Alpha–Beta bidirectional associative memories: theory ... - Springer Link

Neural Processing Letters (2007) 26:1–40 DOI 10.1007/s11063-007-9040-2

Alpha–Beta bidirectional associative memories: theory and applications María Elena Acevedo-Mosqueda · Cornelio Yáñez-Márquez · Itzamá López-Yáñez

Received: 10 October 2006 / Accepted: 22 April 2007 / Published online: 1 June 2007 © Springer Science+Business Media B.V. 2007

Abstract In this work a new Bidirectional Associative Memory model, surpassing every other past and current model, is presented. This new model is based on Alpha–Beta associative memories, from whom it inherits its name. The main and most important characteristic of Alpha–Beta bidirectional associative memories is that they exhibit perfect recall of all patterns in the fundamental set, without requiring the fulfillment of any condition. The capacity they show is 2min(n,m) , being n and m the input and output patterns dimensions, respectively. Design and functioning of this model are mathematically founded, thus demonstrating that pattern recall is always perfect, with no regard to the trained pattern characteristics, such as linear independency, orthogonality, or Hamming distance. Two applications illustrating the optimal functioning of the model are shown: a translator and a fingerprint identifier. Keywords Bidirectional associative memories · Alpha–Beta associative memories · Perfect recall · Fingerprint identifier

1 Introduction The area of Associative Memories, as a relevant part of Computer Science, has achieved ample importance and dynamism in the activities developed by numerous research groups around the globe, specifically among those working in topics related to theory and applications of pattern recognition and classification.

M. E. Acevedo-Mosqueda (B)· C. Yáñez-Márquez · I. López-Yáñez Laboratorio de Inteligencia Artificial, Centro de Investigación en Computación, Instituto Politécnico Nacional, Av. Juan de Dios Bátiz s/n, México, DF 07738, México e-mail: [email protected] C. Yáñez-Márquez e-mail: [email protected] I. López-Yáñez e-mail: [email protected]

123

2

M. E. Acevedo-Mosqueda et al.

The fundamental purpose of an associative memory is to correctly recall complete patterns from input patterns, which may be altered with additive, subtractive, or mixed noise [1]. In the design of an associative memory there are two phases: before the pattern recalling phase comes the learning phase, during which the associative memory is built. In order to do so, associations are made; these associations are pairs of patterns, one input pattern and the other an output pattern. If for every association the input pattern is equal to the output pattern, the resulting memory is autoassociative; otherwise, the memory is heteroassociative. The latter means that an autoassociative memory can be considered as a particular case of an heteroassociative memory [2]. Through out time, Associative Memories have been developed in parallel to Neural Networks, from the conception of the first model of artificial neuron [3], to neural networks models based on modern concepts, such as Morphological Mathematics [4], going through such important works as those by the pioneers in perceptron-like neural networks models [5–7]. In 1982 John J. Hopfield presents to the world his associative memory, model inspired by physics concepts which also has the particularity of an iterative algorithm [8]. This work incited a renewal of researchers interest on topics regarding both associative memories as well as neural networks, which had been dormant for some years. In this sense, neural networks had a recess of 13 years, parting from the publication in 1969 of the book Perceptrons [9] by Minsky and Papert. In it, the authors demonstrated that the perceptron had severe limitations. On the side of associative memories, it was Karl Steinbuch who, in 1961, developed an heteroassociative memory able to work as a binary pattern classifier. He called this first model of associative memory Lernmatrix [10]. Eight years later, Scottish researchers Willshaw et al. [11] presented the Correlograph, elemental optical device capable of working as an associative memory. There is also another important antecedent to Hopfiled memory: two classical model of associative memory were independently presented in 1972 by Anderson [12] and Kohonen [13]. Due to their importance and the similitude of the involved concepts, both models are generically named Linear Associator. The work presented by Hopfield has enormous relevance since his model of neural network demonstrates that the interaction of simple processing elements, similar to neurons, give place to the rise of collective computational properties, such as the stability of memories. However, the Hopfield model of associative memory has two disadvantages: first, it is notable the small pattern recalling capacity, being only 0.15n, where n is the dimension of the stored patterns. Second, the Hopfield memory is only autoassociative; that is, it is incapable of associating patterns which are different. With the intention of solving the second disadvantage of the Hopfield model, in 1988 Kosko [14] created a model of heteroassociative memory parting from the Hopfield memory: the Bidirectional Associative Memory (BAM), which is based on an iterative algorithm, the same as the Hopfield memory. The main characteristic of Kosko BAM is the heteroassociativity, made possible on the basis of the Hopfield model. Kosko proposed a solution based on the same matrix representing the Hopfield model, with which he was capable of realizing the learning phase in both directions: to obtain an output pattern from an input pattern and vice versa, to obtain an input pattern from an output pattern. From this comes the name of bidirectional. Even though the Kosko model was successful in obtaining a heteroassociative memory, the other aforementioned Hopfield memory disadvantage was not solved by the BAM: the Kosko bidirectional associative memory has a very low pattern learning and recovering capacity, dependant on the minimum of the dimensions of the input and output patterns.

123

Alpha–Beta bidirectional associative memories

3

The same year of publication of the BAM, 1988, the first tries of scientific research to improve the BAM pattern learning and recovering capacity appeared [15]. Since then, diverse research groups from around the globe have tried, through the years, to minimize this disadvantage of the BAM. In order to do so, many scientific concepts and mathematical techniques have been used, from multiple training to Householder coding [16] and genetic algorithms [17], and in some cases linear programming. However, the achievements have been modest and scarce. Actually, the vast majority of proposals of new bidirectional associative memory models presented in scientific journals and high level symposiums, do not even guarantee the fundamental set complete recovery. In other words, these models are not capable of recovering all of the learned patterns, and they fail in one or more [14–43]. In the present work, we present the theoretical foundation on which is based the design and functioning of the Alpha–Beta bidirectional associative memories, which exhibit perfect recall of all patterns in the fundamental set.

2 Alpha–Beta associative memories In this section, basic concepts about associative memories are presented. Also, and since Alpha–Beta associative memories are the basis for Alpha–Beta bidirectional associative memories, the theoretical foundation of Alpha–Beta associative memories is presented, as described in [44]. 2.1 Basic concepts The fundamental purpose of an associative memory is to correctly recall complete patterns from input patterns, which may be altered with additive, substractive, or mixed noise. The concepts used in this section are presented in [1,2,44]. An Associative Memory can be formulated as an input output system, idea that is schematized as follows: x

M

y

In this diagram, input and output pattern are represented by column vectors, denoted by x and y, respectively. Every input pattern makes up an association with its corresponding output pattern, similarly to an ordered pair. For instance, the patterns x and y in the former diagram make up the association (x, y). Input and output patterns will be denoted by bold letters, x and y, adding natural numbers as superscript for symbolic discrimination. For instance, to an input pattern x1 corresponds the output pattern y1 , and together they form the association (x1 , y1 ). In the same manner, for a specific positive integer number k, the corresponding association will be (xk , yk ). Associative memory M is represented by a matrix generated from a finite set of associations, known beforehand: this is the fundamental set of associations, or simply fundamental set. The fundamental set is represented as follows: {(xµ, yµ )|µ = 1, 2, . . . , p} where p is a positive, integer number representing the cardinality of the fundamental set.

123

4

M. E. Acevedo-Mosqueda et al.

The patterns that make up the associations of the fundamental set are called fundamental patterns. The nature of the fundamental set gives us an important criterion by which it is possible to classify associative memories: A memory is Autoassociative if it holds that xµ = yµ ∀µ ∈ {1, 2, . . . , p}, then one of the requisites is that n = m. A memory is Heteroassociative is that where ∃µ ∈ {1, 2, . . . , p} for which xµ  = yµ . Notice that there can be heteroassociative memories with n = m. In problems where associative memories intervene, two important phases are considered: the learning phase, where the associative memory is generated from the p associations of the fundamental set, and the recalling phase, where the associative memory operates on an input pattern, as shown in the diagram presented at the beginning of this section. In order to specify the patterns components, the notation of two sets, which we will arbitrarily call A and B, is required. The components of the column vectors representing patterns, both input and output, will be elements from the set A, while the components of matrix M will be elements from the set B. There are no prerequisites or limitations with respect to the choice of the two sets, and therefore they do not need to be different or have special characteristics. This makes the number of possibilities to choose A and B infinite. By convention, every column vector representing an input pattern has n components whose values belong to the set A, and every column vector representing an output pattern has m components whose values belong to the set A. In other words: xµ ∈ An yyµ ∈ Am ∀µ ∈ {1, 2, . . . , p} The jth component of a column vector is indicated by the same letter of the vector, not in bold, putting a j as subscript ( j ∈ {1, 2, . . . , n} or j ∈ {1, 2, . . . , m}, accordingly). The jth µ component of a column vector xµ is represented by x j With the described basic concepts and the former notation, it is possible to express the two phases of an associative memory: 1.

2.

Learning phase (Generation of the associative memory). Find the adequate operators and a way to generate a matrix M that will store the p associations of the fundamental set {(x1 , y1 ), (x2 , y2 ), . . . , (x p , y p )}, where xµ ∈ An and yµ ∈ Am ∀µ ∈ {1, 2, . . . , p}. If ∃µ ∈ {1, 2, . . . , p} such that xµ  =yµ , the memory will be heteroassociative; if m = n and xµ = yµ ∀µ ∈ {1, 2, . . . , p}, the memory will be autoassociative. Recalling phase (Operation of the associative memory). Find the adequate operators and sufficient conditions to obtain the output fundamental pattern yµ , when the memory M is operated with the input fundamental pattern xµ . The latter must hold for every element of the fundamental set and for both modes: autoassociative and heteroassociative.

It is said that an associative memory M exhibits perfect recall if, during the recalling phase, when presented with a pattern xω as input (ω ∈ {1, 2, . . . , p}), M answers with the corresponding output fundamental pattern yω . A bidirectional associative memory is also an input output system, only that the process is bidirectional. The forward direction is described in the same manner as a common associative memory: when presented with an input x, the system delivers an output y. The backward direction takes place presenting to the system an input y in order to receive and output x.

123

Alpha–Beta bidirectional associative memories

5

2.2 Alpha–Beta Associative Memories In this section the definitions of the α and β operations, and the matrix operations that make use of the original operations, are presented. Emphasis is put in the Alpha–Beta associative memory learning and recalling phases, both for the max and min kinds, since they are fundamental to the Alpha–Beta BAM design. Also, we present two Theorems that set the theoretical basis for the Alpha–Beta Bidirectional Associative Memories. The theoretical foundation Theorems for the Alpha–Beta autoassociative memories are numbered according to the original numeration appearing in [44]. The Alpha–Beta associative memories are of two kinds and are able to operate in two different modes. The operator α is useful at the learning phase while the operator β is the basis for the pattern recall phase. The heart of the mathematical tools used in the Alpha–Beta model, are two binary operators designed specifically for these memories. These operators are defined as follows: First, we define the sets A = {0, 1} and B = {0, 1, 2}, then the operators α and β are defined in tabular form: α : A× A→ B

β : B×A→ A

x

y

α(x, y)

x

y

β(x, y)

0 0 1 1

0 1 0 1

1 0 2 1

0 0 1 1 2 2

0 1 0 1 0 1

0 0 0 1 1 1

The sets A and B, the α and β operators, along with the usual ∧ (minimum) y ∨ (maximum) operators, form the algebraic system (A, B, α, β, ∧, ∨) which is the mathematical basis for the Alpha–Beta associative memories. Definition of four matrix operation is required. Of these operations, only four particular cases will be used:
 αmax Operation : Pmxr ∇α Q r xn = f iαj

r

m×n

, where f iαj = ∨ α( pik , qk j ) k=1


 β βmax Operation : Pmxr ∇β Q r xn = f i j

, where f i j = ∨ β( pik , qk j )


 αmi n Operation : Pmxr α Q r xn = h iαj

, where h iαj = ∧ α( pik , qk j )

m×n

β

r

k=1

r

m×n


 β βmi n Operation : Pmxr β Q r xn = h i j

m×n

k=1

β

r

, where h i j = ∧ β( pik , qk j ) k=1

Below are shown some simplifications obtained when these four operations are applied to vectors:

123

6

M. E. Acevedo-Mosqueda et al.

Let x ∈ An and y∈ Am ; then y ∇α xt is a matrix with dimensions m × n, and it holds that: ⎛

∧1k=1 α(y1 , x1 ) ∧1k=1 α(y1 , x2 ) . . . ∧1k=1 α(y1 , xn )

⎜ ⎜ 1 ⎜ ∧k=1 α(y2 , x1 ) ⎜ ⎜ y∇α xt = yα xt = ⎜ ⎜ .. ⎜. ⎜ ⎜ ⎝ ∧1 α(y , x ) k=1

m

1

⎞

⎟ ⎟ ∧1k=1 α(y2 , x2 ) . . . ∧1k=1 α(y2 , xn ) ⎟ ⎟ ⎟ ⎟ .. .. ⎟ ⎟ . ... . ⎟ ⎟ 1 1 ∧ α(y , x ) . . . ∧ α(y , x ) ⎠ k=1

m

2

k=1

m

n

m×n

The ⊗ symbol represents both operations ∇α and α when a column vector of dimension m is operated with a row vector of dimension n: y∇α xt = y ⊗ xt = yα xt Let x ∈ An and P be a matrix of dimension m × n: Operation Pm×n ∇β x yields as a result a column vector of dimension m, whose ith com

ponent has the form: Pm×n ∇β x i = ∨nj=1 β pi j , x j . Operation Pm×n  β x yields as a result a column vector of dimension m, whose ith com

ponent has the form: Pm×n β x i = ∧nj=1 β pi j , x j . 2.3 Alpha–Beta Autoassociative Memories Below are shown some characteristics of Alpha–Beta autoassociative memories: 1. 2. 3.

The fundamental set takes the form {(xµ , xµ )|µ = 1, 2, . . . , p}. Both input and output fundamental patterns are of the same dimension, denoted by n. The memory for both kinds, V (max) and (min). If xµ ∈ An then:  is a squarematrix, V = vi j n×n and  = λi j n×n .

Next, the learning and recalling phases of the Alpha–Beta autoassociative memories max are presented. Learning Phase Step 1. For every µ =1,2,…, p, from the pair (xµ , xµ ) is built the matrix:  µ x ⊗ (xµ )t n×n Step 2. The binary operator max ∨ is applied to the matrices obtained in step 1: p  V = ∨ xµ ⊗ (xµ )t µ=1

The ijth entry is given by the following expression: 

p µ µ vi j = ∨ α xi , x j µ=1

and since α: A × A → B, we have that vi j ∈ B, ∀i ∈ {1, 2, . . . , n}. ∀ j ∈ {1, 2, . . . , n}.

123

Alpha–Beta bidirectional associative memories

7

Recalling Phase The Alpha–Beta autoassociative memories of kind ∨ recalling phase has two possible cases. In the first case, the input pattern is a fundamental pattern. That is, the input is a pattern xω , with ω ∈ {1, 2, . . . , p}. In the second case, the input pattern is NOT a fundamental pattern, but a distorted version of at least one of the fundamental patterns. This means that if the input pattern is x˜ , there must exist at least one index value ω ∈ {1, 2, . . . , p} corresponding to the fundamental pattern in respect to which x˜ is an altered version by one of the three types of noise: additive, substractive or mixed. Case 1 Fundamental pattern. A pattern xω , with ω ∈ {1, 2, . . . , p}, is presented to the autoassociative Alpha–Beta memory of type ∨ and we do operation β : V β x ω . The result of the former operation will be a column vector of dimension n.    p

n n µ µ ω ω ω Vβ x i = ∧ β(vi j , x j ) = ∧ β ∨ α(xi , x j ) , x j j=1

j=1

µ=1

Case 2 Altered pattern. A binary pattern x˜ (an altered pattern of some fundamental pattern xω ), which is a column pattern of dimension n, is presented to the autoassociative Alpha–Beta memory of type ∨ and the operation β is done: Vβ x˜ . As in Case 1, the result of the former operation is a column vector of dimension n, whose ith component is expressed in the following manner:    p

n n µ µ Vβ x˜ i = ∧ β(vi j , x˜ j ) = ∧ β ∨ α(xi , x j ) , x˜ j j=1

j=1

µ=1

Theorem 4.30 Let {(xµ , xµ )|µ = 1, 2, . . . , p} be the fundamental set of an autoassociative Alpha–Beta memory of type ∨ represented by V, and let x˜ ∈ An be a pattern altered with additive noise with respect to some fundamental pattern xω , with ω ∈ {1, 2, . . . , p}. If x˜ is presented to V as input, and also for every i ∈ {1, . . . , n} it holds that ∃ j = j0 ∈ {1, . . . , n}, which is dependant on ω and i such that vi j0 ≤ α(x ω , x˜ j0 ), then recall Vβ x˜ is perfect; that is to say that Vβ x˜ = xω . Theorem 4.30 shows us that autoassociative Alpha–Beta memories of type ∨ are immune to a certain amount of additive noise. Next, the learning and recalling phases of the Alpha–Beta Autoassociative Memories min are presented. Learning Phase Step 1. For every µ =1,2,…, p, from the pair (xµ , xµ ) is built the matrix:  µ x ⊗ (xµ )t n×n Step 2. The binary operator min ∧ is applied to the matrices obtained in step 1: p   = ∧ xµ ⊗ (xµ )t µ=1

The ijth entry is given by the following expression: 

p µ µ λi j = ∧ α xi , x j . µ=1

and since α: A × A → B, we have that λi j ∈ B, ∀i ∈ {1, 2, . . . , n}. ∀ j ∈ {1, 2, . . . , n}.

123

8

M. E. Acevedo-Mosqueda et al.

Recalling Phase The Alpha–Beta autoassociative memories of kind ∧ recalling phase has two possible cases. In the first case, the input pattern is a fundamental pattern. That is, the input is a patternxω , with ω ∈ {1, 2, . . . , p}. In the second case, the input pattern is NOT a fundamental pattern, but a distorted version of at least one of the fundamental patterns. This means that if the input pattern is x˜ , there must exist at least one index value ω ∈ {1, 2, . . . , p} corresponding to the fundamental pattern in respect to which x˜ is an altered version by one of the three types of noise: additive, substractive or mixed. Case 1 Fundamental pattern. A pattern xω , with ω ∈ {1, 2, . . . , p}, is presented to the autoassociative Alpha–Beta memory of type ∧ and we do operation ∇β :  ∇β xω . The result of the former operation will be a column vector of dimension n.    p

n n µ µ ∇β xω i = ∨ β(λi j , x ωj ) = ∨ β ∧ α(xi , x j ) , x ωj j=1

j=1

µ=1

Case 2 Altered pattern. A binary pattern x˜ (an altered pattern of some fundamental pattern xω ), which is a column pattern of dimension n, is presented to the autoassociative Alpha–Beta memory of type ∧ and the operation ∇β is done: ∇β x˜ . As in Case 1, the result of the former operation is a column vector of dimension n, whose ith component is expressed in the following manner:    p

n n µ µ ∇β x˜ i = ∨ β(λi j , x˜ j ) = ∨ β ∧ α(xi , x j ) , x˜ j j=1

j=1

µ=1

Theorem 4.33 Let {(xµ , xµ )|µ = 1, 2, . . ., p} be the fundamental set of an autoassociative Alpha–Beta memory of type ∧ represented by , and let x˜ ∈ An be a pattern altered with substractive noise with respect to some fundamental pattern xω , with ω ∈ {1, 2, . . . , p}. If x˜ is presented to memory as input, and also for every i ∈ {1, . . ., n} it holds that ∃ j = j0 ∈ {1, . . . , n}, which is dependant on ω and i, such that λi j0 ≤ α(x ω , x˜ j0 ), then recall ∇β x˜ is perfect; that is to say that ∇β x˜ = xω . Theorem 4.33 confirms that the autoassociative Alpha–Beta memory of type ∧ are immune to a certain amount of substractive noise.

3 Alpha–Beta Bidirectional Associative Memory Before going into detail over the processing of an Alpha–Beta BAM, we will define the following. In this work we will assume that Alpha–Beta associative memories have a fundamental set denoted by {(xµ , yµ ) | µ = 1, 2, . . ., p}xµ ∈ An and yµ ∈ Am , with A = {0, 1} , n ∈ Z+ , p ∈ Z+ , m ∈ Z+ and 1 < p ≤ min(2n , 2m ). Also, it holds that all input patterns are different; M that is xµ = xξ if and only if µ = ξ . If ∀µ ∈ {1, 2, . . . p} it holds that xµ = yµ , the Alpha–Beta memory will be autoassociative; if on the contrary, the former affirmation is negative, that is ∃µ ∈ {1, 2, . . ., p} for which it holds that xµ  = yµ , then the Alpha–Beta memory will be heteroassociative. Definition 1 (One-hot) Let the set A be A = {0, 1} and p ∈ Z+ , p > 1, k ∈ Z+ , such that 1 ≤ k ≤ p. The kth one-hot vector of p bits is defined as vector h k ∈ A p for which it holds that the kth component is h kk = 1 and the ret of the components are h kj = 0, ∀ j  = k, 1 ≤ j ≤ p.

123

Alpha–Beta bidirectional associative memories

x

9

Stage 1

Stage 2

Stage 4

Stage 3

y

Fig. 1 Graphical schematics of the Alpha–Beta bidirectional associative memory

x Stage 1

⎛ 0 (1) ⎞ ⎜ ⎟ ⎜ 0 ( 2) ⎟ ⎜ ⎟ h ⎟ vk = ⎜ ⎜1( k ) ⎟ ⎜ ⎟ ⎜h ⎟ ⎜ ⎟ ⎝ 0( p) ⎠

Stage 2 Modified Linear Associator

y

Fig. 2 Schematics of the process done in the direction from x to y. Here are shown only Stage 1 and Stage 2. Notice that vkk = 1, vik = 0∀i  = k, 1 ≤ i ≤ p, 1 ≤ k ≤ p

Definition 2 (Zero-hot) Let the set A be A = {0, 1} and p ∈ Z+ , p > 1, k ∈ Z+ , such k that 1 ≤ k ≤ p. The kth zero-hot vector of p bits is defined as vector h ∈ A p for which it k holds that the kth component is h k = 0 and the ret of the components are h kj = 1, ∀ j  = k, 1 ≤ j ≤ p. Definition 3 (Expansion vectorial transform) Let the set A be A = {0, 1} and n ∈ Z+ , y m ∈ Z+ . Given two arbitrary vectors x ∈ An and e ∈ Am , the expansion vectorial transform of order m, τ e : An → An+m , is defined as τ e (x, e) = X ∈ An+m , a vector whose components are: X i = xi for 1 ≤ i ≤ n and X i = ei for n + 1 ≤ i ≤ n + m. Definition 4 (Contraction vectorial transform) Let the set A be A = {0, 1} and n ∈ Z+ , y m ∈ Z+ such that 1 ≤ m < n. Given one arbitrary vector X∈ An+m , the contraction vectorial transform of order m, τ c : An+m → Am , is defined as τ c (X, m) = c ∈ Am , a vector whose components are: ci = X i+n for 1 ≤ i < m. In both directions, the model is made up by two stages, as shown in Fig. 1. For simplicity, first will be described the process necessary in one direction, in order to later present the complementary direction which will give bidirectionality to the model (see Fig. 2). The function of Stage 2 is to offer a yk as output (k = 1, . . . , p) given a xk as input. Now we assume that as input to Stage 2 we have one element of a set of p orthonormal vectors. Recall that the Linear Associator has perfect recall when it works with orthonormal vectors. In this work we use a variation of the Linear Associator in order to obtain yk , parting from a one-hot vector vk in its kth coordinate. For the construction of the modified Linear Associator, its learning phase is skipped and a matrix M representing the memory is built. Each column in this matrix corresponds to each output pattern yµ . In this way, when matrix M is operated with a one-hot vector vk , the corresponding yk will always be recalled. The task of Stage 1 is: given a xk or a noisy version of it (˜xk ), the one-hot vector vk must be obtained without ambiguity and with no condition.

123

10

M. E. Acevedo-Mosqueda et al.

The process in the contrary direction, which is presenting pattern yk (k = 1, . . ., p) as input to the Alpha–Beta BAM and obtaining its corresponding xk , is very similar to the one described above. The task of Stage 3 is to obtain a one-hot vector vk given a yk . Stage 4 is a modified Linear Associator built in similar fashion to the one in Stage 2. 3.1 Theoretical foundation of Stages 1 and 3 Below are presented five Theorems and nine Lemmas with their respective proofs, as well as an illustrative example of each one. This mathematical foundation is the basis for the steps required by the complete algorithm, which is presented in Sect. 3.2. These Theorems and Lemmas numbering corresponds to the numeration used in [45]. By convention, the symbol
will be used to indicate the end of a proof. Theorem 4.1 Let {(x µ , x µ )| µ = 1, 2, . . . , p} be the fundamental set of an autoassociative Alpha–Beta memory of type max represented by V, and let x˜ ∈ An be a pattern altered with additive noise with respect to some fundamental pattern xω with ω ∈ {1, 2, . . . , p}. Lets assume that during the recalling phase, x˜ is presented to memory V

as input, and lets consider an index k ∈ {1, . . . , n}. The kth component recalled Vβ x˜ k is precisely xkω if and only if it holds that ∃r ∈ {1, . . . , n}, dependant on ω and k, such that νkr ≤ α(xkω , x˜r ).

Proof ⇒) By hypothesis we assume that Vβ x˜ k = xkω . By contradiction, now suppose false that ∃r ∈{1, . . . , n} such that νkr ≤ α(xkω , x˜r ). The former is equivalent to stating that ∀r ∈ {1, . . . , n}ν kr >α(xkω , x˜r ), which is the same to saying that ∀r ∈ {1, . . . , n}β (νkr , x˜r ) > β α(xkω , x˜r ), x˜r = xkω . When we take minimums at both sides of the inequality with respect to index r , we have: n

n

∧ β (νkr , x˜r ) > ∧ xkω = xkω

r =1



and this means that Vβ x˜

k

r =1

n

= ∧ β (νkr , x˜r ) > xkω , which contradicts the hypothesis. r =1

⇐) Since the conditions of Theorem 4.30

hold for every i ∈ {1, . . . , n}, we have that Vβ x˜ = xω ; that is, it holds that Vβ x˜ i = xiω , ∀i ∈ {1, . . . , n}. When we fix indexes i that i = k y j0 = r (which depends on ω and k) we obtain the desired result: and j0 such

Vβ x˜ k = xkω .

Example 3.1 Let p = 4 and n = 4. The fundamental set for an associative memory contains tour pairs of patterns {(x µ , x µ )|µ = 1, 2, 3, 4}. Each vector xµ is a column vector with values in the set A4 and the values for each vector components are the following: ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ 1 2 3 4 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ x =⎢ ⎣1⎦, x = ⎣0⎦, x = ⎣1⎦, x = ⎣0⎦ 1 0 0 0 The matrix of the autoassociative Alpha–Beta memory of type max for this fundamental set is: ⎤ ⎡ 1122 ⎢1 1 2 2⎥ ⎥ V=⎢ ⎣2 2 1 2⎦ 1111

123

Alpha–Beta bidirectional associative memories

11

Now, lets suppose we present to matrix V a noisy version x˜ of vector xω with ω = 2, with additive noise, whose components are: ⎡ ⎤ ⎡ ⎤ 1 0 ⎢0⎥ ⎢0⎥ ⎥ ⎢ ⎥ x˜ = ⎢ ⎣ 0 ⎦ then the recalled value will be : Vβ x˜ = ⎣ 0 ⎦ 0 0 We can see that vector x2 was recalled perfectly. However, our interest lies in the recall of components and whether the condition of theorem 4.1 holds. The first component recalled by the autoassociative Alpha–Beta memory of type max is equal to zero, which is precisely the value held by the first component of the second pattern.

This is, Vβ x˜ 1 = 0 = x12 . Now lets see whether the condition of ∃r ∈ {1, . . . , n}, such that νkr ≤ α(xkω , x˜r ), holds. For our example k = 1 and ω = 2. For r = 1, ν11 = 1 and α(x12 , x˜1 ) = α(0, 1) = 0, that is ν11 > α(x12 , x˜1 ), does not hold. For r = 2, ν12 = 1 and α(x12 , x˜2 ) = α(0, 0) = 1, that is ν12 > α(x12 , x˜2 ), does hold. For r = 3, ν13 = 2 and α(x12 , x˜3 ) = α(0, 0) = 1, that is ν13 > α(x12 , x˜3 ), does not hold. For r = 4, ν14 = 2 and α(x12 , x˜4 ) = α(0, 0) = 1, that is ν14 > α(x12 , x˜4 ), does not hold. Therefore, exists r = 2 such that ν12≤ α(x12 , x˜2 ). Lemma 4.1 Let {(Xk , Xk )|k = 1, . . . , p} be the fundamental set of an autoassociative Alpha–Beta memory of type max represented by V, with Xk = τ e (xk , hk ) for k = 1, . . . , p, and let F = τ e (xk , u) ∈ An+ p be a version of a specific pattern Xk , altered with additive p p noise, being u ∈ A the vector defined as u = i=1 hi . If during the recalling phase F is k presented to memory V, then component X n+k will be recalled in a perfect manner; that is

k = 1. Vβ F n+k = X n+k Proof This proof will be done for two mutually exclusive cases. Case 1 Pattern F has one component with value 0. This means that ∃ j ∈ {1, . . . , n + p} k = 1. Then such that F j = 0; also, due to the way vector Xk is built, it is clear that X n+k k α(X n+k , F j ) = α(1, 0) = 2 and since the maximum allowed value for a component of memk , F ). According to Theorem 4.1, X k ory V is 2 we have ν(n+k) j ≤ α(X n+k j n+k is perfectly recalled. Example 3.2 Taking the fundamental set of example 3.1, patterns Xk for k = 1, 2, 3, 4 are built, using the expansion vectorial transform from Definition 3: ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎢0⎥ 1 0 0 1 2 3 4 ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ X = ⎢ ⎥, X = ⎢ ⎥, X = ⎢ ⎥, X = ⎢ ⎢0⎥ 1 0 0 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣0⎦ ⎣1⎦ ⎣0⎦ 0 0 0 1

123

12

M. E. Acevedo-Mosqueda et al.

The matrix of the autoassociative Alpha–Beta memory of type max is: ⎤ ⎡ 11222221 ⎢1 1 2 2 2 2 2 2⎥ ⎥ ⎢ ⎢2 2 1 2 2 2 2 2⎥ ⎥ ⎢ ⎢1 1 1 1 1 2 2 2⎥ ⎥ ⎢ V=⎢ ⎥ ⎢1 1 1 1 1 2 2 2⎥ ⎢2 2 2 2 2 1 2 2⎥ ⎥ ⎢ ⎣2 2 1 2 2 2 1 2⎦ 11222221 Now, having k = 3 we will use vector X3 and obtain its noisy version F, with additive noise: ⎡ ⎤ ⎡ ⎤ 0 0 ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎥ ⎢ 0 3 ⎥ ⎥ ⎢ X = ⎢ ⎥ , the noisy vector F, with additive noise, is: F = ⎢ ⎢1⎥ 0 ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣1⎦ 1 0 When F is presented to memory V, the recalled vector is: ⎡0 ⎤ ⎢0 ⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ 0 V∆ β F = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢⎣0⎥⎦

⎡0 ⎤ ⎢0 ⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ 0 X3 = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ Seventh component ⎢0 ⎥ of both vectors ⎢1⎥ ⎢ ⎥ ⎢⎣0⎥⎦



3 Recalling that for this example n = 4 and k = 3, then Vβ F 4+3 = X 4+3 = 1. Therefore, the seventh component of third vector is perfectly recalled. Case 2 Pattern F does not contain a component with value 0. That is F j = 1∀ j ∈ {1, . . . , n+ p}. This means that it is not possible to guarantee the existence of a value j ∈ {1, . . . , n + p} k , F ), and therefore Theorem 4.1 cannot be applied. However, we such that ν(n+k) j ≤ α(X n+k j

will show the impossibility of Vβ F n+k = 0. The recalling phase of the autoassociative Alpha–Beta memory of type max V, when having vector F as input, takes the following form for the n + kth recalled component:    p

n n µ µ Vβ F n+k = ∧ β(ν(n+k) j , F j ) = ∧ β ∨ α(X n+k , X j ) , F j j=1

j=1

µ=1

µ

k = 1, it is important to notice that X n+k  = 1, Due to the way vector Xk is built, besides X n+k ∀µ  = k, and from here we can establish the following: p

µ

µ

k ∨ α(X n+k , X j ) = α(X n+k , X kj ) = α(1, X kj )

µ=1

123

Alpha–Beta bidirectional associative memories

13

is different from zero regardless of the value of X kj . According to F j = 1∀ j ∈ {1, . . . , n+ p}, we can conclude the impossibility of

n Vβ F n+k = ∧ β(α(1, X kj ), 1) j=1



being zero. That is Vβ F

n+k

=1=

k . X n+k



Example 3.3 Using the matrix V, obtained in Example 3.2, and with k = 1, we get vector F, which is a noisy version by additive noise of X1 , whose component values are: ⎡ ⎤ ⎡ ⎤ 1 1 ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎥ ⎢ 1 1 ⎥ ⎥ ⎢ X = ⎢ ⎥ , the noisy vector F, with additive noise, is: F = ⎢ ⎢1⎥ 1 ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣0⎦ 1 0 When F is presented to memory V, the recalled vector is: ⎡1⎤ ⎢1⎥ ⎢⎥ ⎢1⎥ ⎢⎥ 1 V∆ β F = ⎢ ⎥ ⎢1⎥ ⎢⎥ ⎢1⎥ ⎢1⎥ ⎢⎥ ⎣⎢1⎦⎥

⎡1⎤ ⎢1⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ 1 X1 = ⎢ ⎥ ⎢1⎥ ⎢ ⎥ Fifth component ⎢0 ⎥ of both vectors ⎢0 ⎥ ⎢ ⎥ ⎣⎢0⎦⎥



1 For this example n = 4 y k = 1, then Vβ F 4+1 = X 4+1 = 1. Therefore, the fifth component of the first pattern is perfectly recalled. Theorem 4.2 Let {(Xk , Xk )|k = 1, . . . , p} be the fundamental set of an autoassociative Alpha–Beta memory of type max represented by V, with Xk = τ e (xk , hk ) for k = 1, . . . , p, and let F = τ e (xk , u) ∈ An+ p be a pattern altered with additive  p noise with respect to some specific pattern Xk , being u ∈ A p the vector defined as u = i=1 hi . Lets assume that during the recalling phase, F is presented to memory V as input, and the pattern R = Vβ F ∈ An+ p is obtained. If when taking vector R as argument, the contraction vectorial transform r = τ c (R, n) ∈ A p is done, the resulting vector r has two mutually exclusive possibilities: ∃k ∈ {1, . . . , p} such that r = hk , or r is not a one-hot vector. Proof From the definition of contraction vectorial transform we have that ri = Ri+n = (Vβ F)i+n for 1 ≤ i ≤ p, and in particular, i = k we have rk = Rk+n =

by making k , and since Xk = τ e (xk , hk ), the (Vβ F)k+n . However, by Lemma 4.1 Vβ F n+k = X n+k k value X n+k is equal to the value of component h kk = 1. That is, rk = 1. When considering that rk = 1, vector r has two mutually exclusive possibilities: it can be that r j = 0 ∀ j  = k in which case r = hk ; or happens that ∃ j ∈ {1, . . . , p}, j  = k for which r j = 1, in which case it is not possible that r is a one-hot vector, given Definition 1.



123

14

M. E. Acevedo-Mosqueda et al.

Example 3.4 Using the matrix V obtained in Example 3.2, and with k = 2, the component values of X2 and its respective noisy pattern F, by additive noise, are: ⎡ ⎤ ⎡ ⎤ 0 0 ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎥ ⎢ 0 2 ⎥ ⎥ ⎢ X = ⎢ ⎥ and F = ⎢ ⎢1⎥ 0 ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣0⎦ 1 0 When F is presented to memory V, the recalled vector R is: ⎡ ⎤ 0 ⎢0⎥ ⎢ ⎥ ⎢0⎥ ⎢ ⎥ ⎢0⎥ ⎥ R = Vβ F = ⎢ ⎢0⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ ⎣0⎦ 0 When this vector is taken as argument and the contraction vectorial transform is done we obtain vector r, ⎡ ⎤ 0 ⎢1⎥ ⎥ r=⎢ ⎣0⎦ 0 and according to Definition 1, we can see that r is the second one-hot vector of 4 bits. Now lets follow the same process for k = 4. Then, X4 and its respective noisy pattern F, with additive noise, are: ⎡ ⎤ ⎡ ⎤ 1 1 ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ 4 ⎥ and F = ⎢ 0 ⎥ X =⎢ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣0⎦ 1 1 When F is presented to memory V we obtain vector R: ⎡ ⎤ 1 ⎢1⎥ ⎢ ⎥ ⎢0⎥ ⎢ ⎥ ⎢0⎥ ⎥ R = Vβ F = ⎢ ⎢0⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ ⎣0⎦ 1

123

Alpha–Beta bidirectional associative memories

15

When this vector is taken as argument and the contraction vectorial transform is done we obtain vector r, ⎡ ⎤ 0 ⎢1⎥ ⎢ r=⎣ ⎥ 0⎦ 1 and according to Definition 1, r is not a one-hot vector. Theorem 4.3 Let {(xµ , xµ )|µ = 1, 2, . . . , p} be the fundamental set of an autoassociative Alpha–Beta memory of type min represented by , and let x˜ ∈ An be a pattern altered with substractive noise with respect to some fundamental pattern x ω with ω ∈ {1, 2, . . . , p}. Lets assume that during the recalling phase, x ω is presented  as input, and consider

to memory an index k ∈ {1, . . . , n}. The kth recalled component ∇β x˜ k is precisely xkω if and only if

it holds that ∃r ∈ {1, . . . , n}, dependant on ω and k, such that λkr ≥ α xkω , x˜r .

Proof ⇒) By hypothesis it is assumed that ∇β x˜ k = xkω . By contradiction, now lets ω

suppose it is false that ∃r

. . . , n} such that λkr ≥ α xk , x˜r . That is to say that ω ∈ {1, ∀r ∈ {1, . . . , n}λkr < α xk , x˜r , which is in turn equivalent to ∀r ∈ {1, . . . , n}β (λkr , x˜r ) < β α xkω , x˜r , x˜r = xkω . When taking the maximums at both sides of the inequality, with respect to index r , we have n

n

∨ β (λkr , x˜r ) < ∨ xkω = xkω





r =1

r =1

and this means that ∇β x˜ k = (λkr , x˜r ) < xkω , affirmation which contradicts the hypothesis. ⇐) When conditions for Theorem 4.33 [19]

are met for every i ∈ {1, . . . , n}, we have ∇β x˜ = xω . That is, it holds that f ∇β x˜ i = xiω ∀i ∈ {1, . . . , n}. When indexes i and such that i = k and j0 = r , depending on ω and k, we obtain the desired result

j0 are fixed ∇β x˜ k = xkω .

∨rn=1 β

Example 3.5 The fundamental set components values from example 3.1 are shown below, with p = 4 and n = 4, ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 1 ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎢ 1 0 0 2 3 4 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ x1 = ⎢ ⎣1⎦, x = ⎣0⎦, x = ⎣1⎦, x = ⎣0⎦ 1 0 0 0 The matrix of the autoassociative Alpha–Beta memory of type min for this fundamental set is: ⎤ ⎡ 1101 ⎢1 1 0 1⎥ ⎥ =⎢ ⎣0 0 1 1⎦ 0011 Now lets assume we present matrix  with a noisy version x˜ of vector xω with ω = 4, altered by substractive noise, whose component values are: ⎡ ⎤ ⎡ ⎤ 1 1 ⎢1⎥ ⎢0⎥ ⎥ ⎢ ⎥ x˜ = ⎢ ⎣ 0 ⎦ then the recalled pattern will be ∇β x˜ = ⎣ 0 ⎦ 0 0

123

16

M. E. Acevedo-Mosqueda et al.

As can be seen, vector x4 was perfectly recalled. However, we are interested in corroborating component recall, and also that the condition for Theorem 4.5 is met. The second component recalled by the autoassociative Alpha–Beta memory min is equal

to one which is exactly the second component value of the fourth pattern. That is, ∇β x˜ 2 = 1 = x24 . Now let us see whether the condition of ∃r ∈ {1, . . . , n}, such that λkr ≥ α(xkω , x˜r ), holds. For our example, k = 2 and ω = 4. For r = 1, λ21 = 1 and α(x24 , x˜1 ) = α(1, 1) = 1, this is λ21 ≥ α(x24 , x˜1 ), does hold. For r = 2, λ22 = 1 and α(x24 , x˜2 ) = α(1, 0) = 2, that is λ22 < α(x24 , x˜2 ), does not hold. For r = 3, λ23 = 0 and α(x24 , x˜3 ) = α(1, 0) = 2, that is λ23 < α(x24 , x˜3 ), does not hold. For r = 4, λ24 = 1 and α(x24 , x˜4 ) = α(1, 0) = 2, that is λ24 < α(x24 , x˜4 ), does not hold. Therefore, exists r = 1 such that α(x24 , x˜1 ).  k k

  |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.2 Let  X ,X k = τ e (x k , h¯ k ) for k = 1, . . . , p, Alpha–Beta memory of type min represented by , with X e k n+ p and let G = τ (x , w) ∈ A be a pattern altered with substractive noise with respect to some specific pattern Xk , being w ∈ A p a vector components have values wi = u i −1,  pwhose i and u ∈ A p the vector defined as u = i=1 h . If during the recalling phase, G is k is recalled in a perfect manner. That is, presented to memory , then component X n+k

k ¯ ∇β G n+k = X n+k = 0.

Proof This proof will be done for two mutually exclusive cases. Case 1 Pattern G has one component with value 1. This means that ∃ j ∈ {1, . . . , n + p} k = 0. Because Xk is built, it is clear that X such that G j = 1. Also, due to the way vector  n+k k  of this, α( X n+k , G j ) = α(0, 1) = 0 and, since the minimum allowed value for a component k , G j ). According to Theorem 4.3, X k is of memory  is 0, we have λ(n+k) j ≥ α( X n+k n+k perfectly recalled.

k for k = 1, 2, 3, 4 Example 3.6 Taking the fundamental set of Example 3.1, the patterns X are built, by using the expansion vectorial transform of Definition 3. ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ 1 2 3 4  = ⎢ ⎥, X  = ⎢ ⎥, X  = ⎢0⎥  = ⎢ ⎥, X X ⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣1⎦ ⎣0⎦ ⎣1⎦ 1 1 1 0

123

Alpha–Beta bidirectional associative memories

17

The matrix of the autoassociative Alpha–Beta memory of type min is: ⎤ ⎡ 11010000 ⎢1 1 0 1 0 0 0 0⎥ ⎥ ⎢ ⎢0 0 1 1 0 0 0 0⎥ ⎥ ⎢ ⎢0 0 0 1 0 0 0 0⎥ ⎥ =⎢ ⎢0 0 0 0 1 0 0 0⎥ ⎥ ⎢ ⎢1 1 1 1 0 1 0 0⎥ ⎥ ⎢ ⎣1 1 0 1 0 0 1 0⎦ 00110001 Now, taking k = 3 we use vector  X3 and we obtain its noisy version, with substractive noise: ⎡ ⎤ ⎡ ⎤ 0 0 ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎥ ⎢ 0 3 ⎥ ⎥ ⎢  X = ⎢ ⎥ , the noisy vector G, with substractive noise, is: G = ⎢ ⎢0⎥ 1 ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣0⎦ 0 1 When presenting G to matrix , the recalled pattern is: ⎡0 ⎤ ⎢ ⎥ ⎢0 ⎥ ⎢1 ⎥ ⎢ ⎥ 0 Λ∇ β G = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢0 ⎥ ⎢ ⎥ ⎣⎢1⎦⎥

⎡0 ⎤ ⎢ ⎥ ⎢0 ⎥ ⎢1 ⎥ ⎢ ⎥ 0 X3 = ⎢ ⎥ ⎢1 ⎥ ⎢ ⎥ Seventh component ⎢1 ⎥ of both vectors ⎢0 ⎥ ⎢ ⎥ ⎣⎢1 ⎦⎥



3 = 0. Lets recall that, for this example, n = 4 and k = 3; then ∇β G 4+3 = X 4+3 Therefore, the seventh component of the third vector is perfectly recalled. Case 2 Pattern G has no component with value 1; that is, G j = 0∀ j ∈ {1, . . . , n + p}. This means that it is not possible to guarantee the existence of a value j ∈ {1, . . . , n + p} such that k , G j ), and therefore Theorem 4.3 cannot be applied. However, lets show λ(n+k) j ≥ α( X n+k the impossibility of (∇β G)n+k = 1. Recalling phase of the autoassociative Alpha–Beta memory of type min  with vector G as input, takes the following form for the n + kth recalled component:    p

n n µ µ   ∇β G n+k = ∨ β(λ(n+k) j , G j ) = ∨ β ∧ α( X n+k , X j ) , G j j=1

j=1

µ=1

k is built, besides that X k Due to the way vector X n+k = 0, it is important to notice that µ  X n+k  = 0, ∀µ  = k, and from here we can state that p

µ

µ

 ,X  ) = α( X k , X k ) = α(0, X k ) ∧ α( X n+k j j n+k j

µ=1

123

18

M. E. Acevedo-Mosqueda et al.

k . Taking into account that G j = 0 ∀ j ∈ is different from 2 regardless of the value of X j {1, . . . , n + p}, we can conclude that it is impossible for  



n k , 0 ∇β G n+k = ∨ β α 0, X j j=1



to be equal to 1. That is, ∇β G

n+k

k . =0=X n+k



Example 3.7 By using matrix  obtained in Example 3.6 and with k = 2, we obtain vector G, which is the noisy version, with substractive noise, of X2 , whose component values are: ⎡ ⎤ ⎡ ⎤ 0 0 ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎢ 2 = ⎢ 0 ⎥ , the noisy vector G, with substractive noise, is: G = ⎢ 0 ⎥ X ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣1⎦ 0 1 When presenting G to matrix , the recalled pattern is: ⎡0 ⎤ ⎢0 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ 0 Λ∇ β G = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢⎣0⎥⎦

⎡0 ⎤ ⎢0 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ 0 X2 = ⎢ ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢1 ⎥ Sixth component ⎢ ⎥ of both vectors ⎢⎣1⎥⎦



2 For this example, n = 4 and k = 2, then ∇β G 4+2 = X 4+2 = 0. Therefore, the sixth component of the second pattern is perfectly recalled.   k k

 |k = 1, . . . , p be the fundamental set of an autoassociative  ,X Theorem 4.4 Let X k = τ e (x k , h¯ k ) for k = 1, . . . , p, Alpha–Beta memory of type min represented by , with X and let G = τ e (xk , w) ∈ An+ p be a pattern altered with substractive noise with respect to some specific pattern Xk , being w ∈ A p a vector  p whose components have values wi = u i −1, and u ∈ A p the vector defined as u = i=1 hi . Lets assume that during the recalling phase, G is presented to memory  as input, and the pattern S = ∇β G ∈ An+ p is obtained as output. If when taking vector S as argument, the contraction vectorial transform s = τ c (S, n) ∈ A p is done, the resulting vector S has two mutually exclusive possibilities: ∃k ∈ {1, . . . , p} such that s = h¯ k , or S is not a one-hot vector. Proof From the definition of contraction vectorial transform we have that si = Si+n = (∇β G)i+n for 1 ≤ i ≤ p, and in particular, i = k we have sk = Sk+n =

by making k = τ e (x k , h¯ k ), k , and since X (∇β G)k+n . However, by Lemma 4.2 ∇β G n+k = X n+k k k  ¯ the value X n+k is equal to the value of component h k = 0. That is, sk = 0. When considering that sk = 0, vector s has two mutually exclusive possibilities: it can be that s j = 1∀ j  = k in which case s = h¯ k ; or happens that ∃ j ∈ {1, . . . , p}, j  = k for which s j = 0, in which case it is not possible that s is a zero-hot vector, given Definition 2.



123

Alpha–Beta bidirectional associative memories

19

Example 3.8 By taking matrix  obtained in Example 3.6 and with k = 1, the component values of X1 and its respective noisy pattern G, with substractive noise, are: ⎡ ⎤ ⎡ ⎤ 1 1 ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎥ ⎢ 1⎥ ⎢ ⎥ y G = X1 = ⎢ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣1⎦ 0 1 When presenting G to matrix , the recalled pattern is: ⎡ ⎤ 1 ⎢1⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ ⎢1⎥ ⎥ S = ∇β G = ⎢ ⎢0⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ ⎣1⎦ 1 When this vector is taken as argument and the contraction vectorial transform is done we obtain vector s, ⎡ ⎤ 0 ⎢1⎥ ⎢ s=⎣ ⎥ 1⎦ 1 and according to Definition 2, we can see that s is the first zero-hot vector of 4 bits. 4 and its respective noisy pattern G, Now let us do the same process for k = 4. Then, X with substractive noise, are: ⎡ ⎤ ⎡ ⎤ 1 1 ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎢ 4 = ⎢ 0 ⎥ and G = ⎢ 0 ⎥ X ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣1⎦ 0 0 When presenting G to matrix , the recalled pattern S is: ⎡ ⎤ 1 ⎢1⎥ ⎢ ⎥ ⎢0⎥ ⎢ ⎥ ⎢0⎥ ⎥ S = ∇β G = ⎢ ⎢0⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ ⎣1⎦ 0

123

20

M. E. Acevedo-Mosqueda et al.

When taking this vector as argument and the contraction vectorial transform is done we obtain vector s, ⎡ ⎤ 0 ⎢1⎥ ⎥ S=⎢ ⎣1⎦ 0 According to Definition 2, s is not a zero-hot vector.  k k

  |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.3 Let  X ,X Alpha–Beta memory of type max represented by V, with Xk = τ e (xk , hk ) ∈ An+ p ∀k ∈ {1, . . . , p}. If t is an index such that n + 1 ≤ t ≤ n + p then νi j  = 0 ∀ j ∈ {1, . . . , n + p}. Proof In order to establish that νi j  = 0 ∀ j ∈ {1, . . . , n + p}, given the definition of α, it is µ enough to find, for each ∀t ∈ {n + 1, . . . , n + p}, an index µ for which X t = 1 in the expresp µ µ sion that produces the tjth component of memory V, which is νt j = ∨µ=1 , α(X t , X j ). Due to the way each vector Xµ = τ e (xµ , hµ ) for µ = 1, . . . , p is built, and given the domain of index t ∈ {n + 1, . . . , n + p}, for each t exists s ∈ {1, . . . , p} such that t = n + s. This is s why two useful values to determine the result are µ = s and t = n + s, because X n+s = 1. p µ µ s s s Then, νt j = ∨µ=1 , α(X t , X j ) = α(X n+s , X j ) = α(1, X j ), value which is different from 0. That is, νi j  = 0 ∀ j ∈ {1, . . . , n + p}.

Example 3.9 Let us take vectors Xk for k = 1, 2, 3, 4 from Example 3.2: ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎢ ⎥ 1⎥ 2 ⎢ 0 ⎥ , X3 = ⎢ 0 ⎥ , X4 = ⎢ 0 ⎥ , X = X1 = ⎢ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣0⎦ ⎣1⎦ ⎣0⎦ 0 0 0 1 The matrix of the autoassociative Alpha–Beta memory of type max is: ⎤⎫ ⎡ 11222221 ⎪ ⎪ ⎪ ⎢ 11222222 ⎥⎪ ⎪ ⎥⎪ ⎢ ⎪ ⎢ 22122222 ⎥⎪ ⎪ ⎥⎪ ⎢ ⎢ 11111222 ⎥⎬ ⎥ ⎢ For t ∈ {5, 6, 7, 8} and ∀ j ∈ {1, 2, . . . , 8} νi j  = 0 V=⎢ ⎥ ⎪ ⎢ 11111222 ⎥⎪ ⎪ ⎢ 22222122 ⎥⎪ ⎪ ⎥⎪ ⎢ ⎪ ⎣ 22122212 ⎦⎪ ⎪ ⎪ ⎭ 11222221   k k

 |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.4 Let  X ,X Alpha–Beta memory of type max represented by V, with Xk = τ e (xk , hk ) for k = 1, . . . , p, and let F = τ e (xk , u) ∈ An+ p be an altered version, by additive noise, of a specific pattern p Xk , being u ∈ A p the vector defined as u = i=1 hi . Let us assume that during the recalling phase, F is presented to memory  as input. Given a fixed index t ∈ {n + 1, . . . , n + p} such that t  = n + k, it holds that(Vβ F)t = 1 if and only if the following logic proposition is true: ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2).

123

Alpha–Beta bidirectional associative memories

21

Proof Due to the way vectors Xk = τ e (xk , hk ) and F = τ e (xk , u) are built, we have that Ft = 1 is the component with additive noise with respect to component X tk = 0. ⇒) There are two possible cases: Case 1 Pattern F does not contain components with value 0. That is, F j = 1 j ∈ {1, . . . , n + p}. This means that the antecedent of proposition F j = 0 → νt j = 2 is false, and therefore, regardless of the truth value of consequence νt j = 2, the expression ∀ j ∈ {1, . . . , n+ p}(F j = 0 → νt j = 2) is true. Case 2 Pattern F contains at least one component with value 0. That is, ∃r ∈ {1, . . . , p} such that Fr = 0. By hypothesis (Vβ F)t = 1, which means that the condition for perfect recall of X tk = 0 is not met. In other words, according to Theorem 4.1 expression ¬[∃ j ∈ {1, . . . , n + p} such that νt j ≤ α(X tk , F j )] is true, which is equivalent to ∀ j ∈ {1, . . . , n + p} it holds that νt j ≤ α(X tk , F j ) In particular, for j = r , and taking into account that X tk = 0, this inequality ends up like this: νtr > α(X tk , Fr ) = α(0, 0) = 1. That is, νtr = 2, and therefore the expression ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2) is true. Example 3.10 Let us use vector X3 , with k = 3, from Example 3.9, and build its noisy vector F, with additive noise ⎡ ⎤ ⎡ ⎤ 0 0 ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎥ ⎢ 0⎥ ⎢ ⎥ , F = X3 = ⎢ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣1⎦ 1 0 When presenting F to matrix V, presented in the aforementioned example, we obtain: ⎡0 ⎤ ⎢ ⎥ ⎢0 ⎥ ⎢1 ⎥ ⎢ ⎥ 0 V∆ β F = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢1 ⎥ ⎢ ⎥ ⎣⎢0⎦⎥

⎡0 ⎤ ⎢ ⎥ ⎢0 ⎥ ⎢1 ⎥ ⎢ ⎥ 0 F=⎢ ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢⎣1 ⎥⎦

2

2

2

2

2

(V ∆β F)6 = 1 t = 6

1

2

2

⎡1 1 2 2 2 2 2 1⎤ ⎥ ⎢ ⎢1 1 2 2 2 2 2 2 ⎥ ⎢ 2 2 1 2 2 2 2 2⎥ ⎥ ⎢ 11111 2 2 2 ⎥ V=⎢ ⎢11111 2 2 2 ⎥ ⎥ ⎢ ⎢ 2 2 2 2 2 1 2 2⎥ ⎢2 2 1 2 2 2 1 2⎥ ⎥ ⎢ ⎢⎣ 1 1 2 2 2 2 2 1 ⎥⎦

t=6

123

22

M. E. Acevedo-Mosqueda et al.

⇐) Assuming the following expression is true ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2), there are two possible cases: Case 1 Pattern F does not contain components with value 0. That is, F j = 1∀ j ∈ {1, . . . , n +



n+ p p}. When considering that Vβ F t = ∧ j=1 β νt j , F j , according to the definition of β, it is enough to show that ∀ j ∈ {1, . . . , n + p}νt j  = 0, which is guaranteed by Lemma 4.3.



n+ p n+ p Then, it has been proven that Vβ F t = ∧ j=1 β νt j , F j = ∧ j=1 β νt j , 1 = 1. Case 2 Pattern F contains at least one component with value 0. That is, ∃r ∈ {1, . . . , p} such that Fr = 0. By hypothesis we have that ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2)



n+ p and, in particular, for j = r and νtr = 2, which means that Vβ F t = ∧ j=1 β νt j , F j = β (νtr , Fr ) = β (2, 1) = 1.

 k k

  |k = 1, . . . , p be the fundamental set of an autoassociative  ,X Corollary 4.1 Let X Alpha–Beta memory of type max represented by V, with Xk = τ e (xk , hk ) for k = 1, . . . , p, and let F = τ e (xk , u) ∈ An+ p be an altered version, by additive noise, of a specific pattern p Xk , being u ∈ A p the vector defined as u = i=1 hi . Let us assume that during the recalling phase, F is presented to memory  as input. Given a fixed index t ∈ {n + 1, . . . , n + p} such that t  = n + k, it holds that(V β F)t = 0 if and only if the following logic proposition is true: ∀ j ∈ {1, . . . , n + p}(F j = 0 AND sνt j  = 2). Proof In general, given two logical propositions P and Q, the proposition (P if and only if Q) is equivalent to proposition (¬P if and only if ¬Q). If P is identified with equality (Vβ F)t = 1 and Q with expression ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2), by Lemma 4.4 the following proposition is true: {¬[(Vβ F)t = 1] if and only if ¬[∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2)]}. This expression transforms in the following equivalent propositions: {(Vβ F)t = 0 if and only if ∃ j ∈ {1, . . . , n + p} such that ¬(F j = 0 → νt j = 2)} {(Vβ F)t = 0 if and only if ∃ j ∈ {1, . . . , n + p} such that ¬[¬(F j = 0) OR νt j = 2]} {(Vβ F)t =0 if and only if ∃ j ∈ {1, . . . , n+ p} such that [¬[¬(F j =0)] AND ¬(νt j =2)]} {(Vβ F)t = 0 if and only if ∃ j ∈ {1, . . . , n + p} such that [(F j = 0) AND νt j  = 2]} Example 3.11 Taking X3 and F from Example 3.10, when presenting F to V we have: ⎡0 ⎤ ⎢0 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢ ⎥ 0 V∆ β F = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢⎣0⎥⎦ ⎡0 ⎤ ⎢ ⎥ ⎢0 ⎥ ⎢1 ⎥ ⎢ ⎥ 0 F=⎢ ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢⎣1 ⎥⎦

123

(V ∆β F)5 = 0, t = 5

(V ∆β F)8 = 0, t = 8

1 1 1 1 1 2 2 2

⎡1 1 2 2 2 2 2 1⎤ ⎢ ⎥ ⎢1 1 2 2 2 2 2 2 ⎥ ⎢ 2 2 1 2 2 2 2 2⎥ ⎥ ⎢ 11111 2 2 2 ⎥ V=⎢ ⎢11111 2 2 2 ⎥ ⎢ ⎥ ⎢ 2 2 2 2 2 1 2 2⎥ ⎢ ⎥ ⎢2 2 1 2 2 2 1 2⎥ ⎢⎣ 1 1 2 2 2 2 2 1 ⎥⎦

t=5

Alpha–Beta bidirectional associative memories

23

 k k

  |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.5 Let  X ,X k = τ e (xk , h¯ k ) ∈ An+ p ∀ k ∈ Alpha–Beta memory of type min represented by , with X {1, . . . , p}. If t is an index such that n + 1 ≤ t ≤ n + p then λt j  = 2∀ j ∈ {1, . . . , n + p}. Proof In order to establish that λt j  = 2∀ j ∈ {1, . . . , n + p}, given the definition of α, it is µ enough to find, for each t ∈ {n + 1, . . . , n + p}, an index µ for which  Xt = 0 in the expresp µ tµ ,  sion leading to obtaining the tjth component of memory , which is λt j = ∧µ=1 α(X X j ). k = τ e (x k , h¯ k ) for µ = 1, . . . , p is built, and given In fact, die to the way each vector X the domain of index t ∈ {n + 1, . . . , n + p}, for each t exists s ∈ {1, . . . , p} such that t = n + s; therefore two values useful to determine the result are µ = s and t = n + s, p s s ,X tµ , X µ ) = α(X n+s s ) = α(0, X µ ), value differbecause  Xn+s = 0, then λt j = ∧µ=1 α(X j j j ent from 2. That is, λt j  = 2∀ j ∈ {1, . . . , n + p}.

Example 3.12 Let us take vectors  Xk for k = 1, 2, 3, 4 from Example 3.6 ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 0 0 1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ 1 ⎥ 2 ⎢ 0 ⎥ 3 ⎢ 0 ⎥ 4 ⎢ ⎢0⎥  , X , X , X = = = X1 = ⎢ ⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣1⎦ ⎣0⎦ ⎣1⎦ 1 1 1 0 The matrix of the autoassociative Alpha–Beta memory of type min is: ⎤⎫ ⎡ 11010000 ⎪ ⎪ ⎪ ⎢ 11010000 ⎥⎪ ⎪ ⎥⎪ ⎢ ⎪ ⎢ 00110000 ⎥⎪ ⎪ ⎥⎪ ⎢ ⎢ 00010000 ⎥⎬ ⎥ ⎢ =⎢ ⎥ For t ∈ {5, 6, 7, 8} and ∀ j ∈ {1, 2, . . . , 8} λi j  = 2 ⎪ ⎢ 00001000 ⎥⎪ ⎪ ⎢ 11110100 ⎥⎪ ⎪ ⎥⎪ ⎢ ⎪ ⎣ 11010010 ⎦⎪ ⎪ ⎪ ⎭ 00110001  k k

  |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.6 Let  X ,X k = τ e (x k , h¯ k ) for k = 1, . . . , p, Alpha–Beta memory of type min represented by , with X e k n+ p and let G = τ (x , w) ∈ A be an altered version, by substractive noise, of a specific pattern Xk , being w ∈ A p a vector  p whose components have values wi = u i − 1, and u ∈ A p the vector defined as u = i=1 hi . Let us assume that during the recalling phase, G is presented to memory  as input. Given a fixed index t ∈ {n + 1, . . . , n + p} such that t  = n + k, it holds that (∇β G)t = 0, if and only if the following logical proposition is true ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0). k = τ e (x k , bar hk ) and G = τ e (xk , w) are built, we have Proof Due to the way vectors X tk = 1. that G t = 1 is the component with substractive noise with respect to component X ⇒) There are two possible cases: Case 1 Pattern G does not contain components with value 1. That is, G j = 0∀ j ∈ {1, . . . , n + p}. This means that the antecedent of logical proposition G j = 1 → λt j = 0 is

123

24

M. E. Acevedo-Mosqueda et al.

false and therefore, regardless of the truth value of consequent λt j = 0, the expression ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0) is true. Case 2 Pattern G contains at least one component with value 1. That is, ∃r ∈ {1, . . . , n + p} such that G r = 1. By hypothesis (∇β G)t = 0, which means that the perfect recall conk

dition of X t = 1 is not met. In other words, according to Theorem 4.3 expression ¬[∃ j ∈ tk , G j )] is true, which in turn is equivalent to {1, . . . , n + p} such that λt j ≥ α(X k

∀ j ∈ {1, . . . , n + p} it holds that λt j < α(X t , G j ) tk , G r ) = In particular, for j = r and considering that  Xtk = 1, this inequality yields: λtr < α(X α(1, 1) = 1. That is, λtr = 0, and therefore the expressión ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0) is true. Example 3.13 Let us use vector  X4 , with k = 4, from Example 3.12, and build its noisy vector G, with substractive noise. ⎡ ⎤ ⎡ ⎤ 1 1 ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎢ 4 = ⎢ 0 ⎥ , G = ⎢ 0 ⎥ X ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎣0⎦ ⎣1⎦ 0 0 When presenting G to , shown n the mentioned example, we have ⎡1 ⎤ ⎢ ⎥ ⎢1 ⎥ ⎢0 ⎥ ⎢ ⎥ 0 Λ∇ β G = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢⎣0⎥⎦ ⎡1 ⎤ ⎢1 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ 0 G=⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢0 ⎥ ⎢ ⎥ ⎣⎢0⎦⎥

0 0 0 0 1 0 0 0

(Λ ∇β G)5 = 0 t = 5

⎡1 1 0 1 0 0 0 0 ⎤ ⎢1 1 0 1 0 0 0 0 ⎥ ⎥ ⎢ ⎢0 0 1 1 0 0 0 0⎥ ⎥ ⎢ 0 0 0 1 0 0 0 0⎥ Λ=⎢ ⎥ ⎢ ⎢0 0 0 0 1 0 0 0 ⎥ ⎢1111 0 1 0 0 ⎥ ⎢1 1 0 1 0 0 1 0 ⎥ ⎥ ⎢ ⎣⎢ 0 0 1 1 0 0 0 1 ⎦⎥

t=5

⇐) Assuming the following expression to be true, ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0), there are two possible cases: Case 1 Pattern G does not contain components with value 1. That is, G j = 0∀ j ∈ {1, . . . ,



n+ p n + p}. When considering that ∇β G t = ∨ j=1 β λt j , G j , according to the β definition,

123

Alpha–Beta bidirectional associative memories

25

it is enough to show that ∀ j ∈ {1, . . . , n + p}λt j  = 2, which is guaranteed by Lemma 4.5.



n+ p Then, it is proven that ∇β G t = ∨ j=1 β λt j , G j . Case 2 Pattern G contains at least one component with value 1. That is, ∃r ∈ {1, . . . , n + p} such that G r = 1. By hypothesis we have that ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0)



n+ p and, in particular, for j = r and λtr = 0, which means that ∇β G t = ∨ j=1 β λt j , G j =

β (λtr , G r ) = β (0, 0) = 0.  k k

   Corollary 4.2 Let X , X |k = 1, . . . , p be the fundamental set of an autoassociative k = τ e (xk , h¯ k ) for k = 1, . . . , p, Alpha–Beta memory of type min represented by , with X e k n+ p and let G = τ (x , w) ∈ A be an altered version, by substractive noise, of a specific pattern Xk , being w ∈ A p  a vector whose components have values wi = u i − 1, and u p the vector defined as u = i=1 hi . Let us assume that during the recalling phase, G is presented to memory  as input. Given a fixed index t ∈ {n + 1, . . . , n + p} such that t  = n + k, it holds that (∇β G)t = 1 if and only if the following logic proposition is true: ∃ j ∈ {1, . . . , n + p}(G j = 1 AND λt j  = 0). Proof In general, given two logical propositions P and Q, the proposition (P if and only if Q) is equivalent to proposition (¬P if and only if ¬Q). If P is identified with equality (∇β G)t = 0 and Q with expression ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0), by Lemma 4.6 the following proposition is true: {¬[(∇β G)t = 0] if and only if ¬[∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0)]}. This expression transforms into the following equivalent propositions: {(∇β G)t = 1 if and only if ∃ j ∈ {1, . . . , n + p} such that ¬(G j = 1 → λt j = 0)]} {(∇β G)t = 1 if and only if ∃ j ∈ {1, . . . , n + p} such that ¬[¬(G j = 1) OR λt j = 0]} {(∇β G)t =1 if and only if ∃ j∈{1, . . . , n+ p} such that [¬[¬(G j = 1)] AND ¬(λt j = 0)]} {(∇β G)t = 1 if and only if ∃ j ∈ {1, . . . , n + p} such that [G j = 1 AND λt j  = 0]} Example 3.14 Taking  X4 and G from Example 3.13, when presenting G to  ⎡1 ⎤ ⎢1 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ 0 Λ∇ β G = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢1 ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢⎣0⎥⎦

⎡1 ⎤ ⎢1 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ 0 G=⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢0 ⎥ ⎢0 ⎥ ⎢ ⎥ ⎣⎢0⎦⎥




(Λ ∇β G)7 = 1 t = 7 (Λ ∇β G)6 = 1 t = 6

1 1 1 1 0 1 0 0

⎡1 1 0 1 0 0 0 0 ⎤ ⎥ ⎢ ⎢1 1 0 1 0 0 0 0 ⎥ ⎢0 0 1 1 0 0 0 0⎥ ⎥ ⎢ 0 0 0 1 0 0 0 0⎥ Λ=⎢ ⎢0 0 0 0 1 0 0 0 ⎥ ⎥ ⎢ ⎢1111 0 1 0 0 ⎥ ⎢ ⎥ ⎢1 1 0 1 0 0 1 0 ⎥ ⎣⎢ 0 0 1 1 0 0 0 1 ⎦⎥

t=6

123

26

M. E. Acevedo-Mosqueda et al.

 k k

  |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.7 Let  X ,X k e k k Alpha–Beta  k memory

of type max represented by V, with X = τ (x , h ) for k = 1, . . . , p, k   and let X , X | k = 1, . . . , p be the fundamental set of an autoassociative Alpha–Beta k = τ e (xk , h¯ k ), ∀k ∈ {1, . . . , p}. Then, for memory of type min represented by , with X each i ∈ {n + 1, . . . , n + p} such that i = n + r , with ri ∈ {1, . . . , p}, it holds that: ri )∀ j ∈ {1, . . . , n + p}. νi j = α(1, X rji ) and λi j = α(0, X j k = τ e (xk , h¯ k ) are built, we have Proof Due to the way vectors Xk = τ e (xk , hk ) and X µ ri r i  = 0, besides X = 0 and X µ = 1 ∀µ  = ri such that µ ∈ that X i = 1 and X i i i {1, . . . , p}. Because of this, and using the definition of α, α(X iri , X rji ) = α(1, X rji ) and µ µ µ µ r α(X i , X j ) = α(0, X j ), which implies that, regardless of the values of X ji and X j , it holds µ µ that α(X iri , X rji ) ≥ α(X i , X j ), from whence p

µ

µ

νi j = ∨ α(X i , X j ) = α(X iri , X rji ) = α(1, X rji ) µ=1

ri ) = α(0, X ri ) and α( X µ , X µ ) = α(1, X µ ), which implies that, ri , X We also have α( X i j j i j j ri and X µ , it holds that α( X ri , X ri ) ≤ α( X µ , X µ ), from whence regardless of the values of X j i j j i j p

µ , X µ ) = α( X ri , X ri ) = α(0, X ri ) λi j = ∧ α( X i j j i j µ=1

µ ∈ {1, . . . , p}, ∀ j ∈ {1, . . . , n + p}.



Example 3.15 Let us take vectors Xk for k = 1, 2, 3, 4 from Example 3.2 ⎡ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 1 0 0 1 ⎢1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢2 ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1 ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎢ 1 0 0 0 1 2 3 4 ⎢ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎢ , X , X , X , V = =⎢ ⎥ =⎢ ⎥ =⎢ ⎥ X =⎢ ⎥ ⎢1 ⎢ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢0⎥ ⎢2 ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣2 ⎣0⎦ ⎣0⎦ ⎣1⎦ ⎣0⎦ 1 0 0 0 1

1 1 2 1 1 2 2 1

2 2 1 1 1 2 1 2

2 2 2 1 1 2 2 2

2 2 2 1 1 2 2 2

2 2 2 2 2 1 2 2

2 2 2 2 2 2 1 2

⎤ 1 2⎥ ⎥ 2⎥ ⎥ 2⎥ ⎥ 2⎥ ⎥ 2⎥ ⎥ 2⎦ 1

In this example i ∈ {5, 6, 7, 8} and j ∈ {1, 2, . . . , 8}. Since we are using the autoassociative Alpha–Beta memory of type max, during the learning phase the maximum value must be taken. Using the definition of α, the maximum allowed value is 2, which can be reached through α(1, 0) = 2. With a 0 value the maximum that can be obtained is α(0, 0) = 1. Therefore, the component yielding a maximum value will be that whose value is 1. The only component, out of the four vectors for i = 5, having a value of 1, is the one in X1 . That is, X 51 = 1 will determine a maximum value, then ν5 j = α(X 51 , X rji ) = α(1, X rji ). For i = 6, X 62 = 1, then ν6 j = α(X 62 , X rji ) = α(1, X rji ). r

r

For i = 7, X 73 = 1, then ν7 j = α(X 73 , X ji ) = α(1, X ji ). For i = 8, X 84 = 1, then ν8 j = α(X 84 , X rji ) = α(1, X rji ).

123

Alpha–Beta bidirectional associative memories

27

Now let us take vectors Xk for k = 1, 2, 3, 4 from Example 3.6 ⎡ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 1 0 0 1 ⎢1 ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢0 ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎥ ⎥ ⎢ ⎢ ⎢ ⎢ 4 ⎢0⎥,  = ⎢0 2 = ⎢ 0 ⎥ , X 3 = ⎢ 0 ⎥ ,  1 = ⎢ 1 ⎥ , X X X = ⎢0 ⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1 ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣1 ⎣1⎦ ⎣1⎦ ⎣0⎦ ⎣1⎦ 0 1 1 1 0

1 1 0 0 0 1 1 0

0 0 1 0 0 1 0 1

1 1 1 1 0 1 1 1

0 0 0 0 1 0 0 0

0 0 0 0 0 1 0 0

0 0 0 0 0 0 1 0

⎤ 0 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎦ 1

In the case of the autoassociative Alpha–Beta memory of type min, during the learning phase the minimum value must be taken. The minimum allowed value is 0 and appears when α(0, 1) = 0. Therefore, the component yielding a minimum value will be that whose value is 0. The only component, out of the four vectors for i = 5, having a value of 0, is the one in 1 , X ri ) = α(0, X ri ). X1 . That is, X 51 = 0 will determine a minimum value, then λ5 j = α( X j j 5 62 , X ri ) = α(0, X¯ ri ). 62 = 1, then λ6 j = α( X For i = 6, X j j 73 , X ri ) = α(0, X ri ). 73 = 1, then λ7 j = α( X For i = 7, X j j 84 = 1, then λ8 j = α( X 84 , X ri ) = α(0, X¯ ri ). For i = 8, X j j  k k

  |k = 1, . . . , p be the fundamental set of an autoassociative  ,X Corollary 4.3 Let X Alpha–Beta memory of type max represented by V, with Xk = τ e (xk , hk )∀k ∈ {1, . . . , p},  k k

  ,X  | k = 1, . . . , p be the fundamental set of an autoassociative Alpha– and let X Beta memory of type min represented by , with  Xk = τ e (xk , h¯ k ), ∀k ∈ {1, . . . , p}. Then, νi j = λi j + 1, ∀i ∈ {n + 1, . . . , n + p}, i = n +ri , with ri ∈ {1, . . . , p} and ∀ j ∈ {1, . . . , n}. Proof Let i ∈ {n + 1, . . . , n + p} and j ∈ {1, . . . , n} be two indexes arbitrarily selected. By Lemma 4.7, the expressions used to calculate the ijth components of memories V y  take the following values: ri ) νi j = α(1, X rji )yλi j = α(0, X j ri , there are two possible cases: Considering that for ∀ j ∈ {1, . . . , n}X rji = X j ri r i  . We have the following values: νi j = α(1, 0) = 2 and λi j = Case 1 X j = 0 = X j α(0, 0) = 1, therefore νi j = λi j + 1. ri . We have the following values: νi j = α(1, 1) = 1 y λi j = α(0, 1) = Case 2 X rji = 1 = X j 0, therefore νi j = λi j + 1. Since both indexes i and j were arbitrarily chosen inside their respective domains, the result νi j = λi j + 1 is valid ∀i ∈ {n + 1, . . . , n + p} and ∀ j ∈ {1, . . . , n}.

Example 3.16 For this example we shall use memories V y , presented in Example 3.15. ⎡

1 ⎢1 ⎢ ⎢2 ⎢ ⎢1 V=⎢ ⎢1 ⎢ ⎢2 ⎢ ⎣2 1

1 1 2 1 1 2 2 1

2 2 1 1 1 2 1 2

2 2 2 1 1 2 2 2

2 2 2 1 1 2 2 2

2 2 2 2 2 1 2 2

2 2 2 2 2 2 1 2

⎡ ⎤ 1 1 ⎢1 2⎥ ⎢ ⎥ ⎢0 2⎥ ⎢ ⎥ ⎢ 2⎥ ⎥,  = ⎢0 ⎢0 2⎥ ⎢ ⎥ ⎢1 2⎥ ⎢ ⎥ ⎣1 ⎦ 2 0 1

1 1 0 0 0 1 1 0

0 0 1 0 0 1 0 1

1 1 1 1 0 1 1 1

0 0 0 0 1 0 0 0

0 0 0 0 0 1 0 0

0 0 0 0 0 0 1 0

⎤ 0 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎦ 1

123

28

M. E. Acevedo-Mosqueda et al.

With n = 4 and p = 4, i ∈ {5, 6, 7, 8} and j

∈ {1, 2, 3, 4}.

ν51 = λ51 + 1 = 0 + 1 = 1 ν61 = λ61 + 1 = 1 + 1 = 2 ν71 = λ71 + 1 = 1 + 1 = 2 ν81 = λ81 + 1 = 0 + 1 = 1 ν52 = λ52 + 1 = 0 + 1 = 1 ν62 = λ62 + 1 = 1 + 1 = 2 ν72 = λ72 + 1 = 1 + 1 = 2 ν82 = λ82 + 1 = 0 + 1 = 1 ν53 = λ53 + 1 = 0 + 1 = 1 ν63 = λ63 + 1 = 1 + 1 = 2 ν73 = λ73 + 1 = 0 + 1 = 1 ν83 = λ83 + 1 = 1 + 1 = 2 ν54 = λ54 + 1 = 0 + 1 = 1 ν64 = λ64 + 1 = 1 + 1 = 2 ν74 = λ74 + 1 = 1 + 1 = 2 ν84 = λ84 + 1 = 1 + 1 = 2   k k

 |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.8 Let  X ,X Alpha–Beta of type max represented by V, with Xk = τ e (xk , hk )∀k ∈ {1, . . . , p},

 k memory k   and let X , X | k = 1, . . . , p be the fundamental set of an autoassociative Alpha–Beta e k ¯k k memory of type min represented  p by i, with X = τ (x , h ), ∀k ∈ {1, . . . , p}. Also, if we p define vector u ∈ A as u = i=1 h , and take a fixed index ∀r ∈ {1, . . . , p}, let us consider two noisy versions of pattern X r ∈ An+ p : vector F = τ e (xr , u) ∈ An+ p which is an additive noise altered version of pattern X r , and vector G = τ e (xr , w) ∈ An+ p , which is a substracr , beingw ∈ A p a vector whose components take the tive noise altered version of pattern X values wi = u i − 1∀i ∈ {1, . . . , p}. If during the recalling phase, G is presented as input to memory  and F is presented as input to memory V, and if also it holds that (∇β G)t = 0 for an index t ∈ {n + 1, . . . , n + p}, fixed such that t  = n + r , then (V β F)t = 0. Proof Due to the way vectors Xr , F y G are built, we have that Ft = 1 is the component in the vector with additive noise corresponding to component X tr , and G t = 0 is the component tr . Also, since t  = n + r , in the vector with substractive noise corresponding to component X r r r  we can see that X t  = 1, that is X t = 0, and X t = 1. There are two possible cases: Case 1 Pattern F does not contain any component with value 0. That is, F j = 1 ∀ j ∈ {1, . . . , n + p}. By Lemma 4.3 νt j  = 0∀ j ∈ {1, . . . , n + p}, then β(νt j , F j )∀ j ∈ {1, . . . , n +

n+ p p}, which means that Vβ F t = ∧ j=1 β(νt j , F j ) = 1. In other words, expression (V β F)t = 0 is false. The only possibility for the theorem to hold, is for expression (∇β G)t = 0 to be false too. That is, we need to show that (∇β G)t = 1. According to Corollary 4.2, the latter is true if for every t ∈ {n + 1, . . . , n + p} with t  = n + r , exists j ∈ {1, . . . , n + p} such that (G j = 1 AND λt j  = 0). Now, t  = n + r indicates that ts , X s ) ≤ α( X tµ , X µ )∀µ ∈ ∃s ∈ {1, . . . , p}, s  = r such that t = n+s, and by Lemma 4.7 α( X j j p tµ , X µ ) = α( X ts , X s ), {1, . . . , p}, ∀ j ∈ {1, . . . , n + p}, from where we have λt j = ∧ j=1 α( X j j s ts = X n+s and by noting the equality X = 0, it holds that: s ) ∀ j ∈ {1, . . . , n + p} λt j = α(0, X j r = x r = 1 and X s = x s On the other side, ∀i ∈ {1, . . . , n} the following equalities hold: X i i i i r s and also, taking into account that x  = x , it is clear that ∃h ∈ {1, . . . , p} such that x hs  = x hr ; meaning x hs = 0 = X hs and therefore λth = α(0, 0) = 1

123

Alpha–Beta bidirectional associative memories

29

r = x r = 1, in particular G h = 1. Then Finally, since ∀i ∈ {1, . . . , n} it holds that G i = X i i we have proven that for every t ∈ {n +1, . . . , n + p} with t  = n +r , exists j ∈ {1, . . . , n + p} such that (G j = 1 AND λt j  = 0), and by Corollary 4.2 it holds that (∇β G)t = 1, thus making expression (∇β G)t = 1 is false. Case 2 Pattern F contains, besides the components with value of 1, at least one component with value 0. That is, ∃h ∈ {1, . . . , n + p} such that Fh = 0. Due to the way vectors G and F are built ∀i ∈ {1, . . . , n}G i = Fi and, also, necessarily 1 ≤ h ≤ n and thus Fh = G h = 0. By hypothesis ∃t ∈ {n + 1, . . . , n + p} fixed such that t  = n + r and (∇β G)t = 0, and by Lemma 4.6 ∀ j ∈ {1, . . . , n + p}(G j = 1 → λt j = 0). Given the way vector G is built we have that ∀ j ∈ {n + 1, . . . , n + p}G j = 0, thus making the former expression like this: ∀ j ∈ {1, . . . , n}(G j = 1 → λt j = 0). Let J be a set, proper subset of {1, . . . , n}, defined like this: J = { j ∈ {1, . . . , n}|G j = 1}. The fact that J is a proper subset of {1, . . . , n} is guaranteed by the existence of G h = 0. Now, t  = n + r indicates that ∃s ∈ {1, . . . , p}, s  = r such s )∀ j ∈ {1, . . . , n + p}, that t = n + s, and by Lemma 4.7 νt j = α(1, X sj ) and λt j = α(0, X j s = 1, because if this was not the case, λt j  = 0. This from where we have that ∀ j ∈ J , X j s = 1 = G j which in turn means that patterns Xr and Xs coinmeans that for each j ∈ J X j cide with value 1 in all components with index j ∈ J . Let us now consider the complement of set J , which is defined as J c = { j ∈ {1, . . . , n}|G j = 0}. The existence of at least one value s = 1 is guaranteed by the known fact that xr  = xs . Let us j0 ∈ J c for which G j0 = 0 y X j0 s c  = 0 ∀ j ∈ J then ∀ j ∈ {1, . . . , n} it holds that X s = G j , which would mean that see, if X j j s = 1, this means that ∃ j0 ∈ J c for which xr = xs . Since ∃ j0 ∈ J c for which G j0 = 0 and X j0 F j0 = 0 and X sj0 = 1. Now, β(νt j0 , F j0 ) = β(α(1, X sj0 ), 0) = β(α(1, 1), 0) = β(1, 0) = 0, and finally

Vβ F

t

n+ p

= ∧ β(νt j , F j ) = β(νt j0 , F j0 ) = 0 j=1




Example 3.17 From Example 3.2 let us takeX2 and build the noisy vector F, with additive noise; from Example 3.6 take X2 and build the noisy vector G, with substractive noise, and use the memories V and  shown in Example 3.15. ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 0 0 0 0 ⎢0⎥ ⎢0⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢0⎥ ⎢0⎥ ⎢0⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ ⎥ ⎢ ⎢ 0⎥ ⎢0⎥, X 2 = ⎢ 0 ⎥ , G = ⎢ 0 ⎥ , F = X2 = ⎢ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣0⎦ ⎣0⎦ ⎣1⎦ 1 0 0 1 ⎤ ⎡ ⎤ ⎡ 11010000 11222221 ⎢1 1 0 1 0 0 0 0⎥ ⎢1 1 2 2 2 2 2 2⎥ ⎥ ⎢ ⎥ ⎢ ⎢0 0 1 1 0 0 0 0⎥ ⎢2 2 1 2 2 2 2 2⎥ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎢1 1 1 1 1 2 2 2⎥ ⎥, = ⎢0 0 0 1 0 0 0 0⎥ V=⎢ ⎢0 0 0 0 1 0 0 0⎥ ⎢1 1 1 1 1 2 2 2⎥ ⎥ ⎢ ⎥ ⎢ ⎢1 1 1 1 0 1 0 0⎥ ⎢2 2 2 2 2 1 2 2⎥ ⎥ ⎢ ⎥ ⎢ ⎣1 1 0 1 0 0 1 0⎦ ⎣2 2 1 2 2 2 1 2⎦ 00110001 11222221

123

30

M. E. Acevedo-Mosqueda et al.

Now we present F to V and G to  in order to obtain:

V∆ β F

⎡ 0⎤ ⎢ 0⎥ ⎢ ⎥ ⎢ 0⎥ ⎢ ⎥ 0 = ⎢ ⎥, ⎢ 0⎥ ⎢ ⎥ ⎢ 1⎥ ⎢ 0⎥ ⎢ ⎥ ⎣⎢ 0⎦⎥

Λ∇ β G

⎡ 0⎤ ⎢ 0⎥ ⎢ ⎥ ⎢ 0⎥ ⎢ ⎥ 0 =⎢ ⎥ ⎢ 0⎥ ⎢ ⎥ ⎢ 0⎥ ⎢ 0⎥ ⎢ ⎥ ⎣⎢ 0⎦⎥

With n = 4 and p = 4, t ∈ {5, 6, 7, 8} and for this example r = 2, thus t  = 4 + 2 = 6. Then, for t = 5(∇β G)5 = 0, (Vβ F)5 = 0. for t = 7(∇β G)7 = 0, (Vβ F)7 = 0. for t = 8(∇β G)8 = 0, (Vβ F)8 = 0.  k k

  |k = 1, . . . , p be the fundamental set of an autoassociative Lemma 4.9 Let  X ,X k e k k Alpha–Beta  k memory

of type max represented by V, with X = τ (x , h )∀k ∈ {1, . . . , p}, k   and let X , X | k = 1, . . . , p be the fundamental set of an autoassociative Alpha–Beta e k ¯k k memory of type min represented  p by i, with X = τ (x , h ), ∀k ∈ {1, . . . , p}. Also, if we p define vector u ∈ A as u = i=1 h , and take a fixed index ∀r ∈ {1, . . . , p}, let us consider two noisy versions of pattern X r ∈ An+ p : vector F = τ e (xr , u) ∈ An+ p which is an additive noise altered version of pattern X r , and vector G = τ e (xr , w) ∈ An+ p , which is a substracr , being w ∈ A p a vector whose components take the tive noise altered version of pattern X values wi = u i − 1 ∀i ∈ {1, . . . , p}. If during the recalling phase, G is presented as input to memory  and F is presented as input to memory V, and if also it holds that (V β F)t = 1 for an index t ∈ {n + 1, . . . , n + p}, fixed such that t  = n + r , then (∇β G)t = 1. Proof Due to the way vectors Xr , F y G are built, we have that Ft = 1 is the component in the vector with additive noise corresponding to component X tr , and G t = 0 is the component tr . Also, since t  = n + r , in the vector with substractive noise corresponding to component X tr = 1. There are two possible cases: we can see that X tr  = 1, that is X tr = 0, and X Case 1 Pattern G does not contain any component with value 1. That is, G j = 0∀ j ∈ {1, . . . , n + p}. By Lemma 4.5 λt j  = 2∀ j ∈ {1, . . . , n + p}, thus β(λt j , G j ) = 0∀ j ∈

n+ p {1, . . . , n + p}, which means that ∇β G t = ∨ j=1 β(λt j , G j ) = 0. In other words, expression (∇β G)t = 1 is false. The only possibility for the theorem to hold, is for expression (V β F)t = 1 to be false too. That is, we need to show that (V β F)t = 0. According to Corollary 4.1, the latter is true if for every t ∈ {n + 1, . . . , n + p} with t  = n + r , exists j ∈ {1, . . . , n + p} such that (F j = 0 AND νt j  = 2). Now, t  = n + r indicates that ∃s ∈ µ µ {1, . . . , p}, s  = r such that t = n + s, and by Lemma 4.6 α(X ts , X sj ) ≥ α(X t , X j ) ∀µ ∈ p µ µ {1, . . . , p}, ∀ j ∈ {1, . . . , n + p}, from where we have νt j = ∨µ=1 α(X t , X j ) = α(X ts , X sj ),

123

Alpha–Beta bidirectional associative memories

31

s and by noting the equality X ts = X n+s = 1, it holds that:

νt j = α(1, X sj ) ∀ j ∈ {1, . . . , n + p}

(1)

On the other side, ∀i ∈ {1, . . . , n} the following equalities hold: X ir = xir = 0 and xs , it is clear that ∃h ∈ {1, . . . , p} such that

X is = xis and also, taking into account that xr  = x hs  = x hr ; meaning x hs = 1 = X hs and therefore

νth = α(1, X hs ) = α(1, 1) = 1 Finally, since ∀ i ∈ {1, . . . , n} it holds that Fi = X ir = xir = 0, in particular Fh = 0. Then we have proven that for every t ∈ {n + 1, . . . , n + p} with t  = n + r , exists j ∈ {1, . . . , n + p} such that (F j = 0 AND νt j  = 2), and by Corollary 4.1 it holds that (V β F)t = 0, thus making expression (V β F)t = 1 is false. Case 2 Pattern G contains, besides the components with value of 0, at least one component with value 1. That is, ∃h ∈ {1, . . . , n + p} such that G h = 1. Due to the way vectors G and F are built ∀i ∈ {1, . . . , n}G i = Fi and, also, necessarily 1 ≤ h ≤ n and thus Fh = G h = 0. By hypothesis ∃t ∈ {n + 1, . . . , n + p} fixed such that t  = n + r and (V β F)t = 1, and by Lemma 4.4 ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2). Given the way vector F is built we have that ∀ j ∈ {n + 1, . . . , n + p}G j = 1, thus making the former expression like this: ∀ j ∈ {1, . . . , n + p}(F j = 0 → νt j = 2). Let J be a set, proper subset of {1, . . . , n}, defined like this: J = { j ∈ {1, . . . , n}|F j = 0}. The fact that J is a proper subset of {1, . . . , n} is guaranteed by the existence of G h = 1. Now, t  = n +r indicates that ∃s ∈ {1, . . . , p}, s  = r such s )∀ j ∈ {1, . . . , n + p}, that t = n + s, and by Lemma 4.7 νt j = α(1, X sj ) and λt j = α(0, X j s from where we have that ∀ j ∈ J , X j = 0, because if this was not the case, νt j  = 0. This means that for each j ∈ J X sj = 0 = F j which in turn means that patterns Xr and Xs coincide with value 0 in all components with index j ∈ J . Let us now consider the complement of set J , which is defined as J c = { j ∈ {1, . . . , n}|F j = 1}. The existence of at least one value j0 ∈ J c for which F j0 = 1 y X sj0 = 0 is guaranteed by the known fact that xr  = xs . Let us see, if X sj = 1∀ j ∈ J c then ∀ j ∈ {1, . . . , n} it holds that X sj =F j , which would mean that xr = xs . Since ∃ j0 ∈ J c for which F j0 =1 and X sj0 =0, this means that ∃ j0 ∈ J c for s =0. Now, β(λt j0 , G j0 )=β(α(0, X s ), 1)=β(α(0, 0), 1)=β(1, 1)=1, which G j0 =1 and X j0 j0 and finally

∇β G



n+ p

t

= ∨ β(λt j , G j ) = β(λt j0 , G j0 ) = 1 j=1




Example 3.18 From Example 3.2, let us takeX1 and build the noisy vector F, with additive noise; from Example 3.6 let us takeX1 and build the noisy vector G, with additive noise; and let us use the memories V y  shown in Example 3.15. ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1 1 1 1 ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎢ ⎢1⎥ ⎥ ⎥ ⎢ ⎢ 1 1 1 1 1 ⎥ ⎥ ⎢ ⎥ ⎥ ⎢ ⎢  X = ⎢ ⎥, F = ⎢ ⎥, X = ⎢ ⎥, G = ⎢ ⎢0⎥, 1 1 0 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣1⎦ ⎣0⎦ ⎣0⎦ ⎣1⎦ 1 0 0 1

123

32

M. E. Acevedo-Mosqueda et al.

⎡

1 ⎢1 ⎢ ⎢2 ⎢ ⎢1 V=⎢ ⎢1 ⎢ ⎢2 ⎢ ⎣2 1

1 1 2 1 1 2 2 1

2 2 1 1 1 2 1 2

2 2 2 1 1 2 2 2

2 2 2 1 1 2 2 2

2 2 2 2 2 1 2 2

2 2 2 2 2 2 1 2

⎡ ⎤ 1 1 ⎢1 2⎥ ⎢ ⎥ ⎢0 2⎥ ⎢ ⎥ ⎢ 2⎥ ⎥,  = ⎢0 ⎢0 ⎥ 2⎥ ⎢ ⎢1 ⎥ 2⎥ ⎢ ⎣1 ⎦ 2 0 1

1 1 0 0 0 1 1 0

0 0 1 0 0 1 0 1

1 1 1 1 0 1 1 1

0 0 0 0 1 0 0 0

0 0 0 0 0 1 0 0

0 0 0 0 0 0 1 0

⎤ 0 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎥ ⎥ 0⎦ 1

Now we present F to V and G to  in order to obtain:

⎡1⎤ ⎢1⎥ ⎢⎥ ⎢1⎥ ⎢⎥ ⎢1⎥ , V∆ β F = ⎢1⎥ ⎢⎥ ⎢1⎥ ⎢1⎥ ⎢⎥ ⎢⎣1⎥⎦

⎡1⎤ ⎢1⎥ ⎢ ⎥ ⎢1⎥ ⎢ ⎥ ⎢1⎥ Λ∇β G = ⎢0⎥ ⎢ ⎥ ⎢1⎥ ⎢1⎥ ⎢ ⎥ ⎢⎣ 1 ⎥⎦

With n = 4 and p = 4, t ∈ {5, 6, 7, 8}, and for this example r = 1, thus t  = 4 + 1 = 5. Then, for t = 6(∇β G)6 = 1, (Vβ F)6 = 1. for t = 7(∇β G)7 = 1, (Vβ F)7 = 1. for t = 8(∇β G)8 = 1, (Vβ F)8 = 1.  k k

  |k = 1, . . . , p be the fundamental set of an Theorem 4.5 (Main Theorem) Let  X ,X k e k k autoassociative Alpha–Beta of type max  k kmemory

 represented by V, with X = τ (x , h )∀k ∈  | k = 1, . . . , p be the fundamental set of an autoassociative {1, . . . , p}, and let  X ,X e k ¯k k Alpha–Beta memory of type min represented  pby , with X = τ (x , h ), ∀k ∈ {1, . . . , p}. Also, if we define vector u ∈ A p as u = i=1 hi , and take a fixed index r ∈ {1, . . . , p}, let us consider two noisy versions of pattern X r ∈ An+ p : vector F = τ e (xr , u) ∈ An+ p , which is an additive noise altered version of pattern X r , and vector G = τ e (xr , w) ∈ An+ p , which is a substractive noise altered version of pattern  Xr , being w ∈ A p a vector whose components take the values wi = u i − 1∀i ∈ {1, . . . , p}. Now, let us assume that during the recalling phase, G is presented as input to memory  and F is presented as input to memory V, and patterns S= ∇β G ∈ An+ p and R = Vβ F ∈ An+ p are obtained. If when taking vector R as argument the contraction vectorial transform r = τ c (R, n) ∈ A p is done, and when taking vector S as argument the contraction vectorial transform s = τ c (S, n) ∈ A p is done, then H = (r AND s¯) will be the kth one-hot vector of p bits, where s¯ is the negated of s. Proof From the definition of contraction vectorial transform we have that ri = Ri+n = (Vβ F)i+n and si = Si+n = (∇β G)i+n for 1 ≤ i ≤ p, and in particular, by making i = k

123

Alpha–Beta bidirectional associative memories

33

we have r k = Rk+n

= (Vk β F)k+n and sk = Sk+n

= (∇kβ G)k+n . By Lemmas 4.1 and 4.2  we have Vβ F n+k = X n+k = 1 and ∇β G n+k = X n+k = 0; and thus: Hk = rk AND s¯k = 1 AND ¬0 = 1 AND 1 = 1. Now, by Lemma 4.8 we know that if (∇β G)t = 0 tal que t = i + n is a fixed index with t  = n + k, then (V β F)t = 0; thus: Hi = ri AND s¯i = (Vβ F)t AND ¬(∇β G)t = 0 AND ¬0 = 0 AND 1 = 0 On the other side, by Lemma 4.9 it is known that if (V β F)q = 1 for a fixed index q = i + n such that q  = n + k, then (∇β G)q = 1. According to the latter: Hi = ri AND ¬si = (Vβ F)q AND ¬(∇β G)q = 1 AND ¬1 = 1 AND 0 = 0 Then Hi = 1 for i = k y Hi = 0 for i  = k. Therefore, and according to Definition 1, H will be the kth one-hot vector of p bits.

3.2 Theoretical Foundation of Stages 2 and 4 In this section is presented the theoretical foundation which serves as the basis for the design and operation of Stages 2 and 4, whose main element is an original variation of the Linear Associator. Let {(xµ , yµ )|µ = 1, 2, . . . , p} with A = {0, 1}, xµ ∈ An and yµ ∈ Am be the fundamental set of the Linear Associator. The Learning Phase consists of two stages: • •

For each of the p associations (xµ , yµ ) find matrix yµ · (xµ )t of dimensions m x n. The p matrices are added together to obtain the memory p

M= µ=1

t  yµ · xµ = m i j m×n

in such way that the ijth component of memory M is expressed as: p

mi j =

µ µ

yi x j µ=1

The Recalling Phase consists of presenting an input pattern xω to the memory, where ω ∈ {1, 2, . . . , p}, and do operation ⎡

p

M · xω = ⎣



yµ · x

µ t

⎤ ⎦ · xω

µ=1

The following form of expression allow us to investigate the conditions that must be met in order for the proposed recalling method to give perfect outputs as results: M · x ω = yω ·




xω

t

 · xω +

yµ ·




xµ

t

· xω



µ=ω

For the latter expression to give pattern yω as result, it is necessary that two equalities hold:  • (xω )t · xω = 1  µ t ω • (x ) · x = 0 as long as µ  = ω.

123

34

M. E. Acevedo-Mosqueda et al.

This means that, in order to have perfect recall, vectors xµ must be orthonormals. If that happens, then, for µ = 1, 2, . . . , p, we have: ⎞ ⎛ 1⎞ ⎛ 1 y1 y1 0 0 · · · 0(n) 1 ⎟ ⎜ y1 ⎟

⎜ ⎜ 2⎟ ⎜ y2 0 0 · · · 0(n) ⎟ y1 · (x1 )t = ⎜ . ⎟ · x11 , x21 , . . . , xn1 = ⎜ . . . ⎟ . ⎝ .. ⎠ ⎝ .. .. .. · · · .. ⎠ ym1 ym1 0 0 · · · 0(n) ⎛ ⎞ 0 y12 ⎜0 ⎜ y2 ⎟

⎜ ⎜ 2⎟ y2 · (x2 )t = ⎜ . ⎟ · x12 , x22 , . . . , xn2 = ⎜ . ⎝ .. ⎝ .. ⎠ ⎛

y12 y22 .. . 0 ym2

ym2

⎛ p⎞ 0 y1 ⎜0 ⎜ yp ⎟ ⎜

⎜ 2 ⎟ p p p y p · (x p )t = ⎜ . ⎟ · x1 , x2 , . . . , xn = ⎜ ⎜ .. ⎝ .. ⎠ ⎝. p ym 0 ⎛

Therefore,

⎛

y11 1 p ⎜ t ⎜ y2 yµ · x µ = ⎜ . M= ⎝ .. µ=1 ym1

y12 y22 .. . ym2

y13 y23 .. . ym3

0 0 .. . 0

0 0 .. . 0 0 0 .. . 0

⎞ · · · 0(n) · · · 0(n) ⎟ ⎟ . ⎟ · · · .. ⎠ · · · 0(n)

⎞ p · · · y1(n) p · · · y2(n) ⎟ ⎟ ⎟ .. ⎟ ⎠ ··· . p · · · ym(n)

p⎞ · · · yn p · · · y2 ⎟ ⎟ .. ⎟ ··· . ⎠ p · · · ym

Taking advantage of the characteristic shown by the Linear Asssociator when the input patterns are orthonormals, and given that, by Definition 1, one-hot vectors vk with k = 1, . . . , p are orthonormals, we can obviate the learning phase by avoiding the vectorial operations done by the Linear Associator, and simply put the vectors in order, to form the Linear Associator. Stages 2 and 4 correspond to two modified Linear Associators, built with vectors y and x, respectively, of the fundamental set. 3.3 Algorithm In this section we describe, step by step, the processes required by the Alpha–Beta BAM, in the Learning Phase as well as in the Recalling Phase (by convention only) in the direction x → y, the algorithm for Stages 1 and 2. The following algorithm describes the steps needed by the Alpha–Beta bidirectional associative memory in order to realize the learning and recalling phases in the direction x → y. Learning phase 1. 2.

For each index k ∈ {1, . . . , p} do expansion: Xk = τ e (xk , hk ) Create an Alpha–Beta autoassociative memory of type max V with the fundamental set !  " # Xk , Xk " k = 1, . . . , p

3.

For each index k ∈ {1, . . . , p} do expansion:  Xk = τ e (xk , h¯ k )

123

Alpha–Beta bidirectional associative memories

35

4.

Create an Alpha–Beta autoassociative memory of type min  with the fundamental set #  " ! k " k = 1, . . . , p k , X X

5.

Create a matrix consisting of a modified Linear Associator with patterns yk ⎡ 1 2 p⎤ y1 y1 · · · y1 p ⎢ y1 y2 · · · y2 ⎥ ⎥ ⎢ 2 2 LAy = ⎢ . . . ⎥ ⎣ .. .. · · · .. ⎦ p yn1 yn2 · · · yn

Recalling phase 1. 2.

Present, as input to Stage 1, a vector of the fundamental set xµ ∈ An for some index µ ∈ {1, . . . , p} Build vector u ∈ A p in the following manner: p

u=

hi i=1

3. 4.

Do expansion: F = τ e (xµ , u) ∈ An+ p Operate the Alpha–Beta autoassociative memory max V with F, in order to obtain a vector R of dimension n + p R = Vβ F ∈ An+ p

5. 6.

7. 8. 9.

Do contraction r = τ c (R, n) ∈ A p If (∃k ∈ {1, . . . , p} such that hk = r) it is assured that k = µ (based on Theorem 4.2), and the result is hµ . Thus, operation LAy · r is done, resulting in the corresponding yµ . STOP. Else { Build vector w ∈ A p in such way that wi = u i − 1, ∀i ∈ {1, . . . , p} Do expansion: G = τ e (xµ , w) ∈ An+ p Operate the Alpha–Beta autoassociative memory min  with G, in order to obtain a vector S of dimension n + p S = ∇ β G ∈ An+ p

10. 11.

12.

Do contraction s = τ c (Sµ , n) ∈ A p If (∃k ∈ {1, . . . , p} such that h¯ k = s) it is assured that k = µ (based on Theorem 4.4), and the result is hµ . Thus, operation LAy · s¯ is done, resulting in the corresponding yµ . STOP. Else { Do operation t = r∧¯s, where  is the symbol of the logical AND. The result of this operation is hµ (based on Theorem 4.5). Operation LAy · t is done, in order to obtain the corresponding yµ . STOP.}}

4 Results In this section, two applications that make use of the algorithm described in the former section are presented, illustrating the optimal functioning which the Alpha–Beta bidirectional associative memories exhibit.

123

36

M. E. Acevedo-Mosqueda et al.

Fig. 3 (a) The corresponding translation of departure is partida. (b) With just a part of the Word departure the correct result is still given. (c) The translator works in an optimal manner even if the input word is misspelled

The first application is a Spanish–English/English–Spanish translator. As second application, a fingerprint verifier is presented. For both applications implementation, the programming language Visual C++ 6.0 was used. The programs were executed on a Sony VAIO laptop computer with a Pentium 4 processor at 2.8 GHz. 4.1 Spanish–English/English–Spanish translator This program is able to translate words in English to Spanish and vice versa. For the Learning Phase two text files were used. These files contain 120 words each, in English and Spanish, respectively. With both files the Alpha–Beta bidirectional associative memory is created. The learning process lasts, approximately, 1 06 . The program was executed on a Sony VAIO laptop computer with a Pentium 4 processor at 2.8 GHz. After the Alpha–Beta BAM is built, during the Recalling Phase, a word is written as input, either in English or Spanish, and the translation mode is chosen. The word in the corresponding language appears immediately. An example can be seen in Fig. 3a. The word to be translated is “departure” and its corresponding translation in Spanish is “partida”. The translator presents other advantages as well. For instance, let us suppose that only a part of the word departure is written as input, say “departu”. The program will give as output the word “partida” (see Fig. 3b). Now, let us assume that instead of writing the last “e”, a “w” is written as a typo. The result can be seen in Fig. 3c. In this example we can see that, a misspell, which at the pattern level means that a noisy pattern has been given as input, does not limit the translator performance. The advantages shown by the translator stress out the advantages presented by the Alpha– Beta bidirectional associative memories. These memories are immune to certain amounts and kinds of noise, properties which have not been characterized yet.

123

Alpha–Beta bidirectional associative memories

37

Fig. 4 The bidirectional process of the model is shown in both screens. (a) The input pattern is a fingerprint, its corresponding output pattern a number. (b) A number is chosen as input pattern and its corresponding fingerprint is perfectly recalled

4.2 Fingerprint verifier The first part of the identification consists in building the Alpha–Beta BAM; this is, the learning phase is realized. The input patterns set is made up of 40 fingerprints, which were obtained from the Fingerprint Verification Competition (FPV2000) [46]. Originally, these images have dimensions of 240 × 320 pixels. The Advanced Batch Converter image editor was used to modify the fingerprints dimensions, such that it was possible to present 40 images on one screen. The final dimensions of the images are 80 × 170 pixels. The output patterns set contains 40 images, each representing a number between 1 and 40, inclusive. The dimensions of this images are 10 × 10 pixels. For the verifier implementation it will be assumed that x represents the input patterns set. Given that fingerprints are represented with bidimensional images, and Alpha–Beta BAMs work with column vectors, it becomes necessary to convert the images to a vectorial form. Therefore, input vectors dimension is n = 80 × 170 = 13, 600. In a similar way, the output patterns set s represented by y, and its images are converted to vector of dimension m = 10 × 10 = 100. In this case, the number of trained patterns is p = 40. The learning phase process, whose algorithm was described in the former section, lasts approximately 1 min and 34 s. Once the Alpha–Beta BAM has been built, the program allows to choose any one of the learned fingerprints, or any one number associated to the fingerprints. If the chosen pattern is a fingerprint, the recalled pattern is the number associated to that specific fingerprint (see Fig. 4a). On the other hand, if a number is selected, the result of recall is its corresponding fingerprint (see Fig. 4b). For the recalling of a fingerprint parting from a number, the same algorithm that was described in the prior section is used, only this time the pattern presented to the verifier is a member of the pattern set y. In order to test the efficacy of the verifier, each of the 40 images depicting fingerprints was picked, obtaining in each case the perfect recall of its corresponding number. In the same way, the 40 numbers were individually presented as input; the recall phase delivered in a perfect manner, each of the corresponding fingerprints. From that we can observe that the verifier exhibits perfect recall of all learned patterns. Actually, in this example only 40 pairs of patterns were used; however, the verifier can asso-

123

38

M. E. Acevedo-Mosqueda et al.

ciate a large amount of patterns and its recall will always be perfect. The limit of learned patterns will be given by the available memory in the computer executing the verifier. The perfect recall of patterns shown by the verifier is due to the model on which it is based: the Alpha–Beta bidirectional associative memories.

5 Conclusions In the present work, a model of Bidirectional Associative Memories has been presented; this model is based upon the Alpha–Beta associative memories, from which it inherits its name. The Alpha–Beta bidirectional associative memories present perfect recall of all patterns in the fundamental set. This optimal behavior is mathematically founded through five Theorems and nine Lemmas which are described in Sect. 3. The applications presented in Sect. 4 showed that the Alpha–Beta bidirectional associative memories recall, in a perfect manner, all patterns trained during the Learning Phase. In the translator case, the number of pattern pairs learned was 120, while in the case of the fingerprint verifier, the BAM was trained with 40 pairs of patterns. In both instances, all the recalls were successful. These two examples were used only to illustrate the main, and most important, characteristic of the Alpha–Beta bidirectional associative memories: perfect recall of all the patterns. However, the capacity of these memories is not limited by the model itself, since it has been formally demonstrated that their capacity is 2min(n,m) (being n and m the dimensions of the input and output patterns, respectively). Actually, the only limit to the Alpha–Beta BAM capacity would be given by the characteristics of the equipment on which they are implemented. The translator application indicates that the proposed model is able to manage input patterns with certain levels of noise. The type and amount of noise supported by the Alpha–Beta BAM have not yet been characterized. Acknowledgements The authors would like to thank the Instituto Politécnico Nacional (Secretaría Académica, COFAA, SIP, and CIC), the CONACyT, and SNI for their economical support to develop this work.

References 1. Hassoun MH (1993) Associative neural memories. Oxford University Press, New York 2. Kohonen T (1989) Self-organization and associative memory. Springer-Verlag, Berlin 3. McCulloch W, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133 4. Ritter GX, Sussner P (1996) An Introduction to morphological neural networks. In Proceedings of the 13th international conference on pattern recognition, (IV) Track D pp 709–717 5. Rosenblatt F (1958) The perceptron: a ;probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386–408 6. Widrow B, Lehr MA (1990) 30 years of adaptive neural networks: perceptron, madaline and backpropagation. In: Proceedings of the IEEE 78(9) pp 1415–1441 7. Werbos PJ (1990) Backpropagation through time: what it does and how to do it. In: Proceedings of the IEEE 78(10) pp 1550-1560 8. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. In: Proc Nat Acad Sci 79:2554–2558 9. Minsky M, Papert S (1969) Perceptrons. MIT Press, Cambridge 10. Steinbuch K (1961) Die Lernmatrix. Kybernetik 1(1):36–45 11. Willshaw D, Buneman O, Longuet-Higgins H (1969) Non-holographic associative memory. Nature 222:960–962

123

Alpha–Beta bidirectional associative memories

39

12. Anderson JA (1972) A simple neural network generating an interactive memory. Math Biosci 14:197– 220 13. Kohonen T (1972) Correlation matrix memories. IEEE Trans Comput C-21(4):353–359 14. Kosko B (1988) Bidirectional associative memories. IEEE Trans Syst Man, Cybernet 18(1):49–60 15. Haines K, Hecht-Nielsen R (1988) A BAM with increased information storage capacity. IEEE Int Conf Neural Networks 1:181–190 16. Leung CS, Cheung KF (1991) Householder encoding for discrete bidirectional associative memory. IEEE Int Joint Conf Neural Networks 1:237–241 17. Shen D, Cruz JB, Jr (2005) Encoding strategy for maximum noise tolerance bidirectional associative memory. IEEE Trans Neural Networks 16(2):293–300 18. Wang Y-F, Cruz JB, Jr, Mulligan JH, Jr (1991) Guaranteed recall of all training pairs for bidirectional associative memory. IEEE Trans Neural Networks 1(6):559–567 19. Jiang X, Yau WY (2000) Fingerprint minutiae matching based on the local and global structures. In: Proc internat conf on pattern recognition 1038(-1041):1038–1041 20. Wang Y-F, Cruz JB, Jr, Mulligan JH, Jr (1989) An enhanced bidirectional associative memory. IJCNN Int Joint Conf Neural Networks 1:105–110 21. Tai H-M, Wu C-H, Jong T-L (1989) High-order bidirectional associative memory. Electron Lett 25(21):1424–1425 22. Jeng Y-J, Yeh C-C, Chiveh TD (1990) Exponential bidirectional associative memories. Electron Lett 26(11):717–718 23. Wang Y-F, Cruz JB, Jr, Mulligan JH, Jr (1990) Two coding strategies for bidirectional associative memory. IEEE Trans Neural Networks 1(1):81–92 24. Wang Y-F, Cruz JB, Jr, Mulligan JH, Jr (1990) On multiple training for bidirectional associative memory. IEEE Trans Neural Networks 1(3):275–276 25. Jeng Y-J, Yeh C-C, Chiueh T-D (1992) Generalised stable bidirectional associative memory. Electron Lett 28(15):1396–1398 26. Yu H, Wang Z (1992) Bidirectional associative memories on the matching criterion of weighted Hamming distance. Proceedings of the 31st IEEE conference on decision and control vol 4, pp3481–3484 27. Lee D-L, Wang W-J (1993) Improvement of bidirectional associative memories by using correlation significance. Electron Lett 29(8):688–690 28. Perfetti R (1993) Optimal gradient descent learning for bidirectional associative memories. Electron Lett 29(17):1556–1557 29. Wang W, Lee D (1993) A modified bidirectional decoding strategy based on the BAM structure. IEEE Trans Neural Network 4(4):710–717 30. Khorasani K, Cuffaro A, Grigoriu T (1994) A new learning algorithm for bidirectional associative memory neural networks. IEEE Int Conf Neural Networks 2:1115–1120 31. Xu Z-B, Leung Y, He X-W (1994) Asymmetric bidirectional associative memories. IEEE Trans Syst Man Cybernet 24(10):1558–1564 32. Oh H, Kothari S (1992) A Pseudo-relaxation learning algorithm for bidirectional associative memory. IJCNN 2:208–213 33. Leung CS (1994) Optimum learning for bidirectional associative memory in the sense of capacity. IEEE Trans Syst, Man Cybernet 24(5):791–796 34. Wang Z-O (1996) A bidirectional associative memory based on optimal linear associative memory. IEEE Trans Comput 45(10):1171–1179 35. Sarkar D (1996) A three-stage architecture for bidirectional associative memory. IEEE Int Conf Neural Networks 1:531/4–531/9 36. Haryono, Sadananda R, Phien HN (1997) Orthogonal schemes for bidirectional associative memories. IEEE Trans Syst, Man Cybernet B27(3):543-551. 37. Ritter GX, Diazde Leon JL, Sussner P (1999) Morphological bidirectional associative memories. Neural Networks 12:851–867 38. Wu Y, Pados DA (2000) A feedforward bidirectional associative memory. IEEE Trans Neural Networks 11(4):859–866 39. Liang J, Cao J (2004) Exponential stability of continuous-time and discrete time bidirectional associative memory neural networks with delays. Chaos, Solitons Fractals 22:773–785 40. Lee D-L, Chuang TC (2004) Design of general bidirectional associative memories with improved recall capability. Int J Neural Systems 14(5):325–328 41. Zheng G, Givigi SN, Zheng W (2005) A new strategy for designing bidirectional associative memories. Lecture Notes in Computer Science, vol 3496, Springer-Verlag, pp 398-403 42. Chartier S, Boukadoum M (2006) A Bidirectional heteroassociative memory for binary and grey-level patterns. IEEE Trans Neural Networks 17(2):385–396

123

40

M. E. Acevedo-Mosqueda et al.

43. Arik S (2005) Global asymptotic stability analysis of bidirectional associative memory neural networks with time delays. IEEE Trans Neural Networks 16(3):580–585 44. Park J (2006) Robust stability of bidirectional associative memory neural networks with time delays. Phys Lett A 349:494–499 45. Yáñez-Márquez C (2002) Associative memories based on order relations and binary operators (in Spanish) PhD Thesis. Center for Computing Research, México 46. Acevedo ME (2006) Alpha-Beta bidirectional associative memories (in Spanish) PhD Thesis. Center for Computing Research, México 47. http://bias.csr.unibo.it/fvc2000/download.asp

123