A Model for Hierarchical Associative Memories via Dynamically ...
A Model for Hierarchical Associative Memories via Dynamically Coupled GBSB Neural Networks Rog´erio M. Gomes1 , Antˆ onio P. Braga2, and Henrique E. Borges1 1 CEFET/MG, Laborat´ orio de Sistemas Inteligentes, Av. Amazonas, 7675 - Belo Horizonte - MG - Brasil - CEP 30510-000 {rogerio, henrique}@lsi.cefetmg.br 2 PPGEE-UFMG, Laborat´ orio de Inteligˆencia e T´ecnicas Computacionais, Av. Antˆ onio Carlos, 6627 - Belo Horizonte - MG - Brasil - CEP 31270-010 [email protected]
Abstract. Many approaches have emerged in the attempt to explain the memory process. One of which is the Theory of Neuronal Group Selection (TNGS), proposed by Edelman [1]. In the present work, inspired by Edelman ideas, we design and implement a new hierarchically coupled dynamical system consisting of GBSB neural networks. Our results show that, for a wide range of the system parameters, even when the networks are weakly coupled, the system evolve towards an emergent global associative memory resulting from the correlation of the lowest level memories. Keywords: Hierarchical memories, Coupled neural networks, Dynamical systems, Artificial neural networks, TNGS.
1
Introduction
Presently, studies in neuroscience have revealed, by means of experimental evidences, that memory process can be described as being organized, functionally, in hierarchical levels, where higher levels would coordinate sets of functions of the lower levels [1] [2]. One of the theories that is in compliance with these studies is the Theory of Neuronal Group Selection (TNGS) proposed by Edelman [2]. TNGS establishes that correlations of the localized neural cells in the cortical area of the brain, generate clusters units denoted as: neuronal groups (cluster of 50 to 10.000 neural cells), local maps (reentrant clusters of neuronal groups) and global maps (reentrant clusters of neural maps). A neuronal group (NG) is a set of tightly coupled neurons which fire and oscillates in synchrony. Each neuron belongs only to a single neuronal group, which is spatially localized and functionally hyper-specialized. According to TNGS, NG are the most basic structures in the cortical brain, from which memory and perception processes arise, and can been seen as performing the most primitive sensory-effector correlations. W. Duch et al. (Eds.): ICANN 2005, LNCS 3696, pp. 173–178, 2005. c Springer-Verlag Berlin Heidelberg 2005
174
R.M. Gomes, A.P. Braga, and H.E. Borges
A local map is a composition of NG, also spatially localized in the cortical area. Two local maps, functionally different, can develop reentrant connections, resulting in what Edelman [1] calls categorization. Edelman [1] states that a significant number of different neuronal groups could have the same functionality within a given map, that is, could respond to the same stimuli. A global map is a dynamic structure containing multiple reentrant local maps which are capable of interacting with non-mappable areas of the brain, such as the limbic system [2]. A global map is a set of connected local maps and perform “categorizations” (correlations) of local maps. They are not spatially localized but, in fact, they are spread throughout the cortex. Global maps provide a “global or emergent behaviour” of the cortical activities (perception in action) and generate a complete experience in the world, i.e., an experience with qualia. A continuous selection of existing local maps in a global map by selection of additional reentrant connection allows forming new classification couples. Inspired by these ideas, a model of hierarchically coupled dynamical system, using GBSB (Generalized-Brain-State-in-a-Box) neural networks is described in this paper, which integrates the concepts of dynamical systems theory, TNGS and Artificial Neural Networks (ANNs) aiming at building multi-level memories. This paper is organized as follows. In Section 2 we propose a model of coupled GBSB neural networks and show how multi-level memories may arise within it. Section 3 illustrates the use of the algorithm developed in Section 2 with an example from the literature [3] [4]. Finally, Section 4 concludes the paper and presents some relevant extensions of this work.
2
Proposal for the Construction of Multi-level Memories
In order to develop this new model we use an extension of the original BSB Brain-State-in-a-Box [5] called GBSB (Generalized-Brain-State-in-Box ) [6]. The behaviour of the neural network energy in a discrete BSB model was studied by Golden [7]. Cohen and Grossberg [8] discussed a continuous BSB model based on Liapunov equations, while Hui and Zak [6] discussed the stability of the GBSB model in a non-symmetric diagonally dominant weight matrix case. The GBSB model was chosen due to the fact that its characteristics, which are, in short: asymmetric synapses, different bias and maximum and minimum fire rates, redundancy, non-linear dynamics and self-connection for each neuron. In our proposed model we build a two level hierarchical memory where, in accordance with figure 1, each one of the GBSB networks (A, B and C) plays the role of a neuronal group or, in our case, a first-level memory. In a given cluster, each neuron performs synapses with each other neuron of the same cluster, i.e., the GBSB is a fully connected asymmetric neural network. Beyond this, some selected neurons in a cluster are bidirectionally connected with some selected neurons in the others clusters [4]. These inter-cluster connections can be represented by a weight correlation matrix Wcor , which accounts for the contribution of one cluster to another one due to coupling. An analogous procedure could be followed in order to build higher levels in the hierarchy [1].
A Model for Hierarchical Associative Memories
175
Local Maps
W (i,a)(j,a)
WCor(i,a)(j,b) i
j Wcor(j,b)(i,a)
A
B
Neural Groups
GBSB Nets
C
Fig. 1. Network design
Our proposed coupled GBSB model extends the GBSB model for single networks discussed in [6], by means of adding a fourth term that represents the inter-group connections. Consequently, our new model can be defined by the equation: N a k+1 β(i,a) w(i,a)(j,a) xk(j,a) + β(i,a) f(i,a) + x(i,a) = ϕ xk(i,a) + j=1 ⎞ (1) Nq Nr k + γ(i,a)(j,b) wcor(i,a)(j,b) x(j,b) ⎠ , b=1 b=a
j=1
where the three first terms represent the equations of a GBSB model for Na uncoupled GBSB networks, meaning in our model intra-group synapses (i.e., in the ath network or neuronal group). The sum over j, in the fourth term, labels the Nq neurons in the bth neuronal group that have correlation to neuron i in the ath neuronal group. The strength or density of the inter-group synapses are parameterized by γ(i,a)(j,b) . The activation function ϕ is a linear saturating function whose ith component is defined as follows: ⎧ k ⎪ ⎨ +1 if y(i,a) > +1 k k k k if −1 ≤ y(i,a) ≤ +1, xk+1 ϕ(y(i,a) ) = y(i,a) (2) (i,a) = ϕ(y(i,a) ), ⎪ ⎩ −1 if y k < −1 (i,a) k is the argument of the function ϕ of the equation 1. where y(i,a) In order to complete our model we present now a Lyapunov function (energy) of the coupled system, which can be defined as [9]:
E=
− 21
Nr N a
β(i,a) w(i,a)(j,a) x(i,a) x(j,a) −
a=1 i,j=1
Nr Na
β(i,a) f(i,a) x(i,a)
a=1 i=1
(3) −
Nq Nr N a a,b=1 a=b
i=1 j=1
γ(i,a)(j,b) wcor(i,a)(j,b) x(i,a) x(j,b) ,
176
R.M. Gomes, A.P. Braga, and H.E. Borges
where the first term represents the energy of the individual neuronal groups. The second term gives the contribution to energy due to external factors (i.e., the bias field ). Finally, the third term in equation 3 is due to the inter-group connections. A detailed mathematical analysis of equation 3, describing the energy of the coupled system can be found in [9], where it was shown that it presents two important features: the whole system evolves to a state of minimum energy, even when the neuronal groups are weakly coupled; the inter-group coupling, which establishes the second-level correlations, does not destroy the first-level memories structures.
3
Coupled GBSB Experiments
Computational experiment consisting of three GBSB networks connected (Fig. 1) were conducted and the results were compared with the ones presented in [4]. Although the experiment presented here is a quite simple one, it is intended to make it clear the procedure for the construction of multi-level associative memories. More complex computational experiments will be presented elsewhere. In our simulations each network or neuronal group contains 10 neurons and we selected 6 out of 1024 possible patterns to be stored as our first-level memories. The weight matrix of the individual networks followed the definition proposed in [3]. The selected set of patterns stored as first-level memories was: V1 = [ -1 1 1 1 1 1 -1 -1 -1 -1 ] V2 = [ 1 1 -1 -1 -1 1 -1 -1 1 -1 ] V3 = [ -1 1 1 1 -1 -1 1 -1 -1 -1 ] V4 = [ -1 1 -1 -1 -1 -1 1 -1 1 1 ] V5 = [ 1 -1 -1 1 1 -1 1 1 1 -1 ] V6 = [ 1 1 -1 1 -1 1 1 1 -1 -1 ]
(4)
Each network A, B and C was carefully designed to present the same asymptotically stable fixed point structure presented in [3]. To design these networks we followed the approach of [4]. In addition, amongst the 63 = 216 possible combinations of the 3 sets of first-level memories, we have chosen 2 triplets or global patterns to be our second-level memories (local maps). The arrangement of the global patterns determines the inter-group correlation matrix Wcor by a generalized Hebb rule. The system was initialized in one of the networks A, B or C, randomly, and in one of their first-level memories that establish a correlation in accordance with table 1. The two other networks, in turn, were initialized in one of the 1024 possible patterns, also, randomly. Then, we measured the number of times that the system converges to a configuration of triplets1 , considering networks totally or partially coupled. Neurons that took part of the inter-group connections was chosen randomly. Points in our experiments were averaged over 1000 trials for each value of γ. In our experiment a typical value of β was chosen (β = 0.3) and the number of correlation (triplets) obtained, as a function of γ, was measured considering that 0%, 20%, 60% and 100% of the inter-group neurons were connected. The results 1
Triplet is one of the global patterns selected that constitutes a second-level memory.
A Model for Hierarchical Associative Memories
177
Table 1. Hebbian rule for correlation of the first-level memories, where V(i,ath ) - V(j,bth ) is the correlation between the ith and the j th pattern of the ath and of the bth network, respectively Inter-Groups V(i,A) V(i,B) V(i,C)
V(1,A)
V(j,A)
V(j,B) V(j,C) V(1,A) - V(3,B) V(1,A) - V(5,C) V(2,A) - V(4,B) V(2,A) - V(6,C) V(3,B) - V(1,A) V(3,B) - V(5,C) V(4,B) - V(2,A) V(4,B) - V(6,C) V(5,C) - V(1,A) V(5,C) - V(3,B) V(6,C) - V(2,A) V(6,C) - V(4,B) Global Patterns Selected - V(3,B) - V(5,C) or V(2,A) - V(4,B) - V(6,C)
Coupled networks
Coupled Network
100
100
90
90
80
80
% of correlation
60
% of correlations
100% 60% 20% 0%
70
50 40
70 60 50 40
30
30
20
20 Beta= 0.300
10 0 0
Beta 0.025 Beta 0.050 Beta 0.075 Beta 0.1
1
2
3 4 5 Gamma variation
neurocorr= 60
10 6
7
8
Fig. 2. Triplets obtained to 0%, 20%, 60 % and 100% of inter-group neurons connected
0
0.1
0.2
0.3 0.4 0.5 beta/gamma relation
0.6
0.7
Fig. 3. Triplets obtained to β = 0.025, 0.05, 0.075 e 0.1 as a function of βγ
can be seen in Fig. 2, which shows that even when only 20% of the inter-group neurons were connected, our model presented a recovery rate of global pattern close to 80%. However, when 60% of the inter-group neurons were connected the number of triplets obtained was close to 100%, in practice, the same result of a completely coupled network. We compared our results with the ones achieved in [4] and we observed that in [4], the recovery capacity of global patterns was close 90% for a completely coupled network, while in our model and for a typical value of β, namely β = 0.3, the recovery capacity was close to 100%, even with a network having only 60% of inter-group neurons connected. We have, also, analyzed the influence of the number of correlation (triplets), for a wide range of the parameter β, as a function of βγ relation (Fig. 3). We observed that when β value increases it is necessary an increase of the γ value in such way to improve the recovery capacity. Furthermore, we could, also, infer that a typical value of βγ relation is 0.075, for an specific value of β, namely β = 0.1.
178
4
R.M. Gomes, A.P. Braga, and H.E. Borges
Conclusions
In this paper, we have presented a new proposal of construction of multi-level associative memories using GBSB neural networks that was inspired by TNGS [1]. We derived a new equation for the whole coupled system that extends previous models by means of a term that represents the inter-group connections. We performed numerical computations of a two-level memory system and obtained a recovery rate of global patterns close to 100%, even when the networks are weakly coupled showing that it seems possible to build multi-level memories when new groups of ANNs are interconnected. This present work is currently being generalized in order to include the effects due to different γ values (strength of the inter-groups synapses), such that the model would become more biologically plausible.
References 1. Edelman, G.M.: Neural darwinism: The theory of neuronal group selection. Basic Books, New York (1987) 2. Clancey, W.J.: Situated cognition : on human knowledge and computer representations. Learning in doing. Cambridge University Press, Cambridge, U.K. ; New York, NY, USA (1997) 3. Lillo, W.E., Miller, D.C., Hui, S., Zak, S.H.: Synthesis of brain-state-in-a-box (BSB) based associative memories. IEEE Transactions on Neural Network 5 (1994) 730– 737 4. Sutton, J.P., Beis, J.S., Trainor, L.E.H.: A hierarchical model of neocortical synaptic organization. Mathl. Comput. Modeling 11 (1988) 346–350 5. Anderson, J.A., Silverstein, J.W., Ritz, S.A., Jones, R.S.: 22. In: Distinctive features, categorical perception, probability learning: some applications of a neural model. MIT Press, Cambridge, Massachusetts (1985) 283–325 6. Hui, S., Zak, S.H.: Dynamical analysis of the brain-state-in-a-box (BSB) neural models. IEEE Transactions on Neural Networks 3 (1992) 86–94 7. Golden, R.M.: The brain-state-in-a-box neural model is a gradient descent algorithm. Journal of Mathematical Psychology 30 (1986) 8. Cohen, M.A., Grossberg, S.: Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics 13 (1983) 815–826 9. Gomes, R.M., Braga, A.P., Borges, H.E.: Energy analysis of hierarchically coupled generalized-brain-state-in-box GBSB neural network. In: Proceeding of V Encontro Nacional de Inteligˆencia Artificial - ENIA 2005, S˜ ao Leopoldo, Brazil, (to be published) (2005)
Abstract. Many approaches have emerged in the attempt to explain the memory process. One of which is the Theory of Neuronal Group Selection (TNGS), proposed by Edelman [1]. In the present work, inspired by Edelman ideas, we design and implement a new hierarchically coupled dynamical system consisting of GBSB neural networks. Our results show that, for a wide range of the system parameters, even when the networks are weakly coupled, the system evolve towards an emergent global associative memory resulting from the correlation of the lowest level memories. Keywords: Hierarchical memories, Coupled neural networks, Dynamical systems, Artificial neural networks, TNGS.
1
Introduction
Presently, studies in neuroscience have revealed, by means of experimental evidences, that memory process can be described as being organized, functionally, in hierarchical levels, where higher levels would coordinate sets of functions of the lower levels [1] [2]. One of the theories that is in compliance with these studies is the Theory of Neuronal Group Selection (TNGS) proposed by Edelman [2]. TNGS establishes that correlations of the localized neural cells in the cortical area of the brain, generate clusters units denoted as: neuronal groups (cluster of 50 to 10.000 neural cells), local maps (reentrant clusters of neuronal groups) and global maps (reentrant clusters of neural maps). A neuronal group (NG) is a set of tightly coupled neurons which fire and oscillates in synchrony. Each neuron belongs only to a single neuronal group, which is spatially localized and functionally hyper-specialized. According to TNGS, NG are the most basic structures in the cortical brain, from which memory and perception processes arise, and can been seen as performing the most primitive sensory-effector correlations. W. Duch et al. (Eds.): ICANN 2005, LNCS 3696, pp. 173–178, 2005. c Springer-Verlag Berlin Heidelberg 2005
174
R.M. Gomes, A.P. Braga, and H.E. Borges
A local map is a composition of NG, also spatially localized in the cortical area. Two local maps, functionally different, can develop reentrant connections, resulting in what Edelman [1] calls categorization. Edelman [1] states that a significant number of different neuronal groups could have the same functionality within a given map, that is, could respond to the same stimuli. A global map is a dynamic structure containing multiple reentrant local maps which are capable of interacting with non-mappable areas of the brain, such as the limbic system [2]. A global map is a set of connected local maps and perform “categorizations” (correlations) of local maps. They are not spatially localized but, in fact, they are spread throughout the cortex. Global maps provide a “global or emergent behaviour” of the cortical activities (perception in action) and generate a complete experience in the world, i.e., an experience with qualia. A continuous selection of existing local maps in a global map by selection of additional reentrant connection allows forming new classification couples. Inspired by these ideas, a model of hierarchically coupled dynamical system, using GBSB (Generalized-Brain-State-in-a-Box) neural networks is described in this paper, which integrates the concepts of dynamical systems theory, TNGS and Artificial Neural Networks (ANNs) aiming at building multi-level memories. This paper is organized as follows. In Section 2 we propose a model of coupled GBSB neural networks and show how multi-level memories may arise within it. Section 3 illustrates the use of the algorithm developed in Section 2 with an example from the literature [3] [4]. Finally, Section 4 concludes the paper and presents some relevant extensions of this work.
2
Proposal for the Construction of Multi-level Memories
In order to develop this new model we use an extension of the original BSB Brain-State-in-a-Box [5] called GBSB (Generalized-Brain-State-in-Box ) [6]. The behaviour of the neural network energy in a discrete BSB model was studied by Golden [7]. Cohen and Grossberg [8] discussed a continuous BSB model based on Liapunov equations, while Hui and Zak [6] discussed the stability of the GBSB model in a non-symmetric diagonally dominant weight matrix case. The GBSB model was chosen due to the fact that its characteristics, which are, in short: asymmetric synapses, different bias and maximum and minimum fire rates, redundancy, non-linear dynamics and self-connection for each neuron. In our proposed model we build a two level hierarchical memory where, in accordance with figure 1, each one of the GBSB networks (A, B and C) plays the role of a neuronal group or, in our case, a first-level memory. In a given cluster, each neuron performs synapses with each other neuron of the same cluster, i.e., the GBSB is a fully connected asymmetric neural network. Beyond this, some selected neurons in a cluster are bidirectionally connected with some selected neurons in the others clusters [4]. These inter-cluster connections can be represented by a weight correlation matrix Wcor , which accounts for the contribution of one cluster to another one due to coupling. An analogous procedure could be followed in order to build higher levels in the hierarchy [1].
A Model for Hierarchical Associative Memories
175
Local Maps
W (i,a)(j,a)
WCor(i,a)(j,b) i
j Wcor(j,b)(i,a)
A
B
Neural Groups
GBSB Nets
C
Fig. 1. Network design
Our proposed coupled GBSB model extends the GBSB model for single networks discussed in [6], by means of adding a fourth term that represents the inter-group connections. Consequently, our new model can be defined by the equation: N a k+1 β(i,a) w(i,a)(j,a) xk(j,a) + β(i,a) f(i,a) + x(i,a) = ϕ xk(i,a) + j=1 ⎞ (1) Nq Nr k + γ(i,a)(j,b) wcor(i,a)(j,b) x(j,b) ⎠ , b=1 b=a
j=1
where the three first terms represent the equations of a GBSB model for Na uncoupled GBSB networks, meaning in our model intra-group synapses (i.e., in the ath network or neuronal group). The sum over j, in the fourth term, labels the Nq neurons in the bth neuronal group that have correlation to neuron i in the ath neuronal group. The strength or density of the inter-group synapses are parameterized by γ(i,a)(j,b) . The activation function ϕ is a linear saturating function whose ith component is defined as follows: ⎧ k ⎪ ⎨ +1 if y(i,a) > +1 k k k k if −1 ≤ y(i,a) ≤ +1, xk+1 ϕ(y(i,a) ) = y(i,a) (2) (i,a) = ϕ(y(i,a) ), ⎪ ⎩ −1 if y k < −1 (i,a) k is the argument of the function ϕ of the equation 1. where y(i,a) In order to complete our model we present now a Lyapunov function (energy) of the coupled system, which can be defined as [9]:
E=
− 21
Nr N a
β(i,a) w(i,a)(j,a) x(i,a) x(j,a) −
a=1 i,j=1
Nr Na
β(i,a) f(i,a) x(i,a)
a=1 i=1
(3) −
Nq Nr N a a,b=1 a=b
i=1 j=1
γ(i,a)(j,b) wcor(i,a)(j,b) x(i,a) x(j,b) ,
176
R.M. Gomes, A.P. Braga, and H.E. Borges
where the first term represents the energy of the individual neuronal groups. The second term gives the contribution to energy due to external factors (i.e., the bias field ). Finally, the third term in equation 3 is due to the inter-group connections. A detailed mathematical analysis of equation 3, describing the energy of the coupled system can be found in [9], where it was shown that it presents two important features: the whole system evolves to a state of minimum energy, even when the neuronal groups are weakly coupled; the inter-group coupling, which establishes the second-level correlations, does not destroy the first-level memories structures.
3
Coupled GBSB Experiments
Computational experiment consisting of three GBSB networks connected (Fig. 1) were conducted and the results were compared with the ones presented in [4]. Although the experiment presented here is a quite simple one, it is intended to make it clear the procedure for the construction of multi-level associative memories. More complex computational experiments will be presented elsewhere. In our simulations each network or neuronal group contains 10 neurons and we selected 6 out of 1024 possible patterns to be stored as our first-level memories. The weight matrix of the individual networks followed the definition proposed in [3]. The selected set of patterns stored as first-level memories was: V1 = [ -1 1 1 1 1 1 -1 -1 -1 -1 ] V2 = [ 1 1 -1 -1 -1 1 -1 -1 1 -1 ] V3 = [ -1 1 1 1 -1 -1 1 -1 -1 -1 ] V4 = [ -1 1 -1 -1 -1 -1 1 -1 1 1 ] V5 = [ 1 -1 -1 1 1 -1 1 1 1 -1 ] V6 = [ 1 1 -1 1 -1 1 1 1 -1 -1 ]
(4)
Each network A, B and C was carefully designed to present the same asymptotically stable fixed point structure presented in [3]. To design these networks we followed the approach of [4]. In addition, amongst the 63 = 216 possible combinations of the 3 sets of first-level memories, we have chosen 2 triplets or global patterns to be our second-level memories (local maps). The arrangement of the global patterns determines the inter-group correlation matrix Wcor by a generalized Hebb rule. The system was initialized in one of the networks A, B or C, randomly, and in one of their first-level memories that establish a correlation in accordance with table 1. The two other networks, in turn, were initialized in one of the 1024 possible patterns, also, randomly. Then, we measured the number of times that the system converges to a configuration of triplets1 , considering networks totally or partially coupled. Neurons that took part of the inter-group connections was chosen randomly. Points in our experiments were averaged over 1000 trials for each value of γ. In our experiment a typical value of β was chosen (β = 0.3) and the number of correlation (triplets) obtained, as a function of γ, was measured considering that 0%, 20%, 60% and 100% of the inter-group neurons were connected. The results 1
Triplet is one of the global patterns selected that constitutes a second-level memory.
A Model for Hierarchical Associative Memories
177
Table 1. Hebbian rule for correlation of the first-level memories, where V(i,ath ) - V(j,bth ) is the correlation between the ith and the j th pattern of the ath and of the bth network, respectively Inter-Groups V(i,A) V(i,B) V(i,C)
V(1,A)
V(j,A)
V(j,B) V(j,C) V(1,A) - V(3,B) V(1,A) - V(5,C) V(2,A) - V(4,B) V(2,A) - V(6,C) V(3,B) - V(1,A) V(3,B) - V(5,C) V(4,B) - V(2,A) V(4,B) - V(6,C) V(5,C) - V(1,A) V(5,C) - V(3,B) V(6,C) - V(2,A) V(6,C) - V(4,B) Global Patterns Selected - V(3,B) - V(5,C) or V(2,A) - V(4,B) - V(6,C)
Coupled networks
Coupled Network
100
100
90
90
80
80
% of correlation
60
% of correlations
100% 60% 20% 0%
70
50 40
70 60 50 40
30
30
20
20 Beta= 0.300
10 0 0
Beta 0.025 Beta 0.050 Beta 0.075 Beta 0.1
1
2
3 4 5 Gamma variation
neurocorr= 60
10 6
7
8
Fig. 2. Triplets obtained to 0%, 20%, 60 % and 100% of inter-group neurons connected
0
0.1
0.2
0.3 0.4 0.5 beta/gamma relation
0.6
0.7
Fig. 3. Triplets obtained to β = 0.025, 0.05, 0.075 e 0.1 as a function of βγ
can be seen in Fig. 2, which shows that even when only 20% of the inter-group neurons were connected, our model presented a recovery rate of global pattern close to 80%. However, when 60% of the inter-group neurons were connected the number of triplets obtained was close to 100%, in practice, the same result of a completely coupled network. We compared our results with the ones achieved in [4] and we observed that in [4], the recovery capacity of global patterns was close 90% for a completely coupled network, while in our model and for a typical value of β, namely β = 0.3, the recovery capacity was close to 100%, even with a network having only 60% of inter-group neurons connected. We have, also, analyzed the influence of the number of correlation (triplets), for a wide range of the parameter β, as a function of βγ relation (Fig. 3). We observed that when β value increases it is necessary an increase of the γ value in such way to improve the recovery capacity. Furthermore, we could, also, infer that a typical value of βγ relation is 0.075, for an specific value of β, namely β = 0.1.
178
4
R.M. Gomes, A.P. Braga, and H.E. Borges
Conclusions
In this paper, we have presented a new proposal of construction of multi-level associative memories using GBSB neural networks that was inspired by TNGS [1]. We derived a new equation for the whole coupled system that extends previous models by means of a term that represents the inter-group connections. We performed numerical computations of a two-level memory system and obtained a recovery rate of global patterns close to 100%, even when the networks are weakly coupled showing that it seems possible to build multi-level memories when new groups of ANNs are interconnected. This present work is currently being generalized in order to include the effects due to different γ values (strength of the inter-groups synapses), such that the model would become more biologically plausible.
References 1. Edelman, G.M.: Neural darwinism: The theory of neuronal group selection. Basic Books, New York (1987) 2. Clancey, W.J.: Situated cognition : on human knowledge and computer representations. Learning in doing. Cambridge University Press, Cambridge, U.K. ; New York, NY, USA (1997) 3. Lillo, W.E., Miller, D.C., Hui, S., Zak, S.H.: Synthesis of brain-state-in-a-box (BSB) based associative memories. IEEE Transactions on Neural Network 5 (1994) 730– 737 4. Sutton, J.P., Beis, J.S., Trainor, L.E.H.: A hierarchical model of neocortical synaptic organization. Mathl. Comput. Modeling 11 (1988) 346–350 5. Anderson, J.A., Silverstein, J.W., Ritz, S.A., Jones, R.S.: 22. In: Distinctive features, categorical perception, probability learning: some applications of a neural model. MIT Press, Cambridge, Massachusetts (1985) 283–325 6. Hui, S., Zak, S.H.: Dynamical analysis of the brain-state-in-a-box (BSB) neural models. IEEE Transactions on Neural Networks 3 (1992) 86–94 7. Golden, R.M.: The brain-state-in-a-box neural model is a gradient descent algorithm. Journal of Mathematical Psychology 30 (1986) 8. Cohen, M.A., Grossberg, S.: Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics 13 (1983) 815–826 9. Gomes, R.M., Braga, A.P., Borges, H.E.: Energy analysis of hierarchically coupled generalized-brain-state-in-box GBSB neural network. In: Proceeding of V Encontro Nacional de Inteligˆencia Artificial - ENIA 2005, S˜ ao Leopoldo, Brazil, (to be published) (2005)
