Synchronization of Coupled Connected Neural Networks With Delays
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
2491
Synchronization of Coupled Connected Neural Networks With Delays Wenlian Lu and Tianping Chen
Abstract—In this paper, we investigate synchronization of an array of linearly coupled identical connected neural networks with delays; Variational method is used to investigate local synchronization. Global exponential stability is studied, too. We do not assume that the coupling matrix is symmetric or irreducible. The linear matrix inequality approach is used to judge synchronization with global convergence property.
Therefore, the study of synchronization of coupled neural networks is an important step for both understanding brain science and designing coupled neural networks for practical use. Recurrently connected neural network with delays can be written as
Index Terms—Chaos, delays, dynamical systems, neural network, synchronization.
(1)
I. INTRODUCTION
A
N INCREASING interest has been devoted to the study of dynamical behaviors of recurrently connected neural networks (RCNNs), also called Grossberg–Hopfield neural networks (see [1]–[3]), and many applications have been found in different areas. Most of previous studies are concentrated on such issues as stability analysis [1], [4]–[6] and periodic oscillation [7]. However, it has been shown that the recurrently connected neural networks can exhibit some complicated dynamics, such as chaotic behavior and others, see [8] and other papers. Recently, arrays of coupled systems have attracted much attention of researchers in different research fields. They can exhibit many interesting phenomena, such as spatio-temporal chaos [9], auto waves [10]–[12], spiral waves [13], etc., array of coupled systems is also important in modeling populations of interacting biological systems [14] and many others. For example, fireflies have been known to fire in unison, and this phenomenon has been proved to occur in a group of integrate-and-fire cells [15]. Moreover, experiment and theoretical analysis have revealed that a mammalian brain not only displays in its storage of associative memories, but also modulate oscillatory neuronal synchronization by selective perceive attention [16], [17]. Synchronization of coupled neural networks also have many applications. In [12], [18], [19], the authors proposed so called “the autowave principles for parallel image processing.” In [20], the authors presented an architecture of coupled neural networks to store and retrieve complex oscillatory patterns as synchronization states. In [21], the authors introduced a secure communication system based on coupled cellular neural networks.
Manuscript received March 9, 2004; revised June 9, 2004. This work was supported by the National Science Foundation of China under Grant 69982003 and Grant 60074005. The authors are with Laboratory of Nonlinear Science, Institute of Mathematics, Fudan University, Shanghai 200433, China (e-mail: [email protected]). Digital Object Identifier 10.1109/TCSI.2004.838308
where is the state of the th neuron, denotes the output denotes the output of the th of the th neuron, , is the weight connecting the output of neuron at time the th neuron to the th neuron, is the weight connecting the to the th neuron at time output of the th neuron at time , and denotes the input to the th neuron. We also assume that all differential equations have a unique solution with initial value (2) where . , then the model can be rewritten as following If matrix form: (3) where
, ,
,
,
, ,
, , and . An array of linearly coupled identical delayed neural networks with identical networks can be described componentwise as
(4) where , state of the neurons in the th neural network.
1057-7122/04$20.00 © 2004 IEEE
, denotes the
2492
Let
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
. Thus, the system can be rewritten
as
(5) If , for all written in matrix form as
, then, the system can be
(6) where denotes the coupling configuration of the coupled networks with
(7) and . It should be noted that several authors have investigated the synchronization of linearly coupled networks described as (8) when the coupling matrix is irreducible, symmetric and satisfies (7), local stability analysis via the linearlization technique has been given (see [22]–[24]); In [25], the authors defined a distance between any point and the synchronization manifold. By this distance, the authors proposed a methodology to discuss global convergence for complete regular coupling configuration. Randomly coupled networks are studied in [24], [26], such as small-world lattices and scale-free networks. Most of these papers focus on arrays of systems described by ordinary differential equations without delays. In practice, dynamical systems with delays are more important in theoretical study and applications. Hence, synchronization of coupled systems with delays is interesting and necessary to be investigated. In this paper, motivated by [23], [25], we propose some methodologies to study the local and global stability of the synchronization manifold for arrays of coupled connected neural networks with delays. By these methodologies, we give series of sufficient conditions for arrays of the delayed systems to be synchronized locally and globally. These sufficient conditions are easy to verify by some linear matrix inequality (LMI) tools. Moreover, these methodologies can be extended to study synchronization of arrays of some more general class of delayed systems. As for the coupling matrix, we do not assume that the coupling matrix is symmetric or irreducible. This paper is organized as follows. In Section II, some necessary definitions, lemmas, and hypotheses are given; In Section III, we deal with local stability of synchronization manifold; LMI-based criteria are obtained for global synchronization of linearly coupled identical delayed RCNNs in Section IV.
Some numerical examples to verify theoretical results are given in Section V. In Section VI, we give two examples of applications for image processing and secure communications. We conclude the paper in Section VII. II. PRELIMINARIES In this section, we present some definitions, which are needed throughout the paper. Definition 1: Let . The set is called synchronization manifold. Definiton 2: Synchronization manifold is said to be locally exponential stable, equivalently, the coupled system (5) or (6) is locally exponentially synchronized, if there exist constants , , , such that if , then for sufficient large
, we have if
and
(9) Definiton 3: Synchronization manifold is said to be globally exponential stable, equivalently, the coupled system (5) or , (6) is globally exponentially synchronized, if there exists , , for all , there hold (10) for all , . To measure the distance between any point and synchronization manifold , we use the following definitions introduced in [25]. : is composed of matrices Definiton 4: Set of with columns. Each row (for instance, the th row) of has exactly one entry and one entry , where . All other entries are zeroes. Definiton 5: Set of : are matrices obtained by replacing entry in with , where is the -dimension identity matrix, i.e., is the -dimensional identity matrix , where is Kronecker product. : If , then, for Definiton 6: any pair of indexes and , there exist indexes , where and , and such that and for all . is said to Definiton 7: A matrix of order satisfy condition , if 1)
(11) 2) Real part of eigenvalues of are all negative except an eigenvalue 0 with multiplicity one. denotes the class of functions with LipDefiniton 8: . shitz constant Finally, some lemmas are needed.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
Lemma 1 [27]: If satisfies condition (11) and is irreducible, then has an eigenvalue 0 with multiplicity 1 and corre1) . sponding eigenvector is an eigenvalue of , then its real part 2) If . In [25], the following Lemma 2 and Lemma 3 have been given. and is symLemma 2: If the matrix satisfies condition metric. Then is irreducible if and only if there exists a matrix , such that . Lemma 3: Let where , . Then if and only if (12) holds for some . Remark 1: According to the definition above, we can use to measure the difference between and synchronization manifold .
2493
focusing on the local synchronization for arrays of dynamic systems without delays. Instead, here, we investigate local stability of the manifold in the infinite dimensional Banach Space . IV. GLOBAL EXPONENTIAL ASYMPTOTIC SYNCHRONIZATION In the previous section, we discussed local stability of the synchronization manifold, proved that the trajectory comes back to the manifold after an enough small perturbation. It is an important issue to investigate the global synchronization. In this section, we discuss this issue. Under some mild conditions, we prove the trajectory will converge to the synchronization manifold exponentially from any initial value. , , . Suppose Theorem 2: Let , , and the coupling matrix that satisfies . If there exist , , , , , for all , and an irreducible matrix satisfying such that
III. LOCAL EXPONENTIAL ASYMPTOTIC SYNCHRONIZATION In this section, we investigate local stability of synchronization manifold for the coupled system (5) with the coupling matrix satisfying condition . The coupling matrix has eigenvalues. They are , , where is the , , is multiple, and imaginary unit, . Theorem 1: Suppose that , , , , , and the coupling matrix satisfies condition . If and , such that there exist
(13) hold for , . Then, the synchronization manifold of the coupled system (5) is locally exponentially . stable with convergence rate Proof of Theorem 1 will be given in Appendix I. Following statements in Remark 2 and Remark 3 are direct consequences of Theorem 1 and the properties of -matrices (see [27]). Remark 2: If the condition (13) is replaced by the following inequality:
(14) , are eigenvalues of the matrix . where Then, the synchronization manifold is stable exponentially. Remark 3: It can be seen that conditions (14) are equivalent are matrices, to that , are eigenvalues of the matrix , where is denoted as . where Remark 4: In [22]–[24], local convergence analysis via linearization technique was given. However, they were all
(15) and (16) hold for , where is an identity matrix of order , is the symmetric part of . Then the synchronization manifold of the couple system (5) is globally exponentially stable with convergence rate . Proof of Theorem 2 will be given in Appendix II. , for all , In case all delays are the same we can give the following result, which can be solved by LMI technique. , , . Theorem 3: Let , , and the Suppose that satisfy . If there exist a posicoupling matrix , positive definite diagonal matrices tive constant , , , , , a diagonal matrix , a semi-positive definitive matrix , and an irreducible matrix satisfying , such that
(17) and (18) hold for . Then the synchronization manifold of the system (6) is globally exponentially asymptotically stable with convergence rate .
2494
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Proof of Theorem 3 will be given in Appendix III. If the coupling matrix is symmetric, irreducible, we have the following corollary, which is a direct consequence of Theorem 3. Corollary 1: Suppose the coupling matrix is symmetric, , of which eigenvalues are irreducible and satisfies condition . If we select constants , by following rule: if if if
(19)
Then, . Therefore, (18) in Theorem 3 can be replaced by (19), and (16) in Theorem 2 can be also replaced by (19). Proof: Because is symmetric. It is easy to see that for fixed , , the eigenvalues of the symmetric matrix satisfy
Therefore, under the conditions of the corollary, we have . Corollary 1 is proved. Remark 5: Corollary 1 indicates that it is the maximum of that counts the global synchrononzero eigenvalue are positive. Therefore, the largest nonzero nization if all plays important role when investigate the eigenvalue of synchronization, see [24], [25]. To utilize the LMI technique, we have to fix , for example, , and impose a little more strict constraints on , and define the following matrix: (20) is the -dimensional identity matrix. where Now, we give the following corollary, which is a variant of Theorem 3. , , Corollary 2: Suppose , and the coupling matrix satisfies . If there , positive diagonal matrices exist a positive constant , , , , , a diagonal matrix , a nonnegative definite matrix , and a matrix satisfying
(21) such that
and for all (23) hold for . Then, the synchronization manifold (6) is globally exponentially asymptotically stable with convergence rate . Proof: In fact, it is easy to see that under the conditions of Corollary 2, satisfies conditions of Theorem 3. Because . From the (21) implies that is irreducible. And let , where definition of , one can see that . The orthogonal . One subspace of is described by vector can easily see that . , one has Therefore, for any (24) Corollary 2 is proved. Thus, the general approach to determine whether an array of identical connected neural networks with delays is globally synchronized by Theorem 2, Corollary 1, or Corollary 2 consists of the following two steps. By Theorem 2: , , , and Step 1) Find parameters such that (15) hold for . Step 2) Find matrix such that (16) hold for . By Corollary 1: Step 1) Find parameters such that (15) or (22) is satisfied. Step 2) Verify whether (19) is satisfied. By Corollary 2: , matrices , , , , , such that Step 1) Find (22) is satisfied. such that (23) hold for Step 2) Find matrix . If we can find these parameters in Step 1) and 2), then, the system is globally synchronized by the coupling matrix . It can be seen that if we fix some parameters, then Step 1) and 2) are all LMI problems, which can be solved by some LMI tool, for example, Matlab LMI Control Toolbox. Details of how to manipulate Step 1 and 2 by using Theorem 2, Corollary 1, and Corollary 2 by LMI method are presented in the next section. Remark 6: In [24], [26], all the authors only dealt with the case that the coupling matrix is irreducible and symmetric. But in practice, irreducible or symmetric coupling matrix is not enough. For example, directed graphs, of which coupling matrix may be reducible and asymmetric. Instead, in this paper, we present some criteria which can be used to treat these cases. Remark 7: The methodology we used in discussing global synchronization of an array of linearly coupled dynamic systems can be extended to discuss more general delayed dynamic systems. For example
(25) (22)
The details are a subject of current investigation.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
Fig. 1.
Fig. 2.
Chaotic trajectory of the model (26).
V. NUMERICAL EXAMPLES In this section, we give several simulations to verify theoretical results obtained in previous sections. We show how to use these theories to determine whether the coupling configurations can synchronize several uncoupled neural networks and in what range some parameters can vary to ensure the synchronization. We consider the following two-dimensional (2-D) neural network with delays:
2495
Distance of states x (t), i = 1, 2, 3 of the coupled system (29).
We will explain how the coupled system (29) synchronizes three identical systems defined by (26) with different initial values by using Theorem 2. , , , First, we choose the parameters . Then, by calculation, we find , such that
(26) ,
where , and
hold. It means that (15) is satisfied. Secondly, with Control Toolbox, we obtain
, using the Matlab LMI
(27) The dynamical behavior with initial condition (28) is developed doubled like chaotic attractors in Fig. 1. In the following Examples 1, 2, and 3, we demonstrate how to find the coupling matrix to synchronize three networks with different initial values (0.4,0.6), (5.0, 10.0) and ( 4.0,7.0) by using Theorem 2, Corollary 1, and Corollary 2, respectively. Example 1: In this example, we show how to design a coupled system by Theorem 2. We consider the linearly coupled system with three subsystems defined before
The eigenvalues of and are 0,4.1897, 10.9211; and 0,3.0498, 10.0098, respectively. Thus, are nonnegative definite for , 2, and all the conditions in Theorem 2 are satisfied and the three systems are synchronized. The performance of synchronization of the coupled system is shown in Fig. 2 Fig. 2 indicates the distance among three trajectories by calculating (30)
(29)
Example 2: In this example, we also consider the linearly coupled system with three subsystems
, and
where .
(31)
2496
Fig. 3.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Distance of states x (t), i = 1, 2, 3 of the coupled system (31).
Fig. 4. Dependence of error err(100) on coupling strength c for the coupled system (31).
where the coupling matrix is the following symmetric matrix: The result is
with eigenvalues 0, 6, and 14. Note that , and . Then inequalities (19) are . satisfied, if We will apply Corollary 1 and Theorem 3 to search the constant such that the coupled system is synchronized. . We search by solving the following LMIFirst, let Optimization problem. Minimize under constraints There exists a diagonal matrix There exists a diagonal matrix There exists a symtric matrix There exists a diagonal matrix
As the result, we obtain . Thus, for any , , the synchronizathe system can be synchronized. For tion performance is illustrated by Fig. 3 . with , search Now, for each the constant depending on by solving the following optimization problem. Minimize under constraints There exists a diagonal matrix There exists a diagonal matrix There exists a symtric matrix There exists a diagonal matrix
and
.
It is natural to raise the question. Can we find the sharp bound ? By simulations, such bound exists indeed. we simulate the system (31) for 50 different initial values selected randomly. and plot them in Fig. 4. In each simulation, record , , which indicates that It seems that if the lowest bound is 0.06 approximately. Unfortunately, it is too difficult to prove it theoretically. Example 3: In this simulation, we consider coupled system
(32) with coupling matrix
which is asymmetric and reducible. We show how to synchronize the system by Corollary 2. Step 1) Solve the following LMI-optimization problem. Minimize subject to there exist diagonal matrices and symtric matrix such that
The result is . thus, the matrix in. equality (22) is satisfied with subject to following constraints: Step 2) Search
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
2497
can detect a horseshoe-shaped closed curve by synchronization and de-synchronization of the system. A 2-D array of identical delayed neural networks with 21 21 lattices, is given. At position , , a 2-D defined by (26) is set down. And the array neural network of linear coupled system is written as follows:
(33) where
,
, and are defined in (29). The coefficients , and are defined as follows: If nodes , , ,or are in the lattice but out , , , or of the curve, the corresponding equals 1; otherwise, it equals zero. It is seen that every node is connected with the four closest adjacent nodes. The detection process consists of the following steps. 1) The input image for processing is stored in the array by the way that nodes belonging to the closed curve are isolated from other nodes. 2) We use 1 and 0 to represent synchronization and de-synchronization state, respectively. 3) At first, all nodes in the lattice are set to (0,0) and synchronization states are all set to 0. An autowave is initiated at the node (1,1) by setting (0.4,0.6) as its initial condition, which develops a doubled-like chaotic attractor at the node (1,1) (see Fig. 1). The autowave propagates through other nodes by the way that the state of other nodes is synchronized with node (1,1), except the nodes in the curve are asynchronous with the node (1,1) all the time. In this way, we can detect the closed curve. 4) Numerically, we use explicit Euler method with time step as a scale 0.005. We define to differentiate states of synchronization and de-synchro, then we say nization: Set some threshold , if is synchronous with node (1,1) and denoted by node is asynchronous with node (1,1) 1. Otherwise, node and denoted by 0. Such selection can be realized by zero detector in integrated circuits (VLSI) design. In our numerical example, we set Fig. 6 shows the process of detection, where Fig. 6(a) is the image to be detected. Fig. 6 (b)–(d) shows the snapshot at time , 35, and 50, respectively. Remark 8: In [12], the authors used a bistable system to realize a travelling wave for detection of closed curves by the way and that the nodes in the curve are stable at one equilibrium other nodes out the curve are stable at another equilibrium . However, if a dramatic perturbation of state happens, the state of the nodes belonging to the curve may jump into the attract basin , then the state of these nodes in the curve and out of the of curve may be identical. Thus, the detected curve may not be the wanted. Instead, the synchronization technique we utilize here to detect a closed curve is never influenced by the initial conditions; Even the doubled-like chaotic behavior at (1,1) does not ,
Fig. 5. Distance of states x (t), i = 1, 2, 3 of the coupled system (32).
where
By using Matlab LMI Control Toolbox, we get
The eigenvalues of are 0.0018, 0.0090. By Corollary 2, the system is synchronized. The performance of synchronization is illustrated in Fig. 5.
VI. APPLICATIONS In this section, we give two application examples of synchronization of coupled neural networks with delays. A. Image Processing: Detection of a Closed Curve In [10], [11], Krinsky et al. proposed the so-called “the autowave principles for parallel image processing.” Autowaves represent a particular class of nonlinear waves, which propagate in an active excitable media at the expense of the energy stored in the medium and differ from classical waves by several properties: Annihilation rather than interference when colliding between two waves and no reflection from obstacles. However, the only common feature is diffraction. In [12], the authors described an array of resistively coupled Chua’s circuits which can be designed to implement some elementary aspects of spatial recognition, namely, recognizing open curves from closed ones and locating the shortest path between two locations by travelling waves which is characterized by only two stable equilibrium states. Motivated by these observation, here we present a simple example exploring the possibility of using synchronization and de-synchronization of a 2-D array of linearly coupled connected neural networks with time delays proposed in previous sections for image processing. The model we present here
,
,
2498
Fig. 6.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Snapshots of image processing. (a) Input image. (b) Snapshot at t = 25. (c) Snapshot at t = 35. (d) Snapshot at t = 50.
affect synchronization process. Therefore, the method we use succeeds to detect image under any perturbations. B. Secure Communication Recently, application of chaotic synchronization of coupled systems to secure communication has become as area of active research, see [21], [28] and other papers. A signal containing useful messages is transmitted using a chaotic signal as a broadband carrier, and the synchronization is used to recover the messages at the receiver. Here, we present a secure communication scheme based on coupled delayed neural networks. The system consists of the following two subsystems: Driver subsystem
where
, , , , where
is identical to the message signal signal . A quantity is defined to meaand the recovered sure the error between message signal signal . Fig. 7 indicates the dynamical behaviors of mes, transmitting signal , recovered signal sage signal and the error through time. As Fig. 7 shows, converges to zero, which implies that the message is recovered successfully. VII. CONCLUSION In this paper, we study the synchronization of an array of linearly coupled identical connected neural networks with time delay; criteria for synchronization are given without assuming symmetry and irreducibility of coupling matrix . Simulations and applications are given, too.
, and are defined in (29), and is input messages to be transmitted
APPENDIX I Proof of Theorem 1
Receiver subsystem
, is the coupling strength, and
Direct calculation gives following variational equations of the coupled system (5) at the synchronization manifold : ,
is the transmitting signal comwhere and chaotic carrier signal . posed of message signal and the input message In this simulation, we take signal is a sinusoid signal. The coupled systems can be syn. Hence, recovered chronized, i.e.,
(34) where
,
.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
Fig. 7.
Dynamical behaviors of
m(t), s(t), r(t) and error(t) through time.
Let be the Jordan decomposition, where a block-diagonal matrix
..
and
2499
is
where , variation equation at the manifold
. In terms of is
, the
.
(35)
is the block
..
. (36)
corresponding to the eigenvalue with multiple . is an 1 1 matrix (number) corresponding to , for satisfies condition . Let , , where and , where , , , . Then, we have
where
, . We first discuss the case . Let , and , , , The (35) can be rewritten as
and
,
and prove that , where , , and is the imaginary unit.
(37)
2500
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Define
(38)
It is easy to see that if for all such that, there exists
,
, then
Define the Lyapunov functional holds for all
(39) Then, by some algebra, we have
. Otherwise, there exists , such that, , and is strictly monotone at , is strictly monotone at , and which implies that , furthermore, . Therefore, is bounded and for all and . for all By previous reasoning, we have , which implies (43) By the Jordan decomposition , we have . By Lemma 1, the first row of is . Therefore (44)
which implies (40) Secondly, we will prove that by induction. Supposing that
which implies that converges to orem 1 is proved completely.
for all
exponentially. The-
APPENDIX II
, then Proof of Theorem 2
(41) Denote
, and define
By the assumption imposed on and Lemma 2, there exists a matrix such that . Moreover, let be the matrix defined in Definition denotes th 4 and Definition 5. , , and , row of , where , . Moreover, define if
.. . (42)
where with
for all
is assumed to be . we have
Then
(45) where
Define (46)
(47)
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
and
We will prove that lations give
2501
Therefore
(48) . In fact, direct calcu-
(49)
and is bounded and we conclude that the synchronization manifold is globally exponentially asymptotically stable with rate , namely
It can be seen that (50) Theorem 2 is proved. APPENDIX III Proof of Theorem 3
and
where Hence,
, for all
.
By the assumptions imposed on and Lemma 5, there exists a matrix such that . Moreover, be the matrix defined in Definition let denotes th 4 and Definition 5. , , and , row of , where if , . denote the Kroneker product of matrices and , Let and
Equation (6) can be rewritten as Moreover, it is easy to see that (51) ,
Let ,
, and
2502
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
, , for
. Then, . Define Lyapunov
For simplicity of writing, denote the matrix of (17) by we have
, then
function
(52) Its derivative along the trajectory is
By conditions (18), we have
By the structure of
, following equalities are easy to verify:
Therefore, we have
Moreover, condition (17) implies that fore
is semi-definite. There-
Thus (53) Theorem 3 is proved. ACKNOWLEDGMENT The authors thank the reviewers for their comments, which were very helpful in the revision of the paper. Direct calculation gives
and
REFERENCES [1] M. A. Cohen and S. Grossberg, “Absolute stability of global pattern formation and parallel memory storage by competitive neural networks,” IEEE Trans. Syst., Man, Cybern., vol. SMC-13, pp. 815–826, Apr. 1983. [2] D. Tank and J. J. Hopfield, “Simple neural optimization networks: An A/D converter, signal decision circuit and a linear programming circuit,” IEEE Trans. Circuits Syst., vol. CAS-33, pp. 533–541, Apr. 1986. [3] J. J. Hopfield, “Neurons with graded response have collective computation properties like those of two-stage neurons,” Proc. Nat. Acad. Sci. USA (Biophysical), vol. 81, pp. 3088–3092, 1984. [4] T. Chen, “Convergence of delayed dynamical systems,” Neural Processing Lett., vol. 10, no. 3, pp. 267–271, 1999. [5] T. Chen and S. Amari, “Stability of asymmetric Hopfield neural networks,” IEEE Trans. Neural Netw., vol. 12, pp. 159–163, Jan. 2001. [6] T. Chen, “Global exponential stability of delayed Hopfield neural networks,” Neural Netw., vol. 14, no. 8, pp. 977–980, 2001. [7] Y. Zheng and T. Chen, “Global exponential stability of delayed periodic dynamical systems,” Phys. Lett. A, vol. 322, no. 5–6, pp. 344–355, 2004. [8] H. Sompolinsky and A. Crisanti, “Chaos in random neural networks,” Phys. Rev. Lett., vol. 61, no. 3, pp. 259–262, 1988. [9] A. Zheleznyak and L. O. Chua, “Coexistence of low- and high-dimensional spatio-temporal chaos in a chain of dissipatively coupled Chua’s circuits,” Int. J. Bifur. Chaos, vol. 4, no. 3, pp. 639–672, 1994. [10] V. I. Krinsky, V. N. Biktashev, and I. R. Efimov, “Autowaves principles for parallel image processing,” Phys. D., vol. 49, pp. 247–253, 1991.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
[11] L. Kunbert, K. I. Agladze, and V. I. Krinsky, “Image processing using light-sensitive chemical waves,” Nature, vol. 337, pp. 244–247, 1989. [12] V. Perez-Munuzuri, V. Perez-Villar, and L. O. Chua, “Autowaves for image processing on a two-dimensional CNN array of excitable nonlinear circuits: Flat and wrinkled labyrinths,” IEEE Trans. Circuits Syst. I, vol. 40, pp. 174–181, Feb. 1993. [13] A. Perez-Munuzuri, V. Perez-Munuzuri, and V. Perez-Villar, “Spiral waves or a two-dimensional array of nonlinear circuits,” IEEE Trans. Circuits Syst. I, vol. 40, pp. 872–877, Aug. 1993. [14] J. D. Murray, Mathematics Biology. Berlin, Germany: SpringerVerlag, 1989. [15] R. E. Mirollo and S. H. Strogatz, “Synchronization of pulse-coupled biological oscillators,” SIAM J. Applicat. Math., vol. 50, no. 6, pp. 1645–1662, 1990. [16] P. Fries, J. H. Reynolds, J. H. Rorie, and R. Desimore, “Modulation of oscillatory neuronal synchronization by selective visual attention,” Science, vol. 291, no. 23, pp. 1560–1563, 2001. [17] P. N. Steinmetz, A. Roy, P. J. Fitzgerald, S. S. Hsiao, K. O. Johnson, and E. Niebar, “Attention modulate synchronized neuronal firing in primate somatosensory cortex,” Nature, vol. 404, no. 9, pp. 487–490, 2000. [18] L. O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE Trans. Circuits Syst., vol. 35, pp. 1257–1272, Oct. 1988. , “Cellular neural networks: Applications,” IEEE Trans. Circuits [19] Syst., vol. 35, pp. 1273–1290, Oct. 1988. [20] F. C. Hoppensteadt and E. M. Izhikevich, “Pattern recognition via synchronization in phase-locked loop neural networks,” IEEE Trans. Neural Netw., vol. 11, pp. 734–738, Mar. 2000. [21] Y. Zhang and Z. He, “A secure communication scheme based on cellular neural networks,” in Proc. IEEE Int. Conf. Intelligent Processing Systems, vol. 1, 1997, pp. 521–524. [22] L. M. Pecora, “Synchronization conditions and desynchronization patterns in coupled lmit-cycle and chaotic systems,” Phys. Rev. E, no. 58, pp. 347–360, 1998. [23] L. Pecora, T. Carroll, G. Johnson, D. Mar, and K. S. Fink, “Synchronization stability in coupled oscillator arrays: Solution for arbitrary configuration,” Int. J. Bifurc. Chaos, vol. 10, no. 2, pp. 273–290, 2000.
2503
[24] X. F. Wang and G. Chen, “Synchronization in scale-free dynamical networks: Robustness and fragility,” IEEE Trans. Circuits Syst. I, vol. 49, pp. 54–62, Jan. 2002. [25] C. W. Wu and L. O. Chua, “Synchronization in an array of linearly coupled dynamical systems,” IEEE Trans. Circuits Syst. I, vol. 42, pp. 430–447, Aug. 1995. [26] X. F. Wang and G. Chen, “Synchronization in small-world dynamical networks,” Int. J. Bifurc. Chaos, vol. 12, no. 1, pp. 187–192, 2002. [27] P. A. Horn and C. R. Johnson, Matrix Analysis. New York: Cambridge Univ. Press, 1985. [28] J. R. Terry and G. D. VanWiggeren, “Chaotic communication using generalized synchronization,” Chaos, Solit. Fract., vol. 12, pp. 145–152, 2001.
Wenlian Lu is working toward the Ph.D. degree in mathematics at the Fudan University, Shanghai, China. He has published four papers in respectable international journals and three papers in international conferences. His research interests are neural networks, dynamical systems and complex networks.
Tianping Chen is a Professor of Department of Mathematics, Fudan University, Shanghai, China. His research interests include hamornic analysis, approximation theory, neural networks, signal processing and dynamical systems. Dr. Chen is a receipent of several important awards, including the second prize of the 2002 National Natural Sciences Award of China, the 1997 Outstanding Paper Award of IEEE TRANSACTIONS ON NEURAL NETWORKS, and the 1997 Best paper Award of Japanese Neural Network Society.
2491
Synchronization of Coupled Connected Neural Networks With Delays Wenlian Lu and Tianping Chen
Abstract—In this paper, we investigate synchronization of an array of linearly coupled identical connected neural networks with delays; Variational method is used to investigate local synchronization. Global exponential stability is studied, too. We do not assume that the coupling matrix is symmetric or irreducible. The linear matrix inequality approach is used to judge synchronization with global convergence property.
Therefore, the study of synchronization of coupled neural networks is an important step for both understanding brain science and designing coupled neural networks for practical use. Recurrently connected neural network with delays can be written as
Index Terms—Chaos, delays, dynamical systems, neural network, synchronization.
(1)
I. INTRODUCTION
A
N INCREASING interest has been devoted to the study of dynamical behaviors of recurrently connected neural networks (RCNNs), also called Grossberg–Hopfield neural networks (see [1]–[3]), and many applications have been found in different areas. Most of previous studies are concentrated on such issues as stability analysis [1], [4]–[6] and periodic oscillation [7]. However, it has been shown that the recurrently connected neural networks can exhibit some complicated dynamics, such as chaotic behavior and others, see [8] and other papers. Recently, arrays of coupled systems have attracted much attention of researchers in different research fields. They can exhibit many interesting phenomena, such as spatio-temporal chaos [9], auto waves [10]–[12], spiral waves [13], etc., array of coupled systems is also important in modeling populations of interacting biological systems [14] and many others. For example, fireflies have been known to fire in unison, and this phenomenon has been proved to occur in a group of integrate-and-fire cells [15]. Moreover, experiment and theoretical analysis have revealed that a mammalian brain not only displays in its storage of associative memories, but also modulate oscillatory neuronal synchronization by selective perceive attention [16], [17]. Synchronization of coupled neural networks also have many applications. In [12], [18], [19], the authors proposed so called “the autowave principles for parallel image processing.” In [20], the authors presented an architecture of coupled neural networks to store and retrieve complex oscillatory patterns as synchronization states. In [21], the authors introduced a secure communication system based on coupled cellular neural networks.
Manuscript received March 9, 2004; revised June 9, 2004. This work was supported by the National Science Foundation of China under Grant 69982003 and Grant 60074005. The authors are with Laboratory of Nonlinear Science, Institute of Mathematics, Fudan University, Shanghai 200433, China (e-mail: [email protected]). Digital Object Identifier 10.1109/TCSI.2004.838308
where is the state of the th neuron, denotes the output denotes the output of the th of the th neuron, , is the weight connecting the output of neuron at time the th neuron to the th neuron, is the weight connecting the to the th neuron at time output of the th neuron at time , and denotes the input to the th neuron. We also assume that all differential equations have a unique solution with initial value (2) where . , then the model can be rewritten as following If matrix form: (3) where
, ,
,
,
, ,
, , and . An array of linearly coupled identical delayed neural networks with identical networks can be described componentwise as
(4) where , state of the neurons in the th neural network.
1057-7122/04$20.00 © 2004 IEEE
, denotes the
2492
Let
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
. Thus, the system can be rewritten
as
(5) If , for all written in matrix form as
, then, the system can be
(6) where denotes the coupling configuration of the coupled networks with
(7) and . It should be noted that several authors have investigated the synchronization of linearly coupled networks described as (8) when the coupling matrix is irreducible, symmetric and satisfies (7), local stability analysis via the linearlization technique has been given (see [22]–[24]); In [25], the authors defined a distance between any point and the synchronization manifold. By this distance, the authors proposed a methodology to discuss global convergence for complete regular coupling configuration. Randomly coupled networks are studied in [24], [26], such as small-world lattices and scale-free networks. Most of these papers focus on arrays of systems described by ordinary differential equations without delays. In practice, dynamical systems with delays are more important in theoretical study and applications. Hence, synchronization of coupled systems with delays is interesting and necessary to be investigated. In this paper, motivated by [23], [25], we propose some methodologies to study the local and global stability of the synchronization manifold for arrays of coupled connected neural networks with delays. By these methodologies, we give series of sufficient conditions for arrays of the delayed systems to be synchronized locally and globally. These sufficient conditions are easy to verify by some linear matrix inequality (LMI) tools. Moreover, these methodologies can be extended to study synchronization of arrays of some more general class of delayed systems. As for the coupling matrix, we do not assume that the coupling matrix is symmetric or irreducible. This paper is organized as follows. In Section II, some necessary definitions, lemmas, and hypotheses are given; In Section III, we deal with local stability of synchronization manifold; LMI-based criteria are obtained for global synchronization of linearly coupled identical delayed RCNNs in Section IV.
Some numerical examples to verify theoretical results are given in Section V. In Section VI, we give two examples of applications for image processing and secure communications. We conclude the paper in Section VII. II. PRELIMINARIES In this section, we present some definitions, which are needed throughout the paper. Definition 1: Let . The set is called synchronization manifold. Definiton 2: Synchronization manifold is said to be locally exponential stable, equivalently, the coupled system (5) or (6) is locally exponentially synchronized, if there exist constants , , , such that if , then for sufficient large
, we have if
and
(9) Definiton 3: Synchronization manifold is said to be globally exponential stable, equivalently, the coupled system (5) or , (6) is globally exponentially synchronized, if there exists , , for all , there hold (10) for all , . To measure the distance between any point and synchronization manifold , we use the following definitions introduced in [25]. : is composed of matrices Definiton 4: Set of with columns. Each row (for instance, the th row) of has exactly one entry and one entry , where . All other entries are zeroes. Definiton 5: Set of : are matrices obtained by replacing entry in with , where is the -dimension identity matrix, i.e., is the -dimensional identity matrix , where is Kronecker product. : If , then, for Definiton 6: any pair of indexes and , there exist indexes , where and , and such that and for all . is said to Definiton 7: A matrix of order satisfy condition , if 1)
(11) 2) Real part of eigenvalues of are all negative except an eigenvalue 0 with multiplicity one. denotes the class of functions with LipDefiniton 8: . shitz constant Finally, some lemmas are needed.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
Lemma 1 [27]: If satisfies condition (11) and is irreducible, then has an eigenvalue 0 with multiplicity 1 and corre1) . sponding eigenvector is an eigenvalue of , then its real part 2) If . In [25], the following Lemma 2 and Lemma 3 have been given. and is symLemma 2: If the matrix satisfies condition metric. Then is irreducible if and only if there exists a matrix , such that . Lemma 3: Let where , . Then if and only if (12) holds for some . Remark 1: According to the definition above, we can use to measure the difference between and synchronization manifold .
2493
focusing on the local synchronization for arrays of dynamic systems without delays. Instead, here, we investigate local stability of the manifold in the infinite dimensional Banach Space . IV. GLOBAL EXPONENTIAL ASYMPTOTIC SYNCHRONIZATION In the previous section, we discussed local stability of the synchronization manifold, proved that the trajectory comes back to the manifold after an enough small perturbation. It is an important issue to investigate the global synchronization. In this section, we discuss this issue. Under some mild conditions, we prove the trajectory will converge to the synchronization manifold exponentially from any initial value. , , . Suppose Theorem 2: Let , , and the coupling matrix that satisfies . If there exist , , , , , for all , and an irreducible matrix satisfying such that
III. LOCAL EXPONENTIAL ASYMPTOTIC SYNCHRONIZATION In this section, we investigate local stability of synchronization manifold for the coupled system (5) with the coupling matrix satisfying condition . The coupling matrix has eigenvalues. They are , , where is the , , is multiple, and imaginary unit, . Theorem 1: Suppose that , , , , , and the coupling matrix satisfies condition . If and , such that there exist
(13) hold for , . Then, the synchronization manifold of the coupled system (5) is locally exponentially . stable with convergence rate Proof of Theorem 1 will be given in Appendix I. Following statements in Remark 2 and Remark 3 are direct consequences of Theorem 1 and the properties of -matrices (see [27]). Remark 2: If the condition (13) is replaced by the following inequality:
(14) , are eigenvalues of the matrix . where Then, the synchronization manifold is stable exponentially. Remark 3: It can be seen that conditions (14) are equivalent are matrices, to that , are eigenvalues of the matrix , where is denoted as . where Remark 4: In [22]–[24], local convergence analysis via linearization technique was given. However, they were all
(15) and (16) hold for , where is an identity matrix of order , is the symmetric part of . Then the synchronization manifold of the couple system (5) is globally exponentially stable with convergence rate . Proof of Theorem 2 will be given in Appendix II. , for all , In case all delays are the same we can give the following result, which can be solved by LMI technique. , , . Theorem 3: Let , , and the Suppose that satisfy . If there exist a posicoupling matrix , positive definite diagonal matrices tive constant , , , , , a diagonal matrix , a semi-positive definitive matrix , and an irreducible matrix satisfying , such that
(17) and (18) hold for . Then the synchronization manifold of the system (6) is globally exponentially asymptotically stable with convergence rate .
2494
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Proof of Theorem 3 will be given in Appendix III. If the coupling matrix is symmetric, irreducible, we have the following corollary, which is a direct consequence of Theorem 3. Corollary 1: Suppose the coupling matrix is symmetric, , of which eigenvalues are irreducible and satisfies condition . If we select constants , by following rule: if if if
(19)
Then, . Therefore, (18) in Theorem 3 can be replaced by (19), and (16) in Theorem 2 can be also replaced by (19). Proof: Because is symmetric. It is easy to see that for fixed , , the eigenvalues of the symmetric matrix satisfy
Therefore, under the conditions of the corollary, we have . Corollary 1 is proved. Remark 5: Corollary 1 indicates that it is the maximum of that counts the global synchrononzero eigenvalue are positive. Therefore, the largest nonzero nization if all plays important role when investigate the eigenvalue of synchronization, see [24], [25]. To utilize the LMI technique, we have to fix , for example, , and impose a little more strict constraints on , and define the following matrix: (20) is the -dimensional identity matrix. where Now, we give the following corollary, which is a variant of Theorem 3. , , Corollary 2: Suppose , and the coupling matrix satisfies . If there , positive diagonal matrices exist a positive constant , , , , , a diagonal matrix , a nonnegative definite matrix , and a matrix satisfying
(21) such that
and for all (23) hold for . Then, the synchronization manifold (6) is globally exponentially asymptotically stable with convergence rate . Proof: In fact, it is easy to see that under the conditions of Corollary 2, satisfies conditions of Theorem 3. Because . From the (21) implies that is irreducible. And let , where definition of , one can see that . The orthogonal . One subspace of is described by vector can easily see that . , one has Therefore, for any (24) Corollary 2 is proved. Thus, the general approach to determine whether an array of identical connected neural networks with delays is globally synchronized by Theorem 2, Corollary 1, or Corollary 2 consists of the following two steps. By Theorem 2: , , , and Step 1) Find parameters such that (15) hold for . Step 2) Find matrix such that (16) hold for . By Corollary 1: Step 1) Find parameters such that (15) or (22) is satisfied. Step 2) Verify whether (19) is satisfied. By Corollary 2: , matrices , , , , , such that Step 1) Find (22) is satisfied. such that (23) hold for Step 2) Find matrix . If we can find these parameters in Step 1) and 2), then, the system is globally synchronized by the coupling matrix . It can be seen that if we fix some parameters, then Step 1) and 2) are all LMI problems, which can be solved by some LMI tool, for example, Matlab LMI Control Toolbox. Details of how to manipulate Step 1 and 2 by using Theorem 2, Corollary 1, and Corollary 2 by LMI method are presented in the next section. Remark 6: In [24], [26], all the authors only dealt with the case that the coupling matrix is irreducible and symmetric. But in practice, irreducible or symmetric coupling matrix is not enough. For example, directed graphs, of which coupling matrix may be reducible and asymmetric. Instead, in this paper, we present some criteria which can be used to treat these cases. Remark 7: The methodology we used in discussing global synchronization of an array of linearly coupled dynamic systems can be extended to discuss more general delayed dynamic systems. For example
(25) (22)
The details are a subject of current investigation.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
Fig. 1.
Fig. 2.
Chaotic trajectory of the model (26).
V. NUMERICAL EXAMPLES In this section, we give several simulations to verify theoretical results obtained in previous sections. We show how to use these theories to determine whether the coupling configurations can synchronize several uncoupled neural networks and in what range some parameters can vary to ensure the synchronization. We consider the following two-dimensional (2-D) neural network with delays:
2495
Distance of states x (t), i = 1, 2, 3 of the coupled system (29).
We will explain how the coupled system (29) synchronizes three identical systems defined by (26) with different initial values by using Theorem 2. , , , First, we choose the parameters . Then, by calculation, we find , such that
(26) ,
where , and
hold. It means that (15) is satisfied. Secondly, with Control Toolbox, we obtain
, using the Matlab LMI
(27) The dynamical behavior with initial condition (28) is developed doubled like chaotic attractors in Fig. 1. In the following Examples 1, 2, and 3, we demonstrate how to find the coupling matrix to synchronize three networks with different initial values (0.4,0.6), (5.0, 10.0) and ( 4.0,7.0) by using Theorem 2, Corollary 1, and Corollary 2, respectively. Example 1: In this example, we show how to design a coupled system by Theorem 2. We consider the linearly coupled system with three subsystems defined before
The eigenvalues of and are 0,4.1897, 10.9211; and 0,3.0498, 10.0098, respectively. Thus, are nonnegative definite for , 2, and all the conditions in Theorem 2 are satisfied and the three systems are synchronized. The performance of synchronization of the coupled system is shown in Fig. 2 Fig. 2 indicates the distance among three trajectories by calculating (30)
(29)
Example 2: In this example, we also consider the linearly coupled system with three subsystems
, and
where .
(31)
2496
Fig. 3.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Distance of states x (t), i = 1, 2, 3 of the coupled system (31).
Fig. 4. Dependence of error err(100) on coupling strength c for the coupled system (31).
where the coupling matrix is the following symmetric matrix: The result is
with eigenvalues 0, 6, and 14. Note that , and . Then inequalities (19) are . satisfied, if We will apply Corollary 1 and Theorem 3 to search the constant such that the coupled system is synchronized. . We search by solving the following LMIFirst, let Optimization problem. Minimize under constraints There exists a diagonal matrix There exists a diagonal matrix There exists a symtric matrix There exists a diagonal matrix
As the result, we obtain . Thus, for any , , the synchronizathe system can be synchronized. For tion performance is illustrated by Fig. 3 . with , search Now, for each the constant depending on by solving the following optimization problem. Minimize under constraints There exists a diagonal matrix There exists a diagonal matrix There exists a symtric matrix There exists a diagonal matrix
and
.
It is natural to raise the question. Can we find the sharp bound ? By simulations, such bound exists indeed. we simulate the system (31) for 50 different initial values selected randomly. and plot them in Fig. 4. In each simulation, record , , which indicates that It seems that if the lowest bound is 0.06 approximately. Unfortunately, it is too difficult to prove it theoretically. Example 3: In this simulation, we consider coupled system
(32) with coupling matrix
which is asymmetric and reducible. We show how to synchronize the system by Corollary 2. Step 1) Solve the following LMI-optimization problem. Minimize subject to there exist diagonal matrices and symtric matrix such that
The result is . thus, the matrix in. equality (22) is satisfied with subject to following constraints: Step 2) Search
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
2497
can detect a horseshoe-shaped closed curve by synchronization and de-synchronization of the system. A 2-D array of identical delayed neural networks with 21 21 lattices, is given. At position , , a 2-D defined by (26) is set down. And the array neural network of linear coupled system is written as follows:
(33) where
,
, and are defined in (29). The coefficients , and are defined as follows: If nodes , , ,or are in the lattice but out , , , or of the curve, the corresponding equals 1; otherwise, it equals zero. It is seen that every node is connected with the four closest adjacent nodes. The detection process consists of the following steps. 1) The input image for processing is stored in the array by the way that nodes belonging to the closed curve are isolated from other nodes. 2) We use 1 and 0 to represent synchronization and de-synchronization state, respectively. 3) At first, all nodes in the lattice are set to (0,0) and synchronization states are all set to 0. An autowave is initiated at the node (1,1) by setting (0.4,0.6) as its initial condition, which develops a doubled-like chaotic attractor at the node (1,1) (see Fig. 1). The autowave propagates through other nodes by the way that the state of other nodes is synchronized with node (1,1), except the nodes in the curve are asynchronous with the node (1,1) all the time. In this way, we can detect the closed curve. 4) Numerically, we use explicit Euler method with time step as a scale 0.005. We define to differentiate states of synchronization and de-synchro, then we say nization: Set some threshold , if is synchronous with node (1,1) and denoted by node is asynchronous with node (1,1) 1. Otherwise, node and denoted by 0. Such selection can be realized by zero detector in integrated circuits (VLSI) design. In our numerical example, we set Fig. 6 shows the process of detection, where Fig. 6(a) is the image to be detected. Fig. 6 (b)–(d) shows the snapshot at time , 35, and 50, respectively. Remark 8: In [12], the authors used a bistable system to realize a travelling wave for detection of closed curves by the way and that the nodes in the curve are stable at one equilibrium other nodes out the curve are stable at another equilibrium . However, if a dramatic perturbation of state happens, the state of the nodes belonging to the curve may jump into the attract basin , then the state of these nodes in the curve and out of the of curve may be identical. Thus, the detected curve may not be the wanted. Instead, the synchronization technique we utilize here to detect a closed curve is never influenced by the initial conditions; Even the doubled-like chaotic behavior at (1,1) does not ,
Fig. 5. Distance of states x (t), i = 1, 2, 3 of the coupled system (32).
where
By using Matlab LMI Control Toolbox, we get
The eigenvalues of are 0.0018, 0.0090. By Corollary 2, the system is synchronized. The performance of synchronization is illustrated in Fig. 5.
VI. APPLICATIONS In this section, we give two application examples of synchronization of coupled neural networks with delays. A. Image Processing: Detection of a Closed Curve In [10], [11], Krinsky et al. proposed the so-called “the autowave principles for parallel image processing.” Autowaves represent a particular class of nonlinear waves, which propagate in an active excitable media at the expense of the energy stored in the medium and differ from classical waves by several properties: Annihilation rather than interference when colliding between two waves and no reflection from obstacles. However, the only common feature is diffraction. In [12], the authors described an array of resistively coupled Chua’s circuits which can be designed to implement some elementary aspects of spatial recognition, namely, recognizing open curves from closed ones and locating the shortest path between two locations by travelling waves which is characterized by only two stable equilibrium states. Motivated by these observation, here we present a simple example exploring the possibility of using synchronization and de-synchronization of a 2-D array of linearly coupled connected neural networks with time delays proposed in previous sections for image processing. The model we present here
,
,
2498
Fig. 6.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Snapshots of image processing. (a) Input image. (b) Snapshot at t = 25. (c) Snapshot at t = 35. (d) Snapshot at t = 50.
affect synchronization process. Therefore, the method we use succeeds to detect image under any perturbations. B. Secure Communication Recently, application of chaotic synchronization of coupled systems to secure communication has become as area of active research, see [21], [28] and other papers. A signal containing useful messages is transmitted using a chaotic signal as a broadband carrier, and the synchronization is used to recover the messages at the receiver. Here, we present a secure communication scheme based on coupled delayed neural networks. The system consists of the following two subsystems: Driver subsystem
where
, , , , where
is identical to the message signal signal . A quantity is defined to meaand the recovered sure the error between message signal signal . Fig. 7 indicates the dynamical behaviors of mes, transmitting signal , recovered signal sage signal and the error through time. As Fig. 7 shows, converges to zero, which implies that the message is recovered successfully. VII. CONCLUSION In this paper, we study the synchronization of an array of linearly coupled identical connected neural networks with time delay; criteria for synchronization are given without assuming symmetry and irreducibility of coupling matrix . Simulations and applications are given, too.
, and are defined in (29), and is input messages to be transmitted
APPENDIX I Proof of Theorem 1
Receiver subsystem
, is the coupling strength, and
Direct calculation gives following variational equations of the coupled system (5) at the synchronization manifold : ,
is the transmitting signal comwhere and chaotic carrier signal . posed of message signal and the input message In this simulation, we take signal is a sinusoid signal. The coupled systems can be syn. Hence, recovered chronized, i.e.,
(34) where
,
.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
Fig. 7.
Dynamical behaviors of
m(t), s(t), r(t) and error(t) through time.
Let be the Jordan decomposition, where a block-diagonal matrix
..
and
2499
is
where , variation equation at the manifold
. In terms of is
, the
.
(35)
is the block
..
. (36)
corresponding to the eigenvalue with multiple . is an 1 1 matrix (number) corresponding to , for satisfies condition . Let , , where and , where , , , . Then, we have
where
, . We first discuss the case . Let , and , , , The (35) can be rewritten as
and
,
and prove that , where , , and is the imaginary unit.
(37)
2500
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
Define
(38)
It is easy to see that if for all such that, there exists
,
, then
Define the Lyapunov functional holds for all
(39) Then, by some algebra, we have
. Otherwise, there exists , such that, , and is strictly monotone at , is strictly monotone at , and which implies that , furthermore, . Therefore, is bounded and for all and . for all By previous reasoning, we have , which implies (43) By the Jordan decomposition , we have . By Lemma 1, the first row of is . Therefore (44)
which implies (40) Secondly, we will prove that by induction. Supposing that
which implies that converges to orem 1 is proved completely.
for all
exponentially. The-
APPENDIX II
, then Proof of Theorem 2
(41) Denote
, and define
By the assumption imposed on and Lemma 2, there exists a matrix such that . Moreover, let be the matrix defined in Definition denotes th 4 and Definition 5. , , and , row of , where , . Moreover, define if
.. . (42)
where with
for all
is assumed to be . we have
Then
(45) where
Define (46)
(47)
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
and
We will prove that lations give
2501
Therefore
(48) . In fact, direct calcu-
(49)
and is bounded and we conclude that the synchronization manifold is globally exponentially asymptotically stable with rate , namely
It can be seen that (50) Theorem 2 is proved. APPENDIX III Proof of Theorem 3
and
where Hence,
, for all
.
By the assumptions imposed on and Lemma 5, there exists a matrix such that . Moreover, be the matrix defined in Definition let denotes th 4 and Definition 5. , , and , row of , where if , . denote the Kroneker product of matrices and , Let and
Equation (6) can be rewritten as Moreover, it is easy to see that (51) ,
Let ,
, and
2502
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: REGULAR PAPERS, VOL. 51, NO. 12, DECEMBER 2004
, , for
. Then, . Define Lyapunov
For simplicity of writing, denote the matrix of (17) by we have
, then
function
(52) Its derivative along the trajectory is
By conditions (18), we have
By the structure of
, following equalities are easy to verify:
Therefore, we have
Moreover, condition (17) implies that fore
is semi-definite. There-
Thus (53) Theorem 3 is proved. ACKNOWLEDGMENT The authors thank the reviewers for their comments, which were very helpful in the revision of the paper. Direct calculation gives
and
REFERENCES [1] M. A. Cohen and S. Grossberg, “Absolute stability of global pattern formation and parallel memory storage by competitive neural networks,” IEEE Trans. Syst., Man, Cybern., vol. SMC-13, pp. 815–826, Apr. 1983. [2] D. Tank and J. J. Hopfield, “Simple neural optimization networks: An A/D converter, signal decision circuit and a linear programming circuit,” IEEE Trans. Circuits Syst., vol. CAS-33, pp. 533–541, Apr. 1986. [3] J. J. Hopfield, “Neurons with graded response have collective computation properties like those of two-stage neurons,” Proc. Nat. Acad. Sci. USA (Biophysical), vol. 81, pp. 3088–3092, 1984. [4] T. Chen, “Convergence of delayed dynamical systems,” Neural Processing Lett., vol. 10, no. 3, pp. 267–271, 1999. [5] T. Chen and S. Amari, “Stability of asymmetric Hopfield neural networks,” IEEE Trans. Neural Netw., vol. 12, pp. 159–163, Jan. 2001. [6] T. Chen, “Global exponential stability of delayed Hopfield neural networks,” Neural Netw., vol. 14, no. 8, pp. 977–980, 2001. [7] Y. Zheng and T. Chen, “Global exponential stability of delayed periodic dynamical systems,” Phys. Lett. A, vol. 322, no. 5–6, pp. 344–355, 2004. [8] H. Sompolinsky and A. Crisanti, “Chaos in random neural networks,” Phys. Rev. Lett., vol. 61, no. 3, pp. 259–262, 1988. [9] A. Zheleznyak and L. O. Chua, “Coexistence of low- and high-dimensional spatio-temporal chaos in a chain of dissipatively coupled Chua’s circuits,” Int. J. Bifur. Chaos, vol. 4, no. 3, pp. 639–672, 1994. [10] V. I. Krinsky, V. N. Biktashev, and I. R. Efimov, “Autowaves principles for parallel image processing,” Phys. D., vol. 49, pp. 247–253, 1991.
LU AND CHEN: SYNCHRONIZATION OF COUPLED CONNECTED NEURAL NETWORKS
[11] L. Kunbert, K. I. Agladze, and V. I. Krinsky, “Image processing using light-sensitive chemical waves,” Nature, vol. 337, pp. 244–247, 1989. [12] V. Perez-Munuzuri, V. Perez-Villar, and L. O. Chua, “Autowaves for image processing on a two-dimensional CNN array of excitable nonlinear circuits: Flat and wrinkled labyrinths,” IEEE Trans. Circuits Syst. I, vol. 40, pp. 174–181, Feb. 1993. [13] A. Perez-Munuzuri, V. Perez-Munuzuri, and V. Perez-Villar, “Spiral waves or a two-dimensional array of nonlinear circuits,” IEEE Trans. Circuits Syst. I, vol. 40, pp. 872–877, Aug. 1993. [14] J. D. Murray, Mathematics Biology. Berlin, Germany: SpringerVerlag, 1989. [15] R. E. Mirollo and S. H. Strogatz, “Synchronization of pulse-coupled biological oscillators,” SIAM J. Applicat. Math., vol. 50, no. 6, pp. 1645–1662, 1990. [16] P. Fries, J. H. Reynolds, J. H. Rorie, and R. Desimore, “Modulation of oscillatory neuronal synchronization by selective visual attention,” Science, vol. 291, no. 23, pp. 1560–1563, 2001. [17] P. N. Steinmetz, A. Roy, P. J. Fitzgerald, S. S. Hsiao, K. O. Johnson, and E. Niebar, “Attention modulate synchronized neuronal firing in primate somatosensory cortex,” Nature, vol. 404, no. 9, pp. 487–490, 2000. [18] L. O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE Trans. Circuits Syst., vol. 35, pp. 1257–1272, Oct. 1988. , “Cellular neural networks: Applications,” IEEE Trans. Circuits [19] Syst., vol. 35, pp. 1273–1290, Oct. 1988. [20] F. C. Hoppensteadt and E. M. Izhikevich, “Pattern recognition via synchronization in phase-locked loop neural networks,” IEEE Trans. Neural Netw., vol. 11, pp. 734–738, Mar. 2000. [21] Y. Zhang and Z. He, “A secure communication scheme based on cellular neural networks,” in Proc. IEEE Int. Conf. Intelligent Processing Systems, vol. 1, 1997, pp. 521–524. [22] L. M. Pecora, “Synchronization conditions and desynchronization patterns in coupled lmit-cycle and chaotic systems,” Phys. Rev. E, no. 58, pp. 347–360, 1998. [23] L. Pecora, T. Carroll, G. Johnson, D. Mar, and K. S. Fink, “Synchronization stability in coupled oscillator arrays: Solution for arbitrary configuration,” Int. J. Bifurc. Chaos, vol. 10, no. 2, pp. 273–290, 2000.
2503
[24] X. F. Wang and G. Chen, “Synchronization in scale-free dynamical networks: Robustness and fragility,” IEEE Trans. Circuits Syst. I, vol. 49, pp. 54–62, Jan. 2002. [25] C. W. Wu and L. O. Chua, “Synchronization in an array of linearly coupled dynamical systems,” IEEE Trans. Circuits Syst. I, vol. 42, pp. 430–447, Aug. 1995. [26] X. F. Wang and G. Chen, “Synchronization in small-world dynamical networks,” Int. J. Bifurc. Chaos, vol. 12, no. 1, pp. 187–192, 2002. [27] P. A. Horn and C. R. Johnson, Matrix Analysis. New York: Cambridge Univ. Press, 1985. [28] J. R. Terry and G. D. VanWiggeren, “Chaotic communication using generalized synchronization,” Chaos, Solit. Fract., vol. 12, pp. 145–152, 2001.
Wenlian Lu is working toward the Ph.D. degree in mathematics at the Fudan University, Shanghai, China. He has published four papers in respectable international journals and three papers in international conferences. His research interests are neural networks, dynamical systems and complex networks.
Tianping Chen is a Professor of Department of Mathematics, Fudan University, Shanghai, China. His research interests include hamornic analysis, approximation theory, neural networks, signal processing and dynamical systems. Dr. Chen is a receipent of several important awards, including the second prize of the 2002 National Natural Sciences Award of China, the 1997 Outstanding Paper Award of IEEE TRANSACTIONS ON NEURAL NETWORKS, and the 1997 Best paper Award of Japanese Neural Network Society.
