PDF (2563 KB) - MIT Press Journals
LETTER
Communicated by Richard Hahnloser
Multiperiodicity and Exponential Attractivity Evoked by Periodic External Inputs in Delayed Cellular Neural Networks Zhigang Zeng [email protected] School of Automation, Wuhan University of Technology, Wuhan, Hubei, 430070, China
Jun Wang [email protected] Department of Automation and Computer-Aided Engineering, Chinese University of Hong Kong, Shatin, New Territories, Hong Kong
We show that an n-neuron cellular neural network with time-varying delay can have 2n periodic orbits located in saturation regions and these periodic orbits are locally exponentially attractive. In addition, we give some conditions for ascertaining periodic orbits to be locally or globally exponentially attractive and allow them to locate in any designated region. As a special case of exponential periodicity, exponential stability of delayed cellular neural networks is also characterized. These conditions improve and extend the existing results in the literature. To illustrate and compare the results, simulation results are discussed in three numerical examples. 1 Introduction Cellular neural networks (CNNs) and delayed cellular neural networks (DCNNs) are arrays of dynamical cells that are suitable for solving many complex computational problems. In recent years, both have been extensively studied and successfully applied for signal processing and solving nonlinear algebraic equations. As dynamic systems with a special structure, CNNs and DCNNs have many interesting properties that deserve theoretical studies. In general, there are two interesting nonlinear neurodynamic properties in CNNs and DCNNs: stability and periodic oscillations. The stability of a CNN or a DCNN at an equilibrium point means that for a given activation function and a constant input vector, an equilibrium of the network exists and any state in the neighborhood converges to the equilibrium. The stability of neuron activation states at an equilibrium is prerequisite for most applications. Some neurodynamics have multiple (two) stable equilibria and may be stable at any equilibrium depending on the initial state, which is called Neural Computation 18, 848–870 (2006)
C 2006 Massachusetts Institute of Technology
Multiperiodicity and Exponential Attractivity of CNNs
849
multistability (bistability). For stability, either an equilibrium or a set of equilibria is the attractor. Besides stability, an activation state may be periodically oscillatory around an orbit. In this case, the attractor is a limit set. Periodic oscillation in recurrent neural networks is an interesting dynamic behavior, as many biological and cognitive activities (e.g., heartbeat, respiration, mastication, locomotion, and memorization) require repetition. Persistent oscillation, such as limit cycles, represents a common feature of neural firing patterns produced by dynamic interplay between cellular and synaptic mechanisms. Stimulus-evoked oscillatory synchronization was observed in many biological neural systems, including the cerebral cortex of mammals and the brain of insects. It was also known that time delays can cause oscillations in neurodynamics (Gopalsamy & Leung, 1996; Belair, Campbell, & Driessche, 1996). In addition, periodic oscillations in recurrent neural networks have found many applications, such as associative memories (Nishikawa, Lai, & Hoppensteadt, 2004), pattern recognition (Wang, 1995; Chen, Wang, & Liu, 2000), machine learning (Ruiz, Owens, & Townley, 1998; Townley et al., 2000), and robot motion control (Jin & Zacksenhouse, 2003). The analysis of periodic oscillation of neural networks is more general than stability analysis since an equilibrium point can be viewed as a special case of oscillation with any arbitrary period. The stability of CNNs and DCNNs has been widely investigated (e.g., Chua & Roska, 1990, 1992; Civalleri, Gilli, & Pandolfi, 1993; Liao, Wu, & Yu, 1999; Roska, Wu, Balsi, & Chua, 1992; Roska, Wu, & Chua, 1993; Setti, Thiran, & Serpico, 1998; Takahashi, 2000; Zeng, Wang, & Liao, 2003). The existence of periodic orbits together with global exponential stability of CNNs and DCNNs is studied in Yi, Heng, and Vadakkepat (2002) and Wang and Zou (2004). Most existing studies (Berns, Moiola, & Chen, 1998; Jiang & Teng, 2004; Kanamaru & Sekeine, 2004; Liao & Wang, 2003; Liu, Chen, Cao, & Huang, 2003; Wang & Zou, 2004) are based on the assumption that the equilibrium point of CNNs or DCNNs is globally stable or the periodic orbit of CNNs or DCNNs is globally attractive; hence, CNNs or DCNNs have only one equilibrium point or one periodic orbit. However, in most applications, it is required that CNNs or DCNNs exhibit more than one stable equilibrium point (e.g., Yi, Tan, & Lee, 2003; Zeng, Wang, & Liao, 2004), or more than one exponentially attractive periodic orbit instead of a single globally stable equilibrium point. In this letter, we investigate the multiperiodicity and multistablity of DCNNs. We show that an n-neuron DCNN can have 2n periodic orbits that are locally exponentially attractive. Moreover, we present the estimates of attractive domain of such 2n locally exponentially attractive periodic orbits. In addition, we give the conditions for periodic orbits to be locally or globally exponentially attractive when the periodic orbits locate in a designated position. All of these conditions are very easy to be verified.
850
Z. Zeng and J. Wang
The remaining part of this letter consists of six sections. In section 2, relevant background information is given. The main results are stated in sections 3, 4, and 5. In section 6, three illustrative examples are provided with simulation results. Finally, concluding remarks are given in section 7. 2 Preliminaries Consider the DCNN model governed by the following normalized dynamic equations: dxi (t) a ij f (x j (t)) = − xi (t) + dt n
j=1
+
n
b ij f (x j (t − τij (t))) + ui (t), i = 1, . . . , n,
(2.1)
j=1
where x = (x1 , . . . , xn )T ∈ n is the state vector, A = (a ij ) and B = (b ij ) are connection weight matrices that are not assumed to be symmetric, u(t) = (u1 (t), . . . , un (t))T ∈ n is a periodic input vector with period ω (i.e., there exists a constant ω > 0 such that ui (t + ω) = ui (t) ∀t ≥ 0, ∀i ∈ {1, 2, . . . , n}), τij (t) is the time-varying delay that satisfies 0 ≤ τij (t) ≤ τ (τ is constant), and f (·) is the piecewise linear activation function defined by f (v) = (|v + 1| − |v − 1|)/2. In particular, when b ij ≡ 0 (∀i, j = 1, 2, . . . , n), the DCNN degenerates as a CNN. Let C([t0 − τ, t0 ], D) be the space of continuous functions mapping [t0 − τ, t0 ] into D ⊂ n with the norm defined by ||φ||t0 = max1≤i≤n {supu∈[t0 −τ,t0 ] |φi (u)|}, where φ(s) = (φ1 (s), φ2 (s), . . . , φn (s))T . Denote ||x|| = max1≤i≤n {|xi |} as the vector norm of the vector x = (x1 , . . . , xn )T . ∀φ, ϕ ∈ C([t0 − τ, t0 ], D), where φ(s) = (φ1 (s), φ2 (s), . . . , φn (s))T , ϕ(s) = (ϕ1 (s), ϕ2 (s), . . . , ϕn (s))T . Denote ||φ, ϕ||t0 = max1≤i≤n {supt0 −τ ≤s≤t0 {|φi (s) − ϕi (s)|}} as a measurement in C([t0 − τ, t0 ], D). The initial condition of DCNN model 2.1 is assumed to be φ(ϑ) = (φ1 (ϑ), φ2 (ϑ), . . . , φn (ϑ))T , where φ(ϑ) ∈ C([t0 − τ, t0 ], n ). Denote x(t; t0 , φ) as the state of DCNN model 2.1 with initial condition (t0 , φ). It means that x(t; t0 , φ) is continuous and satisfies equation 2.1 and x(s; t0 , φ) = φ(s), for s ∈ [t0 − τ, t0 ]. Denote (−∞, −1) = (−∞,−1)1 ×[−1,1]0 ×(1,+∞)0 ;[−1,1] = (−∞,−1)0 × [−1, 1]1 × (1, +∞)0 ; (1, +∞)= (−∞, −1)0 × [−1, 1]0 × (1, +∞)1 , = (−∞, +∞) = (−∞, −1) [−1, 1] (1, +∞), so (−∞, +∞)n can be divided into 3n subspaces: (i)
(i)
(i)
n (−∞, −1)δ1 × [−1, 1]δ2 × (1, +∞)δ3 , = {i=1 (i) (i) (i) δ1 , δ2 , δ3 = (1, 0, 0) or (0, 1, 0) or (0, 0, 1), i = 1, . . . , n};
(2.2)
Multiperiodicity and Exponential Attractivity of CNNs
851
and can be divided into three subspaces: 1 = {[−1, 1]n } n 2 = {i=1 (−∞, −1)δ × (1, +∞)1−δ , δ (i) = 1 or 0, i = 1, . . . , n} (i)
(i)
3 = − 1 − 2 Hence, 1 is composed of one region, 2 is composed of 2n regions, and 3 is composed of 3n − 2n − 1 regions. Definition 1. A periodic orbit x ∗ (t) is said to be a limit cycle of a DCNN if x ∗ (t) is an isolated periodic orbit of the DCNN; that is, there exists ω > 0 such that ∀t ≥ t0 , x ∗ (t + ω) = x ∗ (t), and there exists δ > 0 such that ∀ x ∈ {x| 0 < ||x, x ∗ (t)|| < δ, x ∈ n , t ≥ t0 }, where x is not a point on any periodic orbit of the DCNN. Definition 2. A periodic orbit x ∗ (t) of a DCNN is said to be locally exponentially attractive in region if there exist constants α > 0, β > 0 such that ∀t ≥ t0 , x(t; t0 , φ) − x ∗ (t) ≤ β||φ||t exp{−α(t − t0 )}, 0 where x(t; t0 , φ) is the state of the DCNN with any initial condition (t0 , φ), φ(ϑ) ∈ C([t0 − τ, t0 ], ), and is said to be a locally exponentially attractive set of the periodic orbit x ∗ (t). When = n , x ∗ (t) is said to be globally exponentially attractive. In particular, if x ∗ (t) is a fixed point x ∗ , then the DCNN is said to be global exponentially stable. Lemma 1. (Kosaku, 1978). Let D be a compact set in n , H be a mapping on complete metric space (C([t0 − τ, t0 ], D), ||·, ·||t0 ). If H(C([t0 − τ, t0 ], D)) ⊂ C([t0 − τ, t0 ], D), and there exists a constant α < 1 such that ∀φ, ϕ ∈ C([t0 − τ, t0 ], D), ||H(φ), H(ϕ)||t0 ≤ α||φ, ϕ||t0 , then there exists φ ∗ ∈ C([t0 − τ, t0 ], D) such that H(φ ∗ ) = φ ∗ . Consider the following coupled system: dx(t) = −x(t) + Ay(t) + By(t − τ (t)), dt dy(t) = h(t, y(t), y(t − τ (t))), dt
(2.3) (2.4)
where x(t) ∈ n , y(t) ∈ m , A and B are n × m matrices, h ∈ C( × m × m , m ), and C( × m × m , m ) is the space of continuous functions mapping × m × m into m .
852
Z. Zeng and J. Wang
Lemma 2. If system 2.4 is globally exponentially stable, then system 2.3 is also globally exponentially stable. Proof. By applying the variant format of constants, the solution x(t) of equation 2.3 can be expressed as x(t) = exp{−(t − t0 )}x(t0 ) +
t
exp{−(t − s)}(Ay(s) + By(s − τ (s)))ds.
t0
Since equation 2.4 is globally exponentially stable, there exist constants α, β > 0 such that |y(s)| ≤ β exp{−α(s − t0 )}. Hence, |Ay(s) + By(s − τ (s))| ≤ β¯ exp{−α(s − t0 )}, where β¯ = (||A|| + ||B|| exp{ατ })β. Then ¯ − t0 ) exp{−(t − t0 )}; when α = 1, ∀t ≥ t0 , |x(t)| ≤ |x(t0 )| exp{−(t − t0 )} + β(t ¯ when α = 1, |x(t)| ≤ |x(t0 )| exp{−(t − t0 )} + β(exp{−(t − t0 )} + exp{−α(t − t0 )})/|1 − α|; that is, equation 2.3 is also globally exponentially stable. Throughout this article, we assume that N = {1, 2, . . . , n}, N N 1 2 3 N1 N2 , N1 N3 , and N2 N3 are empty. Denote D1 = {x ∈ n | xi ∈ (−∞, −1), i ∈ N1 ; xi ∈ (1, ∞), i ∈ N2 ; xi ∈ [−1, 1], i ∈ N3 }. Note that D1 ⊂ , where is defined in equation 2.2. If N3 is empty, then denote D2 = {x ∈ n |xi ∈ (−∞, −1), i ∈ N1 ; xi ∈ (1, ∞), i ∈ N2 }. 3 Locally Exponentially Attractive Multiperiodicity in a Saturation Region In this section, we show that an n-neuron delayed cellular neural network can have 2n periodic orbits located in saturation regions and these periodic orbits are locally exponentially attractive. Theorem 1. If ∀i ∈ {1, 2, . . . , n}, ∀t ≥ t0 , |ui (t)| < a ii − 1 −
n j=1, j =i
|a ij | −
n
|b ij |,
(3.1)
j=1
then DCNN (see equation 2.1) has 2n locally exponentially attractive limit cycles. Proof. If ∀s ∈ [t0 − τ, t], x(s) ∈ D2 , from equation 2.1, ∀i = 1, 2, . . . , n, dxi (t) = −xi (t) + (a ij + b ij ) − (a ij + b ij ) + ui (t). dt j∈N j∈N 1
2
(3.2)
Multiperiodicity and Exponential Attractivity of CNNs
853
When i ∈ N2 and xi (t) = 1, from equations 3.1 and 3.2, dxi (t) = −1 + (a ij + b ij ) − (a ij + b ij ) + ui (t) > 0. dt j∈N j∈N 1
(3.3)
2
When i ∈ N1 and xi (t) = −1, from equations 3.1 and 3.2, dxi (t) (a ij + b ij ) − (a ij + b ij ) + ui (t) < 0. =1+ dt j∈N j∈N 1
(3.4)
2
Equations 3.3 and 3.4 imply that if ∀φ ∈ C([t0 − τ, t0 ], D2 ), then x(t; t0 , φ) will keep in D2 , and D2 is an invariant set of DCNN (see equation 2.1). So ∀t ≥ t0 − τ, x(t) ∈ D2 . Hence, DCNN, equation 2.1, can be rewritten as equation 3.2. Let x(t; t0 , φ) and x(t; t0 , ϕ) be two states of DCNN, equation 2.1, with initial conditions (t0 , φ) and (t0 , ϕ), where φ, ϕ ∈ C([t0 − τ, t0 ], D2 ). From equations 2.1 and 3.2, ∀i ∈ {1, 2, . . . , n}, ∀t ≥ t0 , d(xi (t; t0 , φ) − xi (t; t0 , ϕ)) = −(xi (t; t0 , φ) − xi (t; t0 , ϕ)). dt
(3.5)
Hence, ∀i = 1, 2, . . . , n, ∀t ≥ t0 , |xi (t; t0 , φ) − xi (t; t0 , ϕ)| ≤ ||φ, ϕ||t0 exp{−(t − t0 )}.
(3.6)
(t)
Define xφ (θ ) = x(t + θ ; t0 , φ), θ ∈ [t0 − τ, t0 ]. Then from equations 3.3 (t)
and 3.4, ∀φ ∈ C([t0 − τ, t0 ], D2 ), xφ ∈ C([t0 − τ, t0 ], D2 ). Define a mapping (ω)
H : C([t0 − τ, t0 ], D2 ) → C([t0 − τ, t0 ], D2 ) by H(φ) = xφ . Then H(C([t0 − τ, t0 ], D2 )) ⊂ C([t0 − τ, t0 ], D2 ), (mω)
and H m (φ) = xφ . We can choose a positive integer m such that exp{−(mω − τ )} ≤ α < 1. Hence, from equation 3.6, ||H m (φ), H m (ϕ)||t0 ≤ max
1≤i≤n
sup θ∈[t0 −τ,t0 ]
|xi (mω + θ; t0 , φ) − xi (mω + θ; t0 , ϕ)|
≤ ||φ, ϕ||t0 exp{−(mω + t0 − τ − t0 )} ≤ α||φ, ϕ||t0 . Based on lemma 1, there exists a unique fixed point φ ∗ ∈ C([t0 − τ, t0 ], D2 ) such that H m (φ ∗ ) = φ ∗ . In addition, H m (H(φ ∗ )) = H(H m (φ ∗ )) = H(φ ∗ ). This shows that H(φ ∗ ) is also a fixed point of H m . Hence, by the uniqueness of the fixed
854
Z. Zeng and J. Wang (ω)
point of the mapping H m , H(φ ∗ ) = φ ∗ ; that is, xφ ∗ = φ ∗ . Let x(t; t0 , φ ∗ ) be a state of DCNN, equation 2.1, with initial condition (t0 , φ ∗ ). Then from equation 2.1, ∀i = 1, 2, . . . , n, ∀t ≥ t0 , dxi (t; t0 , φ ∗ ) = −xi (t; t0 , φ ∗ ) + (a ij + b ij ) − (a ij + b ij ) + ui (t). dt j∈N j∈N 1
2
Hence, ∀i = 1, 2, . . . , n, ∀t + ω ≥ t0 , dxi (t + ω; t0 , φ ∗ ) = −xi (t + ω; t0 , φ ∗ ) + (a ij + b ij ) dt j∈N −
1
(a ij + b ij ) + ui (t + ω)
j∈N2
= −xi (t + ω; t0 , φ ∗ ) + −
(a ij + b ij )
j∈N1
(a ij + b ij ) + ui (t).
j∈N2
This implies x(t + ω; t0 , φ ∗ ) is also a state of DCNN, equation 2.1, with initial (ω) condition (t0 , φ ∗ ). xφ ∗ = φ ∗ implies that ∀t ≥ t0 , x(t + ω; t0 , φ ∗ ) = x(t; t0 , xφ ∗ ) = x(t; t0 , φ ∗ ). (ω)
Hence, x(t; t0 , φ ∗ ) is a periodic orbit of DCNN, equation 2.1, with period ω. From equation 3.5, it is easy to see that any state of DCNN, equation 2.1, with initial condition (t, φ) (φ ∈ C([t0 − τ, t0 ], D2 )) converges to this periodic orbit exponentially as t → +∞. Hence, the isolated periodic orbit x(t; t0 , φ ∗ ) located in D2 is locally exponentially attractive, and D2 is a locally exponentially attractive set of x(t; t0 , φ ∗ ). Since there exist 2n elements in 2 , there exist 2n isolated periodic orbits in 2 . And such 2n isolated periodic orbits are locally exponentially attractive. When the periodic external input u(t) degenerates into a constant vector, we have the following corollary: Corollary 1. If ∀i ∈ {1, 2, . . . , n}, ∀t ≥ t0 , ui (t) ≡ ui (constant), and
|ui | < a ii − 1 −
n j=1, j =i
|a ij | −
n
|b ij |,
j=1
then DCNN (see equation 2.1) has 2n locally exponentially stable equilibrium points.
Multiperiodicity and Exponential Attractivity of CNNs
855
Proof. Since ui (t) ≡ ui (constant), for an arbitrary constant ν ∈ , ui (t + ν) ≡ ui ≡ ui (t). According to theorem 1, DCNN, equation 2.1, has 2n locally exponentially attractive limit cycles with period ν. The arbitrariness of constant ν implies that such limit cycles are fixed points. Hence, DCNN, equation 2.1, has 2n locally exponentially attractive equilibrium points. Remark 1. In theorem 1 and corollary 1, it is necessary for a ii to be dominant such that a ii > 1 + nj=1, j =i |a ij | + nj=1 |b ij |. Remark 2. A main objective for designing associative memories is to store a large number of patterns as stable equilibria or limit cycles such that stored patterns can be retrieved when the initial probes contain sufficient information about the patterns. CNNs and DCNNs are also suitable for very largescale integration (VLSI) implementations of associative memories. It is also expected that they can be applied to association memories by storing patterns as periodic limit cycles. According to theorem 1 and corollary 1, the n-neuron DCNN model, equation 2.1, can store up to 2n patterns in locally exponentially attractive limit cycles or equilibria, which can be retrieved when the input vector satisfies condition 3.1. This implies that the external stimuli also play a major role in encoding and decoding patterns in DCNN associative memories, in contrast with the zero input vector in the bidirectional associate memories and the autoassociative memories based on the Hopfield network.
4 Locally Exponentially Attractive Periodicity in a Designated Region As the limit cycles are stimulus driven (nonautonomous), some information can be encoded in the phases of the oscillating states xi relative to the inputs ui . Hence, it is necessary to find some conditions on the inputs ui , when the periodic orbit x(t) is desired to be located in a designated region. In this section, we give the conditions that allow a periodic orbit to be locally exponentially attractive and located in any designated region.
Theorem 2. If ∀t ≥ t0 , ui (t) < −1 +
(a ij + b ij ) −
j∈N1
ui (t) > 1 +
j∈N1
(a ij + b ij ) −
(a ij + b ij ) −
j∈N2
j∈N2
(a ij + b ij ) +
(|a ij | + |b ij |),
i ∈ N1 ,
j∈N3
(4.1) (|a ij | + |b ij |),
i ∈ N2 ,
j∈N3
(4.2)
856
Z. Zeng and J. Wang ui (t) < 1 − a ii − −
(a ij + b ij ),
ui (t) > a ii − 1 +
|a ij | −
j∈N3 , j =i
j∈N2
−
|b ij | +
j∈N3
(a ij + b ij )
j∈N1
i ∈ N3 , |a ij | +
j∈N3 , j =i
(a ij + b ij ),
j∈N3
(4.3) |b ij | +
(a ij + b ij )
j∈N1
i ∈ N3 ,
(4.4)
j∈N2
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) = τij (t + ω), then DCNN, equation 2.1, has only one limit cycle located in D1 , which is locally exponentially attractive in D1 . Proof. If ∀s ∈ [t0 − τ, t], x(s) ∈ D1 , then from equation 2.1, ∀s ∈ [t0 , t], ∀i = 1, 2, . . . , n, dxi (s) = −xi (s) − (a ij + b ij ) + (a ij + b ij ) + a ij x j (s) dt j∈N1 j∈N2 j∈N3 b ij x j (s − τij (s)) + ui (s). +
(4.5)
j∈N3
When i ∈ N1 and xi (t) = −1, from equations 4.1 and 4.5, dxi (t) (a ij + b ij ) + (a ij + b ij ) ≤1 − dt j∈N1 j∈N2 + (|a ij | + |b ij |) + ui (t) < 0.
(4.6)
j∈N3
When i ∈ N2 and xi (t) = 1, from equations 4.2 and 4.5, dxi (t) ≥ −1 − (a ij + b ij ) − (|a ij | + |b ij |) dt j∈N1 j∈N3 (a ij + b ij ) + ui (t) > 0. +
(4.7)
j∈N2
When i ∈ N3 and xi (t) = 1, from equations 4.3 and 4.5, dxi (t) ≤ −1 − (a ij + b ij ) + (a ij + b ij ) + a ii dt j∈N1 j∈N2 + |a ij | + |b ij | + ui (t) < 0. j∈N3 , j =i
j∈N3
(4.8)
Multiperiodicity and Exponential Attractivity of CNNs
857
When i ∈ N3 and xi (t) = −1, from equations 4.4 and 4.5, dxi (t) (a ij + b ij ) + (a ij + b ij ) − a ii ≥ 1− dt j∈N1 j∈N2 − |a ij | − |b ij | + ui (t) > 0. j∈N3 , j =i
(4.9)
j∈N3
Equations 4.6 to 4.9 imply that if ∀s ∈ [t0 − τ, t0 ], φ(s) ∈ D1 , then x(t; t0 , φ) will keep in D1 , and D1 is an invariant set of DCNN (see equation 2.1). So ∀t ≥ t0 − τ, x(t) ∈ D1 . Hence, ∀t ≥ t0 , DCNN, equation 2.1, can be rewritten as dxi (t) = −xi (t) − (a ij + b ij ) + a ij x j (t) + b ij x j (t − τij (t)) dt j∈N1 j∈N3 j∈N3 (a ij + b ij ) + ui (t), i = 1, 2, . . . , n. +
(4.10)
j∈N2
Let x(t; t0 , φ) and x(t; t0 , ϕ) be two states of DCNN (equation 4.10) with initial conditions (t0 , φ) and (t0 , ϕ), where φ, ϕ ∈ C([t0 − τ, t0 ], D1 ). From equation 4.10, ∀i = 1, 2, . . . , n; ∀t ≥ t0 , d(xi (t; t0 , φ) − xi (t; t0 , ϕ)) = −(xi (t; t0 , φ) − xi (t; t0 , ϕ)) dt + (a ij (x j (t; t0 , φ) − x j (t; t0 , ϕ)) j∈N3
+b ij (x j (t − τij (t); t0 , φ) − x j (t − τij (t); t0 , ϕ))). (4.11) Let yi (t) = xi (t; t0 , φ) − xi (t; t0 , ϕ). Then from equation 4.11, for i = 1, . . . , n; ∀t ≥ t0 , dyi (t) = −yi (t) + a ij y j (t) + b ij y j (t − τi j (t)). dt j∈N j∈N 3
(4.12)
3
4.3 and 4.4, for i ∈ N3 , a ii + |b ii | + j∈N3 , j =i (|a ij | + |b ij |) + From equations | j∈N1 (a ij + b ij ) − j∈N2 (a ij + b ij ) − ui (t)| < 1. Hence, there exists ϑ > 0 such that (1 − a ii ) −
j∈N3 , j =i
|a ij | +
|b ij | exp{ϑτ } + ϑ ≥ 0,
i ∈ N3 .
j∈N3
(4.13)
858
Z. Zeng and J. Wang
Consider a subsystem of equation 4.12: dyi (t) = −yi (t) + a ij y j (t) + b ij y j (t − τi j (t)), t ≥ t0 , i ∈ N3 . dt j∈N j∈N 3
3
(4.14)
Denote || y|| ¯ t0 = maxt0 −τ ≤s≤t0 {||y(s)||}. Then for i ∈ N3 , ∀t ≥ t0 , |yi (t)| ≤ || y|| ¯ t0 exp{−ϑ(t − t0 )}. Otherwise, one of the following two cases holds: Case i. There exist t2 > t1 ≥ t0 , k ∈ N3 , sufficiently small ε1 > 0 such that yk (t1 ) − ¯ t0 exp{−ϑ(t2 − t0 )} = ε1 , and when s ∈ || y|| ¯ t0 exp{−ϑ(t1 − t0 )} = 0, yk (t2 ) − || y|| ¯ t0 exp{−ϑ(s − t0 )} ≤ ε1 , and [t0 − τ, t2 ], for all i ∈ N3 , |yi (s)| − || y|| dyk (t) ¯ t0 exp{−ϑ(t1 − t0 )} ≥ 0, |t=t1 + ϑ|| y|| dt dyk (t) |t=t2 + ϑ|| y|| ¯ t0 exp{−ϑ(t2 − t0 )} > 0. dt
(4.15)
Case ii. There exist t4 > t3 ≥ t0 , j ∈ N3 , sufficiently small ε2 > 0 such that ¯ t0 exp{−ϑ(t3 − t0 )} = 0, yk (t4 ) + || y|| ¯ t0 exp{−ϑ(t4 − t0 )} = −ε2 , and y j (t3 ) + || y|| ¯ t0 exp{−ϑ(s − t0 )} ≥ −ε2 , and when s ∈ [t0 − τ, t4 ], for all i ∈ N3 , |yi (s)| − || y|| dy j (t) dt dy j (t) dt
|t=t3 − ϑ|| y|| ¯ t0 exp{−ϑ(t3 − t0 )} ≤ 0, |t=t4 − ϑ|| y|| ¯ t0 exp{−ϑ(t4 − t0 )} < 0.
(4.16)
It follows from equations 4.13 and 4.14 that for k ∈ N3 , dyk (t) |t=t2 = −yk (t2 ) + (a k j y j (t2 ) + b k j y j (t2 − τk j (t2 ))) dt j∈N 3
|a k j | ≤ || y|| ¯ t0 exp{−ϑ(t2 − t0 )} − 1 + a kk + +
j∈N3 , j =k
|b k j | exp{ϑτ } + ϑ − ϑ|| y|| ¯ t0 exp{−ϑ(t2 − t0 )}
j∈N3
|a k j | + |b k j | +ε1 − 1 + a kk + j∈N3 , j =k
≤ −ϑ|| y|| ¯ t0 exp{−ϑ(t2 − t0 )}.
j∈N3
Multiperiodicity and Exponential Attractivity of CNNs
859
This contradicts equation 4.15. Similarly, it follows from equations 4.13 and 4.14 that dy j (t) dt
¯ t0 exp{−ϑ(t4 − t0 )}. |t=t4 ≥ ϑ|| y||
This contradicts equation 4.16. The two contradictions show that for i ∈ N3 , ∀t ≥ t0 , ¯ t0 exp{−ϑ(t − t0 )}. |yi (t)| ≤ || y|| Hence, according to lemma 2, there exists ϑ¯ > 0 such that ∀i = 1, 2, . . . , n, ∀t ≥ t0 , ¯ − t0 )}. |xi (t; t0 , φ) − xi (t; t0 , ϕ)| ≤ ||φ, ϕ||t0 exp{−ϑ(t
(4.17)
(t)
Define xφ (θ ) = x(t + θ ; t0 , φ), θ ∈ [t0 − τ, t0 ]. From equations 4.6 to 4.9, if (t) ¯ : C([t0 − φ ∈ C([t0 − τ, t0 ], D1 ), then xφ ∈ C([t0 − τ, t0 ], D1 ). Define a mapping H (ω) ¯ ¯ τ, t0 ], D1 ) → C([t0 − τ, t0 ], D1 ) by H(φ) = xφ , then H(C([t0 − τ, t0 ], D1 )) ⊂ (mω)
¯ m (φ) = xφ . C([t0 − τ, t0 ], D1 ), and H Similar to the proof of theorem 1, there exists a periodic orbit x(t; t0 , φ ∗ ) of DCNN, equation 2.1, with period ω such that ∀t ≥ t0 , x(t; t0 , φ ∗ ) ∈ D1 and all other states of DCNN, equation 2.1, with initial condition (t, φ) (φ ∈ C([t0 − τ, t0 ], D1 )) converge to this periodic orbit exponentially as t → +∞. Hence, the isolated periodic orbit x(t; t0 , φ ∗ ) located in D1 is locally exponentially attractive, and D1 is a locally exponentially attractive set of x(t; t0 , φ ∗ ). Remark 3. From equations 4.1 to 4.4, we can see that the input vector u(t) can control the locality of a limit cycle that represents a memory pattern in a designated region. Specifically, when condition 4.1 holds, the part in corresponding coordinate of the limit cycle is located in (1, +∞); when condition 4.2 holds, the part in corresponding coordinate of the limit cycle is located (−∞, −1); when conditions 4.3 and 4.4 hold, the part in corresponding coordinate of the limit cycle is located [−1, 1]. When N3 is empty, we have the following corollary: Corollary 2. Let N1 ∪ N2 = {1, 2, . . . , n}, and N1 ∩ N2 be empty. If ∀t ≥ t0 , ui (t)
j∈N1
(a ij + b ij ) − 1,
i ∈ N1 ,
(4.18)
i ∈ N2 ,
(4.19)
j∈N2
(a ij + b ij ) −
j∈N2
(a ij + b ij ) + 1,
860
Z. Zeng and J. Wang
then DCNN, equation 2.1, has exactly one limit cycle located in D2 , and such a limit cycle is locally exponentially attractive. Proof. Let N3 in theorem 2 be an empty set. According to theorem 2, corollary 2 holds. When the periodic external input u(t) degenerates into a constant, we have the following corollary. Corollary 3. If ∀t ≥ t0 , ui (t) ≡ ui (constant), and ui < −1 +
(a ij + b ij ) −
j∈N1
ui > 1 +
ui < 1 − a ii −
j∈N2
|a ij | −
j∈N3 , j =i
ui > a ii − 1 +
(a ij + b ij ) −
j∈N2
(a ij + b ij ) −
j∈N1
j∈N3 , j =i
(|a ij | + |b ij |),
j∈N3
|b ij | +
j∈N3
|a ij | +
(|a ij | + |b ij |),
i ∈ N1 ,
j∈N3
(a ij + b ij ) +
(a ij + b ij ) −
j∈N1
|b ij | +
j∈N3
i ∈ N2 , (a ij + b ij ),
i ∈ N3 ,
(a ij + b ij ),
i ∈ N3 ,
j∈N2
(a ij + b ij ) −
j∈N1
j∈N2
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) ≡ τij (constant), then DCNN, equation 2.1, has only one equilibrium point located in D1 , which is locally exponentially stable. Proof. Since ui (t) ≡ ui (constant), for arbitrary constant ν ∈ , ui (t + ν) ≡ ui ≡ ui (t). According to theorem 2, DCNN, equation 2.1, has only one limit cycle located in D1 , which is locally exponentially attractive in D1 . The arbitrariness of constant ν implies that such a limit cycle is a fixed point. Hence, DCNN, equation 2.1, has only one equilibrium point located in D1 , which is locally exponentially stable.
5 Globally Exponentially Attractive Periodicity in a Designated Region In order to obtain optimal spatiotemporal coding in the periodic orbit and reduce computational time, it is desirable for a neural network to be globally exponentially attractive to periodic orbit in a designated region. In this section, we give some conditions that allow a periodic orbit to be globally exponentially attractive and to be located in any designated region. Theorem 3. If ∀t ≥ t0 , ui (t) < −1 −
n j=1
(|a ij | + |b ij |),
i ∈ N1 ,
(5.1)
Multiperiodicity and Exponential Attractivity of CNNs
ui (t) > 1 +
n (|a ij | + |b ij |), i ∈ N2 ,
861
(5.2)
j=1
|ui (t)| < 1 − a ii −
n j=1, j =i
|a ij | −
n
|b ij |, i ∈ N3 ,
(5.3)
j=1
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) = τij (t + ω), then DCNN, equation 2.1, has a unique limit cycle located in D1 , and such a limit cycle is globally exponentially attractive. Proof. When i ∈ N1 and xi (t) ≥ −1, from equations 2.1 and 5.1, dxi (t) ≤1+ (|a ij | + |b ij |) + ui (t) < 0. dt j=1 n
(5.4)
When i ∈ N2 and xi (t) ≤ 1, from equations 2.1 and 5.2, dxi (t) (|a ij | + |b ij |) + ui (t) > 0. ≥ −1 − dt j=1 n
(5.5)
When i ∈ N3 and xi (t) ≤ −1, from equations 2.1 and 5.3, n n dxi (t) ≥ 1 − a ii − |a ij | − |b ij | + ui (t) > 0. dt j=1, j =i j=1
(5.6)
When i ∈ N3 and xi (t) ≥ 1, from equations 2.1 and 5.3, n n dxi (t) ≤ −1 + a ii + |a ij | + |b ij | + ui (t) < 0. dt j=1, j =i j=1
(5.7)
Equations 5.4 to 5.7 imply that x(t; t0 , φ) will go into and keep in D1 , where φ ∈ C([t0 − τ, t0 ], n ). So there exists T > 0 such that ∀t ≥ T, x(t) ∈ D1 . Hence, ∀t ≥ T + τ, DCNN, equation 2.1, can be rewritten as dxi (t) = −xi (t) − (a ij + b ij ) + a ij x j (t) + b ij x j (t − τij (t)) dt j∈N j∈N j∈N +
j∈N2
1
3
(a ij + b ij ) + ui (t), i = 1, 2, . . . , n.
3
862
Z. Zeng and J. Wang
Similar to the proof of theorem 2, DCNN, equation 2.1, has a unique limit cycle located in D1 , and such a limit cycle is globally exponentially attractive. Remark 4. By comparison, we can see that if conditions 5.1 to 5.3 hold, then conditions 4.1 to 4.4 also hold. But not vice versa, as will be shown in examples 2 and 3. In other words, the conditions in theorem 3 are stronger than those in theorem 2. When N1 ∪ N2 is empty, we have the following corollary: Corollary 4. If ∀i, j ∈ {1, 2, . . . , n}, τij (t) = τij (t + ω), and ∀t ≥ t0 , |ui (t)| < 1 − a ii −
n
|a ij | −
j=1, j =i
n
|b ij |,
j=1
then the DCNN, equation 2.1, has a unique limit cycle located in [−1, 1]n , which is globally exponentially attractive. Proof. Choose N3 = {1, 2, . . . , n} in theorem 3. According to theorem 3, the corollary holds. When N3 is empty, we have the following corollary: Corollary 5. Let N3 be empty. If ui (t) < −1 −
n (|a ij | + |b ij |),
i ∈ N1 ,
(5.8)
i ∈ N2 ,
(5.9)
j=1
ui (t) > 1 +
n (|a ij | + |b ij |), j=1
then DCNN, equation 2.1, has a unique limit cycle located in D2 . Moreover, such a limit cycle is globally exponentially attractive. Proof. Since N3 is empty, according to theorem 3, corollary 5 holds. Remark 5. Since −1 − nj=1 (|a ij | + |b ij |) ≤ −1 + j∈N1 (a ij + b ij ) − j∈N2 (a ij + b ij ), if condition 5.8 holds, then condition 4.18 also holds, but not vice versa. Similarly, if condition 5.9 holds, then condition 4.19 also holds, but not vice versa. This implies that the conditions in corollary 5 are stronger than those in corollary 2. In addition, corollary 5 shows that a DCNN has a globally exponentially attractive limit cycle if its periodic external stimulus is sufficiently strong.
Multiperiodicity and Exponential Attractivity of CNNs
863
When the periodic external input u(t) degenerates into a constant vector, we have the following corollary: Corollary 6. If ∀t ≥ t0 , ui (t) ≡ ui (constant), and
ui < −1 −
n (|a ij | + |b ij |),
i ∈ N1 ,
j=1
ui > 1 +
n (|a ij | + |b ij |),
i ∈ N2 ,
j=1
|ui | < 1 − a ii −
n
|a ij | −
j=1, j =i
n
|b ij |,
i ∈ N3 ,
j=1
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) ≡ τij (constant), then DCNN, equation 2.1, has a unique equilibrium point located in D1 and is globally exponentially stable at such an equilibrium point. Proof. Since ui (t) ≡ ui (constant), for an arbitrary constant ν ∈ , ui (t + ν) ≡ ui (t) ≡ ui . According to theorem 3, DCNN, equation 2.1, has a unique limit cycle located in D1 , which is globally exponentially attractive. The arbitrariness of constant ν implies that such a limit cycle is a fixed point. Hence, DCNN, equation 2.1, has a unique equilibrium point located in D1 , and such an equilibrium point is globally exponentially stable.
6 Illustrative Examples In this section, we give three numerical examples to illustrate the new results.
6.1 Example 1. Consider a CNN, where
2
0.2
0.2
0.4 0.6
2.4
A = 0.2 2.4
0.5 sin(t)
0.2 u(t) = −0.6 cos(t)
.
−0.2(sin(t) + cos(t))
According to theorem 1, this CNN has 23 = 8 limit cycles, which are locally exponentially attractive. Simulation results with 136 random initial states are depicted in Figures 1 to 3.
864
Z. Zeng and J. Wang 4
x
1
2 0
−2 −4
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
4
x
2
2 0
−2 −4 4
x
3
2 0
−2 −4
Figure 1: Transient behavior of x1 , x2 , x3 in Example 1.
4
3
2
x3
1
0
−1
−2
−3
−4 −4
−3
−2
−1
0 x
1
1
Figure 2: Transient behavior of (x1 , x3 ) in Example 1.
2
3
4
Multiperiodicity and Exponential Attractivity of CNNs
865
4 3 2
x3
1 0
−1 −2 −3 −4 4 4
2 2
0 0
−2
−2 −4
x2
−4
x1
Figure 3: Transient behavior of (x1 , x2 , x3 ) in Example 1.
6.2 Example 2. Consider a CNN, where
2
A = 0.5
0.2 0.4
−1.4 0.4
0.2
0.5 sin(t)
0.5 u(t) = 0.5 cos(t) 0.8
.
0.5(sin(t) + cos(t))
Since u1 (t) < −1 + a 11 − a 13 − Choose N1 = {1}, N2 = {3}, N3 = {2}. |a 12 |; a 22 + |a 21 − a 23 − u2 (t)| < 1; u3 (t) > 1 + a 31 − a 33 + |a 32 |, according to theorem 2, this CNN has a limit cycle located in D1 = {x| x1 < −1, |x2 | ≤ 1, x3 > 1}, which is locally exponentially attractive in D1 . Choose N1 = {3}, N2 = {1}, N3 = {2}. Since u1 (t) > 1 + a 13 − a 11 + |a 12 |, a 22 + |a 21 − a 23 − u2 (t)| < 1, u3 (t) < −1 + a 33 − a 31 − |a 32 |, according to theorem 2, this CNN has a limit cycle located in D1 = {x| x3 < −1, |x2 | ≤ 1, x1 > 1}, which is locally exponentially attractive in D1 . However, since 0.5 sin(t) > 1 + (2 + 0.2 + 0.2) = 3.4 does not hold (i.e., condition 5.1 does not hold), it does not satisfy the conditions in theorem 3. Simulation results with 136 random initial states are depicted in Figures 4 and 5.
866
Z. Zeng and J. Wang 4
x
1
2 0
−2 −4 0
5
10
15 time
20
25
30
5
10
15 time
20
25
30
5
10
15 time
20
25
30
4
x
2
2 0
−2 −4 0 4
x
3
2 0
−2 −4 0
Figure 4: Transient behavior of x1 , x2 , x3 in Example 2.
4 3 2
x3
1 0
−1 −2 −3 −4 4 4
2 2
0 0
−2 x2
−2 −4
−4
x1
Figure 5: Transient behavior of (x1 , x2 , x3 ) in Example 2.
Multiperiodicity and Exponential Attractivity of CNNs
867
x1
5
0
−5
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
x2
5
0
−5
x3
5
0
−5
Figure 6: Transient behavior of x1 , x2 , x3 in Example 3.
6.3 Example 3. Consider a CNN, where
0.2 0.2
0.2
0.2 0.2
0.2
A = 0.2 −2
0.8 sin(t) − 2.6
0.6 u(t) = 0.8 cos(t)
.
0.8(sin(t) + cos(t)) + 2.8
Since u1 (t) < −1 − 3j=1 |a 1 j |, Choose N1 = {1}, N2 = {3}, N3 = {2}. |u2 (t)| < 1 − a 22 − 3j=1, j =2 |a 2 j |, u3 (t) > 1 + 3j=1 |a 3 j |, according to theorem 3, this CNN has a limit cycle located in D1 = {x| x1 < −1, |x2 | ≤ 1, x3 > 1}, which is globally exponentially attractive. Since u1 (t) < −1 + a 11 − |a 12 | − a 13 ; u2 (t) < 1 − a 22 + a 21 − a 23 , u2 (t) > −1 + a 22 + a 21 − a 23 ; u3 (t) > 1 + a 31 + |a 32 | − a 33 , conditions 4.1 to 4.4 also hold. According to theorem 2, this CNN has a limit cycle located in D1 , which is also locally exponentially attractive. However, since a 11 > 0, a 33 > 0 the conditions in Yi et al. (2003) cannot be used to ascertain the complete stability of this CNN. Simulation results with 136 random initial states are depicted in Figures 6 and 7.
868
Z. Zeng and J. Wang
5 4 3
x3
2 1 0 −1 −2 −3 6 4
6 2
4 2
0 0
−2 x2
−4
−2 −4
x1
Figure 7: Transient behavior of (x1 , x2 , x3 ) in Example 3.
7 Concluding Remarks Rhythmicity represents one of most striking manifestations of dynamic behaviors in biological systems. CNNs and DCNNs, which have been shown to be capable of operating in a pacemaker or pattern generator mode, are studied here as oscillatory mechanisms in response to periodic external stimuli. Some information can be encoded in the oscillating activation states relative to external inputs, and these relative phases change as a function of the chosen limit cycle. In this article, we show that the number of locally exponentially attractive periodic orbits located in saturation regions in a DCNN is exponential of the number of the neurons. In view of the fact that neural information is often desired to be encoded in a designated region, we also give conditions to allow a globally exponentially attractive periodic orbit located in any designated region. The theoretical results are supplemented by simulation results in three illustrative examples.
Acknowledgments This work was supported by the Hong Kong Research Grants Council under grant CUHK4165/03E, the Natural Science Foundation of China
Multiperiodicity and Exponential Attractivity of CNNs
869
under grant 60405002 and China Postdoctoral Science Foundation under Grant 2004035579.
References Belair, J., Campbell, S., & Driessche, P. (1996). Frustration, stability and delay-induced oscillation in a neural network model. SIAM J. Applied Math., 56, 245–255. Berns, D. W., Moiola, J. L., & Chen, G. (1998). Predicting period-doubling bifurcations and multiple oscillations in nonlinear time-delayed feedback systems. IEEE Trans. Circuits Syst. I, 45(7), 759–763. Chen, K., Wang, D. L., & Liu, X. (2000). Weight adaptation and oscillatory correlation for image segmentation. IEEE Trans. Neural Networks, 11, 1106–1123. Chua, L. O., & Roska, T. (1990). Stability of a class of nonreciprocal cellular neural networks. IEEE Trans. Circuits Syst., 37, 1520–1527. Chua, L. O., & Roska, T. (1992). Cellular neural networks with nonlinear and delaytype template elements and non-uniform grids. Int. J. Circuit Theory and Applicat., 20, 449–451. Civalleri, P. P., Gilli, M., & Pandolfi, L. (1993). On stability of cellular neural networks with delay. IEEE Trans. Circuits Syst. I, 40, 157–164. Gopalsamy, K., & Leung, I. (1996). Delay induced periodicity in a neural netlet of excitation and inhibition. Physica D, 89, 395–426. Jiang, H., & Teng, Z. (2004). Boundedness and stability for nonautonomous bidirectional associative neural networks with delay. IEEE Trans. Circuits Syst. II, 51(4), 174–180. Jin, H. L., & Zacksenhouse, M. (2003). Oscillatory neural networks for robotic yo-yo control. IEEE Trans. Neural Networks, 14(2), 317–325. Kanamaru, T., & Sekine, M. (2004). An analysis of globally connected active rotators with excitatory and inhibitory connections having different time constants using the nonlinear Fokker-Planck equations. IEEE Trans. Neural Networks, 15(5), 1009– 1017. Kosaku, Y. (1978). Functional analysis. Berlin: Springer-Verlag. Liao, X. X., & Wang, J. (2003). Algebraic criteria for global exponential stability of cellular neural networks with multiple time delays. IEEE Trans. Circuits and Syst. I, 50(2), 268–275. Liao, X., Wu, Z., & Yu, J. (1999). Stability switches and bifurcation analysis of a neural network with continuous delay. IEEE Trans. Systems, Man and Cybernetics, Part A, 29(6), 692–696. Liu, Z., Chen, A., Cao, J., & Huang, L. (2003). Existence and global exponential stability of periodic solution for BAM neural networks with periodic coefficients and time-varying delays. IEEE Trans. Circuits Syst. I, 50(9), 1162–1173. Nishikawa, T., Lai, Y. C., & Hoppensteadt, F. C. (2004). Capacity of oscillatory associative-memory networks with error-free retrieval. Physical Review Letters, 92(10), 108101. Roska, T., Wu, C. W., Balsi, M., & Chua, L. O. (1992). Stability and dynamics of delay-type general and cellular neural networks. IEEE Trans. Circuits Syst. I, 39, 487–490.
870
Z. Zeng and J. Wang
Roska, T., Wu, C. W., & Chua, L. O. (1993). Stability of cellular neural networks with dominant nonlinear and delay-type templates. IEEE Trans. Circuits Syst. I, 40, 270–272. Ruiz, A., Owens, D. H., & Townley, S. (1998). Existence, learning, and replication of periodic motion in recurrent neural networks. IEEE Trans. Neural Networks, 9(4), 651–661. Setti, G., Thiran, P., & Serpico, C. (1998). An approach to information propagation in 1-D cellular neural networks, part II: Global propagation. IEEE Trans. Circuits and Syst. I, 45(8), 790–811. Takahashi, N. (2000). A new sufficient condition for complete stability of cellular neural networks with delay. IEEE Trans. Circuits Syst. I, 47, 793–799. Townley, S., Ilchmann, A., Weiss, M. G., McClements, W., Ruiz, A. C., Owens, D. H., & Pratzel-Wolters, D. (2000). Existence and learning of oscillations in recurrent neural networks. IEEE Trans. Neural Networks, 11, 205–214. Wang, D. L. (1995). Emergent synchrony in locally coupled neural oscillators. IEEE Trans. Neural Networks, 6, 941–948. Wang, L., & Zou, X. (2004). Capacity of stable periodic solutions in discrete-time bidirectional associative memory neural networks. IEEE Trans. Circuits Syst. II, 51(6), 315–319. Yi, Z., Heng, P. A., & Vadakkepat, P. (2002). Absolute periodicity and absolute stability of delayed neural networks. IEEE Trans. Circuits Syst. I, 49(2), 256–261. Yi, Z., Tan, K. K., & Lee, T. H. (2003). Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 15(3), 639–662. Zeng, Z., Wang, J., & Liao, X. X. (2003). Global exponential stability of a general class of recurrent neural networks with time-varying delays. IEEE Trans. Circuits and Syst. I, 50(10), 1353–1358. Zeng, Z., Wang, J., & Liao, X. X. (2004). Stability analysis of delayed cellular neural networks described using cloning templates. IEEE Trans. Circuits and Syst. I, 51(11), 2313–2324.
Received November 30, 2004; accepted June 28, 2005.
Communicated by Richard Hahnloser
Multiperiodicity and Exponential Attractivity Evoked by Periodic External Inputs in Delayed Cellular Neural Networks Zhigang Zeng [email protected] School of Automation, Wuhan University of Technology, Wuhan, Hubei, 430070, China
Jun Wang [email protected] Department of Automation and Computer-Aided Engineering, Chinese University of Hong Kong, Shatin, New Territories, Hong Kong
We show that an n-neuron cellular neural network with time-varying delay can have 2n periodic orbits located in saturation regions and these periodic orbits are locally exponentially attractive. In addition, we give some conditions for ascertaining periodic orbits to be locally or globally exponentially attractive and allow them to locate in any designated region. As a special case of exponential periodicity, exponential stability of delayed cellular neural networks is also characterized. These conditions improve and extend the existing results in the literature. To illustrate and compare the results, simulation results are discussed in three numerical examples. 1 Introduction Cellular neural networks (CNNs) and delayed cellular neural networks (DCNNs) are arrays of dynamical cells that are suitable for solving many complex computational problems. In recent years, both have been extensively studied and successfully applied for signal processing and solving nonlinear algebraic equations. As dynamic systems with a special structure, CNNs and DCNNs have many interesting properties that deserve theoretical studies. In general, there are two interesting nonlinear neurodynamic properties in CNNs and DCNNs: stability and periodic oscillations. The stability of a CNN or a DCNN at an equilibrium point means that for a given activation function and a constant input vector, an equilibrium of the network exists and any state in the neighborhood converges to the equilibrium. The stability of neuron activation states at an equilibrium is prerequisite for most applications. Some neurodynamics have multiple (two) stable equilibria and may be stable at any equilibrium depending on the initial state, which is called Neural Computation 18, 848–870 (2006)
C 2006 Massachusetts Institute of Technology
Multiperiodicity and Exponential Attractivity of CNNs
849
multistability (bistability). For stability, either an equilibrium or a set of equilibria is the attractor. Besides stability, an activation state may be periodically oscillatory around an orbit. In this case, the attractor is a limit set. Periodic oscillation in recurrent neural networks is an interesting dynamic behavior, as many biological and cognitive activities (e.g., heartbeat, respiration, mastication, locomotion, and memorization) require repetition. Persistent oscillation, such as limit cycles, represents a common feature of neural firing patterns produced by dynamic interplay between cellular and synaptic mechanisms. Stimulus-evoked oscillatory synchronization was observed in many biological neural systems, including the cerebral cortex of mammals and the brain of insects. It was also known that time delays can cause oscillations in neurodynamics (Gopalsamy & Leung, 1996; Belair, Campbell, & Driessche, 1996). In addition, periodic oscillations in recurrent neural networks have found many applications, such as associative memories (Nishikawa, Lai, & Hoppensteadt, 2004), pattern recognition (Wang, 1995; Chen, Wang, & Liu, 2000), machine learning (Ruiz, Owens, & Townley, 1998; Townley et al., 2000), and robot motion control (Jin & Zacksenhouse, 2003). The analysis of periodic oscillation of neural networks is more general than stability analysis since an equilibrium point can be viewed as a special case of oscillation with any arbitrary period. The stability of CNNs and DCNNs has been widely investigated (e.g., Chua & Roska, 1990, 1992; Civalleri, Gilli, & Pandolfi, 1993; Liao, Wu, & Yu, 1999; Roska, Wu, Balsi, & Chua, 1992; Roska, Wu, & Chua, 1993; Setti, Thiran, & Serpico, 1998; Takahashi, 2000; Zeng, Wang, & Liao, 2003). The existence of periodic orbits together with global exponential stability of CNNs and DCNNs is studied in Yi, Heng, and Vadakkepat (2002) and Wang and Zou (2004). Most existing studies (Berns, Moiola, & Chen, 1998; Jiang & Teng, 2004; Kanamaru & Sekeine, 2004; Liao & Wang, 2003; Liu, Chen, Cao, & Huang, 2003; Wang & Zou, 2004) are based on the assumption that the equilibrium point of CNNs or DCNNs is globally stable or the periodic orbit of CNNs or DCNNs is globally attractive; hence, CNNs or DCNNs have only one equilibrium point or one periodic orbit. However, in most applications, it is required that CNNs or DCNNs exhibit more than one stable equilibrium point (e.g., Yi, Tan, & Lee, 2003; Zeng, Wang, & Liao, 2004), or more than one exponentially attractive periodic orbit instead of a single globally stable equilibrium point. In this letter, we investigate the multiperiodicity and multistablity of DCNNs. We show that an n-neuron DCNN can have 2n periodic orbits that are locally exponentially attractive. Moreover, we present the estimates of attractive domain of such 2n locally exponentially attractive periodic orbits. In addition, we give the conditions for periodic orbits to be locally or globally exponentially attractive when the periodic orbits locate in a designated position. All of these conditions are very easy to be verified.
850
Z. Zeng and J. Wang
The remaining part of this letter consists of six sections. In section 2, relevant background information is given. The main results are stated in sections 3, 4, and 5. In section 6, three illustrative examples are provided with simulation results. Finally, concluding remarks are given in section 7. 2 Preliminaries Consider the DCNN model governed by the following normalized dynamic equations: dxi (t) a ij f (x j (t)) = − xi (t) + dt n
j=1
+
n
b ij f (x j (t − τij (t))) + ui (t), i = 1, . . . , n,
(2.1)
j=1
where x = (x1 , . . . , xn )T ∈ n is the state vector, A = (a ij ) and B = (b ij ) are connection weight matrices that are not assumed to be symmetric, u(t) = (u1 (t), . . . , un (t))T ∈ n is a periodic input vector with period ω (i.e., there exists a constant ω > 0 such that ui (t + ω) = ui (t) ∀t ≥ 0, ∀i ∈ {1, 2, . . . , n}), τij (t) is the time-varying delay that satisfies 0 ≤ τij (t) ≤ τ (τ is constant), and f (·) is the piecewise linear activation function defined by f (v) = (|v + 1| − |v − 1|)/2. In particular, when b ij ≡ 0 (∀i, j = 1, 2, . . . , n), the DCNN degenerates as a CNN. Let C([t0 − τ, t0 ], D) be the space of continuous functions mapping [t0 − τ, t0 ] into D ⊂ n with the norm defined by ||φ||t0 = max1≤i≤n {supu∈[t0 −τ,t0 ] |φi (u)|}, where φ(s) = (φ1 (s), φ2 (s), . . . , φn (s))T . Denote ||x|| = max1≤i≤n {|xi |} as the vector norm of the vector x = (x1 , . . . , xn )T . ∀φ, ϕ ∈ C([t0 − τ, t0 ], D), where φ(s) = (φ1 (s), φ2 (s), . . . , φn (s))T , ϕ(s) = (ϕ1 (s), ϕ2 (s), . . . , ϕn (s))T . Denote ||φ, ϕ||t0 = max1≤i≤n {supt0 −τ ≤s≤t0 {|φi (s) − ϕi (s)|}} as a measurement in C([t0 − τ, t0 ], D). The initial condition of DCNN model 2.1 is assumed to be φ(ϑ) = (φ1 (ϑ), φ2 (ϑ), . . . , φn (ϑ))T , where φ(ϑ) ∈ C([t0 − τ, t0 ], n ). Denote x(t; t0 , φ) as the state of DCNN model 2.1 with initial condition (t0 , φ). It means that x(t; t0 , φ) is continuous and satisfies equation 2.1 and x(s; t0 , φ) = φ(s), for s ∈ [t0 − τ, t0 ]. Denote (−∞, −1) = (−∞,−1)1 ×[−1,1]0 ×(1,+∞)0 ;[−1,1] = (−∞,−1)0 × [−1, 1]1 × (1, +∞)0 ; (1, +∞)= (−∞, −1)0 × [−1, 1]0 × (1, +∞)1 , = (−∞, +∞) = (−∞, −1) [−1, 1] (1, +∞), so (−∞, +∞)n can be divided into 3n subspaces: (i)
(i)
(i)
n (−∞, −1)δ1 × [−1, 1]δ2 × (1, +∞)δ3 , = {i=1 (i) (i) (i) δ1 , δ2 , δ3 = (1, 0, 0) or (0, 1, 0) or (0, 0, 1), i = 1, . . . , n};
(2.2)
Multiperiodicity and Exponential Attractivity of CNNs
851
and can be divided into three subspaces: 1 = {[−1, 1]n } n 2 = {i=1 (−∞, −1)δ × (1, +∞)1−δ , δ (i) = 1 or 0, i = 1, . . . , n} (i)
(i)
3 = − 1 − 2 Hence, 1 is composed of one region, 2 is composed of 2n regions, and 3 is composed of 3n − 2n − 1 regions. Definition 1. A periodic orbit x ∗ (t) is said to be a limit cycle of a DCNN if x ∗ (t) is an isolated periodic orbit of the DCNN; that is, there exists ω > 0 such that ∀t ≥ t0 , x ∗ (t + ω) = x ∗ (t), and there exists δ > 0 such that ∀ x ∈ {x| 0 < ||x, x ∗ (t)|| < δ, x ∈ n , t ≥ t0 }, where x is not a point on any periodic orbit of the DCNN. Definition 2. A periodic orbit x ∗ (t) of a DCNN is said to be locally exponentially attractive in region if there exist constants α > 0, β > 0 such that ∀t ≥ t0 , x(t; t0 , φ) − x ∗ (t) ≤ β||φ||t exp{−α(t − t0 )}, 0 where x(t; t0 , φ) is the state of the DCNN with any initial condition (t0 , φ), φ(ϑ) ∈ C([t0 − τ, t0 ], ), and is said to be a locally exponentially attractive set of the periodic orbit x ∗ (t). When = n , x ∗ (t) is said to be globally exponentially attractive. In particular, if x ∗ (t) is a fixed point x ∗ , then the DCNN is said to be global exponentially stable. Lemma 1. (Kosaku, 1978). Let D be a compact set in n , H be a mapping on complete metric space (C([t0 − τ, t0 ], D), ||·, ·||t0 ). If H(C([t0 − τ, t0 ], D)) ⊂ C([t0 − τ, t0 ], D), and there exists a constant α < 1 such that ∀φ, ϕ ∈ C([t0 − τ, t0 ], D), ||H(φ), H(ϕ)||t0 ≤ α||φ, ϕ||t0 , then there exists φ ∗ ∈ C([t0 − τ, t0 ], D) such that H(φ ∗ ) = φ ∗ . Consider the following coupled system: dx(t) = −x(t) + Ay(t) + By(t − τ (t)), dt dy(t) = h(t, y(t), y(t − τ (t))), dt
(2.3) (2.4)
where x(t) ∈ n , y(t) ∈ m , A and B are n × m matrices, h ∈ C( × m × m , m ), and C( × m × m , m ) is the space of continuous functions mapping × m × m into m .
852
Z. Zeng and J. Wang
Lemma 2. If system 2.4 is globally exponentially stable, then system 2.3 is also globally exponentially stable. Proof. By applying the variant format of constants, the solution x(t) of equation 2.3 can be expressed as x(t) = exp{−(t − t0 )}x(t0 ) +
t
exp{−(t − s)}(Ay(s) + By(s − τ (s)))ds.
t0
Since equation 2.4 is globally exponentially stable, there exist constants α, β > 0 such that |y(s)| ≤ β exp{−α(s − t0 )}. Hence, |Ay(s) + By(s − τ (s))| ≤ β¯ exp{−α(s − t0 )}, where β¯ = (||A|| + ||B|| exp{ατ })β. Then ¯ − t0 ) exp{−(t − t0 )}; when α = 1, ∀t ≥ t0 , |x(t)| ≤ |x(t0 )| exp{−(t − t0 )} + β(t ¯ when α = 1, |x(t)| ≤ |x(t0 )| exp{−(t − t0 )} + β(exp{−(t − t0 )} + exp{−α(t − t0 )})/|1 − α|; that is, equation 2.3 is also globally exponentially stable. Throughout this article, we assume that N = {1, 2, . . . , n}, N N 1 2 3 N1 N2 , N1 N3 , and N2 N3 are empty. Denote D1 = {x ∈ n | xi ∈ (−∞, −1), i ∈ N1 ; xi ∈ (1, ∞), i ∈ N2 ; xi ∈ [−1, 1], i ∈ N3 }. Note that D1 ⊂ , where is defined in equation 2.2. If N3 is empty, then denote D2 = {x ∈ n |xi ∈ (−∞, −1), i ∈ N1 ; xi ∈ (1, ∞), i ∈ N2 }. 3 Locally Exponentially Attractive Multiperiodicity in a Saturation Region In this section, we show that an n-neuron delayed cellular neural network can have 2n periodic orbits located in saturation regions and these periodic orbits are locally exponentially attractive. Theorem 1. If ∀i ∈ {1, 2, . . . , n}, ∀t ≥ t0 , |ui (t)| < a ii − 1 −
n j=1, j =i
|a ij | −
n
|b ij |,
(3.1)
j=1
then DCNN (see equation 2.1) has 2n locally exponentially attractive limit cycles. Proof. If ∀s ∈ [t0 − τ, t], x(s) ∈ D2 , from equation 2.1, ∀i = 1, 2, . . . , n, dxi (t) = −xi (t) + (a ij + b ij ) − (a ij + b ij ) + ui (t). dt j∈N j∈N 1
2
(3.2)
Multiperiodicity and Exponential Attractivity of CNNs
853
When i ∈ N2 and xi (t) = 1, from equations 3.1 and 3.2, dxi (t) = −1 + (a ij + b ij ) − (a ij + b ij ) + ui (t) > 0. dt j∈N j∈N 1
(3.3)
2
When i ∈ N1 and xi (t) = −1, from equations 3.1 and 3.2, dxi (t) (a ij + b ij ) − (a ij + b ij ) + ui (t) < 0. =1+ dt j∈N j∈N 1
(3.4)
2
Equations 3.3 and 3.4 imply that if ∀φ ∈ C([t0 − τ, t0 ], D2 ), then x(t; t0 , φ) will keep in D2 , and D2 is an invariant set of DCNN (see equation 2.1). So ∀t ≥ t0 − τ, x(t) ∈ D2 . Hence, DCNN, equation 2.1, can be rewritten as equation 3.2. Let x(t; t0 , φ) and x(t; t0 , ϕ) be two states of DCNN, equation 2.1, with initial conditions (t0 , φ) and (t0 , ϕ), where φ, ϕ ∈ C([t0 − τ, t0 ], D2 ). From equations 2.1 and 3.2, ∀i ∈ {1, 2, . . . , n}, ∀t ≥ t0 , d(xi (t; t0 , φ) − xi (t; t0 , ϕ)) = −(xi (t; t0 , φ) − xi (t; t0 , ϕ)). dt
(3.5)
Hence, ∀i = 1, 2, . . . , n, ∀t ≥ t0 , |xi (t; t0 , φ) − xi (t; t0 , ϕ)| ≤ ||φ, ϕ||t0 exp{−(t − t0 )}.
(3.6)
(t)
Define xφ (θ ) = x(t + θ ; t0 , φ), θ ∈ [t0 − τ, t0 ]. Then from equations 3.3 (t)
and 3.4, ∀φ ∈ C([t0 − τ, t0 ], D2 ), xφ ∈ C([t0 − τ, t0 ], D2 ). Define a mapping (ω)
H : C([t0 − τ, t0 ], D2 ) → C([t0 − τ, t0 ], D2 ) by H(φ) = xφ . Then H(C([t0 − τ, t0 ], D2 )) ⊂ C([t0 − τ, t0 ], D2 ), (mω)
and H m (φ) = xφ . We can choose a positive integer m such that exp{−(mω − τ )} ≤ α < 1. Hence, from equation 3.6, ||H m (φ), H m (ϕ)||t0 ≤ max
1≤i≤n
sup θ∈[t0 −τ,t0 ]
|xi (mω + θ; t0 , φ) − xi (mω + θ; t0 , ϕ)|
≤ ||φ, ϕ||t0 exp{−(mω + t0 − τ − t0 )} ≤ α||φ, ϕ||t0 . Based on lemma 1, there exists a unique fixed point φ ∗ ∈ C([t0 − τ, t0 ], D2 ) such that H m (φ ∗ ) = φ ∗ . In addition, H m (H(φ ∗ )) = H(H m (φ ∗ )) = H(φ ∗ ). This shows that H(φ ∗ ) is also a fixed point of H m . Hence, by the uniqueness of the fixed
854
Z. Zeng and J. Wang (ω)
point of the mapping H m , H(φ ∗ ) = φ ∗ ; that is, xφ ∗ = φ ∗ . Let x(t; t0 , φ ∗ ) be a state of DCNN, equation 2.1, with initial condition (t0 , φ ∗ ). Then from equation 2.1, ∀i = 1, 2, . . . , n, ∀t ≥ t0 , dxi (t; t0 , φ ∗ ) = −xi (t; t0 , φ ∗ ) + (a ij + b ij ) − (a ij + b ij ) + ui (t). dt j∈N j∈N 1
2
Hence, ∀i = 1, 2, . . . , n, ∀t + ω ≥ t0 , dxi (t + ω; t0 , φ ∗ ) = −xi (t + ω; t0 , φ ∗ ) + (a ij + b ij ) dt j∈N −
1
(a ij + b ij ) + ui (t + ω)
j∈N2
= −xi (t + ω; t0 , φ ∗ ) + −
(a ij + b ij )
j∈N1
(a ij + b ij ) + ui (t).
j∈N2
This implies x(t + ω; t0 , φ ∗ ) is also a state of DCNN, equation 2.1, with initial (ω) condition (t0 , φ ∗ ). xφ ∗ = φ ∗ implies that ∀t ≥ t0 , x(t + ω; t0 , φ ∗ ) = x(t; t0 , xφ ∗ ) = x(t; t0 , φ ∗ ). (ω)
Hence, x(t; t0 , φ ∗ ) is a periodic orbit of DCNN, equation 2.1, with period ω. From equation 3.5, it is easy to see that any state of DCNN, equation 2.1, with initial condition (t, φ) (φ ∈ C([t0 − τ, t0 ], D2 )) converges to this periodic orbit exponentially as t → +∞. Hence, the isolated periodic orbit x(t; t0 , φ ∗ ) located in D2 is locally exponentially attractive, and D2 is a locally exponentially attractive set of x(t; t0 , φ ∗ ). Since there exist 2n elements in 2 , there exist 2n isolated periodic orbits in 2 . And such 2n isolated periodic orbits are locally exponentially attractive. When the periodic external input u(t) degenerates into a constant vector, we have the following corollary: Corollary 1. If ∀i ∈ {1, 2, . . . , n}, ∀t ≥ t0 , ui (t) ≡ ui (constant), and
|ui | < a ii − 1 −
n j=1, j =i
|a ij | −
n
|b ij |,
j=1
then DCNN (see equation 2.1) has 2n locally exponentially stable equilibrium points.
Multiperiodicity and Exponential Attractivity of CNNs
855
Proof. Since ui (t) ≡ ui (constant), for an arbitrary constant ν ∈ , ui (t + ν) ≡ ui ≡ ui (t). According to theorem 1, DCNN, equation 2.1, has 2n locally exponentially attractive limit cycles with period ν. The arbitrariness of constant ν implies that such limit cycles are fixed points. Hence, DCNN, equation 2.1, has 2n locally exponentially attractive equilibrium points. Remark 1. In theorem 1 and corollary 1, it is necessary for a ii to be dominant such that a ii > 1 + nj=1, j =i |a ij | + nj=1 |b ij |. Remark 2. A main objective for designing associative memories is to store a large number of patterns as stable equilibria or limit cycles such that stored patterns can be retrieved when the initial probes contain sufficient information about the patterns. CNNs and DCNNs are also suitable for very largescale integration (VLSI) implementations of associative memories. It is also expected that they can be applied to association memories by storing patterns as periodic limit cycles. According to theorem 1 and corollary 1, the n-neuron DCNN model, equation 2.1, can store up to 2n patterns in locally exponentially attractive limit cycles or equilibria, which can be retrieved when the input vector satisfies condition 3.1. This implies that the external stimuli also play a major role in encoding and decoding patterns in DCNN associative memories, in contrast with the zero input vector in the bidirectional associate memories and the autoassociative memories based on the Hopfield network.
4 Locally Exponentially Attractive Periodicity in a Designated Region As the limit cycles are stimulus driven (nonautonomous), some information can be encoded in the phases of the oscillating states xi relative to the inputs ui . Hence, it is necessary to find some conditions on the inputs ui , when the periodic orbit x(t) is desired to be located in a designated region. In this section, we give the conditions that allow a periodic orbit to be locally exponentially attractive and located in any designated region.
Theorem 2. If ∀t ≥ t0 , ui (t) < −1 +
(a ij + b ij ) −
j∈N1
ui (t) > 1 +
j∈N1
(a ij + b ij ) −
(a ij + b ij ) −
j∈N2
j∈N2
(a ij + b ij ) +
(|a ij | + |b ij |),
i ∈ N1 ,
j∈N3
(4.1) (|a ij | + |b ij |),
i ∈ N2 ,
j∈N3
(4.2)
856
Z. Zeng and J. Wang ui (t) < 1 − a ii − −
(a ij + b ij ),
ui (t) > a ii − 1 +
|a ij | −
j∈N3 , j =i
j∈N2
−
|b ij | +
j∈N3
(a ij + b ij )
j∈N1
i ∈ N3 , |a ij | +
j∈N3 , j =i
(a ij + b ij ),
j∈N3
(4.3) |b ij | +
(a ij + b ij )
j∈N1
i ∈ N3 ,
(4.4)
j∈N2
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) = τij (t + ω), then DCNN, equation 2.1, has only one limit cycle located in D1 , which is locally exponentially attractive in D1 . Proof. If ∀s ∈ [t0 − τ, t], x(s) ∈ D1 , then from equation 2.1, ∀s ∈ [t0 , t], ∀i = 1, 2, . . . , n, dxi (s) = −xi (s) − (a ij + b ij ) + (a ij + b ij ) + a ij x j (s) dt j∈N1 j∈N2 j∈N3 b ij x j (s − τij (s)) + ui (s). +
(4.5)
j∈N3
When i ∈ N1 and xi (t) = −1, from equations 4.1 and 4.5, dxi (t) (a ij + b ij ) + (a ij + b ij ) ≤1 − dt j∈N1 j∈N2 + (|a ij | + |b ij |) + ui (t) < 0.
(4.6)
j∈N3
When i ∈ N2 and xi (t) = 1, from equations 4.2 and 4.5, dxi (t) ≥ −1 − (a ij + b ij ) − (|a ij | + |b ij |) dt j∈N1 j∈N3 (a ij + b ij ) + ui (t) > 0. +
(4.7)
j∈N2
When i ∈ N3 and xi (t) = 1, from equations 4.3 and 4.5, dxi (t) ≤ −1 − (a ij + b ij ) + (a ij + b ij ) + a ii dt j∈N1 j∈N2 + |a ij | + |b ij | + ui (t) < 0. j∈N3 , j =i
j∈N3
(4.8)
Multiperiodicity and Exponential Attractivity of CNNs
857
When i ∈ N3 and xi (t) = −1, from equations 4.4 and 4.5, dxi (t) (a ij + b ij ) + (a ij + b ij ) − a ii ≥ 1− dt j∈N1 j∈N2 − |a ij | − |b ij | + ui (t) > 0. j∈N3 , j =i
(4.9)
j∈N3
Equations 4.6 to 4.9 imply that if ∀s ∈ [t0 − τ, t0 ], φ(s) ∈ D1 , then x(t; t0 , φ) will keep in D1 , and D1 is an invariant set of DCNN (see equation 2.1). So ∀t ≥ t0 − τ, x(t) ∈ D1 . Hence, ∀t ≥ t0 , DCNN, equation 2.1, can be rewritten as dxi (t) = −xi (t) − (a ij + b ij ) + a ij x j (t) + b ij x j (t − τij (t)) dt j∈N1 j∈N3 j∈N3 (a ij + b ij ) + ui (t), i = 1, 2, . . . , n. +
(4.10)
j∈N2
Let x(t; t0 , φ) and x(t; t0 , ϕ) be two states of DCNN (equation 4.10) with initial conditions (t0 , φ) and (t0 , ϕ), where φ, ϕ ∈ C([t0 − τ, t0 ], D1 ). From equation 4.10, ∀i = 1, 2, . . . , n; ∀t ≥ t0 , d(xi (t; t0 , φ) − xi (t; t0 , ϕ)) = −(xi (t; t0 , φ) − xi (t; t0 , ϕ)) dt + (a ij (x j (t; t0 , φ) − x j (t; t0 , ϕ)) j∈N3
+b ij (x j (t − τij (t); t0 , φ) − x j (t − τij (t); t0 , ϕ))). (4.11) Let yi (t) = xi (t; t0 , φ) − xi (t; t0 , ϕ). Then from equation 4.11, for i = 1, . . . , n; ∀t ≥ t0 , dyi (t) = −yi (t) + a ij y j (t) + b ij y j (t − τi j (t)). dt j∈N j∈N 3
(4.12)
3
4.3 and 4.4, for i ∈ N3 , a ii + |b ii | + j∈N3 , j =i (|a ij | + |b ij |) + From equations | j∈N1 (a ij + b ij ) − j∈N2 (a ij + b ij ) − ui (t)| < 1. Hence, there exists ϑ > 0 such that (1 − a ii ) −
j∈N3 , j =i
|a ij | +
|b ij | exp{ϑτ } + ϑ ≥ 0,
i ∈ N3 .
j∈N3
(4.13)
858
Z. Zeng and J. Wang
Consider a subsystem of equation 4.12: dyi (t) = −yi (t) + a ij y j (t) + b ij y j (t − τi j (t)), t ≥ t0 , i ∈ N3 . dt j∈N j∈N 3
3
(4.14)
Denote || y|| ¯ t0 = maxt0 −τ ≤s≤t0 {||y(s)||}. Then for i ∈ N3 , ∀t ≥ t0 , |yi (t)| ≤ || y|| ¯ t0 exp{−ϑ(t − t0 )}. Otherwise, one of the following two cases holds: Case i. There exist t2 > t1 ≥ t0 , k ∈ N3 , sufficiently small ε1 > 0 such that yk (t1 ) − ¯ t0 exp{−ϑ(t2 − t0 )} = ε1 , and when s ∈ || y|| ¯ t0 exp{−ϑ(t1 − t0 )} = 0, yk (t2 ) − || y|| ¯ t0 exp{−ϑ(s − t0 )} ≤ ε1 , and [t0 − τ, t2 ], for all i ∈ N3 , |yi (s)| − || y|| dyk (t) ¯ t0 exp{−ϑ(t1 − t0 )} ≥ 0, |t=t1 + ϑ|| y|| dt dyk (t) |t=t2 + ϑ|| y|| ¯ t0 exp{−ϑ(t2 − t0 )} > 0. dt
(4.15)
Case ii. There exist t4 > t3 ≥ t0 , j ∈ N3 , sufficiently small ε2 > 0 such that ¯ t0 exp{−ϑ(t3 − t0 )} = 0, yk (t4 ) + || y|| ¯ t0 exp{−ϑ(t4 − t0 )} = −ε2 , and y j (t3 ) + || y|| ¯ t0 exp{−ϑ(s − t0 )} ≥ −ε2 , and when s ∈ [t0 − τ, t4 ], for all i ∈ N3 , |yi (s)| − || y|| dy j (t) dt dy j (t) dt
|t=t3 − ϑ|| y|| ¯ t0 exp{−ϑ(t3 − t0 )} ≤ 0, |t=t4 − ϑ|| y|| ¯ t0 exp{−ϑ(t4 − t0 )} < 0.
(4.16)
It follows from equations 4.13 and 4.14 that for k ∈ N3 , dyk (t) |t=t2 = −yk (t2 ) + (a k j y j (t2 ) + b k j y j (t2 − τk j (t2 ))) dt j∈N 3
|a k j | ≤ || y|| ¯ t0 exp{−ϑ(t2 − t0 )} − 1 + a kk + +
j∈N3 , j =k
|b k j | exp{ϑτ } + ϑ − ϑ|| y|| ¯ t0 exp{−ϑ(t2 − t0 )}
j∈N3
|a k j | + |b k j | +ε1 − 1 + a kk + j∈N3 , j =k
≤ −ϑ|| y|| ¯ t0 exp{−ϑ(t2 − t0 )}.
j∈N3
Multiperiodicity and Exponential Attractivity of CNNs
859
This contradicts equation 4.15. Similarly, it follows from equations 4.13 and 4.14 that dy j (t) dt
¯ t0 exp{−ϑ(t4 − t0 )}. |t=t4 ≥ ϑ|| y||
This contradicts equation 4.16. The two contradictions show that for i ∈ N3 , ∀t ≥ t0 , ¯ t0 exp{−ϑ(t − t0 )}. |yi (t)| ≤ || y|| Hence, according to lemma 2, there exists ϑ¯ > 0 such that ∀i = 1, 2, . . . , n, ∀t ≥ t0 , ¯ − t0 )}. |xi (t; t0 , φ) − xi (t; t0 , ϕ)| ≤ ||φ, ϕ||t0 exp{−ϑ(t
(4.17)
(t)
Define xφ (θ ) = x(t + θ ; t0 , φ), θ ∈ [t0 − τ, t0 ]. From equations 4.6 to 4.9, if (t) ¯ : C([t0 − φ ∈ C([t0 − τ, t0 ], D1 ), then xφ ∈ C([t0 − τ, t0 ], D1 ). Define a mapping H (ω) ¯ ¯ τ, t0 ], D1 ) → C([t0 − τ, t0 ], D1 ) by H(φ) = xφ , then H(C([t0 − τ, t0 ], D1 )) ⊂ (mω)
¯ m (φ) = xφ . C([t0 − τ, t0 ], D1 ), and H Similar to the proof of theorem 1, there exists a periodic orbit x(t; t0 , φ ∗ ) of DCNN, equation 2.1, with period ω such that ∀t ≥ t0 , x(t; t0 , φ ∗ ) ∈ D1 and all other states of DCNN, equation 2.1, with initial condition (t, φ) (φ ∈ C([t0 − τ, t0 ], D1 )) converge to this periodic orbit exponentially as t → +∞. Hence, the isolated periodic orbit x(t; t0 , φ ∗ ) located in D1 is locally exponentially attractive, and D1 is a locally exponentially attractive set of x(t; t0 , φ ∗ ). Remark 3. From equations 4.1 to 4.4, we can see that the input vector u(t) can control the locality of a limit cycle that represents a memory pattern in a designated region. Specifically, when condition 4.1 holds, the part in corresponding coordinate of the limit cycle is located in (1, +∞); when condition 4.2 holds, the part in corresponding coordinate of the limit cycle is located (−∞, −1); when conditions 4.3 and 4.4 hold, the part in corresponding coordinate of the limit cycle is located [−1, 1]. When N3 is empty, we have the following corollary: Corollary 2. Let N1 ∪ N2 = {1, 2, . . . , n}, and N1 ∩ N2 be empty. If ∀t ≥ t0 , ui (t)
j∈N1
(a ij + b ij ) − 1,
i ∈ N1 ,
(4.18)
i ∈ N2 ,
(4.19)
j∈N2
(a ij + b ij ) −
j∈N2
(a ij + b ij ) + 1,
860
Z. Zeng and J. Wang
then DCNN, equation 2.1, has exactly one limit cycle located in D2 , and such a limit cycle is locally exponentially attractive. Proof. Let N3 in theorem 2 be an empty set. According to theorem 2, corollary 2 holds. When the periodic external input u(t) degenerates into a constant, we have the following corollary. Corollary 3. If ∀t ≥ t0 , ui (t) ≡ ui (constant), and ui < −1 +
(a ij + b ij ) −
j∈N1
ui > 1 +
ui < 1 − a ii −
j∈N2
|a ij | −
j∈N3 , j =i
ui > a ii − 1 +
(a ij + b ij ) −
j∈N2
(a ij + b ij ) −
j∈N1
j∈N3 , j =i
(|a ij | + |b ij |),
j∈N3
|b ij | +
j∈N3
|a ij | +
(|a ij | + |b ij |),
i ∈ N1 ,
j∈N3
(a ij + b ij ) +
(a ij + b ij ) −
j∈N1
|b ij | +
j∈N3
i ∈ N2 , (a ij + b ij ),
i ∈ N3 ,
(a ij + b ij ),
i ∈ N3 ,
j∈N2
(a ij + b ij ) −
j∈N1
j∈N2
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) ≡ τij (constant), then DCNN, equation 2.1, has only one equilibrium point located in D1 , which is locally exponentially stable. Proof. Since ui (t) ≡ ui (constant), for arbitrary constant ν ∈ , ui (t + ν) ≡ ui ≡ ui (t). According to theorem 2, DCNN, equation 2.1, has only one limit cycle located in D1 , which is locally exponentially attractive in D1 . The arbitrariness of constant ν implies that such a limit cycle is a fixed point. Hence, DCNN, equation 2.1, has only one equilibrium point located in D1 , which is locally exponentially stable.
5 Globally Exponentially Attractive Periodicity in a Designated Region In order to obtain optimal spatiotemporal coding in the periodic orbit and reduce computational time, it is desirable for a neural network to be globally exponentially attractive to periodic orbit in a designated region. In this section, we give some conditions that allow a periodic orbit to be globally exponentially attractive and to be located in any designated region. Theorem 3. If ∀t ≥ t0 , ui (t) < −1 −
n j=1
(|a ij | + |b ij |),
i ∈ N1 ,
(5.1)
Multiperiodicity and Exponential Attractivity of CNNs
ui (t) > 1 +
n (|a ij | + |b ij |), i ∈ N2 ,
861
(5.2)
j=1
|ui (t)| < 1 − a ii −
n j=1, j =i
|a ij | −
n
|b ij |, i ∈ N3 ,
(5.3)
j=1
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) = τij (t + ω), then DCNN, equation 2.1, has a unique limit cycle located in D1 , and such a limit cycle is globally exponentially attractive. Proof. When i ∈ N1 and xi (t) ≥ −1, from equations 2.1 and 5.1, dxi (t) ≤1+ (|a ij | + |b ij |) + ui (t) < 0. dt j=1 n
(5.4)
When i ∈ N2 and xi (t) ≤ 1, from equations 2.1 and 5.2, dxi (t) (|a ij | + |b ij |) + ui (t) > 0. ≥ −1 − dt j=1 n
(5.5)
When i ∈ N3 and xi (t) ≤ −1, from equations 2.1 and 5.3, n n dxi (t) ≥ 1 − a ii − |a ij | − |b ij | + ui (t) > 0. dt j=1, j =i j=1
(5.6)
When i ∈ N3 and xi (t) ≥ 1, from equations 2.1 and 5.3, n n dxi (t) ≤ −1 + a ii + |a ij | + |b ij | + ui (t) < 0. dt j=1, j =i j=1
(5.7)
Equations 5.4 to 5.7 imply that x(t; t0 , φ) will go into and keep in D1 , where φ ∈ C([t0 − τ, t0 ], n ). So there exists T > 0 such that ∀t ≥ T, x(t) ∈ D1 . Hence, ∀t ≥ T + τ, DCNN, equation 2.1, can be rewritten as dxi (t) = −xi (t) − (a ij + b ij ) + a ij x j (t) + b ij x j (t − τij (t)) dt j∈N j∈N j∈N +
j∈N2
1
3
(a ij + b ij ) + ui (t), i = 1, 2, . . . , n.
3
862
Z. Zeng and J. Wang
Similar to the proof of theorem 2, DCNN, equation 2.1, has a unique limit cycle located in D1 , and such a limit cycle is globally exponentially attractive. Remark 4. By comparison, we can see that if conditions 5.1 to 5.3 hold, then conditions 4.1 to 4.4 also hold. But not vice versa, as will be shown in examples 2 and 3. In other words, the conditions in theorem 3 are stronger than those in theorem 2. When N1 ∪ N2 is empty, we have the following corollary: Corollary 4. If ∀i, j ∈ {1, 2, . . . , n}, τij (t) = τij (t + ω), and ∀t ≥ t0 , |ui (t)| < 1 − a ii −
n
|a ij | −
j=1, j =i
n
|b ij |,
j=1
then the DCNN, equation 2.1, has a unique limit cycle located in [−1, 1]n , which is globally exponentially attractive. Proof. Choose N3 = {1, 2, . . . , n} in theorem 3. According to theorem 3, the corollary holds. When N3 is empty, we have the following corollary: Corollary 5. Let N3 be empty. If ui (t) < −1 −
n (|a ij | + |b ij |),
i ∈ N1 ,
(5.8)
i ∈ N2 ,
(5.9)
j=1
ui (t) > 1 +
n (|a ij | + |b ij |), j=1
then DCNN, equation 2.1, has a unique limit cycle located in D2 . Moreover, such a limit cycle is globally exponentially attractive. Proof. Since N3 is empty, according to theorem 3, corollary 5 holds. Remark 5. Since −1 − nj=1 (|a ij | + |b ij |) ≤ −1 + j∈N1 (a ij + b ij ) − j∈N2 (a ij + b ij ), if condition 5.8 holds, then condition 4.18 also holds, but not vice versa. Similarly, if condition 5.9 holds, then condition 4.19 also holds, but not vice versa. This implies that the conditions in corollary 5 are stronger than those in corollary 2. In addition, corollary 5 shows that a DCNN has a globally exponentially attractive limit cycle if its periodic external stimulus is sufficiently strong.
Multiperiodicity and Exponential Attractivity of CNNs
863
When the periodic external input u(t) degenerates into a constant vector, we have the following corollary: Corollary 6. If ∀t ≥ t0 , ui (t) ≡ ui (constant), and
ui < −1 −
n (|a ij | + |b ij |),
i ∈ N1 ,
j=1
ui > 1 +
n (|a ij | + |b ij |),
i ∈ N2 ,
j=1
|ui | < 1 − a ii −
n
|a ij | −
j=1, j =i
n
|b ij |,
i ∈ N3 ,
j=1
and ∀i ∈ {1, 2, . . . , n}, j ∈ N3 , τij (t) ≡ τij (constant), then DCNN, equation 2.1, has a unique equilibrium point located in D1 and is globally exponentially stable at such an equilibrium point. Proof. Since ui (t) ≡ ui (constant), for an arbitrary constant ν ∈ , ui (t + ν) ≡ ui (t) ≡ ui . According to theorem 3, DCNN, equation 2.1, has a unique limit cycle located in D1 , which is globally exponentially attractive. The arbitrariness of constant ν implies that such a limit cycle is a fixed point. Hence, DCNN, equation 2.1, has a unique equilibrium point located in D1 , and such an equilibrium point is globally exponentially stable.
6 Illustrative Examples In this section, we give three numerical examples to illustrate the new results.
6.1 Example 1. Consider a CNN, where
2
0.2
0.2
0.4 0.6
2.4
A = 0.2 2.4
0.5 sin(t)
0.2 u(t) = −0.6 cos(t)
.
−0.2(sin(t) + cos(t))
According to theorem 1, this CNN has 23 = 8 limit cycles, which are locally exponentially attractive. Simulation results with 136 random initial states are depicted in Figures 1 to 3.
864
Z. Zeng and J. Wang 4
x
1
2 0
−2 −4
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
4
x
2
2 0
−2 −4 4
x
3
2 0
−2 −4
Figure 1: Transient behavior of x1 , x2 , x3 in Example 1.
4
3
2
x3
1
0
−1
−2
−3
−4 −4
−3
−2
−1
0 x
1
1
Figure 2: Transient behavior of (x1 , x3 ) in Example 1.
2
3
4
Multiperiodicity and Exponential Attractivity of CNNs
865
4 3 2
x3
1 0
−1 −2 −3 −4 4 4
2 2
0 0
−2
−2 −4
x2
−4
x1
Figure 3: Transient behavior of (x1 , x2 , x3 ) in Example 1.
6.2 Example 2. Consider a CNN, where
2
A = 0.5
0.2 0.4
−1.4 0.4
0.2
0.5 sin(t)
0.5 u(t) = 0.5 cos(t) 0.8
.
0.5(sin(t) + cos(t))
Since u1 (t) < −1 + a 11 − a 13 − Choose N1 = {1}, N2 = {3}, N3 = {2}. |a 12 |; a 22 + |a 21 − a 23 − u2 (t)| < 1; u3 (t) > 1 + a 31 − a 33 + |a 32 |, according to theorem 2, this CNN has a limit cycle located in D1 = {x| x1 < −1, |x2 | ≤ 1, x3 > 1}, which is locally exponentially attractive in D1 . Choose N1 = {3}, N2 = {1}, N3 = {2}. Since u1 (t) > 1 + a 13 − a 11 + |a 12 |, a 22 + |a 21 − a 23 − u2 (t)| < 1, u3 (t) < −1 + a 33 − a 31 − |a 32 |, according to theorem 2, this CNN has a limit cycle located in D1 = {x| x3 < −1, |x2 | ≤ 1, x1 > 1}, which is locally exponentially attractive in D1 . However, since 0.5 sin(t) > 1 + (2 + 0.2 + 0.2) = 3.4 does not hold (i.e., condition 5.1 does not hold), it does not satisfy the conditions in theorem 3. Simulation results with 136 random initial states are depicted in Figures 4 and 5.
866
Z. Zeng and J. Wang 4
x
1
2 0
−2 −4 0
5
10
15 time
20
25
30
5
10
15 time
20
25
30
5
10
15 time
20
25
30
4
x
2
2 0
−2 −4 0 4
x
3
2 0
−2 −4 0
Figure 4: Transient behavior of x1 , x2 , x3 in Example 2.
4 3 2
x3
1 0
−1 −2 −3 −4 4 4
2 2
0 0
−2 x2
−2 −4
−4
x1
Figure 5: Transient behavior of (x1 , x2 , x3 ) in Example 2.
Multiperiodicity and Exponential Attractivity of CNNs
867
x1
5
0
−5
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
0
5
10
15 time
20
25
30
x2
5
0
−5
x3
5
0
−5
Figure 6: Transient behavior of x1 , x2 , x3 in Example 3.
6.3 Example 3. Consider a CNN, where
0.2 0.2
0.2
0.2 0.2
0.2
A = 0.2 −2
0.8 sin(t) − 2.6
0.6 u(t) = 0.8 cos(t)
.
0.8(sin(t) + cos(t)) + 2.8
Since u1 (t) < −1 − 3j=1 |a 1 j |, Choose N1 = {1}, N2 = {3}, N3 = {2}. |u2 (t)| < 1 − a 22 − 3j=1, j =2 |a 2 j |, u3 (t) > 1 + 3j=1 |a 3 j |, according to theorem 3, this CNN has a limit cycle located in D1 = {x| x1 < −1, |x2 | ≤ 1, x3 > 1}, which is globally exponentially attractive. Since u1 (t) < −1 + a 11 − |a 12 | − a 13 ; u2 (t) < 1 − a 22 + a 21 − a 23 , u2 (t) > −1 + a 22 + a 21 − a 23 ; u3 (t) > 1 + a 31 + |a 32 | − a 33 , conditions 4.1 to 4.4 also hold. According to theorem 2, this CNN has a limit cycle located in D1 , which is also locally exponentially attractive. However, since a 11 > 0, a 33 > 0 the conditions in Yi et al. (2003) cannot be used to ascertain the complete stability of this CNN. Simulation results with 136 random initial states are depicted in Figures 6 and 7.
868
Z. Zeng and J. Wang
5 4 3
x3
2 1 0 −1 −2 −3 6 4
6 2
4 2
0 0
−2 x2
−4
−2 −4
x1
Figure 7: Transient behavior of (x1 , x2 , x3 ) in Example 3.
7 Concluding Remarks Rhythmicity represents one of most striking manifestations of dynamic behaviors in biological systems. CNNs and DCNNs, which have been shown to be capable of operating in a pacemaker or pattern generator mode, are studied here as oscillatory mechanisms in response to periodic external stimuli. Some information can be encoded in the oscillating activation states relative to external inputs, and these relative phases change as a function of the chosen limit cycle. In this article, we show that the number of locally exponentially attractive periodic orbits located in saturation regions in a DCNN is exponential of the number of the neurons. In view of the fact that neural information is often desired to be encoded in a designated region, we also give conditions to allow a globally exponentially attractive periodic orbit located in any designated region. The theoretical results are supplemented by simulation results in three illustrative examples.
Acknowledgments This work was supported by the Hong Kong Research Grants Council under grant CUHK4165/03E, the Natural Science Foundation of China
Multiperiodicity and Exponential Attractivity of CNNs
869
under grant 60405002 and China Postdoctoral Science Foundation under Grant 2004035579.
References Belair, J., Campbell, S., & Driessche, P. (1996). Frustration, stability and delay-induced oscillation in a neural network model. SIAM J. Applied Math., 56, 245–255. Berns, D. W., Moiola, J. L., & Chen, G. (1998). Predicting period-doubling bifurcations and multiple oscillations in nonlinear time-delayed feedback systems. IEEE Trans. Circuits Syst. I, 45(7), 759–763. Chen, K., Wang, D. L., & Liu, X. (2000). Weight adaptation and oscillatory correlation for image segmentation. IEEE Trans. Neural Networks, 11, 1106–1123. Chua, L. O., & Roska, T. (1990). Stability of a class of nonreciprocal cellular neural networks. IEEE Trans. Circuits Syst., 37, 1520–1527. Chua, L. O., & Roska, T. (1992). Cellular neural networks with nonlinear and delaytype template elements and non-uniform grids. Int. J. Circuit Theory and Applicat., 20, 449–451. Civalleri, P. P., Gilli, M., & Pandolfi, L. (1993). On stability of cellular neural networks with delay. IEEE Trans. Circuits Syst. I, 40, 157–164. Gopalsamy, K., & Leung, I. (1996). Delay induced periodicity in a neural netlet of excitation and inhibition. Physica D, 89, 395–426. Jiang, H., & Teng, Z. (2004). Boundedness and stability for nonautonomous bidirectional associative neural networks with delay. IEEE Trans. Circuits Syst. II, 51(4), 174–180. Jin, H. L., & Zacksenhouse, M. (2003). Oscillatory neural networks for robotic yo-yo control. IEEE Trans. Neural Networks, 14(2), 317–325. Kanamaru, T., & Sekine, M. (2004). An analysis of globally connected active rotators with excitatory and inhibitory connections having different time constants using the nonlinear Fokker-Planck equations. IEEE Trans. Neural Networks, 15(5), 1009– 1017. Kosaku, Y. (1978). Functional analysis. Berlin: Springer-Verlag. Liao, X. X., & Wang, J. (2003). Algebraic criteria for global exponential stability of cellular neural networks with multiple time delays. IEEE Trans. Circuits and Syst. I, 50(2), 268–275. Liao, X., Wu, Z., & Yu, J. (1999). Stability switches and bifurcation analysis of a neural network with continuous delay. IEEE Trans. Systems, Man and Cybernetics, Part A, 29(6), 692–696. Liu, Z., Chen, A., Cao, J., & Huang, L. (2003). Existence and global exponential stability of periodic solution for BAM neural networks with periodic coefficients and time-varying delays. IEEE Trans. Circuits Syst. I, 50(9), 1162–1173. Nishikawa, T., Lai, Y. C., & Hoppensteadt, F. C. (2004). Capacity of oscillatory associative-memory networks with error-free retrieval. Physical Review Letters, 92(10), 108101. Roska, T., Wu, C. W., Balsi, M., & Chua, L. O. (1992). Stability and dynamics of delay-type general and cellular neural networks. IEEE Trans. Circuits Syst. I, 39, 487–490.
870
Z. Zeng and J. Wang
Roska, T., Wu, C. W., & Chua, L. O. (1993). Stability of cellular neural networks with dominant nonlinear and delay-type templates. IEEE Trans. Circuits Syst. I, 40, 270–272. Ruiz, A., Owens, D. H., & Townley, S. (1998). Existence, learning, and replication of periodic motion in recurrent neural networks. IEEE Trans. Neural Networks, 9(4), 651–661. Setti, G., Thiran, P., & Serpico, C. (1998). An approach to information propagation in 1-D cellular neural networks, part II: Global propagation. IEEE Trans. Circuits and Syst. I, 45(8), 790–811. Takahashi, N. (2000). A new sufficient condition for complete stability of cellular neural networks with delay. IEEE Trans. Circuits Syst. I, 47, 793–799. Townley, S., Ilchmann, A., Weiss, M. G., McClements, W., Ruiz, A. C., Owens, D. H., & Pratzel-Wolters, D. (2000). Existence and learning of oscillations in recurrent neural networks. IEEE Trans. Neural Networks, 11, 205–214. Wang, D. L. (1995). Emergent synchrony in locally coupled neural oscillators. IEEE Trans. Neural Networks, 6, 941–948. Wang, L., & Zou, X. (2004). Capacity of stable periodic solutions in discrete-time bidirectional associative memory neural networks. IEEE Trans. Circuits Syst. II, 51(6), 315–319. Yi, Z., Heng, P. A., & Vadakkepat, P. (2002). Absolute periodicity and absolute stability of delayed neural networks. IEEE Trans. Circuits Syst. I, 49(2), 256–261. Yi, Z., Tan, K. K., & Lee, T. H. (2003). Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 15(3), 639–662. Zeng, Z., Wang, J., & Liao, X. X. (2003). Global exponential stability of a general class of recurrent neural networks with time-varying delays. IEEE Trans. Circuits and Syst. I, 50(10), 1353–1358. Zeng, Z., Wang, J., & Liao, X. X. (2004). Stability analysis of delayed cellular neural networks described using cloning templates. IEEE Trans. Circuits and Syst. I, 51(11), 2313–2324.
Received November 30, 2004; accepted June 28, 2005.
