A Q-Modification Neuroadaptive Control Architecture for Discrete-Time ...
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
1507
Brief Papers A Q-Modification Neuroadaptive Control Architecture for Discrete-Time Systems Konstantin Y. Volyanskyy, Student Member, IEEE, and Wassim M. Haddad, Fellow, IEEE
Abstract—This brief extends the new neuroadaptive control framework for continuous-time nonlinear uncertain dynamical systems based on a Q-modification architecture to discrete-time systems. As in the continuous-time case, the discrete-time update laws involve auxiliary terms, or Q-modification terms, predicated on an estimate of the unknown neural network weights which in turn involve a set of auxiliary equations characterizing a set of affine hyperplanes. In addition, we show that the Q-modification terms in the discrete-time update law are designed to minimize an error criterion involving a sum of squares of the distances between the update weights and the family of affine hyperplanes. Index Terms—Adaptive control, discrete-time systems, neural networks, Q-modification architecture, uncertainty suppression.
I. Introduction Neural networks have been extensively used for adaptive system identification as well as adaptive and neuroadaptive control of highly uncertain continuous-time dynamical systems [1], [6], [11], [12], [14]–[18]. One of the primary reasons for the large interest in neural networks is their capability to approximate a large class of continuous nonlinear maps from the collective action of very simple, autonomous processing units interconnected in simple ways. Neural networks have also attracted attention due to their inherently parallel and highly redundant processing architecture that makes it possible to develop parallel weight update laws. This parallelism makes it possible to effectively update a neural network on line. Discrete-time extensions of neural network adaptive control methods have also appeared in the literature; see [2]–[4], [7]– [10], [13], [18], and the references therein. To improve robustness and the speed of adaptation of adaptive and neuroadaptive controllers several controller architectures have been proposed in the literature. These include the σ and e-modification architectures used to keep the system parameter estimates from growing without bound in the face of system uncertainty and system disturbances [10], [12], [18]. In the recent papers [19], [20], a new neuroadaptive control architecture for nonlinear uncertain dynamical systems was developed. Specifically, the proposed Manuscript received October 27, 2009; revised March 31, 2010. Date of publication August 13, 2010; date of current version September 1, 2010. This work was supported in part by the Air Force Office of Scientific Research, under Grant FA9550-09-1-0429. The authors are with the School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0150 USA (e-mail: [email protected]; [email protected]). Digital Object Identifier 10.1109/TNN.2010.2047869
framework involved a new and novel controller architecture involving additional terms, or Q-modification terms, in the update laws that were constructed using a moving time window of the integrated system uncertainty. The Qmodification terms were shown to effectively suppress and cancel system uncertainty without the need for persistency of excitation. In this brief, we extend some of the results of [19] and [20] to discrete-time uncertain dynamical systems. As in the continuous-time case, the discrete-time update laws involve auxiliary terms, or Q-modification terms, predicated on an estimate of the unknown neural network weights which in turn involve a set of auxiliary equations characterizing a set of affine hyperplanes. In addition, we show that the Qmodification terms in the update law are designed to minimize an error criterion involving a sum of squares of the distances between the update weights and the family of affine hyperplanes. The proposed approach thus uses a linear subspace projection scheme with a gradient-based search to estimate the neural network weights. The notation used in this brief is fairly standard. Specifically, Z+ denotes the set of nonnegative integers, (·)T denotes transpose, (·)† denotes the Moore–Penrose generalized inverse, tr(·) denotes the trace operator, ln(·) denotes the natural log operator, and · denotes the Euclidean norm.
II. Neuroadaptive Control for Discrete-Time Nonlinear Uncertain Dynamical Systems With a Q-Modification Architecture In this section, we consider the problem of characterizing neuroadaptive full-state feedback control laws for discretetime nonlinear uncertain dynamical systems to achieve reference model trajectory tracking. Specifically, consider the controlled discrete-time nonlinear uncertain dynamical system G given by x(k + 1) = A0 x(k) + B(ˆx(k), u(k)) ˆ + BG(x(k))u(k) x(0) = x0 ,
k ∈ Z+
(1)
where x(k) ∈ Rn , k ∈ Z+ , is the state vector, u(k) ∈ Rm , k ∈ Z+ , is the control input, xˆ (k) x(k), x(k − 1), . . . , x(k − p + 1) is a vector of p-delayed values of the state vector with ˆ p ≥ 1, u(k) u(k − 1), u(k − 2), . . . , u(k − q) is a vector of q-delayed values of the control input with q ≥ 1, A0 ∈ Rn×n and B ∈ Rn×m are known matrices, G : Rn → Rm×m is a known input matrix function such that det G(x) = 0 for all x ∈ Rn , and : Rnp × Rmq → Rm is an unknown nonlinear function representing system uncertainty. Dynamical systems with uncertainty structures given by (1) are discussed in [20].
c 2010 IEEE 1045-9227/$26.00
1508
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
We assume that (ˆx(k), u(k)), ˆ k ∈ Z+ , can be perfectly parameterized as ˆ (ˆx(k), u(k)) ˆ = W T σ(ˆx(k), u(k)),
k ∈ Z+
(2)
where W ∈ Rl×m is an unknown matrix and σ : Rnp × Rmq → Rl is a known bounded Lipschitz continuous function such that, for all xˆ (k) ∈ Rnp and u(k) ˆ ∈ Rmq , k ∈ Z+ , σ(ˆx, u) ˆ ≤ ∗ ∗ σ , where σ > 0. Furthermore, we assume that the state x(k), k ∈ Z+ , is available for feedback. In order to achieve trajectory tracking, we construct a reference system Gref given by xref (k + 1) = Aref xref (k) + Bref r(k),
xref (0) = xref 0 ,
k ∈ Z+ (3)
where xref (k) ∈ Rn , k ∈ Z+ , is the reference state vector, r(k) ∈ Rr , k ∈ Z+ , is a bounded reference input, Aref ∈ Rn×n is Shur, and Bref ∈ Rn×r . Since all the eigenvalues of the matrix Aref lie in the open unit disk, it follows from Lemma 13.2 of [18] that there exists a positive-definite matrix P ∈ Rn×n such that P = ATref PAref + R
(4)
where R ∈ Rn×n is a positive-definite matrix. The goal here is to develop an adaptive control signal u(k), k ∈ Z+ , that guarantees x(k) → xref (k) as k → ∞. The following matching conditions are needed for the main result of this section. Assumption 1: There exist gains Kx ∈ Rm×n and Kr ∈ m×r such that A0 + BKx = Aref and BKr = Bref . R Consider the control law given by u(k) = G−1 (x(k))(un (k) − uad (k)),
k ∈ Z+
(5)
where un (k) = Kx x(k) + Kr r(k),
k ∈ Z+
(6)
and ˆ T (k)σ(ˆx(k), u(k)), ˆ uad (k) = W
k ∈ Z+
(7)
gives the output of the linearly parameterized neural network ˆ and W(k) ∈ Rl×m , k ∈ Z+ , is an update weight matrix. Using (5)–(7) and Assumption 1, the system dynamics (1) can be rewritten as ˜ T (k)σ(ˆx(k), u(k)) ˆ x(k + 1) = Aref x(k) + Bref r(k) + BW x(0) = x0 , k ∈ Z+ (8)
Fig. 1.
Visualization of Q-modification term.
where q(i) = σ(ˆx(i), u(i)), ˆ i = 1, . . . , s ˆ T (i)σ(ˆx(i), u(i)) c(i) = e(i + 1) − Aref e(i) + BW ˆ i = 1, . . . , s.
k ∈ Z+ . Next, define
˜ (k)σ(ˆx(k), u(k)), ˆ e(0) = e0 , k ∈ Z+ e(k + 1) = Aref e(k) + BW (9)
ˆ and note that if W(k), k ∈ Z+ , satisfies
where e0 x0 − xref 0 . Now, for every k ∈ Z+ , consider a linear subspace Ls , where s ≤ min{l, k}, formed by s linearly independent vectors q(i), i = 1, . . . , s, such that BW T q(i) = c(i),
i = 1, . . . , s
(10)
(12)
Note that (10) is a direct consequence of (9). Furthermore, note that q(i), i = 1, . . . , s, and c(i), i = 1, . . . , s, given by (11) and (12) are computable. Hence, although the matrix W is unknown, W satisfies a set of linear equations given by (10). Equation (10) represents a system of s equations in terms of the entries of W, where each of these equations characterizes an affine hyperplane. For example, in the case where n = 1, m = 1, l = 2, s = 1, W = [W1 , W2 ]T , and B = 1, the affine hyperplane (10) is described by a line Ls with q(i), i = 1, . . . , s, being a normal vector to Ls as shown in Fig. 1. Note that the distance from point A to point B shown in Fig. 1, ˆ which is the shortest distance from the weight estimate W(k) to affine hyperplane Ls defined by (10), is given by c(i) − ˆ T (k)q(i). W Next, define the error criterion γQ ˆ ˆ T (k)¯q(k) ρ(W(k), q¯ (k), c¯ (k)) tr c¯ (k) − BW 2 ˆ T (k)¯q(k) T , k ∈ Z+ · c¯ (k) − BW (13) s s where γQ > 0, q¯ (k) i=1 αi (k)q(i), c¯ (k) i=1 αi (k)c(i), and αi (k) ∈ R, i = 1, . . . , s, k ∈ Z+ , are design parameters. Note that (13) is a weighted sum of squares of the distances ˆ between the update weights W(k), k ∈ Z+ , and the family of affine hyperplanes defined by (10). Now, note that the gradient ˆ ˆ of ρ(W(k), q¯ (k), c¯ (k)), k ∈ Z+ , with respect to W(k), k ∈ Z+ , is given by ˆ ∂ρ(W(k), q¯ (k), c¯ (k)) ˆ T (k)¯q(k) T B = −γQ q¯ (k) c¯ (k) − BW ˆ ∂W(k)
˜ W − W. ˆ Defining the tracking error e(k) x(k) − where W xref (k), k ∈ Z+ , the error dynamics are given by T
(11)
ˆ T (k)¯q(k) T B, H(k) γQ q¯ (k) c¯ (k) − BW
ˆ T (k)¯q(k) = c¯ (k), BW
k ∈ Z+
k ∈ Z+
(14)
(15)
(16)
ˆ then H(k), k ∈ Z+ , is zero and the weight estimates W(k), k ∈ Z+ , lie on the collection of the affine hyperplanes defined ˆ by (10). If the weight estimates W(k), k ∈ Z+ , do not satisfy (16), then each nonzero row of the matrix H(k), k ∈ Z+ , is a vector that is orthogonal to the corresponding affine
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
hyperplane defined by (10) and points in the direction of the hyperplane. Finally, using (10), it follows that ˆ T (k)¯q(k)2 ≥ 0, k ∈ Z+ . ˜ tr H T (k)W(k) = γQ c¯ (k) − BW (17) Theorem 1: Consider the nonlinear uncertain dynamical system G given by (1). Assume Assumption 1 holds. Furthermore, assume that for a given γ > 0, there exist positivedefinite matrices P ∈ Rn×n and E ∈ Rm×m such that 1 Im > (E + BT PB) γ(c + σ ∗ 2 )
(18)
where Im is the m × m identity matrix and c > 0. Then, the neuroadaptive control law (5)–(7) with update law given by
1509
along the closed-loop system trajectories is given by ˜ V (e(k), W(k)) ˜ + 1)) − V (e(k), W(k)) ˜ V (e(k + 1), W(k T T ˜ (k)σ(ˆx(k), u(k)) ˆ P = ln 1 + Aref e(k) + BW ˜ T (k)σ(ˆx(k), u(k)) · Aref e(k) + BW ˆ
T − ln 1 + e (k)Pe(k) 1 ˜T ˜ + 1) − 1 tr W ˜ T (k)W(k) ˜ + tr W (k + 1)W(k γ γ
= ln 1 + eT (k) ATref PAref − P e(k) ˜ +2eT (k)ATref PBW(k)σ(ˆ x(k), u(k)) ˆ T T ˜ +σ (ˆx(k), u(k)) ˆ W(k)B PBW T (k)σ(ˆx(k), u(k)) ˆ −1 1 − tr [(k + 1) + Q(k + 1)]T · 1 + eT (k)Pe(k) γ ˜ · 2W(k) − (k + 1) − Q(k + 1) . (24)
k ∈ Z+ Next, let R > 0 and R > 0 be such that R + R = R and 1 2 1 2 (19) define T ˜ (25) ˆ W(k) , k ∈ Z+ η(k) eT (k), σ T (ˆx(k), u(k)) where (k) ∈ Rl×m , k ∈ Z+ , and Q(k) ∈ Rl×m , k ∈ Z+ , are given by
R2 −ATref PB M > 0. (26) E −BT PAref 1 (k + 1) = c + σ T (ˆx(k), u(k))σ(ˆ ˆ x(k), u(k)) ˆ Now, using (4) and the fact that ln(1 + a) ≤ a, a > −1, it ·σ(ˆx(k), u(k)) ˆ (20) follows from (24) that [e(k + 1) − Aref e(k)]T B†
ˆ + 1) = W(k) ˆ W(k + (k + 1) + Q(k + 1),
ˆ ˆ 0, W(0) =W
⎧ ⎨ H(k), if g(k) > 0 Q(k + 1) =
⎩
,
k ∈ Z+
(21)
0l×m , otherwise
and ˆ T (k)¯q(k)2 − tr T (k)H(k) g(k) γQ c¯ (k) − BW 1 (22) − tr H T (k)H(k), k ∈ Z+ 2 guarantees that there exists a positively invariant set Dα ⊂ Rn × Rl×m , with (0, W) ∈ Dα , such that the solution ˆ (e(k), W(k)) ≡ (0, W) of the closed-loop system given by (9), (5)–(7), and (19)–(21) is Lyapunov stable and e(k) → 0 as ˆ 0 ) ∈ Dα . k → ∞ for all (e0 , W Proof: To show Lyapunov stability of the closed-loop system (9), (5)–(7), and (19)–(21), consider the Lyapunov function candidate ˜ = ln(1 + eT Pe) + 1 tr W ˜ TW ˜ V (e, W) γ
(23)
where P > 0 satisfies (4) and γ > 0. Note that V (0, 0) = 0 ˜ > 0 for all and, since P is positive definite and γ > 0, V (e, W) ˜ (e, W) = 0. Now, letting e(k), k ∈ Z+ , denote the solution to (9) and using (19)–(22), it follows that the Lyapunov difference
˜ V (e(k), W(k)) T 1 e (k)R1 e(k) − ηT (k)Mη(k) ≤− T T 1 + e (k)Pe(k) 1 + e (k)Pe(k) 1 ˜ + σ T (ˆx(k), u(k)) ˆ W(k) E + BT PB T 1 + e (k)Pe(k) 1 ˜ T (k)σ(ˆx(k), u(k)) ˆ − tr [(k + 1) + Q(k + 1)]T ·W γ ˜ · 2W(k) − (k + 1) − Q(k + 1) . (27) Using (20), (21), and the fact that ˆ x(k), u(k)) ˆ σ T (ˆx(k), u(k))σ(ˆ < 1, T c + σ (ˆx(k), u(k))σ(ˆ ˆ x(k), u(k)) ˆ
c>0
it follows that ˜ V (e(k), W(k)) T 1 e (k)R1 e(k) − ηT (k)Mη(k) ≤− 1 + eT (k)Pe(k) 1 + eT (k)Pe(k) 1 ˜ + ˆ W(k) E + BT PB σ T (ˆx(k), u(k)) T 1 + e (k)Pe(k) ˜ T (k)σ(ˆx(k), u(k)) ·W ˆ 1 1 − ˆ σ T (ˆx(k), u(k)) T γ c + σ (ˆx(k), u(k))σ(ˆ ˆ x(k), u(k)) ˆ ˜ W ˜ T (k)σ(ˆx(k), u(k)) ·W(k) ˆ 2 ˜ − tr QT (k + 1)W(k) − T (k + 1)Q(k + 1) γ 1 − QT (k + 1)Q(k + 1) . 2
(28)
1510
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
Fig. 3.
Control input versus time.
Fig. 4.
Update weights versus time without the Q-modification controller.
Fig. 2. System states versus time with and without the Q-modification controller.
Finally, using (18), (21), and (17), it follows that ˜ V (e(k), W(k)) ≤ 0,
k ∈ Z+ .
Next, define ˜ ∈ Rn × Rl×m : V (e, W) ˜ ≤ α , α > 0. D˜ α (e, W)
(29)
(30)
˜ ˜ Since V (e(k), W(k)) ≤ 0 for all (e(k), W(k)) ∈ D˜ α and k ∈ Z+ , it follows that D˜ α is positively invariant. Now, since D˜ α is positively invariant, it follows that: ˆ ∈ Rn × Rl×m : (e, W ˆ − W) ∈ D˜ α (31) Dα (e, W) is also positively invariant. Furthermore, it follows from (29) ˆ and [5, Th. 13.10] that the solution (e(k), W(k)) ≡ (0, W) of the closed-loop system given by (9) and (19) is Lyapunov ˆ 0 ) ∈ Dα . stable and e(k) → 0 as k → ∞ for all (e0 , W 2 The term Q(k), k ∈ Z+ , in (19) given by (21) is a discretetime analogue of the Q-modification term introduced in [20] for continuous-time systems. As in the continuous-time case, Q(k), k ∈ Z+ , is an additional term that is introduced to the update law that is designed to minimize the error criterion given by (13) and is constructed based on information of the unknown weights W given by (10) and the property given by (17). Note that for every k ∈ Z+ , the vector Q(k) is directed ˆ ˆ opposite to the gradient ∂ρ(W(k), q¯ (k), c¯ (k))/∂W(k) and, in the case where the system uncertainty is a scalar function, is parallel to q¯ (k), which involves a linear combination of vectors normal to the affine hyperplanes defined by (10). Hence, Q(k), k ∈ Z+ , introduces a component in the update ˆ law (19) that drives the trajectory W(k), k ∈ Z+ , in such a way so that the error criterion given by (13) is minimized. For further details on the utility of the Q-modification term for uncertainty suppression and uncertainty cancelation see [20].
III. Illustrative Numerical Example In this section, we present a numerical example to demonstrate the utility and efficacy of the proposed Qmodification architecture for discrete-time neuroadaptive stabilization. Specifically, consider the nonlinear dynamical system
given by x(k + 1) = A0 x(k) + B(x(k)) + Bu(k), where
A0 =
0 −0.35
1 −0.25
x(0) = x0 ,
,
B=
0 1
k ∈ Z+ (32)
(x(k)) = W T σ(x(k)), k ∈ Z+ , and σ(x) = [sin(x1 ), cos(x2 ), cos(x1 ), sin(x2 ), x1 , x1 |x1 |]T is a known regressor vector. Note that eigenvalues of A0 are λ1 = −0.1250 − 0.5783j and λ2 = −0.1250 + 0.5783j, and hence, lie inside the unit disk. Here, our goal is to achieve stabilization of the uncertain system around the origin. Hence, the reference model is given by (3) with Aref = A0 , Bref = B, xref 0 = [0, 0]T , and r(k) ≡ 0. Next, we use Theorem 1 to design a neuroadaptive controller given by (5)–(7), with un (k) ≡ 0 and update laws given by (19)–(21). Now, with initial conditions x0 = [3, 4]T and ˆ 0 = 06×1 , and W = [1.00, −1.50, 2.50, 3.50, 1.00, 0.50]T , W Figs. 2–5 show the state trajectories, the control input, and the update weight trajectories versus time with and without the Qmodification term activated. It is clear that the Q-modification architecture results in a faster convergence and reduces system and weight oscillations. It is interesting to note that for this example the update weights for both controllers converge to the same values. However, in the presence of persistency of
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
Fig. 5.
Update weights versus time with the Q-modification controller.
excitation it can be shown that the update weights of the Qmodification controller converge to the ideal weights [20]. IV. Conclusion In this brief, we presented a new and novel neuroadaptive controller architecture for nonlinear discrete-time uncertain systems based on the approach introduced in [20]. As in the continuous-time case, this controller architecture provided fast adaptation to effectively suppress system uncertainty while avoiding large parameter (high gain) stabilization which can excite unmodeled system dynamics. Extensions of the present approach to nonlinear in the parameters neural networks as well as output feedback control follow as in [20]. References [1] F. C. Chen and H. K. Khalil, “Adaptive control of nonlinear systems using neural networks,” Int. J. Contr., vol. 55, no. 6, pp. 1299–1317, 1992. [2] F. C. Chen and H. K. Khalil, “Adaptive control of a class of nonlinear discrete-time systems using neural networks,” IEEE Trans. Autom. Contr., vol. 40, no. 5, pp. 791–807, May 1995. [3] H. Deng, H.-X. Lin, and Y.-H. Wu, “Feedback-linearization-based neural adaptive control for unknown nonaffine nonlinear discrete-time systems,” IEEE Trans. Neural Netw., vol. 19, no. 9, pp. 1615–1625, Sep. 2008. [4] S. S. Ge, T. H. Lee, G. Y. Li, and J. Zhang, “Adaptive NN control for a class of discrete-time non-linear systems,” Int. J. Contr., vol. 76, no. 4, pp. 334–354, 2003. [5] W. M. Haddad and V. Chellaboina, Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach. Princeton, NJ: Princeton Univ. Press, 2008. [6] K. J. Hunt, D. Sbarbaro, R. Zbikowski, and P. J. Gawthrop, “Neural networks for control: A survey,” Automatica, vol. 28, pp. 1083–1112, Nov. 1992. [7] C.-L. Hwang and C.-H. Lin, “A discrete-time multivariable neuroadaptive control for nonlinear unknown dynamic systems,” IEEE Trans. Syst. Man Cybern., vol. 30, no. 6, pp. 865–877, Dec. 2000. [8] S. Jagannathan and F. L. Lewis, “Multilayer discrete-time neural net controller with guaranteed performance,” IEEE Trans. Neural Netw., vol. 7, no. 1, pp. 107–130, Jan. 1996. [9] S. Jagannathan, “Control of a class of nonlinear discrete-time systems using multilayer neural networks,” IEEE Trans. Neural Netw., vol. 12, no. 5, pp. 1113–1120, Sep. 2001. [10] S. Jagannathan, Neural Network Control of Nonlinear Discrete-Time Systems. Boca Raton, FL: CRC, 2006. [11] A. U. Levin and K. S. Narendra, “Control of nonlinear dynamical systems using neural networks: Controllability and stabilization,” IEEE Trans. Neural Netw., vol. 4, no. 2, pp. 192–206, Mar. 1993.
1511
[12] F. L. Lewis, S. Jagannathan, and A. Yesildirak, Neural Network Control of Robot Manipulators and Nonlinear Systems. London, U.K.: Taylor and Francis, 1999. [13] L. Lin, P. N. Nikiforuk, and M. M. Gupta, “Fast neural learning and control of discrete-time nonlinear systems,” IEEE Trans. Syst. Man Cybern., vol. 25, no. 3, pp. 478–488, Mar. 1995. [14] K. S. Narendra and K. Parthasarathy, “Identification and control of dynamical systems using neural networks,” IEEE Trans. Neural Netw., vol. 1, no. 1, pp. 4–27, Mar. 1990. [15] J. Park and I. Sandberg, “Universal approximation using radial basis function networks,” Neural Comput., vol. 3, no. 2, pp. 246–257, 1991. [16] M. Polycarpou, “Stable adaptive neural control scheme for nonlinear systems,” IEEE Trans. Autom. Contr., vol. 41, no. 3, pp. 447–451, Mar. 1996. [17] R. Sanner and J. Slotine, “Gaussian networks for direct adaptive control,” IEEE Trans. Neural Netw., vol. 3, no. 6, pp. 837–864, Nov. 1992. [18] J. Spooner, M. Maggiore, R. Ordonez, and K. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques. New York: Wiley, 2002. [19] K. Volyanskyy, W. M. Haddad, and A. J. Calise, “A new neuroadaptive control architecture for nonlinear uncertain dynamical systems: Beyond σ and e-modifications,” in Proc. IEEE Conf. Dec. Contr., Cancun, Mexico, Dec. 2008, pp. 80–85. [20] K. Y. Volyanskyy, W. M. Haddad, and A. J. Calise, “A new neuroadaptive control architecture for nonlinear uncertain dynamical systems: Beyond σ and e-modifications,” IEEE Trans. Neural Netw., vol. 20, no. 11, pp. 1707–1723, Nov. 2009.
Real-Time Simulation of Biologically Realistic Stochastic Neurons in VLSI Hsin Chen, Member, IEEE, Sylvain Sa¨ıghi, Member, IEEE, Laure Buhry, and Sylvie Renaud, Member, IEEE
Abstract—Neuronal variability has been thought to play an important role in the brain. As the variability mainly comes from the uncertainty in biophysical mechanisms, stochastic neuron models have been proposed for studying how neurons compute with noise. However, most papers are limited to simulating stochastic neurons in a digital computer. The speed and the efficiency are thus limited especially when a large neuronal network is of concern. This brief explores the feasibility of simulating the stochastic behavior of biological neurons in a very large scale integrated (VLSI) system, which implements a programmable and configurable Hodgkin-Huxley model. By simply injecting noise to the VLSI neuron, various stochastic behaviors observed in biological neurons are reproduced realistically in VLSI. The noise-induced variability is further shown to enhance the signal modulation of a neuron. These results point toward the development of analog VLSI systems for exploring the stochastic behaviors of biological neuronal networks in large scale. Index Terms—Analog VLSI, Hodgkin-Huxley formalism, neuromorphic VLSI, noise, stochastic behavior, stochastic neurons. Manuscript received August 3, 2009; revised February 24, 2010; accepted April 15, 2010. Date of publication June 21, 2010; date of current version September 1, 2010. This work was supported in part by the National Science Council, Taiwan, under Grant 97-2918-I-007-003, and in part by the European Union in the FP6 Program under Grant 15879 (FACETS). H. Chen is with the Department of Electrical Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan (e-mail: [email protected]). S. Sa¨ıghi, L. Buhry, and S. Renaud are with the IMS Lab, University of Bordeaux, Talence F-33400, France (e-mail: [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNN.2010.2049028
c 2010 IEEE 1045-9227/$26.00
1507
Brief Papers A Q-Modification Neuroadaptive Control Architecture for Discrete-Time Systems Konstantin Y. Volyanskyy, Student Member, IEEE, and Wassim M. Haddad, Fellow, IEEE
Abstract—This brief extends the new neuroadaptive control framework for continuous-time nonlinear uncertain dynamical systems based on a Q-modification architecture to discrete-time systems. As in the continuous-time case, the discrete-time update laws involve auxiliary terms, or Q-modification terms, predicated on an estimate of the unknown neural network weights which in turn involve a set of auxiliary equations characterizing a set of affine hyperplanes. In addition, we show that the Q-modification terms in the discrete-time update law are designed to minimize an error criterion involving a sum of squares of the distances between the update weights and the family of affine hyperplanes. Index Terms—Adaptive control, discrete-time systems, neural networks, Q-modification architecture, uncertainty suppression.
I. Introduction Neural networks have been extensively used for adaptive system identification as well as adaptive and neuroadaptive control of highly uncertain continuous-time dynamical systems [1], [6], [11], [12], [14]–[18]. One of the primary reasons for the large interest in neural networks is their capability to approximate a large class of continuous nonlinear maps from the collective action of very simple, autonomous processing units interconnected in simple ways. Neural networks have also attracted attention due to their inherently parallel and highly redundant processing architecture that makes it possible to develop parallel weight update laws. This parallelism makes it possible to effectively update a neural network on line. Discrete-time extensions of neural network adaptive control methods have also appeared in the literature; see [2]–[4], [7]– [10], [13], [18], and the references therein. To improve robustness and the speed of adaptation of adaptive and neuroadaptive controllers several controller architectures have been proposed in the literature. These include the σ and e-modification architectures used to keep the system parameter estimates from growing without bound in the face of system uncertainty and system disturbances [10], [12], [18]. In the recent papers [19], [20], a new neuroadaptive control architecture for nonlinear uncertain dynamical systems was developed. Specifically, the proposed Manuscript received October 27, 2009; revised March 31, 2010. Date of publication August 13, 2010; date of current version September 1, 2010. This work was supported in part by the Air Force Office of Scientific Research, under Grant FA9550-09-1-0429. The authors are with the School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0150 USA (e-mail: [email protected]; [email protected]). Digital Object Identifier 10.1109/TNN.2010.2047869
framework involved a new and novel controller architecture involving additional terms, or Q-modification terms, in the update laws that were constructed using a moving time window of the integrated system uncertainty. The Qmodification terms were shown to effectively suppress and cancel system uncertainty without the need for persistency of excitation. In this brief, we extend some of the results of [19] and [20] to discrete-time uncertain dynamical systems. As in the continuous-time case, the discrete-time update laws involve auxiliary terms, or Q-modification terms, predicated on an estimate of the unknown neural network weights which in turn involve a set of auxiliary equations characterizing a set of affine hyperplanes. In addition, we show that the Qmodification terms in the update law are designed to minimize an error criterion involving a sum of squares of the distances between the update weights and the family of affine hyperplanes. The proposed approach thus uses a linear subspace projection scheme with a gradient-based search to estimate the neural network weights. The notation used in this brief is fairly standard. Specifically, Z+ denotes the set of nonnegative integers, (·)T denotes transpose, (·)† denotes the Moore–Penrose generalized inverse, tr(·) denotes the trace operator, ln(·) denotes the natural log operator, and · denotes the Euclidean norm.
II. Neuroadaptive Control for Discrete-Time Nonlinear Uncertain Dynamical Systems With a Q-Modification Architecture In this section, we consider the problem of characterizing neuroadaptive full-state feedback control laws for discretetime nonlinear uncertain dynamical systems to achieve reference model trajectory tracking. Specifically, consider the controlled discrete-time nonlinear uncertain dynamical system G given by x(k + 1) = A0 x(k) + B(ˆx(k), u(k)) ˆ + BG(x(k))u(k) x(0) = x0 ,
k ∈ Z+
(1)
where x(k) ∈ Rn , k ∈ Z+ , is the state vector, u(k) ∈ Rm , k ∈ Z+ , is the control input, xˆ (k) x(k), x(k − 1), . . . , x(k − p + 1) is a vector of p-delayed values of the state vector with ˆ p ≥ 1, u(k) u(k − 1), u(k − 2), . . . , u(k − q) is a vector of q-delayed values of the control input with q ≥ 1, A0 ∈ Rn×n and B ∈ Rn×m are known matrices, G : Rn → Rm×m is a known input matrix function such that det G(x) = 0 for all x ∈ Rn , and : Rnp × Rmq → Rm is an unknown nonlinear function representing system uncertainty. Dynamical systems with uncertainty structures given by (1) are discussed in [20].
c 2010 IEEE 1045-9227/$26.00
1508
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
We assume that (ˆx(k), u(k)), ˆ k ∈ Z+ , can be perfectly parameterized as ˆ (ˆx(k), u(k)) ˆ = W T σ(ˆx(k), u(k)),
k ∈ Z+
(2)
where W ∈ Rl×m is an unknown matrix and σ : Rnp × Rmq → Rl is a known bounded Lipschitz continuous function such that, for all xˆ (k) ∈ Rnp and u(k) ˆ ∈ Rmq , k ∈ Z+ , σ(ˆx, u) ˆ ≤ ∗ ∗ σ , where σ > 0. Furthermore, we assume that the state x(k), k ∈ Z+ , is available for feedback. In order to achieve trajectory tracking, we construct a reference system Gref given by xref (k + 1) = Aref xref (k) + Bref r(k),
xref (0) = xref 0 ,
k ∈ Z+ (3)
where xref (k) ∈ Rn , k ∈ Z+ , is the reference state vector, r(k) ∈ Rr , k ∈ Z+ , is a bounded reference input, Aref ∈ Rn×n is Shur, and Bref ∈ Rn×r . Since all the eigenvalues of the matrix Aref lie in the open unit disk, it follows from Lemma 13.2 of [18] that there exists a positive-definite matrix P ∈ Rn×n such that P = ATref PAref + R
(4)
where R ∈ Rn×n is a positive-definite matrix. The goal here is to develop an adaptive control signal u(k), k ∈ Z+ , that guarantees x(k) → xref (k) as k → ∞. The following matching conditions are needed for the main result of this section. Assumption 1: There exist gains Kx ∈ Rm×n and Kr ∈ m×r such that A0 + BKx = Aref and BKr = Bref . R Consider the control law given by u(k) = G−1 (x(k))(un (k) − uad (k)),
k ∈ Z+
(5)
where un (k) = Kx x(k) + Kr r(k),
k ∈ Z+
(6)
and ˆ T (k)σ(ˆx(k), u(k)), ˆ uad (k) = W
k ∈ Z+
(7)
gives the output of the linearly parameterized neural network ˆ and W(k) ∈ Rl×m , k ∈ Z+ , is an update weight matrix. Using (5)–(7) and Assumption 1, the system dynamics (1) can be rewritten as ˜ T (k)σ(ˆx(k), u(k)) ˆ x(k + 1) = Aref x(k) + Bref r(k) + BW x(0) = x0 , k ∈ Z+ (8)
Fig. 1.
Visualization of Q-modification term.
where q(i) = σ(ˆx(i), u(i)), ˆ i = 1, . . . , s ˆ T (i)σ(ˆx(i), u(i)) c(i) = e(i + 1) − Aref e(i) + BW ˆ i = 1, . . . , s.
k ∈ Z+ . Next, define
˜ (k)σ(ˆx(k), u(k)), ˆ e(0) = e0 , k ∈ Z+ e(k + 1) = Aref e(k) + BW (9)
ˆ and note that if W(k), k ∈ Z+ , satisfies
where e0 x0 − xref 0 . Now, for every k ∈ Z+ , consider a linear subspace Ls , where s ≤ min{l, k}, formed by s linearly independent vectors q(i), i = 1, . . . , s, such that BW T q(i) = c(i),
i = 1, . . . , s
(10)
(12)
Note that (10) is a direct consequence of (9). Furthermore, note that q(i), i = 1, . . . , s, and c(i), i = 1, . . . , s, given by (11) and (12) are computable. Hence, although the matrix W is unknown, W satisfies a set of linear equations given by (10). Equation (10) represents a system of s equations in terms of the entries of W, where each of these equations characterizes an affine hyperplane. For example, in the case where n = 1, m = 1, l = 2, s = 1, W = [W1 , W2 ]T , and B = 1, the affine hyperplane (10) is described by a line Ls with q(i), i = 1, . . . , s, being a normal vector to Ls as shown in Fig. 1. Note that the distance from point A to point B shown in Fig. 1, ˆ which is the shortest distance from the weight estimate W(k) to affine hyperplane Ls defined by (10), is given by c(i) − ˆ T (k)q(i). W Next, define the error criterion γQ ˆ ˆ T (k)¯q(k) ρ(W(k), q¯ (k), c¯ (k)) tr c¯ (k) − BW 2 ˆ T (k)¯q(k) T , k ∈ Z+ · c¯ (k) − BW (13) s s where γQ > 0, q¯ (k) i=1 αi (k)q(i), c¯ (k) i=1 αi (k)c(i), and αi (k) ∈ R, i = 1, . . . , s, k ∈ Z+ , are design parameters. Note that (13) is a weighted sum of squares of the distances ˆ between the update weights W(k), k ∈ Z+ , and the family of affine hyperplanes defined by (10). Now, note that the gradient ˆ ˆ of ρ(W(k), q¯ (k), c¯ (k)), k ∈ Z+ , with respect to W(k), k ∈ Z+ , is given by ˆ ∂ρ(W(k), q¯ (k), c¯ (k)) ˆ T (k)¯q(k) T B = −γQ q¯ (k) c¯ (k) − BW ˆ ∂W(k)
˜ W − W. ˆ Defining the tracking error e(k) x(k) − where W xref (k), k ∈ Z+ , the error dynamics are given by T
(11)
ˆ T (k)¯q(k) T B, H(k) γQ q¯ (k) c¯ (k) − BW
ˆ T (k)¯q(k) = c¯ (k), BW
k ∈ Z+
k ∈ Z+
(14)
(15)
(16)
ˆ then H(k), k ∈ Z+ , is zero and the weight estimates W(k), k ∈ Z+ , lie on the collection of the affine hyperplanes defined ˆ by (10). If the weight estimates W(k), k ∈ Z+ , do not satisfy (16), then each nonzero row of the matrix H(k), k ∈ Z+ , is a vector that is orthogonal to the corresponding affine
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
hyperplane defined by (10) and points in the direction of the hyperplane. Finally, using (10), it follows that ˆ T (k)¯q(k)2 ≥ 0, k ∈ Z+ . ˜ tr H T (k)W(k) = γQ c¯ (k) − BW (17) Theorem 1: Consider the nonlinear uncertain dynamical system G given by (1). Assume Assumption 1 holds. Furthermore, assume that for a given γ > 0, there exist positivedefinite matrices P ∈ Rn×n and E ∈ Rm×m such that 1 Im > (E + BT PB) γ(c + σ ∗ 2 )
(18)
where Im is the m × m identity matrix and c > 0. Then, the neuroadaptive control law (5)–(7) with update law given by
1509
along the closed-loop system trajectories is given by ˜ V (e(k), W(k)) ˜ + 1)) − V (e(k), W(k)) ˜ V (e(k + 1), W(k T T ˜ (k)σ(ˆx(k), u(k)) ˆ P = ln 1 + Aref e(k) + BW ˜ T (k)σ(ˆx(k), u(k)) · Aref e(k) + BW ˆ
T − ln 1 + e (k)Pe(k) 1 ˜T ˜ + 1) − 1 tr W ˜ T (k)W(k) ˜ + tr W (k + 1)W(k γ γ
= ln 1 + eT (k) ATref PAref − P e(k) ˜ +2eT (k)ATref PBW(k)σ(ˆ x(k), u(k)) ˆ T T ˜ +σ (ˆx(k), u(k)) ˆ W(k)B PBW T (k)σ(ˆx(k), u(k)) ˆ −1 1 − tr [(k + 1) + Q(k + 1)]T · 1 + eT (k)Pe(k) γ ˜ · 2W(k) − (k + 1) − Q(k + 1) . (24)
k ∈ Z+ Next, let R > 0 and R > 0 be such that R + R = R and 1 2 1 2 (19) define T ˜ (25) ˆ W(k) , k ∈ Z+ η(k) eT (k), σ T (ˆx(k), u(k)) where (k) ∈ Rl×m , k ∈ Z+ , and Q(k) ∈ Rl×m , k ∈ Z+ , are given by
R2 −ATref PB M > 0. (26) E −BT PAref 1 (k + 1) = c + σ T (ˆx(k), u(k))σ(ˆ ˆ x(k), u(k)) ˆ Now, using (4) and the fact that ln(1 + a) ≤ a, a > −1, it ·σ(ˆx(k), u(k)) ˆ (20) follows from (24) that [e(k + 1) − Aref e(k)]T B†
ˆ + 1) = W(k) ˆ W(k + (k + 1) + Q(k + 1),
ˆ ˆ 0, W(0) =W
⎧ ⎨ H(k), if g(k) > 0 Q(k + 1) =
⎩
,
k ∈ Z+
(21)
0l×m , otherwise
and ˆ T (k)¯q(k)2 − tr T (k)H(k) g(k) γQ c¯ (k) − BW 1 (22) − tr H T (k)H(k), k ∈ Z+ 2 guarantees that there exists a positively invariant set Dα ⊂ Rn × Rl×m , with (0, W) ∈ Dα , such that the solution ˆ (e(k), W(k)) ≡ (0, W) of the closed-loop system given by (9), (5)–(7), and (19)–(21) is Lyapunov stable and e(k) → 0 as ˆ 0 ) ∈ Dα . k → ∞ for all (e0 , W Proof: To show Lyapunov stability of the closed-loop system (9), (5)–(7), and (19)–(21), consider the Lyapunov function candidate ˜ = ln(1 + eT Pe) + 1 tr W ˜ TW ˜ V (e, W) γ
(23)
where P > 0 satisfies (4) and γ > 0. Note that V (0, 0) = 0 ˜ > 0 for all and, since P is positive definite and γ > 0, V (e, W) ˜ (e, W) = 0. Now, letting e(k), k ∈ Z+ , denote the solution to (9) and using (19)–(22), it follows that the Lyapunov difference
˜ V (e(k), W(k)) T 1 e (k)R1 e(k) − ηT (k)Mη(k) ≤− T T 1 + e (k)Pe(k) 1 + e (k)Pe(k) 1 ˜ + σ T (ˆx(k), u(k)) ˆ W(k) E + BT PB T 1 + e (k)Pe(k) 1 ˜ T (k)σ(ˆx(k), u(k)) ˆ − tr [(k + 1) + Q(k + 1)]T ·W γ ˜ · 2W(k) − (k + 1) − Q(k + 1) . (27) Using (20), (21), and the fact that ˆ x(k), u(k)) ˆ σ T (ˆx(k), u(k))σ(ˆ < 1, T c + σ (ˆx(k), u(k))σ(ˆ ˆ x(k), u(k)) ˆ
c>0
it follows that ˜ V (e(k), W(k)) T 1 e (k)R1 e(k) − ηT (k)Mη(k) ≤− 1 + eT (k)Pe(k) 1 + eT (k)Pe(k) 1 ˜ + ˆ W(k) E + BT PB σ T (ˆx(k), u(k)) T 1 + e (k)Pe(k) ˜ T (k)σ(ˆx(k), u(k)) ·W ˆ 1 1 − ˆ σ T (ˆx(k), u(k)) T γ c + σ (ˆx(k), u(k))σ(ˆ ˆ x(k), u(k)) ˆ ˜ W ˜ T (k)σ(ˆx(k), u(k)) ·W(k) ˆ 2 ˜ − tr QT (k + 1)W(k) − T (k + 1)Q(k + 1) γ 1 − QT (k + 1)Q(k + 1) . 2
(28)
1510
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
Fig. 3.
Control input versus time.
Fig. 4.
Update weights versus time without the Q-modification controller.
Fig. 2. System states versus time with and without the Q-modification controller.
Finally, using (18), (21), and (17), it follows that ˜ V (e(k), W(k)) ≤ 0,
k ∈ Z+ .
Next, define ˜ ∈ Rn × Rl×m : V (e, W) ˜ ≤ α , α > 0. D˜ α (e, W)
(29)
(30)
˜ ˜ Since V (e(k), W(k)) ≤ 0 for all (e(k), W(k)) ∈ D˜ α and k ∈ Z+ , it follows that D˜ α is positively invariant. Now, since D˜ α is positively invariant, it follows that: ˆ ∈ Rn × Rl×m : (e, W ˆ − W) ∈ D˜ α (31) Dα (e, W) is also positively invariant. Furthermore, it follows from (29) ˆ and [5, Th. 13.10] that the solution (e(k), W(k)) ≡ (0, W) of the closed-loop system given by (9) and (19) is Lyapunov ˆ 0 ) ∈ Dα . stable and e(k) → 0 as k → ∞ for all (e0 , W 2 The term Q(k), k ∈ Z+ , in (19) given by (21) is a discretetime analogue of the Q-modification term introduced in [20] for continuous-time systems. As in the continuous-time case, Q(k), k ∈ Z+ , is an additional term that is introduced to the update law that is designed to minimize the error criterion given by (13) and is constructed based on information of the unknown weights W given by (10) and the property given by (17). Note that for every k ∈ Z+ , the vector Q(k) is directed ˆ ˆ opposite to the gradient ∂ρ(W(k), q¯ (k), c¯ (k))/∂W(k) and, in the case where the system uncertainty is a scalar function, is parallel to q¯ (k), which involves a linear combination of vectors normal to the affine hyperplanes defined by (10). Hence, Q(k), k ∈ Z+ , introduces a component in the update ˆ law (19) that drives the trajectory W(k), k ∈ Z+ , in such a way so that the error criterion given by (13) is minimized. For further details on the utility of the Q-modification term for uncertainty suppression and uncertainty cancelation see [20].
III. Illustrative Numerical Example In this section, we present a numerical example to demonstrate the utility and efficacy of the proposed Qmodification architecture for discrete-time neuroadaptive stabilization. Specifically, consider the nonlinear dynamical system
given by x(k + 1) = A0 x(k) + B(x(k)) + Bu(k), where
A0 =
0 −0.35
1 −0.25
x(0) = x0 ,
,
B=
0 1
k ∈ Z+ (32)
(x(k)) = W T σ(x(k)), k ∈ Z+ , and σ(x) = [sin(x1 ), cos(x2 ), cos(x1 ), sin(x2 ), x1 , x1 |x1 |]T is a known regressor vector. Note that eigenvalues of A0 are λ1 = −0.1250 − 0.5783j and λ2 = −0.1250 + 0.5783j, and hence, lie inside the unit disk. Here, our goal is to achieve stabilization of the uncertain system around the origin. Hence, the reference model is given by (3) with Aref = A0 , Bref = B, xref 0 = [0, 0]T , and r(k) ≡ 0. Next, we use Theorem 1 to design a neuroadaptive controller given by (5)–(7), with un (k) ≡ 0 and update laws given by (19)–(21). Now, with initial conditions x0 = [3, 4]T and ˆ 0 = 06×1 , and W = [1.00, −1.50, 2.50, 3.50, 1.00, 0.50]T , W Figs. 2–5 show the state trajectories, the control input, and the update weight trajectories versus time with and without the Qmodification term activated. It is clear that the Q-modification architecture results in a faster convergence and reduces system and weight oscillations. It is interesting to note that for this example the update weights for both controllers converge to the same values. However, in the presence of persistency of
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 9, SEPTEMBER 2010
Fig. 5.
Update weights versus time with the Q-modification controller.
excitation it can be shown that the update weights of the Qmodification controller converge to the ideal weights [20]. IV. Conclusion In this brief, we presented a new and novel neuroadaptive controller architecture for nonlinear discrete-time uncertain systems based on the approach introduced in [20]. As in the continuous-time case, this controller architecture provided fast adaptation to effectively suppress system uncertainty while avoiding large parameter (high gain) stabilization which can excite unmodeled system dynamics. Extensions of the present approach to nonlinear in the parameters neural networks as well as output feedback control follow as in [20]. References [1] F. C. Chen and H. K. Khalil, “Adaptive control of nonlinear systems using neural networks,” Int. J. Contr., vol. 55, no. 6, pp. 1299–1317, 1992. [2] F. C. Chen and H. K. Khalil, “Adaptive control of a class of nonlinear discrete-time systems using neural networks,” IEEE Trans. Autom. Contr., vol. 40, no. 5, pp. 791–807, May 1995. [3] H. Deng, H.-X. Lin, and Y.-H. Wu, “Feedback-linearization-based neural adaptive control for unknown nonaffine nonlinear discrete-time systems,” IEEE Trans. Neural Netw., vol. 19, no. 9, pp. 1615–1625, Sep. 2008. [4] S. S. Ge, T. H. Lee, G. Y. Li, and J. Zhang, “Adaptive NN control for a class of discrete-time non-linear systems,” Int. J. Contr., vol. 76, no. 4, pp. 334–354, 2003. [5] W. M. Haddad and V. Chellaboina, Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach. Princeton, NJ: Princeton Univ. Press, 2008. [6] K. J. Hunt, D. Sbarbaro, R. Zbikowski, and P. J. Gawthrop, “Neural networks for control: A survey,” Automatica, vol. 28, pp. 1083–1112, Nov. 1992. [7] C.-L. Hwang and C.-H. Lin, “A discrete-time multivariable neuroadaptive control for nonlinear unknown dynamic systems,” IEEE Trans. Syst. Man Cybern., vol. 30, no. 6, pp. 865–877, Dec. 2000. [8] S. Jagannathan and F. L. Lewis, “Multilayer discrete-time neural net controller with guaranteed performance,” IEEE Trans. Neural Netw., vol. 7, no. 1, pp. 107–130, Jan. 1996. [9] S. Jagannathan, “Control of a class of nonlinear discrete-time systems using multilayer neural networks,” IEEE Trans. Neural Netw., vol. 12, no. 5, pp. 1113–1120, Sep. 2001. [10] S. Jagannathan, Neural Network Control of Nonlinear Discrete-Time Systems. Boca Raton, FL: CRC, 2006. [11] A. U. Levin and K. S. Narendra, “Control of nonlinear dynamical systems using neural networks: Controllability and stabilization,” IEEE Trans. Neural Netw., vol. 4, no. 2, pp. 192–206, Mar. 1993.
1511
[12] F. L. Lewis, S. Jagannathan, and A. Yesildirak, Neural Network Control of Robot Manipulators and Nonlinear Systems. London, U.K.: Taylor and Francis, 1999. [13] L. Lin, P. N. Nikiforuk, and M. M. Gupta, “Fast neural learning and control of discrete-time nonlinear systems,” IEEE Trans. Syst. Man Cybern., vol. 25, no. 3, pp. 478–488, Mar. 1995. [14] K. S. Narendra and K. Parthasarathy, “Identification and control of dynamical systems using neural networks,” IEEE Trans. Neural Netw., vol. 1, no. 1, pp. 4–27, Mar. 1990. [15] J. Park and I. Sandberg, “Universal approximation using radial basis function networks,” Neural Comput., vol. 3, no. 2, pp. 246–257, 1991. [16] M. Polycarpou, “Stable adaptive neural control scheme for nonlinear systems,” IEEE Trans. Autom. Contr., vol. 41, no. 3, pp. 447–451, Mar. 1996. [17] R. Sanner and J. Slotine, “Gaussian networks for direct adaptive control,” IEEE Trans. Neural Netw., vol. 3, no. 6, pp. 837–864, Nov. 1992. [18] J. Spooner, M. Maggiore, R. Ordonez, and K. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques. New York: Wiley, 2002. [19] K. Volyanskyy, W. M. Haddad, and A. J. Calise, “A new neuroadaptive control architecture for nonlinear uncertain dynamical systems: Beyond σ and e-modifications,” in Proc. IEEE Conf. Dec. Contr., Cancun, Mexico, Dec. 2008, pp. 80–85. [20] K. Y. Volyanskyy, W. M. Haddad, and A. J. Calise, “A new neuroadaptive control architecture for nonlinear uncertain dynamical systems: Beyond σ and e-modifications,” IEEE Trans. Neural Netw., vol. 20, no. 11, pp. 1707–1723, Nov. 2009.
Real-Time Simulation of Biologically Realistic Stochastic Neurons in VLSI Hsin Chen, Member, IEEE, Sylvain Sa¨ıghi, Member, IEEE, Laure Buhry, and Sylvie Renaud, Member, IEEE
Abstract—Neuronal variability has been thought to play an important role in the brain. As the variability mainly comes from the uncertainty in biophysical mechanisms, stochastic neuron models have been proposed for studying how neurons compute with noise. However, most papers are limited to simulating stochastic neurons in a digital computer. The speed and the efficiency are thus limited especially when a large neuronal network is of concern. This brief explores the feasibility of simulating the stochastic behavior of biological neurons in a very large scale integrated (VLSI) system, which implements a programmable and configurable Hodgkin-Huxley model. By simply injecting noise to the VLSI neuron, various stochastic behaviors observed in biological neurons are reproduced realistically in VLSI. The noise-induced variability is further shown to enhance the signal modulation of a neuron. These results point toward the development of analog VLSI systems for exploring the stochastic behaviors of biological neuronal networks in large scale. Index Terms—Analog VLSI, Hodgkin-Huxley formalism, neuromorphic VLSI, noise, stochastic behavior, stochastic neurons. Manuscript received August 3, 2009; revised February 24, 2010; accepted April 15, 2010. Date of publication June 21, 2010; date of current version September 1, 2010. This work was supported in part by the National Science Council, Taiwan, under Grant 97-2918-I-007-003, and in part by the European Union in the FP6 Program under Grant 15879 (FACETS). H. Chen is with the Department of Electrical Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan (e-mail: [email protected]). S. Sa¨ıghi, L. Buhry, and S. Renaud are with the IMS Lab, University of Bordeaux, Talence F-33400, France (e-mail: [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNN.2010.2049028
c 2010 IEEE 1045-9227/$26.00
