Artificial Spiking Neurons and Analog-to-Digital-to-Analog Conversion
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1455
PAPER
Artificial Spiking Neurons and Analog-to-Digital-to-Analog Conversion Hiroyuki TORIKAI† , Member, Aya TANAKA†† , Nonmember, and Toshimichi SAITO†††a) , Member
SUMMARY This paper studies encoding/decoding function of artificial spiking neurons. First, we investigate basic characteristics of spiketrains of the neurons and fix parameter value that can minimize variation of spike-train length for initial value. Second we consider analog-to-digital encoding based upon spike-interval modulation that is suitable for simple and stable signal detection. Third we present a digital-to-analog decoder in which digital input is applied to switch the base signal of the spiking neuron. The system dynamics can be simplified into simple switched dynamical systems and precise analysis is possible. A simple circuit model is also presented. key words: spiking neurons, analog-to-digital converters, digital-toanalog converters
1.
Introduction
Artificial spiking neurons (ASNs) are nonlinear dynamical systems having rich dynamics and potential applications [1]–[5]. Repeating integrate-and-fire behavior between base and threshold signals, the ASN can exhibit a variety of periodic/chaotic spike-trains and related bifurcation phenomena. The ASNs can be building blocks of pulse-coupled neural networks (PCNNs) that have a variety of synchronization and bifurcation phenomena [6]–[10]. Potential applications of the ASNs and PCNNs are many, including image processing, associative memories, ultra-wide-band communication, and feature selection [7]–[13]. The ASNs and PCNNs have also been studied as simplified models of biological neurons in order to understand neural information processing function where spike-based coding plays important roles [14]–[16]. On the other hand, analog-to-digital and digital-toanalog converters (ADC and DAC) are indispensable systems in order for communication between real analog world and digital signal processing. There are various architectures of ADCs and DACs [17]–[20], e.g., cyclic ADC/DAC are known as basic systems to realize binary encoding/decoding. Improvement of resolution, stability and simplicity are in important issues. The main purpose of this paper is to provide fundamental considerations on spike-based encoding/decoding functions of the ASN and their applications to ADC and DAC. Manuscript received January 11, 2008. The author is with Osaka University, Toyonaka-shi, 560-8531 Japan. †† The author is with TOSHIBA, Tokyo, 105-8001 Japan. ††† The author is with Hosei University, Koganei-shi, 184-0002 Japan. a) E-mail: [email protected] DOI: 10.1093/ietfec/e91–a.6.1455 †
First, we introduce an ASN having periodic sawtooth base signal that can cause a variety of spike-trains [9]. We then analyze relationship between length of spike-trains and the number of spikes by theory and numerical experiments. Based on the analysis we can fix the parameter value that is suitable for spike-based ADC and DAC. Second, we present the ADC based on the ASN (ASN-ADC) where an analog input corresponds to an initial value of the state variable. We present spike-interval modulation (SIM) that encodes a spike-train into a binary sequence based on each inter-spike interval (ISI). Applying the mapping procedure, we clarify that the SIM can realize binary encoding. Third, we present the DAC based on the ASN (ASN-DAC) whose digital input is an inverse sequence of digital output of the ASN-ADC. The input is applied to an ASN as a binary signal that switches the dc component of a base signal. The spike-train is governed by a kind of iterated function systems (IFSs) [21], [22] and the operation of the ASN-DAC can be confirmed theoretically. Forth, we present a simple circuit model of the ASN-DAC with basic PSPICE simulation data. The circuit can be changed into a test circuit of ASN-ADC in [23] by a slight change of the circuit configuration. Preliminary results along these lines can be found in our conference paper [24]. As significance and novelty of this paper, please note the following: (i) The study of ASN-ADAC contributes to bridge between encoding mechanisms of spiking neurons [14]–[16] and ADC/DAC [17]–[20] via nonlinear dynamical system theories [21], [25]. It may be a trigger to develop a new signal processing framework. (ii) The SIM signal in the ASN-ADAC have practical advantages: the signal does not diverge, whereas capacitor voltage of traditional ADCs may diverge, the signal can be detected without phase-locking and thus it is robust for signal delay, and the signal is low power consumption such as the ultra-wide-band communication signals [11], [12]. (iii) The ASN-DAC can be regarded as a novel realization of IFS that can generate fractal patterns and has several applications [21]. (iv) This paper firstly discusses relationship between length of spike-trains and the number of spikes, and proposes the ASN-DAC based on the SIM. 2.
Artificial Spiking Neuron
Integrate-and-fire switching is basic in dynamics of various spiking neuron models including our ASN [23]. Figure 1
c 2008 The Institute of Electronics, Information and Communication Engineers Copyright
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1456
Fig. 1
Dynamics of artificial spiking neuron.
shows basic behavior of dimensionless version of the ASN. x is a state variable which corresponds to a membrane potential of neuron, and τ is a normalized time. The state x repeats integrate-and-fire behavior between a firing threshold th and a base signal b(τ). At each firing moment, the ASN outputs a spike y = 1. The time τ, the state x and the output y are given as dimensionless form, and a circuit model of the ASN is shown in Sect. 6. The dynamics of the ASN is described by the following equation: ⎧ dx ⎪ ⎪ ⎨ = 1, y(τ) = 0 for x(τ) < th, (1) ⎪ dτ ⎪ ⎩ x(τ ) = b(τ ), y(τ ) = 1 if x(τ) = th, +
+
+
where the threshold is normalized (th = 0), the base is period one (b(τ) = b(τ + 1)) and τ+ denotes just after time τ. When an initial state x(0) < th is given, x(τ) rises. If the state x(τ) reaches the threshold th, the state x(τ+ ) is reset to the base b(τ+ ) instantaneously and the ASN outputs a spike y(τ+ ) = 1. Let τn denotes the n-th spike position where n is a positive integer. The dynamics of the spike position τn can be described by the following spike position map g from positive reals to itself: τn+1 = g(τn ) ≡ τn − b(τn ).
(2)
Figure 2 shows a spike position map g. As shown in this figure, by iterating the spike position map g starting from the first spike position τ1 , we can calculate the sequence (τ1 , τ2 , τ3 , · · ·) successively [25]. As shape of the base signal b(τ) varies, various shapes of the spike position map g can be realized. Hence, by adjusting the base signal b(τ), the ASN can generate various periodic and chaotic spike-trains and can exhibit related bifurcation phenomena [1], [6]. In order to study encoding function of the ASN, in this paper we focus on sawtooth shaped base signals b(τ) as follows: b(τ) = −τ − α for 0.5 ≤ τ < 1.5, b(τ + 1) = b(τ)
(3)
where α > −0.5 is an offset parameter. The base signal b(τ) for α = 0 is shown in Fig. 1. Let θn ≡ τ n
(a)
(mod 1)
be the n-th spike phase. The dynamics of the spike phase θn can be described by the following phase map G from an interval [0, 1) into itself:
(b) Fig. 2 Maps corresponding to the ASN in Fig. 1. (a) Spike-position map g. The base signal b(τ) of the corresponding ASN is given by Eq. (3) for the case where its offset parameter is α = 0. (b) Phase map G.
θn+1 = G(θn ) = g(θn )
(mod 1).
(4)
Figure 2(b) shows the phase map G corresponding to the spike position map g in Fig. 2(a). • The phase map G for α = 0 in Fig. 2(b) can be regarded as the Bernoulli shift [25] for the spike phase θn . The Bernoulli shift G can realize conversion of the initial phase θn (analog input) into its binary number representation (digital output) based on the concept of symbolic dynamics [17], [25]. This is a basic idea for the spike position modulation (SPM) studied in [23]. However, in Sect. 4, we present a new approach for AD conversion function of the ASN based on the SIM. • In the previous work [23], we have studied basic characteristics of the spike-train y(t) for the case of α = 0 only. In Sect. 3, we study characteristic of the spike-train y(t) for various values of α and consider which value of α is suitable for the SIM.
3.
Basic Characteristics of Spike-Train
Let us analyze basic characteristics of the spike-train y(τ). Since b(τ) is period one, we can restrict the first spike position in
TORIKAI et al.: ADAC
1457
As a theoretical background of the local minima, we have the following. Proposition 1: Let y(τ) be generated by the ASN having the base signal b(τ) in Eq. (3) and let the number of spikes be l ≥ 2. Then, for α ∈ {0, 1, 2, · · ·}, the variation of the length of spike-trains having l spikes is DV (l) = 2 − 2−l+1 . Proof: First, let us consider the case α = 0. The spike position map g in Fig. 2(a) satisfies this case. As shown in this figure, let Ik ≡ [k − 1, k) be the k-th unit interval, where k = 1, 2, 3, · · ·. Also let the interval Ik be divided into the first half interval Ik− ≡ [k − 1, k − 0.5) and the second half interval Ik+ ≡ [k − 0.5, k). For α = 0, the spike-position map g is to be 2τn + 2 − k for τn ∈ Ik− , τn+1 = g(τn ) ≡ 2τn + 1 − k for τn ∈ Ik+ , (7) g : Ik → Ik+1 .
Spike-train length characteristic for l = 4.
Fig. 3
Recalling the restriction τ1 ∈ I1 for the first spike position, the spike positions satisfy the following property: τn ∈ In ,
n = 1, 2, 3, · · · .
That is, each n-th spike position τn exists on the n-th unit interval In . Hence we have the following relation: τ l = l − 1 + θl .
(8)
Under the condition a = 0, the spike phase map G is to be Fig. 4 Characteristics of spike-train length for l = 33. DA (l) and DV (l) are average and variation for initial state −1 ≤ x(0) < 1, respectively.
without less of generality. The first spike position τ1 is determined by the initial state: τ1 = −x(0) for − 1 < x(0) ≤ 0. Let D(l) denote length of a spike-train y(τ) consisting of l spikes: (5)
Figure 3 shows characteristic of the length D(l) with respect to the initial state x(0). It can be seen that the length D(l) for α = 1 has smaller variation than that for α = 1.4. In order to characterize the length D(l), let us define average and variation of D(l) as follows: 0 D(l)dx, DV (l) ≡ Dmax (l) − Dmin (l) (6) DA (l) = −1
where Dmax (l) and Dmin (l) denote the maximum and the minimum values of D(l) for −1 < x(0) ≤ 0, respectively. Figure 4 shows the average and the variation for l = 33. It can be seen that the average DA increases in proportion to α. On the other hand, the variation DV has complex characteristics and DV seems to have local minimum when α is an integer.
(mod 1)
(9)
as shown in Fig. 2(b). Referring to [25], we have θl = Gl (θ1 ) = 2l θ1
0 ≤ τ1 < 1
D(l) = τl − τ1 .
θn+1 = G(θn ) = 2θn
(mod 1).
(10)
Substituting τ1 = θ1 , Eqs. (10) and (8) into D(l) = τl − τ1 , we obtain D(l) = l − 1 + (2l τ1 (mod 1)) − τ1 . It follows from this equation that the length D(l) becomes the minimum Dmin (l) = l − 2 + 2−l when the first spike position is τ1 = −x(0) = 1 − 2−l (see Fig. 3(a)). Also, the length D(l) becomes the maximum Dmax (l) = l − 2−l when the first spike position is τ1 = −x(0) = 2−l (see Fig. 3 left). Therefore the variation DV (l) is to be DV (l) = Dmax (l) − Dmin (l) = 2 − 2−l+1 . That is, the proposition for α = 0 is proven. Second, let us consider the case α ∈ {0, 1, 2, · · ·}. Repeating a similar consideration to the case of α = 0, we have τl = (α + 1)l − 1 + θl .
(11)
The spike phase map G is given by Eq. (9) and satisfies the relation in Eq. (10). Then we have D(l) = (α + 1)l − 1 + (2l θ1 (mod 1)) − θ1 . Repeating a similar consideration to the case of α = 0, we have Dmin (l) = (α + 1)l − 2 + 2−l and Dmax (l) = (α + 1)l − 2−l . Therefore the variation becomes DV (l) = Dmax (l)−Dmin (l) = 2 − 2−l+1 . Q.E.D.
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1458
Remark 2: • In the case of 0 < α < 1, the spike position map g satisfies a property g : Ik → Ik+1 ∪ Ik+2 . That is, the spike position τn can distribute in a wider range Ik+1 ∪ Ik+2 than the range Ik+1 of the spike position map g for α = 0. This property g : Ik → Ik+1 ∪ Ik+2 is a necessary condition for DV (l) > 2 for all l and α. Detailed analysis (e.g., sufficient condition for DV (l) > 2) is complicated and should be included in future problems. Note that small DV is convenient for the spike-based AD conversion because encoded spike-trains have similar length D(l). Also, small DA is convenient because the coded spiketrains have shorter length D(l). Hence hereafter we fix the offset parameter to α = 0. • If α = 0, the spike position map g satisfies g(Ik− ) = + g(Ik ) = Ik+1 . Such map is referred to as Markov-type: a simple and useful class of maps in application to information coding [11], [12], [22]. Since the slope of g is 2 and is expanding, any spike-train is unstable for the initial state x(0) and is to be chaotic in the sense of dynamical systems defined for unbounded time 0 < τ. However, Ref. [23] presents a circuit in which the spike-trains can be stabilized by adding higher frequency sawtooth signal to the threshold. 4.
Analog-to-Digital Conversion
In this section we study the ASN-ADC based on the SIM. The initial state x(0) of the ASN is used as an analog input. The ASN generates a spike-train y(τ) characterized by the spike positions (τ1 , τ2 , · · · , τl+1 ) whose pattern depends on the analog input x(0). Let us define the n-th ISI by dn = τn+1 − τn .
x ≥ 0. Then we have the following theorem. Theorem 1: The estimation X˜ in Eq. (14) approximates the analog input x(0) as 1 X˜ = − l Int(−2l x(0)). 2
(15)
That is, the analog input x(0) ∈ (−1, 0] is converted into one of the lattice points X ∈ {0, −1·2−l , −2·2−l , · · · , (2l −1)·2−l }. Proof: Since Eq. (7) is equivalent to the Bernoulli shift as shown in Fig. 2(b), binary encoding of θ1 can be realized [25]: l 1, for τn ∈ I1− 1 l −n Int(2 θ ) = P 2 , P = (16) 1 n n 2l 0, for τn ∈ I1+ n=1 where l is the code length. This encoding is referred to as the SPM [23]. Since the spike position map g has periodicity g(τ + 1) = g(τ) + 1, the ISI dn = g(τn ) − τn can be written by a function of the spike phase θn as follows: dn = g(θn ) − θn ≡ h(θn )
(17)
where h is a mapping from I1 to J ≡ [0.5, 1.5] as shown in Fig. 5(a). Note that J is the existence region of the ISI dn . Substituting Eq. (17) into Eq. (7), we obtain θn + 1 for τn ∈ I1− , (18) dn = h(θn ) ≡ θn for τn ∈ I1+ . Note that h is one-to-one and onto as shown in Fig. 5(a). Using h, we can confirm that the SPM in Eq. (16) is equivalent
(12)
To the sequence (d1 , d2 , · · · , dl ) of ISIs, we apply the following SIM 0 for dn ∈ [1, 0.5], Yn = (13) 1 for dn ∈ [0.5, 1), and obtain a digital output Y = (Y1 , Y2 , · · · , Yl ). We can summarize relations between the ASN as ADC and the ASN as neuron model in Table 1. We note that, to obtain the digital sequence with the code length l, the SIM requires the operating time (l + 1). Let us define the estimation of the analog input x(0) by X˜ = −
l
Yn 2−n .
(a)
(14)
n=1
Let a function Int(x) denote the integer part of a real number
x x(0) y(τ) l Y
Table 1 ADC versus neuron model. ASN as ADC ASN as neuron model State variable Membrane potential Analog input Initial state Spike-train for encoding Firing spike-train Code length Number of ISIs Digital output Temporal code by SIM
(b) Fig. 5 Description of spike intervals. (a) Spike position to interval conversion h. (b) Spike interval map F.
TORIKAI et al.: ADAC
1459
to the SIM in Eq. (13), i.e., Pn = Yn .
(19)
Recall that the first spike phase θ1 and the analog input x(0) have the following relation: θ1 = −x(0),
x(0) ∈ (−1, 0].
(20)
From Eqs. (20), (19), (16) and (14), we can derive Eq. (15). Q.E.D. From a mathematical viewpoint, the SIM is equivalent to the SPM when h is one-to-one. However, from a practical viewpoint, the SIM has several advantages such as receiving without clock and deep relation to biological spiking neurons as discussed in Sect. 1. If h is not one-to-one, the SIM is not equivalent to SPM and that may cause interesting function. Consideration of such SIM is in future problems. In order to visualize the SIM, we note that dn+1 is given by dn through the map h as follows. dn+1 = F(dn ) ≡ h ◦ g ◦ h−1 (dn ) = ⎧ ⎪ for dn ∈ [0.5, 0.75), ⎪ ⎪ 2dn , ⎨ 2dn − 1, for dn ∈ [0.75, 1.25), ⎪ ⎪ ⎪ ⎩ 2d − 2, for d ∈ [1.25, 1.5]. n n
(21)
The sequence {d1 , d2 , · · · , dn } of ISIs is governed by the map F. The SIM can be visualized as shown in Fig. 5(b). 5.
Digital-to-Analog Conversion
Here we present the ASN-DAC that realizes inverse operation of the ASN-ADC. Let us assume that the ASN-ADC converts an analog input x(0) into a digital output Y. The required operation for the ASN-DAC is to convert Y into an approximation of the analog input x(0). In order to realize the DA conversion, inverse digital sequence of Y is used as an input, i.e., the digital input to the ASN-DAC is defined by
Y ≡
{Y1 , · · · , Yl }
= {Yl , · · · , Y1 }.
(22)
Using Y we define a continuous-time signal u(τ) that is to be an input to the ASN-DAC: u(τ) = Yn for τ ∈ In ≡ [n − 1, n)
(23)
where 0 ≤ τ < l and n = 1 ∼ l. We then present the dynamics of the ASN-DAC as follows. ⎧ d x˜ ⎪ ⎪ ⎪ =1 for x˜(τ) < th, ⎨ dτ ⎪ ⎪ ⎪ +
+ ⎩ x˜(τ ) = b (τ ) if x˜(τ) = th, (24) b (τ) = b (τ) + 0.5u(τ) 1
b1 (τ) = 0.5τ − 1.5 for 0 ≤ τ < 1 b1 (τ + 1) = b1 (τ) where th = 0, and the initial value x˜(0) can be given arbitrarily in (−1, 0]. b1 (τ) is a periodic sawtooth signal with rising slope. Adding the input signal u(τ) to this b1 (τ), the base
Fig. 6
Dynamics of ASN-DAC (Digital input: Yn = (0, 0, 1)).
Y’ l x˜(0)
Table 2 DAC versus neuron model. ASN as DAC ASN as neuron model Digital input Modulation signal to base b(τ) Code length Number of modulations to b(τ) Initial value that is Initial state of arbitrary in (−1, 0] membrane potential Analog output Membrane potential at τ = l
x˜(l)
signal b (τ) is obtained. When u(τ) is positive (Yn = 1) for τ ∈ In , the offset of the base signal is increased as shown in Fig. 6. The integrate-and-fire switching is the same as that of the ASN-ADC: x˜(τ) rises, x˜ is reset to the base b (τ) just after x˜ reaches th, and the system repeats the integrate-and-fire behavior between th and b (τ). The state x˜(l) at τ = l is used as an analog output. Note that the system is not required to output spikes because neither digital output nor SIM is required in the ASN-DAC. We can summarize relations between the ASN as DAC and the ASN as neuron model in Table 2. Then we have the following theorem. Theorem 2: The difference (error) between the analog output x˜(l) of the ASN-DAC and the analog input x(0) of the ASN-ADC satisfies | x˜(l) − x(0)| < 2−l . Proof: First, we consider the ASN-ADC. Let the first spike phase θ1 of the ASN-ADC is in [2−l m, 2−l (m + 1)), where m ∈ {0, 1, 2, · · · , 2l − 1}. This is equivalent to the following situation: 2−l m ≤ −x(0) < 2−l (m + 1).
(25)
In this case, from Theorem 1, the digital output Y of the ASN-ADC is to be the binary number representation of m. Second, we consider the ASN-DAC. Repeating in a likewise manner as derivation of Eq. (2), we obtain the following spike position map g−1 driven by the digital input Y : (τn + n + 1)/2, for Yn = 0, τn+1 = g−1 (τn ) ≡ (26) (τn + n + 2)/2, for Yn = 1, where g−1 (Ik ) = Ik+1 . The spike position map g−1 is shown in Fig. 7(a). The spike position satisfies τn ∈ In for all n since the first spike position τ1 is in [0, 1). Then we have τ n = n − 1 + θn .
(27)
The spike phase θn is governed by the following map G−1 driven by the digital input Y : θn /2 for Yn = 0, −1 θn+1 = G (θn ) ≡ (28) (θn + 1)/2 for Yn = 1.
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1460
Fig. 8 ASN-ADAC circuit. ICs are TLC4066ID (switch), LM339 (comparator) and CD4538B (monostable multivibrator).
The two voltage sources B1 (t) and U(t) are connected in series and are assumed to be synchronized with a clock signal of period T . In PSPICE simulation B1 (t) and U(t) are given by signal generators. The circuit dynamics is described by ⎧ ⎪ dv ⎪ ⎪ for v < T h, ⎪ ⎪ ⎨ C dt = I0 (31) ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ v(t+ ) = B (t+ ) if v(t) = T h,
(a)
B (t) ≡ B1 (t) + U(t)
(b) Fig. 7 (a) Spike-position map g−1 of ASN-DAC. Digital input is Y = (0, 0, 1). (b) Phase map G−1 of ASN-DAC.
A phase map G−1 is shown in Fig. 7(b). The phase map G−1 can realize inverse operation of the phase map G of the ASN-ADC (Eq. (7)). (G) That is, the map G−1 is to be inverse of the Bernoulli shift and can be regarded as a kind of IFS. Referring to [22], the spike phase θl exists in the interval [2−l m, 2−l (m + 1)). This is equivalent to the following situation: 2−l m ≤ − x˜(l) < 2−l (m + 1).
(29)
From Eqs. (29) and (25), we have | x˜(l) − x(0)| < 2−l . Q.E.D. 6.
Figure 8 shows a simple circuit model of ASN-DAC. In the figure, U(t) is a input signal given by digital input Y and B1 (t) is a periodic sawtooth signal with period T : : U(t) =
U(t + lT ) = U(t)
and b (τ) = where
C(B (T τ)−T h) , I0 T
Eq. (31) is transformed into Eq. (24)
b (τ) = b1 (τ) + 0.5u(τ)
1 u(τ) = 2 CK T I0 Yn for n − 1 ≤ τ < n
Circuit Model
K1 Yn
where T h is a threshold. For an initial value v(0) ∈ (−VD /2, 0], the capacitor voltage v rises by integrating a constant current I0 . When v reaches T h, the comparator triggers the monostable multivibrator MM to output a single spike Y. The spike Y closes a switch S and the v is reset to the base B1 (t) + U(t). It goes without saying that the voltage sources B1 (t) + U(t) must have inner resistor r0 and the reset is completed within finite time. However we ignore r0 and assume instantaneous reset in Eq. (1) for simplicity: it is a routine in analysis of switching circuit. v repeats this integrate-and-fire behavior. If U(t) is positive (Yn = 1) for t ∈ [(n − 1)T, nT ) then the level of B(t) is up. The analog output is given by the voltage at time t = lT : −v(lT ). Ush) ing the following dimensionless variables τ = Tt , x = C(v−T I0 T
for (n − 1)T ≤ t < nT
C(VD +T h) 2 for n − 1 ≤ τ < n b1 (τ) = CK I0 τ − T I0
n = 1 ∼ l, b (τ + l) = b (τ),
and scaling parameters are fixed: K1 = 0.5
n=1∼l (30)
B1 (t) = K2 t − VD for 0 ≤ t < T B1 (t + T ) = B1 (t) where K1 , K2 and VD are scaling parameter fixed afterward.
T I0 I0 T I0 , K2 = 0.5 , VD + T h = 1.5 C C C
We have confirmed basic circuit operations in PSPICE simulation and a typical data is shown in Fig. 9. It should be noted that this circuit is transformed into a model of ASNADC by replacing B1 (t)+U(t) with a periodic sawtooth voltage source proportional to b(τ) and adding a higher periodic signal to T h for stabilization. The actual circuit can be found
TORIKAI et al.: ADAC
1461
Fig. 9 Typical waveform of ASN-DAC for l = 3, digital input Yn = (1, 1, 0). The analog output is −v(3T ) 0.56 [V]. Parameter values are C = 20 [nF] (IC = 0.3), I0 = 0.20 [mA], T = 0.10 [ms], T h 0 [V]. B1 (t) = VSRC (TRAN=pulse(−1.5 V −1 V 0 0.1 ms 0 0 0.1 ms)), U(t) = VSRC (TRAN=pulse(0 V 0.5 V 0 0 0 0.2 ms 0.3 ms)). The operation period is 3T and the initial value is reset every 3T .
in [23] with typical experimental data. We note that the circuit model in Fig. 8 is presented as a breadboard prototype and is not intended to a practical implementation. The practical circuit designs are included in future problems. We also note that the system equations (1) and (27) are presented as dimensionless forms as is often done for analysis of nonlinear dynamical circuits [9], [10], [23]. The dimensionless equations can describe dynamics of the circuit Eq. (34) for wide range of parameter values. 7.
Conclusions
We have studied encoding/decoding properties based on integrate-and-fire dynamics of ASN. We have investigated parameter dependence on spike-train length and have fixed the parameter α = 0 that can realize ASN-ADC with the minimum variation and average of spike-train length. In the ASN-ADC, we have proposed the SIM that has several advantages including robustness for delay. In the ASN-DAC, digital input signal is applied to adjust base level of the ASN and inverse operation of the ASN-ADC is achieved. A simple circuit model is also presented. (a) Circuit design of the ASN-ADAC for with consideration of nonidealities. (b) Analysis of relation between spike-train length and the number of spikes in various spiking neuron models. (c) Analysis of SIM characteristics of a variety of spike-trains along with the change of b(τ); and classification of encoding/decoding schemes for various spike-trains. References [1] R. Perez and L. Glass, “Bistability, period doubling bifurcations and chaos in a periodically forced oscillator,” Phys. Lett. A, vol.90, no.9, pp.441–443, 1982. [2] J.P. Keener, F.C. Hoppensteadt, and J. Rinzel, “Integrate-and-fire models of nerve membrane response to oscillatory input,” SIAM J. Appl. Math., vol.41, pp.503–517, 1981. [3] E.M. Izhikevich, Dynamical systems in neuroscience, MIT Press, 2007.
[4] H. Torikai, T. Saito, and W. Schwarz, “Synchronization via multiplex pulse-train,” IEEE Trans. Circuits Syst. I, vol.46, no.9, pp.1072–1085, 1999. [5] G. Lee and N.H. Farhat, “The bifurcating neuron network 2,” Neural Netw., vol.15, no.1, pp.69–84, 2002. [6] Y. Kon’no, T. Saito, and H. Torikai, “Rich dynamics of pulsecoupled spiking neurons with a triangular base signal,” Neural Netw., vol.18, pp.523–531, 2005. [7] S.R. Campbell, D. Wang, and C. Jayaprakash, “Synchrony and desynchrony in integrate-and-fire oscillators,” Neural Comput., vol.11, pp.1595–1619, 1999. [8] E.M. Izhikevich, “Weakly pulse-coupled oscillators, FM interactions, synchronization, and oscillatory associative memory,” IEEE Trans. Neural Netw., vol.10, no.3, pp.508–526, 1999. [9] H. Torikai and T. Saito, “Synchronization phenomena in pulsecoupled networks driven by spike-train inputs,” IEEE Trans. Neural Netw., vol.15, no.2, pp.337–347, 2004. [10] H. Nakano and T. Saito, “Grouping synchronization in a pulsecoupled network of chaotic spiking oscillators,” IEEE Trans. Neural Netw., vol.15, no.5, pp.1018–1026, 2004. [11] M. Sushchik, N. Rulkov, L. Larson, L. Tsimring, H. Abarbanel, K. Yao, and A. Volkovskii, “Chaotic pulse position modulation: A robust method of communicating with chaos,” IEEE Commun. Lett., vol.4, pp.128–130, 2000. [12] G.M. Maggio, N. Rulkov, and L. Reggiani, “Pseudo-chaotic time hopping for UWB impulse radio,” IEEE Trans. Circuits Syst. I, vol.48, no.12, pp.1424–1435, 2001. [13] A. Linares-Barranco, D. Cascado, G. Jimenez, A. Civit, M. Oster, and B. Linares-Barranco, “Poisson AER generator: Inter-spikeintervals analysis,” Proc. IEEE/ISCAS 2006, pp.3149–3152, 2006. [14] M. Christen, A. Kern, A. Nikitchenko, W.H. Steeb, and R. Stoop, “Fast spike pattern detection using the correlation integral,” Phys. Rev. E, vol.70, 011901, 2004. [15] T. Toyoizumi, K. Aihara, and S. Amari, “Fisher information for spike-based population coding,” Phys. Rev. Lett., vol.97, 098102, 2006. [16] B. Lindner, M.J. Chacron, and A. Longtin, “Integrate-and-fire neurons with threshold noise: A tractable model of how interspike interval correlations affect neuronal signal transmission,” Phys. Rev. E, vol.72, 021911, 2005. [17] P.G.A. Jespers, Integrated converters, Oxford Univ. Press, 2001. [18] L. Chiaburu and S. Signell, “An improved binary algorithmic A/D converter architecture,” Proc. IEEE/ISCAS, I-865–I-868, 2003. [19] G. Cauwenberghs, “A micropower CMOS algorithmic A/D/A converter,” IEEE Trans. Circuits Syst. I, vol.42, pp.913–919, 1995. [20] L. Weyten and S. Audenaert, “Two-capacitor DAC with compensative switching,” Electron. Lett., vol.31, pp.1435–1437, 1995. [21] H.J. Jeffrey, “Chaos game representation of gene structure,” Nucleic Acids Research, vol.18, pp.2163–2170, 1990. [22] T. Saito, J. Shimakawa, and H. Torikai, “D/A converters and iterated function systems,” Nonlinear Dynamics, vol.44, pp.37–43, 2006. [23] H. Hamanaka, H. Torikai, and T. Saito, “Quantized spiking neuron with A/D converision functions,” IEEE Trans. Circuits Syst. II, vol.53, no.10, pp.1049–1053, 2006. [24] A. Tanaka, H. Torikai, and T. Saito, “A/D and D/A converters by spike-interval modulation of simple spiking neurons,” Proc. IEEE/ISCAS, pp.3113–3116, 2006. [25] E. Ott, Chaos in Dynamical Systems, Cambridge Univ. Press., 1993.
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1462
Hiroyuki Torikai received B.E., M.E. and Ph.D. degrees in electrical engineering all from Hosei University, Tokyo, Japan, in 1995, 1997 and 1999, respectively. He is currently an associate professor at the Graduate School of Engineering Science, Osaka University, Osaka, Japan. His research interests are in neural networks, discrete-state dynamical systems, chaos and bifurcation. He is a member of the INNS.
Aya Tanaka received the B.E. and M.E. degrees in electrical engineering all from Hosei University, Tokyo, Japan, in 2005 and 2007, respectively. In 2007 she joined Toshiba Co., Ltd., Tokyo, Japan. Her current research interests are in neural networks and signal processing.
Toshimichi Saito received the B.E., M.E. and Ph.D. degrees in electrical engineering from Keio University, Yokohama, Japan, in 1980, 1982 and 1985, respectively. He is currently a Professor at Hosei University. His research interests include chaos and bifurcation, neural networks, evolutionary computation and power electronics. He served in several editorial boards including the IEEE Trans. Circuits Syst. I (2000–2001) and II (2003–2005). He is a member of the JNNS and a senior member of the IEEE.
1455
PAPER
Artificial Spiking Neurons and Analog-to-Digital-to-Analog Conversion Hiroyuki TORIKAI† , Member, Aya TANAKA†† , Nonmember, and Toshimichi SAITO†††a) , Member
SUMMARY This paper studies encoding/decoding function of artificial spiking neurons. First, we investigate basic characteristics of spiketrains of the neurons and fix parameter value that can minimize variation of spike-train length for initial value. Second we consider analog-to-digital encoding based upon spike-interval modulation that is suitable for simple and stable signal detection. Third we present a digital-to-analog decoder in which digital input is applied to switch the base signal of the spiking neuron. The system dynamics can be simplified into simple switched dynamical systems and precise analysis is possible. A simple circuit model is also presented. key words: spiking neurons, analog-to-digital converters, digital-toanalog converters
1.
Introduction
Artificial spiking neurons (ASNs) are nonlinear dynamical systems having rich dynamics and potential applications [1]–[5]. Repeating integrate-and-fire behavior between base and threshold signals, the ASN can exhibit a variety of periodic/chaotic spike-trains and related bifurcation phenomena. The ASNs can be building blocks of pulse-coupled neural networks (PCNNs) that have a variety of synchronization and bifurcation phenomena [6]–[10]. Potential applications of the ASNs and PCNNs are many, including image processing, associative memories, ultra-wide-band communication, and feature selection [7]–[13]. The ASNs and PCNNs have also been studied as simplified models of biological neurons in order to understand neural information processing function where spike-based coding plays important roles [14]–[16]. On the other hand, analog-to-digital and digital-toanalog converters (ADC and DAC) are indispensable systems in order for communication between real analog world and digital signal processing. There are various architectures of ADCs and DACs [17]–[20], e.g., cyclic ADC/DAC are known as basic systems to realize binary encoding/decoding. Improvement of resolution, stability and simplicity are in important issues. The main purpose of this paper is to provide fundamental considerations on spike-based encoding/decoding functions of the ASN and their applications to ADC and DAC. Manuscript received January 11, 2008. The author is with Osaka University, Toyonaka-shi, 560-8531 Japan. †† The author is with TOSHIBA, Tokyo, 105-8001 Japan. ††† The author is with Hosei University, Koganei-shi, 184-0002 Japan. a) E-mail: [email protected] DOI: 10.1093/ietfec/e91–a.6.1455 †
First, we introduce an ASN having periodic sawtooth base signal that can cause a variety of spike-trains [9]. We then analyze relationship between length of spike-trains and the number of spikes by theory and numerical experiments. Based on the analysis we can fix the parameter value that is suitable for spike-based ADC and DAC. Second, we present the ADC based on the ASN (ASN-ADC) where an analog input corresponds to an initial value of the state variable. We present spike-interval modulation (SIM) that encodes a spike-train into a binary sequence based on each inter-spike interval (ISI). Applying the mapping procedure, we clarify that the SIM can realize binary encoding. Third, we present the DAC based on the ASN (ASN-DAC) whose digital input is an inverse sequence of digital output of the ASN-ADC. The input is applied to an ASN as a binary signal that switches the dc component of a base signal. The spike-train is governed by a kind of iterated function systems (IFSs) [21], [22] and the operation of the ASN-DAC can be confirmed theoretically. Forth, we present a simple circuit model of the ASN-DAC with basic PSPICE simulation data. The circuit can be changed into a test circuit of ASN-ADC in [23] by a slight change of the circuit configuration. Preliminary results along these lines can be found in our conference paper [24]. As significance and novelty of this paper, please note the following: (i) The study of ASN-ADAC contributes to bridge between encoding mechanisms of spiking neurons [14]–[16] and ADC/DAC [17]–[20] via nonlinear dynamical system theories [21], [25]. It may be a trigger to develop a new signal processing framework. (ii) The SIM signal in the ASN-ADAC have practical advantages: the signal does not diverge, whereas capacitor voltage of traditional ADCs may diverge, the signal can be detected without phase-locking and thus it is robust for signal delay, and the signal is low power consumption such as the ultra-wide-band communication signals [11], [12]. (iii) The ASN-DAC can be regarded as a novel realization of IFS that can generate fractal patterns and has several applications [21]. (iv) This paper firstly discusses relationship between length of spike-trains and the number of spikes, and proposes the ASN-DAC based on the SIM. 2.
Artificial Spiking Neuron
Integrate-and-fire switching is basic in dynamics of various spiking neuron models including our ASN [23]. Figure 1
c 2008 The Institute of Electronics, Information and Communication Engineers Copyright
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1456
Fig. 1
Dynamics of artificial spiking neuron.
shows basic behavior of dimensionless version of the ASN. x is a state variable which corresponds to a membrane potential of neuron, and τ is a normalized time. The state x repeats integrate-and-fire behavior between a firing threshold th and a base signal b(τ). At each firing moment, the ASN outputs a spike y = 1. The time τ, the state x and the output y are given as dimensionless form, and a circuit model of the ASN is shown in Sect. 6. The dynamics of the ASN is described by the following equation: ⎧ dx ⎪ ⎪ ⎨ = 1, y(τ) = 0 for x(τ) < th, (1) ⎪ dτ ⎪ ⎩ x(τ ) = b(τ ), y(τ ) = 1 if x(τ) = th, +
+
+
where the threshold is normalized (th = 0), the base is period one (b(τ) = b(τ + 1)) and τ+ denotes just after time τ. When an initial state x(0) < th is given, x(τ) rises. If the state x(τ) reaches the threshold th, the state x(τ+ ) is reset to the base b(τ+ ) instantaneously and the ASN outputs a spike y(τ+ ) = 1. Let τn denotes the n-th spike position where n is a positive integer. The dynamics of the spike position τn can be described by the following spike position map g from positive reals to itself: τn+1 = g(τn ) ≡ τn − b(τn ).
(2)
Figure 2 shows a spike position map g. As shown in this figure, by iterating the spike position map g starting from the first spike position τ1 , we can calculate the sequence (τ1 , τ2 , τ3 , · · ·) successively [25]. As shape of the base signal b(τ) varies, various shapes of the spike position map g can be realized. Hence, by adjusting the base signal b(τ), the ASN can generate various periodic and chaotic spike-trains and can exhibit related bifurcation phenomena [1], [6]. In order to study encoding function of the ASN, in this paper we focus on sawtooth shaped base signals b(τ) as follows: b(τ) = −τ − α for 0.5 ≤ τ < 1.5, b(τ + 1) = b(τ)
(3)
where α > −0.5 is an offset parameter. The base signal b(τ) for α = 0 is shown in Fig. 1. Let θn ≡ τ n
(a)
(mod 1)
be the n-th spike phase. The dynamics of the spike phase θn can be described by the following phase map G from an interval [0, 1) into itself:
(b) Fig. 2 Maps corresponding to the ASN in Fig. 1. (a) Spike-position map g. The base signal b(τ) of the corresponding ASN is given by Eq. (3) for the case where its offset parameter is α = 0. (b) Phase map G.
θn+1 = G(θn ) = g(θn )
(mod 1).
(4)
Figure 2(b) shows the phase map G corresponding to the spike position map g in Fig. 2(a). • The phase map G for α = 0 in Fig. 2(b) can be regarded as the Bernoulli shift [25] for the spike phase θn . The Bernoulli shift G can realize conversion of the initial phase θn (analog input) into its binary number representation (digital output) based on the concept of symbolic dynamics [17], [25]. This is a basic idea for the spike position modulation (SPM) studied in [23]. However, in Sect. 4, we present a new approach for AD conversion function of the ASN based on the SIM. • In the previous work [23], we have studied basic characteristics of the spike-train y(t) for the case of α = 0 only. In Sect. 3, we study characteristic of the spike-train y(t) for various values of α and consider which value of α is suitable for the SIM.
3.
Basic Characteristics of Spike-Train
Let us analyze basic characteristics of the spike-train y(τ). Since b(τ) is period one, we can restrict the first spike position in
TORIKAI et al.: ADAC
1457
As a theoretical background of the local minima, we have the following. Proposition 1: Let y(τ) be generated by the ASN having the base signal b(τ) in Eq. (3) and let the number of spikes be l ≥ 2. Then, for α ∈ {0, 1, 2, · · ·}, the variation of the length of spike-trains having l spikes is DV (l) = 2 − 2−l+1 . Proof: First, let us consider the case α = 0. The spike position map g in Fig. 2(a) satisfies this case. As shown in this figure, let Ik ≡ [k − 1, k) be the k-th unit interval, where k = 1, 2, 3, · · ·. Also let the interval Ik be divided into the first half interval Ik− ≡ [k − 1, k − 0.5) and the second half interval Ik+ ≡ [k − 0.5, k). For α = 0, the spike-position map g is to be 2τn + 2 − k for τn ∈ Ik− , τn+1 = g(τn ) ≡ 2τn + 1 − k for τn ∈ Ik+ , (7) g : Ik → Ik+1 .
Spike-train length characteristic for l = 4.
Fig. 3
Recalling the restriction τ1 ∈ I1 for the first spike position, the spike positions satisfy the following property: τn ∈ In ,
n = 1, 2, 3, · · · .
That is, each n-th spike position τn exists on the n-th unit interval In . Hence we have the following relation: τ l = l − 1 + θl .
(8)
Under the condition a = 0, the spike phase map G is to be Fig. 4 Characteristics of spike-train length for l = 33. DA (l) and DV (l) are average and variation for initial state −1 ≤ x(0) < 1, respectively.
without less of generality. The first spike position τ1 is determined by the initial state: τ1 = −x(0) for − 1 < x(0) ≤ 0. Let D(l) denote length of a spike-train y(τ) consisting of l spikes: (5)
Figure 3 shows characteristic of the length D(l) with respect to the initial state x(0). It can be seen that the length D(l) for α = 1 has smaller variation than that for α = 1.4. In order to characterize the length D(l), let us define average and variation of D(l) as follows: 0 D(l)dx, DV (l) ≡ Dmax (l) − Dmin (l) (6) DA (l) = −1
where Dmax (l) and Dmin (l) denote the maximum and the minimum values of D(l) for −1 < x(0) ≤ 0, respectively. Figure 4 shows the average and the variation for l = 33. It can be seen that the average DA increases in proportion to α. On the other hand, the variation DV has complex characteristics and DV seems to have local minimum when α is an integer.
(mod 1)
(9)
as shown in Fig. 2(b). Referring to [25], we have θl = Gl (θ1 ) = 2l θ1
0 ≤ τ1 < 1
D(l) = τl − τ1 .
θn+1 = G(θn ) = 2θn
(mod 1).
(10)
Substituting τ1 = θ1 , Eqs. (10) and (8) into D(l) = τl − τ1 , we obtain D(l) = l − 1 + (2l τ1 (mod 1)) − τ1 . It follows from this equation that the length D(l) becomes the minimum Dmin (l) = l − 2 + 2−l when the first spike position is τ1 = −x(0) = 1 − 2−l (see Fig. 3(a)). Also, the length D(l) becomes the maximum Dmax (l) = l − 2−l when the first spike position is τ1 = −x(0) = 2−l (see Fig. 3 left). Therefore the variation DV (l) is to be DV (l) = Dmax (l) − Dmin (l) = 2 − 2−l+1 . That is, the proposition for α = 0 is proven. Second, let us consider the case α ∈ {0, 1, 2, · · ·}. Repeating a similar consideration to the case of α = 0, we have τl = (α + 1)l − 1 + θl .
(11)
The spike phase map G is given by Eq. (9) and satisfies the relation in Eq. (10). Then we have D(l) = (α + 1)l − 1 + (2l θ1 (mod 1)) − θ1 . Repeating a similar consideration to the case of α = 0, we have Dmin (l) = (α + 1)l − 2 + 2−l and Dmax (l) = (α + 1)l − 2−l . Therefore the variation becomes DV (l) = Dmax (l)−Dmin (l) = 2 − 2−l+1 . Q.E.D.
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1458
Remark 2: • In the case of 0 < α < 1, the spike position map g satisfies a property g : Ik → Ik+1 ∪ Ik+2 . That is, the spike position τn can distribute in a wider range Ik+1 ∪ Ik+2 than the range Ik+1 of the spike position map g for α = 0. This property g : Ik → Ik+1 ∪ Ik+2 is a necessary condition for DV (l) > 2 for all l and α. Detailed analysis (e.g., sufficient condition for DV (l) > 2) is complicated and should be included in future problems. Note that small DV is convenient for the spike-based AD conversion because encoded spike-trains have similar length D(l). Also, small DA is convenient because the coded spiketrains have shorter length D(l). Hence hereafter we fix the offset parameter to α = 0. • If α = 0, the spike position map g satisfies g(Ik− ) = + g(Ik ) = Ik+1 . Such map is referred to as Markov-type: a simple and useful class of maps in application to information coding [11], [12], [22]. Since the slope of g is 2 and is expanding, any spike-train is unstable for the initial state x(0) and is to be chaotic in the sense of dynamical systems defined for unbounded time 0 < τ. However, Ref. [23] presents a circuit in which the spike-trains can be stabilized by adding higher frequency sawtooth signal to the threshold. 4.
Analog-to-Digital Conversion
In this section we study the ASN-ADC based on the SIM. The initial state x(0) of the ASN is used as an analog input. The ASN generates a spike-train y(τ) characterized by the spike positions (τ1 , τ2 , · · · , τl+1 ) whose pattern depends on the analog input x(0). Let us define the n-th ISI by dn = τn+1 − τn .
x ≥ 0. Then we have the following theorem. Theorem 1: The estimation X˜ in Eq. (14) approximates the analog input x(0) as 1 X˜ = − l Int(−2l x(0)). 2
(15)
That is, the analog input x(0) ∈ (−1, 0] is converted into one of the lattice points X ∈ {0, −1·2−l , −2·2−l , · · · , (2l −1)·2−l }. Proof: Since Eq. (7) is equivalent to the Bernoulli shift as shown in Fig. 2(b), binary encoding of θ1 can be realized [25]: l 1, for τn ∈ I1− 1 l −n Int(2 θ ) = P 2 , P = (16) 1 n n 2l 0, for τn ∈ I1+ n=1 where l is the code length. This encoding is referred to as the SPM [23]. Since the spike position map g has periodicity g(τ + 1) = g(τ) + 1, the ISI dn = g(τn ) − τn can be written by a function of the spike phase θn as follows: dn = g(θn ) − θn ≡ h(θn )
(17)
where h is a mapping from I1 to J ≡ [0.5, 1.5] as shown in Fig. 5(a). Note that J is the existence region of the ISI dn . Substituting Eq. (17) into Eq. (7), we obtain θn + 1 for τn ∈ I1− , (18) dn = h(θn ) ≡ θn for τn ∈ I1+ . Note that h is one-to-one and onto as shown in Fig. 5(a). Using h, we can confirm that the SPM in Eq. (16) is equivalent
(12)
To the sequence (d1 , d2 , · · · , dl ) of ISIs, we apply the following SIM 0 for dn ∈ [1, 0.5], Yn = (13) 1 for dn ∈ [0.5, 1), and obtain a digital output Y = (Y1 , Y2 , · · · , Yl ). We can summarize relations between the ASN as ADC and the ASN as neuron model in Table 1. We note that, to obtain the digital sequence with the code length l, the SIM requires the operating time (l + 1). Let us define the estimation of the analog input x(0) by X˜ = −
l
Yn 2−n .
(a)
(14)
n=1
Let a function Int(x) denote the integer part of a real number
x x(0) y(τ) l Y
Table 1 ADC versus neuron model. ASN as ADC ASN as neuron model State variable Membrane potential Analog input Initial state Spike-train for encoding Firing spike-train Code length Number of ISIs Digital output Temporal code by SIM
(b) Fig. 5 Description of spike intervals. (a) Spike position to interval conversion h. (b) Spike interval map F.
TORIKAI et al.: ADAC
1459
to the SIM in Eq. (13), i.e., Pn = Yn .
(19)
Recall that the first spike phase θ1 and the analog input x(0) have the following relation: θ1 = −x(0),
x(0) ∈ (−1, 0].
(20)
From Eqs. (20), (19), (16) and (14), we can derive Eq. (15). Q.E.D. From a mathematical viewpoint, the SIM is equivalent to the SPM when h is one-to-one. However, from a practical viewpoint, the SIM has several advantages such as receiving without clock and deep relation to biological spiking neurons as discussed in Sect. 1. If h is not one-to-one, the SIM is not equivalent to SPM and that may cause interesting function. Consideration of such SIM is in future problems. In order to visualize the SIM, we note that dn+1 is given by dn through the map h as follows. dn+1 = F(dn ) ≡ h ◦ g ◦ h−1 (dn ) = ⎧ ⎪ for dn ∈ [0.5, 0.75), ⎪ ⎪ 2dn , ⎨ 2dn − 1, for dn ∈ [0.75, 1.25), ⎪ ⎪ ⎪ ⎩ 2d − 2, for d ∈ [1.25, 1.5]. n n
(21)
The sequence {d1 , d2 , · · · , dn } of ISIs is governed by the map F. The SIM can be visualized as shown in Fig. 5(b). 5.
Digital-to-Analog Conversion
Here we present the ASN-DAC that realizes inverse operation of the ASN-ADC. Let us assume that the ASN-ADC converts an analog input x(0) into a digital output Y. The required operation for the ASN-DAC is to convert Y into an approximation of the analog input x(0). In order to realize the DA conversion, inverse digital sequence of Y is used as an input, i.e., the digital input to the ASN-DAC is defined by
Y ≡
{Y1 , · · · , Yl }
= {Yl , · · · , Y1 }.
(22)
Using Y we define a continuous-time signal u(τ) that is to be an input to the ASN-DAC: u(τ) = Yn for τ ∈ In ≡ [n − 1, n)
(23)
where 0 ≤ τ < l and n = 1 ∼ l. We then present the dynamics of the ASN-DAC as follows. ⎧ d x˜ ⎪ ⎪ ⎪ =1 for x˜(τ) < th, ⎨ dτ ⎪ ⎪ ⎪ +
+ ⎩ x˜(τ ) = b (τ ) if x˜(τ) = th, (24) b (τ) = b (τ) + 0.5u(τ) 1
b1 (τ) = 0.5τ − 1.5 for 0 ≤ τ < 1 b1 (τ + 1) = b1 (τ) where th = 0, and the initial value x˜(0) can be given arbitrarily in (−1, 0]. b1 (τ) is a periodic sawtooth signal with rising slope. Adding the input signal u(τ) to this b1 (τ), the base
Fig. 6
Dynamics of ASN-DAC (Digital input: Yn = (0, 0, 1)).
Y’ l x˜(0)
Table 2 DAC versus neuron model. ASN as DAC ASN as neuron model Digital input Modulation signal to base b(τ) Code length Number of modulations to b(τ) Initial value that is Initial state of arbitrary in (−1, 0] membrane potential Analog output Membrane potential at τ = l
x˜(l)
signal b (τ) is obtained. When u(τ) is positive (Yn = 1) for τ ∈ In , the offset of the base signal is increased as shown in Fig. 6. The integrate-and-fire switching is the same as that of the ASN-ADC: x˜(τ) rises, x˜ is reset to the base b (τ) just after x˜ reaches th, and the system repeats the integrate-and-fire behavior between th and b (τ). The state x˜(l) at τ = l is used as an analog output. Note that the system is not required to output spikes because neither digital output nor SIM is required in the ASN-DAC. We can summarize relations between the ASN as DAC and the ASN as neuron model in Table 2. Then we have the following theorem. Theorem 2: The difference (error) between the analog output x˜(l) of the ASN-DAC and the analog input x(0) of the ASN-ADC satisfies | x˜(l) − x(0)| < 2−l . Proof: First, we consider the ASN-ADC. Let the first spike phase θ1 of the ASN-ADC is in [2−l m, 2−l (m + 1)), where m ∈ {0, 1, 2, · · · , 2l − 1}. This is equivalent to the following situation: 2−l m ≤ −x(0) < 2−l (m + 1).
(25)
In this case, from Theorem 1, the digital output Y of the ASN-ADC is to be the binary number representation of m. Second, we consider the ASN-DAC. Repeating in a likewise manner as derivation of Eq. (2), we obtain the following spike position map g−1 driven by the digital input Y : (τn + n + 1)/2, for Yn = 0, τn+1 = g−1 (τn ) ≡ (26) (τn + n + 2)/2, for Yn = 1, where g−1 (Ik ) = Ik+1 . The spike position map g−1 is shown in Fig. 7(a). The spike position satisfies τn ∈ In for all n since the first spike position τ1 is in [0, 1). Then we have τ n = n − 1 + θn .
(27)
The spike phase θn is governed by the following map G−1 driven by the digital input Y : θn /2 for Yn = 0, −1 θn+1 = G (θn ) ≡ (28) (θn + 1)/2 for Yn = 1.
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1460
Fig. 8 ASN-ADAC circuit. ICs are TLC4066ID (switch), LM339 (comparator) and CD4538B (monostable multivibrator).
The two voltage sources B1 (t) and U(t) are connected in series and are assumed to be synchronized with a clock signal of period T . In PSPICE simulation B1 (t) and U(t) are given by signal generators. The circuit dynamics is described by ⎧ ⎪ dv ⎪ ⎪ for v < T h, ⎪ ⎪ ⎨ C dt = I0 (31) ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ v(t+ ) = B (t+ ) if v(t) = T h,
(a)
B (t) ≡ B1 (t) + U(t)
(b) Fig. 7 (a) Spike-position map g−1 of ASN-DAC. Digital input is Y = (0, 0, 1). (b) Phase map G−1 of ASN-DAC.
A phase map G−1 is shown in Fig. 7(b). The phase map G−1 can realize inverse operation of the phase map G of the ASN-ADC (Eq. (7)). (G) That is, the map G−1 is to be inverse of the Bernoulli shift and can be regarded as a kind of IFS. Referring to [22], the spike phase θl exists in the interval [2−l m, 2−l (m + 1)). This is equivalent to the following situation: 2−l m ≤ − x˜(l) < 2−l (m + 1).
(29)
From Eqs. (29) and (25), we have | x˜(l) − x(0)| < 2−l . Q.E.D. 6.
Figure 8 shows a simple circuit model of ASN-DAC. In the figure, U(t) is a input signal given by digital input Y and B1 (t) is a periodic sawtooth signal with period T : : U(t) =
U(t + lT ) = U(t)
and b (τ) = where
C(B (T τ)−T h) , I0 T
Eq. (31) is transformed into Eq. (24)
b (τ) = b1 (τ) + 0.5u(τ)
1 u(τ) = 2 CK T I0 Yn for n − 1 ≤ τ < n
Circuit Model
K1 Yn
where T h is a threshold. For an initial value v(0) ∈ (−VD /2, 0], the capacitor voltage v rises by integrating a constant current I0 . When v reaches T h, the comparator triggers the monostable multivibrator MM to output a single spike Y. The spike Y closes a switch S and the v is reset to the base B1 (t) + U(t). It goes without saying that the voltage sources B1 (t) + U(t) must have inner resistor r0 and the reset is completed within finite time. However we ignore r0 and assume instantaneous reset in Eq. (1) for simplicity: it is a routine in analysis of switching circuit. v repeats this integrate-and-fire behavior. If U(t) is positive (Yn = 1) for t ∈ [(n − 1)T, nT ) then the level of B(t) is up. The analog output is given by the voltage at time t = lT : −v(lT ). Ush) ing the following dimensionless variables τ = Tt , x = C(v−T I0 T
for (n − 1)T ≤ t < nT
C(VD +T h) 2 for n − 1 ≤ τ < n b1 (τ) = CK I0 τ − T I0
n = 1 ∼ l, b (τ + l) = b (τ),
and scaling parameters are fixed: K1 = 0.5
n=1∼l (30)
B1 (t) = K2 t − VD for 0 ≤ t < T B1 (t + T ) = B1 (t) where K1 , K2 and VD are scaling parameter fixed afterward.
T I0 I0 T I0 , K2 = 0.5 , VD + T h = 1.5 C C C
We have confirmed basic circuit operations in PSPICE simulation and a typical data is shown in Fig. 9. It should be noted that this circuit is transformed into a model of ASNADC by replacing B1 (t)+U(t) with a periodic sawtooth voltage source proportional to b(τ) and adding a higher periodic signal to T h for stabilization. The actual circuit can be found
TORIKAI et al.: ADAC
1461
Fig. 9 Typical waveform of ASN-DAC for l = 3, digital input Yn = (1, 1, 0). The analog output is −v(3T ) 0.56 [V]. Parameter values are C = 20 [nF] (IC = 0.3), I0 = 0.20 [mA], T = 0.10 [ms], T h 0 [V]. B1 (t) = VSRC (TRAN=pulse(−1.5 V −1 V 0 0.1 ms 0 0 0.1 ms)), U(t) = VSRC (TRAN=pulse(0 V 0.5 V 0 0 0 0.2 ms 0.3 ms)). The operation period is 3T and the initial value is reset every 3T .
in [23] with typical experimental data. We note that the circuit model in Fig. 8 is presented as a breadboard prototype and is not intended to a practical implementation. The practical circuit designs are included in future problems. We also note that the system equations (1) and (27) are presented as dimensionless forms as is often done for analysis of nonlinear dynamical circuits [9], [10], [23]. The dimensionless equations can describe dynamics of the circuit Eq. (34) for wide range of parameter values. 7.
Conclusions
We have studied encoding/decoding properties based on integrate-and-fire dynamics of ASN. We have investigated parameter dependence on spike-train length and have fixed the parameter α = 0 that can realize ASN-ADC with the minimum variation and average of spike-train length. In the ASN-ADC, we have proposed the SIM that has several advantages including robustness for delay. In the ASN-DAC, digital input signal is applied to adjust base level of the ASN and inverse operation of the ASN-ADC is achieved. A simple circuit model is also presented. (a) Circuit design of the ASN-ADAC for with consideration of nonidealities. (b) Analysis of relation between spike-train length and the number of spikes in various spiking neuron models. (c) Analysis of SIM characteristics of a variety of spike-trains along with the change of b(τ); and classification of encoding/decoding schemes for various spike-trains. References [1] R. Perez and L. Glass, “Bistability, period doubling bifurcations and chaos in a periodically forced oscillator,” Phys. Lett. A, vol.90, no.9, pp.441–443, 1982. [2] J.P. Keener, F.C. Hoppensteadt, and J. Rinzel, “Integrate-and-fire models of nerve membrane response to oscillatory input,” SIAM J. Appl. Math., vol.41, pp.503–517, 1981. [3] E.M. Izhikevich, Dynamical systems in neuroscience, MIT Press, 2007.
[4] H. Torikai, T. Saito, and W. Schwarz, “Synchronization via multiplex pulse-train,” IEEE Trans. Circuits Syst. I, vol.46, no.9, pp.1072–1085, 1999. [5] G. Lee and N.H. Farhat, “The bifurcating neuron network 2,” Neural Netw., vol.15, no.1, pp.69–84, 2002. [6] Y. Kon’no, T. Saito, and H. Torikai, “Rich dynamics of pulsecoupled spiking neurons with a triangular base signal,” Neural Netw., vol.18, pp.523–531, 2005. [7] S.R. Campbell, D. Wang, and C. Jayaprakash, “Synchrony and desynchrony in integrate-and-fire oscillators,” Neural Comput., vol.11, pp.1595–1619, 1999. [8] E.M. Izhikevich, “Weakly pulse-coupled oscillators, FM interactions, synchronization, and oscillatory associative memory,” IEEE Trans. Neural Netw., vol.10, no.3, pp.508–526, 1999. [9] H. Torikai and T. Saito, “Synchronization phenomena in pulsecoupled networks driven by spike-train inputs,” IEEE Trans. Neural Netw., vol.15, no.2, pp.337–347, 2004. [10] H. Nakano and T. Saito, “Grouping synchronization in a pulsecoupled network of chaotic spiking oscillators,” IEEE Trans. Neural Netw., vol.15, no.5, pp.1018–1026, 2004. [11] M. Sushchik, N. Rulkov, L. Larson, L. Tsimring, H. Abarbanel, K. Yao, and A. Volkovskii, “Chaotic pulse position modulation: A robust method of communicating with chaos,” IEEE Commun. Lett., vol.4, pp.128–130, 2000. [12] G.M. Maggio, N. Rulkov, and L. Reggiani, “Pseudo-chaotic time hopping for UWB impulse radio,” IEEE Trans. Circuits Syst. I, vol.48, no.12, pp.1424–1435, 2001. [13] A. Linares-Barranco, D. Cascado, G. Jimenez, A. Civit, M. Oster, and B. Linares-Barranco, “Poisson AER generator: Inter-spikeintervals analysis,” Proc. IEEE/ISCAS 2006, pp.3149–3152, 2006. [14] M. Christen, A. Kern, A. Nikitchenko, W.H. Steeb, and R. Stoop, “Fast spike pattern detection using the correlation integral,” Phys. Rev. E, vol.70, 011901, 2004. [15] T. Toyoizumi, K. Aihara, and S. Amari, “Fisher information for spike-based population coding,” Phys. Rev. Lett., vol.97, 098102, 2006. [16] B. Lindner, M.J. Chacron, and A. Longtin, “Integrate-and-fire neurons with threshold noise: A tractable model of how interspike interval correlations affect neuronal signal transmission,” Phys. Rev. E, vol.72, 021911, 2005. [17] P.G.A. Jespers, Integrated converters, Oxford Univ. Press, 2001. [18] L. Chiaburu and S. Signell, “An improved binary algorithmic A/D converter architecture,” Proc. IEEE/ISCAS, I-865–I-868, 2003. [19] G. Cauwenberghs, “A micropower CMOS algorithmic A/D/A converter,” IEEE Trans. Circuits Syst. I, vol.42, pp.913–919, 1995. [20] L. Weyten and S. Audenaert, “Two-capacitor DAC with compensative switching,” Electron. Lett., vol.31, pp.1435–1437, 1995. [21] H.J. Jeffrey, “Chaos game representation of gene structure,” Nucleic Acids Research, vol.18, pp.2163–2170, 1990. [22] T. Saito, J. Shimakawa, and H. Torikai, “D/A converters and iterated function systems,” Nonlinear Dynamics, vol.44, pp.37–43, 2006. [23] H. Hamanaka, H. Torikai, and T. Saito, “Quantized spiking neuron with A/D converision functions,” IEEE Trans. Circuits Syst. II, vol.53, no.10, pp.1049–1053, 2006. [24] A. Tanaka, H. Torikai, and T. Saito, “A/D and D/A converters by spike-interval modulation of simple spiking neurons,” Proc. IEEE/ISCAS, pp.3113–3116, 2006. [25] E. Ott, Chaos in Dynamical Systems, Cambridge Univ. Press., 1993.
IEICE TRANS. FUNDAMENTALS, VOL.E91–A, NO.6 JUNE 2008
1462
Hiroyuki Torikai received B.E., M.E. and Ph.D. degrees in electrical engineering all from Hosei University, Tokyo, Japan, in 1995, 1997 and 1999, respectively. He is currently an associate professor at the Graduate School of Engineering Science, Osaka University, Osaka, Japan. His research interests are in neural networks, discrete-state dynamical systems, chaos and bifurcation. He is a member of the INNS.
Aya Tanaka received the B.E. and M.E. degrees in electrical engineering all from Hosei University, Tokyo, Japan, in 2005 and 2007, respectively. In 2007 she joined Toshiba Co., Ltd., Tokyo, Japan. Her current research interests are in neural networks and signal processing.
Toshimichi Saito received the B.E., M.E. and Ph.D. degrees in electrical engineering from Keio University, Yokohama, Japan, in 1980, 1982 and 1985, respectively. He is currently a Professor at Hosei University. His research interests include chaos and bifurcation, neural networks, evolutionary computation and power electronics. He served in several editorial boards including the IEEE Trans. Circuits Syst. I (2000–2001) and II (2003–2005). He is a member of the JNNS and a senior member of the IEEE.
