Log-Domain Time-Multiplexed Realization of ... - Gert Cauwenberghs
Log-Domain Time-Multiplexed Realization of Dynamical Conductance-Based Synapses Theodore Yu1,3 and Gert Cauwenberghs2,3 1
Department of Electrical and Computer Engineering, Jacobs School of Engineering 2 Department of Bioengineering, Jacobs School of Engineering t11 3 Institute for Neural Computation t12 1 2 3 4 6 7 University of California San tDiego, CA 92093 t51 tJolla, 1 t1 t1 t1La 1 t1 1
1
t21 t31 t41 t51 t61 t71 t22 t32 t42 t52 t62 t72
1
t3 t23 t33 t43 t53 t63 t73
t12 t22 t32 t42 t52 t62 t72 1
t11
1
2
3
4
5
6
1
1
t21t11 t31t21 t41t31 t51t41 t61t51 t71t61 t71 j=1 j=2 j=3
7
t12 t22t2 t32t2 t42t2 t52t2 t62t2 t72t2 t2 Abstract—We present a compact circuit architecture for analog t3 t3 t3 t3 t3 t3 t3 V1 V2 V3 Vi Vj Ii,j Vτ C 6 27 j = 3 VLSI realization of event-addressable neuromorphic arrays with 1 7t6 t7 t11 t21 t31 j t=41 1t51 j t= t 1 1 t4 t6t5 t(θ) t2t1 t3t(θ)23 t43t33 t53CU (θ)t3 (θ) E ui 3 3 I3leak = τ 3 κ3 3gij3 3δ (θ)3(t − tkj ) conductance-based synaptic dynamics. Synaptic input events are 1 2 3 4 5 6 7 1 V 2 3 V4 5 V6 7V V I 1 jt1=t1 1 tj1 =t12 j = 3 t2 t2 j=1 t2 t2j=2t2 tj=3 142 253 364 4175 526 637 74 5 6 7 t1 t1 j t= 2 t2t1i t2 j t31 i,j 1(θ) j = 2 j(θ)= 3 1 1 11 tt111 tt111 tt111 ttt1111 tt1t11 tt1t11 tt11 t1 t1 t1 time-multiplexed, pooled by synapse type according to common gi (t) = gij tkj = 1 2 3 4 5 6 7 1 2 3V 4 I5 6 I 7 t t t t t 2 2V 2 tV 2 t2 V V I t3 t3 t3 tj3 tr3 gt3 t3tr1 gt2 t31 tt142 tt253 tt364 ttt4175 tt5t26 tt6t37 tt74 t5 t6 t7 V21 V2 2Vj=1 reversal potential and activation dynamics. One such physical V V Ii,j Viτ Cj i,j j=2 3 i j j=3 2 2 22 222 222 222 2222 222 222 22 2 2 τ2 τ τ 1 2 13 24 g 5 (µA/V 6 7 ) Vu (V ) T ime (ms) t t t t t t t synapse element per postsynaptic neuron is provided for each 1 2 3 4 5 6 7 1 1 1 1 1 1 1 7 t3 t3(θ)t3 t3CUt3 t(θ) t11 t21 t31j = t41 1αt51nj(Vt=61) 2tβ =tt)131t11tt122− 233 3 2n 341414113452512522456362363356747347446175 45 55726 56 6637 67 774 7 5 6 (θ) 7 t3 u(θ) 2 1n n(V 1j IleakVj= Iτ r κg gij3 Iδr(θ)g(t − tkj ) i 11 1 311t1tt1t311t1tt1t313t1t1t1t1t31t313tt1t11t1t1t31t313tt1t11t1t1t31t313tt1t11t1t1t31333tt11tt13t133tt11tt13t33tt11tt133t1 t3 t3 Et3 type, selected by type index along with postsynaptic address. A 1 2 3 4 5 6 7 t2 t2 t2 t2 t2 t2 t2 5 6 7 j = 1(θ) j = 2 j(θ)= 3k t12 tV22 j=1t32 Vt42j=2tdn (1i − Vtt1j1n) 1 I2− 2 2β 133n3nt241t41t41t35t5122t52t46t623t633t57t734t7446 t45t575t56t6 t667t7 77 2dtVt= 2 αtn 2V j=3 i,j log-domain encoding of first-order linear dynamics of synaptic 1−n n (t)n(V = ) gβ Itsyn tsyn t2ttI2tsyn t2syn t2tIt2syn t2 tI2Isyn t2syn 3 4 5 6 gi 7 α ij n(Vtj) = 22t2tt22t2Ittsyn 2syn 2t2tItI 2syn 2 2 tI12syn 2 t 2 Isyn 2t22t2tt 2t22tI22I 1 2t2212 2 2t2212 22t2 12 2 t212 tt121 tt211 t1 t1t1 t2t1 t3t1 t4t1 t5 t6 t7 3 3 conductance results in a compact circuit realization witht13 three V3dn i Vj Ii,j Vτ C t23 t33Vjt43 Irt53 g t63 tI73r g t11 11t22 22t133133t244214413155312552426642366135377534772464 645t3575 756t46 6 67t57 7 7t6 τt73 Vτ1 3 τV23 Vg33 (µA/V (1u − βnn (ms) (Vn) ) − T ime t33tt31 t3I3ttm tt13mt3tIt13thtnm tt13tg3thgt13tttgt13t13ttI13ttntgt13tI13tng1tItm 13t13gth 1tItg31g13tI1tgt113 13tI1tn2 1g 3 1Itg4 1 t5 2 t6 t7 dt = )αnV 3t13t3th3t1t3tg13t3t1t3I MOS transistors per synapse element. Circuit simulations show 2 t21313t31t32231113 3t133t321321 313t31t4233113 t3t521413 ttt623251 tt32t7261 t2 (θ) 12 j2 = t71j 2=u(θ) (θ) 2 j = 3 (θ) Ileak = τCUκ gij δ (θ)(t − tkj ) 1 3 54 72 4 65 3 5 76 4 6 7 5 7 6 E 7 j = 1 j = 2 j = 3 i α (V ) β (V ) 1 − n n 1 1 2 2 3 1 2 1 3 2 4 3 5 4 6 n n low-power operation with linear dynamics in conductance. IIsyn t2g22tI2trIsyn I2t2tIgr32tsyn I2ttrg12tt2tg42t2t2tt53Itr2 t6g4 tt57 t6 t7 Isyn IrIttsyn I2tIsyn tI2Irsyn tg2 t2ItI2syn t2IgI22trgsyn syn rt2gr2tI g2tt g2It2syn rt2I gtsyn rt12syn syn syn syn 2tItsyn 2tt 2ttsyn 2I 2rt2I22tI 1
2
3
4
5
6
7
1
1
1
1
1
1
(θ)
T
1
1
syn1
2
syn2
1
11
1
1 1
1
1
(θ) 1
1
T
syn1
2
syn2
1
0
10
0
21
1
20 2
1
2
1
1
1
1
1
1
1
syn1
2
syn2
3
Na
3
43
NK a Na
4 L
4 3 K K
LN a L
4
L
K
1
(θ)
0
0syn 0 00
0 1011syn 01 syn 1 syn 22 21syn 212 1syn 1 20 2syn 2 2syn1 0syn 1 00 210212 02 syn 110syn
2
T
syn2
θ ...
1 1 1 3 1 3 13 13 31 3 j=33 Vi (θ) V I t1 t232125 2t43t3361223 54t72t4343265 3t5t537426 4t6t637525 7tt736V62 j=1t7 72 Vj=2gi(θ)V(t) = gij j tkj i,j= Vj=1 Vj=2 dn Vj=3αnV (1i −Vjn) I− i,j βnIn1 3 2 1 3I3214I4I1 3t tK− t tn t t3 n n dt = ItgntnI3N3g43ht(V It ntnI(V t(V t(V t3− 4)3 )33Kt3gIh33KtI)β β n α )3taIn33ntIn43(V n3ItKβ Iα Inm3h(V It3(V INLa33tLggang3KtLtN1(V tm3m tNga3m tαaβ3nhtInα tagtm t)1tn g3gN)I n3n g34) h31 g3t1 g− g4I3g32 ht3h gn34n m 3) 3115 − K Ktg 3I Ln aN3 Nm j = 33gL3ngL3jg34L= 6 2 7j = 3 t t2 t2 1 t22 t23 tV42j 5tI2r1 6gsyn17 Ir2 gsyn2 I. I NTRODUCTION )1 Vu2 (V3 ) 4T ime 2 2= 4j t= Vj Ir1 gsyn1 Ir2 gsyn2 dnjdn=t11 tj dn 2=Isyn 5 6(ms) 7 1t5It3syn 1τt6tτ 1 1tt71τ2 t1 g t(µA/V 1 t1 tn331gn) I1IgInsyn IIrrI00rg0gdn In2010gsyn Ir1(1 ItII− = = αIn) α (1 − − βsyn − n) β α β(1 −2syn n) I=r00IIα Ig0I11syn IdtIg(1 I30nrIgsyn 01− 3syn 3n 32 −V 3βnn V V I t1 t1 t1 t1 t1 t1 t1 rsyn rdt rsyn gdt r3syn rnn gn n gIsyn grI gI rgsyn rI rsyn r1g2n syn syn syn syn 1syn 0g 01− 2syn syn 22I syn 1r 1syn syn syn 2rg 2syn 0r 21g 2 12V dt 00syn 020syn 22 syn 1syn 2gV 0syn 101syn 212syn 2 j=3 i j i,j 1 2 j=1 3 1 42j=253αn(V 46 )5t7βn6(V 7) 1 − n n t t t t t t t t t t t t t αn(V ) biβn(V ) 1 − n n α (V ) β (Vαα)(V 3 3 3 3 3 3 3 1 2 3 4 5 6 2 2 2 2 2 2 2 Neuromorphic engineering [1] utilizes inspiration from 3 4 IN(V I C 3) 4)n − β β n )n IN4V In 3nh 4n m hI gj t2 t2 t2 t2 t2 t2 t72 αnαn(V )αV )3nh1β(V )a(V )(V 1(V (V 1g)3− )= n 1gβi(V − (V nK− 1IIngi,j − Im In nβ m m h) gIn nI g1 V n(V nm n gnIN(V n gn1n nnaV N ag L1 ah Kn V jK3g− = 2nnLIgVLngKnjjVn= 2)nIβα j4NK)g1Lag− τIrg1L3 Ir2 gsyn2 gdn syn1 1 2 3 dt4 = 5αn(1 6 −7 n) − βnn dn ology and neurobiology to direct and motivate the design and t t t t t t t dn dn dn dn I dndn 3nn 3I2 3 3 3 3 3 jrn(1 = 1Ir− = j 2= dt = αn(1 − n) − βnn I− = − α1= αβ − n) βjn(θ) Ir− In) Inr(1 In rα g− r2 g3syn t1 t2 t3 t4 t5 t6 t7 ==ααdn (1 α (1 n) − β2nr(1 − n n) n Idtr= In(1 Ig− I(θ) Ing0βn) In) (θ) 0n nα n nβ r− rsyn g= g− gsyn gβ r1 Iβ g1(V n0= ngn) ngg− n(θ) r(1 r− CU (θ) 0r 0dt 0dt syn 0 gsyn syn 1dt syn 2nTg dt dt dt 1syn 1 syn22 α2syn 2syn 2) β (V grgsyn −τ )tkjC)1 − n n 3 3 3 3 3 3 3 V1 0syn V113syn1τsyn2(θ) V ij 2 = jn δIi,j(tnV κi V modeling of circuits and systems. By emulating form and t11 t21 t31 t41 Et51 t61 0uti71V10 Ileak j = 1 j = 2 j = 3 α−nn)β(V )− 1V − n In V C (V )nV1(V )β 1nn− 1(Vn − n n j=1 j=2 j=3 )αnβ(Vα )α(V nn1(V nV n n)β n(V V V)dn 1)β inn= jαn(1i,j − n) τ− β (θ) nn (θ)2k 3 CUTdt 6 (θ) architecture in biological systems, neuromorphic engineering t12 t22 t32 t42 αtn52g(Vi(θ) (θ) tE t72 =u(θ) tj ==Vτ (θ)gκi1Vggij(θ) (t − tkjj) Ii,j i2δ gi3 2 (t) i gij I dnleak j=1 j=2β V j=3 Vi V dn dn dn dn (θ) V = α (1 − n) − n = = α− α(1 (1 −n− − n)β − βnβnnCU n n (θ) nn) V k V V V I (1 βn) (θ) (θ) nn) nn− dt seeks to emulate function as well. Investigation of neural t1 t2 t3 t4 tdt5 =t6 αdttn7(1=Edt−α(θ) un dtin n(θ)Ileak tj ) Ijj=2(t) j=3 i j i,j = (θ)κT gij δ (θ)(t − j=1 k (V V)τj T 3 3 3 τ 3 τ 3 τ3 3(θ) I I g (µA/V ) V ime (ms) rg gsyn2 tj = gi1r1 gsyngi2 1 2 gi (t) = gij u 2 i3 1 behavior on large scale requires efficient and realistic modeling Vj Ir1 gsyn1 Ir2 gsyn2 (θ) (θ) = gij α (V tkj ) =β (Vgi1) 1g−i2 n gni3 and implementation of neurons and their synaptic connections θj =.1. . j = 2 τ j =τ13 τ2 gig (t) (µA/V ) Vnu (V )n T ime (ms)αn(V ) βn(V ) 1 − n n Vj=1 Vj=2 Vj=3 Vi Vj Ii,j [2]-[4]. αun(1(V −)n) T −ime βnn (ms) τ τ1 τ2 g (µA/Vdn dn dt)= V dt = αn(1 − n) − βnn Analysis of a variety of different implementations of Vj Ir1 gsyn1 Ir2 gsyn2 1
1
1
conductance-based dynamical synapses, and new circuit that overcomes some of their limitations, are presented in [5]. In particular, [5] analyzes the trade-offs among the different implementations regarding functionality of the temporal dynamics and the required layout size, and offers a circuit with linear dynamics in conductance. Earlier implementations of VLSI synapses such as the pulse current-source synapse [1] and reset-and-discharge synapse [6] suffer from inability to integrate input spikes into continuous output currents and linearly sum postsynaptic currents respectively. Other previous circuits such as the linear charge-and-discharge synapse [7] and current-mirror-integrator synapse [8], [9] , and [10] also suffer from nonlinear summation of postsynaptic currents. The synapse implementations of log-domain integrator synapse [11] and diff-pair integrator synapse [5] implement linear summation of postsynaptic currents, but they require an Mw p-FET or additional transistors. Here we show that a variant on the log-domain implementation gives rise to linear conductance dynamics in more compact form. Here we present a three-transistor realization of a dynamical conductance-based synapse element, serving
αnFig. (V ) β1.n(V )Pooling 1 − n nof
synapses with common reversal potential and activation dynamics, but possibly with different conductances, by time-multiplexing dn = αn(1 − n) −from βnn j presynaptic neurons. events dtinput
multiple synapses with common reversal potential and activation dynamics. The time-multiplexing synapse element pools spike input events from multiple presynaptic source addresses through the address-event representation (AER, [12]) communication framework as seen in Fig. 1. II. S YNAPSE A RRAY A RCHITECTURE This paper focuses upon the architectural design of the pooled synapse input for each neuron within the neural array. We assume that the number of distinct synapse types is limited to a relatively low number k, e.g., k = 8. This assumption is typically valid even in large-scale cortical models. We pool synapses of the same type serving the same postsynaptic terminal into a time-multiplexed synapse element. Synapse elements in the array are activated by presynaptic events presented through an AER input interface [12]. Neurons receiving
1
tt21 tt22t1tt23t2tt24t3tt25t4tt26t5tt27t6 t7 1 13 13 13 13 13 13 3 tt131 2
tt232 2
33 44 jtt32=tt132
tjt535=tt6326 2 2
tjt737= 2
3
(θ)
E (θ) ui
4 2 5j =6 3 7 tj13 =tV231j=2t33j = Vj=1 Vtj=3 3 t3 Vti 3 Vtj3 Ii,j Vτ C t11 t21 t31 t41 t51 t61 t71 (θ) (θ) (θ) Vj=1EV V V Ii,j3g (θ) Vτδ (θ)C(t − tk ) (θ)j=2 j=3 jj = ju=i 1 Ijleak =i = 2 VCV (θ) j t12(a) t22 t32 t42 (θ)t52 τ t62κ t72 ij (θ) (θ) (θ) CV (θ) (θ) (θ) EV u ggjij(θ)Iδi,j tCkj ) I Vgj=3 = (t) k(tV− (θ) V V τ κ j=1 i Vj=2leak i τ = t = t13 t23 ti33 t43 t53 tij63 t73 j (θ) 4k (θ)5 (θ) (θ) (θ) CV (θ)6 t7 k gij tj n) tdn11 ==t21 α t3(θ) t− t= E (θ) ui gi I(t) 1δ−t1(t βn− n1 tj ) leak τ (θ) κ 1gij n1(1 j =2 1dt 3j =4 2 5 j =6 3 7 1 tdn1 (θ) t t t t t t 1 2 3 4 5 6 7 (θ) 1 1 1 1 1 1 α −kβ n t2n(1 =t2−gtijn) dtgi= (t) 2 t2tj tn2= t2 t2 Vj=1 Vj=2 V 3 V4 i 5Vj 6Ii,j7 Vτ C 11 22 j=3 ttdn 21 tt21t1tt231t2tt241t3tt251t4tt261t5tt271t6 t7 = α (1 −3n) − 3 (θ)β3nn 3 3 (θ)dt (θ) 3 n 3CV (θ) E (θ) ui 1 Ileak = (θ) gij δ (θ)(t − tk ) tt31 tt232 jtt333=tτt1434 κtjt535=tt6326 tjt737= 3 j 2 2 2 2 2 2 2 (θ) (θ) gi (t) = gij tkj = 1 = 21 3j =4 2 5j =6 3 7 j Vj=1 3 tt53 Vtti63 V tt13 tt2V3j=2 tt33 tVt4j=3 tt7j3 Ii,j Vτ C 1
1
1
1
1
1
E
dn (θ) (θ) (θ)αn(1CV − n) −(θ)β 7n(t 2 = 3 4 5 g 6 δn(θ) ui t13dt It= leak 3 t3 tτ3(θ)κt3 tij3 t3 (θ) gji =(t) 1
=j
(θ) =gij2
(θ) gi (t) 1
−
k gij1 δ (θ) 2 (t 3 − 4 tj5)
t3 t3 t3 t3 t3 t63 t73
t1 tk j = = 11
(θ) gij
t2 t3 j 1= 21
j
(θ)
(θ)
Ileak =
1 1
E
1
(θ)
2 1
41 52 63 74 11 11 11 11
3 1
t
t
5 1
6 1
7 1
t
t
t
t
(θ)
t
V
E
1
!
u
!
E (θ)
(θ) ui
(θ)
Ileak = (θ)
CV (θ) τ (θ)κ (θ)
gi (t) = gij
tkj )
!τ τ
1
τ2
V
I
V V
Ii,j
k tdn j =
dt
=
t
t
t
(θ)
(θ)
(θ)
(θ)
= αn(1 − n) − βnn
Vj=1 dtV VjT Ii,j(θ) V j=2 (θ)Vj=3(θ) Vi CU τ C gijβnδn(θ)(t − tkj ) I αn(1 =− Vτ CE (θ) ui dn τ (θ)n) κ − dt =leak θ
θ θ (θ) E θ uθ I θ g (θ)=(t)CV − tkj ) θκ τ= ggiji,j δtkj(t = i g δ (t − tkj ) i leak
=
θ
(V ) 1)−Vnu (Vn ) T ime (ms) τ τ1 V τ2τ β g n(µA/V
gij δτ(θ)(t −ntkj ) (θ)
t
Vdnj=1 Vj=2 Vnj=3 Vτ β (V ) V1i −Vjn Ini,j Vτ C
β (V ) 1 − n n
V
t
t13 t23 (θ)t33 t43 t53 t63 t73
1 =j CV = 2 jg =δ3(θ)(t − tk ) (θ) E dn ui j I=leak ij j τ (θ)κ Vdtτ j=β=α 1 − n n) − n(V n1(1) − j = 2 βjnnn= 3
1
V
t
V (θ)V6 j I7i,jk Vτ C (θ)Vj=1 (θ) V 3) V1j=3 4 g (θ) ui V = (VtCV n53 iδn tI13τleakβt23nj=2 t3(t t−3 tj ) 3τ (θ)κt3− tij
j=1 j=2 j=3
j = 1 j = j2 =j 1= j3 = 2 j = 3 t11 t21 t31 t41 t51 t61 t71 dn = α V(1 VVj=1 VVj=2 VVi τ n) VC Ii,j βVn C j − τn Vj=1 1 Vj=2 Ii,j − j=3 i Vjnj=3 t2 t22 t32 dt t42 t52 t62 t72 j=1 j=2 j=3 i j (θ) (θ) (θ) (θ) CV (θ) (θ) (θ) E (θ) CV (θ)I (θ)=(θ) (θ) k u g δ (t − tkj ) E u gij7 δ τ (θ) (tκ− tjij) I3 = 4 iτ5(θ)κ 6leak t13 i t23 tleak 3 t3 t3 t3 t3 (θ) (θ) (θ) = θ θ θ gij(θ) tkj CV g (t)θ= ggθi (t) tk == j = i1 j = 2 ijji= 3j leak i,j τ θκ τ1 τ2 ) gVu(µA/V ) ime Vu (V ) T ime (ms) τ τ1 τ2 gτ (µA/V (V ) T (ms) Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C
t12 t22 t32 t42 t52 t62 t72
3CV1(θ)4 j =(θ) 5 2 (θ) 6j =73 k (θ) 1 (θ)2 j = Vj=3 uVi j=22Ileak δ 2V(tτ − 2 = 2Vτ (θ) 2Ii,j 2 Ctj ) i κ2Vj gij
t
(θ) Vj=1 7 E
1
3 2 43 45 56 6 7 7 t1 t2t13j t= t31 tt31 t3t1 t3 1t3V2tt13V j 1= 1V1j=3 Vj=2 i j j= I3i,j Vτ C
Vj=1
Vτ βn(V ) 1 − n n
1
Fig. 3. The circuit implementing the dn synapse element consisting of 3 MOS transistors. dt = αn(1 − n) − βnn αn(1 − n) − βnn
g (µA/V ) Vu (V ) T ime (ms)
where θ indicates the synapse type. The partitions pool each of the synaptic contributions from the respective presynaptic neurons as: X (θ) X X X (θ) gij f (θ) (t − tkj )(Vi − E (θ) ) Iij =
(θ)
(θ)
t71
E (θ) (θ) (θ) (θ)1 2CV (θ) 6 tk7) 53 6 g 4 7 δ 5 (t − 63 74 t12 t52t22 t= t12 uti22 t32 It42leak 3 2t2 t32tτ2(θ)tκ23 t2 tij 32 3 3 j3
(θ)
gij δ (θ)(t − tkj ) tkj =
gi (t) = gij
t61
1 2 3 4 5 6 7 t13 t23 t33 t43t3 t53t3 t63t3 t73t3 t3 t3 t3
k jtj= = 3
CV (θ) τ (θ)κ
t1 t1 t1 t1 t1 t1 t1
t5 31 1
1
1
1
tV12j=3t22 Vit32Vjt42 Ii,jt52 Vtτ62 Ct2 V V ! τ j=1t t j=2 t t t tt tt tt t t t
Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C E (θ) ui
t41 j=
1
1 2 3 4 5 6 7 2 t32 t42 t52 t62 t72 t2 t2 t13 j t= t33 jt3=4 t33 5 6 7 3 1t31 j t= 32 2
(θ)
CV (θ) τ (θ)κ
=
1
(θ) (θ) Vj=1EV Vj=3 (θ)= Vi 25VCV Ii,j (θ) Vτ (θ)C (θ)j=2 k j 6= t12 jut=i22 1t32Ijleak t42 = t2 τj(θ) t2κ t372gij δ (t − tj ) (b) (θ) (θ) (θ) (θ) CV (θ) (θ) k (θ) EVj=1ui V1j=2Ileak g (θ) δ (t − tj ) = τ4(θ) V= 6 Ii,j7tk V= iκ 5Vgjij τ C i3 (t) ij t3 t23 Vgtj=3 3 t3 t3 t3 t3 j (θ) (θ) (θ)(θ) k (θ) (θ) (θ) k gij tgj ijn)=δ− dn = CV E (θ) ui g1ji = (t I(t) (θ)(1 − β− nt ) leak = α t1 t21 1dtt31j = t41τ n2tκ51 j t=61 3t71 n j dn (θ) (θ) αn(1 n g = (t) =− g n) −tkβn= 1 i 2Vj=3 3 V 4 ij 5V j6I 7 Vτ C Vj=1 Vtdt j=2 t2 2 t2 t2 t2 i t2 j t2 i,j
(θ)
1
(θ)
Ileak =
tt23 tt23 tt23 tt23 tt23 tt23 tt23
t12 t22 t32 t42 t52 t62 t72
j
θ
=
Fig. 2. Illustration of the convolution between the conductance dynamics and conductance strength using two versions (a) a single decay τ , and (b) a rise and fall time τ1 and τ2 .
j
XX θ
k (θ) gi (t)(Vi
k
− E (θ) )
(5)
(θ)
synaptic inputs from these elements further interface through AER arbitration to generate postsynaptic output events [12]. III. S YNAPSE E LEMENT
j
A. Modeling of Conductance Dynamics
(θ)
We assume a general conductance-based synapse with continuous activation dynamics. The postsynaptic membrane receives synaptic current contributions, X XX Iij = gij fij (t − tkj )(Vi − Eij ) (1) j
j
k
where i denotes the post-synaptic neuron, j denotes the presynaptic neuron, k indicates the spiking event number, gij is the conductance strength between neuron i and neuron j, fij (t − tkj ) indicates the conductance dynamics profile, Vi is the membrane voltage of pre-synaptic neuron i, and Eij is the reversal potential between neuron i and neuron j. Synaptic current contributions to postsynaptic neuron i are partitioned according to synapse type as X X X X Iij = Iij (1) + Iij (2) + . . . + Iij (k) (2) j
j
j
∀ i, j
(3)
fij = f (θ) ∀ i, j
(4)
and activation dynamics (θ)
=E
(θ)
The temporal profile of gi (t) is illustrated in Fig. 2. A logdomain recurrence relation expressing this pooled conductance leads to compact realization as described next. B. Linear and Log-Domain Recurrence Relation A general conductance dynamics profile f (θ) can be char(θ) (θ) acterized by two terms: a fall time τ1 and a rise time τ2 . We start by modeling the transient conductance dynamics as a single decaying exponential with time constant τ (θ) as illustrated in Fig. 2(a), and note that the more general case can be implemented by convolution of the activation (θ) functions gij with decaying exponential on shorter time scale as illustrated in Fig. 2(b). The convolution between the conductance dynamics and conductance strength using a single delay τ (θ) is expressed as: (τ (θ)
j
where each partition serves synapses with common synapse parameters in terms of reversal potentials (θ) Eij
where gi (t) denotes the time-multiplexed pooled conductance of synapse element (θ) of postsynaptic neuron i: X (θ) (θ) gi (t) = gij f (θ) (t − tkj ). (6)
X (θ) d (θ) + 1)gi (t) = gij δ(t − tkj ) dt j
(7)
where δ(t − tkj ) is an impulse centered at time tkj , representing a presynaptic input event from neuron j of synapse type θ to postsynaptic neuron j. We utilize a log-domain circuit to exploit the linear relationship between the subthreshold MOSFET gate-source voltage
User: teddy
j=1 j=2 j=3
(θ)
(θ)
(θ)
.
Transient Response
u
=
(θ)
CV (θ) τ (θ)κ
V (mV)
0
tkj =
20.90
20.95
(θ)
0
0
5.0
tkj
(θ) gij
=
4 (θ)5 t12gi(θ)t22(t)t3= 2 t2gijt2
7 6 ttkj2 =t2
(θ) gi (t)
V (mV)
200
10
15 time (ms)
20
6 5 4 3 t73 (ms) t13 t23 (a) )t3 Vut3(Vt3) tT3 ime τ g (µA/V Date: Feb 11, 2010 7:39:24 PM PST
1
4 0 0/V_u
25
30
tezzaron test_synapse_v0 schematic : Feb 11 18:01:17 2010 52
j=1 j=2 j=3
(12)
21.05
7 6 (θ) 4 (θ)5 t(θ) k tCV 1 δt1 (t − tj ) 1 t1 gij τ (θ) κ
Ileak =
(11)
400
CV (θ) τ (θ)κ
(10)Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C 600
21.0 time (ms)
Vu (V ) T ime (ms)
Transient Response
E
8 0/V_in 0
11.05
Graph Window 54
Date: Feb 11, 2010 6:56:11 PM PST
(θ)
gi (t) = gij
11.0 time (ms)
200
(θ)
10.95
(θ)
gij δ (θ)(t − tkj )
400
(θ)
(θ)
tkj =
600
gi (t) = gij
10.9
(θ)
(θ)
Ileak =
V (mV)
0
j=1 j=2 j=3
User: teddy
(tk j −)
Graph Window 60
9:03:16 PM PST
(θ)
1.05
User: teddy
αni(Vi), βni(V αini) (Vi), βni(Vi)
(θ)
Date: Feb 11, 2010
8 0 0 /V_in
E (θ) ui
1.0 time (ms)
(θ) ui 1 t1
.950
αhi(Vi), βhi(Vαihi ) (Vi), βhi(Vi)
(θ)
ui (tkj + ) = ui (tkj − ) + gij e−ui
(θ)
gi (t) = gij
g (µA/V ) V (V ) τ g (µA/V ) Vu (V ) T imeτ (ms) t31 t32 t33 ut34 t35
(θ)
tkj =
(θ)
200
(θ)
and at the arrival of an event tkj , for → 0:
User: teddy
6 5 4 3 t7 t1 t2 g t(µA/V 3 t3) Vt3(V )t3T ime (V ) T3 imeτ3(ms) τ 3g (ms) (µA/V )
(θ)
.900
(θ)
400
(θ) Ileak = t12 t13 (θ) g (t) = t21 i t22 t23
(θ)
0
600
E (θ) ui
t − tkj − (θ) , tkj < t < tk+1 j τ
j=1 j=2 j=3
6(θ)(θ)(θ)7 (θ) k (θ) CV 5 4 CV (θ) tg12ij(θ)δ(θ)Et(t22(θ)−utt(θ)ikj32) Itleak ) = τ κ 2 =tτ2 κ tg2ijE δt2(tui− tIj leak
(θ)
200
gi (t) = gij
(9)
αmi(Vi), βmi(V i)(Vi), βmi(Vi) αmi
=
(θ) ui (tkj )
Graph Window 60
9:03:01 PM PST
Transient Response
− tk ) gij δ (θ)V(t j=1 Vj=2 jVj=3
CV (θ) τ (θ)κ
3 2 (θ) (θ) (θ) t11 Itleak The solution to the integrator with constant delay (10) in 1 t1= u E i between events tkj and tk+1 is: j (θ) ui (t)
Date: Feb 11, 2010
8 0 0 /V_in
τ g (µA/V ) Vu (V ) T ime (ms)
t13 t23 t33 t43 t53 t63 t731
g (pA/V ) Vgu (pA/V (mV ) ) TVime (ms) Vj=2 Vj=3 Vi Vj Ii,j Vτ C Vj=1(ms) ) T ime u (mV
(θ)
V (mV)
(8)
400
j=1 j=2 j=3
i
600
(θ)
(θ)
Ileak =
t12 t22 t32 t42 t52 t62 t72
(θ)
X gij (t) d (θ) ui + 1 = δ(t − tkj ). (θ) dt g (t) j
t13 t23 t33 t43 t53 t63 t73
Vj=3 Vi Vj Ii,j VVj=1 Vi,jj=1 VτVj=2 C Vj=3 Vi Vj Ii,j Vτ C τ CVj=2 Vj=3 Vi Vj I
leading to τ (θ)
t12 t22 t32 t42 t52 t62 t72
Transient Response
E (θ)
i
t72
Graph Window 60
(θ) ui t11
d (θ) gi (t) (θ) dt g (t)
t62
tjk =
t13 t23 t33 t43 t53 t63 t73Vj=1
1
1
(θ)
E (θ) ui
t52
t1 t1 t31 j =t411 jt=51 2 tj61= 3t71
9:01:34 PM PST
8 0 0 /V_in
t42
(θ)
τ g (µA/V ) Vu
j=1 j=2 j=3
=
t31 t32 t33 t34 t35 t36 t37
d (θ) u dt i
t21 t22 t23 t24 t25 t26 t27
=
Vj=1 Vj=2 in the log-domain:
(θ) log gi (t)
Date: Feb 11, 2010
t11 t21 t31 t41 t51 t61 t71
(θ) ui
t11 t12 t13 t14 t15 t16 t17
and channel currents. So we express
t32
j = 1 j = 2 j 1= 3 2 User: teddy
(θ) gi
(θ) k Ileak = CV τ (θ)κ gij δ (t − tj ) t12 t13 t14 t15 t16 t17 (θ) (θ) gi (t) = gij tjk = t21 t22 t23 t24 t25 t26 t27 τ g (µA/V ) Vu (V ) T ime (ms) t31 t32 t33 t34 t35 t36 t37
Vj=2 Vj=3 Vi Vj Ii,j Vτ C
t13 t23 t33 t43 t53 t63 t73
t22
gij δ (θ)(t − tjk )
t12
Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C
t72
j=1 j=2 j=3
t62
T ime (ms) t36 t37
t52
− tjk )
t42
Vi Vj Ii,j Vτ C
t32
(θ) CV (θ) g δ (θ)(t (θ) tτ14 κt15 tij16 t17 (θ) g tk = t24 ij t25 jt26 t27
t22
(θ)
t12
t1 t1 t1 t1 t1 t1 t1
1
t1 t1 t1 t1 t1 t1 t1
1
1
t1 t1 t1 t1 t1 t1 t1
Transient Response
1
Transformed back to the current domain, the resulting con-Vj=1 Vj=2 7 Vτ C 4 i 5Vj t6 Ii,j t31 tV t11 t21Vj=3 (θ) 1 t1 1 t1 ductance gi follows the desired linear dynamics in input (θ) (θ) (θ) (θ) activation: 7 (t − tk 5 g 6 δt(θ) 4 3= CV 2 E (θ) ui t12 Itleak j) t t τ2(θ) κt2 tij 2 2 2 2 (θ) k (θ) k (θ) gi (tj + ) = gi (tj − ) + gij (13) and exponential decaying conductance in between presynaptic events with time constant: (θ)
(θ) −(t−tk j )/τ
(θ)
gi (t) = gi (tkj )e
, tkj < t < tk+1 . j
IV. C IRCUIT A RCHITECTURE
(14)
350
300
V (mV)
250
200
150
(θ) (θ) 4g 5 tk6 =7 3 t13gi t23(t)t= 3 t3 ijt3 tj 3 t3 100
50.0
User: teddy
0 Feb 11, 2010 7:53:34 PM PST Date: 0
5.0
Graph Window 55 10
15 time (ms)
20
) jVu=(V τ g (µA/V 3 (ms) =ime 2 )j T j = 1(b)
25
30
Transient Response
125 /MN2/S
Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C 100
The common reversal potential parameter for each synapse (θ) (θ) (θ) (θ) partition E (θ) is simply implemented as a single nMOS gij δ (θ)(t − tkj ) E (θ) ui Ileak = CV τ (θ) κ transistor operating in the subthreshold region: I (pA)
75.0
50.0
25.0
IN M OS = λI0 e
κn Vg /UT
(e
−Vs /UT
−e
−Vd /UT
),
(15)
where Vg is the gate voltage, Vs is the source voltage, Vd is τ the drain voltage, λ is the W/L ratio of the transistor, I0 is the subthreshold pre-exponential current factor, κn indicates the back gate effect, and UT is the thermal voltage, kT /q. The transistor operates in the subthreshold region while the drain-to-source voltage is less than 4UT . Since the voltages are implemented in log-domain circuits, the resulting output current can be expressed as: (16)
I ∝ κVu (Vi − E (θ) ),
To implement the input recurrence (12) composed of the (θ) input term of the incoming conductance strength value gij (θ)
k
multiplied by a negative exponential e−ui (tj ) , we utilize CMOS technology to implement the negative exponential (θ) k e−ui (tj ) with a single diode-connected pMOS transistor operating in the subthreshold region: IP M OS ∝ e−κp Vg /UT ,
(17)
We activate the pMOS with a short pulse centered at tkj . (θ) The conductance strength gij can in principle be implemented by modulating the pulse voltage logarithmically. Rather than
(θ) gi (t)
(θ)
= gij 0
0
5.0
tkj = 10
15 time (ms)
20
) Vu (V ) T ime (ms) g (µA/V(c)
25
30
Fig. 4. Transistor-level circuit simulation illustrating both the: a) activation (θ) function gi with 3 groups of different activation widths (detail shown in inset); b) log-domain variable u and c) time-domain conductance g.
adding this complication to the circuit and the drivers at the periphery of the array, we modulate the pulse width linearly in (θ) the conductance strength gij . Notice that a back gate effect parameter κn and κp is present in both of the expressions for the input (13) and output (15) of the synapse element. The κ parameter indicates the efficiency of a change the gate voltage and the resultant change in surface potential. This loss in efficiency is due to the bulk terminal in a MOSFET, which can act as another gate terminal (also referred to as the ’back-gate effect’). Fortunately, this effect will have little consequence if the nMOS and pMOS devices have sufficiently close back-gate effects, κn ≈ κp . By virtue of the log-domain transformation, the decaying exponentials e−t/τ in the conductance dynamics fij (t − tkj ) are implemented using a single nMOS transistor operated in
120
implementation is compact, requiring only 3 transistors. This small footprint, coupled with the low-power subthreshold design, make this design a suitable candidate for large-scale implementation of synaptic arrays in addressable neuromorphic systems, with reconfigurable synaptic connectivity as well as individually selectable synaptic dynamics.
Fg (pA/V)
100 80 60
ACKNOWLEDGMENT This research was supported by NIH/NIA R01AG029681.
40
R EFERENCES
20 0 0
20 40 60 Pulse Width (os)
80
Fig. 5. Transistor-level circuit simulation illustrating ∆g for 4 groups of different activation widths.
subthreshold and used as a constant current source to linearly charge capacitor C. As shown in Fig. 2(b), the conductance dynamics fij (t − tkj ) can be extended to a rise time τ1 and fall time τ2 through convolution. This could be accomplished by driving the source of the pMOS with a sequence of pulses. The complete dynamical conductance-based synapse circuit implementation is shown in Fig. 3. The circuit is compact, requiring only 3 transistors to implement. V. C HARACTERIZATION To verify the conductance dynamics, we performed transistor-level simulations (using Spectre and parameters of a 0.13µm CMOS process) of the synapse circuit driven by a train of presynaptic impulses, modulated with three different pulse widths, with relative magnitudes 1, 3, and 5, emulating the effect of three time-multiplexed pooled synapses. The circuit output in response to the sequence of input synaptic events is shown in Fig. 4. To verify the linearity of postsynaptic conductance in presynaptic activation, we studied the dependence of the conductance time profile as a function of pulse width and pulse interval. We observed the step in conductance ∆g for a train of pulses at variable pulse intervals, for four different values of pulse width with relative magnitudes 1, 3, 5, and 7 as shown in Fig. 5. The four distinct and compact groups for each of the four different activation widths indicate the linearity of the conductance according to the convolution model (13) and (14). Furthermore, the centers of the clusters for each of the different activation widths are colinear through the origin, confirming linearity in input pulse width. VI. C ONCLUSION We have formulated a dynamical conductance-based synapse cell in a compact circuit design. Circuit simulations verify log-domain implementation as well as an output magnitude scaled to the input conductance strength. The circuit
[1] Mead, C.A., Analog VLSI and Neural Systems, Addison-Wesley, 1989. [2] Serrano-Gotarredona, R.; Oster, M.; Lichtsteiner, P.; Linares-Barranco, A.; Paz-Vicente, R.; Gomez-Rodriguez, F.; Camunas-Mesa, L.; Berner, R.; Rivas-Perez, M.; Delbruck, T.; Shih-Chii Liu; Douglas, R.; Hafliger, P.; Jimenez-Moreno, G.; Ballcels, A.C.; Serrano-Gotarredona, T.; AcostaJimenez, A.J.; Linares-Barranco, B. ”CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware SensoryProcessing LearningActuating System for High-Speed Visual Object Recognition and Tracking”, IEEE Trans. Neural Networks, vol. 20(9), pp. 1417-1438, 2009. [3] Serrano-Gotarredona, R.; Serrano-Gotarredona, T.; Acosta-Jimenez, A.; Serrano-Gotarredona, C.; Perez-Carrasco, J.A.; Linares-Barranco, B.; Linares-Barranco, A.; Jimenez-Moreno, G.; Civit-Ballcels, A. ”On RealTime AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing”, IEEE Trans. Neural Networks, vol. 19(7), pp. 11961219, 2008. [4] Badoni, D.; Giulioni, M.; Dante, V.; Del Giudice, P. ”An aVLSI recurrent network of spiking neurons with reconfigurable and plastic synapses”, Proc. IEEE ISCAS 2006, pg. 1227-1230, 2006. [5] Bartolozzi, C.; Indiveri, G. ”Synaptic Dynamics in Analog VLSI”, Neural Computation, vol. 19(10), pp. 2581-2603, 2007. [6] Lazzaro, J. P., ”Low-power silicon axons, neurons, and synapses”, in Zaghloul, M.E.; Meador, J.L.; and Newcomb, R.W. (Eds.), Silicon implementation of pulse coded neural networks, Norwell, MA: Kluwer, pp. 153164, 1994. [7] Arthur, J.; Boahen, K. ”Recurrently connected silicon neurons with active dendrites for one-shot learning”, IEEE International Joint Conference on Neural Networks, vol. 3, pp. 1699-1704, 2004. [8] Boahen, K.A. ”Retinomorphic vision systems: Reverse engineering the vertebrate retina”, Unpublished doctoral dissertation, California Institute of Technology, 1997. [9] Hynna, K.; Boahen, K. ”Space-rate coding in an adaptive silicon neuron”, Neural Networks, vol. 14, pp. 645656, 2001. [10] Chicca, E. ”A neuromorphic VLSI system for modeling spike-based cooperative competitive neural networks”, Unpublished doctoral dissertation, ETH Zurich, Zurich, Switzerland, 2006. [11] Merolla, P.; Boahen, K. ”A recurrent model of orientation maps with simple and complex cells” in Thr¨ un, S.; Saul, L.K., Sch¨ olkopf, B. (Eds.), Advances in neural information processing systems, MIT Press. Cambridge, MA, vol. 16, pp. 995-1002, 2004. [12] Boahen, K. ”Point-to-point connectivity between neuromorphic chips using address events”, IEEE Trans. on Circuits and Systems II, vol. 47, pp. 416-434, 2000. [13] Goldberg, D.H.; Cauwenberghs, G.; and Andreou, A.G. ”Probabilistic Synaptic Weighting in a Reconfigurable Network of VLSI Integrate-andFire Neurons”, Proc. IEEE Int. Symp. Circuits and Systems, vol. 3, pp. 241-244, 2001. [14] Vogelstein, R.J.; Mallik, U.; Vogelstein, J.T.; and Cauwenberghs, G. ”Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses”, IEEE Trans. Neural Networks, vol. 18(1), pp. 253-265, 2007. [15] Vogelstein, R.J.; Mallik, U.; Culurciello, E.; Cauwenberghs, G.; and Etienne-Cummings, R. ”Saliency-Driven Image Acuity Modulation on a Reconfigurable Silicon Array of Spiking Neurons”, Adv. Neural Information Processing Systems (NIPS’2004), Cambridge: MIT Press, vol. 17, 2005. [16] Destexhe, A.; Mainen, Z.F.; and Sejnowski, T.J. ”Synthesis of Models for Excitable Membranes, Synaptic Transmission and Neuromodulation Using a Common Kinetic Formalism,” J. Comp. Neuroscience, vol. 1, pp. 195-230, 1994.
Department of Electrical and Computer Engineering, Jacobs School of Engineering 2 Department of Bioengineering, Jacobs School of Engineering t11 3 Institute for Neural Computation t12 1 2 3 4 6 7 University of California San tDiego, CA 92093 t51 tJolla, 1 t1 t1 t1La 1 t1 1
1
t21 t31 t41 t51 t61 t71 t22 t32 t42 t52 t62 t72
1
t3 t23 t33 t43 t53 t63 t73
t12 t22 t32 t42 t52 t62 t72 1
t11
1
2
3
4
5
6
1
1
t21t11 t31t21 t41t31 t51t41 t61t51 t71t61 t71 j=1 j=2 j=3
7
t12 t22t2 t32t2 t42t2 t52t2 t62t2 t72t2 t2 Abstract—We present a compact circuit architecture for analog t3 t3 t3 t3 t3 t3 t3 V1 V2 V3 Vi Vj Ii,j Vτ C 6 27 j = 3 VLSI realization of event-addressable neuromorphic arrays with 1 7t6 t7 t11 t21 t31 j t=41 1t51 j t= t 1 1 t4 t6t5 t(θ) t2t1 t3t(θ)23 t43t33 t53CU (θ)t3 (θ) E ui 3 3 I3leak = τ 3 κ3 3gij3 3δ (θ)3(t − tkj ) conductance-based synaptic dynamics. Synaptic input events are 1 2 3 4 5 6 7 1 V 2 3 V4 5 V6 7V V I 1 jt1=t1 1 tj1 =t12 j = 3 t2 t2 j=1 t2 t2j=2t2 tj=3 142 253 364 4175 526 637 74 5 6 7 t1 t1 j t= 2 t2t1i t2 j t31 i,j 1(θ) j = 2 j(θ)= 3 1 1 11 tt111 tt111 tt111 ttt1111 tt1t11 tt1t11 tt11 t1 t1 t1 time-multiplexed, pooled by synapse type according to common gi (t) = gij tkj = 1 2 3 4 5 6 7 1 2 3V 4 I5 6 I 7 t t t t t 2 2V 2 tV 2 t2 V V I t3 t3 t3 tj3 tr3 gt3 t3tr1 gt2 t31 tt142 tt253 tt364 ttt4175 tt5t26 tt6t37 tt74 t5 t6 t7 V21 V2 2Vj=1 reversal potential and activation dynamics. One such physical V V Ii,j Viτ Cj i,j j=2 3 i j j=3 2 2 22 222 222 222 2222 222 222 22 2 2 τ2 τ τ 1 2 13 24 g 5 (µA/V 6 7 ) Vu (V ) T ime (ms) t t t t t t t synapse element per postsynaptic neuron is provided for each 1 2 3 4 5 6 7 1 1 1 1 1 1 1 7 t3 t3(θ)t3 t3CUt3 t(θ) t11 t21 t31j = t41 1αt51nj(Vt=61) 2tβ =tt)131t11tt122− 233 3 2n 341414113452512522456362363356747347446175 45 55726 56 6637 67 774 7 5 6 (θ) 7 t3 u(θ) 2 1n n(V 1j IleakVj= Iτ r κg gij3 Iδr(θ)g(t − tkj ) i 11 1 311t1tt1t311t1tt1t313t1t1t1t1t31t313tt1t11t1t1t31t313tt1t11t1t1t31t313tt1t11t1t1t31333tt11tt13t133tt11tt13t33tt11tt133t1 t3 t3 Et3 type, selected by type index along with postsynaptic address. A 1 2 3 4 5 6 7 t2 t2 t2 t2 t2 t2 t2 5 6 7 j = 1(θ) j = 2 j(θ)= 3k t12 tV22 j=1t32 Vt42j=2tdn (1i − Vtt1j1n) 1 I2− 2 2β 133n3nt241t41t41t35t5122t52t46t623t633t57t734t7446 t45t575t56t6 t667t7 77 2dtVt= 2 αtn 2V j=3 i,j log-domain encoding of first-order linear dynamics of synaptic 1−n n (t)n(V = ) gβ Itsyn tsyn t2ttI2tsyn t2syn t2tIt2syn t2 tI2Isyn t2syn 3 4 5 6 gi 7 α ij n(Vtj) = 22t2tt22t2Ittsyn 2syn 2t2tItI 2syn 2 2 tI12syn 2 t 2 Isyn 2t22t2tt 2t22tI22I 1 2t2212 2 2t2212 22t2 12 2 t212 tt121 tt211 t1 t1t1 t2t1 t3t1 t4t1 t5 t6 t7 3 3 conductance results in a compact circuit realization witht13 three V3dn i Vj Ii,j Vτ C t23 t33Vjt43 Irt53 g t63 tI73r g t11 11t22 22t133133t244214413155312552426642366135377534772464 645t3575 756t46 6 67t57 7 7t6 τt73 Vτ1 3 τV23 Vg33 (µA/V (1u − βnn (ms) (Vn) ) − T ime t33tt31 t3I3ttm tt13mt3tIt13thtnm tt13tg3thgt13tttgt13t13ttI13ttntgt13tI13tng1tItm 13t13gth 1tItg31g13tI1tgt113 13tI1tn2 1g 3 1Itg4 1 t5 2 t6 t7 dt = )αnV 3t13t3th3t1t3tg13t3t1t3I MOS transistors per synapse element. Circuit simulations show 2 t21313t31t32231113 3t133t321321 313t31t4233113 t3t521413 ttt623251 tt32t7261 t2 (θ) 12 j2 = t71j 2=u(θ) (θ) 2 j = 3 (θ) Ileak = τCUκ gij δ (θ)(t − tkj ) 1 3 54 72 4 65 3 5 76 4 6 7 5 7 6 E 7 j = 1 j = 2 j = 3 i α (V ) β (V ) 1 − n n 1 1 2 2 3 1 2 1 3 2 4 3 5 4 6 n n low-power operation with linear dynamics in conductance. IIsyn t2g22tI2trIsyn I2t2tIgr32tsyn I2ttrg12tt2tg42t2t2tt53Itr2 t6g4 tt57 t6 t7 Isyn IrIttsyn I2tIsyn tI2Irsyn tg2 t2ItI2syn t2IgI22trgsyn syn rt2gr2tI g2tt g2It2syn rt2I gtsyn rt12syn syn syn syn 2tItsyn 2tt 2ttsyn 2I 2rt2I22tI 1
2
3
4
5
6
7
1
1
1
1
1
1
(θ)
T
1
1
syn1
2
syn2
1
11
1
1 1
1
1
(θ) 1
1
T
syn1
2
syn2
1
0
10
0
21
1
20 2
1
2
1
1
1
1
1
1
1
syn1
2
syn2
3
Na
3
43
NK a Na
4 L
4 3 K K
LN a L
4
L
K
1
(θ)
0
0syn 0 00
0 1011syn 01 syn 1 syn 22 21syn 212 1syn 1 20 2syn 2 2syn1 0syn 1 00 210212 02 syn 110syn
2
T
syn2
θ ...
1 1 1 3 1 3 13 13 31 3 j=33 Vi (θ) V I t1 t232125 2t43t3361223 54t72t4343265 3t5t537426 4t6t637525 7tt736V62 j=1t7 72 Vj=2gi(θ)V(t) = gij j tkj i,j= Vj=1 Vj=2 dn Vj=3αnV (1i −Vjn) I− i,j βnIn1 3 2 1 3I3214I4I1 3t tK− t tn t t3 n n dt = ItgntnI3N3g43ht(V It ntnI(V t(V t(V t3− 4)3 )33Kt3gIh33KtI)β β n α )3taIn33ntIn43(V n3ItKβ Iα Inm3h(V It3(V INLa33tLggang3KtLtN1(V tm3m tNga3m tαaβ3nhtInα tagtm t)1tn g3gN)I n3n g34) h31 g3t1 g− g4I3g32 ht3h gn34n m 3) 3115 − K Ktg 3I Ln aN3 Nm j = 33gL3ngL3jg34L= 6 2 7j = 3 t t2 t2 1 t22 t23 tV42j 5tI2r1 6gsyn17 Ir2 gsyn2 I. I NTRODUCTION )1 Vu2 (V3 ) 4T ime 2 2= 4j t= Vj Ir1 gsyn1 Ir2 gsyn2 dnjdn=t11 tj dn 2=Isyn 5 6(ms) 7 1t5It3syn 1τt6tτ 1 1tt71τ2 t1 g t(µA/V 1 t1 tn331gn) I1IgInsyn IIrrI00rg0gdn In2010gsyn Ir1(1 ItII− = = αIn) α (1 − − βsyn − n) β α β(1 −2syn n) I=r00IIα Ig0I11syn IdtIg(1 I30nrIgsyn 01− 3syn 3n 32 −V 3βnn V V I t1 t1 t1 t1 t1 t1 t1 rsyn rdt rsyn gdt r3syn rnn gn n gIsyn grI gI rgsyn rI rsyn r1g2n syn syn syn syn 1syn 0g 01− 2syn syn 22I syn 1r 1syn syn syn 2rg 2syn 0r 21g 2 12V dt 00syn 020syn 22 syn 1syn 2gV 0syn 101syn 212syn 2 j=3 i j i,j 1 2 j=1 3 1 42j=253αn(V 46 )5t7βn6(V 7) 1 − n n t t t t t t t t t t t t t αn(V ) biβn(V ) 1 − n n α (V ) β (Vαα)(V 3 3 3 3 3 3 3 1 2 3 4 5 6 2 2 2 2 2 2 2 Neuromorphic engineering [1] utilizes inspiration from 3 4 IN(V I C 3) 4)n − β β n )n IN4V In 3nh 4n m hI gj t2 t2 t2 t2 t2 t2 t72 αnαn(V )αV )3nh1β(V )a(V )(V 1(V (V 1g)3− )= n 1gβi(V − (V nK− 1IIngi,j − Im In nβ m m h) gIn nI g1 V n(V nm n gnIN(V n gn1n nnaV N ag L1 ah Kn V jK3g− = 2nnLIgVLngKnjjVn= 2)nIβα j4NK)g1Lag− τIrg1L3 Ir2 gsyn2 gdn syn1 1 2 3 dt4 = 5αn(1 6 −7 n) − βnn dn ology and neurobiology to direct and motivate the design and t t t t t t t dn dn dn dn I dndn 3nn 3I2 3 3 3 3 3 jrn(1 = 1Ir− = j 2= dt = αn(1 − n) − βnn I− = − α1= αβ − n) βjn(θ) Ir− In) Inr(1 In rα g− r2 g3syn t1 t2 t3 t4 t5 t6 t7 ==ααdn (1 α (1 n) − β2nr(1 − n n) n Idtr= In(1 Ig− I(θ) Ing0βn) In) (θ) 0n nα n nβ r− rsyn g= g− gsyn gβ r1 Iβ g1(V n0= ngn) ngg− n(θ) r(1 r− CU (θ) 0r 0dt 0dt syn 0 gsyn syn 1dt syn 2nTg dt dt dt 1syn 1 syn22 α2syn 2syn 2) β (V grgsyn −τ )tkjC)1 − n n 3 3 3 3 3 3 3 V1 0syn V113syn1τsyn2(θ) V ij 2 = jn δIi,j(tnV κi V modeling of circuits and systems. By emulating form and t11 t21 t31 t41 Et51 t61 0uti71V10 Ileak j = 1 j = 2 j = 3 α−nn)β(V )− 1V − n In V C (V )nV1(V )β 1nn− 1(Vn − n n j=1 j=2 j=3 )αnβ(Vα )α(V nn1(V nV n n)β n(V V V)dn 1)β inn= jαn(1i,j − n) τ− β (θ) nn (θ)2k 3 CUTdt 6 (θ) architecture in biological systems, neuromorphic engineering t12 t22 t32 t42 αtn52g(Vi(θ) (θ) tE t72 =u(θ) tj ==Vτ (θ)gκi1Vggij(θ) (t − tkjj) Ii,j i2δ gi3 2 (t) i gij I dnleak j=1 j=2β V j=3 Vi V dn dn dn dn (θ) V = α (1 − n) − n = = α− α(1 (1 −n− − n)β − βnβnnCU n n (θ) nn) V k V V V I (1 βn) (θ) (θ) nn) nn− dt seeks to emulate function as well. Investigation of neural t1 t2 t3 t4 tdt5 =t6 αdttn7(1=Edt−α(θ) un dtin n(θ)Ileak tj ) Ijj=2(t) j=3 i j i,j = (θ)κT gij δ (θ)(t − j=1 k (V V)τj T 3 3 3 τ 3 τ 3 τ3 3(θ) I I g (µA/V ) V ime (ms) rg gsyn2 tj = gi1r1 gsyngi2 1 2 gi (t) = gij u 2 i3 1 behavior on large scale requires efficient and realistic modeling Vj Ir1 gsyn1 Ir2 gsyn2 (θ) (θ) = gij α (V tkj ) =β (Vgi1) 1g−i2 n gni3 and implementation of neurons and their synaptic connections θj =.1. . j = 2 τ j =τ13 τ2 gig (t) (µA/V ) Vnu (V )n T ime (ms)αn(V ) βn(V ) 1 − n n Vj=1 Vj=2 Vj=3 Vi Vj Ii,j [2]-[4]. αun(1(V −)n) T −ime βnn (ms) τ τ1 τ2 g (µA/Vdn dn dt)= V dt = αn(1 − n) − βnn Analysis of a variety of different implementations of Vj Ir1 gsyn1 Ir2 gsyn2 1
1
1
conductance-based dynamical synapses, and new circuit that overcomes some of their limitations, are presented in [5]. In particular, [5] analyzes the trade-offs among the different implementations regarding functionality of the temporal dynamics and the required layout size, and offers a circuit with linear dynamics in conductance. Earlier implementations of VLSI synapses such as the pulse current-source synapse [1] and reset-and-discharge synapse [6] suffer from inability to integrate input spikes into continuous output currents and linearly sum postsynaptic currents respectively. Other previous circuits such as the linear charge-and-discharge synapse [7] and current-mirror-integrator synapse [8], [9] , and [10] also suffer from nonlinear summation of postsynaptic currents. The synapse implementations of log-domain integrator synapse [11] and diff-pair integrator synapse [5] implement linear summation of postsynaptic currents, but they require an Mw p-FET or additional transistors. Here we show that a variant on the log-domain implementation gives rise to linear conductance dynamics in more compact form. Here we present a three-transistor realization of a dynamical conductance-based synapse element, serving
αnFig. (V ) β1.n(V )Pooling 1 − n nof
synapses with common reversal potential and activation dynamics, but possibly with different conductances, by time-multiplexing dn = αn(1 − n) −from βnn j presynaptic neurons. events dtinput
multiple synapses with common reversal potential and activation dynamics. The time-multiplexing synapse element pools spike input events from multiple presynaptic source addresses through the address-event representation (AER, [12]) communication framework as seen in Fig. 1. II. S YNAPSE A RRAY A RCHITECTURE This paper focuses upon the architectural design of the pooled synapse input for each neuron within the neural array. We assume that the number of distinct synapse types is limited to a relatively low number k, e.g., k = 8. This assumption is typically valid even in large-scale cortical models. We pool synapses of the same type serving the same postsynaptic terminal into a time-multiplexed synapse element. Synapse elements in the array are activated by presynaptic events presented through an AER input interface [12]. Neurons receiving
1
tt21 tt22t1tt23t2tt24t3tt25t4tt26t5tt27t6 t7 1 13 13 13 13 13 13 3 tt131 2
tt232 2
33 44 jtt32=tt132
tjt535=tt6326 2 2
tjt737= 2
3
(θ)
E (θ) ui
4 2 5j =6 3 7 tj13 =tV231j=2t33j = Vj=1 Vtj=3 3 t3 Vti 3 Vtj3 Ii,j Vτ C t11 t21 t31 t41 t51 t61 t71 (θ) (θ) (θ) Vj=1EV V V Ii,j3g (θ) Vτδ (θ)C(t − tk ) (θ)j=2 j=3 jj = ju=i 1 Ijleak =i = 2 VCV (θ) j t12(a) t22 t32 t42 (θ)t52 τ t62κ t72 ij (θ) (θ) (θ) CV (θ) (θ) (θ) EV u ggjij(θ)Iδi,j tCkj ) I Vgj=3 = (t) k(tV− (θ) V V τ κ j=1 i Vj=2leak i τ = t = t13 t23 ti33 t43 t53 tij63 t73 j (θ) 4k (θ)5 (θ) (θ) (θ) CV (θ)6 t7 k gij tj n) tdn11 ==t21 α t3(θ) t− t= E (θ) ui gi I(t) 1δ−t1(t βn− n1 tj ) leak τ (θ) κ 1gij n1(1 j =2 1dt 3j =4 2 5 j =6 3 7 1 tdn1 (θ) t t t t t t 1 2 3 4 5 6 7 (θ) 1 1 1 1 1 1 α −kβ n t2n(1 =t2−gtijn) dtgi= (t) 2 t2tj tn2= t2 t2 Vj=1 Vj=2 V 3 V4 i 5Vj 6Ii,j7 Vτ C 11 22 j=3 ttdn 21 tt21t1tt231t2tt241t3tt251t4tt261t5tt271t6 t7 = α (1 −3n) − 3 (θ)β3nn 3 3 (θ)dt (θ) 3 n 3CV (θ) E (θ) ui 1 Ileak = (θ) gij δ (θ)(t − tk ) tt31 tt232 jtt333=tτt1434 κtjt535=tt6326 tjt737= 3 j 2 2 2 2 2 2 2 (θ) (θ) gi (t) = gij tkj = 1 = 21 3j =4 2 5j =6 3 7 j Vj=1 3 tt53 Vtti63 V tt13 tt2V3j=2 tt33 tVt4j=3 tt7j3 Ii,j Vτ C 1
1
1
1
1
1
E
dn (θ) (θ) (θ)αn(1CV − n) −(θ)β 7n(t 2 = 3 4 5 g 6 δn(θ) ui t13dt It= leak 3 t3 tτ3(θ)κt3 tij3 t3 (θ) gji =(t) 1
=j
(θ) =gij2
(θ) gi (t) 1
−
k gij1 δ (θ) 2 (t 3 − 4 tj5)
t3 t3 t3 t3 t3 t63 t73
t1 tk j = = 11
(θ) gij
t2 t3 j 1= 21
j
(θ)
(θ)
Ileak =
1 1
E
1
(θ)
2 1
41 52 63 74 11 11 11 11
3 1
t
t
5 1
6 1
7 1
t
t
t
t
(θ)
t
V
E
1
!
u
!
E (θ)
(θ) ui
(θ)
Ileak = (θ)
CV (θ) τ (θ)κ (θ)
gi (t) = gij
tkj )
!τ τ
1
τ2
V
I
V V
Ii,j
k tdn j =
dt
=
t
t
t
(θ)
(θ)
(θ)
(θ)
= αn(1 − n) − βnn
Vj=1 dtV VjT Ii,j(θ) V j=2 (θ)Vj=3(θ) Vi CU τ C gijβnδn(θ)(t − tkj ) I αn(1 =− Vτ CE (θ) ui dn τ (θ)n) κ − dt =leak θ
θ θ (θ) E θ uθ I θ g (θ)=(t)CV − tkj ) θκ τ= ggiji,j δtkj(t = i g δ (t − tkj ) i leak
=
θ
(V ) 1)−Vnu (Vn ) T ime (ms) τ τ1 V τ2τ β g n(µA/V
gij δτ(θ)(t −ntkj ) (θ)
t
Vdnj=1 Vj=2 Vnj=3 Vτ β (V ) V1i −Vjn Ini,j Vτ C
β (V ) 1 − n n
V
t
t13 t23 (θ)t33 t43 t53 t63 t73
1 =j CV = 2 jg =δ3(θ)(t − tk ) (θ) E dn ui j I=leak ij j τ (θ)κ Vdtτ j=β=α 1 − n n) − n(V n1(1) − j = 2 βjnnn= 3
1
V
t
V (θ)V6 j I7i,jk Vτ C (θ)Vj=1 (θ) V 3) V1j=3 4 g (θ) ui V = (VtCV n53 iδn tI13τleakβt23nj=2 t3(t t−3 tj ) 3τ (θ)κt3− tij
j=1 j=2 j=3
j = 1 j = j2 =j 1= j3 = 2 j = 3 t11 t21 t31 t41 t51 t61 t71 dn = α V(1 VVj=1 VVj=2 VVi τ n) VC Ii,j βVn C j − τn Vj=1 1 Vj=2 Ii,j − j=3 i Vjnj=3 t2 t22 t32 dt t42 t52 t62 t72 j=1 j=2 j=3 i j (θ) (θ) (θ) (θ) CV (θ) (θ) (θ) E (θ) CV (θ)I (θ)=(θ) (θ) k u g δ (t − tkj ) E u gij7 δ τ (θ) (tκ− tjij) I3 = 4 iτ5(θ)κ 6leak t13 i t23 tleak 3 t3 t3 t3 t3 (θ) (θ) (θ) = θ θ θ gij(θ) tkj CV g (t)θ= ggθi (t) tk == j = i1 j = 2 ijji= 3j leak i,j τ θκ τ1 τ2 ) gVu(µA/V ) ime Vu (V ) T ime (ms) τ τ1 τ2 gτ (µA/V (V ) T (ms) Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C
t12 t22 t32 t42 t52 t62 t72
3CV1(θ)4 j =(θ) 5 2 (θ) 6j =73 k (θ) 1 (θ)2 j = Vj=3 uVi j=22Ileak δ 2V(tτ − 2 = 2Vτ (θ) 2Ii,j 2 Ctj ) i κ2Vj gij
t
(θ) Vj=1 7 E
1
3 2 43 45 56 6 7 7 t1 t2t13j t= t31 tt31 t3t1 t3 1t3V2tt13V j 1= 1V1j=3 Vj=2 i j j= I3i,j Vτ C
Vj=1
Vτ βn(V ) 1 − n n
1
Fig. 3. The circuit implementing the dn synapse element consisting of 3 MOS transistors. dt = αn(1 − n) − βnn αn(1 − n) − βnn
g (µA/V ) Vu (V ) T ime (ms)
where θ indicates the synapse type. The partitions pool each of the synaptic contributions from the respective presynaptic neurons as: X (θ) X X X (θ) gij f (θ) (t − tkj )(Vi − E (θ) ) Iij =
(θ)
(θ)
t71
E (θ) (θ) (θ) (θ)1 2CV (θ) 6 tk7) 53 6 g 4 7 δ 5 (t − 63 74 t12 t52t22 t= t12 uti22 t32 It42leak 3 2t2 t32tτ2(θ)tκ23 t2 tij 32 3 3 j3
(θ)
gij δ (θ)(t − tkj ) tkj =
gi (t) = gij
t61
1 2 3 4 5 6 7 t13 t23 t33 t43t3 t53t3 t63t3 t73t3 t3 t3 t3
k jtj= = 3
CV (θ) τ (θ)κ
t1 t1 t1 t1 t1 t1 t1
t5 31 1
1
1
1
tV12j=3t22 Vit32Vjt42 Ii,jt52 Vtτ62 Ct2 V V ! τ j=1t t j=2 t t t tt tt tt t t t
Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C E (θ) ui
t41 j=
1
1 2 3 4 5 6 7 2 t32 t42 t52 t62 t72 t2 t2 t13 j t= t33 jt3=4 t33 5 6 7 3 1t31 j t= 32 2
(θ)
CV (θ) τ (θ)κ
=
1
(θ) (θ) Vj=1EV Vj=3 (θ)= Vi 25VCV Ii,j (θ) Vτ (θ)C (θ)j=2 k j 6= t12 jut=i22 1t32Ijleak t42 = t2 τj(θ) t2κ t372gij δ (t − tj ) (b) (θ) (θ) (θ) (θ) CV (θ) (θ) k (θ) EVj=1ui V1j=2Ileak g (θ) δ (t − tj ) = τ4(θ) V= 6 Ii,j7tk V= iκ 5Vgjij τ C i3 (t) ij t3 t23 Vgtj=3 3 t3 t3 t3 t3 j (θ) (θ) (θ)(θ) k (θ) (θ) (θ) k gij tgj ijn)=δ− dn = CV E (θ) ui g1ji = (t I(t) (θ)(1 − β− nt ) leak = α t1 t21 1dtt31j = t41τ n2tκ51 j t=61 3t71 n j dn (θ) (θ) αn(1 n g = (t) =− g n) −tkβn= 1 i 2Vj=3 3 V 4 ij 5V j6I 7 Vτ C Vj=1 Vtdt j=2 t2 2 t2 t2 t2 i t2 j t2 i,j
(θ)
1
(θ)
Ileak =
tt23 tt23 tt23 tt23 tt23 tt23 tt23
t12 t22 t32 t42 t52 t62 t72
j
θ
=
Fig. 2. Illustration of the convolution between the conductance dynamics and conductance strength using two versions (a) a single decay τ , and (b) a rise and fall time τ1 and τ2 .
j
XX θ
k (θ) gi (t)(Vi
k
− E (θ) )
(5)
(θ)
synaptic inputs from these elements further interface through AER arbitration to generate postsynaptic output events [12]. III. S YNAPSE E LEMENT
j
A. Modeling of Conductance Dynamics
(θ)
We assume a general conductance-based synapse with continuous activation dynamics. The postsynaptic membrane receives synaptic current contributions, X XX Iij = gij fij (t − tkj )(Vi − Eij ) (1) j
j
k
where i denotes the post-synaptic neuron, j denotes the presynaptic neuron, k indicates the spiking event number, gij is the conductance strength between neuron i and neuron j, fij (t − tkj ) indicates the conductance dynamics profile, Vi is the membrane voltage of pre-synaptic neuron i, and Eij is the reversal potential between neuron i and neuron j. Synaptic current contributions to postsynaptic neuron i are partitioned according to synapse type as X X X X Iij = Iij (1) + Iij (2) + . . . + Iij (k) (2) j
j
j
∀ i, j
(3)
fij = f (θ) ∀ i, j
(4)
and activation dynamics (θ)
=E
(θ)
The temporal profile of gi (t) is illustrated in Fig. 2. A logdomain recurrence relation expressing this pooled conductance leads to compact realization as described next. B. Linear and Log-Domain Recurrence Relation A general conductance dynamics profile f (θ) can be char(θ) (θ) acterized by two terms: a fall time τ1 and a rise time τ2 . We start by modeling the transient conductance dynamics as a single decaying exponential with time constant τ (θ) as illustrated in Fig. 2(a), and note that the more general case can be implemented by convolution of the activation (θ) functions gij with decaying exponential on shorter time scale as illustrated in Fig. 2(b). The convolution between the conductance dynamics and conductance strength using a single delay τ (θ) is expressed as: (τ (θ)
j
where each partition serves synapses with common synapse parameters in terms of reversal potentials (θ) Eij
where gi (t) denotes the time-multiplexed pooled conductance of synapse element (θ) of postsynaptic neuron i: X (θ) (θ) gi (t) = gij f (θ) (t − tkj ). (6)
X (θ) d (θ) + 1)gi (t) = gij δ(t − tkj ) dt j
(7)
where δ(t − tkj ) is an impulse centered at time tkj , representing a presynaptic input event from neuron j of synapse type θ to postsynaptic neuron j. We utilize a log-domain circuit to exploit the linear relationship between the subthreshold MOSFET gate-source voltage
User: teddy
j=1 j=2 j=3
(θ)
(θ)
(θ)
.
Transient Response
u
=
(θ)
CV (θ) τ (θ)κ
V (mV)
0
tkj =
20.90
20.95
(θ)
0
0
5.0
tkj
(θ) gij
=
4 (θ)5 t12gi(θ)t22(t)t3= 2 t2gijt2
7 6 ttkj2 =t2
(θ) gi (t)
V (mV)
200
10
15 time (ms)
20
6 5 4 3 t73 (ms) t13 t23 (a) )t3 Vut3(Vt3) tT3 ime τ g (µA/V Date: Feb 11, 2010 7:39:24 PM PST
1
4 0 0/V_u
25
30
tezzaron test_synapse_v0 schematic : Feb 11 18:01:17 2010 52
j=1 j=2 j=3
(12)
21.05
7 6 (θ) 4 (θ)5 t(θ) k tCV 1 δt1 (t − tj ) 1 t1 gij τ (θ) κ
Ileak =
(11)
400
CV (θ) τ (θ)κ
(10)Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C 600
21.0 time (ms)
Vu (V ) T ime (ms)
Transient Response
E
8 0/V_in 0
11.05
Graph Window 54
Date: Feb 11, 2010 6:56:11 PM PST
(θ)
gi (t) = gij
11.0 time (ms)
200
(θ)
10.95
(θ)
gij δ (θ)(t − tkj )
400
(θ)
(θ)
tkj =
600
gi (t) = gij
10.9
(θ)
(θ)
Ileak =
V (mV)
0
j=1 j=2 j=3
User: teddy
(tk j −)
Graph Window 60
9:03:16 PM PST
(θ)
1.05
User: teddy
αni(Vi), βni(V αini) (Vi), βni(Vi)
(θ)
Date: Feb 11, 2010
8 0 0 /V_in
E (θ) ui
1.0 time (ms)
(θ) ui 1 t1
.950
αhi(Vi), βhi(Vαihi ) (Vi), βhi(Vi)
(θ)
ui (tkj + ) = ui (tkj − ) + gij e−ui
(θ)
gi (t) = gij
g (µA/V ) V (V ) τ g (µA/V ) Vu (V ) T imeτ (ms) t31 t32 t33 ut34 t35
(θ)
tkj =
(θ)
200
(θ)
and at the arrival of an event tkj , for → 0:
User: teddy
6 5 4 3 t7 t1 t2 g t(µA/V 3 t3) Vt3(V )t3T ime (V ) T3 imeτ3(ms) τ 3g (ms) (µA/V )
(θ)
.900
(θ)
400
(θ) Ileak = t12 t13 (θ) g (t) = t21 i t22 t23
(θ)
0
600
E (θ) ui
t − tkj − (θ) , tkj < t < tk+1 j τ
j=1 j=2 j=3
6(θ)(θ)(θ)7 (θ) k (θ) CV 5 4 CV (θ) tg12ij(θ)δ(θ)Et(t22(θ)−utt(θ)ikj32) Itleak ) = τ κ 2 =tτ2 κ tg2ijE δt2(tui− tIj leak
(θ)
200
gi (t) = gij
(9)
αmi(Vi), βmi(V i)(Vi), βmi(Vi) αmi
=
(θ) ui (tkj )
Graph Window 60
9:03:01 PM PST
Transient Response
− tk ) gij δ (θ)V(t j=1 Vj=2 jVj=3
CV (θ) τ (θ)κ
3 2 (θ) (θ) (θ) t11 Itleak The solution to the integrator with constant delay (10) in 1 t1= u E i between events tkj and tk+1 is: j (θ) ui (t)
Date: Feb 11, 2010
8 0 0 /V_in
τ g (µA/V ) Vu (V ) T ime (ms)
t13 t23 t33 t43 t53 t63 t731
g (pA/V ) Vgu (pA/V (mV ) ) TVime (ms) Vj=2 Vj=3 Vi Vj Ii,j Vτ C Vj=1(ms) ) T ime u (mV
(θ)
V (mV)
(8)
400
j=1 j=2 j=3
i
600
(θ)
(θ)
Ileak =
t12 t22 t32 t42 t52 t62 t72
(θ)
X gij (t) d (θ) ui + 1 = δ(t − tkj ). (θ) dt g (t) j
t13 t23 t33 t43 t53 t63 t73
Vj=3 Vi Vj Ii,j VVj=1 Vi,jj=1 VτVj=2 C Vj=3 Vi Vj Ii,j Vτ C τ CVj=2 Vj=3 Vi Vj I
leading to τ (θ)
t12 t22 t32 t42 t52 t62 t72
Transient Response
E (θ)
i
t72
Graph Window 60
(θ) ui t11
d (θ) gi (t) (θ) dt g (t)
t62
tjk =
t13 t23 t33 t43 t53 t63 t73Vj=1
1
1
(θ)
E (θ) ui
t52
t1 t1 t31 j =t411 jt=51 2 tj61= 3t71
9:01:34 PM PST
8 0 0 /V_in
t42
(θ)
τ g (µA/V ) Vu
j=1 j=2 j=3
=
t31 t32 t33 t34 t35 t36 t37
d (θ) u dt i
t21 t22 t23 t24 t25 t26 t27
=
Vj=1 Vj=2 in the log-domain:
(θ) log gi (t)
Date: Feb 11, 2010
t11 t21 t31 t41 t51 t61 t71
(θ) ui
t11 t12 t13 t14 t15 t16 t17
and channel currents. So we express
t32
j = 1 j = 2 j 1= 3 2 User: teddy
(θ) gi
(θ) k Ileak = CV τ (θ)κ gij δ (t − tj ) t12 t13 t14 t15 t16 t17 (θ) (θ) gi (t) = gij tjk = t21 t22 t23 t24 t25 t26 t27 τ g (µA/V ) Vu (V ) T ime (ms) t31 t32 t33 t34 t35 t36 t37
Vj=2 Vj=3 Vi Vj Ii,j Vτ C
t13 t23 t33 t43 t53 t63 t73
t22
gij δ (θ)(t − tjk )
t12
Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C
t72
j=1 j=2 j=3
t62
T ime (ms) t36 t37
t52
− tjk )
t42
Vi Vj Ii,j Vτ C
t32
(θ) CV (θ) g δ (θ)(t (θ) tτ14 κt15 tij16 t17 (θ) g tk = t24 ij t25 jt26 t27
t22
(θ)
t12
t1 t1 t1 t1 t1 t1 t1
1
t1 t1 t1 t1 t1 t1 t1
1
1
t1 t1 t1 t1 t1 t1 t1
Transient Response
1
Transformed back to the current domain, the resulting con-Vj=1 Vj=2 7 Vτ C 4 i 5Vj t6 Ii,j t31 tV t11 t21Vj=3 (θ) 1 t1 1 t1 ductance gi follows the desired linear dynamics in input (θ) (θ) (θ) (θ) activation: 7 (t − tk 5 g 6 δt(θ) 4 3= CV 2 E (θ) ui t12 Itleak j) t t τ2(θ) κt2 tij 2 2 2 2 (θ) k (θ) k (θ) gi (tj + ) = gi (tj − ) + gij (13) and exponential decaying conductance in between presynaptic events with time constant: (θ)
(θ) −(t−tk j )/τ
(θ)
gi (t) = gi (tkj )e
, tkj < t < tk+1 . j
IV. C IRCUIT A RCHITECTURE
(14)
350
300
V (mV)
250
200
150
(θ) (θ) 4g 5 tk6 =7 3 t13gi t23(t)t= 3 t3 ijt3 tj 3 t3 100
50.0
User: teddy
0 Feb 11, 2010 7:53:34 PM PST Date: 0
5.0
Graph Window 55 10
15 time (ms)
20
) jVu=(V τ g (µA/V 3 (ms) =ime 2 )j T j = 1(b)
25
30
Transient Response
125 /MN2/S
Vj=1 Vj=2 Vj=3 Vi Vj Ii,j Vτ C 100
The common reversal potential parameter for each synapse (θ) (θ) (θ) (θ) partition E (θ) is simply implemented as a single nMOS gij δ (θ)(t − tkj ) E (θ) ui Ileak = CV τ (θ) κ transistor operating in the subthreshold region: I (pA)
75.0
50.0
25.0
IN M OS = λI0 e
κn Vg /UT
(e
−Vs /UT
−e
−Vd /UT
),
(15)
where Vg is the gate voltage, Vs is the source voltage, Vd is τ the drain voltage, λ is the W/L ratio of the transistor, I0 is the subthreshold pre-exponential current factor, κn indicates the back gate effect, and UT is the thermal voltage, kT /q. The transistor operates in the subthreshold region while the drain-to-source voltage is less than 4UT . Since the voltages are implemented in log-domain circuits, the resulting output current can be expressed as: (16)
I ∝ κVu (Vi − E (θ) ),
To implement the input recurrence (12) composed of the (θ) input term of the incoming conductance strength value gij (θ)
k
multiplied by a negative exponential e−ui (tj ) , we utilize CMOS technology to implement the negative exponential (θ) k e−ui (tj ) with a single diode-connected pMOS transistor operating in the subthreshold region: IP M OS ∝ e−κp Vg /UT ,
(17)
We activate the pMOS with a short pulse centered at tkj . (θ) The conductance strength gij can in principle be implemented by modulating the pulse voltage logarithmically. Rather than
(θ) gi (t)
(θ)
= gij 0
0
5.0
tkj = 10
15 time (ms)
20
) Vu (V ) T ime (ms) g (µA/V(c)
25
30
Fig. 4. Transistor-level circuit simulation illustrating both the: a) activation (θ) function gi with 3 groups of different activation widths (detail shown in inset); b) log-domain variable u and c) time-domain conductance g.
adding this complication to the circuit and the drivers at the periphery of the array, we modulate the pulse width linearly in (θ) the conductance strength gij . Notice that a back gate effect parameter κn and κp is present in both of the expressions for the input (13) and output (15) of the synapse element. The κ parameter indicates the efficiency of a change the gate voltage and the resultant change in surface potential. This loss in efficiency is due to the bulk terminal in a MOSFET, which can act as another gate terminal (also referred to as the ’back-gate effect’). Fortunately, this effect will have little consequence if the nMOS and pMOS devices have sufficiently close back-gate effects, κn ≈ κp . By virtue of the log-domain transformation, the decaying exponentials e−t/τ in the conductance dynamics fij (t − tkj ) are implemented using a single nMOS transistor operated in
120
implementation is compact, requiring only 3 transistors. This small footprint, coupled with the low-power subthreshold design, make this design a suitable candidate for large-scale implementation of synaptic arrays in addressable neuromorphic systems, with reconfigurable synaptic connectivity as well as individually selectable synaptic dynamics.
Fg (pA/V)
100 80 60
ACKNOWLEDGMENT This research was supported by NIH/NIA R01AG029681.
40
R EFERENCES
20 0 0
20 40 60 Pulse Width (os)
80
Fig. 5. Transistor-level circuit simulation illustrating ∆g for 4 groups of different activation widths.
subthreshold and used as a constant current source to linearly charge capacitor C. As shown in Fig. 2(b), the conductance dynamics fij (t − tkj ) can be extended to a rise time τ1 and fall time τ2 through convolution. This could be accomplished by driving the source of the pMOS with a sequence of pulses. The complete dynamical conductance-based synapse circuit implementation is shown in Fig. 3. The circuit is compact, requiring only 3 transistors to implement. V. C HARACTERIZATION To verify the conductance dynamics, we performed transistor-level simulations (using Spectre and parameters of a 0.13µm CMOS process) of the synapse circuit driven by a train of presynaptic impulses, modulated with three different pulse widths, with relative magnitudes 1, 3, and 5, emulating the effect of three time-multiplexed pooled synapses. The circuit output in response to the sequence of input synaptic events is shown in Fig. 4. To verify the linearity of postsynaptic conductance in presynaptic activation, we studied the dependence of the conductance time profile as a function of pulse width and pulse interval. We observed the step in conductance ∆g for a train of pulses at variable pulse intervals, for four different values of pulse width with relative magnitudes 1, 3, 5, and 7 as shown in Fig. 5. The four distinct and compact groups for each of the four different activation widths indicate the linearity of the conductance according to the convolution model (13) and (14). Furthermore, the centers of the clusters for each of the different activation widths are colinear through the origin, confirming linearity in input pulse width. VI. C ONCLUSION We have formulated a dynamical conductance-based synapse cell in a compact circuit design. Circuit simulations verify log-domain implementation as well as an output magnitude scaled to the input conductance strength. The circuit
[1] Mead, C.A., Analog VLSI and Neural Systems, Addison-Wesley, 1989. [2] Serrano-Gotarredona, R.; Oster, M.; Lichtsteiner, P.; Linares-Barranco, A.; Paz-Vicente, R.; Gomez-Rodriguez, F.; Camunas-Mesa, L.; Berner, R.; Rivas-Perez, M.; Delbruck, T.; Shih-Chii Liu; Douglas, R.; Hafliger, P.; Jimenez-Moreno, G.; Ballcels, A.C.; Serrano-Gotarredona, T.; AcostaJimenez, A.J.; Linares-Barranco, B. ”CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware SensoryProcessing LearningActuating System for High-Speed Visual Object Recognition and Tracking”, IEEE Trans. Neural Networks, vol. 20(9), pp. 1417-1438, 2009. [3] Serrano-Gotarredona, R.; Serrano-Gotarredona, T.; Acosta-Jimenez, A.; Serrano-Gotarredona, C.; Perez-Carrasco, J.A.; Linares-Barranco, B.; Linares-Barranco, A.; Jimenez-Moreno, G.; Civit-Ballcels, A. ”On RealTime AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing”, IEEE Trans. Neural Networks, vol. 19(7), pp. 11961219, 2008. [4] Badoni, D.; Giulioni, M.; Dante, V.; Del Giudice, P. ”An aVLSI recurrent network of spiking neurons with reconfigurable and plastic synapses”, Proc. IEEE ISCAS 2006, pg. 1227-1230, 2006. [5] Bartolozzi, C.; Indiveri, G. ”Synaptic Dynamics in Analog VLSI”, Neural Computation, vol. 19(10), pp. 2581-2603, 2007. [6] Lazzaro, J. P., ”Low-power silicon axons, neurons, and synapses”, in Zaghloul, M.E.; Meador, J.L.; and Newcomb, R.W. (Eds.), Silicon implementation of pulse coded neural networks, Norwell, MA: Kluwer, pp. 153164, 1994. [7] Arthur, J.; Boahen, K. ”Recurrently connected silicon neurons with active dendrites for one-shot learning”, IEEE International Joint Conference on Neural Networks, vol. 3, pp. 1699-1704, 2004. [8] Boahen, K.A. ”Retinomorphic vision systems: Reverse engineering the vertebrate retina”, Unpublished doctoral dissertation, California Institute of Technology, 1997. [9] Hynna, K.; Boahen, K. ”Space-rate coding in an adaptive silicon neuron”, Neural Networks, vol. 14, pp. 645656, 2001. [10] Chicca, E. ”A neuromorphic VLSI system for modeling spike-based cooperative competitive neural networks”, Unpublished doctoral dissertation, ETH Zurich, Zurich, Switzerland, 2006. [11] Merolla, P.; Boahen, K. ”A recurrent model of orientation maps with simple and complex cells” in Thr¨ un, S.; Saul, L.K., Sch¨ olkopf, B. (Eds.), Advances in neural information processing systems, MIT Press. Cambridge, MA, vol. 16, pp. 995-1002, 2004. [12] Boahen, K. ”Point-to-point connectivity between neuromorphic chips using address events”, IEEE Trans. on Circuits and Systems II, vol. 47, pp. 416-434, 2000. [13] Goldberg, D.H.; Cauwenberghs, G.; and Andreou, A.G. ”Probabilistic Synaptic Weighting in a Reconfigurable Network of VLSI Integrate-andFire Neurons”, Proc. IEEE Int. Symp. Circuits and Systems, vol. 3, pp. 241-244, 2001. [14] Vogelstein, R.J.; Mallik, U.; Vogelstein, J.T.; and Cauwenberghs, G. ”Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses”, IEEE Trans. Neural Networks, vol. 18(1), pp. 253-265, 2007. [15] Vogelstein, R.J.; Mallik, U.; Culurciello, E.; Cauwenberghs, G.; and Etienne-Cummings, R. ”Saliency-Driven Image Acuity Modulation on a Reconfigurable Silicon Array of Spiking Neurons”, Adv. Neural Information Processing Systems (NIPS’2004), Cambridge: MIT Press, vol. 17, 2005. [16] Destexhe, A.; Mainen, Z.F.; and Sejnowski, T.J. ”Synthesis of Models for Excitable Membranes, Synaptic Transmission and Neuromodulation Using a Common Kinetic Formalism,” J. Comp. Neuroscience, vol. 1, pp. 195-230, 1994.
