On-Chip Supervised Learning Rule for Ultra High ... - Semantic Scholar
On-Chip Supervised Learning Rule for Ultra High Density Neural Crossbar using Memristor for Synapse and Neuron Djaafar Chabi1*, Zhaohao Wang1, Weisheng Zhao1,2, Jacques-Olivier Klein1,2 1. IEF, Univ Paris-Sud, Orsay, 91405, France 2. UMR 8622, CNRS, Orsay, 91405, France *Email: [email protected] Abstract—The memristor-based neural learning network is considered as one of the candidates for future computing system because of its advantages such as low power, high density and defect-tolerance. However, its application is still hindered by the limitations originating from huge neuron structure and complicated learning cell. In this paper, we present a memristorbased neural crossbar circuit to implement on-chip supervised learning rule. In our work, activation function of neuron is implemented with simple CMOS inverter to save area overhead. Importantly, we propose a compact learning cell with a crossbar latch consisting of two anti-parallel oriented binary memristors. This scheme allows high density integration and could improve the reliability of learning circuit. We describe the circuit architecture, memristor model and operation process of supervised learning rule. Afterwards we perform transient simulation with CMOS 40nm design kit to validate the function of proposed learning circuit. Analysis and evaluation demonstrate that our circuit show great potential in on-chip learning. Keywords—memristor; supervised learning.
crossbar;
I.
neural
network;
on-chip
INTRODUCTION
Mainstream computing systems are confronted with power and speed bottlenecks originating from the scaling of nanofabrication [1]. As a possible solution, CMOL (CMOS/molecule) architectures were proposed to implement reconfigurable logic with low power and high density [2]. However, they suffer from low reliability caused by process variation and device defect. In this context, emerging neural network with memristor-based [3–4] synapse and crossbar architecture is considered as a promising candidate. It provides numerous advantages such as low power, high density, CMOS compatibility, and defect-tolerance [5]. Currently massive learning algorithms have been implemented in various neural networks. For instance, spike-timing dependent plasticity (STDP) was demonstrated in synapse arrays based on Ag/Si memristor [6], TiN/HfO2 resistive memory [7], TiOx bilayer resistive switching device [8], and phase change material [9], respectively. However, STDP is an unsupervised learning rule so that these prototypes are not well compatible with traditional CMOS digital chips to replace CMOL. In contrast, neuralinspired logic blocks (NLBs) with supervised learning capability [5, 10–11] provide programmability for high density application and attract intensive research effort.
Overall, the performance of neural network depends on the structure, the activation functions and learning mechanism [12]. Firstly, in terms of structure, currently crossbar architecture is widely used to arrange the neural network as it offers high density thanks to nanofabrication technology [5, 6, 10, 11]. Secondly, the activation function of neuron is implemented depending on the specific application, which required normally dozens of transistors [13–15] and large area. Recently [16] presented a compact neuron with Mott memristors, but its integration with synapses array was not demonstrated. Finally and crucially, amongst state-of-art demonstrations of neural network, learning process is mostly achieved with software, which is not suitable for on-chip supervised learning as the network configurations are fixed after the training. Several hardware implementations of learning cell were proposed [10, 13], but they employ conventional CMOS technology and occupy large area. Typically, a learning cell requires a significant number of transistors compared with a synapse consisting of only 1 or 2 memristors. Such complexity limits the usefulness of this approach to networks with high number of neurons. To compensate the area overhead occupied by the neuron, we should increase the number of synapses per neuron. However, due to the defect and the electrical dispersions caused by the current manufacturing technologies, like nanoimprint, this solution will highly affect the reliability of computation which may potentially cancel the benefit of high density. For this reason, a more compact design of learning cell is desired. In this paper, we aim to demonstrate an on-chip supervised learning rule on memristor-based neural crossbar circuit for high-density application. In our circuit, activation function of neuron is implemented with a simple CMOS inverter, where only two transistors are required and they cannot cost large area. Our research focus is to propose a compact learning cell to save area overhead suffered by previous schemes [10, 13]. This learning cell is implemented with two anti-parallel oriented binary memristors [17], where one of memristors can be conditionally open or closed depending on actual output (post-synaptic) and desired output. In this proposed neural network, miniaturization of neuron and learning cell improves integration density and favors multi-layer hierarchy neural network. Moreover, standby power is nearly eliminated thanks to use of non-volatile memristor. The learning capability of proposed neural network is validated by transient simulation on Cadence platform. Analysis and evaluation of performance such as speed is provided as well.
Fig. 1: (a) Architecture of neural crossbar including a memristor crossbar array and CMOS neuron circuits. (b) Memristor is formed at each junction where two nanowires cross. (c) Learning cell to implement the four programming steps and correct the error between the actual output Xj and the expected output Yj.
II.
LEARNING WITH NEURAL CROSSBAR CIRCUIT
A. Architecture Crossbar architecture is a popular way to connect nanodevices, since it allows maximum wire sharing [2]. As presented in the previous work [18], the neural crossbar is organized so as to implement the Hopfield neural network [19]. As shown in Fig. 1, the differential inputs {Xi+, Xi-} are connected with the output Vj through a couple of memristors as synapses. The state is coded {Xi+, Xi-} = {VH , VL} when Xi is active (Xi = H), or {Xi+, Xi-} = {VL , VH} when Xi is inactive (Xi = L). The logic high-level VH corresponds to a positive input voltage whereas the logic low-level VL corresponds to a negative input voltage. Consequently, each couple of memristors {Mij+, Mij-}, with conductance Gij+ and Gijassociated to a differential input {Xi+, Xi-} constitutes a signedsynaptic weight Wij = K (Gij+ - Gij-) from input i with state Xi to output j with state Xj. Each post-synaptic potential Vj is shared among an entire row of synapses connected to the same output neuron j. During a read operation, it is automatically obtained through the linear combination of the inputs weighted by the equivalent conductance of the memristors: ∑ ∑
(1)
where K is a normalizing factor:
∑
(2)
The activation function of the neuron is performed by comparing post-synaptic potential Vj with a fixed threshold (Ref shown in Fig. 1 (a)) in order to generate the bipolar state (1 or -1) Xj of neuron j. B. Analog memristor as synapse As shown in Fig. 1 (b), a memristor is implemented in each intersection of two nanowires, which is able to adjust its conductance G(t)=i(t)/u(t) depending on its current and/or voltage history [6, 8, 20]. In order to be suitable to implement supervised learning rule, programming pulses are required to change the conductance progressively, while during the
operating phase the conductance must remain insensitive to lower voltage pulses. As a consequence, desirable memristors have a "neutral" voltage range where negligible change of the conductance occurs. This is the case of most actual memristive devices (see for example [6, 21]). The neutral voltage range is defined as [Vth-, Vth+] where f(u)=dG/dt≈0. Otherwise, the conductance will increase or decrease when the applied programming pulse exceeds the positive threshold (Vth+) or the negative threshold (Vth-), respectively. This simple model of memristor is used in our learning circuit design and simulation shown in the following sections. C. Supervised learning rule The supervised learning requires an algorithm for adjusting the network weights to minimize the error between the actual output Xj and the expected output Yj. For the proposed neural crossbar we use the simplified (Boolean) version of the Widrow-Hoff’s Mean Least Square “Delta” rule [22]: ∆
(3)
where α corresponds to the learning step. TABLE 1: DELTA LEARNING RULE FOR BINARY INPUTS. CONFIGURATION XI YJ XJ SIGN(∆WIJ ) PROGRAMMING STEPS C0 -1 -1 -1 0 C1 -1 -1 1 1 S1 C2 -1 1 -1 -1 S2 C3 -1 1 1 0 C4 1 -1 -1 0 C5 1 -1 1 -1 S3 C6 1 1 -1 1 S4 C7 1 1 1 0 S1: inverting the input polarity and applying a negative pulse Vp=Vth- to the output Xj. S2: applying a negative pulse Vp=Vth- to the output Xj. S3: inverting the input polarity and applying a positive pulse Vp=Vth+ to the output Xj. S4: apply a positive pulse Vp=Vth+ to the output Xj.
The truth table of this rule is given by Table 1. The learning process includes two cases: i) adjust the synaptic weight Wij by increasing (∆Wij=1) or decreasing (∆Wij=-1) the conductance of memristors (Gij) until the error (Yj-Xj) becomes null, ii) leave unchanged for all the other configurations (∆Wij=0). This can be achieved by designing
Fig. 2: Architecture of neural crossbar for ultra high density on chip learning using analog memristors (M-, M+) as synapses and binary memristors (A, B) to implement the Delta learning rule. The nodes S+ and S- are used to configure the memristors A and B, respectively. The nodes SwPR, SwVJ, SwRS and SwYJ are used to control the NMOS switches (@ 40 nm) during the learning phase.
the four programming steps S1, S2, S3 and S4 to eliminate the four error configurations C1, C2, C5 and C6, respectively. Fig. 1 (c) shows the possible implementation of a learning cell using a traditional CMOS circuit that requires approximately 40 transistors per neuron including learning cell, which may cancel the high density of memristor-based synapses. III.
COMPACTING NEURAL CROSSBAR
The area associated with each neuron including a learning cell is a key challenge for the success of the neuromorphic architecture. In this work we propose to reduce the area overhead by implementing the learning cell using binary memristor crossbar instead of CMOS circuits (see Fig. 2). With such compact architecture of neural crossbar there is no need to integrate large number of synapses. This can open the way to low fan-in neurons and fine grain function decomposition to implement complex functions. In addition, the reliability could be better controlled for neural crossbar with small size. A. Binary memristor crossbar
Fig. 3: A schematic diagram of a Stateful memristor logic. Xj is the signal line, and S+ and S- are the two control lines. The binary memristors A and B connecting the two control lines to the signal line have antiparallel polarity in terms of the sign of the voltage pulses that open or close one of them.
It was demonstrated in [17, 21] that the circuit based on a binary memristor depicted in Fig. 3 is capable of storing information and performing the basic logic operations. Based on these result, we propose in this work to use this circuit for implementing the learning cell in compact way. The error
between the actual output Vj and the desired Yj will be coded in the state of memristors. We assume that we have access to the technology of binary memristor with only two states (ON, OFF). Once the memristors are configured relative to the error state, they will be used to send the programming pulses to the analog synapses. We note that during this phase, the applied pulse to adjust the conductance of the analog memristors must not affect the conductance of binary memristors. For that the threshold voltage VTH+ and VTH- of binary memristors must be higher than the threshold voltage Vth+ and Vth of the analog memristors. The same technology of memristor can be used in our proposed neural crossbar for both analog and binary memristor. Since most of analog memristors can be used as binary memristors, either by handling the programming pulse or by changing the physical parameters (e.g. device size) [23]. As shown in Fig. 3, the two binary memristors A and B are oriented anti-parallel to each other in their polarity (indicated by the direction of the arrows). The operation of this crossbar latch proceeds as follows: 1) Unconditionally open: both switches A and B can be opened simultaneously by applying high pulse (higher than the threshold) with positive bias to S+ and negative bias to S-. This step will be used as “Reset” operation before learning phase. 2) Conditionally close: by applying a threshold pulse with negative bias to S+ and positive bias to S- and depending on the potential presented at the signal line “Xj”, one of the switches A or B will be close. If “Xj” voltage is between zero and positive high level (H), the switch A will be close and the switch B will remain open. Whereas, for an “Xj” voltage between zero and negative low level (L) the switch B will be close and the switch A will remain open. 3) Conditionally open: by applying a threshold pulse with positive bias to S+ and negative bias to S- and depending on the potential presented at the signal line “Xj”, one of the switches A or B will be open. If “Xj” voltage is between zero and negative low level (L), the switch A will be open and the switch B will remain close. Whereas, for an “Xj” voltage
between zero and positive high level (H) the switch B will be open and the switch A will remain close. B. Compact Learning Cell The crossbar of binary memristors shown in Fig. 3 could be used as conditional switches to implement the activation and learning functions of the neuron in a particularly compact way. As the activation function of the neuron implemented here by one inverter (see Fig. 2) will not occupy large area, the binary memristors are used only to implement the learning function, which basically requires a large number of transistors. As shown in Fig. 2, the proposed compact learning cell consists of only four transistors and two binary memristors. Fig. 4 shows the three operations to implement the learning phase: the “Reset” of binary memristors (A and B), the configuration of binary memristors and finally the updating of synaptic weight. The “Reset” operation is used to open unconditionally both memristors A and B. During this operation the signal line Xj is connected to the ground through the transistor controlled by SwRS. The configuration of the binary memristors is used to close one of the memristors (A or B) depending on the error between the actual output “Vjb” and the expected output “Yj” (see Table 2). Firstly, “Vjb” is connected to the signal line “Xj” by activating the transistor control signal “SwVj”, and then supplying a “conditionally close” pulse to the control inputs S+ and S-. Depending on the sign of “Vjb” one of the binary memristors (A or B) will be closed and the other remains open. Secondly, “Xj” is disconnected from “Vjb” and connected to “Yj” by activating the transistor control signal “SwYj”, and then supplying a “conditionally open” to the control inputs S+ and S-. Depending on the sign of the expected output “Yj”, A or B will be opened.
the configuration of the binary memristors is finished, the control signal “SwPR” is activated to connect “Xj” to the postsynaptic “Vj” to perform the learning phase by updating the conductance (synaptic weight). The learning sequences (S4, S2) are implemented by applying the programming pulses VP=Vth+ and VP=Vth- to S-, and then the learning sequences (S1, S3) are implemented by inverting the inputs (IP-) and applying VP=Vth+ and VP=Vth- to S+ (see Fig. 4). TABLE 2: CONFIGURATION OF BINARY MEMRISTORS A AND B. Cond. close pulse Cond. open pulse Vj Xj=Vjb Xj=Yj S+ (A) S- (B) S+ (A) S- (B) L open close open H L close H L H open close open open L H L close open open open H close open open L H close
IV.
SIMULATION AND ANALYSIS
The proposed compact neural crossbar is designed using behavioral models of an analog and a binary memristors and STMicroelectronics CMOS 40 nm low power technology design kit [24]. Electrical simulations of learning logical functions have been performed to demonstrate a functional behavior of the proposed compact neural crossbar. A. Analog memristor model A memristor features non-volatility, very small size and two-terminal synapse-like behaviors [4]. It is considered as one of the most promising nanodevices to be integrated in a crossbar array to build nanoscale neuromorphic computing systems. In our simulation, we use a simplified model of device behavior where the conductance change speed dg(t)/dt is a function of the voltage. Table 3 shows the parameters of the analog memristor model used in this paper. TABLE 3: PARAMETERS OF THE ANALOG MEMRISTOR MODEL Parameter Symbol Default value Read voltage Threshold voltage Programming voltage Programming duration Conductance Increment step
VH/VL Vth+/VthVP+/VPTP GON/GOFF Ginc
1V/-1V 2V/-2V ~2V/~-2V 100 ns 10 uS/100 nS 300 nS
Fig. 4: One learning epoch of compact neural crossbar. Top: voltage S- and S+ as a function of time during learning. Bottom: control signals of the different switches (i.e. @40 nm NMOS transistors).
As shown in Table 2, thanks to these two operations, the node S+ is connected through the memristor A to all the lines Xj corresponding to error-configuration where VjYj=LH, and the node S- is connected through the memristor B to all the lines Xj corresponding to error-configuration VjYj =HL. When
Fig. 5: Electrical simulation results of the analog memristor model implemented using verilog-A. The polarity of the input X1 can be inverted by activating the control signal Ip of the switch at the input (see Fig. 1).
ON and the OFF states (GON/GOFF=10 uS/100 uS). Besides, the threshold voltage for the binary memristor is higher (VTH+/VTH- = 3.6 V/-3.6 V) than the analog memristor (Vth+/Vth-=2 V/ -2V). The parameters of the binary memristor model used in this paper are shown in Table 4. Table 4: Parameters of the binary memristor model
Fig. 6: Electrical simulation results of the binary memristor crossbar model implemented using verilog-A. The basic operations (1), (2) and (3) have been performed for different configurations of the signal line Xj, L and H are the low (-0.2 V) and the high (0.2 V) level, respectively.
As shown in Fig. 5, the model is based on simple linear increment/decrement of the conductance, when the applied voltage between the input (X1) and the output (Vj) exceeds the positive or negative threshold voltage, respectively, and the duration exceeds Tp=100 ns. No conductance change occurs when the across voltage of memristor is lower than the threshold or programming duration is less than Tp. This will allow the neural crossbar to keep the adjusted synaptic weights during the computing operation. B. Binary memristor model The differences between the analog memristor model presented previously and the binary memristor model are the number of level between the ON and the OFF states and the threshold voltage. The analog memristor can adjust its conductance gradually with an increment step (e.g. Ginc=300 nS) but the binary memristor switches abruptly between the
(a)
Parameter
Symbol
Default value
Read voltage Threshold voltage Reset voltage Programming voltage Programming duration Conductance Increment step
VH/VL VTH+/VTHVRS VP+/VPTP GON/GOFF Ginc
2V/-2V 3.6V/-3.6V ~4 V 3.6V/-3.6V 100 ns 10 uS/100 nS 10 uS
Fig. 6 shows the electrical simulations performed to demonstrate the basic operations of the binary memristor crossbar using two memristors oriented anti-parallel like in Fig. 3. The operations, Reset (1), conditionally close (2), conditionally open (3), have been performed here for different configurations of the signal line Xj. As explained previously, during the step (2) Xj is connected to the actual output (Vjb) and during the step (3) Xj is connected to the expected output Yj. The simulation results in Fig. 6 show that depending in Xj voltage one of the memristor A or B will be closed. For the configuration Vjb Yj=LL (VjYj=HL) the memristor B is closed while the memristor A remains open. Unlike the configuration Vjb Yj=HH (VjYj=LH), the memristor A is closed while the memristor B remains open. Otherwise, for the configurations Vjb Yj=LH (VjYj=HH) or VjbYj=HL (VjYj=LL), the step (2) will open the memristor A (B), but as there is no error (Vj=Yj) the step (3) will close it. This result demonstrates the possibility to implement the learning rule using the memristor crossbar. C. Learning of logic function with compact neural crossbar To demonstrate the feasibility of our proposed compact neural crossbar (shown in Fig. 2), we took the learning of
(b)
Fig. 7: Electrical simulation results of compact neural crossbar demonstrating the learning of 2-input AND-function. (a) shows the inputs voltage (X1, X2), output voltage (Vj) and the conductance evolution of the analog memristors (G1+, G1-, G2+, G2-, Gb+, Gb-). (b) shows the control signals (S+, S-), the conductance state (GA, GB) and the common line signal (Xj) of binary memristor (A, B), where they turned ON or OFF depending in the inverse of the postsynaptic output Vjb and the expected output Yj.
logical functions as an example. Fig. 7 shows a transient simulation of learning a 2-input AND-function using compact neural crossbar. The network comprises one output (Vj) and two logical inputs (X1, X2) and one bias input (Bias). Each input corresponds to positive and negative input which gives a total of six physical inputs. The simulation includes three phases. First, the reading phase is performed by applying the truth table to the inputs to show the initial post-synaptic output (Vj) state. Second, the learning phase is performed to adjust the conductance of the analog memristors to eliminate the error between Vj and Yj by repeating the learning steps shown in Fig. 4 for each case of the truth table. Third, the reading operation is performed again to show the output state and check if the neural crossbar has learned the expected function. The conductance of the analog memristors (synapses) has been initialized so that the output follows the input X1. As shown in Fig. 7 (a), at t=0 s the equivalent conductance (synaptic weight) of the couple of memristors connected to the input X1 is G1 =G1+-G1- = 900 nS higher than those of input X2 (G2= -100 nS) and Bias (Gb=100 nS). When the learning phase starts at t=4 us, the conductance of the analog memristors (Gi+, Gi-) increases or decreases depending on the error between Vj and Yj until t=10 us. At this time, the conductance stops evolving which means that the neural crossbar has finished learning the expected function. The reading phase started at t=20 us confirms that (see Fig. 7 (b)). In addition to the high density promised by our proposed compact neural crossbar, simulation results show very promising results in term of learning speed. The learning process can converge quickly after few patterns. For this simulation, for example, only six patterns (1.5×epoch) were sufficient to train the neural crossbar, which corresponds in term of time to 6 us. This can be further optimized by using high speed memristor. By exploiting these characteristics, the learning function can be integrated in the same chip with synapses and performed without affecting drastically the computing performances. V.
ACKNOWLEDGMENT The authors would like to thank the financial support of MOOREA project. REFERENCES [1] [2]
[3] [4] [5] [6]
[7]
[8]
[9]
[10]
[11] [12]
[13]
[14]
[15]
CONCLUSION AND PERSPECTIVES
We have simulated an on-chip supervised learning rule with neural crossbar circuit. In this architecture, synapse is built with a couple of analog memristors, similar to mainstream prototypes. However, the neuron is implemented with a simple CMOS inverter, and learning cell is designed with binary memristors instead of conventional CMOS circuits. Compared with previous neural networks, our scheme is capable of achieving on-chip learning and improves integration density thanks to proposed compact neuron and learning cell. With a behavioral model of memristor and CMOS 40 nm design kit, we performed transient simulation to demonstrate the learning process of our neural crossbar circuit. Simulation results validate the function of proposed learning circuit and show high learning speed. This work paves way for hardware implementation of large-scale multi-layer hierarchy neural network with on-chip learning capability. In the future work, we plan to demonstrate and analyze a large array of this learning circuit for image recognition application with physical memristor such as TiOx memristor and ferroelectric memristor.
[16]
[17]
[18]
[19]
[20] [21] [22] [23] [24]
International Technology Roadmap for Semiconductors (ITRS) 2012. D. B. Strukov and K. K. Likharev. “CMOL FPGA: a reconfigurable architecture for hybrid digital circuits with two-terminal nanodevices,” Nanotechnology, vol. 16, no. 6, pp. 888–900, 2005. L. O. Chua, “Memristor-the missing circuit element,” IEEE Trans. Circuit Theory, vol. 18, pp. 507–519, 1971. D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missing memristor found,” Nature, vol. 453, pp. 80–83, 2008. D. Chabi et al., “Robust learning approach for neuro-inspired nanoscale crossbar architecture,” ACM JETC, vol.10, no. 1, pp. 5, 2014. S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviy, P. Mazumder and W. Lu, “Nanoscale Memristor Device as Synapse in Neuromorphic Systems,” Nano Lett., vol. 10, pp. 1297–1301, 2010. S. Ambrogio, S. Balatti, F. Nardi, S. Facchinetti and D. Ielmini, “Spiketiming dependent plasticity in a transistor-selected resistive switching memory,” Nanotechnology, vol. 24, pp. 384012, 2013. K. Seo et al. “Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device,” Nanotechnology, vol. 22, no. 25, pp. 254023, 2011. D. Kuzum et al., “Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing,” Nano Lett., vol. 12, pp. 2179–2186, 2011. S. Liao et al. “Design and modeling of a neuro-inspired learning circuit using nanotube-based memory devices,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 58, pp. 2172–2181, 2011. K. Gacem et al. “Neuromorphic function learning with carbon nanotube based synapses,” Nanotechnology, vol. 24, no. 38, pp. 384013, 2013. K. Packia Lakshmi and M. Subadra. “A survey on FPGA based MLP Realization for On-chip Learning,” Int. J. Sci. & Eng. Res., vol. 4, no. 1, pp. 1–9, Jan. 2013. G. Cauwenberghs. “An analog VLSI recurrent neural network learning a continuous-time trajectory,” IEEE Trans. Neural Netw., vol. 7, no. 2, pp. 346–361, 1996. G. Indiveri, E. Chicca, and R. Douglas. “A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211–221, 2006. D. H. Goldberg, G. Cauwenberghs, and A. G. Andreou. “Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-andfire neurons,” Neural Netw., vol. 14, no. 6, pp. 781–793, 2001. M. D. Pickett, G. Medeiros-Ribeiro, and R. S. Williams. “A scalable neuristor built with Mott memristors,” Nature Mater., vol. 12, no. 2, pp. 114–117, 2013. P. J. Kuekes, D. R. Stewart, and R. S. Williams. “The crossbar latch: Logic value storage, restoration, and inversion in crossbar circuits,” J. Appl. Phys., vol. 97, no. 3, pp. 034301, 2004. D. Chabi, W. S. Zhao, D. Querlioz, and J.-O. Klein. “Robust neural logic block (NLB) based on memristor crossbar array” in IEEE/ACM NANOARCH, San Diego, 2011, pp. 137–143. D. Tank and J. J. Hopfield. “Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans Circuits and Systems, vol. 33, no. 5, pp.533–541, 1986. L. O. Chua and S. M. Kang. “Memristive devices and systems,” Proc. IEEE, vol. 64, no.2, pp. 209–263, 1976. J. Borghetti et al. “Memristive’switches enable ‘stateful’logic operations via material implication,” Nature, vol. 464, pp. 873–876, 2010. B. Widrow and M. E. Hoff. “Adaptive switching circuits,” IRE WESCON Convention Record, pp. 96–104, 1960. A. Chanthbouala et al. “A ferroelectric memristor,” Nature Mater. vol. 11, pp. 860-864, 2012. CMOS40 Design Rule Manual (STMicroelectronics, 2012).
crossbar;
I.
neural
network;
on-chip
INTRODUCTION
Mainstream computing systems are confronted with power and speed bottlenecks originating from the scaling of nanofabrication [1]. As a possible solution, CMOL (CMOS/molecule) architectures were proposed to implement reconfigurable logic with low power and high density [2]. However, they suffer from low reliability caused by process variation and device defect. In this context, emerging neural network with memristor-based [3–4] synapse and crossbar architecture is considered as a promising candidate. It provides numerous advantages such as low power, high density, CMOS compatibility, and defect-tolerance [5]. Currently massive learning algorithms have been implemented in various neural networks. For instance, spike-timing dependent plasticity (STDP) was demonstrated in synapse arrays based on Ag/Si memristor [6], TiN/HfO2 resistive memory [7], TiOx bilayer resistive switching device [8], and phase change material [9], respectively. However, STDP is an unsupervised learning rule so that these prototypes are not well compatible with traditional CMOS digital chips to replace CMOL. In contrast, neuralinspired logic blocks (NLBs) with supervised learning capability [5, 10–11] provide programmability for high density application and attract intensive research effort.
Overall, the performance of neural network depends on the structure, the activation functions and learning mechanism [12]. Firstly, in terms of structure, currently crossbar architecture is widely used to arrange the neural network as it offers high density thanks to nanofabrication technology [5, 6, 10, 11]. Secondly, the activation function of neuron is implemented depending on the specific application, which required normally dozens of transistors [13–15] and large area. Recently [16] presented a compact neuron with Mott memristors, but its integration with synapses array was not demonstrated. Finally and crucially, amongst state-of-art demonstrations of neural network, learning process is mostly achieved with software, which is not suitable for on-chip supervised learning as the network configurations are fixed after the training. Several hardware implementations of learning cell were proposed [10, 13], but they employ conventional CMOS technology and occupy large area. Typically, a learning cell requires a significant number of transistors compared with a synapse consisting of only 1 or 2 memristors. Such complexity limits the usefulness of this approach to networks with high number of neurons. To compensate the area overhead occupied by the neuron, we should increase the number of synapses per neuron. However, due to the defect and the electrical dispersions caused by the current manufacturing technologies, like nanoimprint, this solution will highly affect the reliability of computation which may potentially cancel the benefit of high density. For this reason, a more compact design of learning cell is desired. In this paper, we aim to demonstrate an on-chip supervised learning rule on memristor-based neural crossbar circuit for high-density application. In our circuit, activation function of neuron is implemented with a simple CMOS inverter, where only two transistors are required and they cannot cost large area. Our research focus is to propose a compact learning cell to save area overhead suffered by previous schemes [10, 13]. This learning cell is implemented with two anti-parallel oriented binary memristors [17], where one of memristors can be conditionally open or closed depending on actual output (post-synaptic) and desired output. In this proposed neural network, miniaturization of neuron and learning cell improves integration density and favors multi-layer hierarchy neural network. Moreover, standby power is nearly eliminated thanks to use of non-volatile memristor. The learning capability of proposed neural network is validated by transient simulation on Cadence platform. Analysis and evaluation of performance such as speed is provided as well.
Fig. 1: (a) Architecture of neural crossbar including a memristor crossbar array and CMOS neuron circuits. (b) Memristor is formed at each junction where two nanowires cross. (c) Learning cell to implement the four programming steps and correct the error between the actual output Xj and the expected output Yj.
II.
LEARNING WITH NEURAL CROSSBAR CIRCUIT
A. Architecture Crossbar architecture is a popular way to connect nanodevices, since it allows maximum wire sharing [2]. As presented in the previous work [18], the neural crossbar is organized so as to implement the Hopfield neural network [19]. As shown in Fig. 1, the differential inputs {Xi+, Xi-} are connected with the output Vj through a couple of memristors as synapses. The state is coded {Xi+, Xi-} = {VH , VL} when Xi is active (Xi = H), or {Xi+, Xi-} = {VL , VH} when Xi is inactive (Xi = L). The logic high-level VH corresponds to a positive input voltage whereas the logic low-level VL corresponds to a negative input voltage. Consequently, each couple of memristors {Mij+, Mij-}, with conductance Gij+ and Gijassociated to a differential input {Xi+, Xi-} constitutes a signedsynaptic weight Wij = K (Gij+ - Gij-) from input i with state Xi to output j with state Xj. Each post-synaptic potential Vj is shared among an entire row of synapses connected to the same output neuron j. During a read operation, it is automatically obtained through the linear combination of the inputs weighted by the equivalent conductance of the memristors: ∑ ∑
(1)
where K is a normalizing factor:
∑
(2)
The activation function of the neuron is performed by comparing post-synaptic potential Vj with a fixed threshold (Ref shown in Fig. 1 (a)) in order to generate the bipolar state (1 or -1) Xj of neuron j. B. Analog memristor as synapse As shown in Fig. 1 (b), a memristor is implemented in each intersection of two nanowires, which is able to adjust its conductance G(t)=i(t)/u(t) depending on its current and/or voltage history [6, 8, 20]. In order to be suitable to implement supervised learning rule, programming pulses are required to change the conductance progressively, while during the
operating phase the conductance must remain insensitive to lower voltage pulses. As a consequence, desirable memristors have a "neutral" voltage range where negligible change of the conductance occurs. This is the case of most actual memristive devices (see for example [6, 21]). The neutral voltage range is defined as [Vth-, Vth+] where f(u)=dG/dt≈0. Otherwise, the conductance will increase or decrease when the applied programming pulse exceeds the positive threshold (Vth+) or the negative threshold (Vth-), respectively. This simple model of memristor is used in our learning circuit design and simulation shown in the following sections. C. Supervised learning rule The supervised learning requires an algorithm for adjusting the network weights to minimize the error between the actual output Xj and the expected output Yj. For the proposed neural crossbar we use the simplified (Boolean) version of the Widrow-Hoff’s Mean Least Square “Delta” rule [22]: ∆
(3)
where α corresponds to the learning step. TABLE 1: DELTA LEARNING RULE FOR BINARY INPUTS. CONFIGURATION XI YJ XJ SIGN(∆WIJ ) PROGRAMMING STEPS C0 -1 -1 -1 0 C1 -1 -1 1 1 S1 C2 -1 1 -1 -1 S2 C3 -1 1 1 0 C4 1 -1 -1 0 C5 1 -1 1 -1 S3 C6 1 1 -1 1 S4 C7 1 1 1 0 S1: inverting the input polarity and applying a negative pulse Vp=Vth- to the output Xj. S2: applying a negative pulse Vp=Vth- to the output Xj. S3: inverting the input polarity and applying a positive pulse Vp=Vth+ to the output Xj. S4: apply a positive pulse Vp=Vth+ to the output Xj.
The truth table of this rule is given by Table 1. The learning process includes two cases: i) adjust the synaptic weight Wij by increasing (∆Wij=1) or decreasing (∆Wij=-1) the conductance of memristors (Gij) until the error (Yj-Xj) becomes null, ii) leave unchanged for all the other configurations (∆Wij=0). This can be achieved by designing
Fig. 2: Architecture of neural crossbar for ultra high density on chip learning using analog memristors (M-, M+) as synapses and binary memristors (A, B) to implement the Delta learning rule. The nodes S+ and S- are used to configure the memristors A and B, respectively. The nodes SwPR, SwVJ, SwRS and SwYJ are used to control the NMOS switches (@ 40 nm) during the learning phase.
the four programming steps S1, S2, S3 and S4 to eliminate the four error configurations C1, C2, C5 and C6, respectively. Fig. 1 (c) shows the possible implementation of a learning cell using a traditional CMOS circuit that requires approximately 40 transistors per neuron including learning cell, which may cancel the high density of memristor-based synapses. III.
COMPACTING NEURAL CROSSBAR
The area associated with each neuron including a learning cell is a key challenge for the success of the neuromorphic architecture. In this work we propose to reduce the area overhead by implementing the learning cell using binary memristor crossbar instead of CMOS circuits (see Fig. 2). With such compact architecture of neural crossbar there is no need to integrate large number of synapses. This can open the way to low fan-in neurons and fine grain function decomposition to implement complex functions. In addition, the reliability could be better controlled for neural crossbar with small size. A. Binary memristor crossbar
Fig. 3: A schematic diagram of a Stateful memristor logic. Xj is the signal line, and S+ and S- are the two control lines. The binary memristors A and B connecting the two control lines to the signal line have antiparallel polarity in terms of the sign of the voltage pulses that open or close one of them.
It was demonstrated in [17, 21] that the circuit based on a binary memristor depicted in Fig. 3 is capable of storing information and performing the basic logic operations. Based on these result, we propose in this work to use this circuit for implementing the learning cell in compact way. The error
between the actual output Vj and the desired Yj will be coded in the state of memristors. We assume that we have access to the technology of binary memristor with only two states (ON, OFF). Once the memristors are configured relative to the error state, they will be used to send the programming pulses to the analog synapses. We note that during this phase, the applied pulse to adjust the conductance of the analog memristors must not affect the conductance of binary memristors. For that the threshold voltage VTH+ and VTH- of binary memristors must be higher than the threshold voltage Vth+ and Vth of the analog memristors. The same technology of memristor can be used in our proposed neural crossbar for both analog and binary memristor. Since most of analog memristors can be used as binary memristors, either by handling the programming pulse or by changing the physical parameters (e.g. device size) [23]. As shown in Fig. 3, the two binary memristors A and B are oriented anti-parallel to each other in their polarity (indicated by the direction of the arrows). The operation of this crossbar latch proceeds as follows: 1) Unconditionally open: both switches A and B can be opened simultaneously by applying high pulse (higher than the threshold) with positive bias to S+ and negative bias to S-. This step will be used as “Reset” operation before learning phase. 2) Conditionally close: by applying a threshold pulse with negative bias to S+ and positive bias to S- and depending on the potential presented at the signal line “Xj”, one of the switches A or B will be close. If “Xj” voltage is between zero and positive high level (H), the switch A will be close and the switch B will remain open. Whereas, for an “Xj” voltage between zero and negative low level (L) the switch B will be close and the switch A will remain open. 3) Conditionally open: by applying a threshold pulse with positive bias to S+ and negative bias to S- and depending on the potential presented at the signal line “Xj”, one of the switches A or B will be open. If “Xj” voltage is between zero and negative low level (L), the switch A will be open and the switch B will remain close. Whereas, for an “Xj” voltage
between zero and positive high level (H) the switch B will be open and the switch A will remain close. B. Compact Learning Cell The crossbar of binary memristors shown in Fig. 3 could be used as conditional switches to implement the activation and learning functions of the neuron in a particularly compact way. As the activation function of the neuron implemented here by one inverter (see Fig. 2) will not occupy large area, the binary memristors are used only to implement the learning function, which basically requires a large number of transistors. As shown in Fig. 2, the proposed compact learning cell consists of only four transistors and two binary memristors. Fig. 4 shows the three operations to implement the learning phase: the “Reset” of binary memristors (A and B), the configuration of binary memristors and finally the updating of synaptic weight. The “Reset” operation is used to open unconditionally both memristors A and B. During this operation the signal line Xj is connected to the ground through the transistor controlled by SwRS. The configuration of the binary memristors is used to close one of the memristors (A or B) depending on the error between the actual output “Vjb” and the expected output “Yj” (see Table 2). Firstly, “Vjb” is connected to the signal line “Xj” by activating the transistor control signal “SwVj”, and then supplying a “conditionally close” pulse to the control inputs S+ and S-. Depending on the sign of “Vjb” one of the binary memristors (A or B) will be closed and the other remains open. Secondly, “Xj” is disconnected from “Vjb” and connected to “Yj” by activating the transistor control signal “SwYj”, and then supplying a “conditionally open” to the control inputs S+ and S-. Depending on the sign of the expected output “Yj”, A or B will be opened.
the configuration of the binary memristors is finished, the control signal “SwPR” is activated to connect “Xj” to the postsynaptic “Vj” to perform the learning phase by updating the conductance (synaptic weight). The learning sequences (S4, S2) are implemented by applying the programming pulses VP=Vth+ and VP=Vth- to S-, and then the learning sequences (S1, S3) are implemented by inverting the inputs (IP-) and applying VP=Vth+ and VP=Vth- to S+ (see Fig. 4). TABLE 2: CONFIGURATION OF BINARY MEMRISTORS A AND B. Cond. close pulse Cond. open pulse Vj Xj=Vjb Xj=Yj S+ (A) S- (B) S+ (A) S- (B) L open close open H L close H L H open close open open L H L close open open open H close open open L H close
IV.
SIMULATION AND ANALYSIS
The proposed compact neural crossbar is designed using behavioral models of an analog and a binary memristors and STMicroelectronics CMOS 40 nm low power technology design kit [24]. Electrical simulations of learning logical functions have been performed to demonstrate a functional behavior of the proposed compact neural crossbar. A. Analog memristor model A memristor features non-volatility, very small size and two-terminal synapse-like behaviors [4]. It is considered as one of the most promising nanodevices to be integrated in a crossbar array to build nanoscale neuromorphic computing systems. In our simulation, we use a simplified model of device behavior where the conductance change speed dg(t)/dt is a function of the voltage. Table 3 shows the parameters of the analog memristor model used in this paper. TABLE 3: PARAMETERS OF THE ANALOG MEMRISTOR MODEL Parameter Symbol Default value Read voltage Threshold voltage Programming voltage Programming duration Conductance Increment step
VH/VL Vth+/VthVP+/VPTP GON/GOFF Ginc
1V/-1V 2V/-2V ~2V/~-2V 100 ns 10 uS/100 nS 300 nS
Fig. 4: One learning epoch of compact neural crossbar. Top: voltage S- and S+ as a function of time during learning. Bottom: control signals of the different switches (i.e. @40 nm NMOS transistors).
As shown in Table 2, thanks to these two operations, the node S+ is connected through the memristor A to all the lines Xj corresponding to error-configuration where VjYj=LH, and the node S- is connected through the memristor B to all the lines Xj corresponding to error-configuration VjYj =HL. When
Fig. 5: Electrical simulation results of the analog memristor model implemented using verilog-A. The polarity of the input X1 can be inverted by activating the control signal Ip of the switch at the input (see Fig. 1).
ON and the OFF states (GON/GOFF=10 uS/100 uS). Besides, the threshold voltage for the binary memristor is higher (VTH+/VTH- = 3.6 V/-3.6 V) than the analog memristor (Vth+/Vth-=2 V/ -2V). The parameters of the binary memristor model used in this paper are shown in Table 4. Table 4: Parameters of the binary memristor model
Fig. 6: Electrical simulation results of the binary memristor crossbar model implemented using verilog-A. The basic operations (1), (2) and (3) have been performed for different configurations of the signal line Xj, L and H are the low (-0.2 V) and the high (0.2 V) level, respectively.
As shown in Fig. 5, the model is based on simple linear increment/decrement of the conductance, when the applied voltage between the input (X1) and the output (Vj) exceeds the positive or negative threshold voltage, respectively, and the duration exceeds Tp=100 ns. No conductance change occurs when the across voltage of memristor is lower than the threshold or programming duration is less than Tp. This will allow the neural crossbar to keep the adjusted synaptic weights during the computing operation. B. Binary memristor model The differences between the analog memristor model presented previously and the binary memristor model are the number of level between the ON and the OFF states and the threshold voltage. The analog memristor can adjust its conductance gradually with an increment step (e.g. Ginc=300 nS) but the binary memristor switches abruptly between the
(a)
Parameter
Symbol
Default value
Read voltage Threshold voltage Reset voltage Programming voltage Programming duration Conductance Increment step
VH/VL VTH+/VTHVRS VP+/VPTP GON/GOFF Ginc
2V/-2V 3.6V/-3.6V ~4 V 3.6V/-3.6V 100 ns 10 uS/100 nS 10 uS
Fig. 6 shows the electrical simulations performed to demonstrate the basic operations of the binary memristor crossbar using two memristors oriented anti-parallel like in Fig. 3. The operations, Reset (1), conditionally close (2), conditionally open (3), have been performed here for different configurations of the signal line Xj. As explained previously, during the step (2) Xj is connected to the actual output (Vjb) and during the step (3) Xj is connected to the expected output Yj. The simulation results in Fig. 6 show that depending in Xj voltage one of the memristor A or B will be closed. For the configuration Vjb Yj=LL (VjYj=HL) the memristor B is closed while the memristor A remains open. Unlike the configuration Vjb Yj=HH (VjYj=LH), the memristor A is closed while the memristor B remains open. Otherwise, for the configurations Vjb Yj=LH (VjYj=HH) or VjbYj=HL (VjYj=LL), the step (2) will open the memristor A (B), but as there is no error (Vj=Yj) the step (3) will close it. This result demonstrates the possibility to implement the learning rule using the memristor crossbar. C. Learning of logic function with compact neural crossbar To demonstrate the feasibility of our proposed compact neural crossbar (shown in Fig. 2), we took the learning of
(b)
Fig. 7: Electrical simulation results of compact neural crossbar demonstrating the learning of 2-input AND-function. (a) shows the inputs voltage (X1, X2), output voltage (Vj) and the conductance evolution of the analog memristors (G1+, G1-, G2+, G2-, Gb+, Gb-). (b) shows the control signals (S+, S-), the conductance state (GA, GB) and the common line signal (Xj) of binary memristor (A, B), where they turned ON or OFF depending in the inverse of the postsynaptic output Vjb and the expected output Yj.
logical functions as an example. Fig. 7 shows a transient simulation of learning a 2-input AND-function using compact neural crossbar. The network comprises one output (Vj) and two logical inputs (X1, X2) and one bias input (Bias). Each input corresponds to positive and negative input which gives a total of six physical inputs. The simulation includes three phases. First, the reading phase is performed by applying the truth table to the inputs to show the initial post-synaptic output (Vj) state. Second, the learning phase is performed to adjust the conductance of the analog memristors to eliminate the error between Vj and Yj by repeating the learning steps shown in Fig. 4 for each case of the truth table. Third, the reading operation is performed again to show the output state and check if the neural crossbar has learned the expected function. The conductance of the analog memristors (synapses) has been initialized so that the output follows the input X1. As shown in Fig. 7 (a), at t=0 s the equivalent conductance (synaptic weight) of the couple of memristors connected to the input X1 is G1 =G1+-G1- = 900 nS higher than those of input X2 (G2= -100 nS) and Bias (Gb=100 nS). When the learning phase starts at t=4 us, the conductance of the analog memristors (Gi+, Gi-) increases or decreases depending on the error between Vj and Yj until t=10 us. At this time, the conductance stops evolving which means that the neural crossbar has finished learning the expected function. The reading phase started at t=20 us confirms that (see Fig. 7 (b)). In addition to the high density promised by our proposed compact neural crossbar, simulation results show very promising results in term of learning speed. The learning process can converge quickly after few patterns. For this simulation, for example, only six patterns (1.5×epoch) were sufficient to train the neural crossbar, which corresponds in term of time to 6 us. This can be further optimized by using high speed memristor. By exploiting these characteristics, the learning function can be integrated in the same chip with synapses and performed without affecting drastically the computing performances. V.
ACKNOWLEDGMENT The authors would like to thank the financial support of MOOREA project. REFERENCES [1] [2]
[3] [4] [5] [6]
[7]
[8]
[9]
[10]
[11] [12]
[13]
[14]
[15]
CONCLUSION AND PERSPECTIVES
We have simulated an on-chip supervised learning rule with neural crossbar circuit. In this architecture, synapse is built with a couple of analog memristors, similar to mainstream prototypes. However, the neuron is implemented with a simple CMOS inverter, and learning cell is designed with binary memristors instead of conventional CMOS circuits. Compared with previous neural networks, our scheme is capable of achieving on-chip learning and improves integration density thanks to proposed compact neuron and learning cell. With a behavioral model of memristor and CMOS 40 nm design kit, we performed transient simulation to demonstrate the learning process of our neural crossbar circuit. Simulation results validate the function of proposed learning circuit and show high learning speed. This work paves way for hardware implementation of large-scale multi-layer hierarchy neural network with on-chip learning capability. In the future work, we plan to demonstrate and analyze a large array of this learning circuit for image recognition application with physical memristor such as TiOx memristor and ferroelectric memristor.
[16]
[17]
[18]
[19]
[20] [21] [22] [23] [24]
International Technology Roadmap for Semiconductors (ITRS) 2012. D. B. Strukov and K. K. Likharev. “CMOL FPGA: a reconfigurable architecture for hybrid digital circuits with two-terminal nanodevices,” Nanotechnology, vol. 16, no. 6, pp. 888–900, 2005. L. O. Chua, “Memristor-the missing circuit element,” IEEE Trans. Circuit Theory, vol. 18, pp. 507–519, 1971. D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missing memristor found,” Nature, vol. 453, pp. 80–83, 2008. D. Chabi et al., “Robust learning approach for neuro-inspired nanoscale crossbar architecture,” ACM JETC, vol.10, no. 1, pp. 5, 2014. S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviy, P. Mazumder and W. Lu, “Nanoscale Memristor Device as Synapse in Neuromorphic Systems,” Nano Lett., vol. 10, pp. 1297–1301, 2010. S. Ambrogio, S. Balatti, F. Nardi, S. Facchinetti and D. Ielmini, “Spiketiming dependent plasticity in a transistor-selected resistive switching memory,” Nanotechnology, vol. 24, pp. 384012, 2013. K. Seo et al. “Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device,” Nanotechnology, vol. 22, no. 25, pp. 254023, 2011. D. Kuzum et al., “Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing,” Nano Lett., vol. 12, pp. 2179–2186, 2011. S. Liao et al. “Design and modeling of a neuro-inspired learning circuit using nanotube-based memory devices,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 58, pp. 2172–2181, 2011. K. Gacem et al. “Neuromorphic function learning with carbon nanotube based synapses,” Nanotechnology, vol. 24, no. 38, pp. 384013, 2013. K. Packia Lakshmi and M. Subadra. “A survey on FPGA based MLP Realization for On-chip Learning,” Int. J. Sci. & Eng. Res., vol. 4, no. 1, pp. 1–9, Jan. 2013. G. Cauwenberghs. “An analog VLSI recurrent neural network learning a continuous-time trajectory,” IEEE Trans. Neural Netw., vol. 7, no. 2, pp. 346–361, 1996. G. Indiveri, E. Chicca, and R. Douglas. “A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity,” IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211–221, 2006. D. H. Goldberg, G. Cauwenberghs, and A. G. Andreou. “Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-andfire neurons,” Neural Netw., vol. 14, no. 6, pp. 781–793, 2001. M. D. Pickett, G. Medeiros-Ribeiro, and R. S. Williams. “A scalable neuristor built with Mott memristors,” Nature Mater., vol. 12, no. 2, pp. 114–117, 2013. P. J. Kuekes, D. R. Stewart, and R. S. Williams. “The crossbar latch: Logic value storage, restoration, and inversion in crossbar circuits,” J. Appl. Phys., vol. 97, no. 3, pp. 034301, 2004. D. Chabi, W. S. Zhao, D. Querlioz, and J.-O. Klein. “Robust neural logic block (NLB) based on memristor crossbar array” in IEEE/ACM NANOARCH, San Diego, 2011, pp. 137–143. D. Tank and J. J. Hopfield. “Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans Circuits and Systems, vol. 33, no. 5, pp.533–541, 1986. L. O. Chua and S. M. Kang. “Memristive devices and systems,” Proc. IEEE, vol. 64, no.2, pp. 209–263, 1976. J. Borghetti et al. “Memristive’switches enable ‘stateful’logic operations via material implication,” Nature, vol. 464, pp. 873–876, 2010. B. Widrow and M. E. Hoff. “Adaptive switching circuits,” IRE WESCON Convention Record, pp. 96–104, 1960. A. Chanthbouala et al. “A ferroelectric memristor,” Nature Mater. vol. 11, pp. 860-864, 2012. CMOS40 Design Rule Manual (STMicroelectronics, 2012).
