A Neuromorphic Circuit that Computes Differential Motion Ko-Chung Tseng and Alice C. Parker Ming Hsieh Department of Electrical Engineering University of Southern California Abstract—Detecting moving objects in a moving background or a dynamic scene is essential to the survival of some animals. The circuitry computing differential motion is found in the biological retina. An object-motion-sensitivity (OMS) ganglion cell remains silent under global motion of the entire image but fires when the image patch in its receptive field moves differently from the background. In this paper, we present a neuromorphic circuit that compares the motion speeds of the central receptive field and peripheral receptive field. We demonstrate that there is a response if motion speeds of the central and peripheral receptive fields are different. However, the response is suppressed if motion speeds of central and peripheral receptive fields are the same. Index Terms—Neuromorphic Circuit, Differential Motion Detection, Receptive Field, Inhibition, Retina.
I. I NTRODUCTION Large retinal shifts such as saccadic eye movements and head movements of humans and other animals may cause great excitation across the entire retina. However, humans do not report this effect perceptually. Although the brain motor command may suppress the perception of saccadic eye movements, research results suggest that retinal motion alone is sufficient to produce this perceptual suppression [1]. The mechanisms that suppress the visual effects of eye movements or head movements are found in the inner retina. In this paper, we present a neuromorphic circuit that compares the motion speed of the central receptive field and peripheral receptive field. If the two motions are at the same speed, the output response is suppressed. Modeling this aspect of the mammalian vision system could be useful for service robots, autonomous vehicles and other applications that require detection of moving objects in a dynamic scene. II. BACKGROUND AND R ELATED W ORK An object-motion-sensitivity (OMS) ganglion cell remains silent under global motion of the entire image but fires when the image patch in its receptive field moves differently from the background [2], [3]. The OMS neurons are highly tuned to detect differential motion between the receptive field center and the periphery. The polyaxonal amacrine cell appears to be a plausible candidate to transmit inhibition from the This material is based upon work supported by the WiSE program at USC, the Viterbi School of Engineering at USC, and the National Science Foundation under Grant No. 0726815, Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Fig. 1. The underlying retinal circuitry for performing differential motion detection found in the biological retina [2], [3]
background region [3]. The inhibition signal may be derived from polyaxonal amacrine cells that inhibit the bipolar cell synaptic terminal, close to the site of transmission but at some electrotonic distance from the soma. The bipolar cells involved in the computation of differential motion detection are mainly transient OFF-type bipolar cells in the salamander [3]. Transient ON-type bipolar cells are believed to be involved in this computation as well [3][4]. The bipolar cells not only relay the visual information from photoreceptors but also shape visual response before transmitting to the inner retina. The details of the signalling cascade in the bipolar cells are given by several researchers [5], [6], [7]. The signaling cascade can be considered a feedback effect that enables the conversion of a sustained input from photoreceptors into a more-transient output [8]. At the axon terminal of the bipolar cells, the output is rectified [9]. The rectified response ensures that each bipolar cell’s vote will be counted and cannot be vetoed by signals of equal magnitude but opposite sign in other parts of the receptive field [9]. The axon terminals of the bipolar cells receive inhibition from the polyaxonal amacrine cells, probably prior to rectification, through glycine receptors. It is believed that this inhibition prevents the ganglion cell from receiving inputs from the bipolar cells when center and surround motion are the same. The proposed underlying biological retinal circuitry for performing differential motion detection is shown in Figure 1. The visual inputs from the central and peripheral receptive fields are relayed by the bipolar cells. The polyaxonal
amacrine cell delivers timed inhibition from the peripheral receptive fields to suppress the responses of central bipolar cell terminals if motion is the same centrally and peripherally. The object-motion-sensitive (OMS) ganglion cell synapses with the bipolar cells of the central receptive field. An OMS ganglion cell remains silent under global motion of the entire receptive field but fires when the image patch in the peripheral receptive field moves differently from the central receptive field. The underlying retinal circuitry compares only the speed of motion of the central receptive field and peripheral receptive field, not the direction and has been shown to prefer fast motion [3]. Both object-motion sensitivity and saccadic suppression involving timed inhibition from globally correlated shifting stimuli are believed to share common circuitry [10]. Andreou and Strohbehn built an analog VLSI implementation inspired by the visual system of a fly in which the delayed photoreceptor responses may correlate with the current responses of neighboring receptors to perform motion detection [11]. Liu proposed a neuromorphic vision chip which was also inspired by motion computation in the fly’s visual system [12]. In their silicon retina, Benson and Delbruck used inhibitory connections in the null direction to perform direction selectivity [13]. Wang and Liu used spiking neurons for motion detection in their aVLSI implementation [14], based on the model proposed by Rao for explaining the formation of direction- and velocity- selective cells in the visual cortex [15]. Tseng and Parker’s neuromorphic circuit modeled a portion of two communicating biological starburst amacrine cells in the retina that performs directional selectivity [16]. However, none of these circuit models detect differential motion of two image patches. III. A C IRCUIT TO D ETECT D IFFERENTIAL M OTION We constructed a neuromorphic circuit that computes differential motion found in the retina (Figure 2). The network consists of a 7-by-70 photoreceptor array, a layer of horizontal cell, 10 bipolar cells, and one sublinear voltage adder that models an amacrine cell. The photoreceptors and the horizontal cell layer that perform contrast enhancement are not described in this paper. Each receptive field is covered by 5 bipolar cells. One bipolar cell connects postsynaptically with 5 photoreceptors and the amacrine cell connects postsynaptically with 5 bipolar cells and presynaptically to axonal terminals in 5 other bipolar cells belong to a different receptive field. The amacrine cell is modeled by using a voltage adder circuit [17] that performs non-linear summations. Modeling the ganglion cells is complicated and is not the focus of this paper. Hence, we used a sublinear voltage adder to sum up the bipolar cells’ responses from the central receptive field and measured the output response of the summation of the bipolar cells. Therefore, we may easily observe the effect of inhibition from the peripheral receptive field under different cases. The bipolar cells that we modeled in the network are transient ONtype bipolar cells in which the mGluR6 (glutamate receptor) cascade causes conversion of a sustained input from photoreceptors into a more transient output [8]. To model the mGluR6
Fig. 2.
The circuit model performs differential motion detection.
Fig. 3. The post-synaptic circuit of a transient-ON bipolar cell that models the mGluR6 cascade mechanism. The left part of the figure is taken from [7].
cascade, the post-synaptic circuit of transient ON bipolar cell in Figure3 has a negative feedback loop. Transistors M1 and M2 correspond to mGluR6-receptor concentration and mGluR6-gated channel respectively. Transistors M3 and M4 form a feedback loop while Transistor M5 provides some delay that shapes the output. The response at the circuit’s output is denoted by L-EPSP (Light-evoked EPSP). The input from a photoreceptor first increases the conductivity of transistor M1 and pulls up L-EPSP. Meanwhile, the increase of LEPSP decreases the conductivity of transistor M2 through the feedback path formed by transistor M3, M4, and M5. It leads to the decrease of L-EPSP. The post-synaptic circuit of transient ON bipolar cell converts a sustained input into a more-transient output. The simulation result is shown in Figure 4. A complete circuit model of a transient ON-type bipolar cell is presented in Figure5. The bipolar cell sums the L-EPSPs nonlinearly using a sublinear adder and rectifies the response at the output using a pass transistor M7. Transistor M6 models the glycine receptor that receives the inhibition from the amacrine cell. IV. S IMULATION E XPERIMENTS The simulations were conducted with TSMC18 CMOS (180nm) technology using the SPECTRE simulator. We used the circuit configuration shown in Figure 1 and applied grating stimuli moving from the right to the left to the receptive field. Grating stimuli consists of black and white bars that
Fig. 4. The simulation result of a post-synaptic circuit of transient-ON bipolar cell.
Fig. 5. The transient-ON bipolar cell model. The post-synaptic circuit models the mGluR6 cascade mechanism. The L-EPSPs are summed and rectified at the output. Transistor M6 pulls down the output voltage if the input from an amacrine cell is asserted.
are represented by the photocurrent of 200nA and 250nA respectively. The black bars span four photoreceptors while the white bars span only one photoreceptor. We tried two different cases of moving grating stimuli. The results are shown in Figure 6. The responses shown in Figure 6 records the summation of the bipolar cells’ responses in the central receptive field during the first 8msec. When the moving grating stimuli are moving at the same frequency, the responses are much smaller than those moving at different frequencies. The smaller response in the upper waveforms of Figure 6 is due to the timely inhibition from the peripheral receptive field. The responses of both cases are small at the beginning of the simulation (before 1m sec) because the moving bars have not filled up the entire receptive field. We simulated the circuit using different speed combinations of the central and peripheral receptive field (both from 50KHz to 5KHz) and measured the maximum output response during the first 10msec. The black and white bars are represented by the photocurrents of 200nA and 250nA respectively. The results are shown in Figure 7. The x-axis and y-axis represent the speeds of moving bars in the peripheral and central receptive fields. The highlighted cells indicate the responses less than 0.35 V. The cells falling on the diagonal line are the cases having the same speed of moving bars in the peripheral
Fig. 6. The responses under different speed combinations. The waveforms imply that the response is suppressed when the moving bars of the central and peripheral receptive field move at the same speed.
and central receptive fields. The responses recorded from those cells are smaller due to the timely inhibition from the peripheral receptive field that suppresses the responses of the bipolar cells in the central receptive field. We also observed that the different speeds of moving bars may still produce small responses in certain cases, i.e. the highlighted cells not falling on the diagonal line. Those cases occur when the speed of the moving bars in the peripheral receptive field is a multiple of that in the central receptive field or when the bars in the peripheral receptive field are moving much faster than the bars in the central receptive field. In the latter case, the cells close to the lower left corner of Figure 7, the inhibition from the peripheral receptive field is too strong to generate the response at the output. Hence, we concluded that an output response above a certain value (i.e. 3.5 V in the case we presented in Figure 7) indicates that the motions from the central receptive field and peripheral receptive field are at different speeds. However, an output response below that certain value does not necessarily imply the motions from the central receptive field and the peripheral receptive field are at the same speed due to a few failing cases that we found. Moreover, the amplitude of the output response does not indicate the magnitude of the speed difference according to the results we observed. We then fixed the input intensity of black bars and swept different input intensities of white bars. We have tried two different combinations of the speeds, i.e. at the same speed and at different speeds. For the case having the same speed, we used both 20KHz for the central and peripheral receptive field. For the case having different speeds, we used 20KHz and 12.5KHz for the central and peripheral receptive field respectively. The maximum output responses by sweeping input photocurrent of the white bars from 200nA to 450nA are recorded in Figure 8. Among the range we swept, the output responses are suppressed if the speeds are the same. V. C ONCLUSION We have presented a neuromorphic circuit that computes differential motion of the central receptive field and peripheral receptive field. We demonstrated that an output response above a certain value indicates that the motions from central receptive
Fig. 7.
The maximum responses under different speed combinations. The highlighted cells indicate the responses less than 0.35 V.
Fig. 8. The maximum output responses by sweeping input photocurrent of the white bars from 200nA to 450nA. The black bars are represented by injecting a fixed photocurrent of 200nA. For the case having the same speed, we used both 20K Hz for the central and peripheral receptive field. For the case having different speed, we used 20K Hz and 12.5K Hz for the central and peripheral receptive field respectively.
field and peripheral receptive field are at different speeds. However, an output response below that certain value does not necessarily imply the movements from the central receptive field and peripheral receptive field are at the same speed due to a few failing cases that we found in the simulations. Moreover, the amplitude of output response does not indicate the magnitude of the difference. This circuit model for detecting differential motion could be useful for future service robots, autonomous vehicles and other applications that require detection of moving objects in a dynamic scene or a moving background in real time. R EFERENCES [1] D. M. MacKay, “Elevation of Visual Threshold by Displacement of Retinal Image,” Nature, vol. 225, pp. 90–92, Jan. 1970. [2] B. P. Olveczky, S. A. Baccus, and M. Meister, “Segregation of object and background motion in the retina,” Nature, vol. 423, pp. 401–408, May 2003. [3] S. A. Baccus, B. P. Olveczky, M. Manu, and M. Meister, “A Retinal Circuit That Computes Object Motion,” J. Neurosci., vol. 28, pp. 6807– 6817, July 2008.
[4] M. N. Geffen, S. E. J. de Vries, and M. Meister, “Retinal ganglion cells can rapidly change polarity from off to on,” PLoS Biol, vol. 5, pp. e65+, March 2007. [5] Y. Shen, J. A. Heimel, M. Kamermans, N. S. Peachey, R. G. Gregg, and S. Nawy, “A transient receptor potential-like channel mediates synaptic transmission in rod bipolar cells.,” The Journal of neuroscience : the official journal of the Society for Neuroscience, vol. 29, pp. 6088–6093, May 2009. [6] C. W. Morgans, J. Zhang, B. G. Jeffrey, S. M. Nelson, N. S. Burke, R. M. Duvoisin, and R. L. Brown, “Trpm1 is required for the depolarizing light response in retinal on-bipolar cells,” Proceedings of the National Academy of Sciences, vol. 106, pp. 19174–19178, November 2009. [7] C. Ribelayga, “Vertebrate vision: Trp channels in the spotlight,” Curr Biol, vol. 20, pp. R278–R280, March 2010. [8] T. Kaur and S. Nawy, “Characterization of Trpm1 desensitization in ON bipolar cells and its role in downstream signalling,” The Journal of Physiology, vol. 590, pp. 179–192, Jan. 2012. [9] J. B. Demb, K. Zaghloul, L. Haarsma, and P. Sterling, “Bipolar cells contribute to nonlinear spatial summation in the brisk-transient (y) ganglion cell in mammalian retina.,” The Journal of neuroscience : the official journal of the Society for Neuroscience, vol. 21, pp. 7447–7454, October 2001. [10] S. A. Baccus, “Timing and computation in inner retinal circuitry.,” Annual review of physiology, vol. 69, pp. 271–290, 2007. [11] A. G. Andreou and K. Strohbehn, “Analog VLSI implementation of the Hassenstein-Reichardt-Poggio models for vision computation,” in Systems, Man and Cybernetics, 1990. Conference Proceedings., IEEE International Conference, pp. 707–710, 1990. [12] S.-C. Liu, “A neuromorphic aVLSI model of global motion processing in the fly,” IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 47, pp. 1458–1467, Dec 2000. [13] R. G. Benson and T. Delbr¨uck, “Direction selective silicon retina that uses null inhibition,” in Advances in Neural Information Processing Systems 4, vol. 4, pp. 756–763, 1991. [14] Y. Wang and S.-C. Liu, “Motion detection using an aVLSI network of spiking neurons,” in Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium, pp. 93–96, May 2010. [15] A. P. Shon, R. P. Rao, and T. J. Sejnowski, “Motion detection and prediction through spike-timing dependent plasticity.,” Network (Bristol, England), vol. 15, pp. 179–198, August 2004. [16] K.-C. Tseng, A. C. Parker, and J. Joshi, “A directionally-selective neuromorphic circuit based on reciprocal synapses in Starburst Amacrine Cells,” in Engineering in Medicine and Biology Society,EMBC, 2011 Annual International Conference of the IEEE, pp. 5674–5677, IEEE, Aug. 2011. [17] H. Chaoui, “CMOS analogue adder,” Electronics Letters, vol. 31, no. 3, pp. 180–181, 1995.