Analog Integrated Circuit for Detection of an ... - Semantic Scholar
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
416
PAPER
Special Section on Analog Circuit Techniques and Related Topics
Analog Integrated Circuit for Detection of an Approaching Object with Simple-Shape Recognition Based on Lower Animal Vision Kimihiro NISHIO†,††a) , Hiroo YONEZU† , Members, and Yuzo FURUKAWA† , Nonmember
SUMMARY A network for the detection of an approaching object with simple-shape recognition is proposed based on lower animal vision. The locust can detect an approaching object through a simple process in the descending contralateral movement detector (DCMD) in the locust brain, by which the approach velocity and direction of the object is determined. The frog can recognize simple shapes through a simple process in the tectum and thalamus in the frog brain. The proposed network is constructed of simple analog complementary metal oxide semiconductor (CMOS) circuits. The integrated circuit of the proposed network is fabricated with the 1.2 µm CMOS process. Measured results for the proposed circuit indicate that the approach velocity and direction of an object can be detected by the output current of the analog circuit based on the DCMD response. The shape of moving objects having simple shapes, such as circles, squares, triangles and rectangles, was recognized using the proposed frog-visualsystem-based circuit. key words: analog integrated circuit, edge detection, motion sensor, shape recognition, vision chip
1.
Introduction
Real-time image processing is needed for information systems such as collision avoidance systems and robotics vision systems. However, in typical image-processing systems, which are constructed with a charge coupled device (CCD) camera and a Neumann-type computer, the realization of real-time processing is difficult because information processing is performed in a time-sequential manner. Lower animals have a number of remarkable real-time vision functions, such as motion detection and simple-shape recognition, although their brains are compact. The processing in such cases is performed in massively parallel nerve networks. Integrated circuits (chips) based on lower animal vision can realize real-time image processing and simple structure. Many researchers including C. Mead designed and developed the motion detection circuits based on the algorithms such as the vertebrate retina and insect vision system [1], [2]. Recently, analog motion detection circuits [3]– [6] of a one- or two-dimensional array have been proposed based on the fly visual system [7]. These circuits were able Manuscript received June 18, 2005. Manuscript revised September 27, 2005. Final manuscript received October 20, 2005. † The authors are with the Department of Electrical and Electronic Engineering, Toyohashi University of Technology, Toyohashi-shi, 441-8580 Japan. †† The author is now with the Department of Electrical and Computer Engineering, Yonago National College of Technology, Yonago-shi, 683-8502 Japan. a) E-mail: [email protected] DOI: 10.1093/ietfec/e89–a.2.416
to operate at high speed because the unit circuits performed information processing in parallel, as in the insect vision system. These circuits are characterized by low power consumption because the metal oxide semiconductor (MOS) transistors used to construct such circuits operate in the subthreshold region. Being analog circuits, the unit circuits were constructed with a small number of transistors. Recently, compact target tracking system for robotics was developed by applying such circuits [2], [3]. Thus, analog circuits based on insect vision systems have superior functionality. Therefore, such circuits should be applied to collision avoidance systems for robotics and vehicles, for example. In this case, circuits for three-dimensional motion detection must be developed by applying previously proposed one- or two-dimensional circuits. The detection of an approaching object is the most basic function in three-dimensional motion detection. The signal of the descending contralateral movement detector (DCMD) [8]–[11] in the locust brain shows a peak value just before collision. This signal has been represented by a simple mathematical formula. In previous studies, calculation results have shown that the approach velocity and direction can be detected based on the peak signal [12], [13]. G. Indiveri proposed a simple network and circuits for the detection of an approaching object based on the locust vision system [14]. Since the output signal of the circuit corresponds to that of the DCMD, the circuit could detect the approach of an object when the object approached from the front. In addition, G. Indiveri et al. proposed a circuit for estimating the time to contact [2], [15]. Since the output signal of the circuit is proportional to the time to contact, the circuit could estimate the time to contact when the object approached from the front. Information of the approach direction is needed for the determination of the escape direction for collision avoidance. However, previously proposed circuits [2], [14], [15] did not contain a function to detect the approach direction, and circuits for detection of the approach direction must therefore be developed. When a predator approaches, the frog can escape, and when prey approaches, the frog can jump toward the prey in order to capture it. Since the frog has a function for discriminating predator from prey, the vision system of the frog differs from that of the locust, for example, which can only escape from an approaching object, and has the new function like capture. This is because the frog has a func-
c 2006 The Institute of Electronics, Information and Communication Engineers Copyright
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
417
tion for simple-shape recognition. The response for simpleshape recognition is observed in the tectum and thalamus in the large-field cells of the frog brain [16], [17]. P. Ewert proposed a model for simple-shape recognition. The shape recognition function of the cell is limited to objects that are large in the lateral direction and objects that are long at the side except that the function for the movement detection of an approaching target into the DCMD. Even if the complex shape is not recognized, predator, prey or other objects can be distinguished by combining simple-shape information with movement information because the movement speed and pattern are limited to a certain extent. Because a pattern is decided as the on the car and walker, it can think that it becomes possible that they are distinguished in the same manner. Thus, shape recognition is required in order to discriminate objects. The only function of the previously proposed circuit [2], [14], [15] is motion detection. Object discrimination is possible if a simple-shape recognition function can be added to the function of the previously proposed circuit. The development of such a circuit is expected to have greater applicability than the previously proposed circuit. Therefore, in the present study, an analog integrated circuit for the detection of an approaching object with simple-shape recognition that is based the lower animal vision is fabricated with the 1.2 µm complementary metal oxide semiconductor (CMOS) process. Measured results indicate that the fabricated chip operates normally. 2.
Biological Model
2.1 Descending Contralateral Movement Detector Figure 1(a) shows the relationship between an object approaching from the right front with constant velocity v and the locust eye. The object is square and has a length of w x (= wy ). The approach direction is θdir,x . The difference between θleft and θright is θsize,x . Figure 1(b) shows the approaching object from the upper front, where θdir,y is the approach direction and θsize,y is the difference between θtop and θbottom . As the object approaches, the size of the image projected on the retina increases rapidly just before collision. Figure 1(c) shows the projected image before collision. The width wpx corresponding to θsize,x and the expansion velocity corresponding to the image angular velocities (θ˙left and θ˙right ) increase just before collision. When the object approaches from the right front, |θ˙left | is larger than |θ˙right |. Conversely, |θ˙left | is smaller than |θ˙right | when the object approaches from the left front. A locust can detect an approaching object by the signal generated by the descending contralateral movement detector (DCMD) in the locust brains. The existence of two types of DCMDs has been reported [10]. One DCMD generates the signal fleft (t), and the other generates fright (t), where fleft (t) and fright (t) are given by following equations [12], [13]:
Fig. 1 Approach detection model. (a) Object approaches from the right front. (b) Object approaches from the upper front. (c) Image projected on the retina before collision. (d) Output signals of DCMDs.
fleft (t) = θ˙left (t) exp(−θsize,x (t)), fright (t) = θ˙right (t) exp(−θsize,x (t)).
(1) (2)
The signals of the DCMDs under the condition of Fig. 1(a) are shown in Fig. 1(d). The signals are first increased by the increase in the angular velocity and are later decreased according to exp(−θsize,x ) [8]–[11]. Therefore, fleft (t) and fright (t) show the peak values just before collision. When the object approaches from the right front, fleft (t) is larger than fright (t). The difference between fleft (t) and fright (t) is fdir,x (t), which is given as follows: fdir,x (t) = fleft (t) − fright (t).
(3)
The absolute value of the peak signal fdir,xp of fdir,x (t) is proportional to that of θdir,x [12], [13]. The sum of fleft (t) and fright (t) is fvel (t), which is given as follows: fvel (t) = fleft (t) + fright (t).
(4)
The peak value fvel,p of fvel (t) is proportional to v [12], [13]. In order to avoid a collision, the locust must escape quickly when the object is large. The peak response was
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
418
found to be generated when the image angle reached a constant threshold angle, which is independent of the approach velocity and the size of the object. Therefore, the time between the occurrence of fvel,p and collision should be proportional to w x [13]. Thus, the collision can be avoided with this model. The approach direction θdir,y in Fig. 1(b) can be detected by applying Eq. (3) to the y-direction. The signal fdir,y (t) for detecting θdir,y is given as fdir,y (t) = fbottom (t) − ftop (t), where ftop (t) and fbottom (t) are given as ftop (t) = θ˙top (t) exp(−θsize,y (t)), fbottom (t) = θ˙bottom (t) exp(−θsize,y (t)),
Iarea , 2 Iperi
(9)
(6) (7)
3.
2.2 Ewert von Seelen Model The Ewert von Seelen model [16] for recognizing simple shapes, which is based on the frog visual system, is shown in Fig. 2. Edge signals generated by the retina are transmitted to the tectum (type I) and the thalamus. The signal Isize,x transmitted to the tectum becomes large when a laterally extending object is projected onto the retina, and the signal Isize,y transmitted to the thalamus becomes large when a vertically extending object is projected on the retina. The excitatory signal Isize,x and the inhibitory signal Isize,y are transmitted to the tectum (type II). The output signal IE of the tectum (type II) becomes constant Iconst when the square object is projected on the retina. The output signal IE is larger than Iconst when a laterally extending object is projected. The output signal IE is smaller than Iconst when a vertically extending object is projected. Thus, IE is given as Isize,x Iconst . Isize,y
IP =
where Iarea and Iperi are the signals of the area and the perimeter of the projected image, respectively, is inserted into the model [18]. As a result, the model can discriminate simple shapes, such as circles, squares and triangles, from the value of IP .
(5)
where θ˙top and θ˙bottom are angular velocities corresponding to the velocities of the top edge and bottom edge, respectively [12]. The approach direction can be detected using fdir,x (t) in Eq. (3) and fdir,y (t) in Eq. (5).
IE =
and θsize,y shown in Figs. 1(a) and (b), respectively. It is impossible to discriminate shape, for example, circle or square, in the model shown in Fig. 2 because both output signals are identical. Thus, the performance function IP , which is given by
(8)
Network for Generating Signals of Angle, Angular Velocity, Area and Perimeter
3.1 Network Construction Generation of the signals indicating the angle, angular velocity, area and perimeter is required for fabricating a chip for approach detection with simple-shape recognition. A network for generating these signals is shown in Fig. 3. The network is constructed using a photodiode (PD), an edge detection circuit (EDC), size detection circuits (SDs) and velocity sensing circuits (VSCs). While the image is projected onto the PD, the photocurrent Ipi, j increases. The photocurrent Ipi, j is digitized by the area detection circuits described in the next section. The output current of the area detection circuit is Iai, j , which becomes constant at the position projecting the image. At all other positions, Iai, j is 0. The signal Iarea , which is proportional to the area, can then be generated as follows: Iai, j . (10) Iarea = On the other hand, Ipi, j is also input to the EDC. The output current Iei, j is the constant current at the edge position. The signal Iperi , which is proportional to the perimeter,
The signals Isize,x and Isize,y are proportional to angles θsize,x
Fig. 2
Model for recognizing simple shapes.
Fig. 3 Network for generating the signals of the angle, angular velocity, area and perimeter.
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
419
can be generated by the EDC and is given as follows: Iperi = Iei, j . (11)
at node a, the Iarea becomes the sum of all Ia,i . The circuit shown in Fig. 4(b) is also used for the realization of Eq. (11).
By summing Iei, j for each row and column, Iex,i and Iey, j are output, respectively. The currents Iex,i and Iey, j are input to the SDs. When currents Iex,i and Iey, j are larger than the threshold current, Idx,i and Idy, j become constant, and when Iex,i and Iey, j are smaller than the threshold current Idx,i and Idy, j are 0. The sum of each Idx,i is Isize,x , which is proportional to θsize,x , and the sum of each Idy, j is Isize,y , which is proportional to θsize,y . The output currents of the VSCs are Iv,right and Iv,left when the edge moves toward the right and left sides, respectively. Thus, Iv,right and Iv,left correspond to angular velocities. Similarly, Iv,top and Iv,bottom also correspond to angular velocities.
3.3 Edge Detection Circuit
3.2 Area Detection Circuit Figure 4(a) shows the unit circuit for area detection. The photocurrent Ip,i , which is proportional to the light intensity, is input. The threshold current Ith is set by providing threshold voltage Vth . When Ip,i is larger than Ith , voltage Va1 is the supply voltage VDD . Then, output current Ia,i is the constant current Ic0 which is set by providing the constant voltage Vc0 . When Ip,i is smaller than Ith , Ia,i is 0 because Va1 is 0. The output current Ia,i becomes the constant current Ic0 when the bright image is projected. Figure 4(b) shows the adder circuit used to generate Iarea in Eq. (10). Each Ia,i generated by the area detection circuit is input to the pMOS current mirror circuit. Since all of the output terminals of the mirror circuits are connected
Fig. 4
Area detection circuit. (a) Unit circuit. (b) Adder circuit.
Figure 5 shows the unit circuit for edge detection based on the vertebrate retina [19], [20]. The photocurrent Ip,i is input to the smoothing circuit (Mh1 and Mh2 ). The current Ih,i is generated by diffusing Ip,i into part of a signal to neighboring circuits. The current Ib,i is generated by subtracting Ih,i from Ip,i . The absolute value of Ib,i at the edge position is larger than at other positions. The current Ib,i is input to the digitizing circuit (DG). The current Ie,i becomes the constant current Ic0 at the edge position, because the voltage Vdg is the supply voltage when Ib,i is larger than the threshold current Ith [12], [21], and becomes 0 at other positions be cause Vdg is 0. The DG in Fig. 4 is also applied to the SD in Fig. 3. 3.4 Velocity Sensing Circuit Figure 6(a) shows a velocity sensing model called the correlation model [7]. The unit cell (Elemental motion detector; EMD) is constructed with the unit neuron of the retina R, delay neuron D and correlator C. Figure 6(b) shows the output signal from each cell when the spot object moves to the right side. The unit cell of the retina R1 first generates the signal IR1 . The delay neuron D1 then indicates the maximum value. When the spot is projected on a position between R1 and R2 , the signal IR1 is not generated and ID is decreased. After time tv , which is inversely proportional to velocity ve , the spot moves to R2 and IR2 is generated. The signals ID and IR2 are input to the correlator C2 , which then generates signal IC . Since the maximum value IC,peak of IC depends on ID , IC,peak is inversely proportional to tv . Therefore, IC,peak is proportional
Fig. 5
Unit circuit for edge detection.
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
420
Fig. 6 Velocity sensing model and unit circuit. (a) Model based on the visual system of the fly. (b) Output signal from each cell. (c) Unit circuit.
to ve . However, IC,peak is not proportional to ve when the wide object shown in Fig. 6(a) is projected since an edge detector such as EDC is used. The EDC outputs the signal IR1 even if the edge of the wide object is projected on the position between R1 and R2 , as shown in Fig. 6(b). Since IC,peak shows a constant maximum value when tv = 0, IC,peak is independent of velocity ve . When the edge of a wide object moves toward the right side, the EMD outputs the positive constant pulsed signal Im,i , and when the edge moves toward the left side, the EMD outputs the negative constant pulsed signal Im,i . The sum of positive Im,i is Iv,right , and the sum of the signals, which is given by rectifying the negative Im,i , is Iv,left . The signal Iv,right when the edge moves toward the right side is shown in Fig. 6(a). When the edge velocity is fast, the interval of pulse td is short. Thus, the velocity can be detected by detecting td . The velocity sensing circuit (VSC) based on the EMD is shown in Fig. 6(c) [3], [4]. A number of circuits based on the EMD have been proposed [3]–[6]. The circuit in Fig. 6(c) is considered to be the simplest structure among the proposed circuits, i.e., the number of MOS transistors
is low. Therefore, this circuit is added to the proposed network. The current Idx,i is an input current. The circuit constructed with capacitor CD and nMOS transistor MD corresponds to the delay neuron D, and the nMOS transistor MC corresponds to the correlator C. The current IC,i is rectified by the pMOS current mirror circuit (M1 and M2 ), and the output current is Im,i . When the edge moves toward the right side, Im,i becomes IC,i because IC,i−1 is 0. When the edge moves to the left side, Im,i represents IC,i−1 . 4.
Analog Circuits for Approach Detection and SimpleShape Recognition
4.1 Approach Detection Circuit Based on DCMD Figure 7(a) shows the approach detection circuit based on the DCMD response. The circuit inputs currents Isize,x , Iv,right and Iv,left , as shown in Fig. 3, which are converted to voltages Vsize,x , Vv,right and Vv,left , respectively. The voltage Vsize,x increases just before collision because Isize,x increases. The voltages Vv,right and Vv,left also increase just before col-
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
421
lision because td in Fig. 6(a) becomes short. Since Vv,right and Vv,left depend on the velocities of the edges, Vv,right and Vv,left correspond to the angular velocities |θ˙left | and |θ˙right | in Eqs. (1) and (2), respectively. Since Vsize,x is the gate voltage of pMOS transistor Ma3 , Iexp decreases just before collision. The Ma3 needs to operate in the subthreshold region. Therefore, Iexp is approximately equivalent to exp(−θsize,x ) in Eqs. (1) and (2). The output current Iright is first increased by Vv,right and then decreased by Iexp , because Iright is equal to Iexp when the voltage Va is large and Iexp is small. Thus, Iright corresponds to fright (t) in Eq. (2) because Iright indicates the peak value just before collision. Similarly, Ileft is first increased by Vv,left and then decreased by Iexp because Ileft is equal to Iexp . Thus, Ileft corresponds to fleft (t) in Eq. (1) because Ileft shows a peak value just before collision. The circuit shown in Fig. 7(a) is used to realize Eqs. (6) and (7). The currents Isize,y , Iv,top and Iv,bottom generated by the network shown in Fig. 3 are input to this circuit, and the resulting output currents are Itop and Ibottom . Since the required processing is equal to that for the circuit shown in Fig. 7(a), Itop and Ibottom correspond to ftop (t) and fbottom (t) in Eqs. (6) and (7), respectively. Figure 7(b) shows the circuit for generating the signals of the approach velocity and direction. The currents Ileft and Iright are input to the pMOS and nMOS current mirror circuits, respectively. Here, Iright is negative because it is input to the nMOS current mirror circuit. The output current Idir,x is obtained by subtracting Iright from Ileft . Therefore, Idir,x corresponds to fdir,x in Eq. (3). Since each output terminal of the pMOS current mirror circuit is connected, the output current Ivel is represented by the sum of Ileft and Iright . Thus, Ivel corresponds to fvel (t) in Eq. (4).
Fig. 7 Approach detection circuit based on DCMD response. (a) Circuit based on Eqs. (1) and (2). (b) Circuit based on Eqs. (3) and (4).
The circuit in Fig. 7(b) is used to realize Eq. (5). Here, Itop and Ibottom are input to the circuit, and the output current is Idir,y , which corresponds to fdir,y (t) in Eq. (5). 4.2 Circuit for Simple-Shape Recognition Realization of Eqs. (8) and (9) requires a circuit that can realize both a division circuit and a square circuit, as shown in Fig. 8 [22]. When all MOS transistors operate in the subthreshold region, the output current Ijout is given as Ijout
Ia Ib . Ic
(12)
The circuit is used as a division circuit by setting Ia to a constant current Iconst and as a square circuit by setting Ia = Ib and Ic = Iconst . The circuit shown in Fig. 8 is based on the Ewert von Seelen model, (EWC) in Eq. (8), by setting Ia = Iconst , Ib = Isize,x , Ic = Isize,y and Ijout = IE . The division and square circuits of Fig. 8 are based on the performance function (PFC) shown in Eq. (9). One circuit is used as a square 2 , where Ijout is made proportional circuit for generating Iperi 2 to Iperi by setting Ia = Ib = Iperi and Ic = Iconst , and the other circuit is used as a division circuit. Ijout becomes IP in Eq. (9) 2 . by setting Ia = Iconst , Ib = Iarea and Ic = Iperi 5.
Experimental Results
5.1 Chip Structure and Experimental Condition A chip for approach detection with simple-shape recognition was fabricated using the double-metal double-poly standard 1.2 µm CMOS process. A photograph of the chip is shown in Fig. 9. Two-dimensional EDCs (15 × 15) having an area of 4.8 × 4.8 mm2 were designed. The size of the PD was 50 × 50 µm2 . The total size of the EDC and the area detection circuit was 180 × 180 µm2 . The fill factor was approximately 10%. Fifteen SDs and VSCs were arranged in the x- and y-directions. The capacitance of CD was fabricated to be 0.5 pF. The capacitance of CD decides the pulsed width td in Fig. 5(a). The width must be set smaller than 1 ms in order to realize high-speed processing. The results obtained with the simulation program with integrated circuit emphasis (SPICE) indicate that the pulsed width is approximately 1 ms when
Fig. 8
Division and square circuit for recognizing simple shapes.
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
422
CD is 0.5 pF. Therefore, the capacitance of CD of the test chip was also fabricated to be 0.5 pF. The capacitance of Ca was fabricated to be 50 pF. When Ca is too small, Vv,right does not increase because the discharge from Ma2 in Fig. 6(a) occurs quickly. Thus, the approach detection circuit cannot output the peak current. When Ca is too large, the area becomes large, and so Ca has to be set to a minimal value so that Iright indicates the peak value. The results obtained with SPICE reveal that the approach detection circuit outputs the peak current when Ca is 50 pF. However, the peak current was not obtained when the capacitance of Ca was set to less than 45 pF. Therefore, the capacitance of Ca of the test chip was also fabricated to be 50 pF. Figure 10 shows a schematic diagram of the measurement equipment. A personal computer (PC) is connected to a projector, from which the input image generated by the PC was projected. The image was projected onto the chip through a lens. The photocurrent was 80 nA and 100 pA, respectively, for white and black images projected on the 50 × 50 µm2 PD. The illumination was about 500 W/m2 and 0.6 W/m2 when the white and black image was projected respectively. The supply voltage was set to 3 V. Vdif and Ith1 in Fig. 4 were set to 1.80 V and 5 nA, respectively. The bias conditions of Vdif in Fig. 5 and Ith in Fig. 4 depend on the illumination conditions. The setup of Vdif is important to generate the diffusing currents by Mh1 and Mh2 of the EDC. The diffusing current is decided by Vdif and Ip,i . When the diffusing current is too large or too small, the edge signal cannot be generated [20]. We confirmed by SPICE that the correct value of Vdif is in the range from 1.75 V to 1.85 V. Therefore,
Fig. 9
Fig. 10
Vdif was set to 1.80 V. In this experiment, Ith generated by Vth has to be set to a value between 100 pA and 80 nA because the photocurrent is in the range between 100 pA and 80 nA. The Vgs − Ids characteristic of the nMOS transistor was obtained by measurement, and Ith was set to 30 nA by setting Vth based on the obtained data. Other bias conditions are independent of the illumination conditions because the edge signals are digitized. Therefore, the correct bias conditions were decided by measuring the test circuits. As a result, Ic0 in Figs. 4(a) and 5 was set to 10 nA and Vbias in Fig. 7(a) was set to 1 V. 5.2 Velocity Sensing Circuit Figure 11(a) shows the measured result under the condition in which the right edge of the rectangle image moved onto the chip for a period of 80 ms. The circuit output the constant pulsed current Iv,right . Thus, Iv,left , Iv,top and Iv,bottom were 0. Figure 11(b) shows the result when the right edge moved onto the chip for a period of 1.5 s. The circuit output
Photograph of the chip.
Schematic diagram of the measurement equipment.
Fig. 11 Transient response of the VSCs. (a) Output current Iv,right under the condition that the right edge moves toward the right side of the chip for 80 ms. (b) Output current Iv,right under the condition that the right edge moves toward the right side of the chip for 1.5 s. (c) Output current Iv,left .
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
423
the constant pulsed current Iv,right . The values of the currents in Fig. 11(a) were equal to those in Fig. 11(b). The interval of pulse td in Fig. 11(b) was longer than that in Fig. 11(a). The velocity was clarified to be detectable by the detection of td . The circuit was measured under the condition that the left edge of the rectangular image moved toward the left side on the chip for a period of 1.5 s. Figure 11(c) shows the output current Iv,left , which was a constant pulsed current. Thus, Iv,right , Iv,top and Iv,bottom were 0. The results shown in Figs. 11(b) and (c) clarified that the moving direction can be detected. Furthermore, the circuits were shown to output constant currents Iv,top and Iv,bottom when the edge moves toward the upper and lower sides, respectively.
Fig. 12 Transient responses of the approach detection circuit. (a) Images projected on the chip. (b) Output current Ivel . (c) Output current Idir,x . (d) Output current Idir,y .
5.3 Approach Detection Circuit Approach detection circuits were measured under the condition in which a square object approaching from the upper right front was projected onto the chip. The values of v, θdir,x (= θdir,y ) and w x (= wy ) were set to 30 m/s, 30◦ and 0.4 m, respectively. Figure 12(a) shows the relationship between the projected image and the chip. When the entire image is projected onto the chip, as shown in right-most photograph of Fig. 12(a), the time to collision tcoll is 0. Figures 12(b), (c) and (d) show output currents Ivel , Idir,x and Idir,y , respectively. The output currents Ivel , Idir,x and Idir,y showed peak values of Ivel,p , Idir,xp and Idir,yp , respectively, just before collision. In addition, the relationship between Ivel,p and v was investigated. Figure 13(a) shows the measured results and the simulation results obtained using SPICE. The Ivel,p was approximately proportional to v, indicating that Ivel corresponds to fvel (t). The measured results and the simulation results showed good agreement. Thus, it was clarified that v can be detected using Ivel,p . We also investigated the dependence of Idir,xp on θdir,x . Figure 13(b) shows the measured results along with the simulation results. The absolute value of Idir,xp was approximately proportional to that of θdir,x , indicating that Idir corresponds to fdir,x (t). The measured results corresponded approximately to the simulation results. The absolute value of Idir,yp was proportional to that of θdir,y , indicating that θdir,y can be detected from Idir,yp . Thus, it was clarified that the
Fig. 13 Characteristics of the approach detection circuit. (a) Dependence of the peak current Ivel,p on velocity v. (b) Dependence of the peak current Idir,xp on direction θdir,x .
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
424
direction can be detected using Idir,xp and Idir,yp . 5.4 Circuit for Simple-Shape Recognition Figure 14(a) shows the measured results of the EWC and each image projected onto the chip. Each image was moved in order to confirm that the circuits could recognize the simple shape of the moving image. Each image approached from the front. The images were projected on the entire area of the chip at 600 ms. In Fig. 8, Ia was set to 10 nA (= Iconst ). When the square image approached the chip, the output current IE was 10 nA (= Iconst ). When a rectangular image having a lateral length that was twice as long as the vertical length, approached the chip, IE was 20 nA (= 2Iconst ). When a rectangular image having a vertical length that was twice as long as the lateral length, moved onto the chip, IE was 5 nA (= 0.5Iconst ). Figure 14(b) shows the measured results of PFC and each image projected onto the chip. Each image approached from the front. When circular, square and triangular images
were projected on the chip, IP became approximately 80 nA, 50 nA and 30 nA, respectively. Figure 15 shows the output current IP of PFC and the calculated results of the performance function. The horizontal axis of the graph indicates the shape S. The gradient of the current IP was approximately equal to that of the calculation results. The output current of PFC thus corresponds to the performance function. Thus, the results shown in Figs. 14 and 15 clarify that simple-shape recognition based on IE and IP is possible, even if the size of the object changes upon approach. 6.
Discussion
6.1 Comparison with Previously Proposed Circuits The test chip for the detection of an approaching object with simple-shape recognition was fabricated using the 1.2 µm CMOS process. It was clarified that, using the fabricated chip, the moving direction and velocity can be detected and, simultaneously, the simple shape can be recognized. New functions for the detection of the approach direction and recognition of the simple shape, which were not realized in previous circuits [2], [14], [15], were incorporated in the newly fabricated chip. 6.2 Device Mismatch The test chip was constructed using simple analog circuits. Although analog circuits have a problem with respect to incorrect operation caused by device mismatches, the fabricated chip operated correctly. It should be considered that the digitizing circuit, which can output a constant current even if the currents vary as a result of device mismatches, was inserted in EDCs and SDs. The pulsed currents shown in Fig. 11 indicate different values due to device mismatches. This problem can be solved to some extent by using DG. Pulsed currents of constant values can be output by inputting Iv,right and Iv,left to DG. Although pulsed currents containing different values,
Fig. 14 Transient responses of the circuit for recognizing simple shapes. (a) Output current IE . (b) Output current IP .
Fig. 15 sults.
Output current of PFC and calculated performance function re-
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
425
as shown in Fig. 11, were input to the approach detection circuits, the circuits operated normally, as shown in Figs. 12 and 13. In biological systems, the output signals In of multiple nerve cells, which contain variation, are assembled [1], where subscript n is the number of cells. The average value of the signals is the nearest correct signal Icor . The CR integrator constructed with Ca and Ma2 shown in Fig. 7(a) has the function for assembling and averaging the signals. Therefore, the approach detection circuits operated normally. 6.3 Output Signals of Proposed Network Currents Ivel , Idir,x and Idir,y showed peak values just before collision, as shown in Fig. 12. Although the output signal in Fig. 1(d) includes only one peak, several peak values appear in Fig. 12. The reason was that Ma2 was included as a discharge pathway in Fig. 7(a). We defined the maximum value in Fig. 12 as the peak value in Fig. 1(d). Note that the values for Ivel,p and Idir,xp in the measurement were larger than those in the simulation. This discrepancy is due to the difference of the measurement and model parameters between SPICE and the fabricated MOS transistors. The test chip was able to recognize the simple shape of a moving object by the EWC and the PFC, as shown in Figs. 13, 14 and 15. However, it is difficult to recognize a simple shape by the use of only PFC. That example is described in the following. The value of IP of the PFC when the ellipse was projected is equal to that when the square was projected. Figure 16 shows the calculation results of the performance function. The solid line shows the performance function when w x /wy = 1, and the dashed line shows the performance function when w x /wy = 3. The value of the ellipse is approximately equal to that of the square, and the value of the rectangle is equal to that of the triangle. It is necessary to discriminate the ellipse and the square, and the method used for discrimination is described in the following. The solid line should be used when the output current IE of EWC is 10 nA, i.e. w x /wy = 1. Then, the shape is square when the output current IP of the PFC is 60 nA, i.e. the performance function is 0.06. The dashed line is used when IE is 30 nA,
Fig. 16
Calculated results of the performance function.
i.e. w x /wy = 3. Then, the shape is an ellipse if IP is 60 nA. Thus, the chip is able to recognize various simple shapes by using the signals of the EWC and the PFC. 6.4 Input Part Motion detection and shape recognition are difficult when a real image was projected on the fabricated chip since the test chip contained only a 15 × 15 EDC. As a result, a white image and a black image, representing an object and the background, respectively, were projected onto the chip. Simulation results obtained using SPICE confirmed that unit circuits of at least 50 × 50 are needed in order to obtain the edge signals of the real image [20]. In the present study, large transistors that are twice as large as the minimum line width were used for the test chip. If a chip is fabricated to have the minimum line width, simple calculations indicate that an edge detection circuit of approximately 30 × 30 can be designed in an area of 4.8 × 4.8 mm2 . In addition, more unit circuits for edge detection can be designed if the chip is designed using a recent process rule. If such a circuit is realized, then motion detection and shape recognition become possible even if the real image is projected. However, even if 50 × 50 or larger edge detection circuits are designed, the proposed chip could only be used in a fixed indoor environment, because the edge detection circuit of the chip has a dynamic range of only approximately 2.5 decades [20], whereas the dynamic range of the vertebrate retina is over six decades [23]. Several attempts have been made to a develop chip that has a wide dynamic range [24]. We also tried to improve the EDC by mimicking the signal processing of rods and cones of the photoreceptor [25]. The realization of a wide dynamic range of more than six decades is expected by mimicking the function for amplifying the photocurrent under low light intensity based on the rod of the photoreceptor. 6.5 Motion Detection in Real Image In this experiment, the background (black image) and an object (white image) were projected onto the chip. It tries to think about the case that an object of a black image and a background of a white image are projected on the chip. In this case, the signals of the width and the expansion velocity can be generated, because these signals are generated by the edge signals. The approach detection circuit and the EWC, which input these signals, operate normally. However, it is considered that faulty operation occurs in the PFC. The signal of the perimeter can be generated, because these signals are generated by the edge signals. However, the signal of the area cannot be generated, because it is generated by the photocurrent of PD. In this case, Iarea is proportional to the area of the background. Therefore, it is necessary to design a network for generating signals of the area of the object, even if the image which turned over is projected on the chip. It is considered that such a network is inserted after the edge detection circuit. Figure 17 shows the network used to gen-
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
426
one target is in the front, only the edges of the front target is output. Thus, motion detection and simple-shape recognition of the target becomes possible by choosing the edges of only one target, even if two or more objects is projected. 6.6 Application
Fig. 17
Network for generating signals of the area.
erate signals of the area from the edge signals. Edge signals are diffused in the directions indicated by the arrows using the smoothing circuit shown in Fig. 5. The current flows into the unit circuit inside the edge. It is possible to generate the signal of the area by digitizing these currents. However, this possibility should be investigated because the scale of the network becomes large. Only a single object was projected onto the chip, and background images, such as trees or buildings, could not be irradiated on the chip in this experiment. The developed chip lacks a function to distinguish an object from the background. The output signal of the approach detection circuit becomes large in proportion to the number of moving edges. In addition, faulty operation occurs in the circuit for simpleshape recognition. Recently, generation of a velocity signal in each pixel and a network that outputs the fastest edge signal was proposed [26]. The operation of the network was confirmed by SPICE. Because the velocity on the edge of an approaching object rapidly becomes large, the velocity of the edge of the target becomes larger than the background. In this condition, because the edge of the background can be removed, the network can output only the edge signal of the target. By using this network for the input part rather that the edge detection circuits, it is possible for each circuit, such as the approach detection circuit, to operate normally even if a moving background is projected onto the chip. When only a single object is projected, movement detection and simple-shape recognition are possible using the proposed circuits. When two or more objects are projected, the approach detection circuit outputs more than two peak currents, and faulty operation occurs in the circuit for simple-shape recognition for that structure. These problems can likely be solved to a certain extent by using a network [26] that outputs the fastest edge signal. For the case of two approaching objects, if one target is large, then the edge velocity increases faster for the large target compared to the small target. Therefore, only the edge of the large target is chosen, and the edge of the small object is removed. Similarly, when the approaching velocity of one target is faster, the edge of the faster target is chosen and that of the slower target is removed, because the edge velocity of the faster target is faster than that of the slow target. For the case in which two objects have the same size and speed, then, if
The fabricated chip is limited by the edge detection circuit with respect to input, as described at section 6.4. The development of a chip that can process real images in various environments by improving the input part of the fabricated chip should be considered. Another application of this chip is for collision avoidance systems for cars or robots, for example. If a problem related to the input part of the fabricated chip can be solved, the object recognition described herein is possible, with the limitations mentioned herein. Since previous chips [2], [14], [15] contain no function for shape recognition, objects cannot be distinguished by the application system of the circuit. Moreover, since the previous circuits [2], [14], [15] can obtain only movement information, it is considered that the application system of this circuit can generate only a signal to flee from an object. Since the proposed circuit is able to recognize objects, the application system of this circuit will be able to perform task such as avoidance and capture. 7.
Conclusion
An integrated circuit of a network for detecting an approaching object with simple-shape recognition was fabricated using the 1.2 µm CMOS process. The proposed network was constructed of simple analog circuits. Measured results for the chip confirmed that the approach direction and velocity of an approaching object can be detected by the peak current generated by the circuit based on the response of DCMD in the locust brain. Moreover, based on the Ewert von Seelen model and the performance function, recognition by the circuit of the simple shapes of moving objects, such as circles, squares, triangles and rectangles, was possible. Acknowledgments This study was supported in part by the Ministry of Education, Science, Sports and Culture through a Grant-inAid for Scientific Research and through the 21st Century COE Program “Intelligent Human Sensing.” Support was also provided by the Toyohashi University of Technology Venture Business Laboratory (TUT-VBL). The VLSI chip in this study has been fabricated in the chip fabrication program of the VLSI Design and Education Center (VDEC), the University of Tokyo in collaboration with OnSemiconductor, Nippon Motorola LTD., HOYA Corporation, and KYOCERA Corporation. References [1] C.A. Mead, Analog VLSI and Neural Systems, Addison Wesley, New York, 1989.
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
427
[2] A. Moini, Vision Chips, Kluwer Academic Publishers, 1999. [3] M. Ohtani, H. Yonezu, and T. Asai, “Analog metal-oxide-silicon IC implementation of motion-detection network based on biological correlation model,” Jpn. J. Appl. Phys., vol.39, pp.1160–1164, 2000. [4] T. Asai, M. Ohtani, and H. Yonezu, “Analog MOS circuits for motion detection based on correlation neural networks,” Jpn. J. Appl. Phys., vol.38, pp.2256–2261, 1999. [5] S.C. Liu and A.U. Viretta, “Fly-like visuomotor responses of a robot using a VLSI motion-sensitive chips,” Biological Cybernetics, vol.85, pp.449–457, 2001. [6] S.C. Liu, “A neuromorphic a VLSI model of global motion processing in the fly,” IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol.47, no.12, pp.1458–1467, 2000. [7] W. Reichardt, Principles of Sensory Communication, Wiley, New York, 1961. [8] N. Hatsopoulos, F. Gabbiani, and G. Laurent, “Elementary computation of object approach by a wide-field visual neuron,” Science, vol.270, pp.1000–1003, 1995. [9] F. Gabbiani, H.G. Krapp, and G. Laurent, “Computation of object approach by a wide-field, motion-sensitive neuron,” J. Neuroscience, vol.19, pp.1122–1141, 1999. [10] F. Gabbiani, C. Mo, and G. Laurent, “Invariance of angular threshold computation in a wide-field looming-sensitive neuron,” J. Neuroscience, vol.21, pp.314–329, 2001. [11] F. Gabbiani, H.G. Krapp, C. Koch, and G. Laurent, “Multiplicative computation in a visual neuron sensitive to looming,” Nature, vol.420, pp.320–324, 2002. [12] K. Nishio, H. Yonezu, M. Ohtani, H. Yamada, and Y. Furukawa, “Analog metal-oxide-semiconductor integrated circuit implementation of approach detection with simple-shape recognition based on visual systems of lower animals,” Optical Review, vol.10, pp.96– 105, 2003. [13] K. Nishio, H. Yonezu, M. Ohtani, H. Yamada, and Y. Furukawa, “Analog integrated circuit for motion detection of approaching object based on insect visual systems,” Optical Review, vol.11, pp.38– 47, 2004. [14] G. Indiveri, “Analog VLSI model of locust DCMD neuron response for computation of object approach,” Eng. Silicon from Neurobiology, vol.10, pp.47–60, 1998. [15] G. Indiveri, J. Kramer, and C. Koch, “System implementations of analog VLSI velocity sensors,” IEEE Micro, vol.16, pp.40–49, 1996. [16] A.P. Ewert, “The neural basis of visually guided behavior,” Scientific American, vol.240, pp.34–42, 1974. [17] D. Ingle, “Disinhibition of tectal neurons by pretectal lesions in the frog,” Science, vol.180, pp.422–424, 1973. [18] M. Brady and A. Yuille, “An extremum principle for shape from contour,” IEEE Trans. Pattern Anal. Mach. Intel., vol.PAMI-6, no.3, pp.288–301, 1984. [19] H. Yamada, T. Miyashita, M. Ohtani, K. Nishio, H. Yonezu, and Y. Furukawa, “Signal formation of image-edge motion based on biological retinal networks and implementation into an analog metaloxide-silicon circuit,” Optical Review, vol.8, pp.336–342, 2001. [20] H. Yamada, T. Miyashita, M. Ohtani, K. Nishio, H. Yonezu, and Y. Furukawa, “An integrated circuit for two-dimensional edgedetection with local adaptation based on retinal networks,” Optical Review, vol.9, pp.1–8, 2002. [21] K. Nishio, M. Ohtani, H. Yamada, Y. Furukawa, H. Yonezu, M. Lee, and J.K. Shin, “An analog MOS circuit for collision avoidance based on a visual system of insects,” Proc. 6th Int. Conf. on Soft Computing and Information, vol.1, pp.718–725, 2000. [22] A.G. Andreou, K.A. Boahen, P.O. Pouliquen, A. Pavasovic, R.E. Jenkins, and K. Strohbehn, “Current-mode subthreshold MOS circuits for analog VLSI neural systems,” IEEE Trans. Neural Netw., vol.2, no.2, pp.205–213, 1991. [23] H. Kolb, “Amacrine cells of the mammalian retina: Neurocircuitry and functional roles,” Eye, vol.11, pp.904–923, 1997.
[24] T. Delbruck and D. Oberhoff, “Self-bias low power adaptive photoreceptor,” Proc. 2004 IEEE Int. Symp. on Circuits and Systems, pp.IV-844–847, 2004. [25] K. Nishio, K. Matsuzaka, and N. Irie, “Analog CMOS circuit implementation of motion detection with wide dynamic range based on vertebrate retina,” Proc. 2004 IEEE Conf. on Cybernetics and Intelligent Systems, 2004. [26] K. Nishio, H. Yonezu, A.B. Kariyawasam, Y. Yoshikawa, S. Sawa, and Y. Furukawa, “Analog integrated circuit for motion detection against moving background based on insect visual systems,” Optical Review, vol.11, pp.24–33, 2004.
Kimihiro Nishio received the B.E., M.E. and Dr.E. degrees in electrical and electronic engineering from Toyohashi University of Technology, Toyohashi, Japan in 1999, 2001 and 2004, respectively. He is now a Research Associate in the Department of Electrical and Computer Engineering, Yonago National College of Technology, Yonago, Japan. His current research interests include hardware implementations of biological neural systems and applications of neuromorphic vision chips.
Hiroo Yonezu received the B.E. degree in electronic engineering from Shizuoka University in 1964 and the Dr.E. degree in the electrical engineering from Osaka University in 1975. In 1964 he joined Nippon Electric Co. Ltd. He made contributions to the research on degradation mechanisms and the improvement of operating life of AlGaAs lasers. Since 1986 he has been a professor with the Department of Electrical and Electronic Engineering, Toyohashi University of Technology. His research has been concerned with basic technologies for future OEICs. He was a councilor of The Japan Society of Applied Physics. He received the SSDM Award from the International Conference on Solid State Devices and Materials in 1995.
Yuzo Furukawa received the B.S., M.S., and Ph.D. degrees in electronic science and engineering from Kyoto University, Kyoto, Japan, in 1995, 1997 and 2000, respectively. In 2000, he joined Toyohashi University of Technology, Toyohashi, Japan, where he is currently a Research Associate in the Department of Electrical and Electronic Engineering. His research interests are hetero-epitaxial growth of semiconductors, optoelectronic devices and analog integrated circuits.
416
PAPER
Special Section on Analog Circuit Techniques and Related Topics
Analog Integrated Circuit for Detection of an Approaching Object with Simple-Shape Recognition Based on Lower Animal Vision Kimihiro NISHIO†,††a) , Hiroo YONEZU† , Members, and Yuzo FURUKAWA† , Nonmember
SUMMARY A network for the detection of an approaching object with simple-shape recognition is proposed based on lower animal vision. The locust can detect an approaching object through a simple process in the descending contralateral movement detector (DCMD) in the locust brain, by which the approach velocity and direction of the object is determined. The frog can recognize simple shapes through a simple process in the tectum and thalamus in the frog brain. The proposed network is constructed of simple analog complementary metal oxide semiconductor (CMOS) circuits. The integrated circuit of the proposed network is fabricated with the 1.2 µm CMOS process. Measured results for the proposed circuit indicate that the approach velocity and direction of an object can be detected by the output current of the analog circuit based on the DCMD response. The shape of moving objects having simple shapes, such as circles, squares, triangles and rectangles, was recognized using the proposed frog-visualsystem-based circuit. key words: analog integrated circuit, edge detection, motion sensor, shape recognition, vision chip
1.
Introduction
Real-time image processing is needed for information systems such as collision avoidance systems and robotics vision systems. However, in typical image-processing systems, which are constructed with a charge coupled device (CCD) camera and a Neumann-type computer, the realization of real-time processing is difficult because information processing is performed in a time-sequential manner. Lower animals have a number of remarkable real-time vision functions, such as motion detection and simple-shape recognition, although their brains are compact. The processing in such cases is performed in massively parallel nerve networks. Integrated circuits (chips) based on lower animal vision can realize real-time image processing and simple structure. Many researchers including C. Mead designed and developed the motion detection circuits based on the algorithms such as the vertebrate retina and insect vision system [1], [2]. Recently, analog motion detection circuits [3]– [6] of a one- or two-dimensional array have been proposed based on the fly visual system [7]. These circuits were able Manuscript received June 18, 2005. Manuscript revised September 27, 2005. Final manuscript received October 20, 2005. † The authors are with the Department of Electrical and Electronic Engineering, Toyohashi University of Technology, Toyohashi-shi, 441-8580 Japan. †† The author is now with the Department of Electrical and Computer Engineering, Yonago National College of Technology, Yonago-shi, 683-8502 Japan. a) E-mail: [email protected] DOI: 10.1093/ietfec/e89–a.2.416
to operate at high speed because the unit circuits performed information processing in parallel, as in the insect vision system. These circuits are characterized by low power consumption because the metal oxide semiconductor (MOS) transistors used to construct such circuits operate in the subthreshold region. Being analog circuits, the unit circuits were constructed with a small number of transistors. Recently, compact target tracking system for robotics was developed by applying such circuits [2], [3]. Thus, analog circuits based on insect vision systems have superior functionality. Therefore, such circuits should be applied to collision avoidance systems for robotics and vehicles, for example. In this case, circuits for three-dimensional motion detection must be developed by applying previously proposed one- or two-dimensional circuits. The detection of an approaching object is the most basic function in three-dimensional motion detection. The signal of the descending contralateral movement detector (DCMD) [8]–[11] in the locust brain shows a peak value just before collision. This signal has been represented by a simple mathematical formula. In previous studies, calculation results have shown that the approach velocity and direction can be detected based on the peak signal [12], [13]. G. Indiveri proposed a simple network and circuits for the detection of an approaching object based on the locust vision system [14]. Since the output signal of the circuit corresponds to that of the DCMD, the circuit could detect the approach of an object when the object approached from the front. In addition, G. Indiveri et al. proposed a circuit for estimating the time to contact [2], [15]. Since the output signal of the circuit is proportional to the time to contact, the circuit could estimate the time to contact when the object approached from the front. Information of the approach direction is needed for the determination of the escape direction for collision avoidance. However, previously proposed circuits [2], [14], [15] did not contain a function to detect the approach direction, and circuits for detection of the approach direction must therefore be developed. When a predator approaches, the frog can escape, and when prey approaches, the frog can jump toward the prey in order to capture it. Since the frog has a function for discriminating predator from prey, the vision system of the frog differs from that of the locust, for example, which can only escape from an approaching object, and has the new function like capture. This is because the frog has a func-
c 2006 The Institute of Electronics, Information and Communication Engineers Copyright
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
417
tion for simple-shape recognition. The response for simpleshape recognition is observed in the tectum and thalamus in the large-field cells of the frog brain [16], [17]. P. Ewert proposed a model for simple-shape recognition. The shape recognition function of the cell is limited to objects that are large in the lateral direction and objects that are long at the side except that the function for the movement detection of an approaching target into the DCMD. Even if the complex shape is not recognized, predator, prey or other objects can be distinguished by combining simple-shape information with movement information because the movement speed and pattern are limited to a certain extent. Because a pattern is decided as the on the car and walker, it can think that it becomes possible that they are distinguished in the same manner. Thus, shape recognition is required in order to discriminate objects. The only function of the previously proposed circuit [2], [14], [15] is motion detection. Object discrimination is possible if a simple-shape recognition function can be added to the function of the previously proposed circuit. The development of such a circuit is expected to have greater applicability than the previously proposed circuit. Therefore, in the present study, an analog integrated circuit for the detection of an approaching object with simple-shape recognition that is based the lower animal vision is fabricated with the 1.2 µm complementary metal oxide semiconductor (CMOS) process. Measured results indicate that the fabricated chip operates normally. 2.
Biological Model
2.1 Descending Contralateral Movement Detector Figure 1(a) shows the relationship between an object approaching from the right front with constant velocity v and the locust eye. The object is square and has a length of w x (= wy ). The approach direction is θdir,x . The difference between θleft and θright is θsize,x . Figure 1(b) shows the approaching object from the upper front, where θdir,y is the approach direction and θsize,y is the difference between θtop and θbottom . As the object approaches, the size of the image projected on the retina increases rapidly just before collision. Figure 1(c) shows the projected image before collision. The width wpx corresponding to θsize,x and the expansion velocity corresponding to the image angular velocities (θ˙left and θ˙right ) increase just before collision. When the object approaches from the right front, |θ˙left | is larger than |θ˙right |. Conversely, |θ˙left | is smaller than |θ˙right | when the object approaches from the left front. A locust can detect an approaching object by the signal generated by the descending contralateral movement detector (DCMD) in the locust brains. The existence of two types of DCMDs has been reported [10]. One DCMD generates the signal fleft (t), and the other generates fright (t), where fleft (t) and fright (t) are given by following equations [12], [13]:
Fig. 1 Approach detection model. (a) Object approaches from the right front. (b) Object approaches from the upper front. (c) Image projected on the retina before collision. (d) Output signals of DCMDs.
fleft (t) = θ˙left (t) exp(−θsize,x (t)), fright (t) = θ˙right (t) exp(−θsize,x (t)).
(1) (2)
The signals of the DCMDs under the condition of Fig. 1(a) are shown in Fig. 1(d). The signals are first increased by the increase in the angular velocity and are later decreased according to exp(−θsize,x ) [8]–[11]. Therefore, fleft (t) and fright (t) show the peak values just before collision. When the object approaches from the right front, fleft (t) is larger than fright (t). The difference between fleft (t) and fright (t) is fdir,x (t), which is given as follows: fdir,x (t) = fleft (t) − fright (t).
(3)
The absolute value of the peak signal fdir,xp of fdir,x (t) is proportional to that of θdir,x [12], [13]. The sum of fleft (t) and fright (t) is fvel (t), which is given as follows: fvel (t) = fleft (t) + fright (t).
(4)
The peak value fvel,p of fvel (t) is proportional to v [12], [13]. In order to avoid a collision, the locust must escape quickly when the object is large. The peak response was
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
418
found to be generated when the image angle reached a constant threshold angle, which is independent of the approach velocity and the size of the object. Therefore, the time between the occurrence of fvel,p and collision should be proportional to w x [13]. Thus, the collision can be avoided with this model. The approach direction θdir,y in Fig. 1(b) can be detected by applying Eq. (3) to the y-direction. The signal fdir,y (t) for detecting θdir,y is given as fdir,y (t) = fbottom (t) − ftop (t), where ftop (t) and fbottom (t) are given as ftop (t) = θ˙top (t) exp(−θsize,y (t)), fbottom (t) = θ˙bottom (t) exp(−θsize,y (t)),
Iarea , 2 Iperi
(9)
(6) (7)
3.
2.2 Ewert von Seelen Model The Ewert von Seelen model [16] for recognizing simple shapes, which is based on the frog visual system, is shown in Fig. 2. Edge signals generated by the retina are transmitted to the tectum (type I) and the thalamus. The signal Isize,x transmitted to the tectum becomes large when a laterally extending object is projected onto the retina, and the signal Isize,y transmitted to the thalamus becomes large when a vertically extending object is projected on the retina. The excitatory signal Isize,x and the inhibitory signal Isize,y are transmitted to the tectum (type II). The output signal IE of the tectum (type II) becomes constant Iconst when the square object is projected on the retina. The output signal IE is larger than Iconst when a laterally extending object is projected. The output signal IE is smaller than Iconst when a vertically extending object is projected. Thus, IE is given as Isize,x Iconst . Isize,y
IP =
where Iarea and Iperi are the signals of the area and the perimeter of the projected image, respectively, is inserted into the model [18]. As a result, the model can discriminate simple shapes, such as circles, squares and triangles, from the value of IP .
(5)
where θ˙top and θ˙bottom are angular velocities corresponding to the velocities of the top edge and bottom edge, respectively [12]. The approach direction can be detected using fdir,x (t) in Eq. (3) and fdir,y (t) in Eq. (5).
IE =
and θsize,y shown in Figs. 1(a) and (b), respectively. It is impossible to discriminate shape, for example, circle or square, in the model shown in Fig. 2 because both output signals are identical. Thus, the performance function IP , which is given by
(8)
Network for Generating Signals of Angle, Angular Velocity, Area and Perimeter
3.1 Network Construction Generation of the signals indicating the angle, angular velocity, area and perimeter is required for fabricating a chip for approach detection with simple-shape recognition. A network for generating these signals is shown in Fig. 3. The network is constructed using a photodiode (PD), an edge detection circuit (EDC), size detection circuits (SDs) and velocity sensing circuits (VSCs). While the image is projected onto the PD, the photocurrent Ipi, j increases. The photocurrent Ipi, j is digitized by the area detection circuits described in the next section. The output current of the area detection circuit is Iai, j , which becomes constant at the position projecting the image. At all other positions, Iai, j is 0. The signal Iarea , which is proportional to the area, can then be generated as follows: Iai, j . (10) Iarea = On the other hand, Ipi, j is also input to the EDC. The output current Iei, j is the constant current at the edge position. The signal Iperi , which is proportional to the perimeter,
The signals Isize,x and Isize,y are proportional to angles θsize,x
Fig. 2
Model for recognizing simple shapes.
Fig. 3 Network for generating the signals of the angle, angular velocity, area and perimeter.
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
419
can be generated by the EDC and is given as follows: Iperi = Iei, j . (11)
at node a, the Iarea becomes the sum of all Ia,i . The circuit shown in Fig. 4(b) is also used for the realization of Eq. (11).
By summing Iei, j for each row and column, Iex,i and Iey, j are output, respectively. The currents Iex,i and Iey, j are input to the SDs. When currents Iex,i and Iey, j are larger than the threshold current, Idx,i and Idy, j become constant, and when Iex,i and Iey, j are smaller than the threshold current Idx,i and Idy, j are 0. The sum of each Idx,i is Isize,x , which is proportional to θsize,x , and the sum of each Idy, j is Isize,y , which is proportional to θsize,y . The output currents of the VSCs are Iv,right and Iv,left when the edge moves toward the right and left sides, respectively. Thus, Iv,right and Iv,left correspond to angular velocities. Similarly, Iv,top and Iv,bottom also correspond to angular velocities.
3.3 Edge Detection Circuit
3.2 Area Detection Circuit Figure 4(a) shows the unit circuit for area detection. The photocurrent Ip,i , which is proportional to the light intensity, is input. The threshold current Ith is set by providing threshold voltage Vth . When Ip,i is larger than Ith , voltage Va1 is the supply voltage VDD . Then, output current Ia,i is the constant current Ic0 which is set by providing the constant voltage Vc0 . When Ip,i is smaller than Ith , Ia,i is 0 because Va1 is 0. The output current Ia,i becomes the constant current Ic0 when the bright image is projected. Figure 4(b) shows the adder circuit used to generate Iarea in Eq. (10). Each Ia,i generated by the area detection circuit is input to the pMOS current mirror circuit. Since all of the output terminals of the mirror circuits are connected
Fig. 4
Area detection circuit. (a) Unit circuit. (b) Adder circuit.
Figure 5 shows the unit circuit for edge detection based on the vertebrate retina [19], [20]. The photocurrent Ip,i is input to the smoothing circuit (Mh1 and Mh2 ). The current Ih,i is generated by diffusing Ip,i into part of a signal to neighboring circuits. The current Ib,i is generated by subtracting Ih,i from Ip,i . The absolute value of Ib,i at the edge position is larger than at other positions. The current Ib,i is input to the digitizing circuit (DG). The current Ie,i becomes the constant current Ic0 at the edge position, because the voltage Vdg is the supply voltage when Ib,i is larger than the threshold current Ith [12], [21], and becomes 0 at other positions be cause Vdg is 0. The DG in Fig. 4 is also applied to the SD in Fig. 3. 3.4 Velocity Sensing Circuit Figure 6(a) shows a velocity sensing model called the correlation model [7]. The unit cell (Elemental motion detector; EMD) is constructed with the unit neuron of the retina R, delay neuron D and correlator C. Figure 6(b) shows the output signal from each cell when the spot object moves to the right side. The unit cell of the retina R1 first generates the signal IR1 . The delay neuron D1 then indicates the maximum value. When the spot is projected on a position between R1 and R2 , the signal IR1 is not generated and ID is decreased. After time tv , which is inversely proportional to velocity ve , the spot moves to R2 and IR2 is generated. The signals ID and IR2 are input to the correlator C2 , which then generates signal IC . Since the maximum value IC,peak of IC depends on ID , IC,peak is inversely proportional to tv . Therefore, IC,peak is proportional
Fig. 5
Unit circuit for edge detection.
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
420
Fig. 6 Velocity sensing model and unit circuit. (a) Model based on the visual system of the fly. (b) Output signal from each cell. (c) Unit circuit.
to ve . However, IC,peak is not proportional to ve when the wide object shown in Fig. 6(a) is projected since an edge detector such as EDC is used. The EDC outputs the signal IR1 even if the edge of the wide object is projected on the position between R1 and R2 , as shown in Fig. 6(b). Since IC,peak shows a constant maximum value when tv = 0, IC,peak is independent of velocity ve . When the edge of a wide object moves toward the right side, the EMD outputs the positive constant pulsed signal Im,i , and when the edge moves toward the left side, the EMD outputs the negative constant pulsed signal Im,i . The sum of positive Im,i is Iv,right , and the sum of the signals, which is given by rectifying the negative Im,i , is Iv,left . The signal Iv,right when the edge moves toward the right side is shown in Fig. 6(a). When the edge velocity is fast, the interval of pulse td is short. Thus, the velocity can be detected by detecting td . The velocity sensing circuit (VSC) based on the EMD is shown in Fig. 6(c) [3], [4]. A number of circuits based on the EMD have been proposed [3]–[6]. The circuit in Fig. 6(c) is considered to be the simplest structure among the proposed circuits, i.e., the number of MOS transistors
is low. Therefore, this circuit is added to the proposed network. The current Idx,i is an input current. The circuit constructed with capacitor CD and nMOS transistor MD corresponds to the delay neuron D, and the nMOS transistor MC corresponds to the correlator C. The current IC,i is rectified by the pMOS current mirror circuit (M1 and M2 ), and the output current is Im,i . When the edge moves toward the right side, Im,i becomes IC,i because IC,i−1 is 0. When the edge moves to the left side, Im,i represents IC,i−1 . 4.
Analog Circuits for Approach Detection and SimpleShape Recognition
4.1 Approach Detection Circuit Based on DCMD Figure 7(a) shows the approach detection circuit based on the DCMD response. The circuit inputs currents Isize,x , Iv,right and Iv,left , as shown in Fig. 3, which are converted to voltages Vsize,x , Vv,right and Vv,left , respectively. The voltage Vsize,x increases just before collision because Isize,x increases. The voltages Vv,right and Vv,left also increase just before col-
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
421
lision because td in Fig. 6(a) becomes short. Since Vv,right and Vv,left depend on the velocities of the edges, Vv,right and Vv,left correspond to the angular velocities |θ˙left | and |θ˙right | in Eqs. (1) and (2), respectively. Since Vsize,x is the gate voltage of pMOS transistor Ma3 , Iexp decreases just before collision. The Ma3 needs to operate in the subthreshold region. Therefore, Iexp is approximately equivalent to exp(−θsize,x ) in Eqs. (1) and (2). The output current Iright is first increased by Vv,right and then decreased by Iexp , because Iright is equal to Iexp when the voltage Va is large and Iexp is small. Thus, Iright corresponds to fright (t) in Eq. (2) because Iright indicates the peak value just before collision. Similarly, Ileft is first increased by Vv,left and then decreased by Iexp because Ileft is equal to Iexp . Thus, Ileft corresponds to fleft (t) in Eq. (1) because Ileft shows a peak value just before collision. The circuit shown in Fig. 7(a) is used to realize Eqs. (6) and (7). The currents Isize,y , Iv,top and Iv,bottom generated by the network shown in Fig. 3 are input to this circuit, and the resulting output currents are Itop and Ibottom . Since the required processing is equal to that for the circuit shown in Fig. 7(a), Itop and Ibottom correspond to ftop (t) and fbottom (t) in Eqs. (6) and (7), respectively. Figure 7(b) shows the circuit for generating the signals of the approach velocity and direction. The currents Ileft and Iright are input to the pMOS and nMOS current mirror circuits, respectively. Here, Iright is negative because it is input to the nMOS current mirror circuit. The output current Idir,x is obtained by subtracting Iright from Ileft . Therefore, Idir,x corresponds to fdir,x in Eq. (3). Since each output terminal of the pMOS current mirror circuit is connected, the output current Ivel is represented by the sum of Ileft and Iright . Thus, Ivel corresponds to fvel (t) in Eq. (4).
Fig. 7 Approach detection circuit based on DCMD response. (a) Circuit based on Eqs. (1) and (2). (b) Circuit based on Eqs. (3) and (4).
The circuit in Fig. 7(b) is used to realize Eq. (5). Here, Itop and Ibottom are input to the circuit, and the output current is Idir,y , which corresponds to fdir,y (t) in Eq. (5). 4.2 Circuit for Simple-Shape Recognition Realization of Eqs. (8) and (9) requires a circuit that can realize both a division circuit and a square circuit, as shown in Fig. 8 [22]. When all MOS transistors operate in the subthreshold region, the output current Ijout is given as Ijout
Ia Ib . Ic
(12)
The circuit is used as a division circuit by setting Ia to a constant current Iconst and as a square circuit by setting Ia = Ib and Ic = Iconst . The circuit shown in Fig. 8 is based on the Ewert von Seelen model, (EWC) in Eq. (8), by setting Ia = Iconst , Ib = Isize,x , Ic = Isize,y and Ijout = IE . The division and square circuits of Fig. 8 are based on the performance function (PFC) shown in Eq. (9). One circuit is used as a square 2 , where Ijout is made proportional circuit for generating Iperi 2 to Iperi by setting Ia = Ib = Iperi and Ic = Iconst , and the other circuit is used as a division circuit. Ijout becomes IP in Eq. (9) 2 . by setting Ia = Iconst , Ib = Iarea and Ic = Iperi 5.
Experimental Results
5.1 Chip Structure and Experimental Condition A chip for approach detection with simple-shape recognition was fabricated using the double-metal double-poly standard 1.2 µm CMOS process. A photograph of the chip is shown in Fig. 9. Two-dimensional EDCs (15 × 15) having an area of 4.8 × 4.8 mm2 were designed. The size of the PD was 50 × 50 µm2 . The total size of the EDC and the area detection circuit was 180 × 180 µm2 . The fill factor was approximately 10%. Fifteen SDs and VSCs were arranged in the x- and y-directions. The capacitance of CD was fabricated to be 0.5 pF. The capacitance of CD decides the pulsed width td in Fig. 5(a). The width must be set smaller than 1 ms in order to realize high-speed processing. The results obtained with the simulation program with integrated circuit emphasis (SPICE) indicate that the pulsed width is approximately 1 ms when
Fig. 8
Division and square circuit for recognizing simple shapes.
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
422
CD is 0.5 pF. Therefore, the capacitance of CD of the test chip was also fabricated to be 0.5 pF. The capacitance of Ca was fabricated to be 50 pF. When Ca is too small, Vv,right does not increase because the discharge from Ma2 in Fig. 6(a) occurs quickly. Thus, the approach detection circuit cannot output the peak current. When Ca is too large, the area becomes large, and so Ca has to be set to a minimal value so that Iright indicates the peak value. The results obtained with SPICE reveal that the approach detection circuit outputs the peak current when Ca is 50 pF. However, the peak current was not obtained when the capacitance of Ca was set to less than 45 pF. Therefore, the capacitance of Ca of the test chip was also fabricated to be 50 pF. Figure 10 shows a schematic diagram of the measurement equipment. A personal computer (PC) is connected to a projector, from which the input image generated by the PC was projected. The image was projected onto the chip through a lens. The photocurrent was 80 nA and 100 pA, respectively, for white and black images projected on the 50 × 50 µm2 PD. The illumination was about 500 W/m2 and 0.6 W/m2 when the white and black image was projected respectively. The supply voltage was set to 3 V. Vdif and Ith1 in Fig. 4 were set to 1.80 V and 5 nA, respectively. The bias conditions of Vdif in Fig. 5 and Ith in Fig. 4 depend on the illumination conditions. The setup of Vdif is important to generate the diffusing currents by Mh1 and Mh2 of the EDC. The diffusing current is decided by Vdif and Ip,i . When the diffusing current is too large or too small, the edge signal cannot be generated [20]. We confirmed by SPICE that the correct value of Vdif is in the range from 1.75 V to 1.85 V. Therefore,
Fig. 9
Fig. 10
Vdif was set to 1.80 V. In this experiment, Ith generated by Vth has to be set to a value between 100 pA and 80 nA because the photocurrent is in the range between 100 pA and 80 nA. The Vgs − Ids characteristic of the nMOS transistor was obtained by measurement, and Ith was set to 30 nA by setting Vth based on the obtained data. Other bias conditions are independent of the illumination conditions because the edge signals are digitized. Therefore, the correct bias conditions were decided by measuring the test circuits. As a result, Ic0 in Figs. 4(a) and 5 was set to 10 nA and Vbias in Fig. 7(a) was set to 1 V. 5.2 Velocity Sensing Circuit Figure 11(a) shows the measured result under the condition in which the right edge of the rectangle image moved onto the chip for a period of 80 ms. The circuit output the constant pulsed current Iv,right . Thus, Iv,left , Iv,top and Iv,bottom were 0. Figure 11(b) shows the result when the right edge moved onto the chip for a period of 1.5 s. The circuit output
Photograph of the chip.
Schematic diagram of the measurement equipment.
Fig. 11 Transient response of the VSCs. (a) Output current Iv,right under the condition that the right edge moves toward the right side of the chip for 80 ms. (b) Output current Iv,right under the condition that the right edge moves toward the right side of the chip for 1.5 s. (c) Output current Iv,left .
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
423
the constant pulsed current Iv,right . The values of the currents in Fig. 11(a) were equal to those in Fig. 11(b). The interval of pulse td in Fig. 11(b) was longer than that in Fig. 11(a). The velocity was clarified to be detectable by the detection of td . The circuit was measured under the condition that the left edge of the rectangular image moved toward the left side on the chip for a period of 1.5 s. Figure 11(c) shows the output current Iv,left , which was a constant pulsed current. Thus, Iv,right , Iv,top and Iv,bottom were 0. The results shown in Figs. 11(b) and (c) clarified that the moving direction can be detected. Furthermore, the circuits were shown to output constant currents Iv,top and Iv,bottom when the edge moves toward the upper and lower sides, respectively.
Fig. 12 Transient responses of the approach detection circuit. (a) Images projected on the chip. (b) Output current Ivel . (c) Output current Idir,x . (d) Output current Idir,y .
5.3 Approach Detection Circuit Approach detection circuits were measured under the condition in which a square object approaching from the upper right front was projected onto the chip. The values of v, θdir,x (= θdir,y ) and w x (= wy ) were set to 30 m/s, 30◦ and 0.4 m, respectively. Figure 12(a) shows the relationship between the projected image and the chip. When the entire image is projected onto the chip, as shown in right-most photograph of Fig. 12(a), the time to collision tcoll is 0. Figures 12(b), (c) and (d) show output currents Ivel , Idir,x and Idir,y , respectively. The output currents Ivel , Idir,x and Idir,y showed peak values of Ivel,p , Idir,xp and Idir,yp , respectively, just before collision. In addition, the relationship between Ivel,p and v was investigated. Figure 13(a) shows the measured results and the simulation results obtained using SPICE. The Ivel,p was approximately proportional to v, indicating that Ivel corresponds to fvel (t). The measured results and the simulation results showed good agreement. Thus, it was clarified that v can be detected using Ivel,p . We also investigated the dependence of Idir,xp on θdir,x . Figure 13(b) shows the measured results along with the simulation results. The absolute value of Idir,xp was approximately proportional to that of θdir,x , indicating that Idir corresponds to fdir,x (t). The measured results corresponded approximately to the simulation results. The absolute value of Idir,yp was proportional to that of θdir,y , indicating that θdir,y can be detected from Idir,yp . Thus, it was clarified that the
Fig. 13 Characteristics of the approach detection circuit. (a) Dependence of the peak current Ivel,p on velocity v. (b) Dependence of the peak current Idir,xp on direction θdir,x .
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
424
direction can be detected using Idir,xp and Idir,yp . 5.4 Circuit for Simple-Shape Recognition Figure 14(a) shows the measured results of the EWC and each image projected onto the chip. Each image was moved in order to confirm that the circuits could recognize the simple shape of the moving image. Each image approached from the front. The images were projected on the entire area of the chip at 600 ms. In Fig. 8, Ia was set to 10 nA (= Iconst ). When the square image approached the chip, the output current IE was 10 nA (= Iconst ). When a rectangular image having a lateral length that was twice as long as the vertical length, approached the chip, IE was 20 nA (= 2Iconst ). When a rectangular image having a vertical length that was twice as long as the lateral length, moved onto the chip, IE was 5 nA (= 0.5Iconst ). Figure 14(b) shows the measured results of PFC and each image projected onto the chip. Each image approached from the front. When circular, square and triangular images
were projected on the chip, IP became approximately 80 nA, 50 nA and 30 nA, respectively. Figure 15 shows the output current IP of PFC and the calculated results of the performance function. The horizontal axis of the graph indicates the shape S. The gradient of the current IP was approximately equal to that of the calculation results. The output current of PFC thus corresponds to the performance function. Thus, the results shown in Figs. 14 and 15 clarify that simple-shape recognition based on IE and IP is possible, even if the size of the object changes upon approach. 6.
Discussion
6.1 Comparison with Previously Proposed Circuits The test chip for the detection of an approaching object with simple-shape recognition was fabricated using the 1.2 µm CMOS process. It was clarified that, using the fabricated chip, the moving direction and velocity can be detected and, simultaneously, the simple shape can be recognized. New functions for the detection of the approach direction and recognition of the simple shape, which were not realized in previous circuits [2], [14], [15], were incorporated in the newly fabricated chip. 6.2 Device Mismatch The test chip was constructed using simple analog circuits. Although analog circuits have a problem with respect to incorrect operation caused by device mismatches, the fabricated chip operated correctly. It should be considered that the digitizing circuit, which can output a constant current even if the currents vary as a result of device mismatches, was inserted in EDCs and SDs. The pulsed currents shown in Fig. 11 indicate different values due to device mismatches. This problem can be solved to some extent by using DG. Pulsed currents of constant values can be output by inputting Iv,right and Iv,left to DG. Although pulsed currents containing different values,
Fig. 14 Transient responses of the circuit for recognizing simple shapes. (a) Output current IE . (b) Output current IP .
Fig. 15 sults.
Output current of PFC and calculated performance function re-
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
425
as shown in Fig. 11, were input to the approach detection circuits, the circuits operated normally, as shown in Figs. 12 and 13. In biological systems, the output signals In of multiple nerve cells, which contain variation, are assembled [1], where subscript n is the number of cells. The average value of the signals is the nearest correct signal Icor . The CR integrator constructed with Ca and Ma2 shown in Fig. 7(a) has the function for assembling and averaging the signals. Therefore, the approach detection circuits operated normally. 6.3 Output Signals of Proposed Network Currents Ivel , Idir,x and Idir,y showed peak values just before collision, as shown in Fig. 12. Although the output signal in Fig. 1(d) includes only one peak, several peak values appear in Fig. 12. The reason was that Ma2 was included as a discharge pathway in Fig. 7(a). We defined the maximum value in Fig. 12 as the peak value in Fig. 1(d). Note that the values for Ivel,p and Idir,xp in the measurement were larger than those in the simulation. This discrepancy is due to the difference of the measurement and model parameters between SPICE and the fabricated MOS transistors. The test chip was able to recognize the simple shape of a moving object by the EWC and the PFC, as shown in Figs. 13, 14 and 15. However, it is difficult to recognize a simple shape by the use of only PFC. That example is described in the following. The value of IP of the PFC when the ellipse was projected is equal to that when the square was projected. Figure 16 shows the calculation results of the performance function. The solid line shows the performance function when w x /wy = 1, and the dashed line shows the performance function when w x /wy = 3. The value of the ellipse is approximately equal to that of the square, and the value of the rectangle is equal to that of the triangle. It is necessary to discriminate the ellipse and the square, and the method used for discrimination is described in the following. The solid line should be used when the output current IE of EWC is 10 nA, i.e. w x /wy = 1. Then, the shape is square when the output current IP of the PFC is 60 nA, i.e. the performance function is 0.06. The dashed line is used when IE is 30 nA,
Fig. 16
Calculated results of the performance function.
i.e. w x /wy = 3. Then, the shape is an ellipse if IP is 60 nA. Thus, the chip is able to recognize various simple shapes by using the signals of the EWC and the PFC. 6.4 Input Part Motion detection and shape recognition are difficult when a real image was projected on the fabricated chip since the test chip contained only a 15 × 15 EDC. As a result, a white image and a black image, representing an object and the background, respectively, were projected onto the chip. Simulation results obtained using SPICE confirmed that unit circuits of at least 50 × 50 are needed in order to obtain the edge signals of the real image [20]. In the present study, large transistors that are twice as large as the minimum line width were used for the test chip. If a chip is fabricated to have the minimum line width, simple calculations indicate that an edge detection circuit of approximately 30 × 30 can be designed in an area of 4.8 × 4.8 mm2 . In addition, more unit circuits for edge detection can be designed if the chip is designed using a recent process rule. If such a circuit is realized, then motion detection and shape recognition become possible even if the real image is projected. However, even if 50 × 50 or larger edge detection circuits are designed, the proposed chip could only be used in a fixed indoor environment, because the edge detection circuit of the chip has a dynamic range of only approximately 2.5 decades [20], whereas the dynamic range of the vertebrate retina is over six decades [23]. Several attempts have been made to a develop chip that has a wide dynamic range [24]. We also tried to improve the EDC by mimicking the signal processing of rods and cones of the photoreceptor [25]. The realization of a wide dynamic range of more than six decades is expected by mimicking the function for amplifying the photocurrent under low light intensity based on the rod of the photoreceptor. 6.5 Motion Detection in Real Image In this experiment, the background (black image) and an object (white image) were projected onto the chip. It tries to think about the case that an object of a black image and a background of a white image are projected on the chip. In this case, the signals of the width and the expansion velocity can be generated, because these signals are generated by the edge signals. The approach detection circuit and the EWC, which input these signals, operate normally. However, it is considered that faulty operation occurs in the PFC. The signal of the perimeter can be generated, because these signals are generated by the edge signals. However, the signal of the area cannot be generated, because it is generated by the photocurrent of PD. In this case, Iarea is proportional to the area of the background. Therefore, it is necessary to design a network for generating signals of the area of the object, even if the image which turned over is projected on the chip. It is considered that such a network is inserted after the edge detection circuit. Figure 17 shows the network used to gen-
IEICE TRANS. FUNDAMENTALS, VOL.E89–A, NO.2 FEBRUARY 2006
426
one target is in the front, only the edges of the front target is output. Thus, motion detection and simple-shape recognition of the target becomes possible by choosing the edges of only one target, even if two or more objects is projected. 6.6 Application
Fig. 17
Network for generating signals of the area.
erate signals of the area from the edge signals. Edge signals are diffused in the directions indicated by the arrows using the smoothing circuit shown in Fig. 5. The current flows into the unit circuit inside the edge. It is possible to generate the signal of the area by digitizing these currents. However, this possibility should be investigated because the scale of the network becomes large. Only a single object was projected onto the chip, and background images, such as trees or buildings, could not be irradiated on the chip in this experiment. The developed chip lacks a function to distinguish an object from the background. The output signal of the approach detection circuit becomes large in proportion to the number of moving edges. In addition, faulty operation occurs in the circuit for simpleshape recognition. Recently, generation of a velocity signal in each pixel and a network that outputs the fastest edge signal was proposed [26]. The operation of the network was confirmed by SPICE. Because the velocity on the edge of an approaching object rapidly becomes large, the velocity of the edge of the target becomes larger than the background. In this condition, because the edge of the background can be removed, the network can output only the edge signal of the target. By using this network for the input part rather that the edge detection circuits, it is possible for each circuit, such as the approach detection circuit, to operate normally even if a moving background is projected onto the chip. When only a single object is projected, movement detection and simple-shape recognition are possible using the proposed circuits. When two or more objects are projected, the approach detection circuit outputs more than two peak currents, and faulty operation occurs in the circuit for simple-shape recognition for that structure. These problems can likely be solved to a certain extent by using a network [26] that outputs the fastest edge signal. For the case of two approaching objects, if one target is large, then the edge velocity increases faster for the large target compared to the small target. Therefore, only the edge of the large target is chosen, and the edge of the small object is removed. Similarly, when the approaching velocity of one target is faster, the edge of the faster target is chosen and that of the slower target is removed, because the edge velocity of the faster target is faster than that of the slow target. For the case in which two objects have the same size and speed, then, if
The fabricated chip is limited by the edge detection circuit with respect to input, as described at section 6.4. The development of a chip that can process real images in various environments by improving the input part of the fabricated chip should be considered. Another application of this chip is for collision avoidance systems for cars or robots, for example. If a problem related to the input part of the fabricated chip can be solved, the object recognition described herein is possible, with the limitations mentioned herein. Since previous chips [2], [14], [15] contain no function for shape recognition, objects cannot be distinguished by the application system of the circuit. Moreover, since the previous circuits [2], [14], [15] can obtain only movement information, it is considered that the application system of this circuit can generate only a signal to flee from an object. Since the proposed circuit is able to recognize objects, the application system of this circuit will be able to perform task such as avoidance and capture. 7.
Conclusion
An integrated circuit of a network for detecting an approaching object with simple-shape recognition was fabricated using the 1.2 µm CMOS process. The proposed network was constructed of simple analog circuits. Measured results for the chip confirmed that the approach direction and velocity of an approaching object can be detected by the peak current generated by the circuit based on the response of DCMD in the locust brain. Moreover, based on the Ewert von Seelen model and the performance function, recognition by the circuit of the simple shapes of moving objects, such as circles, squares, triangles and rectangles, was possible. Acknowledgments This study was supported in part by the Ministry of Education, Science, Sports and Culture through a Grant-inAid for Scientific Research and through the 21st Century COE Program “Intelligent Human Sensing.” Support was also provided by the Toyohashi University of Technology Venture Business Laboratory (TUT-VBL). The VLSI chip in this study has been fabricated in the chip fabrication program of the VLSI Design and Education Center (VDEC), the University of Tokyo in collaboration with OnSemiconductor, Nippon Motorola LTD., HOYA Corporation, and KYOCERA Corporation. References [1] C.A. Mead, Analog VLSI and Neural Systems, Addison Wesley, New York, 1989.
NISHIO et al.: ANALOG INTEGRATED CIRCUIT FOR DETECTION OF AN APPROACHING OBJECT
427
[2] A. Moini, Vision Chips, Kluwer Academic Publishers, 1999. [3] M. Ohtani, H. Yonezu, and T. Asai, “Analog metal-oxide-silicon IC implementation of motion-detection network based on biological correlation model,” Jpn. J. Appl. Phys., vol.39, pp.1160–1164, 2000. [4] T. Asai, M. Ohtani, and H. Yonezu, “Analog MOS circuits for motion detection based on correlation neural networks,” Jpn. J. Appl. Phys., vol.38, pp.2256–2261, 1999. [5] S.C. Liu and A.U. Viretta, “Fly-like visuomotor responses of a robot using a VLSI motion-sensitive chips,” Biological Cybernetics, vol.85, pp.449–457, 2001. [6] S.C. Liu, “A neuromorphic a VLSI model of global motion processing in the fly,” IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol.47, no.12, pp.1458–1467, 2000. [7] W. Reichardt, Principles of Sensory Communication, Wiley, New York, 1961. [8] N. Hatsopoulos, F. Gabbiani, and G. Laurent, “Elementary computation of object approach by a wide-field visual neuron,” Science, vol.270, pp.1000–1003, 1995. [9] F. Gabbiani, H.G. Krapp, and G. Laurent, “Computation of object approach by a wide-field, motion-sensitive neuron,” J. Neuroscience, vol.19, pp.1122–1141, 1999. [10] F. Gabbiani, C. Mo, and G. Laurent, “Invariance of angular threshold computation in a wide-field looming-sensitive neuron,” J. Neuroscience, vol.21, pp.314–329, 2001. [11] F. Gabbiani, H.G. Krapp, C. Koch, and G. Laurent, “Multiplicative computation in a visual neuron sensitive to looming,” Nature, vol.420, pp.320–324, 2002. [12] K. Nishio, H. Yonezu, M. Ohtani, H. Yamada, and Y. Furukawa, “Analog metal-oxide-semiconductor integrated circuit implementation of approach detection with simple-shape recognition based on visual systems of lower animals,” Optical Review, vol.10, pp.96– 105, 2003. [13] K. Nishio, H. Yonezu, M. Ohtani, H. Yamada, and Y. Furukawa, “Analog integrated circuit for motion detection of approaching object based on insect visual systems,” Optical Review, vol.11, pp.38– 47, 2004. [14] G. Indiveri, “Analog VLSI model of locust DCMD neuron response for computation of object approach,” Eng. Silicon from Neurobiology, vol.10, pp.47–60, 1998. [15] G. Indiveri, J. Kramer, and C. Koch, “System implementations of analog VLSI velocity sensors,” IEEE Micro, vol.16, pp.40–49, 1996. [16] A.P. Ewert, “The neural basis of visually guided behavior,” Scientific American, vol.240, pp.34–42, 1974. [17] D. Ingle, “Disinhibition of tectal neurons by pretectal lesions in the frog,” Science, vol.180, pp.422–424, 1973. [18] M. Brady and A. Yuille, “An extremum principle for shape from contour,” IEEE Trans. Pattern Anal. Mach. Intel., vol.PAMI-6, no.3, pp.288–301, 1984. [19] H. Yamada, T. Miyashita, M. Ohtani, K. Nishio, H. Yonezu, and Y. Furukawa, “Signal formation of image-edge motion based on biological retinal networks and implementation into an analog metaloxide-silicon circuit,” Optical Review, vol.8, pp.336–342, 2001. [20] H. Yamada, T. Miyashita, M. Ohtani, K. Nishio, H. Yonezu, and Y. Furukawa, “An integrated circuit for two-dimensional edgedetection with local adaptation based on retinal networks,” Optical Review, vol.9, pp.1–8, 2002. [21] K. Nishio, M. Ohtani, H. Yamada, Y. Furukawa, H. Yonezu, M. Lee, and J.K. Shin, “An analog MOS circuit for collision avoidance based on a visual system of insects,” Proc. 6th Int. Conf. on Soft Computing and Information, vol.1, pp.718–725, 2000. [22] A.G. Andreou, K.A. Boahen, P.O. Pouliquen, A. Pavasovic, R.E. Jenkins, and K. Strohbehn, “Current-mode subthreshold MOS circuits for analog VLSI neural systems,” IEEE Trans. Neural Netw., vol.2, no.2, pp.205–213, 1991. [23] H. Kolb, “Amacrine cells of the mammalian retina: Neurocircuitry and functional roles,” Eye, vol.11, pp.904–923, 1997.
[24] T. Delbruck and D. Oberhoff, “Self-bias low power adaptive photoreceptor,” Proc. 2004 IEEE Int. Symp. on Circuits and Systems, pp.IV-844–847, 2004. [25] K. Nishio, K. Matsuzaka, and N. Irie, “Analog CMOS circuit implementation of motion detection with wide dynamic range based on vertebrate retina,” Proc. 2004 IEEE Conf. on Cybernetics and Intelligent Systems, 2004. [26] K. Nishio, H. Yonezu, A.B. Kariyawasam, Y. Yoshikawa, S. Sawa, and Y. Furukawa, “Analog integrated circuit for motion detection against moving background based on insect visual systems,” Optical Review, vol.11, pp.24–33, 2004.
Kimihiro Nishio received the B.E., M.E. and Dr.E. degrees in electrical and electronic engineering from Toyohashi University of Technology, Toyohashi, Japan in 1999, 2001 and 2004, respectively. He is now a Research Associate in the Department of Electrical and Computer Engineering, Yonago National College of Technology, Yonago, Japan. His current research interests include hardware implementations of biological neural systems and applications of neuromorphic vision chips.
Hiroo Yonezu received the B.E. degree in electronic engineering from Shizuoka University in 1964 and the Dr.E. degree in the electrical engineering from Osaka University in 1975. In 1964 he joined Nippon Electric Co. Ltd. He made contributions to the research on degradation mechanisms and the improvement of operating life of AlGaAs lasers. Since 1986 he has been a professor with the Department of Electrical and Electronic Engineering, Toyohashi University of Technology. His research has been concerned with basic technologies for future OEICs. He was a councilor of The Japan Society of Applied Physics. He received the SSDM Award from the International Conference on Solid State Devices and Materials in 1995.
Yuzo Furukawa received the B.S., M.S., and Ph.D. degrees in electronic science and engineering from Kyoto University, Kyoto, Japan, in 1995, 1997 and 2000, respectively. In 2000, he joined Toyohashi University of Technology, Toyohashi, Japan, where he is currently a Research Associate in the Department of Electrical and Electronic Engineering. His research interests are hetero-epitaxial growth of semiconductors, optoelectronic devices and analog integrated circuits.
