Pulse modulation CMOS image sensor for bio-fluorescence imaging applications Jun Ohta∗ , Takashi Tokuda∗ , Keiichiro Kagawa∗ , Masahiro Nunoshita∗ , Sadao Shiosaka† ∗ Graduate
School of Materials Science School of Biological Science Nara Institute of Science and Technology (NAIST) 8916-5 Takayama, Ikoma, Nara 630–0101, JAPAN Email:
[email protected] † Graduate
Abstract— For wide dynamic range, compatibility with digital circuits, and low-voltage operation, the pulse modulation technique is suitable for an implanted bioimage sensor. We demonstrate bio-fluorescence imaging of the hippocampus in a sliced mouse brain using a pulse modulation-based image sensor. The sensor architecture and system configuration are discussed. In addition, we develop an imaging device for implantation into a mouse brain in order to measure the neural activity in the hippocampus. The device is composed of a pulse modulation image sensor with 128×128 pixels and a fiber illuminator on a polyimide substrate.
I. I NTRODUCTION The application of CMOS technologies to bio-technologies is an emerging field. The system-on-a-chip approach provides a compact, high-sensitivity measurement tool, which enables on-site measurement. Several pioneering studies have investigated two-dimensional voltage sensing of neural activity [1], [2]. Cell luminescence is another signal that has a high potential for use in bio-technology applications and has been studied extensively. Recently, the Stanford group has reported an image sensor for detecting bio-luminescence, a phenomenon in which light is emitted from biological tissues due to a chemical reaction, and thus in the absence of external excitation light [3]. Another type of luminescence that is used extensively in bio-technology is fluorescence. Fluorescence is a phenomenon in which light is emitted by dyed cells that are excited by an external UV source. Fluorescence is also widely used to investigate cell activity. A fluorescence microscope is usually used to capture fluorescence images. Fluorescence microscopes are equipped with a cooled CCD camera and imaging optics. Although the fluorescence microscope is a powerful tool, it cannot be applied to in vivo cell measurement. Our goal is to develop an implantable image sensor for measuring in vivo cell activity, particularly for use in the study of the Long Term Potentiation (LTP) in the hippocampus. Although functional Magnetic Resonance Imaging (fMRI) and Positron Emission Tomography (PET) can acquire in vivo images, the resolution of these techniques is not sufficient for such applications. Other potential applications of the newly developed chip include DNA microarray measurement and protein chips. For the implanted sensor, we introduce a pulse modulation photosensor, which provides high sensitivity, a wide dynamic
0-7803-8834-8/05/$20.00 ©2005 IEEE.
range, and low-voltage operation [4]. We herein describe the architecture and fundamental characteristics of the newly developed sensor and demonstrate fluorescence imaging of a dyed mouse brain slice. Finally, we describe the structure of the implantable sensor, in which a molded sensor chip and a fiber illuminator are integrated on a polyimide substrate. II. P ULSE MODULATION PHOTOSENSING Target materials of small volume (e.g., neural cells or DNA spots) require a bioimaging sensor that has a high sensitivity and a wide dynamic range. In conventional CMOS image sensors, the voltage change of photodiodes (PD) in each pixel caused by photocarrier generation is measured as light intensity. In order to detect extremely low intensity light, long-time exposure or accumulation is effective. However, one disadvantage of long-time accumulation is that the detection limit in the high-intensity region decreases. We propose the application of a pulse modulation measurement scheme for bioimaging sensors. In the pulse modulation method, the light intensity is measured not as a voltage drop, but rather as the time required for a specific voltage drop, as illustrated in Figure 1. For the time measurement, a pulse width modulation (PWM) or time-to-saturation photosensor is realized [6], [5], [7], [8]. In addition, for feedback of the output to the reset transistor of the PD, a pulse frequency modulation (PFM) photosensor is realized [9], [10], [11], because there is no fundamental difference between PWM and PFM, [13], [14], [15]. A PFM photosensor has been proposed to be applied to retinal proshtesis devices [10], [12].
3487
Fig. 1.
Concept of pulse modulation
The pulse modulation scheme provides the advantage of
adaptive photocarrier accumulation. Pixels automatically execute long-time accumulation under low incident light conditions and high-frequency measurement under high incident light conditions. Such feature is suitable to in vivo imaging, becaseu the exciation illumination may not be controlled and thus there would be bright and dark regeion. The trade-off between measurement time and measurable intensity is applied to each pixel independently. Thus, a wider dynamic range is achieved for the captured image. Another advantage of the pulse modulation method is compatibility with logic circuits, because the output of a pulse modulation-based image sensor is a pulse stream. This enables integration of post-processing circuits with minimal circuit area, which is essential for an implanted device.
can detect an intensity change on the order of 10 pW/cm2 in the lowest intensity region. The circuit simulation has found that the total noise level is about 1 mVrms and this value is not critical when the light intensity approaches to the lowest intensity region or the output reaches around at a maximum value.
III. D ESIGN AND CHARACTERISTICS Figure 2 shows a schematic diagram of the newly designed pulse modulation photosensor. The sensor was designed using a 0.6-µm standard CMOS process (2-poly 3-metal). The photosensor consists of a pixel core circuit, a high-gain amplifier, a D-flip-flop (DFF), and other digital components. The PD size is 50 µm × 50 µm. The amplifier is a conventional twostage differential amplifier with an offset cancel circuit having a DC gain of over 70 dB. The output signal of the amplifier is digitized and stored by the DFF, which is synchronously driven by a clock. Fig. 3.
Fig. 2.
Pulse width as a function of incident light intensity
Schematic diagram of the PWM pixel
When the PD voltage drops, the output of the amplifier becomes high and the DFF stores the state. The two reset signals: TIMING Reset and Timing S1 are relayed only when the DFF is high. Thus, in the case that the discharge of the PD is completed between the two latest clocks, a pulse is output and the pixel is reset until the next clock. The interval between two pulses is evaluated as the discharge time. Figure 3 shows the pulse width as a function of incident light intensity. The detectable intensity range extends as wide as 106 (120 dB). The detectable minimum intensity is as low as 2 nW/cm2 . Figure 4 shows the standard deviation of the pulse width as a function of incident light intensity. In the middle-to-low intensity range, the deviation of the pulse width is less than 0.2%. The dark current nonuniformity limits the minimum detectable intensity and we do not calibrate it in the present device. The present result suggests that the photosensor
Fig. 4. Standard deviation of pulse width as a function of incident light intensity
Next, we have evaluated the fluorescence of AMC (7-amino4-metyl-coumarin) solution using the fabricated photosensor for in vivo imaging of neural activity (neuropsin) in the hippocampus [16]. Figure 5 shows the experimental results for output pulse width as functions of AMC concentration and input light intensity, respectively. These detectable regions
3488
TABLE I S ENSOR S PECIFICATIONS
cover the measurement regions in in vitro experiments using sliced mouse brain with AMC, suggesting that the sensor can be applied to the observation of neural activity in the hippocampus using AMC.
Fig. 5.
AMC fluorescence detection
IV. B RAIN IMAGING A. in vitro imaging We have also designed and fabricated a 64×64-pixel pulse modulation image sensor based on the above architecture. Figure 6 illustrates the concept of the device operation and presents a micrograph showing the layout of the device. The specifications are listed in Table I. Only the pixel core in Figure 2 is employed in the pixel. The amplifier and the digital module are implemented on each column. Sixty-four pixels on a selected row are connected to the column units and work as synchronous pulse modulation photosensors in Figure 2. The outputs of the 64 pixels on the selected row are stored in 64 DFFs in the column module as 1-bit/pixel row data. The 64-bit data are read out serially by an X Scanner. The 1-bit/pixel data is transferred to a PC and frame images are reconstructed.
Fig. 6.
Technology Number of Pixels Pixel Size Sensor Size Operation Voltage
Standard 0.6 µm CMOS 64 × 64 20 µm × 20 µm 1150 µm × 1500 µm 5V
Figure 7 shows the configuration of the imaging experiment. The image sensor was molded, and a blue resist layer was spun as a UV cut filter. A DAPI-stained brain slice sample was placed on the assembled image sensor. A UV excitation light having a wavelength between 350 nm and 380 nm was illuminated on the slice, and the fluorescence image at the bottom of the brain slice was captured by the fabricated image sensor. A conventional CCD microscope was also used to monitor the slice and the sensor. Figure 8 shows an onchip fluorescence image of the mouse hippocampus. The structure of the hippocampus is clearly observed. At present, we are applying the newly developed sensor to the fields of Ca2+ imaging and functional imaging using AMC or Green Fluorescent Protein (GFP).
Fig. 7.
Experimental setup for brain slice imaging
Fig. 8.
Experimental results for brain slice imaging
PWM array sensor and layout
We have applied this sensor to an in-vitro on-chip imaging of the hippocampus of a mouse brain.
3489
B. Towards in vivo imaging We are currently developing an in vivo imaging device. Implantation in the mouse brain requires the integration of a UV illuminator near the sensor. Figure 9 shows an assembled image sensor with two PMMA fibers. Part of the clad layers of the fibers is removed so as to transmit light in leaky mode. The next step will be to implant the assembled device into the hippocampus of a mouse brain stained with AMC.
Fig. 9.
Assembled implant device with fiber illuminators
[7] T. L´ule, B. Schneider, and M. B¨ohm, ”Design and Fabiracation of a HighDynamic-Range Image Sensor in TFA Technology,” IEEE J. Solid-State Cir.34, pp.704-711, 1999. [8] D. Stoppa, A. Simoni, L. Gonzo, M. Gottardi, and G.-F. D. Betta, ”Novle CMOS Image Sensor With a 132-dB Dynamic Range,” IEEE J. SolidState Cir.37, pp.1846-1852, 2002. [9] W. Yang, ”A wide-dynamic-range, low power photosensor array,” Digest Int’l Solid-State Circuits Conf. (ISSCC), pp. 230-231, San Francisco, CA, 1994,. [10] J. Ohta, N. Yoshida, K. Kagawa, and M. Nunoshita, ”Proposal of Application of Pulsed Vision Chip for Retinal Prosthesis,” Jpn. J. Appl. Phys. 41, 2322, 2002. [11] E. Culurciello, R. Etienne-Cummings, and K. A. Boahen, ”A bimorphic digital image sensor,” IEEE J. Solid-State Cir., 38, pp. 281-294, 2004. [12] K. Kagawa, K. Yasuoka, D. C. Ng, T. Furumiya, T. Tokuda, J. Ohta, M. Nunoshita, ”Pulse-domain digital image processing for vision chips employing low-voltage operation in deep-submicron technologies,” IEEE Selected Topics Quantum Electron., 10, pp.816-828, 2004. [13] L. McIlrath, ”A low-power low-noise ultrawide-dynamic-range CMOS imager with pixel-parallel A/D conversion,” IEEE J. Solid-State Cir., 36, pp. 846-853, 2001. [14] L. McIlrath, ”A robust O(N log n) algorithm for optimal decoding of first-order SD sequences,” IEEE Trans. Signal Process. 50, pp. 1942-1950, 2002. [15] S. Kavusi and A. El Gamal, ”Quantitative study of high dynamic range SD-based focal plane array architectures,” Proc. SPIE, Infrared Technology and Applications XXX, 5406, 2004. [16] K. Matsumoto-Miyai, A. Ninomiya, H. Yamasaki, H. Tamura, Y. Nakamura, and S. Shiosaka, ”NMDA-Dependent Proteolysis of Presynaptic Adhesion Molecule L1 in the Hippocampus by Neuropsin,” J. Neurosci 23, 7727-7736, 2003.
V. C ONCLUSION We have designed a photosensor having a pulse modulation measurement scheme. The photosensor has a dynamic range of 120 dB with respect to light intensity and is capable of detecting intensity changes on the order of 10 pW/cm2 in lowest intensity region. We have designed and fabricated a 64×64-pixel image sensor based on pulse modulation. The sensor is applied to in vitro on-chip brain slice imaging. Fluorescence imaging of the DAPI-stained hippocampus of a mouse brain has been successfully demonstrated. ACKNOWLEDGMENT This work was supported in part by the Semiconductor Technology Academic Research Center (SATRC). R EFERENCES [1] P. Fromherz, ”Neuroelectronic Interfacing: Semi-conductor Chips with Ion Channels, Nerve Cells, and Brain,” Nanoelectronics and Information Technology, R. Waser, Ed. Wiley-VCH, Berlin, 2003, pp. 781–810. [2] B. Eversmann, ”A 128 × 128 CMOS Bio-Sensor Array for Extracellular Recording of Neural Activity,” Digest of 2003 IEEE Int’l Solid-State Circuits Conf. (ISSCC2003), San Francisco, CA, 2003. [3] H. Eltoukhy, K. Salama, A. El Gamal, M. Ronaghi, and R. Davis, ”A 0.18 µm CMOS 10−6 lux bioluminescence detection system-on-chip,” Digest IEEE Int’l Solid-State Circuits Conf. (ISSCC), pp. 222-223, San Francisco, CA, 2004. [4] D. C. Ng, H. Okamoto, T. Tokuda, K. Kagawa, J. Ohta, and M. Nunoshita, ”A Pulse Modulation CMOS Image Sensor with 120 dB Dynamic Range and 1 nW/cm2 Resolution for Bioimaging Applications,” Ext. Abst. of Int’l Conf. Solid State Device & Materials (SSDM), pp. 384-385, Tokyo, 2004. [5] A. Iwata, M. Nagata, N. Takeda, M. Homma, and T. Morie, ”Pulse Modulation Circuit Architecture and its Application to Functional Image Sensors,” Proc. IEEE International Symposium on Circuits and Systems 2000, #0113-4, pp. II-301-304, May. 2000, Geneva. [6] V. Brajovic and T. Kanade, ”A sorting image sensor and example of massively parallel intensity-to-time processing for low-latency computational sensor,” IEEE Int’l Conf. Robotics and Automation, pp.1638-1643, 1996.
3490