Low-Cost Logarithmic CMOS Image Sensing By ... - IEEE Xplore
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
1212
Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion Ying-Chieh Chuang, Shih-Fang Chen, Shi-Yu Huang, and Ya-Chin King, Member, IEEE
Abstract — In this paper we present a CMOS image sensor design with a better sensitivity to low light intensity. This feature is achieved through a multi-resolution analog-todigital converter (ADC). By doing so, the photo-electrical characteristic of a sensor cell can is finely tuned. A costeffective architecture for realizing the required ADC is also proposed. This architecture leads to a faster conversion time as well as a smaller area. A simulation environment with postlayout accuracy is incorporated to demonstrate the advantages. It shows that a number of images can be captured more clearly than a traditional sensor1.
characteristic is not perfect. From a human’s perspective, an ideal pixel should have a logarithmic output response as shown in Fig. 2. Im ag e S en so r A rra y
D e co de r an d C o n troller
CDS (c o rrelate d do u ble s am p ling )
M u ltiplexo r
Index Terms — sensor, nonlinear ADC, logarithmic, photo diode.
A na lo g-to -D ig ital C o nverte r 8-bit d ig ital ou tp u t
cloc k e na ble
I. INTRODUCTION
Fig. 1. Architecture of a CMOS image sensor.
Recently, CMOS image sensors (CIS) have been emerging as an alternative [1][2] of the traditional sensors using the charge-coupled device (CCD) technology. As shown in Fig. 1, such a CMOS sensor often integrates the sensor cell array (also called pixel array), the periphery, and the analog-todigital processing circuitry on a single chip. This high level of integration leads to a potentially smaller and lower-power product. Besides, extra features such as anti-jittering (image stabilization) or image compression can be easily added on the same chip. However, a number of issues need to be resolved in order to design a high-quality CMOS sensor. For example, the dark current caused by the leakage of the pixels may reduce the quality of the captured image. The device mismatches among pixels and column read-out circuits may contribute to the socalled fixed-pattern noise (FPN) that appears as vertical strip patterns in the image captured. In addition, a pixel’s photo-electrical characteristic that dictates how a pixel reacts to light often needs to be optimized. A typical photo-electrical characteristic is shown in Fig. 2. It can be divided into two regions, the linear region and the saturation region. When the light intensity is relatively low (e.g., lower than 103 Lux), the pixel is in the linear region. In this region, the pixel’s output voltage increases linearly with the light intensity. On the other hand, when the light intensity is high, the pixel will saturate and output the same voltage, regardless of the input light intensity. Such a photo-electrical 1 Y.-C. Chuang is now with Silicon Integrated Systems Corp., Taiwan. (email: [email protected]) S.-F. Chen is now with Silicon Integrated Systems Corp., Taiwan. (e-mail: [email protected]) S.-Y. Huang is with Electrical Engineering Department, National TsingHua University, HsinChu, Taiwan (e-mail: [email protected]) Y.-C. King is with Electrical Engineering Department, National Tsing-Hua University, HsinChu, Taiwan (e-mail: [email protected])
Contributed Paper Manuscript received July 16, 2005
O utput (volt)
desired response
1.5
saturation region 1.0
0.5
linear region
typical response
0 10 3
Light intensity (Lux)
Fig. 2. Typical and desired output responses.
The above discussion reveals two common deficiencies of a CMOS image sensor. First, it is often inadequate when taking indoor pictures under low light intensity. Second, their dynamic range (DR), which is the light intensity range in which a sensor can differentiate, is much lower than their CCD counterpart, making it unable to differentiate pictures with strong light intensity, neither [3]. In this paper, we address the first deficiency. We propose to idealize the linear region of a pixel’s photo-electrical characteristic when performing the analog-to-digital conversion. A nearly logarithmic response is achieved with the aid of a multi-resolution ADC, in which different output codes could correspond to different input range. By doing so, the ADC can perform the signal enhancement (that transforms the linear curve to a nearly logarithmic one) and analog-to-digital translation at the same time. We also propose a cost-effective architecture for the required ADC. The new architecture is based on successive approximation. As compared to a naïve implementation, we show that it can reduce the silicon area significantly, while speeding up the conversion simultaneously.
0098 3063/05/$20.00 © 2005 IEEE
Y.-C. Chuang et al.: Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion
The rest of this paper is organized as follows. Section 2 provides the preliminaries of CMOS image sensors. Section 3 shows how to design the transfer function of the multiresolution ADC. Section 4 introduces the ADC’s architecture. Section 5 gives the simulation results and Section 6 concludes. II. PRELIMINARIES The sensing process of an image is done through scanning the sensor array in a row-major order, one row at a time. The cells on the same row are sensed out simultaneously and latched at each respective column output. After that, they take turn to be converted into a digital output code through the embedded ADC in a serial manner. Such a row sensing process iterate through every row sequentially. VDD
VDD
1213
Hence, a smaller voltage x actually gives a larger voltage at the output of CDS. The detailed operation of CDS is referred to the literature [2]. (Conversion step) In the final step, analog output voltage is converted into a digital code. The granularity of the ADC could be one for each column or one for the entire chip. If there is only one ADC, then each column’s output takes turn to be converted. The overall timing diagrams of the controlling signals, including reset, RS, CL, and SH are shown in Fig. 4, where CL, and SH are two signals to control the operation of CDS (to be explained later). It can be seen that the sensing operations of two consecutive rows are interleaved. When (i+1)-th row is in the integration and CDS steps, the i-th row is also being converted.
source follower
i-th ro w in te g ratio n
i-th ro w res e t
reset
i-th ro w R S CL
X
SH
RS: row select
photo diode
i-th ro w A D C
photo current
(i+ 1 ) ro w in te g ra tio n (i+ 1 )-th ro w re se t
Sensor Array Con.
(i+ 1)-th row R S
column line
CDS MUX ADC
CL SH
Fig. 4. Timing diagrams of controlling signals. Fig. 3: A sensor cell.
In more detail, a typical sensor cell is shown in Fig. 3. It consists a photo diode and three MOS transistors. The sensing process of a pixel takes four steps. (Reset step) In the first step, the storage node x is charged up to a high voltage through a reset transistor. The gate of this transistor is controlled by a signal denoted as reset. (Integration step) In the second step, signal reset goes low and the voltage of the storage node x starts to discharge slowly through the photo diode. The discharging speed is directly proportional to the photo current Iphoto. A stronger light intensity gives rise to a larger photo current, and leading to a smaller output voltage. The time spent in the second step is often referred to as the integration time, which is usually in the range from 0.01 to 0.1 ms. During this period of time, negative photo charges are accumulated and reflected as the voltage drops at node x. (CDS step) In the third step, the voltage at node x is read out through a source follower and connected to its column line through a switch. This switch is controlled by a row selecting signal RS. The voltage level drops by a threshold voltage after the source follower. In order to eliminate the fixed-pattern noise (FPN) mentioned previously, such a output signal will be passed on to a correlated double sampling (CDS) circuit to remove device-related mismatch as shown in Fig. 4. After CDS, the signal is inverted.
The waveform of a pixel’s output is depicted as the dashed line in Fig. 5(a), along with the reset signal waveform as the solid line. After the reset step, we assume that the pixel’s output is charged up to Vreset. While at the end of the integration, it drops to a level denoted as Vx. In the following, we will use these waveforms to explain the operation of a conventional Correlated Double Sampling (CDS) circuit, as shown in Fig. 5(b). The operation of this circuit goes in two stages: tracking and subtraction. Let us first denote the lefthand side node of the center coupling capacitor C1 as α and the right-hand side as β. During the tracking stage, switches controlled by CL and SH are both turned on. Thus, the voltage at α will follow the input signal (which is the pixel’s output), and the voltage at β will be forced to ground. This CDS circuit goes into the subtraction stage when the controlling signal reset has a rising transition. It occurs when the integration ends and the next reset cycle begins. At this moment, signal CL will go low and make the center coupling capacitor C1 floating. Therefore, there will be no charging or discharging on this capacitor. It implies that the voltage difference between the two capacitor’s plates will remain unchanged throughout the subtraction stage. As a result, any voltage change at its left-hand side, i.e., node α, will cause the same amount of voltage change at its right-hand side node β . Since the voltage at α now rises from Vx to Vreset, with an increase of (Vreset – Vx) as shown in Fig. 5(a), the voltage at β
1214
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
will also increase by the same amount and transition from 0 to (Vreset – Vx). Two observations can be made at the output signal of this CDS circuit. First of all, its voltage, i.e., (Vreset – Vx), is truly linearly proportional to the voltage drop at the pixel’s output, as what we have desired. Secondly, it is offset free in the sense that the variations of Vreset among pixels are removed through this precharge-and-subtract process. The final signals read out from the pixels are not dependent on the fluctuations of the reset voltages that could be induced by the non-ideal manufacturing process.
By doing so, the resolution under smaller pixel’s output voltage is actually strengthened, leading to the desired logarithmic response. VDD
VDD
X
photo diode
Circled transistor is in sub-threshold region. Iph
Vreset
voltage
Pixel output Vx time
Reset
column line
Integration stage
(a) Waveforms of signals reset and pixel output. SH
RS
α
pixel output C2
C1
CL
β
Fig. 6. A conventional logarithmic sensor cell.
Control signals CDS output
reset RS CL SH
(b) The schematic and control signals.
Fig. 5. The schematic and operation of a CDS.
Traditionally, a logarithmic sensor can be achieved by a cell structure shown in Fig. 6. In this structure, the reset transistor is diode-connected and operates in the sub-threshold region. Thus, the voltage read out is logarithmically proportional to the photo-current induced by light illumination in the photo diode Iphoto. Although this cell is quite cost-effective, it has two major drawbacks. First of all, the output swing is usually limited to a threshold voltage. Such a small output voltage swing will not only create difficulty for the following analogto-digital converter (ADC), but also make the sensor more vulnerable to the noise. Secondly, it is not easy, if not impossible, to perform the fixed-pattern noise reduction like we did previously [6]. In the following, we propose a low-cost logarithmic sensor architecture that can overcome these two drawbacks. The proposed sensor can be easily adapted from a conventional linear sensor because we will only modify the peripheral ADC, while leaving the cell array intact and keeping the fill-factor unchanged. III. MULTI-RESOLUTION ADC In our sensor design, the key to achieve the desired logarithmic response is only a signal enhancement property built in to the ADC. A multi-resolution ADC as shown in Fig. 7 can realize this property. The analog-to-digital conversion process has now been shaped in a way that the analog voltage step for a digital code is smaller when the input analog voltage is smaller, while larger when the input analog voltage is larger.
However, the resolution of an ADC, referred to as the minimum analog voltage step ΔVmin that can be resolved, is limited. It is not possible to construct a multi-resolution ADC that is capable of enhancing the signal in a truly logarithmic way. Therefore, we use the piecewise-linear approximation [4][5][7]. It means that the transfer curve of this ADC is divided into a number of segments, each of which has its own resolution. These segments can be determined through a process of resolution distribution discussed below. A D C w it h s ig n a l e n h a n c e m e n t CDS o u tp u t
o u tp u t code
o u tp u t code
255
x
L ig h t In t e n s i t y (L u x )
=
a n a lo g in p u t to A D C
L i g h t i n t e n s it y (L u x )
Fig. 7. Logarithm via multi-resolution ADC.
Example 1: In this example, we show how to derive the transfer function of our ADC in detail. Assume that the input range of our ADC is [0, 1.5] V and the output is an 8-bit digital code. In order to have a nearly algorithmic response, we need to use a higher resolution for the input, e.g., 10-bit resolution. This implies that the minimum voltage step ΔVmin that can be resolved is 1.5/210 = 1.5 mV. Next, we assign an integer code range R(i) to each output code i. This integer represents the corresponding input range of a digital code in terms of ΔVmin. To be logarithmic, a smaller output code is assigned a smaller code range as shown in Fig. 7. To cover the entire input range, the summation of the range of every output code should be equal to the overall input range 210 = 1024. It is like a distribution problem that distributes 1024×ΔVmin among
Y.-C. Chuang et al.: Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion
256 codes. Therefore, we arrive at the following guidelines for deriving the ADC’s transfer function. (1) The code range function is non-decreasing, i.e., R(i) ≥ R(j) if i> j, where i and j are two digital codes. (2) R(0) + R(1) + … +R(255) = 1024. Based on these two guidelines, we gradually increase the code range from 1 to 10 as the code number increases from 0 to 255.
In our sensor design, we choose the successive approximation ADC. One naïve way of implementing a multiresolution ADC is done by cascading a linear ADC and a lookup table as shown in Fig. 8. Linear S A-AD C
Table lookup
V in
10-bit code
S /H
com parator
clk
1024 x 8 m apping table
8-bit code
control logic 8-bit
10-bit DAC
V in 8-bit S/H comparator
clk
10-bit control logic
10-bit DAC
IV. COST REDUCTION FOR ADC
10-bit S AR 10-bit
clk
Fig. 8. A naïve architecture for multi-resolution ADC.
The lookup table defines how the original output code is mapped into the desired enhanced code. Since the input resolution is assumed to be 10-bit when deriving the transfer curve, the hardware requirement of such a scheme is relatively high. It requires a 10-bit linear ADC and a 1024×8 lookup table. It turns out that the silicon area of an 8K-bit ROM would be at least two times larger than the linear successive approximation ADC, which includes basic functional blocks such as a sample-and-hold, a comparator, control logic, a 10bit register SAR, and a 10-bit DAC. In this architecture, conversion of one sample takes 12 clock cycles: one for data sampling, 10 clock cycles of successive approximation, and one for the table lookup to derive the final output code. In order to reduce the hardware overhead, we interleave the ADC with the table lookup, resulting in a new architecture, s shown in Fig. 9. The operation of this new architecture proceeds as follows. It takes one cycle to latch in the data. Then, it takes 8 clock cycles to perform multi-resolution successive approximation to compute the final output. Like linear ADC, the temporary result is stored in a register called SAR, which is 8-bit wide only. The main difference here is that every reference voltage to be compared with the input voltage (for deciding the value of one bit in SAR) in every clock cycle is the enhanced version after the mapping table. In this way, the non-linearity is taken into account implicitly. In the following, we use an example to further illustrate this process.
1215
256 x 10 mapping table
8-bit SAR
8-bit code
clk
Fig. 9. A new architecture of multi-resolution ADC.
Example 2: Assume that the resolutions of the input and output are 8 and 4, respectively, and the mapping function from the output space to the input space is {f(‘00’) = ‘000’, f(‘01’) = ‘001’, f(‘10’) = ‘010’, f(‘11’) = ‘100’}. Let the given input be 3.2×ΔVmin. It takes the following two clock cycles to determine the value of SAR. Initially, SAR = ‘10’. In the first cycle, the input voltage is compared with the corresponding voltage of the reference code ‘10’. Through the mapping table, this reference code is mapped into f(‘10’) = ‘010’, which corresponds to 2×ΔVmin. Since 3.2×ΔVmin is larger than this reference voltage, the most significant bit of SAR to set to ‘1’ and SAR becomes ‘11’ temporarily. We are now ready to decide the second bit of SAR. In the second cycle, the reference is ‘11’. After mapping, it becomes f(‘11’) = ‘100’, which corresponds to 4×ΔVmin. Since 3.2×ΔVmin is smaller than this reference voltage, the second most significant bit of SAR to set to ‘0’ and SAR becomes ‘10’, which is also the final output code. The new architecture requires much less hardware. The original 1024×8 table has been reduced to 256×10. The reduction is 69%. Since this table is the dominating factor of the original architecture, it contributes to a much smaller silicon area. In addition, the new architecture is also faster. Originally, it takes 12 cycles to convert a sample, now it takes only 9 cycles. The speed up ratio is (12-9)/12 = 25%. IV. EXPERIMENTAL RESULTS We have designed the proposed architecture using TSMC 0.35 μm 1P4M standard CMOS logic technology. The specification and layout of this chip is shown in Fig. 10. In the left-bottom corner is our ADC. The 256×10 ROM table accounts for about 42% of its total layout area. As mentioned previously, if the proposed interleaved architecture is not applied, then the ADC would have been 2.5 times larger. Our ADC can process 1M samples per second, and operate in the proposed nonlinear mode or in the traditional linear mode depending on the value of a control signal. Its quality in terms of INL (integral non-linearity) and DNL (differential
1216
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
non-linearity) in the linear mode is evaluated through postlayout simulation, as shown in Fig. 11. The maximal absolute values of the DNL and INL are 0.42 LSB and 0.41 LSB, respectively.
Technology
TSMC 0.35 μm 1P4M
power supply
3.3 V
resolution
176 x 144 (QCIF)
frame rate
30 frame/s
pixel size
7.5 x 7.5 μm2
chip size
3041 x 3041 μm2
power dissipation
30 mWatt
176 x 144 Sensor Array
Controller
CDS
Buf ADC ROM
BIST
Periphery
In order to assess the overall image quality of the proposed sensor, we also built a software simulation environment. This environment takes as inputs a two-dimensional light intensity file that mimics the illumination of the sensor array, and a light-to-digital-code transfer function obtained from the previous column model simulation. The output is an image file that could have been produced by our sensor. With such a simulator, we are able to evaluate the quality of a sensor design before silicon. Fig. 13 shows the simulation results of an image. It demonstrates that our design could improve the image quality by enhancing the low-light intensity area.
Fig. 10. Specification and layout. 0 .6
(LSB) DNLDNL (LSB)
0 .4 0 .2 0 .0 -0 .2 -0 .4 -0 .6 0
3 2
64
96
128
1 60
19 2
224
256
O u tp u t c o d e
0 .0
INL INL (LSB) (LSB)
proposed
original
0 .1
Fig. 13. Post-layout simulation results.
-0 .1 -0 .2 -0 .3
VII. CONCLUSION
-0 .4 -0 .5 -0 .6 0
32
64
96
128
160
192
224
We have presented a 176144 CMOS image sensor design.
256
O u tp u t c o d e
Fig. 11. DNL and INL of our ADC in the linear mode.
The overall response of a column model including the sensor cell, CDS, buffer, and A/D converter is shown in Fig. 12. Since we are not able to model the photo diode in SPICE simulation, we replace it with a current source. The magnitudes of the current sources are decided from the measurement results of manufactured pixel test key previously. It can be seen that the overall response matches our target quite well as a smooth logarithmic curve.
We used TSMC 0.35 ˩ m 1P4M CMOS process to manufacture this chip. The main features include a multiresolution scheme and a cost-effective architecture for nonlinear analog-to-digital conversion. These two features combined together improve the sensor’s quality for images under low light intensity. The proposed technique is general. It can be applied to any other sensor design that needs finetuning. During the design process, we also built a simulation environment to assess the effectiveness of this approach throughout the entire design cycle. Post-layout simulation shows that, with only minor area overhead, it is able to capture images more sharply than a naive design.
O utput digital code
REFERENCES [1]
[2]
Illum ination intensity (L ux)
Fig. 12. Overall photo response.
[3]
E. R. Fossum, “CMOS Image Sensors: Electronic Camera-On-A-Chip”, IEEE Transaction on Electron Devices, vol.44, pp. 1689-1698, October 1997. R. H. Nixon , S. E. Kemeny, B. Pain, C. O. Staller, and E. R. Fossum, “256×256 CMOS Active Pixel Sensor Camera-on-a-Chip”, IEEE Journal of Solid-State Circuits, vol.31, pp. 2046-2050, December 1996. S. Decker, R. McGrath, K. Brehmer, and C. Sodini, “A 256×256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output”, IEEE International Solid-State Circuits Conference, pp. 176-177, 1998.
Y.-C. Chuang et al.: Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion [4] [5]
[6]
[7]
C. W. Moreland, “An 8b 150 Msample/s serial ADC”, IEEE International Solid-State Circuits Conference, pp. 272-273, 1995. S. Mortezapour and E. K. F. Lee, “A 1-V, 8-Bit Successive Approximation ADC in Standard CMOS Process”, IEEE Journal of Solid-State Circuits, vol. 35, pp. 642-646, April 2000. S. Otim, D. Joseph, B. Choubey, and S. Collins, “Modeling of High Dynamic Range Logarithmc CMOS Image Sensors,” Proc. of Instrumentation and Measurement Technology Conf., (IMTC), pp. 451456, Vol. 1, 2004. G. Promitzer, “12-bit Low-Power Fully Differential Switched Capacitor Noncalibrating Successive Approximation ADC with 1 MS/s”, IEEE Journal of Solid-State Circuits, vol. 36, pp. 1138-1143, July 2001.
Ying-Chieh Chuang received his BS degree in Power Mechanical Engineering from National Tsing-Hua University in 1994 and MS degree in Electrical Engineering from National Tsing-Hua University in 2002. From 2002 to 2004 he was with Silicon Integrated Systems Corp., designing computing engine of 3D graphics. He joined XGI Technology Inc. in 2004, focusing on shader engine of 3D graphics. His research interests are in 3D graphics concept and computing engine architecture.
Shih-Fang Chen received his MS degree in Electrical Engineering from National Tsing-Hua University, Taiwan, ROC, in 2002. From 2002 to 2003 he was a hardware engineer at Silicon Integrated Systems Corporation(SiS), designing the circuit of 3D accelerator chip. In 2003, he joined the start-up company, XGI Technology, where he is currently a hardware engineer designing the circuit of a graphic processor.
1217
Shi-Yu Huang received his BS, MS degrees in Electrical Engineering from National Taiwan University in 1988, 1992 and Ph.D. degree in Electrical and Computer Engineering from the University of California, Santa Barbara in 1997, respectively. From 1997 to 1998 he was a software engineer at National Semiconductor Corp., Santa Clara, investigating the System-On-Chip design methodology. From 1998 to 1999, he was with Worldwide Semiconductor Manufacturing Corp., designing the high-speed Built-In Self-Test circuits for memories. He joined the faculty of National Tsing-Hua University, Taiwan, in 1999, where he is currently an Associate Professor. Dr. Huang’s research interests are mainly in design automation for VLSI, with an emphasis on power estimation, fault diagnosis, CMOS image sensor design, and lowpower SRAM design. He co-authored a book entitled "Formal Equivalence Checking and Design Debugging" published by Kluwer Academic Publishers in 1998.
Ya-Chin King was born in Taiwan, Republic of China. She received the B.S. degree in electrical engineering from National Taiwan University in 1992, and the M.S. degree in electrical engineering from University of California, Berkeley, in 1994. She received her PhD degree in May of 1999, at University of California, Berkeley, on thin oxide technology and novel quasinonvolatile memory. She joined the faculty of National Tsing-Hua University at Hsinchu, Taiwan in August 1999. She is currently an associate professor of the electrical engineering department at NTHU. Her research topics include: thin gate dielectric, CMOS image sensor and non-volatile memory design.
1212
Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion Ying-Chieh Chuang, Shih-Fang Chen, Shi-Yu Huang, and Ya-Chin King, Member, IEEE
Abstract — In this paper we present a CMOS image sensor design with a better sensitivity to low light intensity. This feature is achieved through a multi-resolution analog-todigital converter (ADC). By doing so, the photo-electrical characteristic of a sensor cell can is finely tuned. A costeffective architecture for realizing the required ADC is also proposed. This architecture leads to a faster conversion time as well as a smaller area. A simulation environment with postlayout accuracy is incorporated to demonstrate the advantages. It shows that a number of images can be captured more clearly than a traditional sensor1.
characteristic is not perfect. From a human’s perspective, an ideal pixel should have a logarithmic output response as shown in Fig. 2. Im ag e S en so r A rra y
D e co de r an d C o n troller
CDS (c o rrelate d do u ble s am p ling )
M u ltiplexo r
Index Terms — sensor, nonlinear ADC, logarithmic, photo diode.
A na lo g-to -D ig ital C o nverte r 8-bit d ig ital ou tp u t
cloc k e na ble
I. INTRODUCTION
Fig. 1. Architecture of a CMOS image sensor.
Recently, CMOS image sensors (CIS) have been emerging as an alternative [1][2] of the traditional sensors using the charge-coupled device (CCD) technology. As shown in Fig. 1, such a CMOS sensor often integrates the sensor cell array (also called pixel array), the periphery, and the analog-todigital processing circuitry on a single chip. This high level of integration leads to a potentially smaller and lower-power product. Besides, extra features such as anti-jittering (image stabilization) or image compression can be easily added on the same chip. However, a number of issues need to be resolved in order to design a high-quality CMOS sensor. For example, the dark current caused by the leakage of the pixels may reduce the quality of the captured image. The device mismatches among pixels and column read-out circuits may contribute to the socalled fixed-pattern noise (FPN) that appears as vertical strip patterns in the image captured. In addition, a pixel’s photo-electrical characteristic that dictates how a pixel reacts to light often needs to be optimized. A typical photo-electrical characteristic is shown in Fig. 2. It can be divided into two regions, the linear region and the saturation region. When the light intensity is relatively low (e.g., lower than 103 Lux), the pixel is in the linear region. In this region, the pixel’s output voltage increases linearly with the light intensity. On the other hand, when the light intensity is high, the pixel will saturate and output the same voltage, regardless of the input light intensity. Such a photo-electrical 1 Y.-C. Chuang is now with Silicon Integrated Systems Corp., Taiwan. (email: [email protected]) S.-F. Chen is now with Silicon Integrated Systems Corp., Taiwan. (e-mail: [email protected]) S.-Y. Huang is with Electrical Engineering Department, National TsingHua University, HsinChu, Taiwan (e-mail: [email protected]) Y.-C. King is with Electrical Engineering Department, National Tsing-Hua University, HsinChu, Taiwan (e-mail: [email protected])
Contributed Paper Manuscript received July 16, 2005
O utput (volt)
desired response
1.5
saturation region 1.0
0.5
linear region
typical response
0 10 3
Light intensity (Lux)
Fig. 2. Typical and desired output responses.
The above discussion reveals two common deficiencies of a CMOS image sensor. First, it is often inadequate when taking indoor pictures under low light intensity. Second, their dynamic range (DR), which is the light intensity range in which a sensor can differentiate, is much lower than their CCD counterpart, making it unable to differentiate pictures with strong light intensity, neither [3]. In this paper, we address the first deficiency. We propose to idealize the linear region of a pixel’s photo-electrical characteristic when performing the analog-to-digital conversion. A nearly logarithmic response is achieved with the aid of a multi-resolution ADC, in which different output codes could correspond to different input range. By doing so, the ADC can perform the signal enhancement (that transforms the linear curve to a nearly logarithmic one) and analog-to-digital translation at the same time. We also propose a cost-effective architecture for the required ADC. The new architecture is based on successive approximation. As compared to a naïve implementation, we show that it can reduce the silicon area significantly, while speeding up the conversion simultaneously.
0098 3063/05/$20.00 © 2005 IEEE
Y.-C. Chuang et al.: Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion
The rest of this paper is organized as follows. Section 2 provides the preliminaries of CMOS image sensors. Section 3 shows how to design the transfer function of the multiresolution ADC. Section 4 introduces the ADC’s architecture. Section 5 gives the simulation results and Section 6 concludes. II. PRELIMINARIES The sensing process of an image is done through scanning the sensor array in a row-major order, one row at a time. The cells on the same row are sensed out simultaneously and latched at each respective column output. After that, they take turn to be converted into a digital output code through the embedded ADC in a serial manner. Such a row sensing process iterate through every row sequentially. VDD
VDD
1213
Hence, a smaller voltage x actually gives a larger voltage at the output of CDS. The detailed operation of CDS is referred to the literature [2]. (Conversion step) In the final step, analog output voltage is converted into a digital code. The granularity of the ADC could be one for each column or one for the entire chip. If there is only one ADC, then each column’s output takes turn to be converted. The overall timing diagrams of the controlling signals, including reset, RS, CL, and SH are shown in Fig. 4, where CL, and SH are two signals to control the operation of CDS (to be explained later). It can be seen that the sensing operations of two consecutive rows are interleaved. When (i+1)-th row is in the integration and CDS steps, the i-th row is also being converted.
source follower
i-th ro w in te g ratio n
i-th ro w res e t
reset
i-th ro w R S CL
X
SH
RS: row select
photo diode
i-th ro w A D C
photo current
(i+ 1 ) ro w in te g ra tio n (i+ 1 )-th ro w re se t
Sensor Array Con.
(i+ 1)-th row R S
column line
CDS MUX ADC
CL SH
Fig. 4. Timing diagrams of controlling signals. Fig. 3: A sensor cell.
In more detail, a typical sensor cell is shown in Fig. 3. It consists a photo diode and three MOS transistors. The sensing process of a pixel takes four steps. (Reset step) In the first step, the storage node x is charged up to a high voltage through a reset transistor. The gate of this transistor is controlled by a signal denoted as reset. (Integration step) In the second step, signal reset goes low and the voltage of the storage node x starts to discharge slowly through the photo diode. The discharging speed is directly proportional to the photo current Iphoto. A stronger light intensity gives rise to a larger photo current, and leading to a smaller output voltage. The time spent in the second step is often referred to as the integration time, which is usually in the range from 0.01 to 0.1 ms. During this period of time, negative photo charges are accumulated and reflected as the voltage drops at node x. (CDS step) In the third step, the voltage at node x is read out through a source follower and connected to its column line through a switch. This switch is controlled by a row selecting signal RS. The voltage level drops by a threshold voltage after the source follower. In order to eliminate the fixed-pattern noise (FPN) mentioned previously, such a output signal will be passed on to a correlated double sampling (CDS) circuit to remove device-related mismatch as shown in Fig. 4. After CDS, the signal is inverted.
The waveform of a pixel’s output is depicted as the dashed line in Fig. 5(a), along with the reset signal waveform as the solid line. After the reset step, we assume that the pixel’s output is charged up to Vreset. While at the end of the integration, it drops to a level denoted as Vx. In the following, we will use these waveforms to explain the operation of a conventional Correlated Double Sampling (CDS) circuit, as shown in Fig. 5(b). The operation of this circuit goes in two stages: tracking and subtraction. Let us first denote the lefthand side node of the center coupling capacitor C1 as α and the right-hand side as β. During the tracking stage, switches controlled by CL and SH are both turned on. Thus, the voltage at α will follow the input signal (which is the pixel’s output), and the voltage at β will be forced to ground. This CDS circuit goes into the subtraction stage when the controlling signal reset has a rising transition. It occurs when the integration ends and the next reset cycle begins. At this moment, signal CL will go low and make the center coupling capacitor C1 floating. Therefore, there will be no charging or discharging on this capacitor. It implies that the voltage difference between the two capacitor’s plates will remain unchanged throughout the subtraction stage. As a result, any voltage change at its left-hand side, i.e., node α, will cause the same amount of voltage change at its right-hand side node β . Since the voltage at α now rises from Vx to Vreset, with an increase of (Vreset – Vx) as shown in Fig. 5(a), the voltage at β
1214
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
will also increase by the same amount and transition from 0 to (Vreset – Vx). Two observations can be made at the output signal of this CDS circuit. First of all, its voltage, i.e., (Vreset – Vx), is truly linearly proportional to the voltage drop at the pixel’s output, as what we have desired. Secondly, it is offset free in the sense that the variations of Vreset among pixels are removed through this precharge-and-subtract process. The final signals read out from the pixels are not dependent on the fluctuations of the reset voltages that could be induced by the non-ideal manufacturing process.
By doing so, the resolution under smaller pixel’s output voltage is actually strengthened, leading to the desired logarithmic response. VDD
VDD
X
photo diode
Circled transistor is in sub-threshold region. Iph
Vreset
voltage
Pixel output Vx time
Reset
column line
Integration stage
(a) Waveforms of signals reset and pixel output. SH
RS
α
pixel output C2
C1
CL
β
Fig. 6. A conventional logarithmic sensor cell.
Control signals CDS output
reset RS CL SH
(b) The schematic and control signals.
Fig. 5. The schematic and operation of a CDS.
Traditionally, a logarithmic sensor can be achieved by a cell structure shown in Fig. 6. In this structure, the reset transistor is diode-connected and operates in the sub-threshold region. Thus, the voltage read out is logarithmically proportional to the photo-current induced by light illumination in the photo diode Iphoto. Although this cell is quite cost-effective, it has two major drawbacks. First of all, the output swing is usually limited to a threshold voltage. Such a small output voltage swing will not only create difficulty for the following analogto-digital converter (ADC), but also make the sensor more vulnerable to the noise. Secondly, it is not easy, if not impossible, to perform the fixed-pattern noise reduction like we did previously [6]. In the following, we propose a low-cost logarithmic sensor architecture that can overcome these two drawbacks. The proposed sensor can be easily adapted from a conventional linear sensor because we will only modify the peripheral ADC, while leaving the cell array intact and keeping the fill-factor unchanged. III. MULTI-RESOLUTION ADC In our sensor design, the key to achieve the desired logarithmic response is only a signal enhancement property built in to the ADC. A multi-resolution ADC as shown in Fig. 7 can realize this property. The analog-to-digital conversion process has now been shaped in a way that the analog voltage step for a digital code is smaller when the input analog voltage is smaller, while larger when the input analog voltage is larger.
However, the resolution of an ADC, referred to as the minimum analog voltage step ΔVmin that can be resolved, is limited. It is not possible to construct a multi-resolution ADC that is capable of enhancing the signal in a truly logarithmic way. Therefore, we use the piecewise-linear approximation [4][5][7]. It means that the transfer curve of this ADC is divided into a number of segments, each of which has its own resolution. These segments can be determined through a process of resolution distribution discussed below. A D C w it h s ig n a l e n h a n c e m e n t CDS o u tp u t
o u tp u t code
o u tp u t code
255
x
L ig h t In t e n s i t y (L u x )
=
a n a lo g in p u t to A D C
L i g h t i n t e n s it y (L u x )
Fig. 7. Logarithm via multi-resolution ADC.
Example 1: In this example, we show how to derive the transfer function of our ADC in detail. Assume that the input range of our ADC is [0, 1.5] V and the output is an 8-bit digital code. In order to have a nearly algorithmic response, we need to use a higher resolution for the input, e.g., 10-bit resolution. This implies that the minimum voltage step ΔVmin that can be resolved is 1.5/210 = 1.5 mV. Next, we assign an integer code range R(i) to each output code i. This integer represents the corresponding input range of a digital code in terms of ΔVmin. To be logarithmic, a smaller output code is assigned a smaller code range as shown in Fig. 7. To cover the entire input range, the summation of the range of every output code should be equal to the overall input range 210 = 1024. It is like a distribution problem that distributes 1024×ΔVmin among
Y.-C. Chuang et al.: Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion
256 codes. Therefore, we arrive at the following guidelines for deriving the ADC’s transfer function. (1) The code range function is non-decreasing, i.e., R(i) ≥ R(j) if i> j, where i and j are two digital codes. (2) R(0) + R(1) + … +R(255) = 1024. Based on these two guidelines, we gradually increase the code range from 1 to 10 as the code number increases from 0 to 255.
In our sensor design, we choose the successive approximation ADC. One naïve way of implementing a multiresolution ADC is done by cascading a linear ADC and a lookup table as shown in Fig. 8. Linear S A-AD C
Table lookup
V in
10-bit code
S /H
com parator
clk
1024 x 8 m apping table
8-bit code
control logic 8-bit
10-bit DAC
V in 8-bit S/H comparator
clk
10-bit control logic
10-bit DAC
IV. COST REDUCTION FOR ADC
10-bit S AR 10-bit
clk
Fig. 8. A naïve architecture for multi-resolution ADC.
The lookup table defines how the original output code is mapped into the desired enhanced code. Since the input resolution is assumed to be 10-bit when deriving the transfer curve, the hardware requirement of such a scheme is relatively high. It requires a 10-bit linear ADC and a 1024×8 lookup table. It turns out that the silicon area of an 8K-bit ROM would be at least two times larger than the linear successive approximation ADC, which includes basic functional blocks such as a sample-and-hold, a comparator, control logic, a 10bit register SAR, and a 10-bit DAC. In this architecture, conversion of one sample takes 12 clock cycles: one for data sampling, 10 clock cycles of successive approximation, and one for the table lookup to derive the final output code. In order to reduce the hardware overhead, we interleave the ADC with the table lookup, resulting in a new architecture, s shown in Fig. 9. The operation of this new architecture proceeds as follows. It takes one cycle to latch in the data. Then, it takes 8 clock cycles to perform multi-resolution successive approximation to compute the final output. Like linear ADC, the temporary result is stored in a register called SAR, which is 8-bit wide only. The main difference here is that every reference voltage to be compared with the input voltage (for deciding the value of one bit in SAR) in every clock cycle is the enhanced version after the mapping table. In this way, the non-linearity is taken into account implicitly. In the following, we use an example to further illustrate this process.
1215
256 x 10 mapping table
8-bit SAR
8-bit code
clk
Fig. 9. A new architecture of multi-resolution ADC.
Example 2: Assume that the resolutions of the input and output are 8 and 4, respectively, and the mapping function from the output space to the input space is {f(‘00’) = ‘000’, f(‘01’) = ‘001’, f(‘10’) = ‘010’, f(‘11’) = ‘100’}. Let the given input be 3.2×ΔVmin. It takes the following two clock cycles to determine the value of SAR. Initially, SAR = ‘10’. In the first cycle, the input voltage is compared with the corresponding voltage of the reference code ‘10’. Through the mapping table, this reference code is mapped into f(‘10’) = ‘010’, which corresponds to 2×ΔVmin. Since 3.2×ΔVmin is larger than this reference voltage, the most significant bit of SAR to set to ‘1’ and SAR becomes ‘11’ temporarily. We are now ready to decide the second bit of SAR. In the second cycle, the reference is ‘11’. After mapping, it becomes f(‘11’) = ‘100’, which corresponds to 4×ΔVmin. Since 3.2×ΔVmin is smaller than this reference voltage, the second most significant bit of SAR to set to ‘0’ and SAR becomes ‘10’, which is also the final output code. The new architecture requires much less hardware. The original 1024×8 table has been reduced to 256×10. The reduction is 69%. Since this table is the dominating factor of the original architecture, it contributes to a much smaller silicon area. In addition, the new architecture is also faster. Originally, it takes 12 cycles to convert a sample, now it takes only 9 cycles. The speed up ratio is (12-9)/12 = 25%. IV. EXPERIMENTAL RESULTS We have designed the proposed architecture using TSMC 0.35 μm 1P4M standard CMOS logic technology. The specification and layout of this chip is shown in Fig. 10. In the left-bottom corner is our ADC. The 256×10 ROM table accounts for about 42% of its total layout area. As mentioned previously, if the proposed interleaved architecture is not applied, then the ADC would have been 2.5 times larger. Our ADC can process 1M samples per second, and operate in the proposed nonlinear mode or in the traditional linear mode depending on the value of a control signal. Its quality in terms of INL (integral non-linearity) and DNL (differential
1216
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
non-linearity) in the linear mode is evaluated through postlayout simulation, as shown in Fig. 11. The maximal absolute values of the DNL and INL are 0.42 LSB and 0.41 LSB, respectively.
Technology
TSMC 0.35 μm 1P4M
power supply
3.3 V
resolution
176 x 144 (QCIF)
frame rate
30 frame/s
pixel size
7.5 x 7.5 μm2
chip size
3041 x 3041 μm2
power dissipation
30 mWatt
176 x 144 Sensor Array
Controller
CDS
Buf ADC ROM
BIST
Periphery
In order to assess the overall image quality of the proposed sensor, we also built a software simulation environment. This environment takes as inputs a two-dimensional light intensity file that mimics the illumination of the sensor array, and a light-to-digital-code transfer function obtained from the previous column model simulation. The output is an image file that could have been produced by our sensor. With such a simulator, we are able to evaluate the quality of a sensor design before silicon. Fig. 13 shows the simulation results of an image. It demonstrates that our design could improve the image quality by enhancing the low-light intensity area.
Fig. 10. Specification and layout. 0 .6
(LSB) DNLDNL (LSB)
0 .4 0 .2 0 .0 -0 .2 -0 .4 -0 .6 0
3 2
64
96
128
1 60
19 2
224
256
O u tp u t c o d e
0 .0
INL INL (LSB) (LSB)
proposed
original
0 .1
Fig. 13. Post-layout simulation results.
-0 .1 -0 .2 -0 .3
VII. CONCLUSION
-0 .4 -0 .5 -0 .6 0
32
64
96
128
160
192
224
We have presented a 176144 CMOS image sensor design.
256
O u tp u t c o d e
Fig. 11. DNL and INL of our ADC in the linear mode.
The overall response of a column model including the sensor cell, CDS, buffer, and A/D converter is shown in Fig. 12. Since we are not able to model the photo diode in SPICE simulation, we replace it with a current source. The magnitudes of the current sources are decided from the measurement results of manufactured pixel test key previously. It can be seen that the overall response matches our target quite well as a smooth logarithmic curve.
We used TSMC 0.35 ˩ m 1P4M CMOS process to manufacture this chip. The main features include a multiresolution scheme and a cost-effective architecture for nonlinear analog-to-digital conversion. These two features combined together improve the sensor’s quality for images under low light intensity. The proposed technique is general. It can be applied to any other sensor design that needs finetuning. During the design process, we also built a simulation environment to assess the effectiveness of this approach throughout the entire design cycle. Post-layout simulation shows that, with only minor area overhead, it is able to capture images more sharply than a naive design.
O utput digital code
REFERENCES [1]
[2]
Illum ination intensity (L ux)
Fig. 12. Overall photo response.
[3]
E. R. Fossum, “CMOS Image Sensors: Electronic Camera-On-A-Chip”, IEEE Transaction on Electron Devices, vol.44, pp. 1689-1698, October 1997. R. H. Nixon , S. E. Kemeny, B. Pain, C. O. Staller, and E. R. Fossum, “256×256 CMOS Active Pixel Sensor Camera-on-a-Chip”, IEEE Journal of Solid-State Circuits, vol.31, pp. 2046-2050, December 1996. S. Decker, R. McGrath, K. Brehmer, and C. Sodini, “A 256×256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output”, IEEE International Solid-State Circuits Conference, pp. 176-177, 1998.
Y.-C. Chuang et al.: Low-Cost Logarithmic CMOS Image Sensing By Nonlinear Analog-To-Digital Conversion [4] [5]
[6]
[7]
C. W. Moreland, “An 8b 150 Msample/s serial ADC”, IEEE International Solid-State Circuits Conference, pp. 272-273, 1995. S. Mortezapour and E. K. F. Lee, “A 1-V, 8-Bit Successive Approximation ADC in Standard CMOS Process”, IEEE Journal of Solid-State Circuits, vol. 35, pp. 642-646, April 2000. S. Otim, D. Joseph, B. Choubey, and S. Collins, “Modeling of High Dynamic Range Logarithmc CMOS Image Sensors,” Proc. of Instrumentation and Measurement Technology Conf., (IMTC), pp. 451456, Vol. 1, 2004. G. Promitzer, “12-bit Low-Power Fully Differential Switched Capacitor Noncalibrating Successive Approximation ADC with 1 MS/s”, IEEE Journal of Solid-State Circuits, vol. 36, pp. 1138-1143, July 2001.
Ying-Chieh Chuang received his BS degree in Power Mechanical Engineering from National Tsing-Hua University in 1994 and MS degree in Electrical Engineering from National Tsing-Hua University in 2002. From 2002 to 2004 he was with Silicon Integrated Systems Corp., designing computing engine of 3D graphics. He joined XGI Technology Inc. in 2004, focusing on shader engine of 3D graphics. His research interests are in 3D graphics concept and computing engine architecture.
Shih-Fang Chen received his MS degree in Electrical Engineering from National Tsing-Hua University, Taiwan, ROC, in 2002. From 2002 to 2003 he was a hardware engineer at Silicon Integrated Systems Corporation(SiS), designing the circuit of 3D accelerator chip. In 2003, he joined the start-up company, XGI Technology, where he is currently a hardware engineer designing the circuit of a graphic processor.
1217
Shi-Yu Huang received his BS, MS degrees in Electrical Engineering from National Taiwan University in 1988, 1992 and Ph.D. degree in Electrical and Computer Engineering from the University of California, Santa Barbara in 1997, respectively. From 1997 to 1998 he was a software engineer at National Semiconductor Corp., Santa Clara, investigating the System-On-Chip design methodology. From 1998 to 1999, he was with Worldwide Semiconductor Manufacturing Corp., designing the high-speed Built-In Self-Test circuits for memories. He joined the faculty of National Tsing-Hua University, Taiwan, in 1999, where he is currently an Associate Professor. Dr. Huang’s research interests are mainly in design automation for VLSI, with an emphasis on power estimation, fault diagnosis, CMOS image sensor design, and lowpower SRAM design. He co-authored a book entitled "Formal Equivalence Checking and Design Debugging" published by Kluwer Academic Publishers in 1998.
Ya-Chin King was born in Taiwan, Republic of China. She received the B.S. degree in electrical engineering from National Taiwan University in 1992, and the M.S. degree in electrical engineering from University of California, Berkeley, in 1994. She received her PhD degree in May of 1999, at University of California, Berkeley, on thin oxide technology and novel quasinonvolatile memory. She joined the faculty of National Tsing-Hua University at Hsinchu, Taiwan in August 1999. She is currently an associate professor of the electrical engineering department at NTHU. Her research topics include: thin gate dielectric, CMOS image sensor and non-volatile memory design.
