A High-Speed, High-Sensitivity Digital CMOS Image ... - IEEE Xplore

766

IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 42, NO. 4, APRIL 2007

A High-Speed, High-Sensitivity Digital CMOS Image Sensor With a Global Shutter and 12-bit Column-Parallel Cyclic A/D Converters Masanori Furuta, Member, IEEE, Yukinari Nishikawa, Toru Inoue, Member, IEEE, and Shoji Kawahito, Senior Member, IEEE

Abstract—This paper presents a high-speed, high-sensitivity 512 512 CMOS image sensor with column parallel cyclic 12-bit ADCs and a global electronic shutter. Each pixel has a charge amplifier for high charge-to-voltage conversion gain despite of using a large-size photodiode, and two sample-and-hold stages for the global shutter and fixed pattern noise (FPN) canceling. High-speed column-parallel cyclic ADC arrays with 12-bit resolution having a small layout size of 0.09 mm2 are integrated at both sides of image array. A technique for accelerating the conversion speed using variable clocking and sampling capacitance is developed. A digital gain control function using 14-bit temporal digital code is also set in the column parallel ADC. The fabricated chip in 0.25- m CMOS image sensor technology achieves the full frame rate in excess of 3500 frames/s. The in-pixel charge amplifier achieves the optical sensitivity of 19.9 V/lx s. The signal full scale at the pixel output is 1.8 V at 3.3-V supply and the noise level is measured to be 1.8 mVrms , and the resulting signal dynamic range is 60 dB. Index Terms—CMOS image sensor, column-parallel cyclic A/D converter, global electronic shutter, high-sensitivity, high-speed imaging, in-pixel charge amplifier.

I. INTRODUCTION

A

PPLICATION of high-speed videography extends over various fields such as optical scientific measurements and motion analysis. These applications often require image sensors with high sensitivity and high grey-scale resolution. Recently, the performance of high-speed vide cameras has been greatly improved by the use of CMOS image sensor technology. A high-speed CMOS image sensor with analog parallel outputs reaches 10 000 frames/s in 512 512 pixels and 3000 frames/s in 1024 1024 pixels [1]. Many applications require a small camera head in high-speed cameras. This motivates the development of high-speed image sensors with on-chip analog-to-digital converters (ADCs) and the resulting digital outputs. For high-speed CMOS image sensors, an in-pixel

Manuscript received November 10, 2006; revised December 10, 2006. This work was supported by the Knowledge Cluster Initiative of Ministry of Education, Culture, Sports, Science and Technology. M. Furuta is with the Research Institute of Electronics and the Graduate School of Electronic Science and Technology, Shizuoka University, Hamamatsu-shi, 432-8011 Japan (e-mail: [email protected]). Y. Nishikawa is with the Graduate School of Electronic Science and Technology, Shizuoka University, Hamamatsu-shi, 432-8011 Japan. T. Inoue is with Photoron Limited, Tokyo, 102-0071 Japan. S. Kawahito is with the Research Institute of Electronics, Shizuoka University, Hamamatsu-shi, 432-8011 Japan. Digital Object Identifier 10.1109/JSSC.2007.891655

ADC [2], [3] and a column-parallel successive approximation ADC [4] have been reported. The in-pixel ADC is an effective architecture to exploit the two-dimensional structure of image sensors. However, a number of transistors are required for the in-pixel high-resolution ADC, and it is difficult to meet the compatibility between the high-resolution of the ADC and the small pixel size. High-speed CMOS image sensors with column-parallel successive approximation ADCs have been demonstrated in many developments [5]–[7]. The practical resolution of 8 to 9 bits is achieved. However, these architectures require a high-precision internal digital-to-analog converter, and it is difficult to achieve 10-bit or more resolutions at the column of the CMOS image sensors. In high-speed image sensors, a global electronic shuttering function is indispensable. A simple and efficient technique for the global electronic shuttering has been reported [4]. In this approach, however, the charge-to-voltage conversion gain is reduced if the photodiode size becomes larger. In this paper, a high-speed, high-sensitivity 512 512 CMOS image sensor with a global electronic shutter is presented. As a key technique for reading high-speed pixel signal with sufficient gray-scale resolution, a high-resolution column-parallel cyclic ADC with a built-in noise canceller is developed. A fully differential cyclic ADC using a single amplifier achieves 12-bit resolution in the 3500 frames/s CMOS image sensor implemented with 0.25- m CMOS technology. The proposed pixel uses a charge amplifier to enhance the sensitivity and sample-and-hold circuits for global electronic shuttering. This approach together with on-chip microlens enhances the optical sensitivity while keeping the wide signal dynamic range. II. IMAGE SENSOR DESIGN A. Architecture Fig. 1 shows a block diagram of the high-speed CMOS image sensor. The sensor has an image array, vertical shift register, column readout circuits with 12-bit ADC arrays, column-wise memories for buffering, and controllers. The array of pixels is accessed in standard row-wise fashion so that all pixels in a row are read out into the column ADCs in parallel. Two ADC arrays with noise cancellers are placed at top and bottom of the image array for the pixel outputs of odd and even columns, respectively. The analog references to the top/bottom ADC arrays are supplied by off-chip circuits. The 512 12-bit digitized image data for one row are multiplexed and read out using 160-bit digital output ports. One column data with more significant 10 bits

0018-9200/$25.00 © 2007 IEEE

FURUTA et al.: A HIGH-SPEED, HIGH-SENSITIVITY DIGITAL CMOS IMAGE SENSOR

767

Fig. 3. Timing diagram of pixel.

Fig. 1. Block diagram of the proposed high-speed CMOS image sensor.

Fig. 2. Equivalent circuit of pixel.

in 8 blocks, or 8 10 bits, are read out in parallel in each upper or lower side, and the 32 column data (10-bit each) in each block are first read out using 32 clocks by multiplexing them. The reset of less significant 2 bits in each column, the total of 2 256 bits data, are read out using another 7 clocks. B. Pixel With an Electronic Shutter A global electronic shutter is an indispensable function in high-speed image sensors. In CMOS implementations of highspeed image sensors, the global electronic shutter can be implemented with a sample-and-hold function in each pixel. A simple and efficient electronic shuttering is to use a separation switch between a photodiode and a floating diffusion node based on a

Fig. 4. Operation of the proposed pixel. (a) Signal sampling. (b) Reset sampling.

non-charge-transfer type pixel (3Tr-type) configuration [4]. In this method, the photo responsivity is not efficiently increased even if a larger-size photodiode is used. This is because the larger-size photodiode reduces the charge-to-voltage conversion gain because of the increased photodiode capacitance. The schematic and timing diagram of the proposed pixel is shown in Figs. 2 and 3, respectively. The pixel consists of a charge amplifier, two series-connected sample-and-hold circuits, and source follower buffers. The in-pixel charge amplifier achieves a relatively large charge-to-voltage gain despite of using a large-size photodiode. NMOS transistor switches controlled by a pulse signal are first turned on to reset two floating nodes; a photodiode cathode terminal and a charge amplifier input. When these switches are turned off, photo-charge

768


Fig. 5. Actual schematic of proposed pixel.

accumulation will start. After the photo-charge is accumulated during the time of shuttering, the charge-amplifier output voltage is given by (1) where is the open-loop gain of the operational amplifier, is the operating point of the charge amplifier, is the phois the feedback capacitance of the todiode capacitance, is the coupling capacitance, and is the charge amplifier, input capacitance of the amplifier. If the amplifier open-loop gain is very large, the output signal , is then simplified to component, (2) where is the number of signal electrons. Equation (2) is determined by means that the conversion gain despite of existing large photodiode capacitance. The use of very small capacitance for results in high sensitivity. A is used to set different reset voltages for coupling capacitor the photodiode and the charge amplifier input. This technique allows the higher reset voltage to be set to the photodiode while maintaining a large output swing of the charge amplifier. The resulting signal swing at the pixel source follower output can be as large as 1.8 V at 3.3-V supply. Fig. 4 shows the sample-and-hold operation for signal and reset levels. First, the signal level of the in-pixel charge amby turning the switches plifier is sampled at a capacitor and on. It is hold when the switch controlled by is turned off. For global shuttering opercontrolled by ation, the control pulse for all the rows of the image through , are activated simultaarray, or neously as shown in Fig. 3. Then, the photodiode and charge amplifier are reset through the reset switches controlled by . A sampling capacitor is connected to the amplifier output during the reset operation. After reset switches are turned off, is turned off to hold the reset the switch controlled by level. The two output levels are read out as a differential signal through a couple of source follower buffers. After this sampleand-hold operation for signal and reset levels, which is performed in vertical blanking interval, the readout sequence will start. The readout operation is performed in a row-wise fashion

, and for fixed pattern with pixel selection signal, noise (FPN) canceling. Fig. 5 shows the actual pixel circuit. The proposed pixel consists of one photodiode, twelve transistors and five capacitors. A single-ended cascode amplifier with transistor M3, M4, M5, , and M6 is used for the internal amplifier. The capacitors , and are of MOS structure. The differential signal output is used for reducing the cross-talk noise. C. Column-Parallel Cyclic ADCs Fig. 6 shows the circuit schematic of the cyclic ADC with a built-in noise canceller. It is designed with a fully differential switched-capacitor amplifier and two comparators (1.5-bit ADC) for 1.5-bit-per-cycle algorithm. Although a cyclic or algorithmic ADC has moderate conversion speed [8], two sets of 256-channel cyclic ADC arrays provide in total very high conversion speed. The fully differential circuit brings accurate analog operation and high noise immunity and is suitable for high-resolution ADCs. A wide dynamic range CMOS image sensor integrating a column-parallel cyclic ADC has been reported [9]. The one channel of this ADC consists of three gain stages for noise canceling and a two-stage cyclic A/D conversion. It requires three amplifiers and twelve capacitors. In the proposed method, an amplifier and capacitors are shared for the noise canceller and two sets of the multiplying DACs. The power dissipation and the size can be sufficiently small for the column integration. The amplifier shared column cyclic ADC with a built-in noise canceller, and a modified configuration for this cyclic ADC have been reported [10], [11]. In these ADCs, each 1.5-bit ADC requires a full cycle, and the cycle time and the size of capacitors are constant. The ADC in this paper, to accelerate the read-out speed for the high-speed image sensor, the following three techniques are introduced. 1) Adding another capacitor set, each 1.5-bit ADC is performed in a half cycle. 2) The conversion cycle time and the size of capacitors are reduced in the latter conversion steps for less significant bits (see Figs. 7 and 10). 3) Using a capacitor for input signal sampling, a part of noise canceling operation is overlapped with a part of the A/D conversion in the previous sample (see Fig. 7). The timing diagram of the cyclic ADC with noise cancellation mode is shown in Fig. 7. Fig. 8 depicts the equivalent


769

Fig. 6. Schematic of the cyclic ADC with noise cancelling mode.

Fig. 7. Timing diagram of the cyclic ADC.

circuits for the pixel FPN canceling. In the input signal sampling phase, Fig. 8(a), the differential pixel outputs are conand , and nected to the bottom plates of capacitors the switch controlled by is first turned on to connect the and . The signal level and reset level top plates of and , respectively, through in the pixel are given to the pixel source follower buffers. The difference is sampled by off. In order to fit the pixel turning the switch controlled by signal swing into the full scale range of the ADC, reference and are connected to the bottom plates of signals and , respectively. Four capacitors, , , , and , for cyclic A/D conversion are connected between the input terminal and output terminal of the operational amplifier, and these terminals are short-circuited for charging to zero by , , , and on. In turning the switches controlled by

the charge-transfer phase, Fig. 8(b), the bottom plates of and are connected to the amplifiers, and then the input terminals of two buffer amplifiers in a pixel are short-circuited by on. This operation results turning the switch controlled by in FPN canceling of the charge amplifier and buffer amplifiers. is given by After these operations, the differential output

(3)

, where source voltage when the switch controlled by

,

is the is turned on,

770


Fig. 9. Equivalent circuit for the cyclic ADC mode. (a) Odd phase. (b) Even phase.

Fig. 8. Equivalent circuit for the noise cancellation mode. (a) Input sampling. (b) Charge transfer.

and

is a source-follower gain. Using , and (3) is rewritten as

, ,

(4) where

. In the operation of signal readout, is approximately equal to , and hence is simplified to (5) Equation (5) suggests that a perfect noise canceling of the pixel FPN can be achieved even though the capacitor mismatches be, , , and exist. The dark level is shifted tween to , so that the full scale of the A/D conversion can be used. , The signal component can be amplified by the ratio of in our design. As shown in Fig. 7, the pixel though signal sampling is started at the timing of the 10th cycle of the A/D conversion for the previous sample, to accelerate the total readout speed. At the beginning of the cyclic ADC mode, the bottom plates of and are connected to the output of the amplifier as shown in Fig. 8(b). On and after this phase, two comparators in

a sub-ADC operate every half cycle. In the 1.5-bit/cycle algois used, and is determined rithm, a digital code as (if (if (if

) ) ).

The signed digit values 1, 0, and 1 correspond to the comparator output codes (0,0), (1,0) and (1,1), respectively. An important property of the 1.5-bit/cycle algorithm is , up to that the comparator offset, can be corrected in digital domain [12]. This property of the 1.5-bit/cycle algorithm greatly relaxes the comparator precision and leads to a great reduction of power dissipation of comparators. In the next phase, or odd half-clock phase, Fig. 9(a), the top and are connected to the amplifier, while plates of the bottom plates are connected to a 1.5-bit DAC using the deand cision results of the comparators, and the charges in depending on the DAC output are transferred to and . The bottom plates of and are connected the amand are connected plifier, while the top plates of the and each other. In this phase, the switches controlled by are turned on, and the switches controlled by and are turned off. The amplifier outputs are sampled by and when the switch controlled by is turned off. In the even half-cycle phase, Fig. 9(b), the roles of the capacitor set with and and that with and are exchanged, and the same operation is performed. These two operations shown in Fig. 9(a) and (b) are alternately repeated until the required resolution is obtained. In the th cycle, the ideal output if there is no mismatches in capacitors, and no other errors, is given by


771

Fig. 10. Timing chart of the cyclic ADC.

Fig. 12. Measured photoelectric conversion characteristics. (ADC full scale range: 2 V, frame rate: 1000 frames/s).

Fig. 11. Chip micrograph.

In a cyclic ADC, the resolution is determined by the number of cycles and is independent of the circuit complexity, though the device sizes are carefully designed to meet the required signal-to-noise ratio (SNR) and the linearity. Using this feature, high-resolution of 12-bit and digital gain control function are realized. The temporal resolution of the ADC is designed to be 14 bits using 13 half cycles. The 14-bit digital code is used for digital gain of 2 or 4 when it is fit to the 12-bit output. For accelerating of the conversion speed, the clock speed is increased and the size of sampling capacitors is reduced in the phase of less significant bits as shown in Fig. 10. III. EXPERIMENTAL RESULTS A prototype high-speed CMOS image sensor is designed and implemented. A chip microphotograph of the developed highspeed image sensor with 0.25- m one-poly four-metal CMOS image sensor technology is shown in Fig. 11. The image array has 512 512 pixels with the pixel size of 20 m 20 m. The chip size is 17 mm 12 mm. Two 256 cyclic ADC arrays are integrated at the upper and lower sides of the image array. The

Fig. 13. Measured 12-bit differential nonlinearity of column ADC.

size of one ADC channel is 40 m 2200 m. The designed pixel and cyclic ADC dissipate 6 W per pixel and 0.43 mW per column at supply voltage of 3.3 V and sampling frequency of 2 MHz. Fig. 12 shows photo-conversion characteristics of the image sensor, measured at 1000 frames/s. The output voltage is expressed as 10-bit digital code, and the ADC full scale is 2 V. The sensitivity is as high as 19.9 V/lx s. The sensitivity is two times higher than that of the high-speed image sensor which claim high sensitivity [4], while the measured noise level is comparable. This means that the noise equivalent illumination level is improved by factor of 2. The sensor exhibits a linear response of the output voltage on incident light intensity and the signal full scale at the pixel output is 1.8 V.

772


Fig. 14. Sample image obtained the sensor at: (a) 2500 frames/s, (b) 3000 frames/s, (c) 3500 frames/s, and (d) 4000 frames/s.

Fig. 13 shows the measured linearity of the 12-bit ADC integrated at the column of the developed image sensor without any calibration. The measured 12-bit differential nonlinearity is within 0.81/ 0.72. Random noise is measured by monitoring the signal from a certain pixel under dark condition using

where is a signal of the th frame, is the average of all is the number of frames used for the noise the frame, and measurement. The input-referred noise, including the pixel and column cyclic ADC is measured to be 1.8 mV . Therefore, the signal dynamic range is 60 dB. The nonlinearity of the cyclic ADC is caused by the capacitor mismatch, charge injection due to switch transistors and noises coupling from other channels or circuits. The column-to-column FPN due to the offset deviation of the cyclic ADC is measured to be 0.63% (pp). The FPN can be further reduced if the residual FPN is canceled in digital do-

main using a line memory. The result of DNL measurements of the ADC suggests that the capacitance mismatch is smaller than 0.1%. The column-to-column gain error is caused by the capacitance mismatch, and it is also smaller than 0.1%. The sample images taken by the developed sensor are shown in Fig. 14(a)–(d). Except for digital FPN canceling, no other processing was performed on the image. In the measurement of Fig. 14, a stationary object is captured at high frame rate from 2500 through 4000 frames/s. In this measurement, a prototype camera board is used, and the image data are acquired by using a logic analyzer. Though a logic analyzer is a tool for analyzing logic circuits, it is also useful for a tool for high-speed data capturing. At 4000 frames/s, some parts of the image are degraded, and so the highest frame rate is determined to be 3500 frames/s. Fig. 15 shows four frames from a video sequence at 2000 frames/s. The real high-speed scene of Fig. 15 was captured with an implemented camera system with a large-size video memory, but because of the camera system design, the maximum speed was limited to 2000 frames/s. The performance of the sensor is summarized in Table I.


773

Fig. 15. 2000 frames/s image sequence.

TABLE I PERFORMANCE SUMMARY

cyclic ADCs has been described. The proposed pixel with a charge amplifier and sample-and-hold circuits achieves the high-sensitivity and the global electronic shuttering. The column-parallel 12-bit cyclic ADC with the noise canceling function integrated at the column has a sufficient conversion speed and linearity. The fully differential pixel signal readout provides high noise immunity, and accurate canceling of the pixel-related FPN. The fabricated chip in 0.25- m CMOS imager technology achieves the full frame rate in excess of 3500 frames/s. REFERENCES [1] “Ultima APX-RS,” Photron Limited [Online]. Available: http://www. photron.com [2] D. X. D. Yang, B. Fowler, and A. E. Gamal, “A Nyquist-rate pixel-level ADC for CMOS image sensors,” IEEE J. Solid-State Circuits, vol. 34, no. 3, pp. 348–356, Mar. 1999. [3] S. Kleinfelder, S. Lim, X. Liu, and A. E. Gamal, “A 10,000 frames/s CMOS digital pixel sensor,” IEEE J. Solid-State Circuits, vol. 36, no. 12, pp. 2049–2059, Dec. 2001. [4] A. I. Krymski and N. Tu, “A 9-V/lux-s 5000-frames/s 512 512 CMOS sensor,” IEEE Trans. Electron Devices, vol. 50, no. 1, pp. 136–143, Jan. 2003. [5] A. Krymski, D. Van Blerkom, A. Andersson, N. Bock, B. Mansoorian, and E. R. Fossum, “A high speed, 500 frames/s, 1024 1024 CMOS active pixel sensor,” in Proc. VLSI Circuits Symp., Jun. 1999, pp. 137–138.

2

IV. CONCLUSION In this paper, a high-speed, high-sensitivity digital CMOS image sensor with a global shutter and 12-bit column-parallel

2

774


[6] B. Bansoorian, H. Y. Yee, S. Huang, and E. R. Fossum, “A 250 mW, 60 frames/s 1280 720 pixel 9 b CMOS digital image sensor,” in IEEE ISSCC 1999 Dig. Tech. Papers, Feb. 1999, pp. 312–313. [7] A. Krymski and K. Tajima, “CMOS image sensor with integrated 4 Gb/s camera link transmitter,” in IEEE ISSCC 2006 Dig. Tech. Papers, Feb. 2006, pp. 504–505. [8] K. Nagaraj, “Efficient circuit configurations for algorithmic analog to digital converters,” IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 40, no. 12, pp. 777–785, Dec. 1993. [9] S. Decker, R. Daniel, K. Brehmer, and C. G. Sodini, “A 256 256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output,” IEEE J. Solid-State Circuits, vol. 33, no. 12, pp. 2081–2091, Dec. 1998. [10] M. Mase, S. Kawahito, M. Sasaki, Y. Wakamori, and M. Furuta, “A wide dynamic range CMOS image sensor with multiple exposure-time signal outputs and 12-bit column-parallel cyclic A/D converters,” IEEE J. Solid-State Circuits, vol. 40, no. 12, pp. 2787–2795, Dec. 2005. [11] J. H. Park, M. Mase, S. Kawahito, M. Sasaki, Y. Wakamori, and Y. Ohta, “A 142 dB dynamic range CMOS image sensor with multiple exposure time signals,” in Proc. 2005 ASSCC, Nov. 2005, pp. 85–88, A2L-3. [12] S. H. Lewis and P. R. Gray, “A pipelined 5-MSample/s 9-bit analog-to-digital converter,” IEEE J. Solid-State Circuits, vol. 22, no. 6, pp. 954–961, Dec. 1993.

2

2

Masanori Furuta (S’01–M’04) was born in Mie, Japan, in 1975. He received the B.E. and M.E. degrees in information and computer sciences from Toyohashi University of Technology, Toyohashi, Japan, in 1998 and 2000, respectively, and the Ph.D. degree from Shizuoka University, Hamamatsu, Japan, in 2004. Currently, he is a Research Associate in the Research Institute of Electronics, Shizuoka University, Hamamatsu, Japan. His research interest is in high-speed CMOS image sensor and high-speed low-power A/D converter design.

Yukinari Nishikawa received the B.E. degrees in the advanced course of electronic-mechanical engineering from Suzuka National College of Technology, Suzuka, Japan, and the M.E. degree from Shizuoka University, Hamamatsu, Japan, in 2002 and 2004, respectively. He is currently pursuing the D.E. degree at Shizuoka University. His research interests are in image compression algorithm, and digital circuit design for imaging devices and systems. Mr. Nishikawa is a member of the Institute of Image Information and Television Engineers.

Toru Inoue (M’05) was born in Kanagawa, Japan, in 1974. He received the B.E. and M.E. degrees in applied eco-technology from Toyohashi University of Technology, Toyohashi, Japan, in 1998 and 2000, respectively. His Master research focused on the gate stack technologies of advanced MOSFETs. Since 2000, he has been an Analog IC Designer with Photron, Ltd., Product Development Group, Tokyo, Japan. His current research interest is in the development of high-speed CMOS image sensors.

Shoji Kawahito (M’86–SM’00) was born in Tokushima, Japan, in 1961. He received the B.E. and M.E. degrees in electrical and electronic engineering from Toyohashi University of Technology, Toyohashi, Japan, in 1983 and 1985, respectively, and the D.E. degree from Tohoku University, Sendai, Japan, in 1988. In 1988, he joined Tohoku University as a Research Associate. From 1989 to 1999, he was with Toyohashi University of Technology. From 1996 to 1997, he was a Visiting Professor at ETH, Zurich. Since 1999, he has been a Professor with the Research Institute of Electronics, Shizuoka University. His research interests are in mixed analog/digital circuit design for imaging and sensing devices and systems. Prof. Kawahito received the Outstanding Paper Award at the 1987 IEEE International Symposium on Multiple-Valued Logic, the Special Feature Award in LSI Design Contest at the 1998 Asia and South Pacific Design Automation Conference, Beatrice Winner Award at the 2005 IEEE International Solid-State Circuits Conference. He is a member of the Institute of Electronics, Information and Communication Engineers of Japan, the Institute of Image Information and Television Engineers of Japan, the International Society for Optical Engineering, and a senior member of the IEEE.