SPIE Proceedings [SPIE Electronic Imaging 2004 - San Jose, CA (Sunday 18 January 2004)] Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications

Improving the Sensitivity of a Vision Chip Using theSoftware A-D Conversion Method

Daisuke Takeuchia, Shingo Kagamia, Takashi Komuroa and Masatoshi Ishikawaa

aDepartment of Information Physics and Computing, Graduate School of Information Scienceand Technology, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656 JAPAN

ABSTRACT

We developed a new method of pixel-level Analog-to-Digital (A-D) conversion for vision chips, removing FixedPattern Noise (FPN) at the same time. Vision chips are CMOS image sensors integrating a processing element(PE) and a photodetector (PD) in each pixel. The chip can handle high frame rate images in real time becauseits processing speed is high due to the parallel processing and also because it does not need high-bandwidthcommunication. Pixel-level A-D conversion is an essential technology for vision chips because digital operationsmust be performed in each pixel. The vision chip, which we have developed, contains a programmable PE in eachpixel, and it directly controls the behavior of the PD with the use of the software. In our developed method, thechip controls the reference voltage to cancel the FPN by using this feature. We applied this method to our visionchip, and confirmed that the FPN was reduced and the sensitivity improved. We made a test chip including onlyPDs to solve the problem on the existing vision chip. As a result of applying this method to the test chip, thedetectable minimum illuminance improved about 40 times in comparison with applying our existing method.

Keywords: Vision Chip, CMOS Image Sensor, Pixel-level A-D conversion, Fixed Pattern Noise

1. INTRODUCTION

Digital vision chips are CMOS image sensors integrating a digital processing element (PE) and a photo detector(PD) in each pixel.1 The chip can handle high frame rate images in real time because its processing speed ishigh due to parallel processing and also because it does not need high-bandwidth communication.

Pixel-level A-D conversion is an essential technology for vision chips because digital operations must beperformed in each pixel. The vision chip, which we developed before, contains a programmable PE in each pixel,and it directly controls the behavior of the PD with software.

In the vision chip, the exposure time is short due to the high frame rate and the fill factor is low due tothe area of PE implemented in each pixel. Therefore, it is important to improve the sensitivity. Although theterm, ‘sensitivity’, is commonly known as the ratio of output voltage to input charge, we use it to refer to thedetectable minimum illuminance here.

In this paper, we will first explain the structure of the vision chip we developed. Next, we will propose amethod to remove the FPN of the vision chip. In this developed method, the output pixel values of the visionchip are obtained with the FPN removed. After that, we will describe the experiments that we applied thisdeveloped method to the vision chip. In addition, we will show the result of this developed method applied tothe test chip that includes only PDs. In this chip, the problem on the existing vision chip was solved.

1.1. The Structure of the Vision Chip

Figure 1 shows the structure of the vision chip, which integrates a PD and a PE in each pixel.2 A PE consistsof an arithmetic-and-logical unit and memory, and we can give instructions to the PE. Various kinds of imageprocessing algorithm can be implemented by software.

Further author information: E-mail: Daisuke [email protected], Shingo [email protected],Takashi [email protected], Masatoshi [email protected]; Telephone: +81 3 5841 6937; Fax: +813 5841 6952

Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications V,edited by Morley M. Blouke, Nitin Sampat, Ricardo J. Motta, Proc. of SPIE-IS&T Electronic

Imaging, SPIE Vol. 5301 © 2004 SPIE and IS&T · 0277-786X/04/$15

138

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/17/2013 Terms of Use: http://spiedl.org/terms

Processing Element

(PE)

Photo Detector

(PD)

Input Image Lens

Processing Element

(PE)

Photo Detector

(PD)

Output

Input Image Lens

Processing Element

(PE)

Photo Detector

(PD)

Input Image Lens

Processing Element

(PE)

Photo Detector

(PD)

Output

Input Image Lens

Figure 1. A model of the vision chip.

1.2. The photodetector in the vision chip

The A-D conversion in the pixel from the light intensity to a digital value is indispensable in the vision chipbecause digital processing is performed in the pixel. Figure 2 shows the circuit of the photodetector of the visionchip. The PD consists of a photodiode, a resetting transistor and a comparator.

－

＋

Vdd

Vref

Vout

Photodiode

ResetVin

for PE

Comparator

－

＋

－

＋

－

＋

Vdd

Vref

Vout

Photodiode

ResetVin

for PE

Comparator

Figure 2. The circuit of the photodetector.

The procedure of the A-D conversion is as follows. The reset signal is first turned on and the photodiodevoltage Vin is equated with the power-supply voltage Vdd in Fig. 2. Then the reset signal is turned off. Theseare repeated at regular intervals. Photocurrent I flows when the photodiode receives light, and Vin descends.The speed, at which Vin descends, is proportional to the amount of photocurrent that flows in the photodiode.

Let the time when the reset signal was turned off be t = 0. The relation between Vin and I is given by Eq.(1). The parasitic capacitance of the photodiode is represented as C.

CVdd − CVin = It. (1)

Next, Equation (2) is derived from Eq. (1). Figure 3 shows the behavior that Vin descends as the time passes.

Vin = − I

Ct + Vdd (2)

Vin, which is discharged by the photocurrent I, is compared with the reference voltage Vref . The output ofthe comparator changes from the logical value 0 to the logical value 1, when Vin falls below Vref . The comparator

SPIE-IS&T/Vol. 5301 139


t

V

Vdd

Vref

tmax0

0 ttout tmax

Vin

1

t

V

Vdd

Vref

tmax0

0 ttout tmax

Vin

1

Figure 3. The principle of the A-D conversion.

output is sampled at some rate in the PE after resetting the photodiode, and the light intensity of each pixelcan be converted to a digital value by counting the sampled data using the function of PE.

Instead of keeping the reference voltage constant and sampling the comparator output at regular intervals, wecan regulate the voltage and the intervals dynamically with the software since the pixel operation is programmablycontrolled. We call this concept software A-D conversion. Kagami et al. derived an algorithm to achieve arbitraryconversion scales subject to the condition that the influence of noise is minimized.3 However, their outputssuffered from severe FPN and could not achieve satisfactory sensitivity.

1.3. Noise in CMOS image sensors

The noise in CMOS image sensors can be classified roughly into random noise and fixed pattern noise. FPN isthe variation in the pixel values of the image sensor under uniform illumination. The main cause of the FPN isthe variation of the characteristic of the transistor built in the pixel. The FPN is typically tens of times largerthan the random noise, and it is very important to remove this.

There are many methods of canceling FPN. One method is Correlated Double Sampling (CDS). In thismethod, two samples are taken, the pixel value in the reset state, and the actual signal. Then the two valuesare used as a differential value. However, implementing a circuit for CDS in each pixel makes the area larger.The method of removing FPN without using CDS is needed because the size of the pixel should be as smallas possible. In another method, canceling FPN can be performed by taking the difference between the pixelvalues and the offset values. The offset values are calculated beforehand and stored in the memory. This methodrequires a large amount of memory for storing the whole offset values. Therefore, it is unsuitable to apply thismethod to our vision chip. By using the developed method, we need no additional circuits in the pixel.

2. THE DEVELOPED METHOD

We developed a new method of removing the fixed pattern noise. This method does not require a large amountof memory and needs no additional circuits.

In this developed method, the reference voltage Vref in Fig. 2 is controlled by the software to remove theoffset component of FPN. The offset FPN appears as the variation of the voltage as shown in Fig. 4 when thephotodiode voltage Vin and reference voltage Vref are compared. The variation in the time scale is caused by theFPN when the reference voltage Vref is constant, and the influence appears as shown in Fig. 4.

140 SPIE-IS&T/Vol. 5301


t

V

Vdd

Vref

tmax0

0 ttout tmax

Vin

1

t

V

Vdd

Vref

tmax0

0 ttout tmax

Vin

1

Figure 4. The influence of FPN.

The developed method removes the variation in the time scale caused by the FPN. It is performed by varyingthe reference voltage Vref with time. Vref is ramped up from a minimum voltage to a maximum voltage twicewith the same slope within a frame period tmax. The light intensity at the PD can be estimated by measuring thetime between the first time when Vin drops under Vref and the second time when Vin drops under Vref . Althoughthere is a variation in the offset of the comparator, it does not affect the time described above. Therefore, FPNis reduced.

The detailed procedure of the developed method is as follows. First, the frame period tmax, the maximumreference voltage Vmax and the minimum reference voltage Vmin are given. Next, the time when the secondreference voltage Vref2(t) represented in Eq. (4) is input is given. The value of the time is given by multiplyingtmax by constant r (0 < r < 1). The value of the constant r is set to the value to meet the requirement that Vin

falls below the reference voltage Vref1 represented in Eq.(3) even if there is a variation of the voltage.

Functions Vref1(t) and Vref2(t) represented in Eqs. (3) and (4) respectively are given in terms of the parameterstmax, Vmax, Vmin and r. The reference voltage is supplied according to these functions.

Vref1(t) =Vmax − Vmin

(1 − r)tmax(t − rtmax) + Vmax (0 ≤ t ≤ rtmax) (3)

Vref2(t) =Vmax − Vmin

(1 − r)tmax(t − tmax) + Vmax (rtmax ≤ t ≤ tmax) (4)

The interval (t2 − t1) between the time t1, when Vin drops under Vref1, and the time t2, when Vin dropsunder Vref2, is measured. The variation of the time scale from the influence of the variation of the voltage canbe removed by calculating the value (tmax − (t2 − t1)). Figure 5 illustrates the method.

Let α be the amount of photocurrent normalized by the capacitance of the photodiode. The pixel value P (α)is shown as the following Eq. (5) obtained from Eqs. (2), (3) and (4).

P (α) = (1 − r)tmax

(1 − 1

1 + (1−r)tmaxVmax−Vmin

α

)(5)



t

Vmax=Vdd

V

tmaxrtmax0

Vmin

Vref2

Vref1

Vin

1

0 t1 t2 tmax t

t

Vmax=Vdd

V

tmaxrtmax0

Vmin

Vref2

Vref1

Vin

1

0 t1 t2 tmax t

Figure 5. The developed method. The reference voltage is supplied according to the reference voltage functions Vref1

and Vref2.

3. EXPERIMENTS

We examined the effectiveness of the developed method, in our existing vision chip2 and the test chip includingonly PDs that has been newly designed to improve the performance at the lower illuminance.

3.1. Experiments on our vision chip

The developed method was applied to the vision chip to prove the effectiveness. The vision chip, which Komuroet al. developed, has 64×64 pixels in a 5.4mm×5.4mm area using 0.35µm TLM CMOS process.2 The chipsurface was irradiated flatly by an LED array lighting without lens.

Figure 6 shows the graphs of maximum, average, and minimum of the output pixel values when the referencevoltage Vref was set to a constant 3.0V and when the reference voltage Vref was varied by using the developedmethod. The image was captured in 6-bit gray-scale. The values of the parameters in the developed methodare as follows: tmax = 1.8[ms], Vmin = 2.0[V], Vmax = 3.3[V] and r = 0.2. Figure 7 shows the root-mean-square(RMS) of the output pixel values. The RMS of the output pixel values was not calculated at the illuminancewhere the minimum of the output pixel values is equal to zero. FPN is reduced by using the developed method.However, the RMS of the output pixel values at the lower illuminance is larger than that at the higher illuminance.

Figure 8 shows the images obtained by irradiating the light of the illuminance of 240lx and Figure 9 showsthe images of a wristwatch.

3.2. Experiments on the test chip

In the vision chip used in the experiment, which is described in Section 3.1, the reset voltage Vrs is equal to thepower supply voltage Vdd. Therefore, an output value obtained in lower illuminance that is smaller than a truevalue as shown in Fig. 10(a). To solve the problem, we made a test chip in which the voltage Vrs can be variableas shown in Fig. 10(b). This test chip, which was fabricated in 0.35 µm CMOS process, includes only PDs andintegrates 1×8 pixels. Figure 11 shows the layout of the test chip.

We performed experiments to confirm that the noise was reduced in a low illuminance by using the developedmethod with the voltage Vrs lower than the power-supply voltage. The experiments were performed under thefollowing three conditions: when the reference voltage Vref was set to a constant 2.5V, when the reference voltageVref was varied by using the developed method with the voltage Vrs = Vdd = 3.3[V], and when the referencevoltage Vref was varied by using the developed method with the voltage Vrs = 3.1[V]. The parameters in the



0

10

20

30

40

50

0 100 200 300 400 500 600

digitalized value

illuminance (lx)

"constant"

(a)

0

10

20

30

40

50

0 100 200 300 400 500 600

digitalized value

illuminance (lx)

"developed"

(b)

Figure 6. The A-D conversion characteristics. (a) The reference voltage Vref was set to a constant 3.0V. (b) The referencevoltage Vref was varied by using the developed method.

0

2

4

6

8

10

0 100 200 300 400 500 600

RMS (%)

illuminance (lx)

"constant"

(a)

0

2

4

6

8

10

0 100 200 300 400 500 600

RMS (%)

illuminance (lx)

"developed"

(b)

Figure 7. The RMS of the output pixel values. (a) The reference voltage Vref was set to a constant 3.0V. (b) Thereference voltage Vref was varied by using the developed method.



(a) (b)

Figure 8. These images were captured by irradiating the light at the illuminance of 240lx. (a) The reference voltage Vref

was set to a constant 3.0V. (b) The reference voltage Vref was varied by using the developed method.

(a) (b)

Figure 9. Images of the wristwatch. (a) The reference voltage Vref was set to a constant 3.0V. (b) The reference voltageVref was varied by using the developed method.



t

Vmax=Vdd

V

tmaxrtmax0

Vmin

Vref2

Vref1

Vin

1

0t1 t2 tmax t

false true

t

Vmax=Vdd

V

tmaxrtmax0

Vmin

Vref2

Vref1

Vin

1

0t1 t2 tmax t

false true

(a)

Vin

t

Vmax=Vdd

V

tmaxrtmax0

Vmin

Vref2

Vref1

1

0 t1 t2 tmax t

Vrs

Vin

t

Vmax=Vdd

V

tmaxrtmax0

Vmin

Vref2

Vref1

1

0 t1 t2 tmax t

Vrs

(b)

Figure 10. The influence of the FPN in the case that the reset voltage Vrs is equal to the power-supply voltage Vdd (a),and in the case that Vrs is lower than Vdd (b).

Figure 11. A part of the layout of the test chip.

developed method are set as follows: Vmin = 1.5[V], Vmax = 3.3[V] and r = 0.2. The PD integration time tmax is1ms. The maximum pixel value was set to 1000. The chip surface was flatly irradiated by an LED array lightingwithout a lens.

Figure 12(a) shows the graph of maximum, average, and minimum of the output pixel values when thereference voltage Vref was set to a constant 3.0V. Under this condition, the minimum detectable illuminancewas 338lx. Figure 12(b) shows the graph of maximum, average, and minimum of the output pixel values whenthe reference voltage Vref was varied by using the developed method with the voltage Vrs = 3.3[V]. Under thiscondition, the minimum detectable illuminance was 66lx. Figure 12(c) shows the graph of maximum, average,and minimum of the output pixel values when the reference voltage Vref was varied by using the developedmethod with the voltage Vrs = 3.1[V]. Under this condition, the minimum detectable illuminance was 8.8lx.Figure 13 shows the RMS of the output pixel values. The RMS of the output pixel values was not calculatedat the illuminance where the minimum of the output pixel values is equal to zero. FPN is reduced by using thedeveloped method. Moreover, the RMS when Vref was varied by using the developed method with Vrs = 3.1[V] issmaller at the lower illuminance than that when Vref was varied by using the developed method with Vrs = 3.3[V].

These experimental results show that the minimum detectable illuminance was improved by applying thedeveloped method. In addition, it was confirmed that the method is effective to set the voltage Vrs lower thanthe power-supply voltage.

Based on these experimental results, we are at work on making a new vision chip in which the reset voltageVrs is variable.



0

50

100

150

200

250

300

350

400

0 50 100 150 200 250 300 350 400 450 500

digitalized value

illuminance (lx)

"constant"

(a)

0

50

100

150

200

250

300

350

400

0 50 100 150 200 250 300 350 400 450 500

digitalized value

illuminance (lx)

"Vrs=3.3V"

(b)

0

50

100

150

200

250

300

350

400

0 50 100 150 200 250 300 350 400 450 500

digitalized value

illuminance (lx)

"Vrs=3.1V(proposed)"

(c)

Figure 12. A-D conversion characteristics. (a) The reference voltage Vref was set to a constant 3.0V. (b) The referencevoltage Vref was varied by using the developed method with the reset voltage Vrs = 3.3[V]. (c) The reference voltage Vref

was varied by using the developed method with the reset voltage Vrs = 3.1[V].



0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 50 100 150 200 250 300 350 400 450 500

RMS (%)

illuminance (lx)

"constant2.5V"

(a)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 50 100 150 200 250 300 350 400 450 500

RMS (%)

illuminance (lx)

"Vrs=3.3V"

(b)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 50 100 150 200 250 300 350 400 450 500

RMS (%)

illuminance (lx)

"Vrs=3.1V(proposed)"

(c)

Figure 13. The RMS of the output pixel value.(a) The reference voltage Vref was set to a constant 3.0V. (b) The referencevoltage Vref was varied by using the developed method with the voltage Vrs = 3.3[V]. (c) The reference voltage Vref wasvaried by using the developed method with the voltage Vrs = 3.1[V].



4. CONCLUSION

We developed a new pixel-level A-D conversion method for our vision chip. In this developed method, softwareprograms control reference voltages to cancel FPN, which is the main cause of the low sensitivity of the visionchip. We applied this developed method to the vision chip that we have developed, and confirmed that FPN wasreduced compared with the case that the reference voltage was constant. However, the variation of the outputpixel values at the lower illuminance was larger than that at the higher illuminance. We postulated that thecause was that the reset voltage Vrs was equal to the power-supply voltage Vdd. We made a test chip includingonly PDs, to solve this problem on our existing vision chip. As a result of applying this developed method tothe test chip, the detectable minimum illuminance improved about 40 times in comparison with the case thatthe reference voltage was constant. Further improvement of the developed method and the experiments on theimproved vision chip need to be performed.

REFERENCES1. M. Ishikawa and T. Komuro, “Digital vision chips and high-speed vision systems (invited),” in 2001 Sympo-

sium on VLSI Circuits, pp. 1–4, 2001.2. T. Komuro, S. Kagami, and M. Ishikawa, “A dynamically reconfigurable SIMD processor for a vision chip,”

IEEE Journal of Solid-State Circuits Vol.39, No.1, 2004(to appear).3. S. Kagami, T. Komuro, and M. Ishikawa, “A software-controlled pixel-level A-D conversion method for digital

vision chips,” in 2003 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, 2003.



Documents

SPIE Proceedings [SPIE Electronic Imaging 2004 - San Jose, CA (Sunday 18 January 2004)] Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications