Enhanced Angle Sensitive Pixels forLight Field Imaging

Sriram Sivaramakrishnan, Albert Wang, Patrick R. Gill and Alyosha MolnarSchool of Electrical and Computer Engineering, Cornell University

Ithaca, NY 14853Email: ss2462@cornell.edu

Abstract—Previously demonstrated angle sensitive pixels(ASPs) have been shown to enable integrated digital light-field imaging in CMOS, but suffer from significantly reducedpixel quantum efficiency and increased sensor size. This workdemonstrates ASP devices that use phase gratings and a pair ofinterleaved diodes to double pixel density and improve quantumefficiency by a factor of 4.

I. INTRODUCTION

A conventional image sensor captures only a map of lightintensity at the image plane. This two dimensional capturemethod often requires substantial downstream processing oradditional imaging hardware to extract three dimensionalinformation about visual scenes. For instance, active 3-D imag-ing techniques such as time-of-flight [1] require coordinatingillumination with image sensing to extract information aboutdepth. Visual information is represented more completely asa collection of directed light rays in space constituting a 4Dlight field [2]. A light field image sensor simplifies recoveryof 3-D spatial information by passive capture of both the localintensity and incident angle of light.

We have previously demonstrated pixel-scale photodetectorssensitive to both the intensity and local incident angle of thelight they see (Fig. 1). These angle-sensitive pixels (ASPs)form the basis of a monolithic, single-chip light-field digitalimage sensor. Arrays of ASPs have been used for lensless3D localization of microscale objects [3] as well as post-capture range finding and computational refocus [4] in digitalphotography.

Angle sensitive pixels employ a pair of metal diffractiongratings placed above photodiodes to achieve sensitivity toincident angle and have been fabricated in a standard CMOSmanufacturing process. This structure incurs a significant costin terms of physical pixel size and quantum efficiency (QE).This work demonstrates an ASP which addresses both of thesedrawbacks without compromising basic function or CMOSmanufacturing compatibility. The proposed pixel structureimplements phase gratings in back-end dielectric on top ofpairs of interleaved diodes. This design improves QE of ASPsby a factor of 4 and doubles pixel density.

II. BACKGROUND

Current ASP designs consist of a standard photodiodeplaced beneath two sets of diffraction gratings made usingdifferent interconnect metalization layers as shown in Fig. 2.

Identical intensity information

Sharp image, undefined angle

blurred image, converging angles

blurred image, diverging angles

Distinct angle information

Sharp image,undefined angle

Blurred image,converging angle

Blurred image,diverging angle

Identical intensity information,distinct angle information.

Fig. 1. Angle sensitive pixels (ASPs) extract local intensity and angle ofincidence, both of which are used here to determine the depth of the objectrelative to the focal plane.

PassivationInter-metal Dielectric

(SiO2)

n well p sub

Metal1

M 6

M 5

M 4

M 3

M 2 Analyzer grating

Amplitude grating

d

d/2(Type I)

Fig. 2. Schematic cross-section of a reported ASP with grating pitch d (∼1μm) and grating offset of d/2 which corresponds to ASP phase, α = π.The amplitude grating that generates self-images and the analyzer grating areimplemented in CMOS back-end metalization layers.

Upon illumination, the top grating generates strong intensitypatterns at half integer multiples of the Talbot depth [5],zT = 2d2/λ for incident light of wavelength λ. These patternshave periodicity identical to the grating pitch, d. As theincident angle of light changes, these Talbot patterns shift

8.6.1 IEDM11-191978-1-4577-0505-2/11/$26.00 ©2011 IEEE

in response. The function of the second (analyzer) grating isto selectively transmit the diffracted light to the photodiodebelow it. As shown in Fig. 3a light passes through to thephotodiode only when the intensity peaks align with gaps inthe analyzer grating. This produces the characteristic angledependent output response shown in Fig. 3b.

The output of an ASP is modeled by the simple equation:Vout = I0(1 + mcos(βθ + α)) where I0 and θ are incidentintensity and angle, and m, β and α are set by the geometryof the gratings [3]. The parameter m, referred to as the mod-ulation depth, represents the strength of the angular responsewith reference to the intensity response. It takes values in therange 0 ≤ m ≤ 1. Larger values of m provide strongerdirectional selectivity. For a given top grating design, themaximum modulation depth is obtained when the analyzergrating is placed at depths, z = nzT

2 , where n is a positiveinteger. For this condition, the angular sensitivity, β, is givenby:

β =2πz

ηd(1)

where z is the grating separation, η is the refractive index ofthe inter-layer dielectric and d is the grating pitch. The phaseof the ASP response, denoted by α, is set by the relative offsetof the two gratings.

Information about the angle of incident light can be ex-tracted from the lateral shifts in the diffraction pattern. How-ever, the analyzer grating of a single ASP samples only onephase of the periodic intensity pattern. So the output of a singleangle-sensitive pixel cannot distinguish between changes inintensity and incident angle. We require the differential signalfrom a pair of ASPs whose phases (α’s) differ by π to un-ambiguously recover angle information. Demonstrated angle-sensitive pixel arrays [3] have used four ASPs, where eachASP’s response has identical m and β parameters but distinctvalues for α (α = 0, π/2, π, 3π/2) to obtain a full quadraturedescription of incident angle.

Since several ASPs are required to completely characterizeangle, angle information is captured at a significantly reducedspatial resolution as compared to intensity information. Inaddition, the metal gratings used to achieve angle sensitiv-ity block a significant fraction of incident light from thephotodiode. As a result, the quantum efficiency of reportedASP devices is 6-10 times less than an equivalent, exposedphotodiode without gratings. This reduced sensitivity limitsthe usefulness of angle-sensitive pixels in low-light and high-speed imaging applications.

III. DEVICE STRUCTURE AND FABRICATION

Incident angle shifts can be detected more efficiently bysampling multiple phases of the intensity pattern. This can bedone using multiple diodes to detect different portions of theintensity pattern. We choose a grating pitch that generates aTalbot pattern at the surface of the substrate and place a pairof interleaved diodes below the top grating. The outputs of thisdiode pair record complementary phases of the Talbot pattern.This structure, therefore, enables the direct capture of angle

(a) (b)

Fig. 3. (a)An FDTD simulation for two angles of incidence showing shiftin diffraction pattern and associated selective transmission of light by theanalyzer grating. (b) Simulated outputs of ASPs with four different phases.

(a)

(b)

P substrate

N+ diffusion

(a)Amplitude gratings over

interleaved diodes (Type II)

P substrate

Fig. 4. (a) Top view and cross section of a pair of interleaved diodes. (b)An ASP that uses interleaved diodes instead of an analyzer grating and ann-well diode to detect complementary phases of a single diffraction pattern.

information while eliminating the lower metal grating. Fig. 4shows the Finite-difference time-domain (FDTD) simulationand schematic structure of an ASP that uses a pair of inter-leaved N+/p-substrate diffusion diodes.

The function of the top grating in an ASP is to generatea periodic intensity pattern sensitive to the angle of incidentlight. Amplitude gratings used in existing designs block morethan half of the incident light, degrading quantum efficiency.We can greatly increase the amount of transmitted light byreplacing the metal gratings by phase gratings. Phase grat-ings have been studied previously in the context of TalbotArray Illuminators [6]. They have been shown to generateintensity patterns with identical distribution and periodicityas the grating itself. These self-images occur at depths zT /4and 3zT /4 for gratings with a phase step of π/2 and 50%duty cycle [7]. Fig. 5 shows two ASP designs with periodic,rectangular binary phase structures implemented in the inter-metal dielectric layer stack. FDTD simulations of one of thefabricated grating designs, shown in Fig. 6, show that theFresnel diffraction patterns generated by phase gratings shiftlaterally with incident angle.

We fabricated phase gratings by simple post processing ofmetal gratings using methods similar to those used in CMOS

8.6.2IEDM11-192

(c) (d)

(a) (b)

n well p sub

Metal1

M 6

M 5

M 4

M 3

M 2

n+ p sub

Metal1

M 6

M 5

M 4

M 3

M 2

Phase gratingsover

Analyzer gratings(Type III)

Phase gratingsover

interleaved diodes(Type IV)

Fig. 5. (a) ASP structure with phase gratings instead of amplitude gratings.(b) ASP using both phase gratings and interleaved diodes.

Fig. 6. Simulation of diffraction in a phase grating for two angles ofincidence. Patterns shift laterally similar to amplitude gratings.

MEMS [8]. The process flow (Fig. 7) requires only a singlecrude mask step to protect bond pads and two etch steps.First, both the CMOS passivation and inter-metal oxide wereetched anisotropically using a CHF3/O2 plasma. The durationof this etch sets the phase step of the grating. The second etchremoves metalization using a Cl2/BCl3 dry etch followed by ashort wet etch to remove aluminum and a wet NH4OH/H202

etch to remove TiN. This processing was performed on astandard 180nm CMOS die, and is easily scalable to a fullsized imager array. A scanning electron microscopy (SEM)image of a fabricated 0.6μm pitch phase grating is shown inFig. 8. The gratings have a depth of 220nm and a duty cycle of30%. The grating deviates from the designed 50% duty cyclebecause the width of the metal gratings used as etch maskswere not corrected for their trapezoidal cross-section.

M 6

Oxide Etch

Al Etch

(a) (b) (c)

Fig. 7. Two-step process flow for fabrication of phase gratings fromamplitude gratings.

d=0.6μm

t=0.2μm

Fig. 8. SEM image of a fabricated phase grating.

IV. RESULTS

We experimentally characterized four variants of ASPslisted in Table I. The angular response of the ASPs weremeasured under green (523nm, spectral half width 20nm)plane wave illumination over an 80o range of incident an-gles. Fig. 9 shows the measured four phase output for eachof the four ASP types. All four ASP structures trace thedesired quadrature angle response. In the case of ASPs withinterleaved photodiodes, complementary phase responses weremeasured from a single 8μm × 8μm pixel. Newer ASPdesigns, types II, III and IV, exhibit modulation depths in therange 0.2-0.3 as compared to amplitude grating ASPs withpeak modulation depths above 0.4. For interleaved diode basedASPs this degradation could be due to stray carriers from thesubstrate regions between and below the interleaved diodes.Deviations from the ideal phase step of π/2 likely result inpoorer modulation of intensity patterns generated by the phasegratings. Further characterization is required to establish theeffect of variations in the grating step.

For each ASP structure, the dependence of output on gratingpitch and position was characterized. Measurements weremade for ASPs with grating pitch, d, ranging from 0.6μm to3.2μm and 4 different vertical grating separations allowed bythe CMOS metal stack. The measured and simulated variationof modulation depth, m, with grating pitch is shown in Fig. 10for an amplitude grating (type I) ASP with analyzer gratingdepth, z = 2.76μm and a phase grating ASP (type III) withanalyzer grating depth, z = 2.33μm. For a desired angularsensitivity of 12, (1) predicts a grating pitch, d = 1.03μm forthe type I ASP and d = 0.87μm for the type III ASP. Thesecalculations are consistent with measured results.

The QE of all the ASPs were measured and normalizedto an nwell-p-substrate diode without gratings. The measuredquantum efficiency of the four variants of ASPs (Fig. 11)confirms that the efficiency loss caused by metal gratingscan be recovered by a combination of phase gratings andinterleaved diodes.

V. CONCLUSION

Three pixel scale structures that respond to incident angleof light are proposed. These devices use operational principlesfrom previous ASP designs to exhibit characteristic angle de-pendent outputs but greatly improve upon quantum efficiency

8.6.3 IEDM11-193

TABLE IFABRICATED ANGLE SENSITIVE PIXEL STRUCTURES

ASP Top Analyzer Photodiode Signalstype Grating grating type /pixel

I Amplitude Amplitude N-well/p-subtrate 1II Amplitude None Interleaved n+/p-subtrate 2III Phase Amplitude N-well/p-subtrate 1IV Phase None Interleaved n+/p-subtrate 2

Type I: Amplitude gratingsover analyzer gratings

Type II: Amplitude gratingsover interleaved diodes

Type III: Phase gratings overanalyzer gratings

Type IV: Phase gratings overinterleaved diodes

(b)

(a)

(c)

(d)

α=0α=π/2

α=3π/2α=π

m=0.38, β=11.7

m=0.22, β=20

m=0.26, β=12.7

m=0.23, β=20

Fig. 9. Measured output of four ASP structures showing all ASP structuresachieve a characteristic angle sensitive output response similar to the currentpublished device [3].

and size. They can be deployed as large arrays manufacturedin standard CMOS for use in light field image sensors.

ACKNOWLEDGMENT

The authors would like to acknowledge Changhyuk Lee andYoon Ho Daniel Lee for help with simulation and design. The

β = 12.86m=0.32d=0.86μm

β = 11.5m=0.39d=0.98μm

Fig. 10. Measured variation of modulation depth and angular sensitivity withgrating pitch. The highest modulation depth of m = 0.32 is obtained whengrating pitch, d = 0.86μm.

AmplitudeGratings/analyzergratings

AmplitudeGratings/interleaveddiodes

PhaseGratings/analyzergratings

PhaseGratings/interleaveddiodes

I II III IV

Fig. 11. Measured quantum efficiency of four ASP variants, relative to aconventional active CMOS pixel with an n-well/p-substrate photodiode.

project was supported by grant number 5R21EB009841 fromNIH and performed in part at the Cornell NanoScale Facility,a member of NNIN, supported by NSF (Grant ECS-0335765).

REFERENCES

[1] C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design andcharacterization of a cmos 3-d image sensor based on single photonavalanche diodes,” Solid-State Circuits, IEEE Journal of, vol. 40, no. 9,pp. 1847 – 1854, Sept. 2005.

[2] M. Levoy and P. Hanrahan, “Light field rendering,” in Proceedings of the23rd annual conference on Computer graphics and interactive techniques,ser. SIGGRAPH ’96. New York, NY, USA: ACM, 1996, pp. 31–42.

[3] A. Wang, P. Gill, and A. Molnar, “Light field image sensors based on thetalbot effect,” Appl. Opt., vol. 48, no. 31, pp. 5897–5905, Nov 2009.

[4] ——, “An angle-sensitive CMOS imager for single-sensor 3D photog-raphy,” in Solid-State Circuits Conference Digest of Technical Papers(ISSCC), 2011 IEEE International, Feb. 2011, pp. 412 –414.

[5] H. F. Talbot, “Facts relating to optical science,” Philosophical Magazine,vol. 9, no. 56, p. 401, 1836.

[6] A. W. Lohmann and J. A. Thomas, “Making an array illuminator basedon the talbot effect,” Appl. Opt., vol. 29, no. 29, pp. 4337–4340, Oct1990.

[7] T. J. Suleski, “Generation of lohmann images from binary-phase talbotarray illuminators,” Appl. Opt., vol. 36, no. 20, pp. 4686–4691, Jul 1997.

[8] G. Fedder, S. Santhanam, M. Reed, S. Eagle, D. Guillou, M.-C. Lu, andL. Carley, “Laminated high-aspect-ratio microstructures in a conventionalcmos process,” Sensors and Actuators A: Physical, vol. 57, no. 2, pp. 103– 110, 1996.

8.6.4IEDM11-194