Digital: Converting Light to Data

SensorsSimplified

•  Cameras’Sensors(andFilm)whereRaysofLightbecomePixels

ObjecBves

•  PhotographicProcessesforDigitalCapture•  TheFiveSimplifiedlayersofaCCD•  TwobenefitsofusingtheCameraRaw

Reading:hHps://en.wikipedia.org/wiki/Charge-coupled_device

Digital:ConverBngLighttoData•  CCD:Charge-CoupledDevice,adeviceforconverBngelectricalcharge,intoadigitalvalue

•  Pixelsarerepresentedbycapacitors,whichconvertandstore(accumulates)incomingphotonsaselectroncharges

•  WillardBoyleandGeorgeE.Smith,1969(WonaNoblePrizeinPhysicsin2009).

© 2015 Irfan Essa, Georgia Tech, All Rights Reserved

Digital: Converting Light to Data

CCD: Charge-Coupled Device, a device for converting electrical charge, into a digital value

Pixels are represented by capacitors, which convert and store incoming photons as electron charges

Willard Boyle and George E. Smith, 1969 (Won a Noble Prize in Physics in 2009). Smith and Boyle

commons.wikimedia.org



CCD: Charge-Coupled Device, a device for converting electrical charge, into a digital value

Pixels are represented by capacitors, which convert and store incoming photons as electron charges

Willard Boyle and George E. Smith, 1969 (Won a Noble Prize in Physics in 2009). Smith and Boyle

commons.wikimedia.org

hHps://en.wikipedia.org/wiki/Charge-coupled_device

•  Microlenses:CaptureLightanddirectittothelightsensiBveareasofthesensor.–  AddedLenslayer

•  HotMirror:SimpleFiltering.Letsvisiblelightpass,butreflectslightsintheinvisiblepartofthespectrum(dependsonkindoflighttocapture–e.g.,infrared,ultraviolet-UVcamera).AlsoprovidesAnB-aliasing.

•  ColorFilter:BayerArrayseparatelightintoRGB.

•  PhotoDiodes:ColorBlind.ButmeasureintensiBes,andenergyconvertedtoelectrons

•  Deple8onLayer:WhereElectronArecollected(partofphotodiodeslayer)

Digital:ConverBngLighttoData

•  MicrolensArray:CaptureLightanddirectittothelightsensiBveareasofthesensor.– AddedLenslayer

Digital:ConverBngLighttoData

DualMicrolensDesignsShrinksPixelSizeSonyCCD

HotMirror

•  CutsIRlight.•  Hotfilter,thatFiltersoutinfraredlighttomiBgatesensorsthataresensiBvetotheinfrareds(750nm1250),andcontaminatedcolors.–  E.g.,Fires

•  ColdFilter–ColdMirrorfiltersIRintothepahtway(morecommoninfrontoflens.

•  BayerFilteronasensor

•  Pixel:– squareof4colorsquares.

•  Letsonly1colorlightthrough.– 3PaHernsoneforeachcolor

– CombinaBongivesusthefinalimage



Images: Wikipedia

Bayer filter on a sensorCombine

BayerFilter

ConvertLighttoData


Raw input in Bayer mosaic format RGB color planes


?

ConvertBayertoRGB

•  R=200•  G=170,180•  B=153

ConvertBayertoRGB

•  R=200•  G=AVG(170,180)=175

•  B=153

BayerFilter

½Green

1/4Blue

1/4Red

WiththismethodsensoronlycapturesRawcolordatafor1/3oftheimage.UseDemosaicinordertoesBmatetheremaining2/3oftheimage.

Demosaic•  CombineBayerFilter&Interpola8ontoesBmateothercolorsfromtheincompletesamplebyBayerFilterMechansism

•  BayerFilter

hHps://en.wikipedia.org/wiki/Demosaicing

hHp://www.csee.wvu.edu/~xinl/papers/demosaicing_survey.pdf

OtherFilters

•  Foveon:stacksphotodiodestocapturealllightatasinglelocaBon.

•  FujifilmSuperCCD(moreneighbors)àClaimsmoredatatoworkwithforinterpolaBon

cross over small amounts of other colors beyond those in their chromic make-up.This allows crossover through the green filter meaning that energies from the red andblue spectral areas will affect the green sites on the sensor. The second reason forhaving more green filters than red or blue is that blue is the noisiest channel andrestricting its use produces a cleaner image.

Another way to capture the color is via the Foveon X3® chip. This sensor usesa natural attribute of silicon to separate the colors. Depending on wavelength, a colorwill penetrate to varying depths into silicon, similar to what occurs with color film.This factor, along with the stacking of three layers of sites, allows the sensor to capturethe light’s color at each stack of photoreceptors. While theoretically there should beno crossover, there is some absorption of colors as the different wavelengths travelthrough succeeding sensitivity layers.

Sensors are also manufactured that do not have color capture filters or do notacquire color. These cameras are designed to produce monochromatic, black and white images or are used with tricolor photography. Using a monochro-matic sensor to produce tricolor photography, filters are placed in the light path to

36 s e n s o r s

Figure 3-5 Arrangement of filters for

photocells for a matrix or Bayer array.

Figure 3-6 and Figure 3-7 The Foveon X3®

chip stacks photodiodes to capture all light

at a single location. The sensitive areas vary

as shown in the colorized patent drawing.

ch03.qxd 12/4/04 02:42 PM Page 36

hHp://www.vividlight.com/arBcles/2514.htm

CameraRAWFileFormat

•  Containsminimallyprocesseddatafromthesensor

•  Imageencodedinadevice-dependentcolorspace•  CapturesradiometriccharacterisBcsofthescene•  Viewableimagefromthecamera'ssensordata•  LikeaphotographicnegaBve– Hasawiderdynamicrangeorcolor;preservesmostoftheinformaBonofthecapturedimage

Documents

Digital: Converting Light to Data