View
229
Download
10
Embed Size (px)
Pixels,Numbers,andProgramsComputational PhotographyComputationalPhotography
StevenL.Tanimoto
Pixels,Numbers,andPrograms;S.Tanimoto 1ComputationalPhotography
Outline
Wh t i t ti l h t h ?Whatiscomputationalphotography?AutofocustechniquesHi h d i (HDR) h hHighdynamicrange(HDR)photographyCatadioptric camerasSeparatingimagesduetodirectandindirectlightingOtherdevelopments
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 2
WhatisComputationalPhotography?
Computational photography is an emerging technology forComputationalphotographyisanemergingtechnologyforacquiringimagesthroughacombinationofoptics,sensors,andcomputers.
Itincludesautomaticcameracontrol,computationofpixelvaluesusingmultiplesourcesofinformation,aswellasg p ,computingalternativerepresentationsofvisualinformationto2Dimages(e.g.,3Dmodels).
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 3
Autofocustechniques
Focusing a camera lens was once a tedious chore.Focusingacameralenswasonceatediouschore.Instant(film)camerasusedfixedfocus,limitingphotographytowelllitscenesandfast(highASA)p g p y ( g )films.Acousticrangefinding wasusedonsomecamerastog f gdeterminethedistancefromthecameratothefirstsurfaceinthemiddleofthefieldofview.Digitalcameraschangetheeconomicsofautofocus.Now,techniquesbasedonsharpnessmeasurements
ff tiPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography
areeffective.4
Autofocustechniques(cont.)
Brennersharpness:
Tenengrad sharpness:
HereGx andGy arethehorizontalandverticalgradientoperators
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography
x y g pusedintheSobel edgedetector.
5
Autofocustechniques(cont.)
x
Frame#1 Frame#10
Sharpnessasafunctionoffocussetting.Bestfocus
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 6
gisatthe10th frame.
HighDynamicRangePhotography
The dynamic range of a photo or camera is the rangeThedynamicrangeofaphotoorcameraistherangeofbrightnesslevelsthatitcanaccuratelycapture.Historically,dynamicrangewaslimitedby(a)filmy, y g y ( )technology,and(b)digitallightsensortechnology.
Thephotographerschallenge:takeapictureofascenethatcontainsbothbrightanddarkregions,withoutlosingthedetailsinoneoftheseregions.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 7
HDR(cont.)
The brightly lit area is capturedThebrightlylitareaiscapturedeffectivelyinthisphoto(takenwitha1/60sec.exposure).
Butthelowerhalfofthephotop(theindoorpartofthescene)istoodarkandlackingincontrast.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 8
HDR(cont.)
Here, the darkest areas areHere,thedarkestareasarecapturedeffectively(takenwitha15sec.exposure).
Buttheupperhalfofthephotopp p(theoutdoorpartofthescene)istoobrightandlackingincontrast.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 9
HDR(cont.)
Now, both areas have someNow,bothareashavesomevisiblestructure(takenwitha1/2sec.exposure).
Butneithershowsdetailswell.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 10
HDR(cont.)
HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherangerange.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 11
HDR(cont.)
HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherange Th id l i i i frange. Theidea:getluminosityinfoabouteachpixelfromeachimage,butweighttheevidenceaccordingtowhateachimageisgood at
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography
goodat.
12
Catadioptric Imaging
Using optical systems that combine lenses andUsingopticalsystemsthatcombinelensesandmirrors,anamorphicimagescanbecapturedthatcontainentirepanoramas.Computerscaninverttheanamorphicdistortion.
CourtesyofProf.ShreeNayar ofColumbiaUniversity.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 13
Catadioptric Imaging
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 14
Catadioptric Cameras
TheOneshot360(RemoteReality,Inc.).Threeothercatadioptric cameras.http://www.cs.columbia.edu/CAVE/projects/cat_cam_360/
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 15
SeparatingDirect&IndirectLighting
Thecolorofanobject(specificallyatapointPontheobject)inascenedepends on two different physical effects:dependsontwodifferentphysicaleffects:a. Directreflectionoflightfromthelightsourceoffthesurfaceofthe
object(atpointP),andb Light coming from point P on the object that did not come directlyb. LightcomingfrompointPontheobjectthatdidnotcomedirectly
fromthelightsource:i.lightthathascomefromthelightsourcebutbouncedoffother
surfaces before getting to PsurfacesbeforegettingtoP.ii.lightthathasenteredthesurfacematerial(e.g.,thepaintlayer,
whichistypicallyadielectricmaterialandispartiallytranslucent)elsewhere from point P but emerges from point PelsewherefrompointPbutemergesfrompointP.
Weareoftenunawareof(ii).However,usingcomputationalphotography,we can separate the effects of (i) and (ii) The results are amazing
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography
wecanseparatetheeffectsof(i)and(ii).Theresultsareamazing.
16
Direct&Indirect(cont.)
indirectlightsource surfacesin
thescene
P
camera
Rt
RadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 17
Direct&Indirect(cont.)
indirectlightsource
BarriertocastashadowonP
P
camera
Rg
RadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.By measuring R and R then we can compute R as R R
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 18
BymeasuringRt andRg thenwecancomputeRd asRt Rg
ResultsofSeparation
Byusingstructuredlight(projectionofcheckerboards),itisnotnecessaryto have a separate shadow image for each pixel. In theory, only 2 imagestohaveaseparateshadowimageforeachpixel.Intheory,only2imagesareneeded,butinpracticeitsbesttohave10to20images,accordingtoKrishnanandNayar.
Rt Rd Rg
In this example most of the radiance from global illumination is due toInthisexample,mostoftheradiancefromglobalilluminationisduetointerreflectance amongtheeggs.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 19
ResultsofSeparation
AscenewithgrapesandcheeseMuch of the natural color (as seen in R ) comes from R and not RMuchofthenaturalcolor(asseeninRt )comesfromRg andnotRd .
Rt Rd Rg
Here,muchoftheradiancefromglobalilluminationisduetolightpassingthroughthetranslucentgrapesorcheese.Thecheese,shownwithdirectilluminationonly,issomewhatunappetizing.Thegrapesalsoappearblue.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography
Applications:imageenhancement,imageanalysis.20
OtherDevelopments
3Dcameras cameratakestwoormoreshotsofsamescene,then builds 3D model of scenethenbuilds3Dmodelofscene.Redeyereduction.Alreadystandardistheautomaticuseofapreflashtoconditionthepupilsofphotosubjectstoclosedown.Redeyecanalsobereducedusingfacedetectionmethodsandspecializedalgorithms.Light field cameras By capturing a sample of the complete lightLightfieldcameras.Bycapturingasampleofthecompletelightfieldattheviewpoint,operationssuchasfocusingonspecificobjectscanbeperformedlater.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 21
OtherDevelopments(cont.)
IncameraHDR.Automaticexposurebracketingisleadingtheway to built in HDR capturewaytobuiltinHDRcapture.Motionblurinversion.Thecameracanestimateitsmotionusingacombinationofaccelerometerdataandvideodata.Themotioninfocanthenbeusedtopartiallyinvertthemotionblur.Allsmilessnapshots.Usingfacedetection,smiledetectionalgorithms and inpainting techniques it is possibly to synthesizealgorithms,andinpainting techniques,itispossiblytosynthesizeanimageofagroupofpeopleinwhicheveryoneissmiling,eventhoughineachcomponentshot,therewassomeonenotsmiling.
Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 22
