Elements of Computer Graphics - KAISTvclab.kaist.ac.kr/cs580/slide13-HDR.pdf · 2018-07-02 · 18/07/02 5 Min H. Kim KAIST CS580 Computer Graphics Environment cube maps • During

18/07/02

1

MinH.Kim KAISTCS580ComputerGraphics

CS580:ComputerGraphics

MinH.KimKAISTSchoolofComputing


ElementsofComputerGraphics

2Rendering

MaterialmodelGeometry Light

Virtualphotography

18/07/02

2


HDRENVIRONMENTMAP

3


CapturingEnvironmentMaps•  Photographingalightprobeproducesanenvironmentmaprepresentingincidentradiancefromalldirections.

4 B(w) = 2( w ⋅ n)n − w

18/07/02

3


CapturingEnvironmentMaps

5

•  Howtoremovethecamerafromtheenvironmentmap?


SphericalEnvironmentMap&CubeMap•  Galileo’sTomb

6

http://www.pauldebevec.com/Probes/

18/07/02

4


Environmentcubemaps•  Texturescanalsobeusedtomodeltheenvironmentinthedistancearoundtheobjectbeingrendered.

•  Inthiscase,wetypicallyuse6squaretexturesrepresentingthefacesofalargecubesurroundingthescene.

7


Environmentcubemaps•  Eachtexturepixelrepresentsthecolorasseenalongonedirectionintheenvironment.

•  Thisiscalledacubemap.GLSLprovidesacube-texturedatatype,samplerCubespecificallyforthispurpose.

8

18/07/02

5


Environmentcubemaps•  Duringtheshadingofapoint,wecantreatthematerialatthatpointasaperfectmirrorandfetchtheenvironmentdatafromtheappropriateincomingdirection.

9


Environmentmapshader•  Wecalculateinthepreviouslecture.•  Thisbouncedvectorwillpointpointstowardstheenvironmentdirection,whichwouldbeobservedinamirroredsurface.

•  Bylookingupthecubemap,usingthisdirection,wegivethesurfacetheappearanceofamirror.

10

B(v)

18/07/02

6


Environmentmapshader•  Fragmentshader

11

#version130uniformsampler2DuTexUnit0;invec3nNormal;invec4vPosition;outvec4fragColor;vec3reflect(vec3w,vec3n){returnn*(dot(w,n)*2.0)-w;//bouncevector}voidmain(){vec3normal=normalize(vNormal);vec3reflected=reflect(normalize(vec3(-vPosition)),normal);vec4texColor0=textureCube(uTexUnit0,reflected);fragColor=vec4(texColor0.r,texColor0.g,texColor0.b,1.0);;}

[0,0,0,1]t


Environmentmapshader•  -vPositionrepresentstheviewvector•  textureCubeisaspecialGLSLfunctionthattakesadirectionvectorandreturnsthecolorstoredatthisdirectioninthecubetexturemap.

•  Hereweassumeeye-coordinates,butframechangesmaybeneeded.

12

v

18/07/02

7


Environmentmapshader•  Thiscanbeusedforrefraction.

13


HIGH-DYNAMIC-RANGEIMAGING

MeasureRadianceas2DImages

14

Acknowledgements:someofslidesarecourtesyofProf.PaulDebevec(USC),Prof.AlexeiEfros(CMU),andProf.FredoDurand(MIT)

18/07/02

8


TheProblem

•  Limitofdigitalcamera

15


Problem:DynamicRange

•  Therealworldishighdynamicrange

16

1 1,500

2,000,000,000400,000

25,000

18/07/02

9


10-6 106

10-6 106

Real world

Picture

Low contrast

High dynamic range

Multipleexposurephotography

17CourtesyofF.Durand


10-6 106

10-6 106

Real world

Picture

Low contrast

High dynamic range



18/07/02

10


10-6 106

10-6 106

Real world

Picture

Low contrast

High dynamic range




10-6 106

10-6 106

Real world

Picture

Low contrast

High dynamic range



18/07/02

11


10-6 106

10-6 106

Real world

Picture

Low contrast

High dynamic range




Howdowevaryexposure?•  Options:– Shutterspeed– Aperture–  ISO– Neutraldensityfilter

22

18/07/02

12


Tradeoffs•  Shutterspeed–  Range:~30secto1/4000sec(6ordersofmagnitude)–  Pros:reliable,linear–  Cons:sometimesnoiseforlongexposure

•  Aperture–  Range:~f/1.4tof/22(2.5ordersofmagnitude)–  Cons:changesdepthoffield–  Usefulwhendesperate

23


Tradeoffs•  ISO–  Range:~100to1600(1.5ordersofmagnitude)

–  Cons:noise–  Usefulwhendesperate

•  Neutraldensityfilter–  Range:upto4densities(4ordersofmagnitude)&canbestacked

–  Cons:notperfectlyneutral(colorshift),–  notveryprecise,needtotouchcamera(shake)–  Pros:workswithstrobe/flash,goodcomplementwhendesperate

24

18/07/02

13


HDRimageusingmultipleexposure

•  GivenNphotosatdifferentexposure•  RecoveranHDRcolorforeachpixel

25


Ifweknowtheresponsecurve

•  Justlookuptheinverseoftheresponsecurve•  Buthowdowegetthecurve?

26

Pixel value

scene value

18/07/02

14


Calibratingtheresponsecurve•  Twobasicsolutions–  Varysceneluminanceandseepixelvalues

•  Assumeswecontrolandknowsceneluminance

–  Varyexposureandseepixelvalueforonesceneluminance•  Butnotethatwecanusuallynotvaryexposuremorefinelythanby1/3stop

•  Bestofboth:–  Varyexposure–  Exploitthelargenumberofpixels

27


TheAlgorithm•  Imageseries

28

• 3

• 1 •

2

t = 1/100 sec

• 3

• 1 •

2

t = 1 sec

• 3

• 1 • 2

t = 1/1000 sec

• 3

• 1 •

2

t = 10 sec

• 3

• 1 •

2

t = 1/10 sec

Z = f (H )H = E ⋅ Δtlog(H )= log(E)+ log(Δt)where Z is pixel value, H is exposure,E is radiance, Δt is exposure time.

CourtesyofPaulDebevec

18/07/02

15


Responsecurve

•  Exposureisunknown,fittofindasmoothcurve

29log Exposure

curve

Pixe

l val

ue 3

1

2

log Exposure

Pixe

l val

ue

Assumingunitradianceforeachpixel

Afteradjustingradiancestoobtainasmoothresponse


TheMath

•  Letg(z)bethediscretelogarithmicinverseresponsefunction

•  Foreachpixelsiteiineachimagej,want:

30g(Zij )= log(Ei )+ log(Δt j )

Z = f (H ) → logH = log f −1(Z ) → logH = g(Z )

Z = f (H )H = E ⋅ Δtlog(H )= log(E)+ log(Δt)where Z is pixel value, H is exposure,E is radiance, Δt is exposure time.

18/07/02

16


TheMath•  Solvetheoverdeterminedlinearsystem:

31

log(Ei )+ log(Δt j )− g(Zij )⎡⎣ ⎤⎦2

j=1

P

∑ +λ g ''(z)2z=Zmin

Zmax

∑i=1

N

∑

Dataterm Regularizationterm


TheMath•  Howtoreconstructradiance:

•  Weightingfactorhandlesthenoisearoundjointsofpieces

32

logEi =

w(Zij ) g(Zij )− log(Δt j )( )j=1

P

∑

w(Zij )j=1

P

∑

18/07/02

17


Matlabcodefunction[g,lE]=gsolve(Z,B,l,w) %Zispixelvalue,Bisexposurefactorn=256;A=zeros(size(Z,1)*size(Z,2)+n+1,n+size(Z,1));b=zeros(size(A,1),1);k=1; %Includethedata-fittingequationsfori=1:size(Z,1)forj=1:size(Z,2)wij=w(Z(i,j)+1);A(k,Z(i,j)+1)=wij;A(k,n+i)=-wij;b(k,1)=wij*B(i,j);k=k+1;endendA(k,129)=1; %Fixthecurvebysettingitsmiddlevalueto0k=k+1;fori=1:n-2 %IncludethesmoothnessequationA(k,i)=l*w(i+1);A(k,i+1)=-2*l*w(i+1);A(k,i+2)=l*w(i+1);k=k+1;Endx=A\b; %SolvethesystemusingSVDg=x(1:n);lE=x(n+1:size(x,1));

33

CourtesyofPaulDebevec


Result:digitalcamera•  Reconstructedcameraresponse

34logExposure

Pixelvalue

RecoveredresponsecurveKodakDCS4601/30to30sec

18/07/02

18


Reconstructedradiancemap

35

CourtesyofP

aulD

ebev

ec


Result:colorfilm•  KodakGold,ASA100

36

18/07/02

19


Recoveredresponsecurves

37

Red Green

Blue RGB


TheRadianceMap•  Measuresceneradiancebyusingdigitalcamera

38

18/07/02

20


TheRadianceMap

39

Linearlyscaledtodisplaydevice


HDRimageprocessing

•  Importantalsofordepthoffieldpost-process

40

Motionblurappliedtolow-dynamic-rangepicture

Motionblurappliedtohigh-dynamic-rangepicture

Realmotion-blurredpicture

ImagesfromDebevec&Malik1997

18/07/02

21


SmarterHDRcapture•  Automaticexposurealignment•  Ghostremoval•  Lensflareremoval•  ImplementingHDRincameras

41

ImagesGregWard


Imageregistration

•  Howtorobustlycompareimagesofdifferentexposure?

•  Useablackandwhiteversionoftheimagethresholdedatthemedian– Median-ThresholdBitmap(MTB)

•  Findthetranslationthatminimizesdifference

•  Accelerateusingpyramid

42

18/07/02

22


Alignmentresult

43


HDRVideo

•  GenerateHDRvideobyrapidlyvaryingtheexposureofeachframe(automaticexposurecontrolandstitchingneighboringframes)

44

Kangetal.SIG2003

18/07/02

23


Hardwaresolutions

45


HDRImagingHardware

•  Mosaicneutral-densityfilterforspatialvaryingexposureimaging.Fourdifferentexposuresofneutraldensityfilterareinstalledinfrontofthedetectorarray.Thedifferencebetweenneutraldensityise3=4e2=16e1=64e0.NayarandMitsunaga[2000].

46

18/07/02

24


Fileformats•  PortableFloatMap(.pfm)– 4bytes(4x8=32bits)perchannel– 12bytesperpixel(=4bytesx3ch.)– TextheadersimilartoJeffPoskanzer’s.ppmimageformat(FloatingPointTIFFsimilar):

47


Fileformats

•  Radianceformat(.pic,.hdr)– 4bytes(4x8=32bits)perpixel–  (145,215,87,149)=(145,215,87)*2^(149-128)(1190000,1760000,713000)

–  (145,215,87,103)=(145,215,87)*2^(103-128)(0.00000432,0.00000641,0.00000259)

48Ward,Greg."RealPixels,"inGraphicsGemsIV,editedbyJamesArvo,AcademicPress,1994

18/07/02

25


Fileformats

•  ILM’sOpenEXR(.exr)– 2bytes(2x8=16bits)perchannel(2x3=6bytes(16x3bits)perpixel)

– Losslesscompressionsupport– Half-precisionfloat– 65504(maxhalfprecision)– http://www.openexr.net/– Multi-channelsupported

49


SunnybrookHDRdisplay•  Usebrightsource+two8-bitmodulators•  Transmissionmultipliestogether•  Over10,000:1dynamicrangepossible

50

18/07/02

26


HighDynamicRangeDisplay

51


Howitworks

52

18/07/02

27


WhatifedgecontrastexceedsLCDrange?•  Observerscannottellwhenthishappensbecausetheeyehaslimitedlocalcontrastcapacityduetoscattering

53


HDRDisplay

54

BrightSideDR37-P(nowDolby)

18/07/02

28


Displaytheinformation•  Matchlimitedcontrastofthemedium•  Preservedetails

55

10-6 106

10-6 106

Real world

Picture

Low contrast

High dynamic range


Insightoftonemapping

•  TumblinandRushmeier(1993)’stone-reproductionoperatorcomprisesreal-worldobservations,inversedisplayobservations,andaninversedisplaydevicefunctionthatachievesaperceptualmatchbetweenreal-worldobservationandtheobservationofthereproducedimageonthedisplay.

56

18/07/02

29


Howhumansdealwithdynamicrange?

•  We'resensitivetocontrast(multiplicative)–  Aratioof1:2isperceivedasthesamecontrastasaratioof100to200

– Makessensebecauseilluminationhasamultiplicativeeffect–  Usethelogdomainasmuchaspossible

•  Dynamicadaptation(verylocalinretina)–  Pupil–  Neural/chemical

•  Differentsensitivitytospatialfrequencies

57


ContrastSensitivity•  SineWavegrating•  Whatcontrastisnecessarytomakethegratingvisible?

58

18/07/02

30


ContrastSensitivityFunction(CSF)

•  CampbellRobsoncontrast-sensitivitychart

59

loga

rithm

icdecreaseofcon

trast

logarithmicincreaseofspatialfrequency


ContrastSensitivityFunction(CSF)

•  Lowsensitivitytolowfrequencies

•  Importanceofmediumtohighfrequencies

•  Mostmethodstodealwithdynamicrange,reducingthecontrastoflowfrequencies

•  Butkeepthecolor

60

18/07/02

31


Contrastreduction•  Input:high-dynamic-rangeimage– Floatingpointperpixel

61 CourtesyofF

redo

Duran

d


Naïvetechnique•  Scenehas1:10,000contrast,displayhas1:100•  Simplestcontrastreduction?

62 CourtesyofF

redo

Duran

d

18/07/02

32


Naïve:Gammacompression

•  XàXϒ(whereϒ=0.45inthiscase)•  But...colorsarewashed-out.Why?

63 CourtesyofF

redo

Duran

d


Gammacompressiononintensity

•  ColorsareOK,butdetails(intensityhigh-frequency)areblurred

64 CourtesyofF

redo

Duran

d

18/07/02

33


Oppenheim1968,Chiuetal.1993

•  Reducecontrastoflow-frequencies•  Keephighfrequencies

65 CourtesyofF

redo

Duran

d


Thehaloartifact

•  Forstrongedges•  Becausetheycontainhighfrequency

66 CourtesyofF

redo

Duran

d

18/07/02

34


Whyhaloartifacthappen

67


DurandBilateralFiltering•  Donotbluracrossedges•  Non-linearfiltering

68 CourtesyofF

redo

Duran

d

18/07/02

35


StartwithGaussianfiltering

69CourtesyofFredoDurand




18/07/02

36





Gaussianfilterasweightedaverage


18/07/02

37


Theproblemofedges



PrincipleofBilateralfiltering


18/07/02

38


Bilateralfiltering



Bilateralfiltering


18/07/02

39


Normalizationfactor



Bilateralfilteringisnon-linear


18/07/02

40


OtherView



Handlinguncertainty


18/07/02

41


DurandandDorsey(SIG2002)•  FollowingOppenheimetal,fromtheinputimageand

illuminanceimage,thereflectanceimagecouldbereconstructedbydividingtheinputandilluminanceimage.

•  Thesmoothilluminationiscalled“baselayer”,whereastheresultofthedivisioniscalled“detaillayer”

•  CompressionisdoneinthelogarithmicchannelsoftheluminancechannelintheYxycolorspace

81


Durand’scontrastreduction

82DurandandDorsey(SIG2002)

18/07/02

42







18/07/02

43







18/07/02

44







Documents

Elements of Computer Graphics - KAISTvclab.kaist.ac.kr/cs580/slide13-HDR.pdf · 2018-07-02 · 18/07/02 5 Min H. Kim KAIST CS580 Computer Graphics Environment cube maps • During