PowerPoint Presentation

776 Computer VisionJan-Michael FrahmFall 2015

Camera

Camera

object point emits light in all directions

put a sensor to capture the image

block most of the rays with a barrier called aperture

sensor (CCD, CMOS, etc.)

Capture an ObjectBarrier is known as apertureblocks of most of the raysreduces blur

image source: S. SeitzCapture an ObjectBarrier is known as apertureblocks of most of the raysreduces blurPinhole camera (camera obscura)

Interior of camera obscura(Sunday Magazine, 1838)Camera obscura(France, 1830)

Capture an ObjectBarrier is known as apertureblocks of most of the raysreduces blurPinhole camera (camera obscura)infinitely small aperture (pinhole)captures all rays going through the pinhole (pencil of rays)image is captured on image plane

image source: S. SeitzCamera ProjectionProject 3D point [X,Y,Z] into image point [x,y]

Zcamera in origin [0,0,0]no rotation, translation of the cameraProperties of Pinhole CameraWhat is preservedpoints project to pointslines, incidence (except for parallel lines, lines through focal point)planes project to planes

What is lostdepth (all points on a ray project to the same point)angles, length

Pinhole Camera Projectionzxy

focal lengthsensor0Image point is the intersection of the ray from the object point through the origin 0 with the image planeImage point can be derived through similar triangles

Projection eliminates last componentM =(X,Y,Z)

0Camera ProjectionIs this a linear transformation?Can we make it a linear transformation?

MVector relative to 0:a1a3a2Point in affine coordinates:e2e3e10

Not linear due to division by Z10Camera ProjectionUnified notation by including origin 0 into the representation

homogenous representation of MMa1a3a2e2e3e10

Special transformation: RotationRigid transformation: Angles and lengths preservedR is orthonormal matrix defined by three angles around three coordinate axes

ezeyexaRotation with angle a around ez

12Projective geometry in 2D Projective space is space of rays emerging from 0view point 0 forms projection center (focal point) for all rays rays v emerge from viewpoint into sceneray g is called projective point, defined as scaled v: g=lvxyw

0

13Projective and homogeneous pointsw=1

(R2)wx

yProjective space is space of rays emerging from 0view point 0 forms projection center for all rays rays v emerge from viewpoint into sceneray g is called projective point, defined as scaled v: g=lv014Finite and infinite pointsAll rays g that are not parallel to intersect at an affine point v on .w=1O

The ray g(w=0) does not intersect . Hence v is not an affine point but a direction. Directions have the coordinates (x,y,0)TProjective space combines affine space with infinite points (directions).(R2)15Pinhole Camera Projectionzxy

focal lengthsensor0M=(X,Y,Z)XYZ00

Image planeCamera centerZ (Optical axis)xy(R2)Perspective projectionPerspective projection models pinhole camera:scene geometry is affine R3 space with coordinates M=(X,Y,Z,1)T camera focal point in 0=(0,0,0,1)T, camera viewing direction along Zimage plane (x,y) in (R2) aligned with plane (X,Y) at Z0= f (focal length) scene point M projects onto point m on plane surface

XYf0

Image planeCamera centerZ (Optical axis)xy(R2)17Projective TransformationProjective Transformation maps P onto p

XYO

Projective Transformation linearizes projection

18Perspective ProjectionDimension reduction from R3 into R2 by projection onto (R2) XY0

(R2)

19Perspective Projection

Dimension reduction from R3 into R2 by projection onto (R2) XY0

(R2)20Projection in General PoseRotation [R]Projection center CMWorld coordinatesProjection:m

21XYImage center c= (cx, cy)TProjection centerZ (Optical axis)Pixel scale f= (fx,fy)TxyPixel coordinates m = (y,x)TImage plane and image sensorA sensor with picture elements (Pixel) is added onto the image planeImage sensor

Image-sensor mapping:

Pixel coordinates are related to image coordinates by affine transformation K with five parameters:Image center c=(cx,cy)T defines optical axisPixel size and pixel aspect ratio defines scale f=(fx,fy)T image skew s to model angle between pixel rows and columns22Projection matrix PCamera projection matrix P combines:inverse affine transformation Tcam-1 from general pose to originPerspective projection P0 to image plane at Z0 =1affine mapping K from image to sensor coordinates

World to camera coord. trans. matrix(4x4)Perspectiveprojection matrix(3x4)Camera to pixel coord. trans. matrix (3x3)=2Dpoint(3x1)3Dpoint(4x1)23Camera Projection Arbitrary CameraCamera rotated with R and translated with T

Orthographic ProjectionSpecial case of perspective projectionDistance from center of projection to image plane is infinite

Also called parallel projectionWhats the projection matrix?

ImageWorldSlide by Steve Seitz25Is this a linear function?Projection propertiesMany-to-one: any points along same visual ray map to same point in imagePoints pointsBut projection of points on focal plane is undefinedLines lines (collinearity is preserved)But lines through focal point (visual rays) project to a pointPlanes planes (or half-planes)But planes through focal point project to linesslide: S. Lazebnik26Vanishing points

Each direction in space has its own vanishing pointAll lines going in that direction converge at that pointException: directions parallel to the image plane

slide: S. Lazebnik27Vanishing pointsEach direction in space has its own vanishing pointAll lines going in that direction converge at that pointException: directions parallel to the image planeHow do we construct the vanishing point of a line?image planecameracenterline on ground planevanishing pointslide: S. Lazebnik28Facing Real CamerasThere are undesired effects in real situationsperspective distortion

Perspective distortionProblem for architectural photography: converging verticals

Where do they converge to?

image source: F. Durand30Perspective distortionProblem for architectural photography: converging verticals

Solution: view camera (lens shifted w.r.t. film)Source: F. DurandTilting the camera upwards results in converging verticalsKeeping the camera level, with an ordinary lens, captures only the bottom portion of the building

Shifting the lens upwards results in a picture of the entire subjecthttp://en.wikipedia.org/wiki/Perspective_correction_lens

31Perspective distortionProblem for architectural photography: converging verticalsResult:

Source: F. Durand

shifted lenstilted camera with regular lens32Perspective distortion

image: WikipediaWhich image is captured with a shifted lens?Perspective distortionWhat does a sphere project to?

Image source: F. Durand34Perspective distortionWhat does a sphere project to?

slide: S. Lazebnik35Perspective distortionThe exterior columns appear biggerThe distortion is not due to lens flawsProblem pointed out by Da Vinci

Slide by F. Durand36Perspective distortion: People

slide: S. Lazebnik37Facing Real CamerasThere are undesired effects in real situationsperspective distortion

Camera artifactsHome-made pinhole camera

http://www.debevec.org/Pinhole/Why soblurry?Slide by A. Efros39Shrinking the apertureWhy not make the aperture as small as possible?Less light gets throughDiffraction effectsSlide by Steve Seitz

40Shrinking the aperture

41Facing Real CamerasThere are undesired effects in real situationsperspective distortion

Camera artifactsaperture is not infinitely smallAdding a lensA lens focuses light onto the filmThin lens model:Rays passing through the center are not deviated(pinhole projection model still holds)

Slide by Steve Seitz43thin lens is a lens with a thickness (distance along the optical axis between the two surfaces of the lens) that is negligible compared to the radii of curvature of the lens surfacesAdding a lensA lens focuses light onto the filmThin lens model:Rays passing through the center are not deviated(pinhole projection model still holds)All parallel rays converge to one point on a plane located at the focal length f

Slide by Steve Seitzfocal pointf44Adding a lensA lens focuses light onto the filmThere is a specific distance at which objects are in focusother points project to a circle of confusion in the image

circle of confusionSlide by Steve Seitz45Thin lens formulaWhat is the relation between the focal length (f), the distance of the object from the optical center (D), and the distance at which the object will be in focus (D)?fDDSlide by Frdo Durandobjectimage planelens46This is the relation between the focal length (f), the distance of the object from the camera (D), and the distance at which the object will be in focus (D)Thin lens formulafDDSimilar triangles everywhere!Slide by Frdo Durandobjectimage planelens47Thin lens formulafDDSimilar triangles everywhere!yyy/y = D/DSlide by Frdo Durandobjectimage planelens48Thin lens formulafDDSimilar triangles everywhere!yyy/y = D/Dy/y = (D-f)/fSlide by Frdo Durandobjectimage planelens49Thin lens formulafDD1DD11f+=Any point satisfying the thin lens equation is in focus.Slide by Frdo Durandobjectimage planelens50And the set of all such points forms a plane parallel to the image (plane of focus).Real LensesZoom lens

image: Simal51slide: S. LazebnikLens Flaws: Chromatic AberrationLens has different refractive indices for different wavelengths: causes color fringing

Near Lens Center

Near Lens Outer Edge52Lens flaws: Spherical aberrationSpherical lenses dont focus light perfectlyRays farther from the optical axis focus closer

slide: S. Lazebnik53Lens flaws: Vignetting

slide: S. Lazebnik54

No distortionPin cushionBarrelRadial DistortionCaused by imperfect lensesDeviations are most noticeable near the edge of the lens

slide: S. Lazebnik55Radial DistortionBrowns distortion modelaccounts for radial distortion accounts for tangential distortion (distortion caused by lens placement errors)

typically K1 is used or K1, K2, P1, P2

(xu, yu) undistorted image point as in ideal pinhole camera(xd,yd) distorted image point of camera with radial distortion(xc,yc) distortion centerKn n-th radial distortion coefficientPn n-th tangential distortion coefficientFacing Real CamerasThere are undesired effects in real situationsperspective distortion

Camera artifactsaperture is not infinitely smalllensvignetting, radial distortionDepth of Field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htmSlide by A. Efros58Depth of field is the range of distance within the subject that is acceptably sharp.How can we control the depth of field?Changing the aperture size affects depth of fieldA smaller aperture increases the range in which the object is approximately in focusBut small aperture reduces amount of light need to increase exposure

Slide by A. Efros59F Number of the Cameraf number (f-stop) ratio of focal length to aperture

Varying the aperture

Large aperture = small DOFSmall aperture = large DOFSlide by A. Efros61Facing Real CamerasThere are undesired effects in real situationsperspective distortion

Camera artifactsaperture is not infinitely smalllensvignetting, radial distortiondepth of fieldField of View

Slide by A. EfrosWhat does FOV depend on?63fField of ViewSmaller FOV = larger Focal Length

Slide by A. EfrosfFOV depends on focal length and size of the aperture

64Field of View / Focal Length

Large FOV, small fCamera close to carSmall FOV, large fCamera far from the car

Sources: A. Efros, F. Durand65Field of View / Focal Length

66

Same effect for facesstandardwide-angletelephotoSource: F. Durand67The dolly zoomContinuously adjusting the focal length while the camera moves away from (or towards) the subjecthttp://en.wikipedia.org/wiki/Dolly_zoom

slide: S. Lazebnikhuman visual perception uses both size and perspective cues => this is an unsettling effect68The Dolly Zoom