Basic Mathematics of Projection

7/30/2019 Basic Mathematics of Projection

1/63

Projective Geometry and

Camera Models

Computer Vision

CS 543 / ECE 549

University of Illinois

Derek Hoiem

01/20/11


2/63

Administrative Stuff

Office hours Derek: Wed 4-5pm + drop by Ian: Mon 3-4pm, Thurs 3:30-4:30pm

HW 1: out Monday Prob1: Geometry, today and Tues Prob2: Lighting, next Thurs Prob3: Filters, following week

Next Thurs: Im out, David Forsyth will cover


3/63

Last class: intro

Overview of vision, examples of state of art

Logistics


4/63

Next two classes: Single-view Geometry

How tall is this woman?

Which ball is closer?

How high is the camera?

What is the camera

rotation?

What is the focal length ofthe camera?


5/63

Todays class

Mapping between image and world

coordinates

Pinhole camera model

Projective geometry

Vanishing points and lines

Projection matrix


6/63

Image formation

Lets design a camera Idea 1: put a piece of film in front of an object Do we get a reasonable image?

Slide source: Seitz


7/63

Pinhole camera

Idea 2: add a barrier to block off most of the rays This reduces blurring

The opening known as the aperture

Slide source: Seitz


8/63

Pinhole camera

Figure from Forsyth

f

f = focal lengthc = center of the camera

c


9/63

Camera obscura: the pre-camera

First idea: Mo-Ti, China (470BC to 390BC)

First built: Alhacen, Iraq/Egypt (965 to 1039AD)

Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill

Photo by Seth Ilys


10/63

Camera Obscura used for Tracing

Lens Based Camera Obscura, 1568


11/63

First Photograph

Oldest surviving photograph

Took 8 hours on pewter plate

J oseph Niepce, 1826

Photograph of the first photograph

Stored at UT Austin

Niepce later teamed up with Daguerre, who eventually created Daguerrotypes


12/63

Point of observation

Figures Stephen E. Palmer, 2002

Dimensionality Reduction Machine (3D to 2D)

3D world 2D image


13/63

Projection can be trickySlide source: Seitz

lid i


14/63

Projection can be trickySlide source: Seitz


15/63

Projective Geometry

What is lost?

Length

Which is closer?

Who is taller?


16/63

Length is not preserved

Figure by David Forsyth

B

C

A


17/63

Projective Geometry

What is lost?

Length

Angles

Perpendicular?

Parallel?


18/63

Projective Geometry

What is preserved?

Straight lines are still straight


19/63


Parallel lines in the world intersect in the image at a

vanishing point


20/63


oVanishing Point

oVanishing Point

Vanishing Line


21/63


Vanishingpoint

Vanishing

line

Vanishingpoint

Vertical vanishingpoint

(at infinity)

Slide from Efros, Photo from Criminisi


22/63


Photo from online Tate collection


23/63

Note on estimating vanishing points

Use multiple lines for better accuracy but lines will not intersect at exactly the same point in practice

One solution: take mean of intersecting pairs

bad idea!Instead, minimize angular differences


24/63

Vanishing objects


25/63

Projection: world coordinatesimage coordinates

CameraCenter

(tx, ty, tz)

=

Z

Y

X

P.

.

. f Z Y

=

v

up

.

OpticalCenter

(u0, v0)

v

u


26/63

Homogeneous coordinates

Conversion

Converting to homogeneous coordinates

homogeneous imagecoordinates

homogeneous scenecoordinates

Converting from homogeneous coordinates


27/63

Homogeneous coordinates

Invariant to scaling

Point in Cartesian is ray in Homogeneous

=

=

w

y

wx

kw

ky

kwkx

kwky

kx

wy

x

k

Homogeneous

Coordinates

Cartesian

Coordinates


28/63

Basic geometry in homogeneous coordinates

Line equation: ax + by + c = 0

Append 1 to pixel coordinate to gethomogeneous coordinate

Line given by cross product of two points

Intersection of two lines given by cross

product of the lines

=

1

i

i

i v

u

p

jiij ppline =

jiij linelineq =

=

i

i

i

i

c

b

a

line

Another problem solved by homogeneous


29/63

Another problem solved by homogeneous

coordinates

Cartesian: (Inf, Inf)Homogeneous: (1, 1, 0)

Intersection of parallel linesCartesian: (Inf, Inf)Homogeneous: (1, 2, 0)

Slide Credit: Saverese

P j ti t i


30/63

Projection matrix

[ ]XtRKx =x: Image Coordinates: (u,v,1)K: Intrinsic Matrix (3x3)

R: Rotation (3x3)t: Translation (3x1)X: World Coordinates: (X,Y,Z,1)

Ow

iw

kw

jwR,T


31/63

Interlude: when have I used this stuff?


32/63

When have I used this stuff?

Object Recognition (CVPR 2006)


33/63


Single-view reconstruction (SIGGRAPH 2005)


34/63


Getting spatial layout in indoor scenes (ICCV

2009)


35/63


Inserting photographed objects into images

(SIGGRAPH 2007)

Original Created

h h d h ff


36/63


Inserting synthetic objects into images

Projection matrix


37/63

[ ]X0IKx =

=

1

0100

000

000

1z

y

x

f

f

v

u

w

K

Slide Credit: Saverese

Projection matrix

Intrinsic AssumptionsUnit aspect ratioOptical center at (0,0)No skew

Extrinsic AssumptionsNo rotationCamera at (0,0,0)

k l


38/63

Remove assumption: known optical center

[ ]X0IKx =

=

1

0100

00

00

1

0

0

z

y

x

vf

uf

v

u

w

Intrinsic AssumptionsUnit aspect ratioNo skew


R i i l


39/63

Remove assumption: square pixels

[ ]X0IKx =

=

1

0100

00

00

1

0

0

z

y

x

v

u

v

u

w

Intrinsic AssumptionsNo skew


i k d i l


40/63

Remove assumption: non-skewed pixels

[ ]X0IKx =

=

1

0100

00

0

1

0

0

z

y

x

v

us

v

u

w

Intrinsic Assumptions Extrinsic AssumptionsNo rotationCamera at (0,0,0)

Note: different books use different notation for parameters

O i d d T l d C


41/63

Oriented and Translated Camera

Ow

iw

kw

jw

t

R

All t l ti


42/63

Allow camera translation

[ ]XtIKx =

=

1

100

010

001

100

0

0

1

0

0

z

y

x

t

t

t

v

u

v

u

w

z

y

x

Intrinsic Assumptions Extrinsic AssumptionsNo rotation

3D R t ti f P i tSlide Credit: Saverese


43/63

3D Rotation of Points

Rotation around the coordinate axes, counter-clockwise:

=

=

=

100

0cossin

0sincos

)(

cos0sin

010

sin0cos

)(

cossin0

sincos0

001

)(

z

y

x

R

R

R

p

p

y

z

All t ti


44/63

Allow camera rotation

[ ]XtRKx =

=

1100

0

1333231

232221

131211

0

0

z

y

x

trrr

trrr

trrr

v

us

v

u

w

z

y

x

D f f d


45/63

Degrees of freedom

[ ]XtRKx =

=

1100

0

1333231

232221

131211

0

0

z

y

x

trrr

trrr

trrr

v

us

v

u

w

z

y

x

5 6

V i hi P i t P j ti f I fi it


46/63

Vanishing Point = Projection from Infinity

[ ]

=

=

=

R

R

R

z

y

x

z

y

x

z

yx

KpKRptRKp

0

=

R

R

R

z

y

x

vf

uf

v

u

w

100

0

0

1

0

00

u

z

fxu

R

R +=

0v

z

fyv

R

R +=

O th hi P j ti


47/63

Orthographic Projection

Special case of perspective projection

Distance from the COP to the image plane is infinite

Also called parallel projection Whats the projection matrix?

Image World

Slide by Steve Seitz

=

11000

0010

0001

1z

y

x

vu

w

Scaled Orthographic Projection


48/63

Scaled Orthographic Projection

Special case of perspective projection

Object dimensions are small compared to distance tocamera

Also called weak perspective

Whats the projection matrix?

Image World


=

1000

000

000

1z

y

x

s

ff

vu

w


49/63

Suppose we have two 3D cubes on the ground

facing the viewer, one near, one far.1. What would they look like in perspective?

2. What would they look like in weak perspective?

Photo credit: GazetteLive.co.uk

Beyond Pinholes: Radial Distortion


50/63

Beyond Pinholes: Radial Distortion

Image from Martin Habbecke

Corrected Barrel Distortion

Things to remember


51/63

Things to remember

Vanishing points andvanishing lines

Pinhole camera modeland camera projectionmatrix

Homogeneouscoordinates

Vanishingpoint

Vanishingline

Vanishingpoint

Vertical vanishingpoint

(at infinity)

[ ]XtRKx =

Next class


52/63

Next class

Applications of camera model and projectivegeometry

Recovering the camera intrinsic and extrinsic

parameters from an image Recovering size in the world

Projecting from one plane to another

Questions


53/63

Questions

What about focus aperture DOF FOV etc?


54/63

What about focus, aperture, DOF, FOV, etc?

Adding a lens


55/63

Adding a lens

A lens focuses light onto the film There is a specific distance at which objects are in

focus

other points project to a circle of confusion in the image

Changing the shape of the lens changes this distance

circle ofconfusion

Focal length aperture depth of field


56/63

Focal length, aperture, depth of field

A lens focuses parallel rays onto a single focal point

focal point at a distancefbeyond the plane of the lens

Aperture of diameter D restricts the range of rays

focal point

F

optical center

(Center Of Projection)

Slide source: Seitz

The eye


57/63

The eye

The human eye is a camera Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris Whats the film?

photoreceptor cells (rods and cones) in the retina

Depth of fieldSlide source: Seitz


58/63

Depth of field

Changing the aperture size or focal length

affects depth of field

f / 5.6

f / 32

Flower images from Wikipedia http://en.wikipedia.org/wiki/Depth_of_field

Varying the apertureSlide from Efros
http://en.wikipedia.org/wiki/Depth_of_fieldhttp://en.wikipedia.org/wiki/Depth_of_fieldhttp://en.wikipedia.org/wiki/Image:Jonquil_flowers_at_f5.jpghttp://en.wikipedia.org/wiki/Image:Jonquil_flowers_at_f32.jpg


59/63

Varying the aperture

Large aperture = small DOF Small aperture = large DOF

Shrinking the aperture


60/63


Why not make the aperture as small as possible? Less light gets through

Diffraction effects




61/63



Relation between field of view and focal length


62/63

Relation between field of view and focal length

Field of view (angle width) Film/Sensor Width

Focal lengthfdfov2

tan 1=

Dolly Zoom or Vertigo Effect


63/63

Dolly Zoom or Vertigo Effect

http://www.youtube.com/watch?v=Y48R6-iIYHs

Zoom in whilemoving away

How is this done?
http://www.youtube.com/watch?v=Y48R6-iIYHshttp://www.youtube.com/watch?v=Y48R6-iIYHs

Documents

Basic Mathematics of Projection