11

CS6825: Image FormationCS6825: Image Formation

How are images How are images created.created.

22

Image FormationImage Formation

Images are formed when a Images are formed when a SENSORSENSOR registers registers RADIATIONRADIATION that has that has interacted with interacted with PHYSICAL OBJECTSPHYSICAL OBJECTS

33

Types of ImagesTypes of Images Photography: reflected Photography: reflected

lightlight Range images: Range images:

distancedistance Tomography: tissue Tomography: tissue

densitydensity Infrared: heatInfrared: heat

We will concentrate on We will concentrate on the first typethe first type

(gray scale and color).(gray scale and color).

44

Digital ImagesDigital Images

are 2D arrays (matrices) of numbers:

55

Digital imagesDigital images

light intensities, or colorlight intensities, or color distances, or distances, or other physical quantities.other physical quantities.

Depending on the type of image, the numbers, pixel values, represent:

Color image and3 fields –red, green,blue

Blow up – shows pixels

Pixel values representDepth – white nearer

66

Intensity ImagesIntensity Images

Light coming from the world hits the sensor.Light coming from the world hits the sensor.

77

Pinhole CameraPinhole Camera

Pinhole camerasPinhole cameras Abstract camera model - box with a Abstract camera model - box with a

small hole in itsmall hole in it Pinhole cameras work in practicePinhole cameras work in practice

88

Distant objects appear smallerDistant objects appear smaller Even though B is larger than C it appears Even though B is larger than C it appears

the same size because it is more distant.the same size because it is more distant.

99

Physical parameters involved:Physical parameters involved:

Optical parameters of the lensOptical parameters of the lens• Characterize the sensorCharacterize the sensor

Photometric parametersPhotometric parameters• Characterize the light reflected off the Characterize the light reflected off the

objectobject Geometric parametersGeometric parameters

• Determine the relative position of the Determine the relative position of the object wrt the sensorobject wrt the sensor

1010

Optical ParametersOptical Parameters

Lens typeLens type

1111

Optical ParametersOptical Parameters

Focal lengthFocal length

Lens Types

1212

Optical ParametersOptical Parameters Field of viewField of view Characterized byCharacterized by

• Vertical and horizontal Vertical and horizontal angles of viewangles of view

1313

Optical ParametersOptical Parameters

Angular aperturesAngular apertures

Larger aperture (larger diameter of opening)•Lets in more light, but, can have

Smaller aperture (smaller diameter of opening)

1414

Optical ParametersOptical Parameters Larger Aperture means:Larger Aperture means:

• More light comes in (can have faster shutter More light comes in (can have faster shutter speed …take quick picture and hence speed …take quick picture and hence capture faster moving objects).capture faster moving objects).

• BUT, also means incoming light is BUT, also means incoming light is uncollimated rays, resulting in a uncollimated rays, resulting in a sharp focus sharp focus only for rays with a certain focal lengthonly for rays with a certain focal length. This . This means that a greater aperture results in an means that a greater aperture results in an image that is image that is sharp around what the lens is sharp around what the lens is focusing on and blurred otherwise. focusing on and blurred otherwise. SMALLER DEPTH OF FIELD SMALLER DEPTH OF FIELD

For an apertureShutter speed too low

Object near flower are not in focusFor this larger aperture

1515

Optical ParametersOptical Parameters Smaller Aperture means:Smaller Aperture means:

• Less light comes in (need greater Less light comes in (need greater exposure time…not good for fast moving exposure time…not good for fast moving objects…get a blur).objects…get a blur).

• BUT, also means incoming light is highly BUT, also means incoming light is highly collimated rays, collimated rays, resulting in a sharp focus resulting in a sharp focus at the image plane for objects an a more at the image plane for objects an a more varied distance from the point focused on varied distance from the point focused on in the scene. GREATER DEPTH OF FIELDin the scene. GREATER DEPTH OF FIELD

Objects around flower are in focus withsmaller aperture

1616

Photometric ParametersPhotometric Parameters

Type, intensity, and direction of Type, intensity, and direction of illuminationillumination

Reflectance properties of the viewed Reflectance properties of the viewed objectobject

Effects of the sensor’s structure on Effects of the sensor’s structure on the amount of light reaching the the amount of light reaching the photoreceptorsphotoreceptors

1717

Geometric ParametersGeometric Parameters

Type of projection Type of projection • how we model how light goes from an how we model how light goes from an

object into the camera.object into the camera.

Position and orientation of the sensorPosition and orientation of the sensor

Perspective distortions - lens Perspective distortions - lens aberations.aberations.

1818

A model –Perspective ProjectionA model –Perspective Projection Based on simple pinhole camera, which is not Based on simple pinhole camera, which is not

reality, but works for many applicationsreality, but works for many applications

Math Model

1919

Pinhole Camera ModelPinhole Camera ModelHow to calculate the image location p’ from the Object point P.How to calculate the image location p’ from the Object point P.

XX

YY

ZZ

ff

OO

PP

ppxx

yy

•Non-linear equationsNon-linear equations•Any point on the ray OP has image p Any point on the ray OP has image p !!!!

x’ = f (X/Z)y’ = f (Y/Z)

’

’

’

2020

Perspective Projection Perspective Projection ParametersParameters

f’f’ = focal length of lens = focal length of lens O = focal point (like pinhole in camera, center of O = focal point (like pinhole in camera, center of

iris in eye)iris in eye) P = point in 3D world (coordinates [x y z])P = point in 3D world (coordinates [x y z]) P’ = point in 2D image plane (coordinates [x’ y’ P’ = point in 2D image plane (coordinates [x’ y’

z’] z’ is fixed)z’] z’ is fixed)

2121

Understanding Perspective Understanding Perspective ProjectionProjection

Know the following to understand perspective Know the following to understand perspective projection:projection:

Points go to PointsPoints go to Points Lines go to LinesLines go to Lines Planes go to whole image or Half-planesPlanes go to whole image or Half-planes Polygons go to PolygonsPolygons go to Polygons

Preserves the “perspective”of what you see in a scene.Remember from your grade school – the idea of “PerspectiveDrawing” it is the same model!

2222

How things really work with a LensHow things really work with a Lens

2323

The reason for lenses!The reason for lenses!

Without a lens, the lightfrom a object point is NOT focused into a single image point, but,as shown on the top,goes to a ellipse type shape. This is notwhat we perceive ashumans (we haveour own lenses).

The lens can focus theobject point to a pointin the image.

2424

But….Lenses can have But….Lenses can have problemsproblems

Unfortunately, lens are not perfect.Unfortunately, lens are not perfect.

There are many models for lens imperfections. There are many models for lens imperfections. Perspective projection is a very SIMPLE model of Perspective projection is a very SIMPLE model of what actually happens…..consider sesors can what actually happens…..consider sesors can have problems to (not uniform reception, etc).have problems to (not uniform reception, etc).

Spherical Aberration-point P does not focus to a single point in image.

2525

Remember Lens Distance from Remember Lens Distance from image plane – focal point.image plane – focal point.

Also, know that when you FOCUS a camera, you Also, know that when you FOCUS a camera, you are moving the lens increasing and decreasing are moving the lens increasing and decreasing the focal length ….hence bringing into “focus” the focal length ….hence bringing into “focus” (hence appearing as one point in the image) (hence appearing as one point in the image) different depth object points in the scenedifferent depth object points in the scene

Only center point P2 is focusedPerfectly to a point. P1 and P2 are“blurry” not focused to a point …but,Could be if moved lens…focal distanceChanges.

P1

P2

P3

2626

More on Depth of FieldMore on Depth of Field Reinforcing last slide…here a different image Reinforcing last slide…here a different image

showing what is in focus and what is not.showing what is in focus and what is not. When not focuses on, the point becomes a circle, When not focuses on, the point becomes a circle,

called circle of confusion. Here only the cat B is called circle of confusion. Here only the cat B is focused on perfectly. Both Dog and Rabbit focused on perfectly. Both Dog and Rabbit create circles of confusion and will be blurry.create circles of confusion and will be blurry.

2727

ConclusionConclusion Many kinds of imagesMany kinds of images Discussed 2D image FormationDiscussed 2D image Formation Simple model of how points in image register in Simple model of how points in image register in

the image plane is “Perspective Projection”.the image plane is “Perspective Projection”. Discussed Perspective Project ModelDiscussed Perspective Project Model Described why we need lenses.Described why we need lenses. Discussed what focusing a camera does.Discussed what focusing a camera does.