Computational PhotographyComputational PhotographyComputer Science 203.4790Semester B 2009-2010Lecture: Sunday 12:00-15:00 Room: 303
Dr. Hagit Hel-Orhagit@cs.haifa.ac.ilOffice: 415Office Hours: by appointment
Course Internet Site: http://cs.haifa.ac.il/courses/compPhoto
The CameraThe Camera
A camera is a device that takes photos of images Camera Obscura (Latin = "dark chamber")
19th century camera Sonys smile recognition camera
Camera DevelopmentsCamera Developments
motion capture(dry plates)
quality and size (35 mm)
1826 - Earliest surviving photograph. This image required an eight-hour exposure.
Computational PhotographyComputational Photography
Computational photography refers broadly to computational imaging techniques that enhance or extend the capabilities of digital photography. The output of these techniques is an ordinary photograph, but one that could not have been taken by a traditional camera.
Steve Mann: The Cyberman
Goal: Record a richer, multiGoal: Record a richer, multi--layered visual experiencelayered visual experience1.1. Overcome limitations of todayOvercome limitations of todays camerass cameras
2.2. Support better postSupport better post--capture processingcapture processing
3.3. Enables new classes of recording the visual signal Enables new classes of recording the visual signal
4.4. Synthesize Synthesize impossibleimpossible photosphotos
Single View Modeling
Segmentation and Matting
Data Driven Synthesis
Multi exposure enhancement
Multi exposure enhancement
Blending and Composition
Panoramas and feature-based registration
Single exposure enhancement
Acquisition and camera model
Intro and image formation
Pre-requisites / prior knowledge Course Home Page:
http://cs.haifa.ac.il/courses/compPhoto Messages Lecture slides and handouts Matlab guides Homework, Grades
Exercises: Programming in Matlab, ~3 Assignments Final project
Administration (Cont.)Administration (Cont.)
Matlab software: Available in PC labs Student version
Grading policy: Final Grade will be based on:
Exercises (40%) , Final project (60%) Exercises will be weighted Exercises can be submitted in pairs
Office Hours: by email appointment
Further ReadingsFurther Readings
Related papers New book: Computational
Photography by R. Raskarand J. Tumblin
SyllabusSyllabus Image Formation
Image formation HVS pathwayColor models
Acquisition and camera modelCamera model + perspective projectionsSensorsNoise models & DistortionsSampling (spatial+temporal) and quantizationCamera parametersCamera Parameters trade-offs.
Single exposure enhancementWhite BalancingDe-mosaicingDe-noisingDe-blurringGeometrical distortion correction
Panoramas and feature based registrationImage featuresSIFTPanoramasFeature based registrationPanoramasHomographyRANSAC Image stitching
Syllabus Syllabus cont.cont. Blending and Composition
Pyramid blendingOptimal cutSeam CarvingGraph-cutGradient domain editing
Appearance based registrationSimilarity measures Lucas Kanade optical flowMulti-modal registrationApplications
Multi exposure enhancement (2 weeks)HDRSuper-resolutionmulti-exposure fusion
Data Driven SynthesisTexture synthesisVideo textureQuiltingImage analogiesSuper-ResolutionImage Completion
Syllabus Syllabus cont.cont. Segmentation and Matting
Segmentation using Graph cut.mean-shiftSpectral clusteringInteractive and semi-automatic Matting
Single View ModelingCamera CalibrationMeasurements in affine camera3D reconstruction
Light FieldPlenoptic function and the LumiographRe-sampling the plenoptic function
1. Image Formation1. Image Formation
Taking a picture HVS pathway Color models
2. Camera Model and Acquisition2. Camera Model and Acquisition
Perspective projections Camera pipeline and parameters Sensors and optics Sampling and quantization Noise models & Distortions Camera Parameters trade-offs.
3. Single Exposure Enhancement3. Single Exposure Enhancement
White Balancing De-mosaicing De-noising De-blurring Geometrical distortion correction
Difference in white point
4. Panoramas and Feature Based Registration4. Panoramas and Feature Based Registration
Image features SIFT Feature based registration Panoramas Homography RANSAC Image stitching
5. Blending and Composition5. Blending and Composition
Pyramid blending Gradient domain editing Optimal cut Graph-cut
6. Appearance Based Registration (warping?)6. Appearance Based Registration (warping?)
Similarity measures Lucas Kanade optical flow Multi-modal registration Applications
7. Multi Exposure Enhancement 7. Multi Exposure Enhancement
HDR Super-resolution Different-exposures fusion
8. Data Driven Synthesis 8. Data Driven Synthesis Texture synthesis Video texture Quilting Image analogies Super-Resolution Image Completion
9. Segmentation and Matting 9. Segmentation and Matting
Segmentation using Graph cut. Mean-shift Spectral clustering Interactive and semi-automatic Matting
10. Single View Modeling10. Single View Modeling Camera Calibration 3D reconstruction Metrology
Flagellation by Pietro della Francesca (1416-92, Italian Renaissance period)Animation by Criminisi et al., ICCV 99
11. Light Field11. Light Field
Plenoptic function and the Lumiograph Re-sampling the plenoptic function
Computational PhotographyComputational PhotographyTodayTodays Topic s Topic -- Image FormationImage Formation
What is an image ? What is color ?
Model3D ObjectGeometric Modeling
The Visual SciencesThe Visual Sciences
What is an Image ?What is an Image ? An image is a projection of a 3D scene into a 2D projection plane. An image can be defined as a 2 variable function I(x,y) , where for
each position (x,y) in the projection plane, I(x,y) defines the light intensity at this point.
The Pinhole Camera ModelThe Pinhole Camera Model
Pinhole model: Captures pencil of rays all rays through a single point The point is called Center of Projection (COP) The image is formed on the Image Plane Effective focal length f - distance from COP to Image Plane
Slide by Steve Seitz
Projection Model (where)Projection Model (where)
The coordinate system We will use the pin-hole model as an approximation Put the optical center (Center Of Projection) at the origin Put the image plane (Projection Plane) in front of the COP The camera looks down the negative z axis
Slide by Steve Seitz
Funny things happenFunny things happen
Parallel lines arenParallel lines arentt
Lengths canLengths cant be trusted...t be trusted...
The Shading Model (what)The Shading Model (what)
Shading Model: Given the illumination incident at a point on a surface, what is reflected?
ShadingShading Model ParametersModel Parameters
The factors determining the shading effects are:
The light source properties: Positions, Electromagnetic Spectrum, Shape.
The surface properties: Position, orientation, Reflectance properties.
The eye (camera) properties: Position, orientation, Sensor spectrum sensitivities.
Newtons Experiment, 1665 Cambridge.Discovering the fundamental spectral components of light.
Light and the Visible SpectrumLight and the Visible Spectrum
The light SpectrumThe light Spectrum
Electromagnetic Radiation - Spectrum
Gamma X rays Infrared Radar FM TV AMUltra-violet
400 nm 500 nm 600 nm 700 nmWavelength in nanometers (nm)
Wavelength in meters (m)
Monochromators measure the power or energy at different wavelengths
The Spectral Power Distribution (SPD) of a light is a function e() which defines the energy at each wavelength.
400 500 600 7000
Light ParametersLight Parameters
ExamplesExamples of Spectral Power Distributionsof Spectral Power Dis