Upload
jordan-mark-stanley
View
215
Download
0
Embed Size (px)
Citation preview
Lecture 12Lecture 12 Stereo Reconstruction IIStereo Reconstruction II
Mata kuliah : T0283 - Computer VisionTahun : 2010
January 20, 2010 T0283 - Computer Vision 3
Learning ObjectivesLearning Objectives
After carefullyAfter carefully listening this lecture, students will listening this lecture, students will be able to do the following :be able to do the following :
demonstrate 3D stereo computation by solving demonstrate 3D stereo computation by solving point-point- correspondence problems and correspondence problems and fundamentalfundamental matrix.matrix.
Calculate object-depth information using Calculate object-depth information using disparity and disparity and triangulation techniquestriangulation techniques
January 20, 2010 T0283 - Computer Vision 4
An algorithm for stereo An algorithm for stereo reconstructionreconstruction
1. For each point in the first image determine the corresponding point in the second image
(this is a search problem)
2. For each pair of matched points determine the 3D point by triangulation
(this is an estimation problem)
January 20, 2010 T0283 - Computer Vision 5
Epipolar lineEpipolar line
Epipolar constraint• Reduces correspondence problem to 1D
search along an epipolar line
January 20, 2010 T0283 - Computer Vision 6
Algebraic representation of epipolar Algebraic representation of epipolar geometrygeometryWe know that the epipolar geometry defines a We know that the epipolar geometry defines a mappingmapping
x l/
point in first image
epipolar line in second
image• the map only depends on the cameras P, P/ (not on structure)
• it will be shown that the map is linear and can be written as
January 20, 2010 T0283 - Computer Vision 7
Stereo correspondence algorithms
January 20, 2010 T0283 - Computer Vision 8
Problem statementProblem statement
GivenGiven:: two images and their associated cameras two images and their associated cameras computecompute
corresponding image points.corresponding image points.
Algorithms may be classified into two types:Algorithms may be classified into two types:
1.1. Dense:Dense: compute a correspondence at every pixel compute a correspondence at every pixel
2.2. Sparse:Sparse: compute correspondences only for features compute correspondences only for features
The methods may be top down or bottom upThe methods may be top down or bottom up
January 20, 2010 T0283 - Computer Vision 9
Top down matching Top down matching
1. Group model (house, windows, etc) independently in each image
2. Match points (vertices) between images
January 20, 2010 T0283 - Computer Vision 10
Bottom up matchingBottom up matching• epipolar geometry reduces the correspondence search from 2D to a 1D search on corresponding epipolar lines
• 1D correspondence problem
b/
a/
bca
CBA
c/
January 20, 2010 T0283 - Computer Vision 11
Example image pair – parallel Example image pair – parallel camerascameras
January 20, 2010 T0283 - Computer Vision 12
First imageFirst image
January 20, 2010 T0283 - Computer Vision 13
Second imageSecond image
January 20, 2010 T0283 - Computer Vision 14
Dense correspondence algorithmDense correspondence algorithm
Search problem (geometric constraint): for each point in the left image, the corresponding point in the right image lies on the epipolar line (1D ambiguity)
Disambiguating assumption (photometric constraint): the intensity neighbourhood of corresponding points are similar across images
Measure similarity of neighbourhood intensity by cross-correlation
Parallel camera example – epipolar lines are corresponding rasters
epipolar line
January 20, 2010 T0283 - Computer Vision 15
Intensity profiles
• Clear correspondence between intensities, but also noise and ambiguity
January 20, 2010 T0283 - Computer Vision 16
Normalized Cross CorrelationNormalized Cross Correlation
region A region B
vector a vector b
write regions as vectors
a
b
January 20, 2010 T0283 - Computer Vision 17
Cross-correlation of neighbourhood Cross-correlation of neighbourhood regionsregions
epipolar line
January 20, 2010 T0283 - Computer Vision 18
left image bandright image band
cross correlation
1
0
0.5
x
January 20, 2010 T0283 - Computer Vision 19
left image band
right image band
cross correlation
1
0
x
0.5
target region
January 20, 2010 T0283 - Computer Vision 20
Why is cross-correlation such a poor measure in the second case?
1. The neighborhood region does not have a “distinctive” spatial intensity distribution
2. Foreshortening effects
front-parallel surfaceimaged length the
same
slanting surfaceimaged lengths differ
January 20, 2010 T0283 - Computer Vision 21
Limitations of similarity constraintLimitations of similarity constraint
Textureless surfaces Occlusions, repetition
Non-Lambertian surfaces, specularities
January 20, 2010 T0283 - Computer Vision 22
Results with window searchResults with window search
Window-based matching Ground truth
Data
January 20, 2010 T0283 - Computer Vision 23
Sketch of a dense correspondence Sketch of a dense correspondence algorithmalgorithm
For each pixel in the left imageFor each pixel in the left imageccompute the neighbourhood cross correlation ompute the neighbourhood cross correlation along the corresponding epipolar line in the right along the corresponding epipolar line in the right imageimagetthe corresponding pixel is the one with the he corresponding pixel is the one with the highest cross correlationhighest cross correlation
ParametersParameterssize (scale) of neighbourhoodsize (scale) of neighbourhoodsearch disparity search disparity
Other constraintsOther constraintsuniquenessuniquenessorderingorderingssmoothmoothness ofness of disparity field disparity field
ApplicabilityApplicabilitytextured scene, largely fronto-paralleltextured scene, largely fronto-parallel
January 20, 2010 T0283 - Computer Vision 24
Example dense correspondence algorithm
left image right image
January 20, 2010 T0283 - Computer Vision 25
3D Reconstruction
intensity = depthright image depth map
January 20, 2010 T0283 - Computer Vision 26
range map
Pentagon exampleleft image right image
January 20, 2010 T0283 - Computer Vision 27
RectificationRectification
e e /
For converging cameras epipolar lines are not paralle
January 20, 2010 T0283 - Computer Vision 28
Project images onto plane parallel to baseline
epipolar plane
January 20, 2010 T0283 - Computer Vision 29
Rectification continuedRectification continued
Convert converging cameras to parallel camera geometry by an image mapping
Image mapping is a 2D homography (projective transformation)
January 20, 2010 T0283 - Computer Vision 30
Example original stereo pair
rectified stereo pair
January 20, 2010 T0283 - Computer Vision 31
Note• image movement (disparity) is inversely proportional to depth Z • depth is inversely proportional to disparity
Example: depth and disparity for a parallel camera stereo rig
Then, y/ = y, and the disparity
Derivation
x
x /d
January 20, 2010 T0283 - Computer Vision 32
TriangulationTriangulation
January 20, 2010 T0283 - Computer Vision 33
Problem statementProblem statementGiven:Given: corresponding measured (i.e. noisy) points corresponding measured (i.e. noisy) points x x and and xx// , and cameras (exact) P and P , and cameras (exact) P and P//, compute the 3D point , compute the 3D point XX
Problem: in the presence of noise, back projected rays do not intersect
rays are skew in space
Measured points do not lie on corresponding epipolar lines
C C /
x x /
January 20, 2010 T0283 - Computer Vision 34
1. Vector solution1. Vector solution
C C /
Compute the mid-point of the shortest line between the two rays
January 20, 2010 T0283 - Computer Vision 35
2. Linear triangulation (algebraic solution)2. Linear triangulation (algebraic solution)
January 20, 2010 T0283 - Computer Vision 36
January 20, 2010 T0283 - Computer Vision 37
3. Minimizing a geometric/statistical error3. Minimizing a geometric/statistical error