Upload
saransh-vora
View
222
Download
0
Embed Size (px)
Citation preview
8/18/2019 CS6320 CV F2012 Rectification
1/40
Image Rectification (Stereo)
Guido Gerig
CS 6320 Spring 2012
8/18/2019 CS6320 CV F2012 Rectification
2/40
Example: converging cameras
courtesy of Andrew Zisserman
8/18/2019 CS6320 CV F2012 Rectification
3/40
8/18/2019 CS6320 CV F2012 Rectification
4/40
Stereo image rectification
In practice, it is
convenient if image
scanlines are the
epipolar lines.
8/18/2019 CS6320 CV F2012 Rectification
5/40
Image Rectification
http://en.wikipedia.org/wiki/Image_rectification
If the two cameras are aligned to be coplanar , the search is
simplified to one dimension - a horizontal line parallel to the
baseline between the cameras
http://en.wikipedia.org/wiki/Coplanarhttp://en.wikipedia.org/wiki/Baselinehttp://en.wikipedia.org/wiki/Baselinehttp://en.wikipedia.org/wiki/Coplanar
8/18/2019 CS6320 CV F2012 Rectification
6/40
Stereo image rectification
• Image Reprojection– reproject image planes
onto commonplane parallel to linebetween optical centers
– a homography (3x3transform)applied to both inputimages
– pixel motion is horizontalafter this transformation
–
C. Loop and Z. Zhang.Computing RectifyingHomographies for StereoVision. IEEE Conf.Computer Vision andPattern Recognition,1999.
http://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdfhttp://research.microsoft.com/~zhang/Papers/TR99-21.pdf
8/18/2019 CS6320 CV F2012 Rectification
7/40
Stereo image rectification:example
Source: Alyosha Efros
8/18/2019 CS6320 CV F2012 Rectification
8/40
l l
Image Rectification
p p
P
O O
l l p
p
ee
• Common Image Plane
• Parallel Epipolar Lines
• Search Correspondences
on scan line
• Epipoles → ∞
8/18/2019 CS6320 CV F2012 Rectification
9/40
All epipolar lines are parallel in the rectified image plane.
Image Rectification
8/18/2019 CS6320 CV F2012 Rectification
10/40
Image Rectification
8/18/2019 CS6320 CV F2012 Rectification
11/40
]R [TE
T
IR
x
]0,0,[ d 0 0 0
0 0 d
0 –d 0
0Epp
Essential matrix example: parallel cameras
For the parallel cameras,
image of any point must lie on
same horizontal line in each
image plane.Grauman
8/18/2019 CS6320 CV F2012 Rectification
12/40
8/18/2019 CS6320 CV F2012 Rectification
13/40
Example I: compute the fundamental matrix for a parallel camera stereo rig
• reduces to y = y/
, i.e. raster correspondence (horizontal scan-lines)
f
f
XY
Z
8/18/2019 CS6320 CV F2012 Rectification
14/40
f
f
XY
Z
Geometric interpretation ?
8/18/2019 CS6320 CV F2012 Rectification
15/40
Image pair rectification
Goal: Simplify stereo matching by “warping” the
images
Apply projective transformation so that epipolar lines
correspond to horizontal scanlines
e
e
map epipole e to (1,0,0)
try to minimize image distortion
problem when epipole in (or close to) the image
He
0
0
1
8/18/2019 CS6320 CV F2012 Rectification
16/40
Planar rectification
Bring two views
to standard stereo setup
(moves epipole to)
(not possible when in/close to image)
Image Transformations: Find
homographies H’ and H sothat after transformations, Fn
becomes F of parallel cameras:
8/18/2019 CS6320 CV F2012 Rectification
17/40
Algorithm Rectification
Following Trucco & Verri book pp. 159
• known T and R between cameras
• Rotate left camera so that epipole el goes to infinity along horizontal axis
• Apply same rotation to right camera torecover geometry
• Rotate right camera by R-1
• Adjust scale
8/18/2019 CS6320 CV F2012 Rectification
18/40
8/18/2019 CS6320 CV F2012 Rectification
19/40
From: Trucco & Verri, Introductory Techniques for 3-D Computer
Vision, pp. 157-161
8/18/2019 CS6320 CV F2012 Rectification
20/40
Rectification ctd.
before
after
8/18/2019 CS6320 CV F2012 Rectification
21/40
Stereo matching withgeneral camera configuration
8/18/2019 CS6320 CV F2012 Rectification
22/40
Image pair rectification
8/18/2019 CS6320 CV F2012 Rectification
23/40
8/18/2019 CS6320 CV F2012 Rectification
24/40
Other Material /Code
• Epipolar Geometry, Rectification:http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.html
•
Fusiello, Trucco & Verri:Tutorial, Matlabcode etc:http://profs.sci.univr.it/~fusiello/demo/rect/
http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.htmlhttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.htmlhttp://profs.sci.univr.it/~fusiello/demo/rect/http://profs.sci.univr.it/~fusiello/demo/rect/http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.htmlhttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO2/rectif_cvol.html
8/18/2019 CS6320 CV F2012 Rectification
25/4025
Goal: 3D from Stereo viaDisparity Map
F&P
Chapter 11
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y),y)
8/18/2019 CS6320 CV F2012 Rectification
26/40
Example: Stereo to Depth Map
8/18/2019 CS6320 CV F2012 Rectification
27/40
Run Example
Demo for stereo reconstruction (out of date):http://mitpress.mit.edu/e- journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.html
Updated Webpages:
http://research.microsoft.com/en-
us/um/people/zhang/INRIA/softwares.html
SFM Example:
http://research.microsoft.com/en-
us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.html
Software:
http://research.microsoft.com/en-
us/um/people/zhang/INRIA/softwares.html
http://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://research.microsoft.com/en-us/um/people/zhang/INRIA/softwares.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.htmlhttp://mitpress.mit.edu/e-journals/Videre/001/articles/Zhang/CalibEnv/CalibEnv.html
8/18/2019 CS6320 CV F2012 Rectification
28/40
Example: Zhengyou Zhang
Fundamental matrix between thetwo cameras:
http://www.inria.fr/robotvis/personnel/zzhang/zzhang-eng.htmlhttp://www.inria.fr/robotvis/personnel/zzhang/zzhang-eng.html
8/18/2019 CS6320 CV F2012 Rectification
29/40
Points have been
extracted using
Harris cornerdetector, point
matches via
fundamental
matrix F and
search alongepipolar lines.
8/18/2019 CS6320 CV F2012 Rectification
30/40
Point matches found by acorrelation technique
8/18/2019 CS6320 CV F2012 Rectification
31/40
3D reconstruction representedby a pseudo stereogram
http://research.microsoft.com/en-
us/um/people/zhang/INRIA/SFM-Ex/SFM-Ex.html
8/18/2019 CS6320 CV F2012 Rectification
32/40
Additional Materials: Forward
Translating Cameras
8/18/2019 CS6320 CV F2012 Rectification
33/40
8/18/2019 CS6320 CV F2012 Rectification
34/40
8/18/2019 CS6320 CV F2012 Rectification
35/40
8/18/2019 CS6320 CV F2012 Rectification
36/40
f
f
X YZ
first image second image
8/18/2019 CS6320 CV F2012 Rectification
37/40
8/18/2019 CS6320 CV F2012 Rectification
38/40
8/18/2019 CS6320 CV F2012 Rectification
39/40
e
e’
8/18/2019 CS6320 CV F2012 Rectification
40/40
Summary: Properties of the Fundamental matrix