Embed Size (px)
DESCRIPTION
David Fofi 1. « Uncalibrated Vision based on Structured Light ». Joaquim Salvi 2. El Mustapha Mouaddib 3. 1 Le2i UMR CNRS 5158 Université de Bourgogne Le Creusot, France [email protected]. 2 VICOROB - IIiA Universitat de Girona Girona, España [email protected]. - PowerPoint PPT Presentation
Citation preview
« Uncalibrated « Uncalibrated Vision based on Vision based on Structured Structured Light »Light »
Joaquim Salvi2
David Fofi1
El Mustapha Mouaddib3
3CREA EA 3299Université de Picardie Jules VerneAmiens, [email protected]
2VICOROB - IIiAUniversitat de GironaGirona, Españ[email protected]
1Le2i UMR CNRS 5158 Université de BourgogneLe Creusot, [email protected]
0. Outline0. Outline
1. Introduction
2. Tools for uncalibrated vision
3. Uncalibrated reconstruction
4. Experimental results
5. Conclusion
……………………………
I. IntroductionI. Introduction
1. Structured light vision
2. Calibration vs uncalibration
……………………………
IMAGE PLANEIMAGE PLANE PATTERN FRAMEPATTERN FRAME
« Structured light vision »« Structured light vision »
………………..J. Salvi, J. Batlle, E. Mouaddib, "A robust-coded pattern projection for dynamic measurement of moving scenes", Pattern Recognition Letters, 19, pp. 1055-1065, 1998.
J. Batlle, E. Mouaddib, J. Salvi, "Recent progress in coded structured light to solve the correspondence problem. A survey", Pattern Recognition, 31(7), pp. 963-982, 1998.
……………………...……………………………
DRAWBACKS OF HARD-CALIBRATION:
•Off-line process (calibration pattern, etc.)
•Has to be repeated each time one of the parameters is modified
Working with a camera with automatic focus and aperture is NOT possible.
Visual adaptation to the environment is not allowed!
A slide or LCD projector needs to be focused.
RECONSTRUCTION FROM RECONSTRUCTION FROM UNCALIBRATED SENSOR...UNCALIBRATED SENSOR...
« Calibration vs uncalibration »« Calibration vs uncalibration » ……………………...……………………………
II. Tools for II. Tools for uncalibrated visionuncalibrated vision
1. Test of spatial colinearity
2. Test of coplanarity
3. Stability of the cross-ratio
4. Validity of the affine model
……………………………
« Test of spatial colinearity »« Test of spatial colinearity »
SRQP
',',',',,,,,, srqpsrqpSRQPk
Cross-ratio within the pattern and cross-ratio within the image are equals if the points are colinear.
……………………...……………………………
« Test of coplanarity »« Test of coplanarity »
po' p'
q'r'
s'
o
qr
s
{o;p,q,r,s}={o';p',q',r'}
Cross-ratio within the pattern and cross-ratio within the image are equals if the point are colinear.
……………………...……………………………
« Stability of the cross-ratio »« Stability of the cross-ratio »
0 5 10 15 20 25 30 35 40 45 500
0.005
0.01
0.015
0.02
0.025
0.03
0.035
Noise
ErroronCross-Ratios
0 5 10 15 20 25 30 35 40 45 500
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
Noise
ErroronCross-Ratios
Error on cross-ratios with a noise from 0 to 0.5d (d is the distance between two successive points)
Nota: to compare cross-ratios a projective distance is necessary. Method of the random cross-ratios.
………………..K. Aström, L. Morin, "Random cross-ratios", Research Report n°rt 88 imag-14, LIFIA, 1992.
……………………...……………………………
« Validity of the affine model »« Validity of the affine model » ……………………...……………………………
2'2
'
2'
2'
2'
2'
22
2''
2''
22
nnmm
nnmmnnmm
nmnm
nmnmnmnm
vvuu
vvuuvvuu
vvuu
vvuuvvuu
affine projection
mn
m'n'
Valid if 0
III. Uncalibrated III. Uncalibrated reconstructionreconstruction
1. Projective reconstruction
2. Structured light limitations
3. Euclidean constraints through structured lighting
……………………………
Recover the scene structure from n images and m points and...
Intrinsic parameters
Extrinsic parameters
Scene geometry
Points matching
PROJECTIVE PROJECTIVE RECONSTRUCTIONRECONSTRUCTION
« Projective reconstruction »« Projective reconstruction »
……………………...……………………………
PARAMETERS ESTIMATION APPROACH, CANONICAL REPRESENTATIONPARAMETERS ESTIMATION APPROACH, CANONICAL REPRESENTATION
« Structured light limitations »« Structured light limitations »
………………..R. Mohr, B. Boufama, P. Brand, “Accurate projective reconstruction”, Proc. of the 2nd ESPRIT-ARPA-NSF Workshop on Invariance, Azores, pp. 257-276, 1993.
Q.-T. Luong, T. Viéville, "Canonical representations for the geometries of multiple projective views", Proc. of the 3 rd Euro. Conf. on Computer Vision, Stockholm (Sweden), 1994
+
=
MOVEMENT OF THE PROJECTOR
MOVEMENT OF THE 3-D POINTS
THE PATTERN SLIDES ALONG THE OBJECTS
RECONSTRUCTION FROM TWO VIEWS (i.e. one view and one
pattern projection)
CAMERA + PROJECTOR
HETEROGENEITY OF THE SENSOR
INTRINSIC PARAMETERS CANNOT BE ASSUMED
CONSTANT
……………………...……………………………
n images composed by m points... njmiijij ,...,1 ,...,1 , PAp
2
)(34
)(33
)(32
)(31
)(24
)(23
)(22
)(21
2
)(34
)(33
)(32
)(31
)(14
)(13
)(12
)(11
)(34
)(33
)(32
)(31
)(24
)(23
)(22
)(21
)(34
)(33
)(32
)(31
)(14
)(13
)(12
)(11
ij ij ij
ij
ij
ij
ij
ij
ij
ij
iji
ji
ji
ji
ji
ji
ji
ji
j
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
ij
tazayaxa
tazayaxaV
tazayaxa
tazayaxaUd
tazayaxa
tazayaxaV
tazayaxa
tazayaxaU
pij : image point
Aj : projection matrix
Pj : object point
(Uij, Vij) : pixel co-ordinates
« The parameters estimation approach »« The parameters estimation approach » ……………………...……………………………
WPAWpPAp 1
A unique solution cannot be performed because...
W is a 4x4 invertible matrix… a collineation of the 3-D space4x4 - 1 (scale factor) = 15 degrees of freedom, thus...
5 co-ordinates object points assigned to AN ARBITRARY PROJECTIVE BASIS.
A RECONSTRUCTION WITH RESPECT TO A RECONSTRUCTION WITH RESPECT TO A PROJECTIVE FRAME A PROJECTIVE FRAME
(distances, angles, parallelism are not preserved)
« The parameters estimation approach »« The parameters estimation approach » ……………………...……………………………
Euclidean transformations form a sub-group of projective transformations...
A collineation W upgrades projective reconstruction to Euclidean one.
TRANSLATING EUCLIDEAN KNOWLEDGE OF THE TRANSLATING EUCLIDEAN KNOWLEDGE OF THE SCENE INTO MATHEMATICAL CONSTRAINTS ON THE SCENE INTO MATHEMATICAL CONSTRAINTS ON THE ENTRIES OF W.ENTRIES OF W.
Matching projective points with their corresponding Euclidean points ? YES, BUT ...
Euclidean co-ordinates of points are barely available…
… if they are: pattern cross-points have to be projected exactly onto these object points.
projeucl PWP
« From projective to Euclidean »« From projective to Euclidean »
………………..B. Boufama, R. Mohr, F. Veillon, "Euclidean constraints for uncalibrated reconstruction", Proc. of the 4th Int. Conf. on Computer Vision, Berlin (Germany), pp. 466-470, 1993.
……………………...……………………………
PROJECTED PROJECTED SQUARESQUARE
ONTO A ONTO A PLANAR PLANAR SURFACESURFACE
IMAGE IMAGE CAPTURECAPTURE
A B
C D
The sensor behaviour is assumed to be affine...
BDACCDAB , BDACCDAB // ,//
« Parallelogram constraints »« Parallelogram constraints » ……………………...……………………………
Pattern
Vert. plane
Horiz. plane
•Points belonging to horizontal or vertical plane...
•Arbitrary distance between two planes...
•Cross-point as origin…
« Alignment constraints »« Alignment constraints » ……………………...……………………………
'''' or '''' CABA zzzzCABA
otherwise… reduced orthogonality constraint:
« Orthogonality constraints »« Orthogonality constraints » ……………………...……………………………
A'B' ·A'C' = (xA' - xB')(xA' - xC')+ (yA' - yB')(yA' - yC')+ (zA' - zB')(zA' -zC') = 0
(xA' - xB')(xA' - xC')+ (yA' - yB')(yA' - yC') = 0
Light stripes
Light planes
Projected lines
Planar surfaces
A
C
B
A' B'
C'
« Example »« Example » ……………………...……………………………
An alignment constraint : xA' = xB' (relation between unknown Euclidean
points)
We have: [xA' ; yA' ; zA' ; tA']T = W· [xA ; yA ; zA ; tA]T [xB' ; yB' ; zB' ; tB']T = W· [xB ; yB ; zB ; tB]T
Then: W1i·xA = W1i·xB (relation between known projective points)
… same way for the other constraints…
The set of equations is solved by a non-linear optimisation method as Levenberg-Marquardt.
15 independent constraints are necessary (W is a 44 matrix defined up to a scale factor)
IV. Experimental IV. Experimental resultsresults
1. Colinearity
2. Coplanarity
3. Euclidean reconstruction
……………………………
« Colinearity »« Colinearity »
Theoretical (pattern) cross-ratio = 1.3333Measured (image) cross-ratio = 1.3287Projective error =6.910-4
Decision = the points are colinear
Theoretical cross-ratio = 1.3333Measured cross-ratio = 1.3782Projective error =6.210-3
Decision = the points are not colinear
……………………...……………………………
« Coplanarity »« Coplanarity »
Theoretical cross-ratio = 2Measured cross-ratio = 1.96Projective error =2.210-3
Decision = the points are coplanar
Theoretical cross-ratio = 2Measured cross-ratio = 2.186Projective error =5.910-3
Decision = the points are not coplanar
……………………...……………………………
« Euclidean reconstruction: synthetic data »« Euclidean reconstruction: synthetic data »
Real co-ordinates Errors on estimate co-ordinates
X Y Z X Y Z 100 -50 4000 0.518 -0.267 3.95 300 -50 2000 -0.65 -0.242 -1.5 700 -50 4000 0.614 -0.33 6.43 500 -400 4020 -1.132 -1.768 -4.332 300 50 4000 0.091 0.397 2.597 500 50 2000 0.076 -0.119 0.449 900 50 4000 0.13 0.171 2.007 300 -430 3000 0.505 -0.911 5.079 450 75 2500 0.76 -1.154 4.016 705 -120 1000 0.603 -0.827 0.829 Mean relative error (%) 0.518 1.539 0.169
re-projection of 3D points onto the image planes (circles: synthetic points, crosses:
re-projections)
……………………...……………………………
-200
-150
-100
-50
-150
-100
-50
00
20
40
60
80
xy
z
0 20 40 60 80 100 1200
10
20
30
40
50
60
70
80
90
x
y
0
50
100
150
0
50
1000
20
40
60
80
xy
z
« Euclidean reconstructions »« Euclidean reconstructions »
-180 -160 -140 -120 -100 -80 -60-140
-120
-100
-80
-60
-40
-20
x
y
……………………...……………………………
V. ConclusionV. Conclusion
……………………………
•Projective reconstruction from a single pattern projection and a single image capture.
•Pattern projection used to retrieve geometrical knowledge of the scene: uncalibrated Euclidean reconstruction.
•Structured lighting ensures there is known scene structure.
•Structured light provides numerous contraints.
•Tests of colinearity and coplanarity can be used to retrieve projective basis (5 points, no 4 of them being coplanar, no 3 of them being colinear).
![Enforcing Consistency Constraints in Uncalibrated Multiple ...wojtek/papers/multiHomogr.pdf · Enforcing Consistency Constraints in Uncalibrated Multiple ... tection [25, 46] or enhanced](https://img.pdfslide.us/doc/110x75/5f0d905a7e708231d43afb29/enforcing-consistency-constraints-in-uncalibrated-multiple-wojtekpapersmultihomogrpdf.jpg)
