A Unified Approach to Calibrate a Network of Camcorders & ToF Cameras
M2SFA2 Marseille France 2008
Li Guan Marc Pollefeys
{lguan, marc}@cs.unc.edu
UNC-Chapel Hill, USA ETH-Zurich, Switzerland
2
ToF Camera (RIM sensor)
• Theory– Time of Flight
Fig. from 3DV system website
• Products– Canesta cameras– Swiss Ranger– PMD cameras– ZCam
3
• Advantage – high frame-rate (50 fps.)
– Depth image + amplitude image
• Drawback– low resolution (e.g. 176x144, SR3100)
– depth measurement is still not stable
• Solution for reconstruction:– A network of ToF cameras & video camcorders
– Challenges• calibration
• robust shape estimation
http://www.3dcgi.com/images/face_2d_3d.jpg
3D Sensors (cont.)
4
• Recovering sensor location, orientation and imaging parameters
• Traditional calibration target– Checkerboard
Z. Zhang ICCV’99
J.-Y. Bouget’s toolbox
– Laser pointer, etcT.Svoboda MIT press ’05
Svobod’s toolbox
• Our proposal– A sphere
with unknown radius
Calibration of the Sensor Network
5
• Video Camcorder– Observation: due to projective distortion, the image of a sphere is
an ellipse, and sphere center is NOT the center of the ellipse,– An ellipse is defined with 5 parameters– If we know the intrinsics of the camera, it can be simplified to 3
Hough transform
Sphere Center Extraction
6
Hough Transform• Given the
undistorted optical center position, the ellipse detection is a 3-parameter Hough transform
– Radius of the sphere tangent to the cone at plane Z=-1
– Row and Col of the sphere center in the image
• Fit the final result to get sub-pixel accuracy
7
• ToF Camera– Observation: intensity highlight in the “amplitude image”
Detect & track the sphere highlight
Fit parabolic surface to get sub-pixel
accuracy
Sphere Center Extraction (cont.)
Camera optical center
8
• Setup– 4 fixed position vision
sensors• 2 Canon HG10,
1920x1080, 25Hz
• 2 SR3100, 176x144,20Hz
Calibration Result
9
• Radius recovery
• Scale recovery
Sphere Radius & Scale Recovery
R = 0.0248
S = 11.3386
R’ = RS =0.0248x11.3386 = 0.2824m
Measured circumference = 1.7925m, the actual radius = 0.2853m
11
Sensor Fusion Framework
• Notations– as the binary state space– as the sensor models– as the sensor observations
(L. Guan, J.-S. Franco, M. Pollefeys, 3DPVT 2008)
13
ResultsFor MATLAB code, check out
http://www.cs.unc.edu/~lguan
Volume size 2563
Threshold at 0.875
Computation Time ~ 3 min. (MATLAB)