Stereo Image Processing Using Opencv

3D RECONSTRUCTION

FROM 2D IMAGESJARGALSAIKHAN IVEEL (09VL10F)

You are using demo versionPlease purchase full version from www.technocomsolutions.com


Table of contents

¤ Introduction

¤ Objective

¤ Camera model

¤ Epipolar geometry¤ Epipolar geometry

¤ Implementation

¤ Result

¤ Conclusion



Introduction

¤ Stereo vision

� Based on human vision system by using two or more 2D views

� Attempts to obtain depth information of 2D scene objects.objects.

¤ Usage

� Tracking moving objects in 3D space

�Gather distance information

� Construct a 3D spatial model of a scene etc.



Objective

¤ To implement a reconstruction algorithm for a

stereo-vision system.

¤ Designing hardware implementation for a such ¤ Designing hardware implementation for a such

system based on the FPGA.



Camera model

¤ Pin-hole model

n Where

n Cx, Cy – alignment to the optical axis

n Fx, Fy - focal length



Camera model

¤ Parameters

� Instrinsics

� Extrinsics



Camere model

¤ Extrinsics parameter Pc = R(Pw − T)

� R - Rotational matrix

� T – Translation V



Camera model

¤ Rotating points by θ (in this case, around the Z-axis) is the same as counterrotating the coordinate axis by θ; by simple trigonometry, we can see how rotation changes the coordinates of a point



Chessboard Calibration

¤ cvFindChessboardCorners()

� Finds a position of the corners of the chessboard in a

given image



cvDrawChessboardCorners()



Stereo imaging pipeline

¤ Undistortion� Mathematically remove radial

and tangential lens distortion

¤ Rectification � Images are row-aligned and

rectified.

¤ Correspondence.� Disparity map- differences

coordinates of the same features.

¤ Triangulation. � Reprojection, and the output is a

depth map.



Essential & Fundamental Matrices

¤ Essential Matrice� E – contains information about the translation and rotation which relate

physical space of two camera

¤ Fundamental MatriceF – contains information about the translation and rotation which relate pixel ¤ F – contains information about the translation and rotation which relate pixel

coordinates of two camera ( essential matrices + intrinsics parameters of both camera)



To Compute F-Matrice

¤ cvFindFundamentalMat

� Calculates the fundamental matrix from the

corresponding points in two images



Epipolar lines

¤ Given a feature in one image, its matching view in the other image

must lie along the corresponding epipolar line.



To Compute Epipolar lines

¤ cvComputeCorrespondEpilines()

� For points in one image of a stereo pair, computes the

corresponding epilines in the other image.



Rectification

¤ Stereo image transformations in corresponding epipolar lines in all images become collinear in all images become collinear with each other and with the image scanning lines.



Rectification

¤ Bouguet’s algorithm for stereo rectification

� Arguments are the rotation matrix and translation (R,

T) between the stereo images

�Minimizes the resulting reprojection distortions

�Maximizins common viewing area



cvStereoRectify()

¤ Computes rectification transforms for each head of

a calibrated stereo camera

�Outputs are Rotation matrice & Translation vector for

both stereo cameras.



Stereo Correspondence

¤ Stereo correspondence� matching a 3D point in the two different camera views

� can be computed only over the visual areas in which the views of the two cameras overlap.

¤ cvFindStereoCorrespondenceBM()� Implemented as a fast and effective block-matching stereo algorithm in OpenCV

� Algorithm Stagesn Prefiltering to normalize image brightness a 1. nd enhance texture.

n Correspondence search along horizontal epipolar lines using an SAD window.

n Postfiltering to eliminate bad correspondence matches.



cvFindStereoCorrespondenceBM()



Implementation in OpenCV

¤ Simplified stages of the code

� Calibrate the stereo web camera using Chessboard

Pattern technique

� Rectification of the stereo pair images based on the

Bouguet’s algorithm.

� Finding the feature correspondence on the rectified

images and calculating the output disparity image.



Result : Calibration



Result : Rectification



Result : Disparity Map



Conclusion

¤ Camera model and Epipolar geometry

¤ Stereo vision and its implementation pipeline

¤ Implementation based on the OpenCV Library



Documents

Stereo Image Processing Using Opencv