3D reconstruction Slides

7/30/2019 3D reconstruction Slides

1/12

3D Reconstruction

Tutorial

By

(Waqar S. Qureshi)

Vision and Graphics Lab, Asian Institute ofTechnology, Thailand


2/12

What is 3D-Reconstruction?

In computer vision, 3D reconstruction is amethod to find the shape of an object or theappearance of a scene in an image[1].

This requires to calculate the depth estimatesof the 2D-points of an image of a 3D scene orobject.

Today, we will discuss how to get 3D-pointsestimates using monocular camera

[1] http://en.wikipedia.org/wiki/3D_reconstruction


3/12

3D-Reconstruction from image sequence

What do we need

Appearance of a 3D-scene, shape of theobject of interest

3D-points or depth estimates of a scene orobject

What do we have

Sequence of Images from a monocularcamera

How to solve it?

Use Multi-view geometry and linear algebra


4/12


5/12

3D-Reconstruction

Types:

Spacial 3D-reconstruction

2-D feature correspondences in spacial domain

are extracted and then 3D depths areestimated

Dense 2-D feature correspondences in spacialdomain are extracted and then 3D depths areestimated


6/12

3D-Reconstruction

Overview of the method taken from Atima M.Sc thesis presentation AIT


7/12

SURF Correspondences

[1]- Find SURF correspondences between two images as follows:

Find the Key-point and descriptor in the first image.

Find the Key-point and descriptor in the second image.

Find the Matching Key-points correspondence from image-1 to image-2

using K-nearest neighbor.

Find the Matching Key-points correspondence from image-2 to image-1using K-nearest neighbor.

Remove all the non-symmetric correspondences in both the images.

Remove all the correspondences which are not consistent with theepipolar geometry (Fundamental Matrix).

Find the fundamental matrix with only the left-over matches.


8/12

SURF Correspondences


9/12

3D-Reconstruction

[1] - Find the pair of frame having the best GRIC ratio and use themto initialize reconstruction. Use only SURF correspondences to findthe fundamental matrix and the homography matrix for calculating theGRIC.

[2] - Recover the motion for the pair of frames. We call it aInitialization/Recover Motion

Get correspondence between the frames

Get Camera P1 and P2 using essential matrix method by Hertley (2004).

Use triangulation to find the 3D-points using the equation X . P = x

Check if enough points lie in front of the cameras.(only 10-15 outliers)

Add the pair of recovered cameras to the vector of cameras.


10/12

Triangulation

[3] - Estimate the 3D points using cameras recovered in step 2. This isknown as Triangulation

Find the dense correspondence between the frames to be triangulated

Check if initialization=True

if initialization=True then find all the new 2D-point correspondences in the set of

correspondences which are not already recovered.

Get camera matrices P1 and P2 from camera matrices set for frames.

For new 2D-points correspondences find the 3D points (triangulate).

Find the re-projection error for the 3D points recovered in previous step and remove

outliers Check if the points lie in-front of the cameras.

Add the 3D-points to the vector of 3D points.


11/12

3D-Reconstruction

[4] - Set the initialization = TRUE.

[5] - Resection the next i-th frame

Check if Initialization=True if not return

For the number of times we need to mergeframes do...


12/12

Re-sectioning

Get the point-correspondences between the i-th frame to be re-sectionedand the j-th frame recovered.

Find the new 2Dto3D correspondences by matching the pointcorrespondences from step ii to 2D-projections of the i-th frame in the 3D-points vectors.

If the 2Dto3D correspondences are not greater then threshold do not dore-sectioning for this frame.

Get all the 2Dto3D pairs and find Camera matrix P using Ransack.

Remove the outliers and find the camera matrix P with all the inlier usingDLT.

Add camera to the vector of cameras.

[6] - Save all the 3D-Point cloud with their corresponding 2D projections

and frame Ids for visualization.

Documents

3D reconstruction Slides