3D Reconstruction and Camera Calibration from 2D Images 3D Reconstruction and Camera Calibration from

3D Reconstruction

and

Camera Calibration

from

2D Images

by

Arne Henrichsen

Submitted to the Department of Electrical Engineering in fulfilment of the requirements for the degree of

Master of Science in Engineering

at the

UNIVERSITY OF CAPE TOWN

December 2000

© University of Cape Town 2000

Make some space

Declaration

I, Arne Henrichsen, declare that this dissertation is my own work, except where otherwise stated.

It is being submitted for the degree of Master of Science in Engineering at the University of

Cape Town and it has not been submitted before for any degree or examination, at any other

university.

Signature of Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Cape Town

December 2000

i

Acknowledgements

I would like to thank my supervisor, Professor Gerhard de Jager, for the opportunity he gave me

to work in the Digital Image Processing group and for his enthusiasm and guidance throughout

this project.

I also wish to thank:

• The National Research Foundation and DebTech for their generous financial support.

• All the members of the Digital Image Processing group for providing an interesting and

stimulating working environment, and in particular Fred Nicolls for his help with my

many questions.

• Marthinus C. Briers from the Department of Geomatics at UCT for his help in calibrating

my camera.

• David Liebowitz from the Department of Engineering Science at Oxford for helping me

understand his calibration method.

iii

Abstract

A 3D reconstruction technique from stereo images is presented that needs minimal intervention

from the user.

The reconstruction problem consists of three steps, each of which is equivalent to the estimation

of a specific geometry group. The first step is the estimation of the epipolar geometry that exists

between the stereo image pair, a process involving feature matching in both images. The second

step estimates the affine geometry, a process of finding a special plane in projective space by

means of vanishing points. Camera calibration forms part of the third step in obtaining the

metric geometry, from which it is possible to obtain a 3D model of the scene.

The advantage of this system is that the stereo images do not need to be calibrated in order

to obtain a reconstruction. Results for both the camera calibration and reconstruction are

presented to verify that it is possible to obtain a 3D model directly from features in the images.

v

Contents

Declaration i

Acknowledgements iii

Abstract v

Contents vi

List of Figures xi

List of Tables xv

List of Symbols xix

Nomenclature xxi

1 Introduction 1

1.1 General Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 The 3D reconstruction problem . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Outline of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Stratification of 3D Vision 7

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

vii

viii CONTENTS

2.2 Projective Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2.1 Homogeneous Coordinates and other Definitions . . . . . . . . . . . 8

2.2.2 The Projective Plane . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.3 The Projective Space . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3 Affine Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3.1 The Affine Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3.2 The Affine Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4 Metric Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.4.1 The Metric Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.4.2 The Metric Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 Euclidean Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.6 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3 Camera Model and Epipolar Geometry 25

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2 Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2.1 Camera Calibration Matrix . . . . . . . . . . . . . . . . . . . . . . . 27

3.2.2 Camera Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Epipolar Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4 Fundamental Matrix Estimation 31

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.2 Linear Least-Squares Technique . . . . . . . . . . . . . . . . . . . . . . . . 31

CONTENTS ix

4.3 Minimising the Distances to Epipolar Lines . . . . . . . . . . . . . . . . . . 33

4.4 Least-Median-of-Squares method . . . . . . . . . . . . . . . . . . . . . . . 35

4.5 RANSAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5 Corner Matching 39

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.2 Establishing Matches by Correlation . . . . . . . . . . . . . . . . . . . . . . 40

5.3 Support of each Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.4 Initial Matches by Singular Value Decomposition . . . . . . . . . . . . . . . 43

5.5 Resolving False Matches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6 Camera Calibration 51

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.2 Classical Calibration Methods . . . . . . . . . . . . . . . . . . . . . . . . . 52

6.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

6.2.2 Estimating the Perspective Projection Matrix . . . . . . . . . . . . . 53

6.2.3 Extracting the Camera Calibration Matrix . . . . . . . . . . . . . . . 54

6.2.4 Other Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6.3 Selfcalibration using Kruppa’s Equations . . . . . . . . . . . . . . . . . . . 55

6.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

6.3.2 Kruppa’s Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.4 Selfcalibration in Single Views . . . . . . . . . . . . . . . . . . . . . . . . . 58

6.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

6.4.2 Some Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

6.4.3 Calibration Method 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 59

x CONTENTS

6.4.4 Calibration Method 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 62

6.4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.5 Calibration using a Planar Pattern . . . . . . . . . . . . . . . . . . . . . . . 66

6.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

6.5.2 Homography between the Planar Object and its Image . . . . . . . . 66

6.5.3 Calculating the Camera Calibration Matrix . . . . . . . . . . . . . . 69

6.5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

7 Stratified 3D Reconstruction 73

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

7.2 3D Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

7.2.1 Projective Reconstruction . . . . . . . . . . . . . . . . . . . . . . . 74

7.2.2 Affine Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . 75

7.2.3 Metric Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . 76

7.2.4 Reconstruction Results . . . . . . . . . . . . . . . . . . . . . . . . . 77

7.3 3D Textured Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

7.3.1 Rectification of Stereo Images . . . . . . . . . . . . . . . . . . . . . 80

7.3.2 Dense Stereo Matching . . . . . . . . . . . . . . . . . . . . . . . . . 83

7.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

7.4 Other Reconstruction Results . . . . . . . . . . . . . . . . . . . . . . . . . . 86

7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

8 Conclusions 89

A Feature Extraction 91

A.1 Corner Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

CONTENTS xi

A.1.1