3D Reconstruction and Camera Calibration from 2D Images 3D Reconstruction and Camera Calibration from

  • View
    2

  • Download
    1

Embed Size (px)

Text of 3D Reconstruction and Camera Calibration from 2D Images 3D Reconstruction and Camera Calibration...

  • 3D Reconstruction

    and

    Camera Calibration

    from

    2D Images

    by

    Arne Henrichsen

    Submitted to the Department of Electrical Engineering in fulfilment of the requirements for the degree of

    Master of Science in Engineering

    at the

    UNIVERSITY OF CAPE TOWN

    December 2000

    © University of Cape Town 2000

  • Make some space

  • Declaration

    I, Arne Henrichsen, declare that this dissertation is my own work, except where otherwise stated.

    It is being submitted for the degree of Master of Science in Engineering at the University of

    Cape Town and it has not been submitted before for any degree or examination, at any other

    university.

    Signature of Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    Cape Town

    December 2000

    i

  • Acknowledgements

    I would like to thank my supervisor, Professor Gerhard de Jager, for the opportunity he gave me

    to work in the Digital Image Processing group and for his enthusiasm and guidance throughout

    this project.

    I also wish to thank:

    • The National Research Foundation and DebTech for their generous financial support.

    • All the members of the Digital Image Processing group for providing an interesting and

    stimulating working environment, and in particular Fred Nicolls for his help with my

    many questions.

    • Marthinus C. Briers from the Department of Geomatics at UCT for his help in calibrating

    my camera.

    • David Liebowitz from the Department of Engineering Science at Oxford for helping me

    understand his calibration method.

    iii

  • Abstract

    A 3D reconstruction technique from stereo images is presented that needs minimal intervention

    from the user.

    The reconstruction problem consists of three steps, each of which is equivalent to the estimation

    of a specific geometry group. The first step is the estimation of the epipolar geometry that exists

    between the stereo image pair, a process involving feature matching in both images. The second

    step estimates the affine geometry, a process of finding a special plane in projective space by

    means of vanishing points. Camera calibration forms part of the third step in obtaining the

    metric geometry, from which it is possible to obtain a 3D model of the scene.

    The advantage of this system is that the stereo images do not need to be calibrated in order

    to obtain a reconstruction. Results for both the camera calibration and reconstruction are

    presented to verify that it is possible to obtain a 3D model directly from features in the images.

    v

  • Contents

    Declaration i

    Acknowledgements iii

    Abstract v

    Contents vi

    List of Figures xi

    List of Tables xv

    List of Symbols xix

    Nomenclature xxi

    1 Introduction 1

    1.1 General Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

    1.2 The 3D reconstruction problem . . . . . . . . . . . . . . . . . . . . . . . . . 2

    1.3 Outline of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

    2 Stratification of 3D Vision 7

    2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

    vii

  • viii CONTENTS

    2.2 Projective Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

    2.2.1 Homogeneous Coordinates and other Definitions . . . . . . . . . . . 8

    2.2.2 The Projective Plane . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    2.2.3 The Projective Space . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    2.2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

    2.3 Affine Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

    2.3.1 The Affine Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

    2.3.2 The Affine Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

    2.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

    2.4 Metric Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

    2.4.1 The Metric Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

    2.4.2 The Metric Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    2.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

    2.5 Euclidean Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

    2.6 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

    3 Camera Model and Epipolar Geometry 25

    3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

    3.2 Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

    3.2.1 Camera Calibration Matrix . . . . . . . . . . . . . . . . . . . . . . . 27

    3.2.2 Camera Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

    3.3 Epipolar Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

    4 Fundamental Matrix Estimation 31

    4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

    4.2 Linear Least-Squares Technique . . . . . . . . . . . . . . . . . . . . . . . . 31

  • CONTENTS ix

    4.3 Minimising the Distances to Epipolar Lines . . . . . . . . . . . . . . . . . . 33

    4.4 Least-Median-of-Squares method . . . . . . . . . . . . . . . . . . . . . . . 35

    4.5 RANSAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

    5 Corner Matching 39

    5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

    5.2 Establishing Matches by Correlation . . . . . . . . . . . . . . . . . . . . . . 40

    5.3 Support of each Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

    5.4 Initial Matches by Singular Value Decomposition . . . . . . . . . . . . . . . 43

    5.5 Resolving False Matches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

    5.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    6 Camera Calibration 51

    6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

    6.2 Classical Calibration Methods . . . . . . . . . . . . . . . . . . . . . . . . . 52

    6.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

    6.2.2 Estimating the Perspective Projection Matrix . . . . . . . . . . . . . 53

    6.2.3 Extracting the Camera Calibration Matrix . . . . . . . . . . . . . . . 54

    6.2.4 Other Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

    6.3 Selfcalibration using Kruppa’s Equations . . . . . . . . . . . . . . . . . . . 55

    6.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

    6.3.2 Kruppa’s Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

    6.4 Selfcalibration in Single Views . . . . . . . . . . . . . . . . . . . . . . . . . 58

    6.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

    6.4.2 Some Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

    6.4.3 Calibration Method 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 59

  • x CONTENTS

    6.4.4 Calibration Method 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 62

    6.4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

    6.5 Calibration using a Planar Pattern . . . . . . . . . . . . . . . . . . . . . . . 66

    6.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

    6.5.2 Homography between the Planar Object and its Image . . . . . . . . 66

    6.5.3 Calculating the Camera Calibration Matrix . . . . . . . . . . . . . . 69

    6.5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

    7 Stratified 3D Reconstruction 73

    7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

    7.2 3D Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

    7.2.1 Projective Reconstruction . . . . . . . . . . . . . . . . . . . . . . . 74

    7.2.2 Affine Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . 75

    7.2.3 Metric Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . 76

    7.2.4 Reconstruction Results . . . . . . . . . . . . . . . . . . . . . . . . . 77

    7.3 3D Textured Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

    7.3.1 Rectification of Stereo Images . . . . . . . . . . . . . . . . . . . . . 80

    7.3.2 Dense Stereo Matching . . . . . . . . . . . . . . . . . . . . . . . . . 83

    7.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

    7.4 Other Reconstruction Results . . . . . . . . . . . . . . . . . . . . . . . . . . 86

    7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

    8 Conclusions 89

    A Feature Extraction 91

    A.1 Corner Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

  • CONTENTS xi

    A.1.1