36
Close Range Photogrammetry Saju John Mathew EE 5358 Monday, 24 th March 2008 University of Texas at Arlington Monday, March 24, 2008 EE 5358 Computer Vision 1

Close Range Photogrammetry

Embed Size (px)

Citation preview

Page 1: Close Range Photogrammetry

Close Range Photogrammetry

Saju John Mathew EE 5358

Monday, 24th March 2008

University of Texas at Arlington

Monday, March 24, 2008 EE 5358 Computer Vision 1

Page 2: Close Range Photogrammetry

Overview

• Definitions• Equipment• Mathematical Explanations• Working• Applications

Monday, March 24, 2008 EE 5358 Computer Vision 2

Page 3: Close Range Photogrammetry

Close Range Photogrammetry(CRP)

Photogrammetry is a measurement technique where the coordinates of the points in 3D of an object are calculated by the measurements made in two photographic images(or more) taken starting from different positions.

CRP is generally used in conjunction with object to camera distances of not more than 300 meters (984 feet).

Monday, March 24, 2008 EE 5358 Computer Vision 3

Page 4: Close Range Photogrammetry

Vertical Aerial Photographs

Monday, March 24, 2008 EE 5358 Computer Vision 4

University of Texas at Arlington at approx. 30 meters

University of Texas at Arlington at approx. 200 meters

Page 5: Close Range Photogrammetry

CRP

Monday, March 24, 2008 EE 5358 Computer Vision 5

Page 6: Close Range Photogrammetry

Acquisition of Data: Camera

Cameras can be broadly classified into two:

• Metric• Single Cameras• Stereometric Cameras

• Non-metric

Monday, March 24, 2008 EE 5358 Computer Vision 6

Page 7: Close Range Photogrammetry

Photogrammetric Camera that enables geometrically accurate reconstruction of the optical model

of the object scene from its stereo photographs

Single Cameras • Total depth of field • Photographic material• Nominal focal length• Format of photographic material• Tilt range of camera axis and number of intermediate

stops

Monday, March 24, 2008 EE 5358 Computer Vision 7

Metric Cameras

Page 8: Close Range Photogrammetry

Metric Cameras (contd.)

Stereometric Cameras• Base Length• Nominal Focal Length• Operational Range• Photographic Material• Format of photographic material• Tilt range of optical axes and

number of intermediate tilt stops

Monday, March 24, 2008 EE 5358 Computer Vision 8

Page 9: Close Range Photogrammetry

Non-metric Cameras

Cameras that have not been designed especially for photogrammetric purposes:

• A camera whose interior orientation

is completely or partially unknown

and frequently unstable.

Monday, March 24, 2008 EE 5358 Computer Vision 9

Page 10: Close Range Photogrammetry

Non-metric Cameras

Advantages• General availability• Flexibility in focusing range• Price is considerably less than for metric cameras• Can be hand-held and thereby oriented in any direction

Disadvantages• Lenses are designed for high resolution at the expense of high

distortion• Instability of interior orientation (changes after every exposure)• Lack of fiducial marks• Absence of level bubbles and orientation provisions precludes

the determination of exterior orientation before exposure

Monday, March 24, 2008 EE 5358 Computer Vision 10

Page 11: Close Range Photogrammetry

Data Reduction

• Analog 1900 to 1960

• Analytical 1960 onwards

• Semi-analytical

• Digital 1980 onwards

Monday, March 24, 2008 EE 5358 Computer Vision 11

Page 12: Close Range Photogrammetry

Analytical Photogrammetry

Based on camera parameters, measured photo coordinates and ground control

Accounts for any tilts that exist in photos

Solves complex systems of redundant equations by implementing least squares method

Monday, March 24, 2008 EE 5358 Computer Vision 12

Page 13: Close Range Photogrammetry

ReviewCollinearity Condition

The exposure station

of a photograph, an object

point and its photo image

all lie along a straight

line.

Monday, March 24, 2008 EE 5358 Computer Vision 13

AZa

Xa

YaXL

YL

ZL

X

Z

Y

Ox

yL

aTilted photo

plane

f xa

ya

Page 14: Close Range Photogrammetry

Image Coordinate System

• Ground Coordinate System - X, Y, Z

wrt Ground Coordinate System

• Exposure Station Coordinates – XL, YL, ZL

• Object Point (A) Coordinates – Xa, Ya, Za

• Rotated coordinate system parallel to ground

coordinate system (XYZ) – x’, y’, z’

wrt Rotated Coordinate System Rotated image coordinates – xa’, ya’, za’

xa’ , ya’ and za’ are related to the measured

photo coordinates xa, ya, focal length (f) and the

three rotation angles omega, phi and kappa.

Monday, March 24, 2008 EE 5358 Computer Vision 14

Z

Y

X

A

Za

Ya

Xa

YL

XL

ZL

x’

z’y’

L

za’

ya’

Xa’

Page 15: Close Range Photogrammetry

Rotation Formulas

Developed in a sequence of three independent two-dimensional rotations.• ω rotation about x’ axis

x1 = x’

y1 = y’Cosω + z’Sinω

z1 = -y’Sinω + z’Cosω

• φ rotation about y’ axis

x2 = -z1Sinφ + x1Cosφ

y2 = y1

z2 = z1Cosφ + x1Sinφ

• κ rotation about z’ axis

x = x2Cosκ + y2Sin κ

y = -x2Sinκ + y2Cosκ

z = z2

Monday, March 24, 2008 EE 5358 Computer Vision 15

Page 16: Close Range Photogrammetry

Rotation Matrix

X = MX’

Rotation Matrix

The sum of the squares of the three “direction cosines” in any row or in any

column is unity. M -1 = MT

X’ = MTX

Monday, March 24, 2008 EE 5358 Computer Vision 16

x = m11x’ + m12y’ + m13z’ y = m21x’ + m22y’ + m23z’ z = m31x’ + m32y’ + m33z’

x = x’(CosφCosκ) + y’(SinωSinφCosκ + CosωSinκ) + z’(-CosωSinφCosκ + SinωSinκ) y = x’(-CosφSinκ) + y’(-SinωSinφSinκ + CosωCosκ) + z’(CosωSinφSinκ + SinωCosκ ) z = x’(Sinφ) + y’(-SinωCosφ) + z’(CosωCosφ)

'

'

'

333231

232221

131211

z

y

x

mmm

mmm

mmm

z

y

x

Page 17: Close Range Photogrammetry

Collinearity Condition Equations

Collinearity condition equations developed from similar triangles

* Dividing xa and ya by za

* Substitute –f for za

* Correcting the offset of Principal

point (xo, yo)

Monday, March 24, 2008 EE 5358 Computer Vision 17

AL

a

LA

a

LA

a

ZZ

z

YY

y

XX

x

''' x = m11x’ + m12y’ + m13z’ y = m21x’ + m22y’ + m23z’ z = m31x’ + m32y’ + m33z’

aLA

LAa

LA

LAa

LA

LAa

aLA

LAa

LA

LAa

LA

LAa

aLA

LAa

LA

LAa

LA

LAa

zZZ

ZZmz

ZZ

YYmz

ZZ

XXmz

zZZ

ZZmz

ZZ

YYmz

ZZ

XXmy

zZZ

ZZmz

ZZ

YYmz

ZZ

XXmx

'''

'''

'''

333231

232221

131211

)()()(

)()()(

)()()(

)()()(

333231

232221

333231

131211

LALALA

LALALAoa

LALALA

LALALAoa

ZZmYYmXXm

ZZmYYmXXmfyy

ZZmYYmXXm

ZZmYYmXXmfxx

Page 18: Close Range Photogrammetry

Collinearity Equations

• Nonlinear• Nine unknowns

• ω, φ, κ

• XA, YA and ZA

• XL, YL and ZL

Taylor’s Theorem is used to linearize the nonlinear equations

substituting

Monday, March 24, 2008 EE 5358 Computer Vision 18

)()()(

)()()(

)()()(

)()()(

333231

232221

333231

131211

LALALA

LALALAoa

LALALA

LALALAoa

ZZmYYmXXm

ZZmYYmXXmfyy

ZZmYYmXXm

ZZmYYmXXmfxx

0

)(

)(!

)(

n

nn

axn

af

)()()(

)()()(

)()()(

232221

131211

333231

LALALA

LALALA

LALALA

ZZmYYmXXms

ZZmYYmXXmr

ZZmYYmXXmq

Page 19: Close Range Photogrammetry

Linearizing Collinearity Equations

Rewriting the Collinearity Equations

• F0 and G0 are functions F and G evaluated at the initial approximations for the nine unknowns

• dω, dφ, dκ are the unknown corrections to be applied to the initial approximations

• The rest of the terms are the partial derivatives of F and G wrt to their respective unknowns at the initial approximations

Monday, March 24, 2008 EE 5358 Computer Vision 19

ao

ao

yq

sfyG

xq

rfxF

aAA

AA

AA

LL

LL

LL

o

aAA

AA

AA

LL

LL

LL

o

ydZZ

GdY

Y

GdX

X

G

dZZ

GdY

Y

GdX

X

Gd

Gd

Gd

GG

xdZZ

FdY

Y

FdX

X

F

dZZ

FdY

Y

FdX

X

Fd

Fd

Fd

FF

000

000000

000

000000

Taylor’s Theorem

Page 20: Close Range Photogrammetry

Applying LLSM to Collinearity Equations

Monday, March 24, 2008 EE 5358 Computer Vision 20

• Residual terms must be included in order to make the equations consistent

J = xa – Fo ; K = ya – Go

b terms are coefficients equal to the partial derivatives

Numerical values for these coefficient terms are obtained by using initial approximations for the unknowns.

The terms must be solved iteratively (computed corrections are added to the initial approximations to obtain revised approximations) until the magnitudes of corrections to initial approximations become negligible.

yaAAALLL

xaAAALLL

VKdZbdYbdXbdZbdYbdXbdbdbdb

VJdZbdYbdXbdZbdYbdXbdbdbdb

262524262524232221

161514161514131211

Page 21: Close Range Photogrammetry

Analytical Stereomodel

Mathematical calculation of three-dimensional ground coordinates of points in the stereomodel by analytical photogrammetric techniques

Three steps involved in forming an Analytical Stereomodel:

• Interior Orientation• Relative Orientation• Absolute Orientation

Monday, March 24, 2008 EE 5358 Computer Vision 21

Page 22: Close Range Photogrammetry

Analytical Interior Orientation

Requires camera calibration information and quantification of the effects of atmospheric refraction.

2D coordinate transformation is used to relate the comparator coordinates to the fiducial coordinate system to correct film distortion.

Lens distortion and principal-point information from camera calibration are used to refine the coordinates so that they are correctly related to the principal point and free from lens distortion.

Atmospheric refraction corrections are applied.

Monday, March 24, 2008 EE 5358 Computer Vision 22

Page 23: Close Range Photogrammetry

Analytical Relative Orientation

Process of determining the elements of exterior orientation

Fix the exterior orientation elements of the left photo of the stereopair to zero values

Common method in use to find these elements is through Space Resection by Collinearity(see slide below)

Each object point in the stereomodel contributes 4 equations

5 unknown orientation elements + 3 unknowns(X, Y & Z)

Monday, March 24, 2008 EE 5358 Computer Vision 23

Page 24: Close Range Photogrammetry

Space Resection by Collinearity

Formulate the collinearity equations for a number of control points whose X, Y and Z ground coordinates are known and whose images appear in the tilted photo. The equations are then solved for the six unknown elements of exterior orientation which appear in them.

Space Resection collinearity equations for a point A

A two dimensional conformal coordinate transformation is used

X = ax’ – by’ + Tx X, Y – ground control coordinates for the point

Y = ay’ + bx’ + Ty x’, y’ – ground coordinates from a vertical photograph

a, b, Tx, Ty – transformation parameters

Monday, March 24, 2008 EE 5358 Computer Vision 24

yaLLL

xaLLL

VKdZbdYbdXbdbdbdb

VJdZbdYbdXbdbdbdb

262524232221

161514131211

Page 25: Close Range Photogrammetry

Analytical Absolute Orientation

Utilizes a 3D conformal coordinate transformationRequires a min. of 2 horizontal and 3 vertical control

pointsStereomodel coordinates of control points are related to

their 3D coordinates in a cartesian coordinate systemCoordinates of all stereomodel points in the ground

system can be computed by applying the transformation parameters

Monday, March 24, 2008 EE 5358 Computer Vision 25

Page 26: Close Range Photogrammetry

Bundle Adjustment

Adjust all photogrammetric measurements to ground

control values in a single solution

Unknown quantities • X, Y and Z object space coordinates of all object points• Exterior orientation parameters of all photographs

Measurements • x and y photo coordinates of images of object points• X, Y and/or Z coordinates of ground control points• Direct observations of the exterior orientation parameters of the photographs

Monday, March 24, 2008 EE 5358 Computer Vision 26

Page 27: Close Range Photogrammetry

Bundle Adjustment-Observations

Photo Coordinates - Fundamental Photogrammetric Measurements made with a comparator or analytical plotter. According to accuracy and precision the coordinates are weighed

Control Points – determined through field survey Exterior Orientation Parameters – especially helpful in understanding the angular

attitude of a photograph Regardless of whether exterior orientation parameters were observed, a least squares

solution is possible since the number of observations is always greater than the number of unknowns.

xij, yij – measured photo coordinates of the image

of point j on photo i related to fiducial axis system

xo, yo – coordinates of principal points in fiducial axis

system

f - focal length/principal distance X j, Yj, Zj – coordinates of point j in object space

m11i, m12i, …….,m33i – rotation parameters for photo i XLi, YLi, ZLi – coordinates of incident nodal point

of camera lens in object space

Monday, March 24, 2008 EE 5358 Computer Vision 27

)()()(

)()()(

)()()(

)()()(

333231

232221

333231

131211

LijiLijiLiji

LijiLijiLijioij

LijiLijiLiji

LijiLijiLijioij

ZZmYYmXXm

ZZmYYmXXmfyy

ZZmYYmXXm

ZZmYYmXXmfxx

Page 28: Close Range Photogrammetry

Bundle Adjustment - Weights

Photo coordinates σ0

2 – reference variance

σxij2 , σyij

2 – variances in xij and yij resp.

σxijyij = σyijxij – covariance of xij and yij

• Ground Control coordinates• σXj

2, σYj2, σZj

2 – variances in Xj00, Yj

00, Zj00 resp.

• σXjYj = σYjXj – covariance of Xj00 with Yj

00

• σXjZj = σZjXj – covariance of Xj00 with Zj

00

• σYjZj = σZjYj – covariance of Yj00 with Zj

00

• Exterior Orientation Parameters

Monday, March 24, 2008 EE 5358 Computer Vision 28

1

2

2

2

ijijij

ijijij

yxy

yxx

oijW

2

2

10

01

ij

ij

y

x

ijW

1

2

2

2

20

..

jjjjj

jjjjj

jjjjj

ZYZXZ

ZYYXY

ZXYXX

W

1

2

2

2

2

2

2

.

iLiLiLiLiLiiLiiLiiL

iLiLiLiLiLiiLiiLiiL

iLiLiLiLiLiiLiiLiiL

iLiiLiiLiiiiii

iLiiLiiLiiiiii

iLiiLiiLiiiiii

ZYZXZZZZ

ZYYXYYYY

ZXYXXXXX

ZYX

ZYX

ZYX

iW

Page 29: Close Range Photogrammetry

Direct Linear Transformation (DLT)

This method does not require fiducial marks and can be solved without supplying initial approximations for the parameters

Collinearity equations along with the correction for lens distortion

δx, δy – lens distortion

fx – pd in the x direction

fy – pd in the y direction

Rearranging the above two equations

Monday, March 24, 2008 EE 5358 Computer Vision 29

)()()(

)()()(

)()()(

)()()(

333231

2322210

333231

1312110

LALALA

LALALAyya

LALALA

LALALAxxa

ZZmYYmXXm

ZZmYYmXXmfxy

ZZmYYmXXm

ZZmYYmXXmfxx

1

1

11109

8765

11109

4321

ZLYLXL

LZLYLXLy

ZLYLXL

LZLYLXLx

ya

xa

LmfmyL

LmfmyL

LmfmyL

LZmYmXmfxL

LmfmxL

LmfmxL

LmfmxL

y

y

y

LLLx

x

x

x

/)(

/)(

/)(

/)(

/)(

/)(

/)(

233307

223206

213105

13121104

133303

123202

113101

Page 30: Close Range Photogrammetry

DLT(contd.)

The resulting equations are solved iteratively using LSM

Advantages - No initial approximations are required for the unknowns. Limitations - Requirement of atleast six 3D object space control points - Lower accuracy of the solution as compared with a rigorous bundle adjustment

Monday, March 24, 2008 EE 5358 Computer Vision 30

)(

/

/

/

/)(

333231

3311

3210

319

23222108

LLL

LLLy

ZmYmXmL

LmL

LmL

LmL

LZmYmXmfyL

18

4

1

11109

765

321

L

L

LLL

LLL

LLL

Z

Y

X

L

L

L

Page 31: Close Range Photogrammetry

Analytical Self Calibration

The equations take into account adjustment of the calibrated focal length, principal-point offsets and symmetric radial and decentering lens distortion.

xa, ya – measured photo coordinates related to fiducials

xo, yo – coordinates of the principal point

= xa – xo where

= ya - yo

Monday, March 24, 2008 EE 5358 Computer Vision 31

q

sfyxpyxprprkrkrkyyy

q

rfyxpyxprprkrkrkxxx

aaaaaaaaaa

aaaaaaaaaa

]2)3()[1()(

]2)3()[1()(

122

222

36

34

22

10

222

122

36

34

22

10

axay

)()()(

)()()(

)()()(

232221

131211

333231

LALALA

LALALA

LALALA

ZZmYYmXXms

ZZmYYmXXmr

ZZmYYmXXmq

Page 32: Close Range Photogrammetry

Analytical Self Calibration(contd.)

k1, k2, k3 = symmetric radial lens distortion coefficients

p1, p2, p3 = decentering distortion coefficients

f = calibrated focal length

r, s, q = collinearity equation terms

Provides a calibration of the camera under original conditions which existed when the photographs were taken.

Geometric Requirements

- Numerous redundant photographs from multiple locations are required, with sufficient roll diversity

- Many well-distributed image points be measured over the entire format to determine lens distortion parameters

The numerical stability of analytical self calibration is of serious concern.

Monday, March 24, 2008 EE 5358 Computer Vision 32

222aaa yxr

Page 33: Close Range Photogrammetry

Applications

Automobile ConstructionMachine Construction, Metalworking, Quality ControlMining EngineeringObjects in MotionShipbuildingStructures and BuildingsTraffic EngineeringBiostereometrics

Monday, March 24, 2008 EE 5358 Computer Vision 33

Page 34: Close Range Photogrammetry

Biomedical Applications

Linear tape and caliper measurements of inherently irregular three-dimensional biological structures are inadequate for many purposes.

Subtle movements produced by breathing, pulsation of blood, and reflex correction for control of postural stability.

Short patient involvement times, avoids contact with the patient and thereby avoiding risk of deforming the area of interest and spreading infection.

All medical photogrammetric measurements require further interpretation and analysis to allow meaningful information to be given to the end-user.

Monday, March 24, 2008 EE 5358 Computer Vision 34

Page 35: Close Range Photogrammetry

Bibliography

• Kamara, H.M. (1979). Handbook of Non-Topographic Photogrammetry, American Society of Photogrammetry.

• Wolf , Paul R., Dewitt, Bon A. (2000). Elements of Photogrammetry, McGraw Hill.

• Devarajan, Venkat and Chauhan, Kriti (Spring 2008). Lecture Notes: Mathematical Foundation of Photogrammetry, EE 5358 University of Texas at Arlington.

• Karara, H.M. (1989). Non-Topographic Photogrammetry, American Society for Photogrammetry and Remote Sensing.

• Mitchell, H.L. and Newton, I. (2002). Medical photogrammetric measurement: overview and prospects. ISPRS Journal of Photogrammetry & Remote Sensing, 56, 286-294.

Monday, March 24, 2008 EE 5358 Computer Vision 35

Page 36: Close Range Photogrammetry

Acknowledgments

Dr. Venkat Devarajan

Kriti Chauhan

Monday, March 24, 2008 EE 5358 Computer Vision 36