Recognition of object by finding correspondences between features of a model and an image. Alignment repeatedly hypothesize correspondences between minimal

Recognition of object by finding correspondences between features of a model and an image.

Alignment repeatedly hypothesize correspondences between minimal set of features of a model and an image and then tries to find model poses.

For computing poses a model of projection must be selected.

A minimal number of points needed to compute a model pose is three.

Alignment

General idea:

1. Given an input image and a candidate model, establish correspondence between them.

2. Determine transformation from the model to the image

3. Apply the recovered transformation to the model

4. Compare the transformed model with the viewed object

5. Based on this comparison choose the best model

Alignment cont.

General steps before alignment : 1. Selection of object of interest in the picture.2. Segmentation – delineation of a sub-part of the image

to which subsequent recognition process will be applied.

3. Image description – extraction of information which will be used for matching the viewed object with stored object models

4. Extracting an alignment key. Alignment key is an information used to bring the viewed object and models into alignment.

Before, During and After Alignment

Alignment1. Viewed object is brought into correspondence with a

large number of models stored in the memory.2. Individual alignmentsGeneral steps after alignment:

1. Indexing (classification)– use some criteria to “filter out” unlikely models.

Matching

Before, During and After Alignment

We consider a work “3D pose from 3 corresponding points under weak perspective projection” of T.D.Alter

The problem is to determine the pose of 3 points in space given 3 corresponding points in image.

It gives direct expressions for 3 matched model points in image coordinates and an expression of a position in the image of any additional, unmatched model point.

3D pose from 3 corresponding points

Hypothesize a correspondence between three model points and three image points.Compute the 3D pose of the model from three-point correspondence.Predict the image positions of the remaining model points and extended features using the 3D pose.Verify whether the hypothesis is correct by looking in the image near the predicted positions of the model features for corresponding image features.

Alignment Algorithm

Fig.1 Model points undergoing perspective projectionto produce image points

The perspective solution

0 1 2, ,m m m

0 1 2, ,i i i

Let image points be extended as follows: Then

The problem is: givenfind a,b, and c. From the law of cosines:

Given a, b, and c, we can compute the 3D locations of themodel points:

The perspective solution cont.

0 1 2, ,i i i

( , ) ( , , )x y x y f

01 0 1 02 0 2 12 1 2ˆ ˆ ˆ ˆ ˆ ˆcos , cos , cosi i i i i i

01 02 12 01 02 12, , , cos ,cos ,cosR R R

2 2 201 012 cosa b ab R

2 2 202 022 cosa c ac R

2 2 212 122 cosb c bc R

0 0 1 1 2 2ˆ ˆ ˆ, ,m ai m bi m ci

Approximate perspective projection closely in many cases.

Less complicated.

Conceptually simpler.

We do not need to know the camera focal length and the central point.

Fewer solutions (four for perspective an two for weak perspective).

Justification of the weak perspective approximation

Weak-Perspective Solution

Fig.2 Model points undergoing orthographic projection plusScale to produce image points

0 1 2, ,m m m

0 1 2, ,i i i

To recover the 3D pose of the model we should know the distances between the model points

and distances between the image points

The parameters of the geometry in Fig. 2 are (will be proved

later ):

See eq. (7)-(13).

Weak-Perspective Solution cont.

01 02 12( , , )R R R01 02 12( , , )d d d

2b b acs

a

2 2 2 21 2 01 01 02 02( , ) ( ( ) , ( ) )h h sR d sR d

1 2 1 2

1( , ) ( , )H H h h

s

Computing the Weak-Perspective Solution

From Fig. 3 we have three constraints:

2 2 21 01 01

2 2 22 02 02

2 2 21 2 12 12

( ) (1)

( ) (2)

( ) ( ) (3)

h d sR

h d sR

h h d sR

Computing the Weak-Perspective Solution cont.

Multiplying (3) by –1 and adding all three gives

Squaring (4) and using (1) and (2) to eliminate and

which leads to biquadratic in s :

2 2 2 2 2 2 21 2 01 02 12 01 02 122 ( ) ( ) (4)h h s R R R d d d

21h 2

2h

2 2 2 2 2 2 2 2 2 2 2 2 2 201 01 02 02 01 02 12 01 02 124( )( ) ( ( ) ( )) (5)s R d s R d s R R R d d d

4 22 0 (6)as bs c

where

The positive solutions of biquadratic are


2 2 2 2 2 201 02 01 02 12

01 02 12 01 02 12 01 02 12 01 02 12

4 ( )

( )( )( )( )

a R R R R R

R R R R R R R R R R R R

2 2 2 2 2 2 2 2 2 201 02 02 01 01 02 12 01 02 12

2 2 2 2 2 2 2 2 2 2 2 201 01 02 12 02 01 02 12 12 01 02 12

2 2 ( )( )

( ) ( ) ( )

b R d R d R R R d d d

d R R R d R R R d R R R

2 2 2 2 2 201 02 01 02 12

01 02 12 01 02 12 01 02 12 01 02 12

4 ( )

( )( )( )( )

c d d d d d

d d d d d d d d d d d d

2

(10)b b ac

sa

From (1),(2) and (4)


2 2 2 21 2 01 01 02 02( , ) ( ( ) , ( ) ) (11)h h sR d sR d

2 2 2 2 2 2 201 02 12 01 02 121 ( )

(12)1

if d d d s R R R

otherwise

1 2 1 2

1( , ) ( , ) (13)H H h h

s

The solution fails when the model triangle degenerates to a line, at which case a=0.


Let the image points be

Given we can invert the projection to get the tree model points:

where unknown w can not be recovered.

Image location of a fourth model point.

0 0 0

1 1 1 1

2 2 2 2

1( , , )

1( , , ) (14)

1( , , )

m x y ws

m x y h ws

m x y h ws

0 0 0 1 1 1 2 2 2( , ), ( , ), and ( , )i x y i x y i x y

1 2, ,s h h

Denote the model points in arbitrary model coordinate

frame. Using solve the following vector

equation for the “extended affine coordinates”, of

Let

Using the three model points with


0 1 2, , andp p p

( , , )

ip

3p

3 1 0 2 0 1 0 2 0 0( ) ( ) ( ) ( ) (15)p p p p p p p p p p

01 1 0 01 1 0

02 2 0 02 2 0

,

(16)

x x x y y y

x x x y y y

0 0 1 1 2 2, , andp m p m p m

1 0 01 01 1

1( , , ) (17)p p x y h

s

2 0 02 02 2

1( , , ) (18)p p x y h

s

1 0 2 0 01 2 02 1 02 1 01 2 01 02 02 012

1( ) ( ) ( , , ) (19)p p p p y h y h x h x h x y x y

s

Substituting (17)-(19) into (15) we’ll get


3 01 01 1 02 02 2

01 2 02 1 01 2 02 1 01 02 02 01 0 02

01 2 02 101 02 0

01 2 02 101 02 0

01 021 2

1 1( , , ) ( , , )

1 1)

1

m x y h x y hs s

y h y h x h x h x y x y x y ws s

y h y hx x x

s sx h x h

y y ys

x yh h

02 01 ) (20)x y

ws

To project, first apply the scale factor s :

Then project orthographically to get the image location of the

fourth point:


01 2 02 13 01 02 0

01 2 02 101 02 0

01 02 02 011 2 ) (21)

y h y hsm x x x

sx h x h

y y ys

x y x yh h w

s

3 01 02 01 2 02 1 0

01 02 01 2 02 1 0

( ) ( )

( ) ) (22)

sm x x y H y H x

y y x H x H y

3D pose from 3 corresponding points under weak perspective projection” T.D.Alter., MIT A.I.Memo No. 1378, 1992

3D Pose from 3 Points Using Weak Perspective T.D.Alter, IEEE Transactions on Pattern Analysis and Machine Intelligence,v.16 No.8,1994

References

Documents

Recognition of object by finding correspondences between features of a model and an image. Alignment repeatedly hypothesize correspondences between minimal