The Shape of Things to Come Shape Theory for Object ...md9c/presentations/UVA... · The Shape of Things to Come*: Shape Theory for Object Recognition Michael D. DeVore SIE Graduate

The Shape of Things to Come*:Shape Theory for Object Recognition

Michael D. DeVore

SIE Graduate Seminar

University of Virginia

November 7, 2003

*Perhaps better put as “The Shape of Things Coming Toward You”

2

Definition of Object Recognition

• Collect measurements from a scene and decide whatobjects are present

• May also be interested in other info about the object– Dimensions

– Orientation

– Lighting conditions

– etc.

Object Recognition

Definition

Measurementand shape

ResearchLaboratory

Need for ShapeTheory

Need from a ShapeTheory

ProcrusteanAnalysis

Examples

SensorPlatform

InferenceSystem

ModelDatabase

HondaCivic

3

Measurement Variability

• Image variation (randomness) due to:– Lighting variation

– Surface color variation

– Shape variation

Object Recognition

Definition


ResearchLaboratory



ProcrusteanAnalysis

Examples

4

Shape is Fundamental

• Some sensors collect shape data directly

LADAR image of a ChevroletBlazer and a Hummer

From “LADAR puts the puzzletogether” by Robert Hauge. SPIE’sOE Magazine, April 2003.

A recognition system must (implicitly or explicitly)account for shape and shape variation

Object Recognition

Definition


ResearchLaboratory



ProcrusteanAnalysis

Examples

5

Research Laboratory

• Equipped to collect image data under computer control

• Structured-light laser to collect shape data

• Beowulf cluster for analysis

Object Recognition

Definition


ResearchLaboratory



ProcrusteanAnalysis

Examples

6

Research Laboratory

• Collaborators at UVa– Xin Wu

– Josh Stafford

– Shankha Basu

– 4th Year Capstone GroupSaptarshi Adhikari, James Fang,

Shadie Wadie, Guenevere Lindgren

– David Luebke (Asst. Prof., CS)

Object Recognition

Definition


ResearchLaboratory



ProcrusteanAnalysis

Examples

7

Fields: Anthropology

• In many fields, shape data are of central importance• Anthropological studies often concerned with the

evolution of physical features

Object Recognition


Anthropology

Archaeology

Medicine

Others


ProcrusteanAnalysis

Examples

1. Modern Human2. Neanderthal

3. Australopithecine4. Chimpanzee

“The Order of Man: A Biomathematical Anatomy of the Primates,” C. E. Oxnard

8

Fields: Archaeology

• Archaeological investigations often focus onevaluation of the shape of artifacts

Iron Age Brooches

Hodson, Sneath, and Doran. Biometrika 1966.

Object Recognition


Anthropology

Archaeology

Medicine

Others


ProcrusteanAnalysis

Examples

9

Fields: Medicine

• Medical studies often focus on the shape of organs

Location of the hippocampus

3D shape model of ahippocampus

From web site of Sarang Joshi,http://cis.jhu.edu/~sarang

Object Recognition


Anthropology

Archaeology

Medicine

Others


ProcrusteanAnalysis

Examples

10

Fields: Others

• Astronomy

• Biology

• Geology

• Cartography

• Manufacturing

• etc.

Object Recognition


Anthropology

Archaeology

Medicine

Others


ProcrusteanAnalysis

Examples

11

Requirements of a Theory

1. Represent the shape of an arbitrary object

2. Quantify the difference between shapes

3. Represent randomness in shape

Object Recognition



Requirements

CasualDefinition

Landmarks

ProcrusteanAnalysis

Examples

12

Casual Definition

• Shape of a data set is the collection of attributes thatremain unchanged under transformations of:– Scale

– Location

– Rotation

• Two data sets have the same shape if:– can be made equal by transformations of scale, location,

and rotation

• Shape distance between two data sets:– smallest distance between sets under all possible

transformations of scale, location, and rotation

Object Recognition



Requirements

CasualDefinition

Landmarks

ProcrusteanAnalysis

Examples

13

Geometrical Figures

• Data set to represent a geometric figure:– Labeled collection of landmark location

• Landmarks chosen to:– Give a partial geometric characterization

– Be identifiable across individuals

• We will use the shape of leaves as an example in 2D

Object Recognition



Requirements

CasualDefinition

Landmarks

ProcrusteanAnalysis

Examples

14

Leaf Landmarks

• Landmarks will be the tips and valleys of the mainlobes

Object Recognition



Requirements

CasualDefinition

Landmarks

ProcrusteanAnalysis

Examples

15

Landmark Representation

• Procrustean Analysis:– What must I do to one set of landmarks to make it match

another set?

• N landmarks represented by a matrix

X = [x1, x2, …, xN]

where xm is a column vector of length 2 for points inthe plane (3 for points in space)

• Matrix X is an element of Euclidian space ¬2N

Object Recognition



ProcrusteanAnalysis

LandmarkRepresentation

Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

16

Pre-Shape

• Location is defined as

• Scale is defined as

• By pre-shape we mean the data set with location andscale removed

†

x = 1N

xnn=1

N

Â

†

x = xn - x 2

n=1

N

Â

†

t X( ) =1x

x1 - x , x2 - x ,K , xN - x [ ] = [t1,t 2,K,t N ]

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

17

Example: Pre-Shape

Original Landmarks After Location Removal

Pre-Shape (location and scale removed)

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

18

Example: Pre-Shape Retains Orientation

Small Maple Leaf and Landmarks

Small Maple Leaf Pre-Shape Big Maple Leaf Pre-Shape

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

19

Space of Pre-Shapes

• We can think of a pre-shape t as N points in ¬2

• We can think of a pre-shape t as 1 point in ¬2N

• Each t is an element of

(A 2N-2 dimensional hyperplane containing the origin)

and an element of

(A 2N-1 dimensional hypersphere centered at the origin)

†

F 2N-2 = x1,x2,K,xN( ) Œ ¬2N : xnn=1

N

Â = 0Ï Ì Ó

¸ ˝ ˛

†

S2N-1 = x1,x2,K,xN( ) Œ ¬2N : xn2

n=1

N

Â =1Ï Ì Ó

¸ ˝ ˛

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

20

Space of Pre-Shapes

• The intersection, S*2N-3 = F2N-2 « S2N-1, is a 2N-3

dimensional hypersphere made by “slicing” S2N-1

• Call S*2N-3 the pre-shape space

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

21

Orientation of a Pre-Shape

• Not generally possible to “remove” orientation like wedid with location and scale

• Rotation of the original figure corresponds to a rotationof t

• Set of all possible rotations of t is called the orbit of t

• The orbit defines a set of equivalence classes

• Two pre-shapes t and n are equivalent if they have thesame orbit: O(t) = O(n)

†

O t( ) =cosq sinq

-sinq cosq

È

Î Í

˘

˚ ˙ t : 0 £ q < 2p

Ï Ì Ó

¸ ˝ ˛

Ã S*2N-3

†

SN = O t( ) : t Œ S*2N-3{ }

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

22

Definition of Shape

• By the shape of a labeled set of points, X = [x1, x2, …,xN] we mean the orbit of its pre-shape O(t(X))

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

23

Distance Between Pre-Shapes

• Pre-shapes lie on the unit hypersphere S*2N-3

• Great circle distance between t and n is:

d(t, n) = cos-1(t • n)

• Values between 0 and p

Recall:t • n = cos(q)

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

24

Distance Between Shapes

• Distance between shapes O(t) and O(n) is theminimum distance between pre-shapes in the orbits

• To get a closed-form solution:– Note that cos-1 is minimized when its argument is

maximized (both t and n are unit vectors)

– Differentiate the dot product with respect to q

– Set the result equal to zero

†

d O t( ),O n( )[ ] = minq1 ,q 2

dcosq1 sinq1

-sinq1 cosq1

È

Î Í

˘

˚ ˙ t ,

cosq2 sinq2

-sinq2 cosq2

È

Î Í

˘

˚ ˙ n

Ê

Ë Á

ˆ

¯ ˜

= minq

d t ,cosq sinq

-sinq cosq

È

Î Í

˘

˚ ˙ n

Ê

Ë Á

ˆ

¯ ˜

= minq

cos-1 t ,cosq sinq

-sinq cosq

È

Î Í

˘

˚ ˙ n

Ê

Ë Á

ˆ

¯ ˜

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

25

Distance Between Shapes

• Solution:

†

tan q( ) =

t n,1n n,2 - t n,2n n,1( )n=1

N

Â

t n,1n n,1 + t n,2n n,2( )n=1

N

Â

†

d O t( ),O t( )( ) = cos-1 t n,1n n,1 + t n,2n n,2( )n= 0

N

ÂÈ

Î Í

˘

˚ ˙

2

+ t n,2n n,1 - t n,1n n,2( )n= 0

N

ÂÈ

Î Í

˘

˚ ˙

2Ê

Ë

Á Á

ˆ

¯

˜ ˜

Object Recognition



ProcrusteanAnalysis


Pre-Shape

Space of Pre-Shapes

Orientation

PreciseDefinition

Distance

Examples

Example: Leaf Shapes

Big Maple Leaf

Small Maple Leaf

Big Ivy Leaf

Small Ivy Leaf Small Tulip Poplar Leaf

Big Tulip Poplar Leaf

27

Example Leaf Shape Distances

• Distances between shapes for each pair of leaves

00.11160.77560.63960.64470.5763SmallPoplar

0.111600.71380.58970.57430.5185BigPoplar

0.77560.713800.23720.31970.3420SmallIvy

0.63960.58970.237200.38800.3609Big Ivy

0.64470.57430.31970.388000.1364SmallMaple

0.57630.51850.34200.36090.13640BigMaple

SmallPoplar

BigPoplar

SmallIvy

Big IvySmallMaple

BigMaple

Object Recognition



ProcrusteanAnalysis

Examples

28

Concluding Remarks

• Randomness in the object geometryfi Randomness in the pre-shape

fi Distribution over the orbits

• Extensions from vector spaces to function spaces– Shape representations that account for the entire edge

• Eliminate landmark identification– Distance metrics not dependent on point correspondence

• Joint representation of color and shape– Random color over a random shape

Object Recognition



ProcrusteanAnalysis

Examples

Documents

The Shape of Things to Come Shape Theory for Object ...md9c/presentations/UVA... · The Shape of Things to Come*: Shape Theory for Object Recognition Michael D. DeVore SIE Graduate