Animating Animal Motion From Still

Presenting:Shahar Ben EzraSaar Nakibli

Supervisor: Hayley Binia Wolman

Computer Graphics LabElectrical Engineering, Technion, Israel

June 2009

[1]

[1] Xuemiao Xu , Animating Animal Motion From Still, Siggraph 2008

Project Goal

• Create an animation movie by reconstructing the animal motion from a still picture

Algorithm

Still Image

Animation Movie

Quick Demonstration

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Snapshot Extraction•Extract the snapshots using a tool called “MVP-PIE” developed at the CGM lab at the Technion.

•This stage is not part of our project but it’s a necessary step in order to extract the snapshots from the source image.

Overview

Find distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Shape Context

Descriptor

head

tail

Shape Context (Cont.)

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

50 100 150 200 250 300 350 400120

140

160

180

200

220

240

260

280

300

320

•User Interaction: choose two anchor points that define the movement direction of the animal.

•Rotate the snapshot in the angle defined by the user.

•Find the contour levels of the snapshot using the Matlab Image Proccesing Toolbox.

•Take the last contour level and decimate it, and display it on the XY plane.

Shape Context (Cont.)

-200 -100 0 100 200 300 400 500 600-300

-200

-100

0

100

200

300

400

500370

•Find the contour of the snapshot

•For every point on the contour draw the circles and bins as shown

•Create the “matrix descriptor “

•Invariant to translation, rotation and scale.

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Distance Matrix

* *

1 1,

k l

k l i j i ji M j Mk l

D S S f h f hM M

2

,1

1

2

Ni j

i jk i j

h k h kC

h k h k

k

lDistance between Snapshot-K and Snapshot-L:

Point on snapshot K

Closest point on snapshot L

How do we find ? *jh

Distance between two matrix descriptors

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Path Finding •Find the optimal path between the snapshots using the distances matrix•Looking to find a path which minimizes the Energy function:

• Local similarity:

• Sampling uniformity:

• Global distinction:

•We use the Simulated Annealing Optimization Algorithm• Avoid “getting stuck” on a local minimum, because of the

Temperature factor. • Ignore outliers that don’t belong in the motion cycle.• Ignore snapshots that are too similar to other snapshots.

1 2 1s u dC I C C C 1,s i iC E D I I

1,u i iC Var D I I

,

,

1

i ji j j i

d

D I I

CN N

•Initialize a path •While (T>Limit)• Loop K times• Choose a new path length Lcurr • Generate new Path Pcurr at length Lcurr and price Ccurr • If (Ccurr – Cold ) < 0• Accept current path (trivial). Update parameters.

• Else if exp{(Ccurr – Cold )/T} < rand[0,1] • Accept current path. Update parameters.

• Else• Reject current path.

• Decrease Temprature T = T*Annealing_Factor• End•End

Path Finding (Cont.)

Motion Cycle

Full Cycle:

Half Cycle:

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Pose Consistency

Pivot

•In order to create smooth and realistic animation, all the animals have to be in the same pose relative to the camera.•Affine transformation includes: • Translation• scale• rotation

•All snapshots are translated relative to a pivot snapshot.

0 50 100 150 200 250 3000

100

200

300

400

500

600

700

800

900

4

0 50 100 150 200 250 3000

100

200

300

400

500

600

700

800

0 50 100 150 200 250 3000

100

200

300

400

500

600

700

800

900

Appearance Consistency

Pivot histogram:

•Every snapshot has its own color and texture that can vary from one snapshot to another.

•Histogram standardization of all snapshots will make the final animation look smoother.

8

4

53.9pivot

68.7before 54.1after

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

TPS Morphing1

2

3 4 5

6

789

10•Finding animation points:• Points which are not static

during the animal motion

•Morph between the source points and target points• Source Image is registered to the green points• Destination Image is registered to the red points.

Overview

Shape Context

DescriptorFind

distances

Path Finding

6,3,4,1,2,5

Consistency Refinement

TPSMorphing

Creating Animation

snapshots Shape context descriptors

Distances matrix

Motion PathConsistent snapshots

Animation between two

snapshotsFinal Movie

Source Image

Snapshot Extraction

Creating AnimationMorph three frames between two snapshots

Source Destination

0.750.25 0.5

0.50.75 0.25

0.250.5

0.751

0

0 50 120

150 200

Creating Animation (Cont.)•Blend the morphed frames into the background image, by the following formula:

•The Threshold Value (T) was empirically found and set to T=120.

, ,

,, ,

BW

BW

Image i j if Image i j TFrame i j

Background i j if Image i j T

T=50 T=120

T=150 T=200

Results - Turtles

Output Animation:

Input Images:

Results - Turtles

Output Animation:

Input Images:

Results - Tadpoles

Output animation:

Input Image: