Oklahoma State University

Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis

Xin Fan and Guoliang Fan

Visual Computing and Image Processing Lab

School of Electrical and Computer Engineering

Oklahoma State University

4th Joint IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum(OTCBVS'07)Minneapolis, MN, USA, June 22, 2007

2

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Problem StatementIntroduction Problem Statement

Related Work

Generative model

Experimental Results

Conclusions

Motion models Deal with object movements Why important?

complex motion patterns, e.g., maneuvering no good appearance model, e.g., low SNR provide good prediction for robust and efficient tracking

Challenges Hardly predict maneuvering actions Model constraints

A motion model that incorporates constraints

3

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Our observations Maneuvering actions are due to forces and torques.

forces and torques cause kinematic changes

Newton equations for rigid body motion Rigid body motion VS point motion Newton equations

Forces are dependent on kinematics Limited output power of engines.

Uncertainties exist, e.g., air resistance, road friction, mechanical instability, etc.

Introduction Problem Statement

Related Work

Generative model

Experimental Results

Conclusions z

4

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Problem FormulationIntroduction Problem Statement Problem Formulation

Related Work

Generative model

Experimental Results

Conclusions

Switching statistical models for maneuvering variables (forces and torques)

Maneuvering actions are due to forces and torques.

Newton equations to define kinematics evolution densities

Newton equations of rigid body motion

Rayleigh distribution to model velocity-force constraints

Physical constraints reveal how forces are dependent on kinematics.

Organize these dependencies with a probabilistic graphical model

5

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Related work

White Gaussian noise acceleration (WGNA) Point target assumption

Miller’s condition mean estimation Rigid Newton dynamics Jump-diffusion process, not sequentially

Switching Linear dynamic system (SLDS) or Jump Markov linear system (JMLS)

Discrete switching variables for maneuvering actions No explicit physical dynamics

Inference algorithms IMM works for Gaussian densities BP works for tree structures Sampling based approximation

Introduction Problem Statement Problem Formulation

Related Work

Generative model

Experimental Results

Conclusions

6

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative Model - StructureIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure

Experimental Results

Conclusions

Generative model How forces and torques generate kinematics changes how kinematics generate observations

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

Cause variables

Effect variables

Effect variables

Cause variables

Observations Frames

Velocity

Position

Orientation

Forces

Torques

7

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model-Cause variablesIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables

Experimental Results

Conclusions

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

Cause variables

Effect variables

Effect variables

Cause variables

Observations

Switching continuous probabilistic models Specify three switching normal distributions for forces. Ternary uniform mixture for torques (angular velocity)

8

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model – Temporal constraintsIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints

Experimental Results

Conclusions

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

Cause variables

Effect variables

Effect variables

Cause variables

Observations

Newton equations Investigate the dynamics of 3D rigid motion Define the kinematic dependence by Newton equ

ations of rigid body motion.

9

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model – Temporal constraintsIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints

Experimental Results

Conclusions

Newton equations for 3D rigid motion

τh

fp

Simplified for ground vehicles

x y

z

f

xy

x

z

z

y

x fv

v

v

v

0

1

0

0

z

y

x

v

v

y

x

cossin

sincos

zzzJ

p-linear momentum and f- force

h - angular momentum and τ- torque

10

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model – Temporal constraintsIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints

Experimental Results

Conclusions

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

11,1 ttztt nT

vttttztt nfT 1111,1

~ vωvv

rtttt nvT 111 )( Arr

kinematics dependency via Newton equations

Velocity

Orientation

Position

11

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model- VF constraintsIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints

Experimental Results

Conclusions

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

Cause variables

Effect variables

Effect variables

Cause variables

Observations

Rayleigh distribution for velocity-force constraints

Driving force conditional on velocities

Resistance force conditional on velocities

12

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model - Likelihood

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

Cause variables

Effect variables

Effect variables

Cause variables

Observations

Simple template matching to define likelihood

Introduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood

Experimental Results

Conclusions

13

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Generative model - InferenceIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference

Experimental Results

Conclusions

ft-1 ft+1ft

rt-1 rt+1rt

vt-1 vt-1vt

Zt-1 Zt+1Zt

Cause variables

Effect variables

Effect variables

Cause variables

Observations

Predict with temporal densities

Evaluate weights with likelihood

MCMC to generate samples of forces

SMC based inference algorithm

14

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Experiments – Simulated data

Compared with a particle filter (PF) for JMLS Tracking with coupled linear and angular motion

No coupling

Introduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference

Experimental Results Simulated data

Conclusions

Ours

JMPF

0 10 20 30 40 50 60

Time

Orie

ntat

ion

Ground Truth

GMJMPF

0 10 20 30 40 50 60

Time

For

ce

Ground Truth

GM

JMPF

0 10 20 30 40 50 60Time

Vel

ocity

Ground Truth

GM

JMPF

-10 0 10 20 30 40 50 60 70-20

0

20

40

60

80

100

120

140

X Coordinate

Y C

oord

inat

e

Ground Truth

GM

JMPF

15

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Experiments – Simulated data

Compared with a particle filter (PF) for JMLS Tracking with coupled linear and angular motion

Has coupling

Introduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference

Experimental Results Simulated data

Conclusions

JMPF

Ours-20 0 20 40 60 80 100 120 140 160 180

-10

0

10

20

30

40

50

60

70

80

90

X Coordinate

Y-c

oord

inat

e

Ground Truth

GM

JMPF

0 10 20 30 40 50 60-5

0

5

10

Time

Vel

ocity

Ground Truth

GMJMPF

0 10 20 30 40 50 60Time

For

ce

Ground Truth

GM

JMPF

0 10 20 30 40 50 60

Time

Orie

ntat

ion

Ground Truth

GMJMPF

16

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Experiments – Simulated data

Compared with a particle filter (PF) for JMLS Tracking with velocity-force constraints

Introduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference

Experimental Results Simulated data

Conclusions

0 10 20 30 40 50 60Time

Orie

ntat

ion

Ground truth

Proposed method

JMPF

0 10 20 30 40 50 60Time

Forc

e

Ground truth

Proposed method

JMPF

0 10 20 30 40 50 60Time

Vel

ocity

Ground Truth

GM

JMPF

0 10 20 30 40 50 60X-Coordinate

Y C

oord

inat

e

Ground Truth

GM

JMPF

17

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Experiments – Real world video

Compared with constant velocity constant turn (CVCT) model

Introduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference

Experimental Results Simulated data Real world video

Conclusions

OursCVCT

0 50 100 150 200 250 300

Time

Orie

ntat

ion

GMCVCT

250 300 350 400 450 5000

50

100

150

200

250

300

X Coordinate

Y C

oo

rdin

ate

GMCVCT180th Frame

0 50 100 150 200 250 300

Time

Ve

loci

ty

GM

CVCT

18

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.

Conclusions and future workIntroduction Problem Statement Problem Formulation

Related Work

Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference

Experimental Results Simulated data Real world video

Conclusions

Conclusions Graphical model for maneuvering targets, which encode

the Newtonian dynamics in a probabilistic framework. Explicitly and directly build the cause-effect relationship Feedback constraint from velocity to the forces

Future work Handle multiple views. Multiple targets with data association