Oliver Williams

University of Cambridge

Andrew Blake

Microsoft Research,

Cambridge

Roberto Cipolla

University of Cambridge

A Sparse Probabilistic Learning Algorithm

for Real-Time Tracking

exploit temporal

coherence

object detector Self starting

Self recovering

Efficient

Robust Tracking

Background: Support Vector Tracking

Avidan [CVPR 2001]

SVM on object class

Treat score as function of

position

-100 -80 -60 -40 -20 0 20 40 60 80 100-3

-2

-1

0

1

2

3

Translation (pixels)S

VM

Sco

re

subimage

image gradient

Learn right versus left

Training for displacement

positive example

negative example

SVM

Translation (pixels)

SVM

Sco

re

-u

+u

Calibrate state update from SVM score

Relevance Vector

Machine

Probabilistic

Statistical filter

Want sparsity

Computational efficiency

Temporal Fusion

Tipping [NIPS 2000]

SVM in Bayesian setting

Learns continuous function from training set {zi,ti}

RVM Evaluation Equation

weights

kernel function

test subimagetraining subimage

Setting some wi to zero sparse solution

Bayesian Training

Sparsity encouraged by zero-mean prior :

Training data modelled by Gaussian likelihood function

hyperparameter

provided target

“true” underlying value noise parameter

-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5

Occam’s Razor

Sometimes, lower Occam

penalty paid by removing an

example…

…and explaining data with

more noise

-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5-5 -4 -3 -2 -1 0 1 2 3 4 5

Relevance Vector Machine

Posterior

Train RVM by finding hyperparameters

Maximise marginal likelihood

“Prune” vector i when i ! 0

Creating a Training Set

δX1

Select a few “seed” stills

Simulate translation, scaling and rotation

labelled training set

δX3

X2

RVM Tracking

X1 X2 X3 XN

4D RVM learns mapping

{zi}

{ti}

Input Space (images)

State space (u,v,s,)

R400

R4

RVM

RVM Tracking

Frame t Frame t+1

XtXt+1

xt x*t+1 xt+1

RVM Tracking

At each frame…

RVM

X

Statistical filter

Xt+1

Extract subimage

Random variable

New state estimate

Motion Model

x*t+1

xt+1

Initialization & Recovery

Algorithm trains from “seed”

subimages

Provided by localisation algorithm

Localisation algorithm also used for

Initialization (frame 0)

Validation

Re-initialization

Results: Computational Efficiency

Why track?

0 10 20 30 40 50 60 70 80 9010

1

102

103

CP

U t

ime

/ fra

me

(m

s)

Frame Number

Restarting

Results: Computational Efficiency

Localize with boosting [Viola, Jones 2001, Li et al. 2002]

0 10 20 30 40 50 60 70 80

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time (s)

CP

U U

tiliz

ation

Boosting

RVM

Automatic Camera Management

Use position/scale information to control digital pan and

zoom

Conclusions

Future Work

RVM sparsity-smoothness tradeoff

Robustness to illumination and occlusion

Tracking 3D motion

Hybrid RVM tracking is…

Self starting SVM/Boosting detector

Self recovering SVM/Boosting validation

Efficient Temporal fusion +

sparsity

Questions?

Results: Cars

Algorithm is not specific to any class of objects

cinematographer

START

Establishment

Shot

Medium View

Zoom Withdraw

Pan &

Withdraw

Pan

Detect

New Pos

Big Change

Lost

Lost

1D example

Tutorial: toy data

100 Training

examples

12 Relevant vectors

remain

0 10 20 30 40 50 60 70 80 90 100-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

Results: Smoothness

0 100 200 300 400 500 600 7000

5

10

15

20

25

30

35

0 100 200 300 400 500 600 7000

5

10

15

20

25

30

35

RVM Boosting