Tracking with Online Tracking with Online Appearance ModelAppearance Model

Bohyung HanBohyung Han

bhhan@cs.umd.edubhhan@cs.umd.edu

IntroductionIntroduction

Tracking algorithmTracking algorithm– Deterministic: mean-shiftDeterministic: mean-shift– Probabilistic: Condensation algorithmProbabilistic: Condensation algorithm

Model templateModel template– Fixed in most of tracking algorithmsFixed in most of tracking algorithms– Requires to be updated for the robust trackingRequires to be updated for the robust tracking– But, how?But, how?

ReferenceReference[1] A. Jepson, D. Fleet, T. El-Maraghi, “Robust Online [1] A. Jepson, D. Fleet, T. El-Maraghi, “Robust Online

Appearance Models for Visual Tracking,” CVPR 2001Appearance Models for Visual Tracking,” CVPR 2001[2] S. Zhou, R. Chellappa, B. Moghaddam, [2] S. Zhou, R. Chellappa, B. Moghaddam,

“Appearance Tracking Using Adaptive Models in A “Appearance Tracking Using Adaptive Models in A Particle Filter,” ACCV 2004Particle Filter,” ACCV 2004

Related WorkRelated Work

Adaptive color feature selectionAdaptive color feature selection– Stern and EfrosStern and Efros

choose 5 feature spaces (RG, rg, HS, YQ, CbCr) choose 5 feature spaces (RG, rg, HS, YQ, CbCr) and switch amongst them in each frameand switch amongst them in each frame

– Collins and LiuCollins and Liu build ranking system for the feature selectionbuild ranking system for the feature selection

Feature value weightingFeature value weighting– ComaniciuComaniciu

assigns different weight for each pixel considering assigns different weight for each pixel considering the backgroundthe background

Target model updateTarget model update Adaptive process model in particle filterAdaptive process model in particle filter

Basic IdeaBasic Idea

Three componentsThree components– StableStable: learned with a long-term course: learned with a long-term course– WanderingWandering: 2-frame transient component: 2-frame transient component– LostLost (outlier) [1] or Fixed [2] (outlier) [1] or Fixed [2]

IdeaIdea– By identifying stable properties of By identifying stable properties of

appearance, we can weight them more appearance, we can weight them more heavily for motion estimation.heavily for motion estimation.

– On-line EM algorithm for the parameter On-line EM algorithm for the parameter estimationestimation

Probabilistic mixture [1]Probabilistic mixture [1]

– : observation: observation– : mixing probabilities: mixing probabilities– : mean and covariance of : mean and covariance of

stable componentstable component Log-likelihood of observation historyLog-likelihood of observation history

)();(

);()|( 11

tllttss

ttwwtttt

dpmdpm

ddpm,d,dp

q

mq

WSL Appearance WSL Appearance ModelModel

),,( lsw mmmmtd

),( 2,, tstst q

tkkttktttt d,dpkS,L ),|(log)()|( 1mqmqd

WSF Appearance WSF Appearance Model Model Probabilistic mixture [2]Probabilistic mixture [2]

– Every component is modeled with Every component is modeled with Gaussian.Gaussian.

Observation likelihoodObservation likelihood

)();(

);()|( 11

tffttss

ttwwtttt

dpmdpm

ddpm,d,dp

q

mq

p

j fswititittitt jjjZNjmp

1 ,,

2,,, ))(),();(()()|( xz

EM AlgorithmEM Algorithm

Purpose & methodologyPurpose & methodology– Need to estimate parameters mixing Need to estimate parameters mixing

probabilities and Gaussian parameters for probabilities and Gaussian parameters for stable componentstable component

– Online approximate EM algorithmOnline approximate EM algorithm Sketch of estimation processSketch of estimation process

– Parameters in previous stepParameters in previous step– new mixing probabilities by computing the new mixing probabilities by computing the

posterior responsibility probabilitiesposterior responsibility probabilities– S: using the first- and second-moment imagesS: using the first- and second-moment images– W, F: very simpleW, F: very simple– L: no parameter (uniform distribution)L: no parameter (uniform distribution)

EM AlgorithmEM Algorithm

Batch stepBatch step Incremental Incremental modificationmodification

EM AlgorithmEM Algorithm

Parameter EstimationParameter Estimation

Tracking [1]Tracking [1]

Motion-based TrackingMotion-based Tracking– Wavelet-based appearance modelWavelet-based appearance model– Maximizes the sum of data log likelihood Maximizes the sum of data log likelihood

and log prior by EM algorithmand log prior by EM algorithm

Tracking [2]Tracking [2]

Appearance-based Tracking [2]Appearance-based Tracking [2]– Particle filterParticle filter– Adaptive process modelAdaptive process model– Variable number of particlesVariable number of particles– Online target model updateOnline target model update