View
215
Download
0
Category
Preview:
Citation preview
Ensemble Tracking
Shai AvidanIEEE TRANSACTIONS ON PATTERN ANALYSIS AND
MACHINE INTELLIGENCE
February 2007
outline
Prior knowledge : AdaboostIntroductionEnsemble trackingImplementation issuesExperiments
Adaboost
Resampling for Classifier Design Bagging
Use multiple versions of a training set• Each created by drawing n’<n samples from D with repla
cement (i.e. if a sample is drawn, it is not removed from D but is reconsidered in the next sampling)
• Each data set is used to train a different component classifier
• The final classification decision is based on the vote of the component classifiers
Adaboost
Boosting To generate complementary classifiers by training th
e next component classifier on the mistakes of the previous ones
• Using a subset of the training data that is most informative given the current set of component classifiers
Adaboost trains a weak classifier on increasingly more difficult examples and combines the result to produce a strong classifier that is better than any of the weak classifiers.
}1,1{)( xhkWeak classifier :
max
1
)()(K
kkk xhxg }1,1{)]([ xgsignyStrong classifier :
Adaboost
AdaBoost(adaptive boosting) Use the same training set over and over
Each training pattern receives a weight Wk(i) The probability that the i-th pattern is drawn to take the kth
component classifier. Uniform initialization W1(i)=1/n
If a training pattern is accurately classified hk(xi)=yi, its chance of used again is reduced
Otherwise, hk(xi)yi
keiWiW kk
)()(1 )1
ln(2
1
t
tt E
E
},..,1),,{( niyx ii }1,1{ iy
keiWiW kk )()(1
)( using on measurederror training iWDE kk
Adaboost
Final decision
k
kk xhxg )()(
Adaboost
Kmax component classifiers
}1,1{)( xhk
max
1
)()(K
kkk xhxg
}1,1{)]([ xgsigny
},..,1),,{( niyx ii }1,1{ iy
n
iii xgyJ
1
))(exp(
1
11 )()(
t
kkkt xhxg
11 ~ t()~() 11 thh
Adaboost
At the t step
)()()()( 11
xhxgxhxg ttt
t
kkkt
n
iitit xgyJ
1
))(exp(
n
iitititi xhyxgy
11 ))()(exp(
n
iititt xhyiw
1
))(exp()(
Adaboost
tt eEeE tt )1(
0)1( tt eEeEJ
ttt
t
tt EeE t 12
)1
ln(2
1
t
tt E
E
Adaboost
))()(()(1
1)( ittitiiti xhxgyxgyt eeiw
)()()( )(1 ittiittiiti xhyt
xhyxgy eiwee
Introduction
Considering tracking as a binary classification problem.
Ensemble tracking as a method for training classifiers on time-varying distributions.
Ensemble of weak classifiers is trained online to distinguish between the object and the background.
Introduction
Introduction
Ensemble tracking maintains an implicit representation of the foreground and the background instead of describing foreground object explicitly alone.
Ensemble is not template-based methods. Those maintains the spatial integrity of the objects and are especially suited for handling rigid objects.
Introduction
Ensemble tracking extends traditional mean-shift tracking in a number of important directions: Mean-shift tracking usually works with
histogram of RGB colors. This is because gray-scale images do not provide enough information for tracking and high-dimensional feature spaces cannot be modeled with histograms due to exponential memory requirements.
Introduction
This is in contrast to existing methods that either represent the foreground object using the most recent histogram or some ad hoc combination of the histograms of the first and last frames.
Introduction
Other advantages: It breaks the time consuming training phase into
a sequence of simple and easy to compute learning tasks that can be performed online.
It can also integrate offline and online learning seamlessly.
Integrating classifier over time improves the stability of the tracker in cases of partial occlusions or illumination changes.
In each frame, we keep the K “best” weak classifiers, discard the remaining T-K new weak classifiers, train T-K new weak classifiers on the newly available data, and reconstruct the strong weak classifier.
The margin of the weak classifier h(x) is mapped to a confidence measure c(x) by clipping negative margins to zero and rescaling the positive margins to the range [0,1].
Ensemble update
Ensemble tracking
During Step 7 of choosing the K best weak classifier, weak classifiers do not perform much better than chance.
We allow up to existing weak classifiers to be removed this way because a large number might be a sign of occlusion and keep the ensemble unchanged for this frame.
Implementations issues
Outlier Rejection
Implementations issues
Implementations issues
Multiresolution Tracking
Implementations issues
experiments
The first version uses five weak classifiers, each working on an 11D feature vector per pixel that consists of an 8-bin local histogram of oriented gradients calculated on a 5x5 window as well as the pixel R, G, and B valuse.
To improve robustness, we only count edges that are above some predefined threshold, which war set to 10 intensity values.
experiments
We found that the original feature space was not stable enough and used a nonlinear version of that feature space instead.
We use only three, instead of five weak classifiers. Three levels of the pyramid In each frame, we drop one weak classifier and add a
newly trained weak classifier.
],,[ 32iii xxx
experiments
We allow the tracker to drop up to two weak classifiers per frame because dropping more than that might be could be a sign of occlusion and we therefore do not update the ensemble in such a case.
experiments
Results on Color Sequences: a pedestrian crossing the streat
experimentsResults on Color Sequences:
tracking a couple walking with a hand-held camera.
experimentsResults on Color Sequences:
tracking a face exhibiting out-of-plane rotations
experimentsResults on Color Sequences: tracking a red car that is undergoing out-of-plane rotations and partial occlusions.
11D feature vector, single scale, an ensemble of three classifier was enough to obtain robust and stable tracking
experimentsAnalyze the importance of the update scheme for tracking:
experimentsAnalyze how often are the weak classifiers updated?
experimentsAnalyze how does their weight change over time.
experimentsAnalyze how does this method compare with a standard AdaBoost classifier that trains all its weak classifiers on a given frame?
experimentsResults on gray-scale sequence :
experiments
Results on IR sequence:
experiments
Handling long-period occlusion Classification rate is the fraction of the number pixels
that were correctly classified As long as the classification rate is high, the tracking
goes unchanged. When the classification level drops(<0.5), switch to
prediction mode. Once occlusion is detected we start sampling, according
to the particle filter, possible location where the object might appear.
In each such location, compute the classification score. If it is above a threshold (0.7), then tracking resumes.
experimentsHandling occlusions:
experiments
Recommended