View
6
Download
0
Category
Preview:
Citation preview
Multiple Targets Tracking in World Coordinate with a Single, Minimally Calibrated Camera.Wongun Choi Silvio Savarese
Department of Electrical Engineering and Computer Science, University of Michigan
Conclusion• Joint estimation of camera and targets’ state helps improve tracking performance.• Other geometric cues further stabilize the system.• Better target association can be achieved by modeling interaction between targets.
Challenges• Unknow (uncalibrated) camera motion.
• Background clutter.
• Detector failure (missed detection).
• Occlusion between targets.
Interaction Model
Zk(t-1) Zk(t) Zk(t+1)
Zj(t-1) Zj(t) Zj(t+1)
Zi(t-1) Zi(t) Zi(t+1) • Pairwise interaction Model
• Two exclusive mode of interactions- Repulsion: people want to keep a distance from others.
- Group Interaction: People moving as a group tend to move together.
- Targets’ motions are dependent.
P (Zt|Zt−1) =i<j
ψ(Zit, Zjt; βijt)i<j
P (βijt|βij(t−1))Ni=1
P (Zit|Zi(t−1))Repulsion
Group Interaction
ψ(Zit, Zjt; βijt) =
ψg(Zit, Zjt), if βijt = 1ψr(Zit, Zjt), otherwise
- Switch variables to select one of the above models.
References[1] Ess, A., Leibe, B., Schindler, K., , van Gool, L.: A mobile vision system for robust multi-person tracking. In: CVPR. (2008)
[2] Khan, Z., Balch, T., Dellaert, F.: Mcmc-based particle tering for tracking a variable number of interacting targets. PAMI (2005)
[3] Pellegrini, S., Ess, A., Schindler, K., van Gool, L.: You'll never walk alone: Modeling social behavior for multi-target tracking. In: ICCV. (2009)
[4] Hoiem, D., Efros, A., Hebert, M.: Putting objects in perspective. In: CVPR. (2006)
Overview• Objective : robustly track multiple targets in 3D.
• Modeling interaction between targets.
• MCMC particle ltering for ecient estimation.
• Ground feature points for robust camera estimation.
• Joint estimation of camera and target’s motion.
−36 −34 −32 −30 −28 −26 −24 −22 −20 −18 −1618
20
22
24
26
28
30
32
34Frame 657
3D Trajectory
Ground Plane Feature
Frame 657
114
121122123124
Projected Target
2D TrajectoryKLT Feature
Experimental Results• Improved detection accuracy.
0 0.5 1 1.5 2 2.5 30.6
0.65
0.7
0.75
0.8
0.85
Recall/FPPI
FPPI
Reca
ll
Full ModelNo GeometryNo InteractionBaseline [9]
• Example tracking results on ETH sequence.
12 34 51314 1516 171819 20
117 121
122
121
161 162
165
167
121162
166
−20 −15 −10 −5 0 5 10 150
5
10
15
20
25 Frame 6
−30 −25 −20 −15 −10 −5 0 50
5
10
15
20
25
30
35
40Frame 96
−50 −45 −40 −35 −30 −25 −20 −1510
15
20
25
30
35
40
45Frame 600
−80 −75 −70 −65 −60 −55 −50 −45 −4015
20
25
30
35
40
45
50Frame 995
−80 −75 −70 −65 −60 −55 −50 −45 −4015
20
25
30
35
40
45
50 Frame 1005
Frame 176
42
4344
45
Frame 1064
121
177178
179
• Accurate localization of camera
• Eect of interaction model.
4 25 31 333435
Frame 533
24
24
2728 30
Frame 479
424
28 30 3233
Frame 533
2 425
2829 3133
Frame 479
−15 −10 −5 0 5 10 150
5
10
15
20
25Frame 479
Group Interaction
−15 −10 −5 0 5 10 150
5
10
15
20
25Frame 533
Group Interaction
−15 −10 −5 0 5 10 150
5
10
15
20
25Frame 479
−15 −10 −5 0 5 10 150
5
10
15
20
25Frame 533
• Eect of geometric features.
1 2
34
Frame 89
1 2
34
Frame 223
1 2
34
Frame 89
1 2
68
Frame 223
−15 −10 −5 0 5 10 150
5
10
15
20
25 Frame 89
−15 −10 −5 0 5 10 150
5
10
15
20
25 Frame 223
−15 −10 −5 0 5 10 150
5
10
15
20
25Frame 89
−15 −10 −5 0 5 10 150
5
10
15
20
25Frame 223
Recall/FPPI on ETH dataset
ETH [1]Recall
FPPI
0.498 0.404 0.338
0.781 0.431 0.262
0.673 0.616 0.484
2.772 1.593 0.638
Our AlgorithmRecall
FPPI
0.556 0.541 0.519
0.792 0.442 0.267
0.339 0.421 0.497
2.792 1.608 0.647
Method Seq.#2 Seq.#3
Estimation : MCMC Particle Filtering• Motivation
- Non-linear and non-gaussian density function.
• Proposal Distribution Q(Ωt; Ωt)
- Choose one of the variables : camera parameter, target states, or ground features.- Sample from Gaussian distribution centered on current sample.
- Randomly change the class of a target or a ground feature.
- Direct sampling from high dimensional space is not ecient.Randomly Select one
Human? Non-human?How fast?Where? ...New random sample X’
Accept/Reject(Metropolis Hasting)
Joint Model
X2
τ1
X3
X1
τ3
τ2
ϴZ1
G2
Z3Z2
G3
G1
Xt-1 Xt
θt-1 θt
Zt-1 Zt
Xt+1
θt+1
Zt+1
τt-1 τt
Gt-1 Gt
τt+1
Gt+1
Graphical Representation World Illustration
• Sequential bayesian formulationP (Ωt|χt) ∝ P (Ωt, χt|χt−1) = P (χt|Ωt)
P (Ωt|Ωt−1)P (Ωt−1|χt−1)dΩt−1
P (χt|Ωt) = P (Xt, Yt|Zt,Θt)P (τt|Gt,Θt)P (Ωt|Ωt−1) = P (Zt|Zt−1)P (Θt|Θt−1)P (Gt|Gt−1)
• Observation models- Simplied camera projection function [4].
• Motion models- Target’s individual motion modeled as a rst order linear dynamic model.- Ground features assumed to be static.- Camera assumed to move along the viewing direction.
• Indicator variable- Target : indicate whether this target is valid human or not.
- Ground features : indicate whether the feature is on static ground plane.
Time t
Time t+1
Recommended