Upload
jeffery-quinn
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Counting Crowded Moving Objects
Vincent Rabaud and Serge BelongieDepartment of Computer Science and Engineering
University of California, San Diego{vrabaud,sjb}@cs.ucsd.edu
Presentation by: Yaron KoralIDC, Herzlia, ISRAEL
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
2
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
3
Motivation
• Counting crowds of people• Counting herds of animals• Counting migrating cells
• Everything goes as long as the crowdis homogeneous!!
4
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
5
Challenges
• The problem of occlusion– Inter-object– Self occlusion
• Large number of independent motions– Dozens of erratically moving objects– Require more than two successive
frames
6
Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
7
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
8
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
9
Harris Corner Detector – What are Good Features? C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
• We should easily recognize a corner by looking through a small window
• Shifting a window in any direction should give a large change in intensity
Harris Detector: Basic Idea
“flat” region:no change in all directions
“edge”:no change along
the edge direction
“corner”:significant
change in all directions
Harris Detector: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
Change of intensity for shift in [u,v] direction:
IntensityShifted intensity
Window function
orWindow function w(x,y)=
Gaussian1 in window, 0 outside
Harris Detector: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
yx vIuIyxIvyuxI ,,
v
uMvu
v
u
III
IIIyxwvu
v
uyxIyxIyxwvuE
yyx
yxx
yxyx
2
2
,
2
),(
,,,,
For small [u,v]:
We have:
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
For small shifts [u,v] we have a bilinear approximation:
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
where M is a 22 matrix computed from image derivatives:
Harris Detector: MathematicsDenotes by ei the ith eigen-vactor of
M whose eigen-value is i:
Conclusions:
0 iiTi M ee
max1, ,maxarg e vuEvu
maxmax eE
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
Intensity change in shifting window: eigenvalue analysis
1, 2 – eigenvalues of M
direction of the slowest change
direction of the fastest change
)max(-1/2
)min(-1/2
Ellipse E(u,v) = const
Harris Detector: Mathematics
1
2
“Corner”1 and 2 are large,
1 ~ 2;
E increases in all directions
1 and 2 are small;
E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points
using eigenvalues of M:
Sum of Squared Differences – Tracking FeaturesTracking Features
• SSD is optimal in the sense of ML when1. Constant brightness assumption2. i.i.d. additive Gaussian noise
Ayx
tyxtvyuxvu,
2,,1,,,SSD RI
Exhaustive Search
• Loop over all parameter space• No realistic in most cases
– Computationally expensive• E.g. to search 100X100 image in 1000X1000
image using only translation~1010 operations!
– Explodes with number of parameters– Precision limited to step size
The Problem
Find (u,v) that minimizes the SSD
over region A.
Assume that (u,v) are constant over all A
Ayx
yxvyuxvuSSD,
2,,, RI
Iterative Solution
• Lucas Kanade (1981)– Use Taylor expansion of I (the optical
flow equation)
– Find
Ayx
tyxvuvu
vuvuSSD,
2
,,min,min III
0 tyx vu III
Feature Tracking with KLT(We’re back to crowd counting…)• KLT is a feature tracking algorithm• Driving Principle:
– Determine the motion parameter oflocal window W from image I to consecutive image J
– The center of the window defines the tracked feature
23
Feature Tracking with KLT
• Given a window W– the affine motion parameters A and d
are chosen to minimize the dissimilarity
24
Feature Tracking with KLT
• It is assumed that only d matters between 2 frames. Therefore a variation of SSD is used
25
• A window is accepted as a candidate feature if in the center of the window, both eigenvalues exceed a predefined threshold t
min(min(λλ11,, λ λ22) > t) > t
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
33
Increased Efficiency #1• Associating only one window with
each feature– Giving a uniform weight function that
depends on 1/(window area |w|)– Determining quality by comparing:
• Computation of different Z matrices is accelerated by “integral image”[1]
34
|| w
z
[1] Viola & Jones 2004
Increased Efficiency #2
• Run on sample training frames first– Determine parameters that lead to the
optimal windows sizes– Reduces to less than 5% of the possible
parameter set– All objects are from the same class
35
Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
36
Feature Re-Spawning
• Along time, KLT looses track:– Inter-object occlusion– Self occlusion– Exit from picture– Appearance change due to perspective
and articulation
• KLT recreates features all the time– Computationally intensive– Weak features are renewed
37
Feature Re-Spawning
• Re-Spawn features only at specific locations in space and time
• Propagate them forward and backward in time– Find the biggest “holes”– Re-spawn features
in frame with theweighted average oftimes
38
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
39
Trajectory Conditioning
• KLT tracker gives a set of trajectories with poor homogeneity– Don’t begin and end at the same times– Occlusions can result in trajectory
fragmentation– Feature can lose its strength resulting in
less precise tracks
• Solution: condition the data– Spatially and temporally
40
Trajectory Conditioning
• Each trajectory is influenced by its spatial neighbors
• Apply a box to each raw trajectory• Follow all neighbor trajectories from
the time the trajectory started
41
Algorithm Highlights
• Feature Tracking with KLT• Increased Efficiency• Feature Re-Spawning• Trajectory Conditioning• Trajectory Clustering
42
Trajectory Clustering
• Determine number of object at time tt by clustering trajectories
• Since at time tt objects may be close, focus attention on a time interval (half-width of 200 frames)
• Build connectivity graph– At each time step, the present features form the
nodes of a connectivity graph G – Edges indicate possible membership to a common
object.
43
Trajectory Clustering• Connectivity Graph
– Bounding Box: as small as possible, able to contain every possible instance of the object
– If two features do not stay in a certain box, they do not belong to the same object.
– The 3 parameters of this box are learned from training data.
44
Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.
Articulation factor
Trajectory Clustering• Rigid parts merging
– Features share similar movement during whole life span, belong to a rigid part of an objectrigid part of an object, and consequently to a common objectcommon object
– RANSAC is applied to sets of trajectories • Within time window • Connected in graph G
45
Trajectory Clustering
• Agglomerative Clustering– At each iteration, the two closest sets are
considered – If all features are linked to each other in the
connectivity graph, they are merged together.
– Otherwise, the next closest sets are considered
– Proceed until all possible pairs are analyzed
46
AGENDA
• Motivation• Challenges• Algorithm• Experimental Results
47
Experimental results
• Datasets– USC: elevated view of a crowd
consisting of zero to twelve persons– LIBRARY: elevated view of a crowd of
twenty to fifty persons– CELLS: red blood cell dataset consisting
of fifty to hundred blood cells
48
Experimental results
49
Experimental results
50
Estimated
Ground Truth
Experimental results
51
Conclusion
• A new way for segmenting motions generated by multiple objects in crowd
• Enhancements to KLT tracker• Conditioning and Clustering
techniques
52
Thank You!Thank You!
53