Real-time foreground object detection & tracking with moving camera
P93922005 Martin Chang
Motivation More and more moving cameras
Handheld devices Cell phone PDA
Hard to track object with moving camera
Hard to learn background with moving camera
Previous Work Thompson, W.B. and Pong, T.C. Detecting moving objects. Int
ernational Journal of Computer Vision, 4(1):39-57. (January 1990).
Stationary Camera K Daniilidis, C Krauss, M Hansen, G Sommer. Real-Time Track
ing of Moving Objects with an Active Camera. Real-Time Imaging, 1998.
Two degrees of freedom of a camera platform E Hayman, JO Eklundh - Procs. Statistical Background Subtra
ction for a Mobile Observer. IEEE Intl. Conf. on Computer Vision.
Moving foreground object static background Mobile observer
Steps
1. Find good feature to track2. Track features3. Classify foreground and
background features 4. Decide region of foreground
object5. Track foreground object
Video Demo
Step 1: Find good feature to track
Finding good feature to track Shi and Tomasi ‘s method
Step 2: Track features
Optical flow
Step 3: Classify foreground and background features
Classify feature points Optical flow
Moving direction of feature Length of moving direction
MTF of neighbor image patch Doesn’t work, due to
With cheap camera Low resolution video
Idea: how to identify foreground features? 1/3
Background Object
Case 1: The camera rotates The background image moves more
Idea: how to identify foreground features? 2/3
Case 2: The background moves The background image moves more
Background object
Idea: how to identify foreground features? 3/3
Background object
Case 3: The object moves The foreground image moves more
Classify Features
KMeans Hard to separate them well
Marginal KMeans Filter unreliable features
Angle issue 1° is similar to 359 °
KMeans
Marginal KMeans (Margin=1/2)
Marginal KMeans (Margin=1/4)
Step 4: Foreground Object Detection 1/2
Two two-class problems Classify foreground and background
features Cluster features
Calculate the occlusion rate The region of foreground object should be
Compact Less noise (background features)
Foreground Object Detection 2/2
Measure our confidence Geometry approach Check foreground and background
regions
Step 5: Foreground Object Tracking
1. Object detection2. If foreground object is never
detectedGo to Step 1
3. Object tracking4. Go to Step 1
Development Platform
Microsoft Visual C++ .NET 2003 Cheap webcam (USB 1.1) OpenCV
Future Work
Find parameters by machine learning
Detect finite candidate objects Cue: color moment
Multiple object detection(!)
Conclusion
The bottleneck is camera’s data transporting speed (USB 1.1)
Real time is possible OpenCV is useful
Demo
Thanks!