Using Adaptive Tracking To Classify And Monitor Activities In A Site

W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee

Objectives to calibrate the distributed sensors, to construct rough site models, to classify detected objects, to learn common patterns of activity for

different object classes, and to detect unusual activities

Motivation The system should provide statistical descriptions of typical

activity patterns, e.g., normal vehicular volume or normal pedestrian traffic paths for a given time of day

it should detect unusual events, by spotting activities that are very different from normal patterns, e.g., unusual volumes of traffic, or a specific movement very different from normal observation

it should detect unusual interactions between objects, e.g., a person parking a car in front of a building, exiting the car, but not entering the building

Hypothesis motion tracking is sufficient to support a

range of computations about site activities

Hardware The system observes activities with a “forest

of sensors” distributed around the site. Each sensor unit is a compact packaging of camera, on-board computational power, local memory, communication capability and possibly locational instrumentation (e.g., GPS)

Robust Adaptive Tracker

Robust Adaptive Tracker contd.. Color encodes direction and intensity encodes speed

Using The Tracker To CalibrateWhen using multiple cameras it is essential to coordinate the individual video streams and find a common coordinate frame.Procedure -

Each camera independently tracks an object. Correspondence is established between points in the

video frames from different cameras. For each camera pair, small but sufficient set of points is

sampled from a larger set of point correspondences and a least square solution to the homographies is found.

This solution is compared with the current solution by calculating the mean square error on all the point correspondences and the best fit is retained.

Using The Tracker To Calibrate contd...

Using The Tracker For Site Modeling Objective is to determine the pose of the ground plane

relative to a camera. Estimate the height of the object. By using the estimate of the ground plane and the height

the distance to the object can be estimated. The portion between the un-occluded object and the

camera can be carved out as free space. When the object is occluded, this places a lower bound

on the occluding portion in the site. Since the ground plane is known these obstacles can be placed in the world coordinates of the site.

Using The Tracker For Site Modeling contd..

Using The Tracker to Classify Classify objects – size and the aspect

ratio of the tracked objects can be used to classify and label the objects in the site

Classify actions – tracks of the moving objects can be used to classify activities by clustering the tracks on the basis of common attributes

Using The Tracker to Classify contd..

Clustering Algorithm #1-

1) Numeric Iterative Hierarchical Clustering Algorithm

2) MDL cut

Using The Tracker to Classify contd..

Using The Tracker to Classify contd..