Fahad Fazal Elahi Guraya

People Tracking via a Modified CAMSHIFT People Tracking via a Modified CAMSHIFT Algorithm

(DCABES 2009)

Fahad Fazal Elahi Guraya, Pierre-Yves Bayleand Faouzi Alaya Cheikh

Department of Computer Science and Media Technology, Gjovik University College Gjovik , Norway

Email: [email protected]

OutlineIntroductionIntroduction

Problem Statement

T ki Al ithTracking Algorithms

Mean-shift tracking

CAMSHIFT tracking

Extended CAMSHIFT tracking

Implementation

Experimental Resultsp

Conclusion

2Gjøvik University College [email protected]

Introduction

3Multi-Camera Cooperated Object Detection, Tracking, and Event Analysis

Gjøvik University College [email protected]

I t d ti (A li ti )Introduction (Applications)Motion-based recognition: g

Human identification based on gait, automatic object detection, etc.

Automated Video surveillance: Monitoring a scene to detect suspicious activities or unlikely events

Video indexing:Automatic annotation and retrieval of the videos in multimedia databasesAutomatic annotation and retrieval of the videos in multimedia databases

Human-computer interaction:Gesture recognition, eye gaze tracking for data input to computers, etc.

Traffic monitoring: Real-time gathering of traffic statistics to direct traffic flow

Vehicle navigation:Vehicle navigation:Video-based path planning and obstacle avoidance capabilities

4 Gjøvik University College [email protected]

P bl St t tProblem Statement

Frame 4Frame 1 Frame 2 Frame 3

Can we estimate the position of the object?Can we estimate the position of the object?

Can we estimate its velocity?

Can e predict future positions?Can we predict future positions?


Problem StatementProblem StatementGiven a Sequence of Images/framesGiven a Sequence of Images/framesFind center of moving objectCamera might be moving or stationaryCamera might be moving or stationary

We Assume:We Assume:We can find object in individual frames

The Problem:The Problem:Track across multiple frames

A fundamental problem in the field of video analysis


Tracking AlgorithmsTracking Algorithms

a) Point Tracking by Bayesian Filters (Differ in representing probability densities (pdf))Kalman FiltersParticle FiltersParticle FiltersGrid-based approachMulti-hypothesis(MHT) filter

b) KernelTrackingMean-shift trackingMean-shift trackingContinuously Adaptive Mean-shift (CAMSHIFT)Modified CAMSHIFT with MotionKanade-Lucas-Tomasi Feature Tracker (KLT)Support Vector Machines (SVM) TrackerSupport Vector Machines (SVM) TrackerEigen Tracking

c,d) SilhouetteTrackingContour evolution state space modelsContour evolution by variational methods

7

Contour evolution by variational methodsShape Matching hough transform


Mean shift TrackingMean shift Tracking

• Introduced by Y. Cheng. in “Mean Shift, Mode Seeking, and Clustering” PAMI 1995

• Mean shift algorithm climbs the gradient of a probability distribution to find the nearest domain mode (peak)

@R C lli CVPR 2003 @C i i PAMI 2003

8

@R. Collins CVPR 2003 @Comaniciu PAMI 2003


M hift T kiMean shift Tracking

Given a likelihood image, find the optimal location of the tracked object

The likelihood image is generated by computing, at each pixel, the probability that the pixel belongs to the object based on the di ib i f h fdistribution of the feature

Obtain mean-shift vector y by maximizing the Bhattacharyya ffi i hi h i i l i i i i h dicoefficient, which is equivalent to minimizing the distance

maximize:maximize:

where Bhattacharyya

9

Bhattacharyya coefficient for a single bin u

Continuously Adaptive Mean-SHIFT (CAMSHIFT) Tracking

Modified form of Mean-shift tracking algorithm

Introduced by GR Bradski. in “Computer vision face tracking for use in a perceptual user interface”. Intel Technology Jounal 1998

Differs from Mean-shift: Search window adjusts itself in size Differs from Mean-shift: Search window adjusts itself in size.

If we have well-segmented distributions(face) then CAMSHIFT will automatically adjust itself for the size of face as the person moves closer or further from camera.

search window

size

Re-centre the search

window, l it

Both search window size and centre are

Current search window size is the object size; it t i th

Target

centre*M N

( , )c cx y

rescale its size

stable its centre is the centre of object

Initialization

Target

EndCondition

YesControl ResolutionFoundCondition

Satisfied?

No10

CAMSHIFT TrackingCAMSHIFT Tracking1. Choose the initial location of the 2D mean shift search window

2. Calculate the color probability distribution in the 2D region centered at the search window location in an ROI slightly larger than mean shift window size

3 Run Mean Shift algorithm to find the search window center Store the zeroth3. Run Mean Shift algorithm to find the search window center. Store the zerothmoment(M00) area or size and centriod location

4. For the next video frame, center the search window at the mean location stored in Step-3 and set the window size to a function of the zeroth moment M00 found there. Go to Step-2.

11 [email protected]

Extended CAMSHIFT trackingExtended CAMSHIFT tracking

Optical Flow computation by Lucas-Kanade MethodOptical Flow computation by Lucas-Kanade Method

Brightness Constancy, Temporal Persistance, Spatial CoherenceCoherence

Vid F M ti Di ti

12

Video Frame Motion Direction


Extended CAMSHIFT trackingExtended CAMSHIFT trackingPurposed Method: Compute motion information of the moving objects and add it linearly to the back-projected g j y p jcolor histogram

Steps:pSelect initial location of the person i.e ROI (Region of interest)Compute equalized histogram of the ROI p q gCompute back projection image using the current histogram for the next frameCompute the motion of the blob using Lucas-Kanade algorithmUpdate the back-projection image using motion information

Combine direction of motion with back projection image linearly, givemore weightage to pixels moving in same direction as in the previousframe and less weightage to pixels moving in the other directions

13

g g p g

Use updated back-projection image to track object/s in the new frame

Extended CAMSHIFT TrackingExtended CAMSHIFT TrackingPros:Pros:

Computationally efficient (robust statistics and probability distributions) -Working in real-time: fast processing) g : ast p gRobust to image noise Robust to distractors (e.g. other objects)( g j )Irregular object motion (linear/non-linear)Robust to partial/full occlusionpRobust to background-foreground color

Cons:Need manual input to initialize template window


ImplementationpOpenCV Computer Vision Library/Visual C++


I l t tiImplementation

VideoFrames Background

ModelingBackground Modeling

Post Processing (Shadow Removal)

Post Processing (Shadow Removal)

Frames

Object/People Object/People Object/People Object/People

ObjectTracks

Object/People Detection

Object/People Detection

Object/People Tracking

Object/People Tracking


Implementation(Background Modeling)Implementation(Background Modeling)

Recursive techniquesqRunning Gaussian average (RGA)Gaussian mixture model (GMM)( )GMM with adaptive number of Gaussians(AGMM)Approximated median filtering (AMF)

Non-recursive techniquesMedian filteringMediod filteringEigenbackgrounds (EigBG)

[email protected]øvik University College Norway

Implementation (GMM)Implementation (GMM)

Data is represented by a mixture of N GaussiansData is represented by a mixture of N Gaussians

Different Gaussians represent different colors


Implementation(Shadows Removal)Implementation(Shadows Removal)

T diti l C t

We want to realize this!Traditional Contour


Implementation(Shadows Removal)Implementation(Shadows Removal)

Original background (B): Brightness and chromaticity similar to those of the same pixel in the background image.

Shaded background (S): Similar chromaticity but lower brightness.

Moving foreground object (F): Chromaticity different from g g j ( ) ythe expected values in the background image.

Gjøvik University College Norway20 [email protected]

E i t l R ltExperimental ResultsShadow Removal Morphological filteringShadow Removal Morphological filtering

Closing operationStructuring element: 3x3 Structuring element: 3x3

Removing small regions

Before AfterBackground subtraction Shadows removal

21

E i t l R ltExperimental ResultsObject/People DetectionObject/People Detection

Contours detectionBounding box Bounding box

Contours and boundingboxes for 2 separate persons

Contours and bounding boxes for 2persons starting to overlap


Experimental ResultspBack projection images (with/without) motion information

Backprojection person.1 using color feature Backprojection person.2 using color feature

23 Backprojection person.1 using color and motion features

Backprojection person.2 using color and motion features

Experimental ResultsExperimental ResultsTracking Results using CamShift + Optical Flow

Tracking windows aroundmoving personsmoving persons

Tracking Without Motion Information Tracking Using Motion Information

ConclusionConclusion

A modified CAMSHIFT algorithm is presentedA modified CAMSHIFT algorithm is presented

The Algorithm use color and motion features

Th Al ith i t t d d ifi d f t f idThe Algorithm is tested and verified for a set of videos

As future work the algorithm should be tested/verified for i d / d id i h h d d i l/f ll indoor/outdoor videos with strong shadows and partial/full occlusion of objects

Th l i h h ld l b d d difi d f The algorithm should also be tested and modified for multiple objects and multiple camera tracking
