Object Tracking with ProbabilisticHausdorff Distance Matching

Sang-Cheol Park and Seong-Whan Lee�

Department of Computer Science and Engineering, Korea University,Anam-dong, Seongbuk-ku, Seoul 136-713, Korea

{scpark, swlee}@image.korea.ac.kr

Abstract. This paper proposes a new method of extracting and track-ing a nonrigid object moving while allowing camera movement. For ob-ject extraction we first detect an object using watershed segmentationtechnique and then extract its contour points by approximating theboundary using the idea of feature point weighting. For object track-ing we take the contour to estimate its motion in the next frame by themaximum likelihood method. The position of the object is estimated us-ing a probabilistic Hausdorff measurement while the shape variation ismodelled using a modified active contour model. The proposed method ishighly tolerant to occlusion. Because the tracking result is stable unlessan object is fully occluded during tracking, the proposed method can beapplied to various applications.

1 Introduction

Thanks to the recent surge of interest in computer based video surveillance,video-based multimedia service and interactive broadcasting, the problem oftracking objects in video has attracted a lot of researchers around the world.Object tracking is so basic and in high demand that it is an indispensable compo-nent in many applications including robot vision, video surveillance, object basedcompression, etc. The problem of object tracking can be divided into two sub-problems, extraction of a target object and tracking it over time. In general eventhose subtasks are not easy because of cluttered backgrounds and frequent occlu-sions. However, we can readily find a large number of studies on diverse methodsusing a variety of features such as color, edge, optical flow, and contour [1].

A target object under tracking involves motion, which we classify into threetypes; in the increasing difficulty of analysis, they are (1) object motion with astatic camera, (2) camera motion over a static scene/object, and (3) simultaneousmovement of camera and objects. To speak in terms of applications, the first typeof motion has been heavily studied by the students in video surveillance, whilethe second type has been a prime target of analysis in video compression. Thelast type of hybrid motion, though more involved, has been an important topicin object based services, object based video compression, and interactive video.� To whom all correspondence should be addressed.

D.S. Huang, X.-P. Zhang, G.-B. Huang (Eds.): ICIC 2005, Part I, LNCS 3644, pp. 233–242, 2005.c© Springer-Verlag Berlin Heidelberg 2005

234 S.-C. Park and S.-W. Lee

This paper discusses an efficient method of extracting the contours of andtracking an interesting object in video from a non-static camera. First, objectextraction steps estimate the rough contour of an object, and then locate anddelineate the exact contour. Since the initial contour estimation starts from agiven seed or a rough estimate, it is desired to make the initial error as smallas possible. For this, we have developed a modified watershed algorithm basedon Gaussian weighted edges. The resulting contour is then used to estimate themotion vectors and locate the objects in the succeeding frames. In the objecttracking step, there are two subtasks, global motion estimation using the Haus-dorff matching method and local deformation analysis using the modified activecontour model.

The overall procedure is illustrated in Figure 1.

Fig. 1. The block diagram of the proposed method

2 Proposed Contour Extraction and Motion Tracking

2.1 Object Extraction with Watershed Algorithm

The watershed transform is a well-known method of choice for image segmen-tation in the field of mathematical morphology. The watershed transform canbe classified as a region based segmentation approach. To date a number of al-gorithms have been developed to compute watershed transforms. They can bedivided into two classes, one based on the specification of a recursive algorithmby Vincent and Soille[2], and the other based on distance functions by Meyer[4].The details of the watershed algorithm of this paper are based on the worksdeveloped by Nguyen[3], and Vincent and Soille[2].

Object Tracking with Probabilistic Hausdorff Distance Matching 235

Fig. 2. Watershed segmentation process steps, (a) an input image and selected featurepoints (b) an edge image (c) initial small regions (d) merged region (e) segmentedresult (f) extracted object

Initial Watershed Segmentation. An important idea of the proposed methodhas a Gaussian weight to the initial feature points of an interesting object. Thishelps removing the need for selecting all detailed feature points of an object.Let X = (X1, ..., Xn) be an ordered sequence of feature points Xi = [xi, yi]t,i = 1, ..., n. And let us denote the magnitude of the displacement from the originto X be m(X), and the Gaussian weight for the feature point X as G(X).Then the weighted magnitude of the displacement vector to X for the watershedsegmentation is given by

Mnew(Xi) = G(Xi) ∗ M(Xi)

G(X) = 1√2πσ

e−x2+y2

2σ2 (1)

where ∗ denotes 2D convolution operation. The watershed algorithm proceedsaccording to Equation 1 with a sequence of weighted feature points. The water-shed segmentation algorithm is a region based technique. The result is a set ofimage regions dividing the input image.

Figure 2 illustrates the overall object extraction process, in which a numerousfragmental regions are produced and then merged into a single complete contour.In the region merging step, we simply merge the regions one by one until we’releft with only two regions, foreground and background.

2.2 Contour Tracking of the Interesting Object

An object in an image is defined by a contour. A contour is sufficient for ourtask. The proposed tracking method takes the contour generated in precedingstep to analyze the object contour’s motion. The motion estimate is based onprobabilistic Hausdorff distance measurement between the previous contour andall possible candidate regions.

236 S.-C. Park and S.-W. Lee

Fig. 3. Multi-level canny edge, (a) an input frame (b) the 1st level edge (c) the 2ndlevel edge (d) the 3rd level edge

Global Motion Tracking with Hausdorff Distance. For the candidatefeature set in the next frame, we use the feature map which is calculated bymulti-level canny edge detector.

These level edge maps are used to calculate the Hausdorff distance, then eachlevel has the different weight. If we use three level edge maps, the real distancecan be calculated by the following Equation 2

HD = HL1(M, R) ∗ ω1 + HL2(M, R) ∗ ω2 + HL3(M, R) ∗ ω3 (2)

where ωi is the weight coefficient and HL1, HL2 and HL3, are the Hausdorffdistance calculated from the first, second, and third level edge map respectively.

The Hausdorff matching score is the Hausdorff distance between two finitefeature point sets[6]. The Hausdorff distance is used to measure the degree ofsimilarity between the extracted contour and a search region in video frames.Let us assume that we are given by a sequence of reference feature points, M =(m1, m2, ..., mk) computed in the preceding frame, and let a new sequence ofimage feature points, R = (r1, r2, ..., rn) from the current frame. The Hausdorffdistance between M and R can be written by

H(M, R) = max(h(M, R), h(R, M)) (3)

where

h(M, R) = maxm∈M

minr∈R

‖m − r‖ (4)

where ‖‖ is the distance between two points, e.g., Euclidean norm. This distanceis sensitive to noise and occlusion, we extend the function h(M, R) so that themodified Hausdorff can measure the partial distance between a reference featureand an image feature. We use the likelihood function to find the position of theobject.

First, to estimate the position of the model using the maximum likelihood,we define the set of distance measurements between reference feature pointsand input image feature points. Let d(m,R) be the partial distance, we usesome measurements of the partial distance. We consider the following distancemeasurements:

D1(M, R) = minm∈M

d(m, R) (5)

D2(M, R) =75 Kthm∈Md(m, R) (6)

Object Tracking with Probabilistic Hausdorff Distance Matching 237

D3(M, R) =90 Kthm∈Md(m, R) (7)

where the partial distance is defined as d(m,R) = minr∈R ‖m − r‖ and xK thm∈M

represents the K th ranked distance that K/mk = x%[6].

D4(M, R) = minm∈M

∑

W

[I(m) − I(R)]2 (8)

D5(M, R) = minm∈M

∑

W

|I(m) − I(R)| (9)

There are several distance measurements to estimate the partial distance,e.g., Euclidean distance, city-block distance, the distance transform, the sum ofsquared difference, optical-flow feature tracking method which can be used forthe distance measurement defined as Equation 8.

Probabilistic Hausdorff Measurement. Let s be the position of an object inthe current frame, then distance measurements are denoted D1 (s),D2 (s),...,Dl (s).We use these distances to match maximum likelihood. In order to estimatethe contour motion problem, we use the Olson’s maximum-likelihood match-ing method[5]. Then the joint probability of the distance measurements Di(s)can be written as

p(D1(s), ..., Dm(s)|s) =l∏

i=1

p(Di(s)) (10)

where p(Di(s)) is the probability density function of normalized distance ofDi(s) at the position s , that is, p(Di(s)) = (1 − Di(s)), p must be maximizedby Equation 10. The likelihood for p is formulated as the product of the priorprobability of the position s and the probability in Equation 11:

L(s) = p(s)m∏

i=1

p(Di(s)). (11)

For convenience, we will take the logarithm of Equation 11 as follows Equa-tion 12.

ln L(t) = ln p(t) +m∑

i=1

ln p(Di(t)). (12)

We search s that maximizes this likelihood function, then this result uses toestimate the global motion for an interesting object.

Local Deformation with Active Contour Motion Model. The conven-tional active contour model consists of 2 terms, internal and external energyterms. The discrete energy function in the active contour model is defined asfollows:

E(t) =∑

t

[Eint(t) + Eext(t)] (13)

238 S.-C. Park and S.-W. Lee

In this paper we propose a modified active contour motion model that addsan external motion term. The proposed energy function contains four terms:continuity, curvature, image force, and motion estimation confidence as follows:

E(t) =∑

i

[αEcont(vi(t)) +βEcurv(vi(t)) + γEimg(vi(t)) + ηEmatch(vi(t))] (14)

where contour energy functions, continuity energy, Econt(v(t)), curvature energy,Ecurv (v(t)), image force (edge/gradient), Eimg(v(t)) and motion estimation con-fidence, Ematch(v(t)). This last term in the above equation measures the motionvariation between feature points in current frame and those in the previousframe. It can be defined as

Emacth(t) =n∑

i

|v̄(t) − vi(t)|2 (15)

where v̄ is the average motion vector at time t, vi is the motion vector of the ithfeature point in frame t.

The position of the contour must be predicted for the efficient tracking. Topredict the position of the contour in the next frame, we calculate the followingprediction energy, the predicted position of the contour at time t + 1

Epre(t+1) = Ereal(t)+∑

vi∈V

[ε·(Evi

real(t)−Evi

real(t−1))+(1−ε)·(Evi

real(t)−Epre(t))]

(16)

where Epre(t+1) is the predicted position of the contour at time t + 1 andEreal(t) is the real energy term of the moved contour at time t and ε is the match-ing rate between Ereal(t) and Epre(t). We proposed the probabilistic method toextract and to track an object using watershed and Hausdorff matching.

3 Experimental Results and Analysis

We have proposed the object tracking method using the modified Hausdorff dis-tance matching and active contour algorithm. It has been tested over a set ofVHS video clips recorded. The background includes streets, cars, indoor labo-ratory scenes. All the deformable targets are the human movements freely. Thetest images have been in monochrome MPEG-1 format with the dimension of320 x 240. For comparison of the performance, we will refer to the method ofKass’s active contour[7].

3.1 Simple Tracking

Figure 4 shows the three frames of a video sequence used in the work of Astrom[8].We took the clip to compare the tracking performance with their method. Thebackground of the video is relatively simple. The target objects include a cup,

Object Tracking with Probabilistic Hausdorff Distance Matching 239

Fig. 4. Astrom and Kahl’s test images

a pimento, a banana, an archetype and several polygonal objects. They are sta-tionary, but the camera moves, apparently panning, tilting and zooming.

For the tracking performance tr(t) at time t , we calculated the ratio of thenumber of pixels of the real area npr (t) to that of the segmented area npe(t).Also, for occlusion rate, occ(t), at time, t , we used the pixel difference betweenpixels of the real area npr (t) and that of the occluded area npo(t).

tr(t) =npe(t)npr(t)

(17)

occ(t) =npo(t)npr(t)

(18)

Figure 5 shows the value of tr(t) over the sequence of the video clip. Thetwo straight lines downward show the performance degradation averaged overthe entire sequences.

Fig. 5. The comparison to the tracking performance

3.2 Long Sequence Tracking Performance

Figure 6 shows selected samples from a video of 180 frames (numbered from946 to 1126) where the human upper body is the target of tracking. The ob-ject boundary in the first frame was manually given by a user. It consists of a

240 S.-C. Park and S.-W. Lee

sequence of feature points appropriately defining the boundary. Note that theboundaries of subsequent tracking frames are highly accurate. The performanceof tracking over the long sequences is summarized in Table 1 and Figure 6. Theproposed method records 5 percent over the standard snake and template mod-els. Moreover the accuracy degrades much slowly compared with those of snakeand template.

Fig. 6. Human body tracking in a parking lot

Table 1. Experiment over a long sequence video

average number of mis-tracked pixels tracking rateSnake model 255 88%

Adaptive template 287 86%Proposed method 174 93%

Fig. 7. Tracking rate and mis-tracked pixels for each frame

Fig. 8. The experimental result of a person tracking in a room

Object Tracking with Probabilistic Hausdorff Distance Matching 241

The object boundary was manually selected by a user in the first frame,several feature points of a object were simply selected by a user. Furthermore,a human body sequence captured in our lab was used to check the performanceexperiment in the different environment.

Figure 8 shows a successful result under the pose variation of the man. Inorder to compare the experimental results quantitatively, we measured the suc-cessful tracking rate and the mistaken pixels of tracking process from each framein Figure 7.

4 Conclusions and Future Works

This paper proposed an effective contour tracking method for a deformable ob-ject which was described by a set of feature points. The proposed method workssuccessfully in spite of non-static camera and nonlinear deformation over timewith cluttered backgrounds. The extracting result has shown that the proposedmethod separates well target foreground objects from the background by ana-lyzing the background texture.

Through a series of experimental results, we could confirm that the proposedmethod was quite stable and produced good results under object shape deforma-tion and non-static camera motion. One problem with the implementation of theproposed system is that it sometimes fails if an object moves too fast. Anotherproblem is that processing time is so slow, so our proposed method is not suitedto realtime application. We believe that these problems can easily be overcome.The real issue of the paper is the performance and robustness of extracting andtracking deformable objects.

Acknowledgments

This research was supported by the Intelligent Robotics Development Program,one of the 21st Century Frontier R&D Programs funded by the Ministry ofCommerce, Industry and Energy of Korea.

References

1. Leymarie, F., Levine, M.D.: Tracking Deformable Objects in the Plane Using anActive Contour Model. IEEE Trans. on Pattern Analysis and Machine Intelligence,Vol. 15, No. 6 (1996) 617–634

2. Vincent, L., Soille, P.: Watershed indigital spaces: an efficient algorithm based onimmersion simulations. IEEE Trans. on Pattern Analysis and Machine Intelligence,Vol. 13 (1991) 583–598

3. Nguyen, H.T., Worring, M., Boongaard, R.: Watersnakes : Energy-Driven WatershedSegmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 25,No. 3 (2003) 330–342

4. Meyer, F.: Topographic distance and watershed lines. Signal Processing, Vol. 38(1994) 113–125

242 S.-C. Park and S.-W. Lee

5. Olson, C.F.: Maximum-Likelihood Image Matching. IEEE Trans. on Pattern Analy-sis and Machine Intelligence, Vol. 24, No. 6 (2002) 853–857

6. Huttenlocher, D.P., Klanderman, G.A., Rucklidge, W.J.: Computing Images Usingthe Hausdorff Distance. IEEE Trans. on Pattern Analysis and Machine Intelligence,Vol. 15, No. 9 (1993) 850–863

7. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active Contour Models. In Proc.of International Conference on Computer Vision, London, England, Vol. 1 (1987)259–268

8. Astrom, K., Kahl, F.: Motion Estimation in Image Sequences Using the Deformationof Apparent Contours. IEEE Trans. on Pattern Analysis and Machine Intelligence,Vol. 21, No. 2 (1999) 114–127