[IEEE 2012 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR) - Xian, China (2012.07.15-2012.07.17)] 2012 International Conference on Wavelet Analysis and

Proceedings of the 2012 International Conference on Wavelet Analysis and Pattern Recognition, Xian, 15-17 July, 2012

CROWD MOVEMENT SEGMENTATION USING VELOCITY FIELDHISTOGRAM CURVE

WEI LI1, JIU-HONG RUAN1, HUA-AN ZHA02

1Department of Transportation and Logistics Engineering, Shandong Jiaotong University, Jinan 250023, China2Department of Computer Science and Electrical Engineering, Kumamoto University, Kumamoto 860-8555, Japan

E-MAIL: [email protected]@[email protected]

Abstract:This paper presents a fast, accurate and novel approach for

the problem of flow segmentation in dense and very dense movingcrowds. First, optical flow method is used to remove backgroundnoise of scenes. Second, the angle information of foreground velocity field is turned into gray level image and histogram curve is employed to find out extreme points. Finally, the updated minimumpoints can be utilized to segment foreground crowd into differentflows. Compared to other methods, the proposed approach is efficient and fast, the processing speed is 17 frames per second.

Keywords:Crowd analysis; crowd flow segmentation; optical flow; his

togram curve

1. Introduction

With the increase of crowd phenomenon and the development of computer vision, the research of crowd analysis (CA)is attracting more and more interests. The technique of CA isbeing widely employed in different fields, such as traffic monitoring, medical image processing, underwater animal detectionand so on [1]. However, when there are more than one flowin crowd, it is necessary to segment different flows before extracting feature vectors. As shown in Fig.1, the crowd movetoward two directions, in order to precisely analyze the crowd,we need to separate different crowd flows. This procedure isnamed crowd flow segmentation (CFS).

There exist two challenging problems in the research of CFS,the one is how to achieve real-time segmentation, the other ishow to exactly segment flows in crowd. So far, two kinds ofsegmentation methods have been proposed. S. Ali [2] proposeda particle dynamics approach for CFS, a set of continuous optical flow field of crowd video was employed to calculate the fi-

978·1-4673·1535·7/121$31.00 ©2012 IEEE

Figure 1: Example of crowd with multiple flows.

nite time Lyapunov exponent field (FTLE), in which the boundaries of different crowd flows could be detected by constructingthe Lagrangian coherent structures (LCS), then this approachused the normalized cuts algorithm to accomplish flow segmentation. This method is quite time-consuming because thecalculation of FTLE is very large and the processing of normalized cuts is complex. Besides, this method can only detectthe large flows in the foreground. Another method is based onfeature clustering. In [3], the authors extracted optical flow angle information as features and used K-mean clustering methodto segment optical flow field instead of crowd flows, the experimental results showed many "blocks" appeared after segmentation, so the authors proposed block absorbing method toremove these "blocks". In [4] the authors evaluated the performance of another two clustering methods, self-tuning spectralclustering and nonlinear dimension reduction Isomap, the literature concluded that, to deal with one image the execution timeis less than 5 seconds on a MATLAB implementation. So theclustering methods are also very time-consuming.

In this paper, we propose a new CFS approach based on optical flow angle histogram analysis. Compared to the above

191


methods, our approach has small computation and is easy tocarry out, since we only use the histogram curve to find outsegmentation points. The segmentation results are also quitegood, both large flows and small flows can be detected. Therest of this paper is organized as follows. Section 2 gives adetailed introduction of our approach and some experimentalresults will be shown in Section 3 to verify the performance ofour approach. Finally, the paper is concluded in Section 4.

2. The proposed approach

The processing of our approach contains three steps. First,the crowd foreground is extracted from the scene. Second,obtaining the histogram curve of optical flow angle of crowd.Third, getting the segmentation points from the curve and segmenting crowd flows. We will introduce each step in detail.

2.1. Background removal

Figure 2: Angle gray image and histogram curve.

8(x, y)fint(x, y) = [3600 x 255]

8(x, y)fdb(x, y) = 3600

(1)

(2)

Since we only care about the moving crowd in the scene, thebackground area should be removed at first. In our approach, areal-time optical flow method is employed to compute the velocity field [5], and then the background is removed by settingthreshold. The optical flow field of foreground is then used toobtain a histogram curve.

2.2. Obtaining histogram curve

In the research of image segmentation, gray level histogramanalysis is efficient method, the histogram curve shows the graydistribution of image. In our research, we considered to employthis theory in CFS.

Supposing the original image size is M x N, f f =(x, y, u,V)T is the optical flow vector of foreground, wherep = (x, y)T is image location coordinate and V = (u,v)T

is velocity vector. We denote R(x, y) as the magnitude and8(x, y) as the angle of a vector, where 0° ~ 8(x, y) < 360°.To getting histogram curve of optical flow, we should changethe angle information into gray level image. Eq(l) and (2) canbe used to obtain integer type and normalized gray images.Usually we denote pixel value 255 and 1 as foreground andoas background, but when using Eq(l) and (2), the foregroundpixel with 8(x, y) = 0° is also treated as background, whichleads to foreground information lost. To solve this problem,we propose Eq(3), which changes foreground and backgroundpixels respectively, so in the angle gray image g(x, y), pixelvalue of 255 means background, pixel value of [0, 250) meansforeground.

(x ) = {[e3~O~) x 250], if p(x,y) is in foreground9 ,y 255, if p(x,y) is in background

(3)Figure2(a) shows the angle gray image of Fig.l, which is

calculated through Eq(3), Fig.2(b) shows the histogram curveof (a), in which the vertical line at right represents the background area, the "w"-like curve shows the angle distribution ofthe foreground velocity field. Noted that in angle gray image,pixel value 0 and 250 mean the same angle in velocity field,therefore, from Fig.2(b) we can see the crowd in Fig.l mainlymove toward two directions, 0° and 180°. But this is just theobserved result, we propose an auto-detection approach to findout segmentation points in histogram curve.

2.3. Segmenting crowd flows

Apparently, in Fig.2(b) the wave peaks mean the two moving directions of crowd and the wave valleys could be selectedas segmentation points. However, the curve is quite rough, itis difficult to detect exact wave valleys. Therefore, we utilize Gaussian smooth method as histogram preprocessing, thesmooth model is shown in Eq(4),

gs(w) = gs(w - 2) *gs(w - 2), w E {5,7,9, ...} (4)

with the original setting gs(3) = [0.2261 0.5478 0.2261]. Withthe increasing of w, the length of smooth model lengs(w)changes as Eq(5):

192


Step 1 Removing maximum point 255, which represents background.

Step2 Iflmaxi - maxjl ~ 31,oneofmaXi,maXj should beremoved. If N(maxi) 2:: N(maxj), maxj is removed;otherwise, max; is removed. N(maxi) means the pixelnumber of gray value maxi.

If selecting large parameter, like w = 11, the length ofsmooth model will be 33, so that the first 16 points and thelast 16 points on the curve can not be smoothed. So w shouldnot be large. However, it can not be too small, otherwise thecurve will be still rough. In our test, w = 7 is a proper setting.

Next, we calculate the differential curve of smoothen histogram and detect zero crossing points to find out maximumand minimum values. Figure3(a) shows the smoothen curve ofFig.2(b) and Fig.3(b) is the smoothen differential curve of (a).By detecting zero crossing points on Fig.3(b), we can obtain themaximum group and the minimum group, which are {max} ={3, 5,8,40, 57,69, 124,162,177, 190,204,220,240,246, 255} and {min} = {4, 6, 38, 51, 66, 70, 161, 172,187, 195,209,221, 242}

As can be seen, in the two groups some of elements have similar values, such as 3 and 5. If changing into angle value theybecome 4° and 7°, which represent very close directions. Forcrowd movement, two flows with 4° and 7° moving directionsshould be treated as the same flow. Therefore, some extremepoints having similar value as other point should be removed.We assume that, in velocity field if the angle difference of twopixels is smaller than 45°, they are treated as in the same flow.Consequently, we set a threshold T = 31 to remove similarextreme points, the detail approach is described as follow:

Figure 3: Smoothen histogram curve and smoothendifferential curve.

w-3-2-

lengs(w) = 3 + E 2i.

i=l

(5)

Figure 4: Foreground mask and CFS result.

Step 3 If max; < 31, maxj > 219 andImaxi - maxj + 2501 ~ 31, one of maau, maxjshould be removed. The following processing is same asStep 2. After the above disposing {max} is updated as{max'}.

Step 4 Between two maximums of {max/}, finding out properminimum point from {min}. If there are more than oneminimum between two maximums, keep the one with thesmallest number of pixel and removing all the others. Finally, we can obtain a new group of minimum points{min/}, which can be used to segment histogram curve.

After that, the above {max} and {min} tum to {max/} ={124, 246} and {min/} = {70, 187}. The consequent segmentation result is shown in Fig.4, Fig.4(a) shows the foregroundarea and Fig.4(b) is the segmentation image. As can be seen,crowd foreground is segmented into two flows which are represented in blue and white. The CFS result is in accordance withthe observed result, that verifies the efficiency of our approach.

3. Experimental results

Our experiments contain two parts, in the first part, we givetwo groups of image result obtained by our approach; in thesecond part, we compare our approach with three other CFSmethods mentioned above.

Figure5 and 6 show the CFS results of marathon game videoand convention video, each group contains four images, theoriginal image, the angle gray image, the smoothen histogramcurve and the segmentation image. In Fig.5 the marathon players run in "U" shape and the crowd can be segmented intothree flows, which is in accordance with segmentation result. InFig.6, people in convention move around the center. Althoughthe histogram curve is quite flat, our approach still found outthree segmentation points, so that the foreground crowd aredivided into three flows. These experiments showed that our

193


Figure 5: Our CFS result of marathon game image.

Figure 7: Comparison of four CFS methods .

4. Conclusion

Figure 6: Our CFS result of convention image.

This paper researches on CFS and proposes a new approach.This approach makes use of optical flow field to extract crowdforeground at first, then turns the angle information of velocityfield into gray level image and automatically detects segmentation points on histogram curve of angle gray image. Someexperimental results showed that, compared to other CFS methods, this approach has the best performance.

However, when determining the segmentation points fromextreme points, the setting threshold is not proper for allvideos, sometimes a few of blocks appear after segmentation.As future work, it is necessary to propose better method tosolve this problem.

approach could deal with different types of crowd, includingeasily segmented crowd(Fig.l) and crowd with flat histogram Acknowledgementscurve(Fig.5 and 6).

Figure7 shows CFS results of fluid dynamics method(Fig.7(a)), spectral clustering method (Fig.7(b)), k-mean clustering method (Fig.7(c)) and our approach (Fig.7(d)). As canbe seen that fluid dynamics method can not detect small flow incrowd, spectral clustering method can not detect exact boundary between flows and k-mean clustering method producesmany small blocks after segmentation. The performance of ourapproach is the best in these four methods. Besides, the processing of our approach is the fastest, the speed is 17 framesper second. Therefore, these experiments proved that the proposed CFS approach is better than the existing methods.

This work was supported by National Natural ScienceFoundation of China (No. 61174054) and Colleges Scienceand Technology Planning Project of Shandong Province (No.JI0LG51).

194


References

[1] J. Junior, S.R. Musse and C.R. Jung, "Crowd AnalysisUsing Computer Vision Techniques", Signal ProcessingMagazine, pp. 66-77, 2010.

[2] S. Ali and M. Shah, "A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis", IEEE Conference on Computer Vision and PatternRecognition , pp. 1-6, 2007.

[3] W. Li, X. Wu and H. Zhao, "New Techniques of Foreground Detection, Segmentation and Density Estimationfor Crowded Objects Motion Analysis", Transactions ofInformation Processing Society of Japan, 52(4), pp.1820-1830,2011.

[4] G. Eibl and N. Brandle, "Evaluation of ClusteringMethods for Finding Dominant Optical Flow Fields inCrowded Scenes", The 19th International Confence onPattern Recognition, pp. 1-4, 2008.

[5] C. Zach, T. Pock and H. Bischof, "A Duality Based Approach for Realtime TV-Ll Optical Flow". In Proc. 29thDAGM Symposium on Pattern Recognition, 2007.

195

Documents

[IEEE 2012 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR) - Xian, China (2012.07.15-2012.07.17)] 2012 International Conference on Wavelet Analysis and