١٥٦٧

Detection of Abnormal Behaviors Based on Spatial Analysis for Intelligent Video Surveillance systems

Fatemeh Ziaeetabar and Nasrollah Moghadam Charkari Computer Engineering Department, Tarbiat Modares university, Iran, Tehran

Fatemeh.ziaeetabar@modares.ac.ir charkari@modares.ac.ir

Abstract—Video surveillance technology is one of the most important topics in the security technology domain. However, the existing video surveillance systems relay on human observers and have some limitations. Therefore, it is necessary these systems equipped to an internal intelligence and detect abnormality automatically. In this paper, we have proposed a novel method for recognizing abnormal behaviors based on the spatial analysis. We have studied both misuse detection and anomaly detection approaches and compared their results. Misuse detection is related to definition of suspicious area in the dataset scene while anomaly detection clusters the scene to some regions by considering the abundance of the people passing. In addition, we have defined a new parameter which decision about level of abnormality is determined according it. It can update automatically from new input behaviors. Experimental results which have been done on the CAVIAR dataset indicate our method has an accuracy of 92% and it is almost compatible with the hand labeled results.

Keywords: spatial analysis; misuse detection; anomaly detection; abnormal behavior; video surveillance

I. INTRODUCTION Video surveillance systems are used in various places such as stations, airports, subways, shopping centers, and banks, because these systems can record aspect and action of suspicious person in image data [1]. We apply video surveillance systems for crime prevention, because they can collect information and evidence about criminality. For reaching this aim, some observers must be employed and always monitor a lot of dynamic image shot by multiple cameras, without tiredness or fatigue, but it is impossible and if we do not employ observers or they do not monitor well due to above reasons, crime is not detected online and video surveillance acts just as a post-mortem tool. To overcome this limitation, it is necessary to have intelligent processes which are able to highlight the salient data and filter out normal conditions that do not pose a threat to security [2]. In order to create such intelligent systems, we should have a good understanding of human abnormal behaviors. Abnormality in a human

behavior can indicate itself in a person's trajectory, his type of activity or places that he moves there. For example if a person fights with another and his arms or legs stretched, he has abnormal behavior. Or a crinkle trajectory of a person in a public scene is abnormal and indicates he is drunk. Staying or walking on the some regions of a scene for a long time is abnormal also. In this paper, we studied about the last item and consider it from two different approaches, misuse detection model and anomaly detection model. There are samples of both of approaches in the literatures. Vaswani et al [3] selected the second approach in their study. They used a very low resolution video where the moving objects are small enough to be modeled as point objects in a 2D plane. They model a behavior by the polygonal shape of the configuration of the point during the time; then they calculated the mean shape of a behavior and finally applied this model to find the common shape of walking paths in an airport scenario. Thus any trajectory which is different enough from the common shape is labeled as suspicious. But this approach has a big shortcoming, since the averaging feature is not always reliable as the common pattern. In [4], a vision system that monitors behavior in a site over extended periods of time is presented. The system used a distributed set of sensors to cover the site, and an adaptive tracker detects multiple moving objects using the sensors. The approach involved learning motion pattern of activities for different object classes to detect unusual activities. Detection of suspicious behaviors has also been investigated in the area of network-based intrusion detection systems. Such systems detect an intrusion by examining packets in the network. Approaches in intrusion detection are classified into misuse detection model and anomaly detection model [5]. In our study, the misuse detection approach determines some special regions like sides of the corridor or the regions around the chairs or the air conditions and

١٥٦٨

whenever people stay there more than a time threshold, a warning signal will be activated. In the anomaly detection approach, first segment the scene in to the several regions, divide each region to some grids, than track people in half scenarios of CAVIAR dataset and determine how many times people move on these grids. Thus, assign a number which is related to the abundance of tracked point, to each region. A region with larger attributed number has higher normality level. One of the innovations of our research is flexibility of our systems, because when a new input entered, we determine its related regions duo to passing grids and increase the abundance number of those regions. This parameter indicates abnormality level of each region. The structure of the paper has been organized as follow: section 2 defines misuse detection approach and its application in spatial analysis for detecting abnormality. Section 3 talks about detection of suspicious and abnormal behavior according to anomaly detection method. Section 4 presents experimental results on CAVIAR dataset. Finally in section 5, we have a conclusion about current issue.

II. SPATIAL ANALYSIS BY MISUSE DETECTION MODEL Some of the abnormalities are related to the spatial domain. The image of the scene is observed and some parts of scene are determined as special regions in advance. People often walk along the middle of corridor and moving on places such as sides of the corridor for long time is not normal. Therefore, whenever a person stays in abnormal regions for equal or more than ζ, it is assumed as abnormal, thus, spatial warning in misuse detection approach which is called “mis_sp_warning” will be activated and set to 1 and if the person stays equal or more than 2ζ seconds, the warning signal will be set to 2. “mis_sp_warning(1)” is the name of this warning for person with object id 1. In this paper, CAVIAR dataset has been employed, it contains video clips from a corridor in a shopping center in Portugal. Special regions which have abnormality are presented with 11 rectangles as shown in fig. 1. Algorithm 1 describes this method well, where s0, …, s10 are suspicious regions, num_frames is the number

of frames in the CAVIAR dataset scenario, ζ is the time thershold and count is the number of seconds that the person with object id 1 stays in the suspicious regions. Note that the camera is 25 frames per seconds.

III. SPATIAL ANALYSIS BY ANOMALY DETECTION MODEL

In the previous section, we discussed about abnormal behavior according to the misuse detection model. Now we complete this issue by adding another approach which used anomaly detection model. One of the good points of our system operation is its dynamic behavior. Therefore, it is possible at first a region labeled as abnormal but as the time is passing and samples are increasing, its type converts to normal. In this section we explain our method base on this fact that in a public place, the number of normal behaviors is more than abnormal.

Algorithm 1 finding spatial warning by misuse detection model

//Definition of s0, s1, …, s10 as suspicious regions 1. count=0; 2. mis_sp_warning(1)=0; 3. for all suspicious regions(SR) 4. for i=1:num_frames-25 5. for j=0:25:num_frames 6. if((x_1(i+j),y_1(i+j)) € SR) 7. count=count+1 8. end if 9. end for 10. end for 11. if(count>=2ζ) 12. mis_sp_warning=2 13. else if(count>=ζ) 14. mis_sp_warning=1 15. end if 16. end if

١٥٦٩

Figure1. Abnormal regions of CAVIAR dataset scene

Algorithm 2 Calculating "commonality index" of the defined regions in the dataset scene

for s=1:num_scenario for j=1:14 for i=1:num_frames for k=1:num_persons if(x_k(i) < xr(j) && x_k(i) > xl(j)) if(y_k(i) < yu(j) && y_k(i) > yd(j)) region_count(s,j)=region_count(s,j)+1; end if end if end for end for commonality_index(j)= Σs region_count(s,j) end for end for

Initially, the scene plane is segmented to 14 different regions, then a parameter which is called "commonality_index" will assigned to each region according to the number of times that tracked people cover their grids. Finally all regions classify to normal and abnormal region groups. Now we have detected those regions that are more common as a pathway. Thus, if a trajectory path covers the regions with higher "commonality index", it means that is common trajectory and classify as normal while trajectories with traverse the regions with low "commonality index" assumed as abnormal and activate "anomal_sp_warning" signal which indicates value of warning signal for spatial analysis by anomaly detection method. The segmentation of the data set scene has been shown in fig. 2. Algorithm 2, describes this method well. After looking at the fig. 3 consider, num_scenario is the number of scenarios in CAVIAR dataset that we employ for training stage, num_frames and num_persons are the number of frames and present persons in the current scenario respectively. Region_count(s,j) is the number of times that persons in jth scenario move on sth region. Finally "commonality_index(j)" is the number of times that any person in any scenario has moved on jth region. If this parameter is high indicates that region is a common area for passing and movement is normal there but low value of this parameter show the region is an abnormal area. The innovation of our study is employing two different approaches and their comparison. Meanwhile the behaviors are classified in to the several groups according to the values of some parameters. Thus there is a difference between warning and alarm signals.

Figure 3. The name of dimensions of each region; l, r, u, d indicates left, right, up, down respectively

(xl, yu)

(xl, yd)

(xr, yu)

(xr, yd)

١٥٧٠

IV. EXPRIMENTAL RESULT In the CAVIAR dataset, there are 9 scenarios and 49 people in the different states. State is defined according to the regions that a person passes and his trajectory. If the trajectory of a person is in the defined abnormal area, means that person has suspicious behavior. First we employ misuse detection model on this dataset and receive results, then apply anomaly detection model.

A. Misuse detection results Table 1 has a perfect review on outputs. We have considered ζ= 150. Thus if a person stays 300 frames equal 12 seconds in the defined special regions, his behavior is suspicious according to his trajectory. If he spent between 6 to 12 second there, his behavior has lower level of abnormality and if the time duration that a person is in the special regions is less than 6 seconds, it is normal. In the following table Horizontal cells indicate persons with 1-7 object ids in each scenario and vertical cells indicate number of scenario. If a scenario has not object id i (i: 1-7), its related cell has filled by color. As we consider 35 persons have disable spatial warning signal. 7 persons have spatial warning signal with value 1 while 7 other persons have spatial warning signal with value 2. Fig. 4 includes 3 samples of behavior types, green line indicates normal trajectory, blue line shows abnormal trajectory and red line is a suspicious trajectory. Fig. 5 represents the abundance of each type of behavior in whole of CAVIAR dataset according to the misuse detection approach.

Figure 4. A sample of normal (green), abnormal (blue) and suspicious (red) trajectory in the dataset scene

Figure 2. Segmentation of a CAVIAR dataset scene to 14 regions

(xl, yu)

(xl, yd)

١٥٧١

Figure5. The abundance of normal, abnormal and suspicious behaviors

in the CAVIAR dataset Table 1: the value of spatial warning signal for all persons in all of the scenarios by misuse detection model

scenario1 2 3 4 5 6 7 8 9 Person1 0 0 0 1 0 0 0 0 0 Person2 1 0 0 2 0 2 0 2 0 Person3 0 0 0 2 1 1 0 1 Person4 0 0 0 2 0 0 0 0 Person5 0 0 0 0 0 0 2 Person6 0 0 2 0 1 Person7 1 0 0 Trajectories of the current dataset have been labeled by hand and in 46 cases, system decision regarding the behaviors is exactly correct. Thus our accuracy is 93.8%

B. Anomaly detection results In the anomaly detection analysis, first the CAVIAR dataset scene is divided to 14 regions as it has been shown in fig.2. Because the length of the corridor is larger than its width and most of people walk along the corridor not across of it, the proposed segmentation is horizontal. According to the algorithm 2, "commonality_index" of each region is calculated based on the total number of times that tracked people traverse its grid. As it is clear in fig. 6, each grid is assumed as a point in a region and whenever a person move on it, the value of "commonality_index" parameter of the related region is increased.

The values of "commonality_index" of all regions after 5th scenario have been written in the table 2. Regions si (i=0:13) have number 1, 2, …, 14, respectively. As it is clear in the table2, the regions such as 1, 3, 4 and 2 are common in people observed trajectories and the regions such as 11, 10 and 14 have lower commonality value. Whenever a trajectory passes on more common region it has higher level of normality than one which traverses on less common regions. But we have employed this strategy on 5 scenarios of our dataset. Since 6th scenario, the decision about type of input trajectories is done according to the previous stage, thus as the new inputs entered, our learning system will be improved according to the new observations and abundance of passing on the regions will be increased, table 3 includes "commonality_index" of each region after 6,7,8 and 9 scenario. Table 2: The abundance of movement on each region after 5 th scenario

The number of region Commonality_index after 5th stage

1 2446 2 1108 3 1974 4 1285 5 689 6 502 7 539 8 328 9 184

10 71 11 35 12 216 13 158 14 28

Suspicious Abnormal Normal

١٥٧٢

Table3: The abundance of movement on each region after 6th, 7th, 8th and 9th scenarios 6th 7th 8th 9th 1 5398 9678 14650 20551 2 2651 4446 7004 10124 3 4399 7006 9839 13155 4 2934 4814 6932 10037 5 1611 2758 4265 6429 6 1132 1967 3249 4948 7 1267 2275 3618 5355 8 902 2560 4424 6744 9 474 1170 1988 2992 10 142 578 1126 1836 11 70 638 1323 2144 12 432 1008 1657 2313 13 316 567 832 1097 14 56 179 302 431 Thus, it is possible that a walking path which includes abnormal regions initially and it was assumed as an abnormal trajectory, converts to a normal trajectory due to increase of "commonality_index" of the related regions. Table 3 includes "commonality_index" of each region after 6th, 7th, 8th and 9th scenario. Figure 7 and 8 show the abundance of each region after 5th scenario and 7th scenario. As it is seen, the order of the regions has changed during the time, for example in the first diagram region 8 has the 8th popular state while in the second diagram it has 5th state. Consider trajectory of the 4th person in the 7th scenario which is clear in the fig. 8. The trajectory passes from regions 6, 7, 8, 9, 10, 11, 12 which have 8th, 7th, 5th, 9th, 12th, 11th and 10th "commonality_index", respectively. If we assume their sum as a good measure, the total sum of these numbers for the current trajectory is 65, while after observing the 9th stage, the above order converts to 8th, 7th, 5th, 9th, 12th, 11th and 10th and total sum will be 62. Thus, if a new trajectory same as the current one entered, the level of abnormality will be lower and the behavior is assumed more normal. Therefore, the second scenario works better during the time passing and increase of observations. This difference can become more after a lot of observations.

Figure 7. The abundance of the regions after observing 5 first scenarios

Figure 8. The abundance of the regions after observing all 9 scenarios

١٥٧٣

Fig 8. The trajectory of 4th persons in the 7th scenario

V. CONCLUSION

REFERENCES [1] D. Duque, H. Santos, and I. P. Cortez, “Prediction of Abnormal Behaviors for Intelligent Video Surveillance Systems,” in computational Intelligence and Data Mining, 2007. Proceedings of the 2007 IEEE Symposium, 2007, pp.362-376 [2] A. Wiliem, V. K. Madasu, W. W. Boles, and P. K. Yarlagadda, “A context-based approach for detecting suspicious behaviours,” 03-Dec-2009. [Online]. Available: http://dicta2009.vu.edu.au/. [Accessed: 30-Jun-2012]. [3] N. Vaswani, A. Roy Chowdhury, and R. Chellappa, “Behavior recognition using the dynamics of the configuration of interacting objects,” in Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, 2003, vol. 2, p. II – 633–40 vol.2. [4] O. Faugeras, “Three-Dimensional Computer Vision,” Nov. 1993. [5] A. Wiliem, “Robust suspicious behaviour detection for smart surveillance systems,” 2010. [Online]. Available: http://eprints.qut.edu.au/41567/. [Accessed: 30-Jun-2012].

In the recent world, due to increase of crimes and terrorism activities, it is necessary to notice much regarding security problems. Thus, video surveillance technology has found much importance. It is better these systems equipped to internal intelligence and differentiate between normal, abnormal and suspicious behaviors. In this paper, we classified human behaviors through the spatial analysis and classify. This analysis employed by two different approaches, which are misuse detection and anomaly detection. In the first approach, staying in the special parts of the scene for a long time assumed as abnormal and in second we decided about a trajectory type by abundance of the related regions that the person has passed on them. In addition, our system has automatically update ability and it causes we have more precise decisions. Experimental results showed our two parts method, works very well. As a future work, we can expand our method by adding trajectory and pose estimation analysis.