Covert Behaviour Detection in a Collaborative Surveillance ... · plane, in subsection 2.2. 3. The autonomous calibration of each camera image plane, to the ground plane, using homography,

Covert Behaviour Detection in a Collaborative Surveillance System

Punarjay Chakravarty David Rawlinson Ray JarvisNational ICT Australia and Monash University

[email protected],[email protected],[email protected]

Abstract

Marzouqi and Jarvis’ covert path transform[Marzouqi and Jarvis, 2003] allows a robot toplan a path that minimizes robot visibility, inan effort to avoid detection. This paper usesthe same transform to measure the overtnessof human movements, in an attempt to iden-tify deliberately covert and suspicious humanbehaviour.

In a series of trials we detect and track bothanonymous individuals and known targets (sen-tries) using multiple cameras. Prior to trackingthe system is calibrated automatically by simul-taneous observations of a mobile robot in bothimage and ground planes. Subsequently, peopleare tracked on the ground plane and the pro-posed covertness metrics indicate the extent towhich their trajectories are covert or overt withrespect to the environment and with respect topatrolling sentries. Both these metrics showdifferences between normal and covert humantrajectories.

1 Introduction and Related Work

The algorithms for covert behaviour detection describedin this paper are part of a larger collaborative surveil-lance system developed during the first author’s PhDcandidature [Chakravarty, 2011a]. The system com-prises a mobile robot and a network of fixed surveil-lance cameras. The robot is capable of detecting andtracking people within the field of view (FOV) of itson-board panoramic camera as it patrols its environ-ment [Chakravarty and Jarvis, 2008]. It is also capableof autonomously calibrating all the external cameras tothe ground plane of the environment it is operating in[Chakravarty, 2011b]. When an intruder is out of rangeof the robot’s on-board sensing, but is detected in oneof the external surveillance cameras, the system is ableto guide the robot to the ground plane position of the

detected intruder using the calibration information forthat camera [Chakravarty and Jarvis, 2009].

Once the robot has calibrated all fixed cameras itis possible to track other targets on the ground plane.Ground plane tracking enables analysis of target trajec-tories, and classification as covert or overt.

Our method of determining the covert/overtness ofhuman paths was inspired by the Dark Path algorithm[Marzouqi and Jarvis, 2003], previously used in the sim-ulation of robot war games. Given a map of the en-vironment, the Dark Path is the most covert route be-tween start and goal locations. The Dark Path is a setof connected, minimally observable non-obstacle points,produced by transforming an obstacle map into a gen-eral visibility map. Areas that are observable from manylocations are more “visible”.

Existing work has focused on use of the covert pathtransform to allow a robot agent to avoid detection. Theproposal here, is to concentrate on analysis of humantrajectories, rather than scrutinize the characteristics ofindividuals’ appearance, gait, size or shape. Also pro-posed, is analysis of targets’ positions with respect toeach other: how do the targets interact? In our work,the overtness of a human target is given by computingtwo functions at each point on his or her trajectory:

1. Observability with respect to other targets (humanor robot sentries).

2. Observability with respect to all non-obstacle pointsin the environment.

Both metrics of observability are measured in termsof line-of-sight, which we achieve by ray-tracing betweenlocations.

[Hu et al., 2004a] provide a survey of the differentmethods of analyzing the behaviour of tracked people.In the literature, the determination of covert or overt-ness of peoples’ trajectories is less comprehensively re-searched than recognition of individuals’ characteristicsbut some pertinent work on classifying behaviour basedon trajectories is described below. A common feature ofthese works is that model parameters are derived from

Proceedings of Australasian Conference on Robotics and Automation, 7-9 Dec 2011, Monash University, Melbourne Australia.

Page 1 of 9

training data, rather than application of a prior model asin our case. In consequence, these works tend to detectstatistical anomalies rather than specific characteristicsof interest.

[Hu et al., 2004b] train a neural network using normaltrajectories and are subsequently able to use the networkto detect trajectories that deviate from the learnt nor-mal ones. This works well in situations where a largebody of human traffic behaves consistently, as is oftenthe case in public transport stations. However, it can-not be applied in low traffic secure environments, wherethere is insufficient ordinary behaviour for learning.

In other work [Brand et al., 1997; Natarajan andNevatia, 2007], computer-synthesized agents are usedto simulate the trajectories of people meeting and in-teracting with each other. These agents are used totrain Coupled Hidden Markov Models (HMMs) and Cou-pled Hidden Semi Markov Models, mathematical mod-elling frameworks for classifying behaviours like “Fol-low”, “Approach, Talk and Continue Separately”, “Ap-proach, Talk and Continue Together”, etc.

[Nguyen et al., 2005] also use track trajectories ob-tained from an overhead camera to model a person’s ac-tivities in a room. Track segments between landmarks(door, chair, TV, etc.) in a room are modelled as prim-itive behaviours and sequences of these are strung to-gether to model complex behaviours such as Short Meal,Normal Meal, etc. Primitive and complex behavioursare mapped into a hierarchical HMM, whose parametersare trained using 45 training sequences. 43 test sequencebehaviours are perfectly identified by their algorithm.

HMMs are a convenient framework for describing se-quences of discrete movements. However, in trajectoryanalysis and many other real-world problems it can bedifficult to decompose a stream of continuous measure-ments into discrete states. These problems are com-pounded by weakly or partially ordered sequences withmany permutations. For example, activity 3 could some-times follow activity 1, but other times follow activity 2.In addition, the set of states and partial ordering mightnot be known a priori.

[Bui et al., 2008] have tried to address these problemsin trajectory analysis. They carried out experiments inwhich students carried GPS devices around campus for afew days. A density-based clustering algorithm was firstused to cluster GPS locations according to time spent ateach location. Sequences of GPS locations were associ-ated with 7 clusters, identified by location (Bank, Wat-son Theatre, etc.) and tagged with activities (Banking,Lecture 1, etc.). An example of partial ordering of astudent’s activities is: “Lecture 2 follows Group Meet-ing 1 but could also follow Lecture 1; Group Meeting1 follows Lecture 1, but could also follow Breakfast.”They introduce the Hidden Permutation Model (HPM)

as a means of learning and recognizing high-level rou-tines which consist of permutations of “hidden” partiallyordered activities. They demonstrated better recall andprecision for recognition of activities, compared to otherbaseline methods.

Statistical modelling of activities or behaviour has thedrawback that observations have to be hand-labelled forthe models to be learnt, which can be a time consumingand laborious process. There is also the danger that anoverly complicated model is over-fitted to noisy data andwill perform poorly on unseen datasets.

[Dee and Hogg, 2004; Dee and Hogg, 2005] describe asystem that does not use statistical models to determinethe most commonly taken routes, and is able to inter-pret trajectories not previously observed during train-ing. This system can also deal with moving non-targetobjects such as cars.

Their first system [Dee and Hogg, 2004] determinesthe goal-directedness of a person by using the directionof the person’s motion, along with a model of possiblegoal locations for the person at each point on his/her tra-jectory, given the location of obstacles in the scene. Ob-stacles are marked off manually in the image plane, andlike our work, ray-tracing is used to find out a polygonalarea of visibility in an arc of one radian on either side ofthe direction of motion of the tracked entity. Obstaclevertices falling within this polygonal area are marked asgoals. In subsequent work [Dee and Hogg, 2005], theyoutline an algorithm that uses a modified Hausdorff mea-sure to compare points on the trajectory followed by aperson with points on trajectories to known goal sitesgenerated by the system.

Our work does not require statistical modelling of ob-served historical track trajectories, to identify outliers.Instead it classifies behaviour based on functions of tra-jectories’ relative visibility, and visibility with respect tothe environment. Our primary contributions are:

1. Metrics for classification of peoples’ behaviour ascovert or overt, based on track trajectory relativeto other persons and trajectory with respect to theenvironment.

2. Description of a practical framework in which thesemetrics can be computed, using typical surveillancecameras.

The remainder of the paper is organised as follows:Section 2 describes the autonomous calibration of exter-nal cameras to the ground plane using a mobile robot.Section 3 introduces the two visibility metrics we use todetermine the covertness of track trajectories. Section4 discusses the experiments conducted; their results arepresented in Section 5. Concluding remarks are givenin Section 6.


Page 2 of 9

2 Autonomous Camera Calibration andTarget Tracking

The system described in this paper comprises two low-resolution surveillance video cameras, a mobile robot,and observation of people in the environment visible tothe cameras. The cameras are used to track people andthe robot. The robot has two roles. First, it enablesautomatic computation of the relationship between everycamera and a common coordinate system on the groundplane. Second, the robot can function as an autonomoussentry. The trajectories of people observed by any or allcameras are then analysed in an attempt to detect covertbehaviour.

This section describes camera-calibration and target-tracking components that enable people to be tracked onthe ground plane. Subsections and their content are asfollows:

1. The localization of the robot on the ground plane,in subsection 2.1.

2. The detection and tracking of targets in the imageplane, in subsection 2.2.

3. The autonomous calibration of each camera imageplane, to the ground plane, using homography, insub-section 2.3.

For a more detailed explanation of the algorithmsdiscussed in this section, the reader is referred to[Chakravarty and Jarvis, 2009].

2.1 Particle Filtering for Mobile RobotLocalization

The mobile robot is equipped with a Hokuyo scanninglaser range finder and is able to localize itself on theground plane. Localization is achieved using a 2-D mapof obstacles on the plane of the robot’s Hokuyo sensor,and a particle-filter localization algorithm [Chakravartyand Jarvis, 2009]. The map is provided a priori, butthere are well known methods for autonomously creat-ing occupancy maps of this type [Thrun et al., 2005].Each particle (x, y, θ) represents a possible pose of theholonomic robot on the map. Laser range-finder readingspredicted by each particle are compared to actual read-ings from the sensor. Particles (poses) that agree withobserved laser measurements are given higher weightsthan particles that don’t match. Particles are also movedbetween cycles of the algorithm using odometry datathe robot’s wheel encoders. Particles are resampled ac-cording to their weights so that particles with higherweights survive and particles with lower weights die out.Over time, particles cluster around the actual pose ofthe robot.

2.2 Target Detection and Tracking

Mixture of Gaussians (MoG) [Stauffer and Grimson,2000] background subtraction is used to detect targetsin the image plane. The background is modelled at eachpixel using a mixture of weighted Gaussian probabilitydensity functions in the 3 colour channels R, G and B,each considered independent of the other. During aninitialization phase, the first 100 frames are stored inmemory, and 3 clusters are found for each pixel in theR-G-B space using a k-means clustering process. Themeans and standard deviations of each cluster are usedto initialize the Gaussians, whose weights are initializedto be low.

Thereafter, the pixel intensity for every new image, ischecked against the Gaussians modelling the backgroundat that pixel. If it is within 2.5 standard deviations of themean of any of the currently modelled Gaussians in all 3colour channels, then the pixel is considered part of thebackground and the corresponding Gaussian weightedhigher. If this is not the case, then the pixel is consideredforeground and a new Gaussian is introduced into thebackground model, centred around this intensity value,with a low initial weight.

Large connected-components of foreground pixels areassociated to form targets. Small components are as-sumed to be noise and removed. Targets are tracked bya bank of kalman filters. A new filter is initialized oneach target, provided it is well separated from currentlytracked targets. Targets are designated as robot/human.The robot is identified by the colour of its red base.

Once the robot enters the image plane of a camera, thesystem begins the process of calibrating the image planeto the ground plane. The robot begins a visual servoingprocedure [Chakravarty and Jarvis, 2009] that makes itorbit the centre of the image and keeps it within thecamera FOV until the calibration process is completed.Simultaneous localization of the robot in the image andground planes allows the system to collect a set of corre-sponding points, allowing computation of a homographybetween image and ground planes. This process is dis-cussed below.

2.3 Camera Calibration usingHomography

Homography is used for calibration of the image planesof the fixed cameras to a common ground plane. Let thepoints (collected during robot movement) in the groundplane be represented using the homogeneous vector X=(X,Y, Z), and their counterparts in the image plane byx= (x, y, 1). The coordinates x, y of each target in theimage plane are the centre of the base of the bounding-box of each foreground component. Under perspectiveprojection, the points are related by:


Page 3 of 9

Figure 1: Homography yields a matrix H which trans-forms points x, y in the camera image plane to groundplane coordinates X,Y, Z. Subsequently it is possible tocompute the ground-plane position of any moving ob-ject, allowing tracking of human targets.

X = Hx (1)

Written in vector notation, we have:

XYZ

=

h11 h12 h13h21 h22 h23h31 h32 h33

xy1

(2)

Equation 2 can be re-arranged to give the homoge-neous equation system for n points Ah = 0.

To fit the equation system into single column format-ting, A is split into A = [A1|A2], where :

A1 =

xi yi 1 0 0 00 0 0 xi yi 1

...

2nx6

(3)

A2 =

−xiXi −yiXi −Xi

−xiYi −yiYi −Yi...

2nx3

(4)

and

h =

h11h12h13h21h22h23h31h32h33

(5)

The elements of the transformation vector h are ob-tained from the eigen vector corresponding to the leasteigen value of ATA. For each pair of points on the imageplane and the ground plane that the robot traverses, wefill up two rows of the matrix A, which becomes a 2nx9matrix, where n is the number of points. The SingularValue Decomposition (SVD) of A:

A = UDVT (6)

results in the eigenvalues being arranged in decreasingorder along the diagonal of the matrix D and the corre-sponding eigenvectors along the columns of the matrixV. The values of the eigenvector associated with thesmallest eigenvalue make up the values of the vector h.

The set of corresponding points collected during therobot’s movement in front of the camera are subjectto errors due to transient localization errors in imageor ground planes. RANSAC (RANdom SAmpling Con-sensus) [Fischler and Bolles, 1981] is employed to filterout these erroneous points. RANSAC is normally usedwhen a mathematical model needs to be fitted to a setof points. In this case, the model is the homography ma-trix H (equation 1), whose 9 parameters need to be es-timated. The method randomly samples a set of points,calculates the model from these points, and then uses thesame model to find the set of inlier points (those pointswhich agree to the calculated model within a pre-definedthreshold). A refined model is then calculated from allthe inlier points. This process is repeated for either apre-determined number of iterations or until the reduc-tion in error between successive iterations falls below athreshold.

Once the homography is calculated for a camera, theground plane location corresponding to any arbitrarypoint in the image plane can now be found using equa-tion 1. The tracking algorithms discussed in the previoussub-section can now find ground plane positions for tar-gets detected on the image plane and track them on theground plane (Figure 2).

Tracking on the ground plane gives the trajectoriesof people moving through an environment in a conve-nient common format, even during transitions betweendifferent camera views. In experiments conducted for


Page 4 of 9

Figure 2: Cameras’ homography matrices are found bythe mobile robot during a calibration phase. Afterwards,people moving within view of any cameras are trackedrelative to a common ground plane system shared byall cameras. A map of obstacles in the environment isprovided a priori or generated by the robot during ex-ploration and calibration. In this figure, obstacles areshown in green and free space is black. Two targetsare being tracked; one target is currently simultaneouslyvisible in both cameras.

Figure 3: Visibility map of the sentry: ray-tracing is usedto determine the field of view of each sentry and therebythe visibility of intruders with respect to sentries.

this paper, we explored two metrics for the analysis ofthese trajectories. These are discussed in the followingsection.

3 Covert Behaviour Detection Metrics

Our emphasis is on human targets’ trajectories ratherthan characteristics of their appearance. The latter havebeen widely studied but are difficult to exploit given typ-ical low resolution overhead cameras that are most com-monly used in surveillance systems. Ordinary low qual-ity cameras and typical camera positioning are employedin our experiments. Two metrics are used to determinethe overtness of a tracked target at each point on its tra-jectory. These are described in the following sections.

3.1 Visibility With Respect to Sentry

For each sentry, a visibility map (Figure 3) is used tomeasure overtness of targets’ positions. The map repre-sents the extent of a sentry’s field of vision, given his/herposition in the environment. Ray-tracing is used to gen-erate the visibility map of the sentry. Rays are extendedin angular increments around the sentry’s current posi-tion until an obstacle is hit; viewing angles are limited to90 degrees on either side of the sentry’s direction of mo-tion. We assume that the sentry is looking the directionit is moving. We also assume that the first confirmedtrack is the sentry. Many appearance features could beused to identify sentries - particularly robot ones - butfor simplicity, in this study we always allow the sentryto enter the environment first. Additional tracked tar-gets are assumed to not be sentries. Generation of thesentry’s visibility map is formalized in Algorithm 1.

When non-sentry targets are detected, the visibilitymap is sampled in a window around the targets to as-certain whether or not they are in the sentry’s FOV atthat point in time. The number of pixels in this win-dow around the intruder’s position that are “hit” by thetraced rays is given as the visibility count. This metric isdetermined at each point of the intruder’s trajectory andthe visibility measure of the intruder with respect to thesentry at any time is a moving average of this visibilitycount.

3.2 Visibility With Respect toEnvironment

A generalized visibility map (first used by [Marzouqi andJarvis, 2003] for covert path planning for a mobile robot)is constructed to measure the visibility of the trackedsuspect with regard to his environment, regardless ofthe position of the sentry. This reflects the fact that theintruder may not know the trajectories of sentries andinstead may simply try to avoid observation. Rays aretraced until they hit obstacles in all directions aroundeach unoccupied cell in a map of the environment. The


Page 5 of 9

Algorithm 1 Visibility with respect to Sentry

1: Definition :2: V isSentry(M,xs, ys, θs, d)3: Inputs :4: Obstacle map M , a matrix of size w × h with ob-

stacles given value 1 and pixels representing emptyspace 0.

5: x position of sentry in the map xs,6: y position of sentry in the map ys,7: Bearing of sentry in map θs8: Incremental distance in pixels for ray tracing d9: Output :

10: Completed visibility map with respect to sentry Vs,a matrix of equal dimension to M , representing thesentry’s field of view in the environment

11: Procedure :12: Initialize every element in Vs to zero.13: x′ = xs,y

′ = ys14: for θ = θs : θs ± π/2 do15: while M [x′, y′] is not an obstacle do16: Vs[x

′, y′] = 117: x′ = x′ + d · cos(θ)18: y′ = y′ + d · sin(θ)19: end while20: end for

Algorithm 2 Generalized Visibility

1: Definition :2: V isGeneral(M,d)3: Inputs :4: Obstacle map M , a matrix of size w × h with ob-

stacles given value 1 and pixels representing emptyspace 0.

5: Incremental distance in pixels for ray tracing d6: Output :7: Completed generalized visibility map Vg8: Procedure :9: Initialize every element in Vg to zero.

10: for x=0:w do11: for y=0:h do12: if M [x, y] is not an obstacle then13: for θ = 0 : 2π do14: x′ = x+ d · cos(θ)15: y′ = y + d · sin(θ)16: while M [x′, y′] is not an obstacle do17: Vg[x, y] = Vg[x, y] + 118: x′ = x′ + d · cos(θ)19: y′ = y′ + d · sin(θ)20: end while21: end for22: end if23: end for24: end for

Figure 4: Generalized visibility map of the environment:Darker areas are less visible than lighter areas. Blackareas are obstacles.

number of cells that are visible from the current cell arerecorded for each cell in the environment. The gener-ation of the generalized visibility map is given in Algo-rithm 2. This is a time-consuming method (with O(N2)complexity, where N is the number of free pixels in theenvironment), but can be done offline and only needs tobe computed once. For each position in the environment,the corresponding value in this map (shown in Figure 4)gives a generalized visibility value. Generalized visibilityis measured at each point on each intruder’s trajectoryand is averaged over time, giving the intruder’s visibilitywith respect to the environment for the entire trajectory.

Both these measures are used to determine covertnessof intruders in experiments detailed in the next section.

4 Experiments

The objective of the experiment reported in this sectionis to determine whether there are useful, measurable dif-ferences between deliberately covert and normal humanbehaviour. In these trials the sentries are humans, butthey could easily be robots following predefined patrolroutes. The metrics used for covert behaviour detectionhave already been described in Section 3. This exper-iment also demonstrates effective multi-target trackingbetween multiple camera views, in a low-traffic environ-ment.

An arena was set up with obstacles offering hidingplaces. All parts of the arena were visible from at leastone of two cameras mounted overhead. The map of thearena and an image to ground plane mapping for eachcamera were provided to the system. Actors were re-cruited to play the roles of sentry and intruder. In total16 trials were conducted (each trial being the traver-sal of an intruder from one point in the arena to an-other, while a sentry patrols). In each case the sentries


Page 6 of 9

Figure 5: A snapshot of target tracking on the groundplane: Red rectangles indicate the bounding boxes oftargets detected in the image planes of the 2 cameraswith overlapping fields of view. Red and yellow ellipsoidsencircling their ground plane positions show the proba-bility distribution of their tracks (sentry and intruderrespectively) on the ground plane. Rays are traced fromthe sentry’s position outwards in a 180 degree arc aroundhis direction of motion to estimate his field of view. Bluebars on the bottom left labelled cov1 (intruder’s visibil-ity with respect to sentry) and cov2 (intruder’s visibilitywith respect to environment) show the averaged visibil-ity/covertness measures up to that point in the trajec-tory of the intruder. The smaller the bars, the morevisible their trajectories.

were asked to patrol the environment at a relatively con-stant speed in all the trials. The intruders were askedto pursue a covert path in 8 trials, and a normal (non-suspicious)/overt path in the other 8. The instructionsto the actors were given exactly as detailed below. Thecameras were obvious but the actors were not explicitlymade aware of them.

Instructions for the covert path actors were as follows:“Move from the START to the GOAL without being seenby the guard. You can crouch and hide behind obstacles.Walk/crawl slowly: You must not run. Instead, planyour route to avoid observation by the guard. You cantake as long as you like. The guard will repeat his patrolroute continuously. Try to minimise any view of you theguard has, but don’t worry if you’re seen temporarily.Just keep going.”

Instructions for the overt path actors: “Move from theSTART to the GOAL. Walk slowly: you must not run.You can take as long as you like. You may encounteranother subject during your movement. You can chooseany route you like.”

A screen snapshot of target tracking during the ex-periment is displayed in Figure 5. Video streams fromthe 2 cameras show intruder and sentry positions, andthe multi-target tracking system described in Section 2allows ground-plane positions to be shown also. Uncer-tainty ellipses for each target’s filter are displayed.

5 Results

The two measures of covertness of a person walkingthrough the environment (detailed in Section 3) are:

1. Visibility with respect to patrolling sentry.

2. Visibility with respect to environment.

Figure 6 displays the average visibility of the intruderwith respect to the sentry over 8 trials. Covert runs areshown in blue, and overt ones in red.

Figure 7 shows the average visibility of the intruderwith respect to the environment over 8 trials. Again,blue and red bars are used to show covert and overttrajectories respectively.

It can be seen that covert (blue) trajectories have con-sistently lower visibility values than overt (red) trajec-tories, both with respect to the sentry (metric 1) andwith respect to the environment (metric 2). There is amarked difference between overt and covert trajectorieswith respect to sentry visibility. However, with respectto the environment the difference is less distinct. Never-theless the generalized visibility with respect to the en-vironment was less for every covert trial than any overttrial. Using a single-tailed T-test and assuming the twosets have unequal variance, the p-value obtained for met-ric 1 is 5×10−5 and the p-value for metric 2 is 1.71×10−6.These low p-values lead us to conclude that these results


Page 7 of 9

Figure 6: Visibility of intruder with respect to sentry(averaged over trajectory lengths) for 8 covert (blue) and8 overt (red) trials in our experiment.

Figure 7: Visibility of intruder with respect to envi-ronment (averaged over trajectory lengths) for 8 covert(blue) and 8 overt (red) trials.

reflect changes in behaviour caused by differing inten-tions of the actors between experimental conditions.

A video clip showing two covert and two overt runs isavailable athttp://www.youtube.com/watch?v=pgn88_azriE

6 Conclusions and Future Work

We have shown that using a network of typical fixed se-curity cameras in a low-traffic indoor environment wecan distinguish the behaviour of actors having differing(presumed) intentions. This result implies another con-clusion, namely that persons intent on evasion choose tomove in a different way to “normal” individuals. How-ever, without a much larger and more strictly controlledexperiment in real environments, it is difficult to drawfirm conclusions about human behaviour in general.

This paper contributes evidence that the intent toavoid observation changes behaviour in a way that can beautonomously and reliably detected by typical securityequipment. While many technologies can help to iden-tify potentially threatening or unauthorised persons, feware able to exploit the low-resolution fixed camera net-works that are prolific today. With such data it is hardto model facial appearance, expression, limb movementsor other factors that might arouse suspicion. But it isrelatively easy to model trajectories.

Although between covert and overt trials differencesin general visibility were less marked than differencesin visibility with respect to sentries, general visibilityis potentially equally or more useful. Mobile sentries- either robot or human - are expensive and thereforecoverage is sparse. It is very useful to be able to monitorbehaviour in areas not regularly or constantly patrolledby sentries.

A useful extension to this work could be dynamic sen-try path planning. Any unknown persons who happenedto avoid observation by sentries could cause patrols tobe rerouted to intercept. Interception urgency could de-pend on the covertness of observed behaviour. Robotsentries could be routed to make innocent evasion highlyunlikely, and therefore highly suspicious.

The authors also hope to examine human trajecto-ries for other criteria that highlight suspicious persons.For example, acceleration or other moments of motionmay be significant. These criteria may not be obvious:Although the authors expected that prolonged proxim-ity to obstacles would be a good indicator of “hiding”,ad-hoc observations during our experiment contradictedthis. People tend to move close to obstacles for efficiencyand convenience, as well as covertness.

This work may have a number of direct applicationsin low-traffic secure environments, such as public placesout of hours, and private business premises. Many falsealarms could be eliminated and security improved by


Page 8 of 9

http://www.youtube.com/watch?v=pgn88_azriE

causing the system to autonomously instruct humanor robot guards to intercept individuals who have, bychance or design, thus far avoided inspection.

References

[Chakravarty, 2011a] P. Chakravarty Surveillance & In-tervention: Collaboration Between a Robot & FixedCameras. PhD Thesis, Monash University, 2011.

[Chakravarty and Jarvis, 2008] P. Chakravarty and R.Jarvis. People Tracking from a Moving PanoramicCamera. In Australasian Conference on Robotics andAutomation (ACRA), 2008.

[Chakravarty, 2011b] P. Chakravarty Surveillance & In-tervention: Collaboration Between a Robot & FixedCameras, chapter Autonomous Calibration of Cam-eras to Ground, pages 126–136. PhD Thesis, MonashUniversity, 2011.

[Chakravarty and Jarvis, 2009] P. Chakravarty and R.Jarvis. External Cameras & A Mobile Robot: A Col-laborative Surveillance System. In Australasian Con-ference on Robotics and Automation (ACRA), 2009.

[Thrun et al., 2005] S. Thrun, W. Burgard and DieterFox. Mapping, Probabilistic Robotics, pages 279–485,MIT Press, 2005.

[Marzouqi and Jarvis, 2003] M. Marzouqi and R. Jarvis.Covert Path Planning for Autonomous Robot Naviga-tion in Known Environments. In Australasian Confer-ence on Robotics and Automation (ACRA), 2003.

[Hu et al., 2004a] W. Hu, T. Tan, L. Wang and S.J.Maybank. A Survey on Visual Surveillance of Ob-ject Motion and Behaviors. In IEEE Transactions onSystems, Man and Cybernetics, pages 334–352, Vol.34, 2004.

[Hu et al., 2004b] W. Hu, D. Xie, T. Tan and S.J. May-bank. Learning Activity Patterns using Fuzzy Self-organizing Neural Network. IEEE Transactions onSystems, Man and Cybernetics, pages 1618–1626, Vol.34, 2004.

[Brand et al., 1997] M. Brand, N. Oliver and A. Pent-land. Coupled Hidden Markov Models for ComplexAction Recognition. In Computer Vision and PatternRecognition (CVPR), pages 994–999, 1997.

[Natarajan and Nevatia, 2007] P. Natarajan and R.Nevatia. Coupled Hidden Semi Markov Models forActivity Recognition. In IEEE Workshop on Motionand Video Computing, Vol. 0, 2007.

[Nguyen et al., 2005] N.T, Nguyen, D.Q. Phung, S.Venkatesh and H. Bui. Learning and Detecting Ac-tivities from Movement Trajectories using the Hierar-chical Hidden Markov Model. In Computer Vision and

Pattern Recognition (CVPR), pages 955–960, Vol. 2,2005.

[Bui et al., 2008] H. Bui, D. Phung, S. Venkatesh and H.Phan. The Hidden Permutation Model and Location-Based Activity Recognition. In 23rd National Confer-ence on Artificial Intelligence (AAAI), pages 1345–1350, Vol. 3, 2008.

[Dee and Hogg, 2004] H.M. Dee and D.C. Hogg. Detect-ing Inexplicable Behaviour. In British Machine VisionConference (BMVC), pages 477–486, 2004.

[Dee and Hogg, 2005] H.M. Dee and D.C. Hogg. Navi-gational Strategies and Surveillance. In IEEE Interna-tional Workshop on Visual Surveillance, pages 73–81,2005.

[Stauffer and Grimson, 2000] C. Stauffer and W. Grim-son Learning Patterns of Activity using Real-timeTracking. In IEEE Transactions on Pattern Recogni-tion and Machine Intelligence (TPAMI), pages 747–757, Volume 22, 2000.

[Fischler and Bolles, 1981] M. Fischler and R. Bolles.Random Sample Consensus: A Paradigm for ModelFitting with Applications to Image Analysis and Au-tomated Cartography. In Communications of theACM, pages 381–395, Vol. 24, 1981.


Page 9 of 9

Documents

Covert Behaviour Detection in a Collaborative Surveillance ... · plane, in subsection 2.2. 3. The autonomous calibration of each camera image plane, to the ground plane, using homography,