7/28/2019 00959025

1/4

IMAGE STABILIZATION ALGORITHMS FO R VIDEO-SURVEILLANCE APPLICATIONSLucio Marcenaro, Gianni Vernazza and Carlo S. Regazzoni

University ofGenoa, Department of Biophysical and Electronic Engineering (DIBE)Via allOp era Pia 11/A 1-16145 Genov a (Italy)Phone: +39-0 10-3532792 Fax: +39-0 10-3532134e-mail: carlo@dibe.unige.it

ABSTRACTIn this paper, an image-stab ilization algorithm is presented thotis specifically oriented toward v ideo-surveillanc e applications.The proposed approach is based on a novel motion-compensation method that is an adaptation of a well-knownimage-stabilization algorithm for visualization purposes tovideo-surveillance applications. In particular, the illustratedmethods take into account the specificity of typical video-surveillance app lications, where objects moving in a scene ofe ncover a large part of an image thus causing the failure of classicimage-stabilization techniques. In the second part of the paper,evaluation methods fo r image stabilization algorithm s arediscussed.

1. INTRODUCTIONIn the past few years, the market of video-surveillance systemshas considerably grown. Video-surveillance sensors are usuallyrepresented by cameras that acquire video sequences to betransmitted to a remote control center. In first-generation video-surveillance systems, acquired images are presented to thehuman operator, who has to search for potentially dangeroussituations. This paper deals with second-generation surveillancesystems, where the images acquired by the sensors are processedby an automatic system that can detect and locate objects movingwithin a scene and possibly, recognize and classify theirtypologies and behaviors. Static video-surveillance cameras areoften mounted on poles, thus they may be affected by vibrationsand unwanted movements, for example, due to atmosphericdisturbances. Such interferences are extremely harmful toautomatic video-surveillance systems as they cause aconsiderable degradation of automatic event recognition. Image-processing methods adopted by this kind of systems typically usean image of an empty scene as a reference image for objectdetection and location. An unwanted movement in the camerashot often causes an incorrect superposition of the current andreference images as well as destructive consequences for typicalchange-detection algorithms. In the present paper, a novelimage-stabilization algorithm is described together with themethods for evaluating the obtained performances, with specialattention on automatic video-surveillance systems.

This work was partially supported by the Ministry of Universities andScientific Research (MURST) of the Italian Government an d by theBritish Council.

0-7803-6725-1/01/$10.0002001 IEEE

The paper is organized as follows: in Section I , the principles ofimage-registration and motion-compensation techniques areoutlined. In Section 2, the need of image-stabilization algorithmsfor video-surveillance applications is highlighted. In Sections 3and 4, the method proposed for image stabilization in video-surveillance systems is detailed and analyzed. Sections 5 and 6deal with possible evaluation methods and results of thedescribed algorithms respectively. Conclusions are drawn inSection 7.

2. IMAGE-REGISTRATION AND MOTION-COMPENSATION TECHNIQUESMany well-known motion estimation techniques can be found inliterature. These algorithms are sometimes defined by the termimage registration, which is related to the evaluation of themovements between successive images in a sequence. Image-registration methods constitute the basis for image-stabilizationand mosaicing techniques. Such algorithms are needed in severalapplications such as:

Wide-area surveillance: image-mosaicing algorithms allowthe surveillance of very large areas (industrial plants, etc) byusing a small number of cameras;Cartography: sequences of aerial images can be combined togenerate maps;Outdoor surveillance: image stabilization is needed tocompensate for unwanted sensor movements (e.g., highwaysurveillance) [11;Automatic-vehicle driving: image stabilization is needed toattenuate vibration due to mechanical vehicle movements.

Image registration can be defined as the estimation of thecorrespondences between an input image and a reference frameproviding the reference system for the movement to be estimated.Image registration algorithms are used to determine thedisplacement between two images: the geometricaltransformation is represented by the model of the cameramovement. Image-registration algorithms can be divided intotwo categories:

feature-based techniques [ 2 ] : the movement is estimated bytracking a set of features in the images; in this way, it ispossible to find the translations and rotations that occurredbetween the considered frames. The feature-selection stagecan be very complex because the robustness of the methodmostly depends on this step.dense-matching techniques [3, 41: hese methods estimateparameters related to the model of movement, while

349

7/28/2019 00959025

2/4

minimizing the cost functional associated, for instance, withthe differences between analogous regions in the framesconsidered.

A method based on feature tracking is proposed by Morimotoand Chellappa in [ 2 ] , Hansen et al. [SI propose an algorithmbased on global optical-flow estimation. A method widely usedfor motion estimation ilj the Block Matching Algorithm (BMA),which is based on the subdivision of the images of a sequenceinto blocks: each block. is then tracked in the sequence and theresults of the tracking phase are used for motion compensation.However, this method is very time-consuming, then it is notuseful when a real-time requirement must be met. Manyalgorithms have been proposed to improve algorithmperformances [6 , 71. The second step for image stabilization liesin motion compensation: motion parameters that have beenestimated through mvstion registration techniques are nowapplied to the sequence in order to stabilize the images byplacing them in a common reference system. In a general case,motion compensation should compensate for unwantedmovements of the camera, while preserving the ones due tomoving objects or to global camera-shot movements. A blockscheme of a general image-stabilization system is depicted inFigure 1.

Figure 1 Block diagiam of a generic image stabilizationsystem.The Motion Estimation module evaluates inter-frame motion,and the Motion Compensation module calculates the globaltransformation that is needed to stabilize the current image.Finally the Image Composition module modifies the consideredimage according to the results of the Motion Compensationmodule, thus generating the stabilized sequence or, if required,the mosaicing image.In the following, only static-camera surveillance systems will beconsidered, hence only the motion due to movements of objectsin a scene will be preserved.

stable features from the images and to track them in thesequence.Unfortunately, the contents of video-surveillance images areconsiderably different because, typically, moving objectsconstitute a considerable part of an image, often covering a largearea of the background and making unreliable the features placedbeneath an object in a certain frame. Besides, the featuresdetected within the bounding box of the object must be rejectedbecause of their instability.

4. PROPOSED METHODS4.1. The grid methodThis method uses a fixed set of points on a grid that issuperimposed on an image. The method evaluates the motiontransformation to be applied to the image by minimizing themean square error between the cor responding pixels in theimages of a sequence. First, a grid of points is placed on both thereference and current images; sets of translations are applied tothe grid and, for each set, a correlation index is calculated as:

wherep,',, denotes the pixels with the coordinates ( j ,k ) in theimage t , where t =O is the reference image and t= i is the currentimage; m and n represent the numbers of points on the grid alongthe horizontal and vertical axes, respectively.Equation (1 ) defines a correlation index between the consideredimages when a translation vector (a,b)is applied to the images.The index is computed on the discrete set of points representedby the grid; in order to minimize this term, an exhaustive searchis performed over a certain range of translations, and the vectorcorresponding to the maximum correlation is selected for motioncompensation.Figure 2 shows a typical video-surveillance image (the imagesequence presented in [9] has been used for the present paper)where a reference grid (black squares) and the translated one(white sauares) have been SuDerimmsed.

3. IMAGE STABILIZATIONFOR SURVEILLANCEAPPLICATIONSA standard video-surveillance application will be considered in

the following. Automatic video-surveillance systems aim toprocess an input image sequence in order to detect and locateobjects present in a scene, classify them, and interpret theirbehavior. Subsequently, they can send an alarm signal to informthe human operator that something dangerous is happening. Atypical video-surveillance system [8] uses a reference image(background) that dtepicts the scene without any objects. Bysubtracting and threslnolding between the current acquired imageand the background, the automatic system is able to detectobjects in the scene. Because the low-level image-processingalgorithms operate 011 the corresponding pixels of the consideredcouple of images, it is very important for the images to be in thesame reference system: this is not guaranteed, in particular, if theinstallation of the seiisor suffers from vibrations.The algorithms that have been developed up to now work onwide panoramic images where moving objects do not cover alarge percentage of a scene. In this case, a feature-basedstabilization algorithm works well, being able to select highly

Figure 2 Schematic representationof the grid method.A validation algorithm is then used to discard the grid points thatcorrespond to moving objects in the image: each point isassociated with a confidence coefficient that is incrementedwhen a certain point can be successfully tracked, whereas it isdecremented when no corresponding point is found in thecurrent processed image. The confidence coefficient is computedas a percentage of correct tracking; it is initialized with 1 andupdated as follows:

(2)No.of times that the point was trackedNo . of processed framesA point can be correctly tracked when a pixel with the samefeatures is detected within a certain research area with respect to

C =

350

7/28/2019 00959025

3/4

the reference frame. Finally, the point is actually tracked if theabove coefficient is above a certain threshold, hence the point isdefined as trackable.If the method considers only the information about the pointslying on the grid, it turns to be very fast but still reliable forvideo-surveillance applications when the grid is dense enough tocover a large part of the image (10% is considered to beenough).4.2. Feature-based methodThe second method considered is based on the feature trackerproposed in [lo] and used in [ l l ] for stabilization of aerialimages. A set of points is selected from a reference imagebyapplying a criterion able to detect corners in the image. For eachconsidered area, a two-dimensional gradient is evaluated, and, ifa corner point is detected, its typology is classified by evaluatingthe following functions for each pixel in the reference image(Fig. 3):~val,_,I P , ~ -~,-~,~.~l+l

7/28/2019 00959025

4/4

the system have been evaluated through the analysis of thesuperposition of real and estimated bounding boxes, as proposedin [14] .

6. RESULTSThe proposed methods were evaluated by calculating the GT Fand ITF indexes for different change-detection thresholds. Boththe grid method and the feature-tracking method were comparedwith the results obtained by using uncompensated sequences.The GTF index was used to evaluate motion compensation withrespect to an initial reference image; through the IT F index, onecan estimate the correlation between temporally adjacent images.Figures 4a and 4b show that, in both cases, the curve thatrepresents the uncompensated sequence is always below theother lines: this means rhat, in both cases, the proposed methodsare able to compensate for unwanted motion; moreover, the gridmethod achieves higher performances. Considering that thegraphs are on a dB-like scale, it can be concluded that theillustrated methods provide good performances.Figure 5displays the FLOC curves calculated for the consideredsystem. Each point of the curves wa s calculated by varying themotion range of the images to be stabilized. The figure showsthat the proposed methods ensure higher correct-detection-to-false alarm probability ratios than the system based onuncompensated images. The graph also points out that, in thiscase, the grid method works better than the feature-tracking one.The working area highlighted in the figure refers to movementsin the range of 5 to 15 pixels: in this area, the stabilizationmethods reach a maximum gain.

yl 1u.J I r m mlr.h*.

(b)Figure 4 Evaluations of the modified (a) GTF and (b) ITF for theconsidered methods

7. CONCLUSIONSIn conclusion, this palper has shown a possible evolution of well-known motion-compensation and image-stabilization methods.The proposed methods are able to filter unwanted motion, whilepreserving the movements of the objects in a scene. Evaluationmethods have been developed for the proposed image-

stabilization video-surveillance algorithms. The validity of theadopted approach is demonstrated by the measures on the outputof the system considered as well as by the probabilitiescalculated on a complete video-surveillance system.

V . . L l l U h .

Figure 5 ROC curves for a video-surveillance system using theconsidered methods8. REFERENCES

[ I ] S . Peleg, B . Rousso, A. Rav-Acha. and A. Zomet. Mosaicing onadaptive manifolds. IEEE Trans. on Pattern Analysis and Marlt ineIntelligence, Vol. 12 , No . 10, pp. 1144-1154,October 2000.[2] C.H. Morimoto and R. Chellappa. Automatic digital imagestabilization. EEE In t . Cot$ on P ar tem Recogni t ion , August 1996.[3] C. Kuglin and D. Hines. The phase correlation image alignmentmethod. IEEE Conference on Cybernetics and So ciety, September 1975.[4] R. Szeliski. Image mosaicing for tele-reality application. TechnicalReport CRL 94/%Digital Equipment Corporation, Cambridge ResearchLab, May 1994.[SI M. Hansen, P. Anandan, K. Dana, G. Va n der Wal, and P. Burt.Real-time scene stabilization and mosaic construction. DARPA ImageUnderstanding Workshop.November 1 994.[6] M.J. Chen et al. A new block-matching criterion for motionestimation and its implementation. IEEE Trans. Circuit and System forVideo Teclinology. vol. 5, p. 231-236, June 1995.[71 L.M. Po and W.C. Ma. A novel four-step search algorithm for fastblock motion estimation. IEEE Trans. Circuit arid System fo r V ideoTechnology, vol. 3, no. I , June 1996.[SI L. Marcenaro, F. Oberti, and C.S. Regazzoni. Short-memory shapemodels for ground-plane predictive object tracking. Proc. First IEEElnr. Workshop on Perfonnance Evaluation of Tracking andSurveillance, ETS2000, Grenoble, France, pp.50-56.2000.[9] First IEEE Int. Workshop on Performance Evaluation of Trackingand Surveillance, PETS2000, Grenoble, France, 2000.[IO] C. Tomasi and T. Kanade. Detection and tracking of point features.Technical Report CMU-CS-91-132. Camegie Mellon University,Pittsburgh, PA, April 1991.[Ill A. Censi, A. Fusiello, and V . Roberto. Image stabilization byfeature tracking. Image Analysis arid Processing ICIAP, pp. 665-667,September 1999.[12] C. Morimoto and R. Chellappa. Fast electronic digital imagestabilization.Proc. Int. Conf on Pattern Recognition, Vienna, Austria,August 1996.[13] H. Van Trees. Classical Detection and Estimation Theory -Detection, Estimation and Modulation Theory,John Wiley & Sons, Inc.,1968,pp . 19-46.[I41 F. Oberti, E. Stringa, and G. Vernazza. Performance evaluationcriterion for characterizing video-surveillance systems, Real-Timelmuging Journal, 2001 (in press).

352