A Foreground Extraction Algorithm Based on Adaptively

Adjusted Gaussian Mixture Models

Tianci HUANG JingBang QIU and Takeshi IKENAGA

Graduate School of Information Production and System Waseda University

N355 2-7 Hibikino Wakamatsu Kitakyushu

Fukuoka Japan 808-0135

E-mail bond0060ruriwasedajp

AbstractmdashBackground subtraction is a widely used method for moving object detection in computer vision field To cope with highly dynamic and complex environments the mixture of models has been proposed In this paper a background subtraction method is proposed based on the popular Gaussian Mixture Models technique and a scheme is put forward to adaptively adjust the number of Gaussian distributions aiming at speeding up execution Moreover edge-based image is utilized to weaken the effect of illumination changes and shadows of moving objects The final foreground mask is extracted by the proposed data fusion scheme Experimental results validate the performance of proposed algorithm in both computational complexity and segmentation quality

Keywords-Background subtraction Gaussian Mixture Model

Motion detection

I INTRODUCTION

The ability to estimate non-stationary temporal distributions efficiently and accurately is a key element for any robust vision system Background subtraction is a method typically used to segment moving regions in video sequences by comparing each new frame to a model of the scene background It has been used successfully for indoor and outdoor applications Generally speaking there are two types of background subtraction algorithms pixel modeling and global modeling Pixelwise approaches take each pixel as an output from an independent random process while global modeling considers the spatial correlations One of simple adaptive background differencing technique has become a simplest solution since it is sensitive to motion in background and not influenced by the gradual illumination changes however its high sensitivity makes this algorithm come out with a very bad result in complex background especially with periodic noise existing In an actual scene the complex background such as snowy or windy conditions make the conventional algorithm unfit for the real surveillance systems Stauffer and Grimson [25] modeled each pixel as a mixture of Gaussians and an online EM Algorithm was proposed by P KaewTraKulPong et al [6] to update the model Even though 3 to 5 Gaussian distributions are able to model a multi-modal background with difficult situations like shaking branches of tree clutter and so forth there is a fact that this kind of pixel-based background modeling is sensitive to noise and illumination change Lots of effort made to modify the model or integrate

III GMM BASED ADAPTIVELY ADJUSTED MECHANISM

Even though K (3 to 5) Gaussian distributions are capable of modeling a multimodal background the number of Gaussian distributions brings a huge computational load for surveillance system If K equals to 5 it achieves to 320x240x5=384000 with complex computation for learning process of RGB 3 channels to update weight mean and variance which makes the surveillance system difficult to achieve real time let alone the higher resolution for current needs In fact not all the pixels of the background change

repetitively To the pixels where less repetitive motion occurs such as the areas of ground houses and parking lot in the scene of Fig 1(a) only the first Fig 1(b) and second Fig 1(c) highest weighted Gaussians at each pixel are adequate to model the multi-possibilities for background and as the Fig 1(d) (e) (f) show most of third fourth and fifth highest weighted Gaussians were much less updated

except the car-passing area where should be foreground area Based on the analysis above Adaptively Adjustment Mechanism was proposed to reduce the number of the Gaussian distributions which offered less contribution to the multi-possibilities in the process of modeling background From the analysis and experiments different model number can be adopted for different pixels The update of weight mean and variance for our proposal is based on online EM algorithm [6] which allowed fast convergence on a stable background model for GMMproposed by P KaewTraKulPong et al The main procedure

of the adaptive Elimination Mechanism for GMM is as follows

1048697 E-step As the online EM algorithm does we begin our estimating of the Gaussian Mixture Model by

expected sufficient statistics which is called E-step Due to the unpredictable factor and possibilities for the complexity of background pixel and the first L frames is very important for Gaussian models to dominant background component and achieve stable adaptationswe keep the number of Gaussians models K for each pixel fixed during E-step Experiments also show these could provide a good estimate which helps to improve the accuracy for M-step process For initialization part

VI CONCLUSIONS

This paper presents a new background subtraction scheme based on the Gaussian Mixture Models We proposed an Adaptively Adjusted Mechanism to reduce less useful Gaussian distributions which contribute less to background modeling Through this method the execution speed for

video is improved Additionally an image segmentation based on edge information has been introduced By lessening the influence from illumination changes and excluding the shadow of moving object the

quality of segmentation mask is enhanced consequently Experimental results validate the gains in both processing speed and segmentation quality The proposed algorithm works very well in various complex background for both outdoor and indoor surveillance system

Histogram-based Foreground Object Extraction for Indoor

and Outdoor ScenesMandar Kulkarni

_

IPCV Lab Electrical Engg Department

Indian Institute of Technology Madras

Chennai India

mandareln40gmailcom

ABSTRACT

Extracting foreground objects is an important task in many video processinganalysis systems In this paper we propose a technique for foreground object extraction under static camera condition In our approach the spatial histogram of a single background image is modeled as Mixture of Gaussians and this model is updated after every few frames To extract the foreground input frames are compared with current background frame model and foreground pixels are classified according to intensity differences To mitigate the errors caused due to movement of the background objects (eg tree leaves in outdoor scenes) we also incorporate optical flow in an efficient manner We demonstrate performance of our approach on various indoor and outdoor scenes

Foreground extraction is an important task in many computervision applications In this paper we propose an method which models the histogram of an initial background frame by the mixtures of Gaussians Generally a natural background includes large objects such as trees road floor buildings walls etc each of which contains pixels with similar intensity values but whose intensities differ considerably from each other Hence the histogram of the background frame containing multiple objects is usually multi-modal and can be approximated by the Mixture of Gaussian The number of Gaussians is determined by the number of objects present in the background We also update the background histogram model at regular intervals to adapt to illumination variations over time We use the Expectation Maximization (EM) algorithm to find maximum likelihood parameters of every Gaussian

III GMM BASED ADAPTIVELY ADJUSTED MECHANISM

Even though K (3 to 5) Gaussian distributions are capable of modeling a multimodal background the number of Gaussian distributions brings a huge computational load for surveillance system If K equals to 5 it achieves to 320x240x5=384000 with complex computation for learning process of RGB 3 channels to update weight mean and variance which makes the surveillance system difficult to achieve real time let alone the higher resolution for current needs In fact not all the pixels of the background change

repetitively To the pixels where less repetitive motion occurs such as the areas of ground houses and parking lot in the scene of Fig 1(a) only the first Fig 1(b) and second Fig 1(c) highest weighted Gaussians at each pixel are adequate to model the multi-possibilities for background and as the Fig 1(d) (e) (f) show most of third fourth and fifth highest weighted Gaussians were much less updated

except the car-passing area where should be foreground area Based on the analysis above Adaptively Adjustment Mechanism was proposed to reduce the number of the Gaussian distributions which offered less contribution to the multi-possibilities in the process of modeling background From the analysis and experiments different model number can be adopted for different pixels The update of weight mean and variance for our proposal is based on online EM algorithm [6] which allowed fast convergence on a stable background model for GMMproposed by P KaewTraKulPong et al The main procedure

of the adaptive Elimination Mechanism for GMM is as follows

1048697 E-step As the online EM algorithm does we begin our estimating of the Gaussian Mixture Model by

expected sufficient statistics which is called E-step Due to the unpredictable factor and possibilities for the complexity of background pixel and the first L frames is very important for Gaussian models to dominant background component and achieve stable adaptationswe keep the number of Gaussians models K for each pixel fixed during E-step Experiments also show these could provide a good estimate which helps to improve the accuracy for M-step process For initialization part

VI CONCLUSIONS

This paper presents a new background subtraction scheme based on the Gaussian Mixture Models We proposed an Adaptively Adjusted Mechanism to reduce less useful Gaussian distributions which contribute less to background modeling Through this method the execution speed for

video is improved Additionally an image segmentation based on edge information has been introduced By lessening the influence from illumination changes and excluding the shadow of moving object the

quality of segmentation mask is enhanced consequently Experimental results validate the gains in both processing speed and segmentation quality The proposed algorithm works very well in various complex background for both outdoor and indoor surveillance system

Histogram-based Foreground Object Extraction for Indoor

and Outdoor ScenesMandar Kulkarni

_

IPCV Lab Electrical Engg Department

Indian Institute of Technology Madras

Chennai India

mandareln40gmailcom

ABSTRACT

Extracting foreground objects is an important task in many video processinganalysis systems In this paper we propose a technique for foreground object extraction under static camera condition In our approach the spatial histogram of a single background image is modeled as Mixture of Gaussians and this model is updated after every few frames To extract the foreground input frames are compared with current background frame model and foreground pixels are classified according to intensity differences To mitigate the errors caused due to movement of the background objects (eg tree leaves in outdoor scenes) we also incorporate optical flow in an efficient manner We demonstrate performance of our approach on various indoor and outdoor scenes

Foreground extraction is an important task in many computervision applications In this paper we propose an method which models the histogram of an initial background frame by the mixtures of Gaussians Generally a natural background includes large objects such as trees road floor buildings walls etc each of which contains pixels with similar intensity values but whose intensities differ considerably from each other Hence the histogram of the background frame containing multiple objects is usually multi-modal and can be approximated by the Mixture of Gaussian The number of Gaussians is determined by the number of objects present in the background We also update the background histogram model at regular intervals to adapt to illumination variations over time We use the Expectation Maximization (EM) algorithm to find maximum likelihood parameters of every Gaussian

VI CONCLUSIONS

This paper presents a new background subtraction scheme based on the Gaussian Mixture Models We proposed an Adaptively Adjusted Mechanism to reduce less useful Gaussian distributions which contribute less to background modeling Through this method the execution speed for

video is improved Additionally an image segmentation based on edge information has been introduced By lessening the influence from illumination changes and excluding the shadow of moving object the

quality of segmentation mask is enhanced consequently Experimental results validate the gains in both processing speed and segmentation quality The proposed algorithm works very well in various complex background for both outdoor and indoor surveillance system

Histogram-based Foreground Object Extraction for Indoor

and Outdoor ScenesMandar Kulkarni

_

IPCV Lab Electrical Engg Department

Indian Institute of Technology Madras

Chennai India

mandareln40gmailcom

ABSTRACT

Extracting foreground objects is an important task in many video processinganalysis systems In this paper we propose a technique for foreground object extraction under static camera condition In our approach the spatial histogram of a single background image is modeled as Mixture of Gaussians and this model is updated after every few frames To extract the foreground input frames are compared with current background frame model and foreground pixels are classified according to intensity differences To mitigate the errors caused due to movement of the background objects (eg tree leaves in outdoor scenes) we also incorporate optical flow in an efficient manner We demonstrate performance of our approach on various indoor and outdoor scenes

Foreground extraction is an important task in many computervision applications In this paper we propose an method which models the histogram of an initial background frame by the mixtures of Gaussians Generally a natural background includes large objects such as trees road floor buildings walls etc each of which contains pixels with similar intensity values but whose intensities differ considerably from each other Hence the histogram of the background frame containing multiple objects is usually multi-modal and can be approximated by the Mixture of Gaussian The number of Gaussians is determined by the number of objects present in the background We also update the background histogram model at regular intervals to adapt to illumination variations over time We use the Expectation Maximization (EM) algorithm to find maximum likelihood parameters of every Gaussian

quality of segmentation mask is enhanced consequently Experimental results validate the gains in both processing speed and segmentation quality The proposed algorithm works very well in various complex background for both outdoor and indoor surveillance system

Histogram-based Foreground Object Extraction for Indoor

and Outdoor ScenesMandar Kulkarni

_

IPCV Lab Electrical Engg Department

Indian Institute of Technology Madras

Chennai India

mandareln40gmailcom

ABSTRACT

Extracting foreground objects is an important task in many video processinganalysis systems In this paper we propose a technique for foreground object extraction under static camera condition In our approach the spatial histogram of a single background image is modeled as Mixture of Gaussians and this model is updated after every few frames To extract the foreground input frames are compared with current background frame model and foreground pixels are classified according to intensity differences To mitigate the errors caused due to movement of the background objects (eg tree leaves in outdoor scenes) we also incorporate optical flow in an efficient manner We demonstrate performance of our approach on various indoor and outdoor scenes

Foreground extraction is an important task in many computervision applications In this paper we propose an method which models the histogram of an initial background frame by the mixtures of Gaussians Generally a natural background includes large objects such as trees road floor buildings walls etc each of which contains pixels with similar intensity values but whose intensities differ considerably from each other Hence the histogram of the background frame containing multiple objects is usually multi-modal and can be approximated by the Mixture of Gaussian The number of Gaussians is determined by the number of objects present in the background We also update the background histogram model at regular intervals to adapt to illumination variations over time We use the Expectation Maximization (EM) algorithm to find maximum likelihood parameters of every Gaussian

component To detect the foreground objects we compare input frame with the current background histogram model Pixels showing higher intensity deviations than background pixels are classified as foreground objects The threshold for foreground classification is computed from the current background

model We also account for the fact that if a classified foreground object remains stationary for long time its corresponding pixels are re-classified as background To improve the results under significant background motion we also incorporate optical flow efficiently in our framework We provide various qualitative and quantitative results on indoor and outdoor scenes An illustration of our approach is shown in Fig 1 Note the different intensities of the background objects such as the road building etc These differences show up in the multi-modal histogram (Fig1 (b)) where blue line indicates histogram of background frame and red line indicates Gaus sian approximation of the histogram The foreground extraction results for the scene are shown in Figs1 (df) for the corresponding frames in Figs1 (ce)