A neural approach to extract foreground from human movement imagesS.Conforto, M.Schmid, A.Neri, T.D’Alessio

Compute Method and Programs in Biomedicine 82(2006) 73-80

Che-Wei Sung 2007/12/24

Outline

Introduction Materials and methods

Subtraction techniques Neural approach

Qualitative evaluation of results Objective evaluation of results Conclusions

Introduction

The capture of human movement is a hot topic for surveillance, control and analysis.

In the framework of human movement analysis often consists of separating the moving subject (i.e. foreground) from the background by techniques based on temporal or spatial.

Introduction

Temporal data can be used in two different ways, subtraction and flow, while spatial techniques is applying markers on foreground.

Mixed approaches have been presented, but none can be considered as outperforming in general terms.

The work in this paper is to development of a markerless capture system for movement analysis application by making the ANN “learn” the background.

Materials and methods

The moving subject is detected by analyzing the differences between the background scene.

, corresponding to the background image

,represents the generic s-th image frame extracted from the video sequence gathering the moving subject over the background scene.

( )smI P

( )bI P

Materials and methods - Subtraction1. Compute the image difference

2. For each row of , calculate the vectors of mean value , and standard deviation

3. Determine the 3D-classification interval , if a pixel lies inside the

domain , it is classified as background,

vice versa as foreground.4. Detect the largest connected area that considered

as actual foreground.

( ) { ( ), ( ), ( )} ( ) ( )s R G B sdiff diff diff diff b mI P I P I P I P I P I P

( )sdiffI P

{ , , }R G Brow row row rowμ μ μ μ{ , , }R G B

row row row rowσ σ σ σ

{ , , }R R G G B Brow row row row row row μ σ μ σ μ σ

{ , ,R R R R R G G G G Grow row row row row row row rowP Pμ -σ < <μ +σ μ -σ < <μ +σ

}B B B B Brow row row rowPμ -σ < <μ +σ

Materials and methods - ANN

Neural network makes use of a Kohonen map, composed of (8×8) neurons

Materials and methods - ANN

In this work, background image is partitioned into blocks of (8×8) pixels, and arranged in a mono-dimensional vector composed of

H = (64×3) =192 components for training data.

Materials and methods - ANN

Assume the image is subdivided into B blocks of size (8×8), the training input vector Vb={b=1,2…B} and the size of each synaptic weight vector is randomly initialized in [0,1], where h=1,2…H

hWij

Materials and methods - Training1. One input vector Vb is randomly extracted from the

training set, and feeds the network.

2. In each neuron nij, the distance dij,b(k) between Vb

and (k) is calculated:

3. The best match neuron nBM(k) is defined as the nij whose corresponding vector (k) is at the minimum distance from Vb.

Wij

Wij

2,

1

( ) ( ( ) )H

h hij b ij b

h

d k W k V

Materials and methods - Training4. The weight vectors are updated by using typical Ko

honen neighborhood procedure.

where

( ) ( )( ( ))( 1) {

( )

h h hij ij b ijh

ij hij j

W k k V W kW k

W k n

λ ij BMn N

ij BMn N

ij BMn n

ij BMn N

ij BMn N

Materials and methods - Training

The training has been considered as complete when, for the 98% of training samples, the association between each Vb and the corresponding best match neuron is not altered

Materials and methods - Testing1. undergoes Data Shaping, creating a set of

vectors .

2. For each block, the best match neuron is identified by considering the minimum Euclidean distance criterion.

( )smI P

bS

2,

1

( )H

h hBM b BM b

h

d W S

Materials and methods - Testing3. is used to build up a distance matrix, whose el

ements are rearranged respecting the spatial of

, where each element occupies the position of block.

4. For each row of distance matrix, the mean value

and the standard deviation are calculated.

Blocks with corresponding distance values outside the range are considered as foreground.

5. A segmentation mask is built up by marking pixels with 0 for background, 1 for foreground.

,BM bd

( )smI P

rowμ rowσ

row rowμ σ

Qualitative evaluation of results The proposed algorithm have been applied to

analyze human body movement during three motor tasks: gait, pitching a ball and standing up from a chair.

The training of Kohonen’s map has met convergence after around 90000 presentations of background blocks.

Qualitative evaluation of results

Qualitative evaluation of results

Objective evaluation of results quality_indexs = 0.3shape_regs +

0.33temp_stabs + 0.37contrasts

shape_regs: the regularity of segmented object shape.

temp_stabs: the stability along the video sequence of extracted object.

contrasts: the contrast between the inside and the outside of the object evaluated along the border.

Objective evaluation of results

Conclusions

The work proposes a new unsupervised approach for foreground extraction in human movement images based on ANN and the presented results demonstrate it is suitable.