Robust Video Surveillance for Fall Detection Based on Human Shape Deformation

Robust Video Surveillance for Fall Detection Based on Human Shape Deformation

Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline RousseauIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 5, MAY 2011

outline

Introduction Our System and data set Falls Characteristics

Shape deformation▪ mean matching cost▪ full Procrustes distance

Fall Detection Using GMM Experimental Results Conclusion

Introduction (1/2) Establish new healthcare systems to

ensure the safety of elderly people at home. Falls are one of the major risks for old

people living alone. Fall detection wearable sensor:

Accelerometers or help buttonsProblem:-forget to wear-unconscious after the fall-recharged regularly

Introduction (2/2) Computer vision systems has overcome

these problems. A camera provides a vast amount of

information on his/her environment▪ Monocular Systems▪ Bounding box[8]▪ Only placed sideways▪ Occluding objects

▪ Multi-Camera Systems▪ Auvinet et al.[17] reconstructed 3-D silhouette of an

elderly person ▪ Need to be calibrated▪ The video sequences need to be synchronize

Our System and data set (1/2)

Uncalibrated multi-camera system

Low-cost IP cameras, 30 frames/s, 720 × 480 pixels

Wide angle to cover all the room

Our System

and data set

Total of 75 different events , more than 12 min

Falls Characteristics

1. Lack of significative movement2. A lying position3. A person lying on the ground4. Vertical speed5. An impact shock6. Body shape change

Silhouette Edge Point Extraction The silhouette is extracted by a background

subtraction

N = 250 landmarks * Canny edge detector[25]

Matching Using Shape Context (1/2) Shape context[20] is a way of

describing shapes.

Matching cost for pair (pi, qj):

, K=5*12 bins

Matching Using Shape Context (2/2) Minimizing the total matching cost given a

permutation π (i)

Use the Hungarian algorithm[27] for bipartite matching Time complexity: O(n^3) Bad landmarks due to segmentation errors or partial occlusions ▪ Add dummy points (not easy to choose).▪ Match only the most reliable points in our implement (mini Cij = minj Cij)

mean matching cost: i j

bipartite graph

N∗: the total number of best matching points.

Procrustes analysis

Procrustes analysis [21] has been widely used to compare shapes. Detect abnormal shape deformation for fall

detection▪ Step1 : image registration(one translation, no rotation,

no scaling)▪ Step2: Compute full Procrustes distance for compare.

centered landmarks Zc :

1

11

kl

Z

Zc

two centered vectors : v = (v1, · · · , vk)w = (w1, · · ·,wk).

full Procrustes distance :

Fall feature

mean matching cost full Procrustes distance Consider 2 feature (F1, F2)

CfD

1) F1 representing the fall : F1 will high in case of fall

2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low

Fall Detection Using GMM Model normal activity data with a Gaussian Mixture

Model(GMM). GMM: weighted sum of Gaussian(normal) distributions

M : the number of components in the mixture P (j) : the mixing coefficients The jth Gaussian probability density function p (x | j)

▪ d is the dimensionality of the input space

expectation-maximization (EM)algorithm by maximizing the data likelihood

GMM Classifier : only tell normal or abnormal!

Training and test the dataset Leave-One-Out Cross-Validation

1. Divided the dataset into N video sequences2. One sequence is removed3. Training using the N − 1 remaining sequences

(falls are deleted)4. This sequence is classified with the resulting

GMM.5. Repeat N times6. Count the number of errors, classification

error rate

GMM Classifier Analysis

1. True Positives (TP): falls correctly detected;2. False Negatives (FN): falls not detected;3. False Positives (FP): normal activities detected

as a fall;4. True Negatives (TN): normal activities not

detected as a fall;5. Sensitivity: Se = TP/ (TP + FN);6. Specificity: Sp = TN/ (TN + FP);7. Accuracy: Ac = (TP+TN) / (TP+TN+FP+FN) ;8. Classification error rate: Er = (FN+FP) /

(TP+TN+FP+FN) .

Experimental Results Shape matching : C++ using the OpenCV

library [33] Fall detection : MATLAB using the NETLAB

toolbox [32] to perform the GMM classification.

The original video sequences frame : 30 frames/s 5 frames/s was sufficient to detect a fall Intel Core 2 Duo processor (2.4 GHz) The computational time of the shape matching

step is about 200 ms

Number of GMM Components

train a GMM with three components for our experiment.

Classification Results Normalize training data.

Detection threshold depends on the sensitivity.

Receiver operating characteristic (ROC) analysis

false positives

true positives

Ensemble Classifier

Simply majority vote on all cameras (>= 3 vote) In fig. 9 : error rate 10%2.7%

Comparative Study with Other 2-D Features (1/2)

Comparative Study with Other 2-D Features (2/2)

Occlusions and Other Difficulties

Conclusion

We presented a new GMM classification method to detect falls By analyzing human shape deformation

Robust to large occlusions and other segmentation difficulties