Face Processing System

Presented by: Harvest JangGroup meeting Fall 2002

OutlineIntroductionSystem Architecture

Pre-processingFace detectionFace tracking

ProblemsFuture work

IntroductionAn automatic system is needed to provide more human-computer interactive services to improve the life style

Security SystemInteractive application/gameRobot visualization

This is a challenging task for integrating different techniques

face detectionface trackingface recognition

System Architecture

Three main componentsPre-processingFace detectionFace tracking

Fig 1: System architecture

Pre-processing1. Transform to YCrCb color space2. Use ellipse color model to locate the

fresh color

Fig 2: 2D projection in the CrCb subspace (gray dots represent skin color samples and black dots represent non-skin tone color)

Pre-processing3. Perform

morphological operation to reduce noise

4. Skin segmentation to find face candidates

Fig 3: Pre-processing step (a) binary skin mask, (b) original images (c) binary skin mask after morphological operation and (d) Face candidates

Comparison on three face detection approach

Motivation for the comparisonUsing different infrastructure systems for detecting faces in a large imageOnly shows one result pair of detection rate and false alarm rate

Using experimental result to compareSparse Network of Winnows (SNoW)Support Vector Machines (SVM)Neural Network

Comparison on three face detection approach

Sparse Network of Winnows (SNoW)

Primitive features architecture

Winnow update rule

),()(256 yxIxwy

learner’sprediction

correctresponse

update action update name

1 0 wi: wi/ if xi1

wi unchanged if xi0

demotion step

0 1 wi: wi if xi1

wi unchanged if xi0

>1 fixed parameter

promotion step

Table 1: Winnow update rule

Comparison on three face detection approach

Support Vector Machines (SVM)Find a hyperplane that leaves the maximum margin between two classes which will have the smallest generalization errorThe margin is defined as the sum of the distances of the hyperplane from the closest point of the two classes

Neural NetworkBack-propagation methodx1

x2

xn

w1

w2

wn

x0 = 1w0

n

0iii xw

otherwise 1-

0 if 1 i

n

0ii

n21

xwxxxo ,,

otherwise 1-

0 if 1:notation Vector

xww ,xsgnxo

Fig 4: SVM

Fig 5: Neural Network

Comparison on three face detection approaches

CBCL face database from MITTraining set (2429 face pattern, 4548 non-face pattern with 19x19 pixel)Testing set (472 face pattern, 23573 non-face pattern with 19x19 pixel)

To better represent the detectability of each model, ROC curve is used to replace single point of criterion response

Comparison on three face detection approaches

Similar signal forSNoWNeural NetworkSVM model with linear kernel

SVM model with polynomial kernel

much stronger signalthe classification between face and non-face patterns is much better

Fig: 6: ROC curves of different face detection method

Comparison on three face detection approaches

In our system, success to detect face is more important than processing more windowsFace tracking method to help for tracking the detected faceSVM model with polynomial kernel of second degree is chosen.

(in sec) NN SNoW

SVM with linear

SVM with polynomial

Mean 35.57 63.60

65.22 61.13

S.T.D. 0.029 1.378

0.306 0.410Table 2: The average processing time of each model for testing 24045 images ten times

Face detectionTo detect different size of faces, the region is resized to various scalesA 19x19 search window is searching around the re-sized regionsHistogram equalization is performed to the search windowAfter the first presence of a face at the larger scale, face recognition using SVM classifier will be performed to retrieve the information of that person.

Face detection - Example

Histogram equalization

Transform to various scale

Apply a 19x19 search window

Face trackingWhy using face tracking

Face detection method has missing rate and false alarm rateThe missing rate and false alarm rate for losing an object will be reducedThe performance speed is much better than face detection method

Conditional Density Propagation – Condensation algorithm

Sampling-based tracking method

The posterior probability density at timestep t

using a set of N random samples, denoted as with weights There are three phases to compute the density iteratively at each time step t for the set of random samples to track the movement

Face tracking

)|( tt Zxp

},,1,{ )( NnS nt )(n

t

Face trackingSelection phase:

The element with high weights in the sethas a higher probability to be chosen to the predictive steps

Predictive phase:The sample set for the new time-step is generated by independent Brownian motionThe weights are approximately from the effective prior density for time-step t+1

)(ntS

)(1ntS

)(1nt

)|( 1 tt Zxp

tttttttt dxZxpxxpZxp )|(),|()|( 11

Face trackingMeasurement (Update) phase

Given in terms of likelihoodThe model is expressed as a histogram for face color HUE (HSV color space)Calculate the measurement

where is a normalization factor

)|()|()|( 111111 ttttttt ZxpxzpcZxp

)|( 11 tt xzp

1tc

Face tracking

Fig 7: One time step in the Condensation algorithm

Face trackingThe color input image is being masked by the binary skin mask

reduce the localization periodconverge less sensitive from the background noiseconverge to the face boundary much faster.

ProblemsThe face detection rate is still lowThe condensation algorithm doesn’t provide verification for the tracking object

Two adjacent object with similar color will cause problem

Future workImprove the accuracy of face detectionImplement the face tracking verification step using face recognition

Thank you!