Topic 5. Human Faces

TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu

Topic 5. Human Faces

Human face is extensively studied in vision. Depending on the applications, there are a long list of tasks [5]:1. Detection and Recognition: Face detection (finding all faces in a picture), facial feature detection (eyes, lips, …), Face localization (detecting a single face in image), Face recognition or identification (from a database, classification) Face authentication (verifying claim, bank id), Age/gender recognition, Face tracking (location and pose over time) Facical expression recognition (affective states), aesthetic study.

2. Modeling and Photorealistic Synthesis: Appearance models, deformable templates, lighting models, facial action units, face hallucination (high resolution from low resolution), pose adjustment, image editing (removing wrinkles, eye glass, red-eye etc.)

3. Artistic rendering Sketch, portrait, caricature, cartoon, painting, …


Face Image Databases

The CMU Rowley dataset


Face Image Databases

The CMU Schneidrman and Kanade Dataset


References.

1. P. Hallinan, G. Gordon, A. Yuille, P. Giblin, and D. Mumford, 2D and 3D Patterns of the Face, A.K. Peters, Ltd. Book chapters 2-4. (handouts).

2. D.H. Ballard, "Generaling the Hough transform to detect arbitrary shapes", (in handbook). 3. P. Viola and M. Jones, "Robust Real Time Object Detection", 4. F. Fleuret and D. Geman, " Coarse-to-fine face detection", IJCV 41(1/2),2001. 5. M.H. Yang, D. Kriegman, N. Ahuja, “Detecting faces in images, a survey”, PAMI

vol.24,no.1, January, 2002.

6 T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", ECCV 1998 7. C. Liu, S. C. Zhu, and H. Y. Shum, "Learning inhomogeneous Gibbs models of faces by

minimax entropy", ICCV 2001.

8. Y. Tian, T. Kanade, and J. Cohn, "Recognizing action units for facial expression analysis" PAMI, Feb, 2001. 9. H. Chen, Y. Q. Xu, H. Y. Shum, S. C. Zhu, and N. N. Zhen, "Example-based facial sketch

generation with non-parametric sampling", ICCV 2001.


Outline

We proceed in three steps:

• A survey on face detection and recognition techniques

2. Mathematical models of face images

3. Face synthesis: photorealistic and non-photorealistic.


Face Detection Methods [5]


Face vs non-face Clsutering

6 clusters in a 19 x19 space (Sung and Poggio)


Distance Measure

D1

D2

For each input image, it measures two distances for each cluster center: D1 is the Mahalanobis distance and D2 is the Euclidean distance.

Thus Sung and poggio have 2 x 6 x 2 = 24 features for classification in a multiple layer perceptron.


Deformable Face Template

Deformable face template by Fishler and Elschlager 1973. M. Fishler and R. Elschlager, “The representation and matching of pictorial structures”,

IEEE Trans. on Computer. Vol.C-22, 67-92, 1973.


Local Deformation and Global Transform

Geometric variations of faces: (Hallinan, Yuille, Mumford et al)


Deformable Model of Facial Features

Eye template using parabolic curves by Yuille et al 1989-92. A.L.Yuille, D. Cohen, and P.Hallinan, “Feature extraction from faces using deformable templates”, CVPR 89, IJCV 92.

We can derive meaningful diffusion equations from the energy functionals.


Upper Face Action Units


Lower Face Action Units


Templates for Various States


Templates for Various States


Features for Action Unit Recognition


Classification from Feature Vector


Recognition Rate


Apparence Model: Landmarks on a face

400 images each labeled with 122 points.


Eigen-vectors for Geometry and Photometry


Apparence Model


Face Localization and Recognition


A Linear HMM Model for Face


Face Detection


Sample of the 4D space


Multi-scale Detection


Edge Features


Decision Tree


Examples of Decision Trees


Bounds Analysis


Some Examples


Face Prior Learning: Experimental Details

• 83 key points defined on face

• 720 individuals with all kinds of types

• Dimension reduced to 33 by PCA

• 40000 samples drawn by the inhomogeneous Gibbs sampler in each Monte Carlo integration

• 50 features pursuit

• Total runtime: about 5 days on a PIII 667, 256MB PC


Obs & Syn Samples (1)

Observed

faces

Synthesized faces without any features


Synthesis Samples

Synthesized faces with 20 features



Synthesis Samples




50 Observed Histograms


50 Synthesized Histograms

Documents

Topic 5. Human Faces