Automatic Pose Estimation of 3D Facial Models Yi Sun and Lijun Yin Department of Computer Science State University of New York at Binghamton Binghamton,

Automatic Pose Estimation Automatic Pose Estimation of 3D Facial Modelsof 3D Facial Models

Yi Sun and Lijun Yin

Department of Computer Science State University of New York at Binghamton

Binghamton, New York, 13902 USA

19th International Conference on Pattern Recognition December 8th, 2008

December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn

22

IntroductionIntroduction Pose estimation plays an essential role in

many computer vision applications, such as human computer interaction (HCI), monitoring driver attentiveness, face recognition, and automatic model editing.

2D image/Videos based. Active infra-red illumination based. 3D facial models based


33

MotivationMotivation Problem? Invariant to illumination, robust

to pose variations, deal with different expressions, none-facial outliers, noise, partial facial data missing, etc.

Geometric surface representation Identify facial features (inner eye corners,

nose tip). Machine learning plus structure based to

estimate pose orientation.


44

3D facial expressions database – 3D facial expressions database – samplessamples

First row: raw faces; second row: clean faces

First row: 3D textured faces; second row: 3D shaded faces. From left to right: neutral, anger, disgust, fear, happy, sad, and surprise.


55

ObservationObservation


66

System frameworkSystem framework


77

Candidate eye inner cornerCandidate eye inner corner Use decision tree method to determine the Use decision tree method to determine the

proper thresholdproper threshold


88

Noise erase Noise erase (1)(1) Erase candidate far away from othersErase candidate far away from others Erase points having limited number of Erase points having limited number of

neighboring candidates neighboring candidates


99

Inner eye corner clustering Inner eye corner clustering Apply 2-means clustering approach to find Apply 2-means clustering approach to find

the two inner eye cornersthe two inner eye corners


1010

Sparse candidate Sparse candidate eliminationelimination


1111

Noise tip determination (1)Noise tip determination (1) Fit a flat surface (reference plane)

toto two clusters by solving the optimization problem::

0=+×+×+× dzcybxa

( )∑n

i iD1=

2min

Where, 222 ++

+×+×+×=

cba

dzcybxaD iii

i

n is the number of candidates in two clusters


1212

Noise tip determination (2)Noise tip determination (2)


1313

Symmetry planeSymmetry plane


1414

Automatic frontal view Automatic frontal view transformtransform


1515

ExperimentsExperiments

Tested with 2500 range models from Tested with 2500 range models from BU-3DFE databaseBU-3DFE database

Tested with both raw facial models Tested with both raw facial models and clean facial modelsand clean facial models

Estimated pose orientation less than Estimated pose orientation less than 5 degrees - correct5 degrees - correct

Raw data: 92.1%Raw data: 92.1%Clean data: 96.4%Clean data: 96.4%


1616

Sample results (1)Sample results (1) Same subject, different poses


1717

Sample results (2)Sample results (2)

Different subjects, different poses


1818


Same subject with clean and raw modelsSame subject with clean and raw models


1919


Different expressionsDifferent expressions


2020

ConclusionConclusion

Propose a fully automatic 3D face pose estimation approach.

Based on 3D wire-frame model. Feasible with respect to various subjects,

large pose variations, different expressions, and data with noise/none-facial outliers.


2121

AcknowledgementAcknowledgement

This material is based upon the work supported in part by the National Science Foundation under grants IIS-0414029 and IIS-0541044, and the NYSTAR's James D. Watson Investigator Program.


2222

Thank you!Thank you!

Documents

Automatic Pose Estimation of 3D Facial Models Yi Sun and Lijun Yin Department of Computer Science State University of New York at Binghamton Binghamton,