View
215
Download
1
Category
Tags:
Preview:
Citation preview
Automatic Pose Estimation Automatic Pose Estimation of 3D Facial Modelsof 3D Facial Models
Yi Sun and Lijun Yin
Department of Computer Science State University of New York at Binghamton
Binghamton, New York, 13902 USA
19th International Conference on Pattern Recognition December 8th, 2008
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
22
IntroductionIntroduction Pose estimation plays an essential role in
many computer vision applications, such as human computer interaction (HCI), monitoring driver attentiveness, face recognition, and automatic model editing.
2D image/Videos based. Active infra-red illumination based. 3D facial models based
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
33
MotivationMotivation Problem? Invariant to illumination, robust
to pose variations, deal with different expressions, none-facial outliers, noise, partial facial data missing, etc.
Geometric surface representation Identify facial features (inner eye corners,
nose tip). Machine learning plus structure based to
estimate pose orientation.
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
44
3D facial expressions database – 3D facial expressions database – samplessamples
First row: raw faces; second row: clean faces
First row: 3D textured faces; second row: 3D shaded faces. From left to right: neutral, anger, disgust, fear, happy, sad, and surprise.
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
55
ObservationObservation
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
66
System frameworkSystem framework
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
77
Candidate eye inner cornerCandidate eye inner corner Use decision tree method to determine the Use decision tree method to determine the
proper thresholdproper threshold
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
88
Noise erase Noise erase (1)(1) Erase candidate far away from othersErase candidate far away from others Erase points having limited number of Erase points having limited number of
neighboring candidates neighboring candidates
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
99
Inner eye corner clustering Inner eye corner clustering Apply 2-means clustering approach to find Apply 2-means clustering approach to find
the two inner eye cornersthe two inner eye corners
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1010
Sparse candidate Sparse candidate eliminationelimination
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1111
Noise tip determination (1)Noise tip determination (1) Fit a flat surface (reference plane)
toto two clusters by solving the optimization problem::
0=+×+×+× dzcybxa
( )∑n
i iD1=
2min
Where, 222 ++
+×+×+×=
cba
dzcybxaD iii
i
n is the number of candidates in two clusters
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1212
Noise tip determination (2)Noise tip determination (2)
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1313
Symmetry planeSymmetry plane
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1414
Automatic frontal view Automatic frontal view transformtransform
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1515
ExperimentsExperiments
Tested with 2500 range models from Tested with 2500 range models from BU-3DFE databaseBU-3DFE database
Tested with both raw facial models Tested with both raw facial models and clean facial modelsand clean facial models
Estimated pose orientation less than Estimated pose orientation less than 5 degrees - correct5 degrees - correct
Raw data: 92.1%Raw data: 92.1%Clean data: 96.4%Clean data: 96.4%
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1616
Sample results (1)Sample results (1) Same subject, different poses
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1717
Sample results (2)Sample results (2)
Different subjects, different poses
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1818
Sample results (3)Sample results (3)
Same subject with clean and raw modelsSame subject with clean and raw models
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
1919
Sample results (4)Sample results (4)
Different expressionsDifferent expressions
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
2020
ConclusionConclusion
Propose a fully automatic 3D face pose estimation approach.
Based on 3D wire-frame model. Feasible with respect to various subjects,
large pose variations, different expressions, and data with noise/none-facial outliers.
December 8th, 2008December 8th, 2008 19th International Conference on Pattern Recognitio19th International Conference on Pattern Recognitionn
2121
AcknowledgementAcknowledgement
This material is based upon the work supported in part by the National Science Foundation under grants IIS-0414029 and IIS-0541044, and the NYSTAR's James D. Watson Investigator Program.
Recommended