3D Face Modeling Michaël De Smet. Topics to Discuss 3D Morphable Models 3D face reconstruction Face...

3D Face Modeling

Michaël De Smet

Topics to Discuss

3D Morphable Models 3D face reconstruction Face recognition Lip synchronization

Topics to Discuss

3D Morphable Models

Statistical model of shape and texture Derived from laser scans

USF DARPA HumanID 3D Face Database Processing

Hole filling Surface smoothing Albedo estimation Dense correspondence

3D Morphable Models

Topics to Discuss

3D Face Reconstruction

Fitting the 3DMM to one or more images of the same face

•Scale•Rotation•Translation•Illumination

•Shape•Texture

Optimization problem with > 100 parameters

Feature points Feature alignment Fitting result

Dealing with Occlusions

Without occlusionhandling

With occlusionhandling

Topics to Discuss

Face RecognitionFit the 3DMM to an image of an unknown face

•Scale•Rotation•Translation•Illumination

•Shape•Texture

Compare to database Recognition

Face Recognition

In controlled settings, almost perfect recognition is possible

Pose 1100.0%

Pose 2100.0%

Pose 3N/A

Pose 4100.0%

Pose 595.7%

Training view

Face Recognition

Uncontrolled environments are challenging Face orientation unknown Difficult illumination Facial expressions Occlusions Low resolution Motion blur …

Face Recognition

European parliament video: 21 persons, 86% correct

Face Recognition

VRT news broadcasts: 12 persons

Face Recognition

VRT news broadcasts: 12 persons

82.3% correctrecognition

Topics to Discuss

Lip Synchronization Speech driven animation Texture based, i.e. shape is fixed Strategy:

Extract 3D model of speaker’s face Track rigid motion of the face in video Extract texture for each frame Compute PCA model of texture Train ANN to link phonemes and PCA

coefficients (visemes)

System Overview

AutomaticAutomaticPhonePhone

RecognitionRecognition

NeuralNeuralNetworkNetwork

FaceFaceSynthesisSynthesis

SpeechSpeechfeature vectorsfeature vectors

FacialFacialfeature vectorsfeature vectors

Training Setup

AutomaticAutomaticPhonePhone

RecognitionRecognition

NeuralNeuralNetworkNetworkTrainingTraining

FaceFaceAnalysisAnalysis

SpeechSpeechfeature vectorsfeature vectors

FacialFacialfeature vectorsfeature vectors

Video Processing

3D face model acquisition Rigid motion tracking Normalized texture extraction Texture feature extraction (PCA)

Conclusion

3DMMs are a very powerful tool for face modeling

Many applications in computer vision and computer graphics

3D Face Modeling Michaël De Smet. Topics to Discuss 3D Morphable Models 3D face reconstruction Face...

Documents

Face Identification by Fitting a 3D Morphable Model using ...mi.informatik.uni-siegen.de/publications/romdhani_eccv02.pdf · Face Identiﬁcation by Fitting a 3D Morphable Model using

Gaussian Mixture 3D Morphable Face Modelepubs.surrey.ac.uk/842230/1/2017_GM3DMM.pdfGaussian Mixture 3D Morphable Face Model Paul Koppen a, Zhen-Hua Fenga,b,, Josef Kittler , Muhammad

Neural 3D Morphable Models: Spiral Convolutional Networks ......Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation Giorgos

3D face reconstruction from a single 2D imagecs6240/past-projects/3Dface/3Dface.pdf · Morphable Model – Problem Definition • Inputs – : 3D surface point in the reference face

A Morphable Model For The Synthesis Of 3D Faces · morphable 3D face model is a consequent extension of the interpo-lation technique between face geometries, as introduced by Parke

Face recognition based on fitting a 3D morphable …bebis/CS790Q/PaperPresentations/...Face Recognition Based on Fitting a 3D Morphable Model Volker Blanz and Thomas Vetter, Member,

Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Face De-Occlusion Using 3D Morphable Model and ......Face De-occlusion using 3D Morphable Model and Generative Adversarial Network Xiaowei Yuan and In Kyu Park {xiaoweichn@qq.com pik@inha.ac.kr}

A 3D Morphable Model Learnt From 10,000 Facesopenaccess.thecvf.com/content_cvpr_2016/papers/Booth_A_3... · 2017-04-04 · A 3D Morphable Model learnt from 10,000 faces James Booth⋆

A 3D Morphable Model of Craniofacial Shape and Texture ...openaccess.thecvf.com/content_ICCV_2017/papers/Dai... · A 3D Morphable Model of Craniofacial Shape and Texture Variation

Face Poser: Interactive Modeling of 3D Facial Expressions ...graphics.cs.cmu.edu/projects/face_poser/3dface_sca07_final.pdf · BBPV03] built a morphable model from 3D scans via Prin-cipal

Face Recognition Using a Uniﬁed 3D Morphable Modeludrc.eng.ed.ac.uk/sites/udrc.eng.ed.ac.uk/files...face recognition performance on Multi-PIE and AR databases. We also present baseline

Parametric T-Spline Face Morphable Model for Detailed

Dense 3D Face Decoding over 2500FPS: Joint …...Linear 3D Morphable Models. For the past two decades, the method of choice for representing and generating 3D faces was Principal Component

3D Morphable Face Modelshelper.ipam.ucla.edu/publications/glws1/glws1_15712.pdf · 2019. 4. 8. · Probabilistic 3D Morphable Model 5 Probabilistic PCA Gerig et al (2018). Morphable

Neural 3D Morphable Models: Spiral Convolutional Networks ... · 3D Morphable Model [5] and the COMA autoencoder [39], as well other graph convolutional operators, including the initial

MVF-Net: Multi-View 3D Face Morphable Model …openaccess.thecvf.com/content_CVPR_2019/papers/Wu_MVF...MVF-Net: Multi-View 3D Face Morphable Model Regression Fanzi Wu2†∗ Linchao

A Biophysical 3D Morphable Model of Face Appearanceopenaccess.thecvf.com/content_ICCV_2017_workshops/papers/... · 2017-10-20 · A Biophysical 3D Morphable Model of Face Appearance

Occlusion-aware 3D Morphable Face Models - unibas.chgravis.dmi.unibas.ch/...Occlusion-aware_3DMMs.pdf · occlusion-aware 3d morphable face models 3 (a) target (b) z, trimmed 70% (c)

A 3D Morphable Model learnt from 10,000 facesaroussos/Booth_etal_3DMMlearnt... · 2016-04-27 · A 3D Morphable Model learnt from 10,000 faces James Booth?Anastasios Roussos Stefanos