Matthias Wimmer, Sylvia Pietzsch, Freek Stulp and Bernd Radig

Chair for Image Understanding

Institute for Computer Science

Technische Universität München

matthias.wimmer@cs.tum.edu

Learning Robust Objective Functions with Application to Face Model Fitting

Christoph Mayer

11.06.2007 2/13Technische Universität München

Christoph Mayer

Facial Expression Recognition

Natural Human-Computer Interaction

tactile channel

visual channel

audatory channel

olfactory channel

gustatory channel

auditory channel

visual channel

tactile channel

olfactory channel

gustatory channel

11.06.2007 3/13Technische Universität München

Christoph Mayer

Model-based Image Interpretation

Objective Function Calculates how well a parameterized model matches an image.

The model contains a parameter

vector p hat represents the model’s

configuration.

Fitting Algorithm

Searches for the model that matches the image best

by minimizing the objective function.

11.06.2007 4/13Technische Universität München

Christoph Mayer

Ideal Objective Functions

P1: Correctness Property:

The global minimum corresponds to the best model fit.

P2: Uni-modality Property:

The objective function has no local extrema.

¬ P1 P1

¬P2

P2

11.06.2007 5/13Technische Universität München

Christoph Mayer

Introducing Objective Functions

a) image b) along perpendicular c) edge values d) designed objective function e) ideal objective function f) training samples g) learned objective function

11.06.2007 6/13Technische Universität München

Christoph Mayer

Traditional Approach

Shortcomings: Requires domain knowledge. Based on designer’s intuition. Time-consuming.

Manually design the

objective function

Manually evaluate on test images

designed objective function

good

not good

11.06.2007 7/13Technische Universität München

Christoph Mayer

Learning the Objective Function (Step 1)Manually annotate images with ideal parameterization

Ideal objective function

11.06.2007 8/13Technische Universität München

Christoph Mayer

Learning the Objective Function (Step 2)Manually annotate images with idealparameterization

Ideal objective function

Automatically generate further

image annotations

result = 0 result = 0.2result = 0.3

11.06.2007 9/13Technische Universität München

Christoph Mayer

Learning the Objective Function (Step 3)

Number of features: 6 styles · 3 sizes · 25 locations = 450

Locations

Styles

Sizes

Manually annotate images with idealparameterization

Ideal objective function

Automatically generate further

image annotations

Manually specifya set of

image features

11.06.2007 10/13Technische Universität München

Christoph Mayer

Learning the Objective Function (Step 4+5)

Automatically obtain calculation rules of objective function.Mapping of feature values to the value of the objective function.

Machine learning by Model Trees.Select the most relevant features.

Manually annotate images with idealparameterization

Ideal objective function

Automatically generate further

image annotations

Manually specifya set of

image features

Automatically generate

training data

learned objective function

11.06.2007 11/13Technische Universität München

Christoph Mayer

Evaluation of local objective functions

Designed Learned

Evaluation of displacement and face turning.

Weak global minimum using the designed objective function.

Strong global minimum using the learned objective function.

11.06.2007 12/13Technische Universität München

Christoph Mayer

Evaluation of the global function 95% of models

are located at 0.12 using the learned objective function.

95% of models are located at 0.16 using a state-of-the-art approach.

11.06.2007 13/13Technische Universität München

Christoph Mayer

Conclusion

Evaluation in natural dialog situations.

Application for Daimler-Crysler in car-driving

situations.

Integration of a three-dimensional face model.

11.06.2007 14/13Technische Universität München

Christoph Mayer

Thank you!