4EyesFace-Realtime face detection, tracking, alignment and recognition

Changbo Hu, Rogerio Feris and Matthew Turk

Overview

Introduction Face Detection and Pose tracking Face Alignment Face Recognition Conclusions

Introduction

Detection Pose tracking Alignment Recognition

Introduction

Why this is a difficult problem? Facial Expressions, Illumination Changes, Pose,

etc. Object

Develop a fully automatic system, suitable for real-time applications to locate and track human faces, then to align and recognize the face.

Evaluate it on a large dataset.

Face Detection

[Viola and Jones, 2001]

Simple features, which can be computed very fast.

A variant of Adaboost is used both to select the features and to train the classifier.

Classifiers are combined in a “cascade” which allows background regions of the image to be quickly discarded.

Face detection

Pose tracking

Based on Kentaro Toyama’s IFA framework

Face Alignment

Active Appearance Model (AAM)

Statistical Shape Model (PCA)

Statistical Texture Model (PCA)

Face alignment

Problem: Partial Occlusion

Active Wavelet Networks (AWN) (on BMVC’03) Main idea: Replace AAM texture model by a

wavelet network

Face Alignment

Similar performance to AAM in images under normal conditions.

More robust against partial occlusions.

Face Alignment

Using 9 wavelets, the system requires only 3 ms per iteration. In general, at most 10 iterations are sufficiently for good convergence (PIV 1.6Ghz).

Multi-View Face Alignment

View selection by pose tracker

Multi-View Face Alignment

Face recognition online recognition

HMM based face recognition

Face recognition

Large dataset evaluation FERET DataSet 1196 different individuals1196 different individuals With ground truth of eye cornersWith ground truth of eye corners

Face recognition

Face recognition

Face Recognition

Face Recognition

Conclusion We develop a system to do human

face detection, tracking, alignment and recognition

In this system, we invented new methods AWN and extent to multi-view AWN

We implement the related detection and pose tracking

Evaluate our method on large dataset