Automatic Face Feature Localization for Face Recognition Christopher I. Fallin honors thesis defense: May 1, 2009 advisor: Dr. Patrick J. Flynn May 1,

Automatic Face Feature Localization for Face Recognition

Christopher I. Fallinhonors thesis defense: May 1, 2009

advisor: Dr. Patrick J. Flynn

May 1, 2009 1Chris Fallin - thesis defense

Outline

• Face recognition– Methods of evaluation

• Elastic Bunch Graph Matching– Gabor Jets– Bunch Graphs and Feature Localization

• My Contributions: Automatic Fiducial Points– Information Content model– Fiducial point placement– results

Face Recognition


• Subfield of biometrics:– life (bio)– measure (metric)– Extract identifying

information from measures of human traits

• Face recognition: digital images of face

• 2D, 3D, infrared, multimodal, …

http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaces.jpg

May 1, 2009 Chris Fallin - thesis defense 4

Image Set

Face Recognition Evaluation

May 1, 2009 Chris Fallin - thesis defense 5http://en.wikipedia.org/wiki/File:Roc-general.png – used under terms of GNU FDL

System Decision

ROC curvesY N

Actual

Y

N

http://en.wikipedia.org/wiki/File:Roc-general.png


EER = 11.1%

Rank-one Score


Gallery A B C D E F GA 0.89 0.70 0.10 0.52 0.34 0.48 0.37B 0.70 0.73 0.45 0.82 0.12 0.43 0.44

Probe C 0.10 0.45 0.92 0.89 0.23 0.82 0.13D 0.52 0.82 0.89 0.56 0.20 0.38 0.14E 0.34 0.12 0.23 0.20 0.82 0.52 0.23F 0.48 0.43 0.82 0.38 0.52 0.84 0.11G 0.37 0.44 0.13 0.14 0.23 0.11 0.99

Rank-one: 5/7 = 71.4%

Elastic Bunch Graph Matching (EBGM)

• Wiskott et al., USC/Bochum, mid-90s• Basis of ZN-Face, successful commercial system• We use Face Identity Evaluation System, from

Colorado State• Face features represented by Gabor filter

responses• Features are localized

– Fit an elastic graph onto the features by localization: local optimization problems


A Face Graph


[Wiskott99]

Gabor Jets

• Vector of filter responses to 40 Gabor kernels– 5 wavelengths– 8 orientations– Each is complex-valued

• Gabor jets capture information well: Gokberk et al. get 91% rank-one with fixed grid– On FERET: 78.5% max, with

12 grid points


Bunch Graphs

• Each feature has a bunch of canonical jets

• Represents typical features

• Best-match at each feature point for novel images

May 1, 2009Chris Fallin - thesis defense

11

[Wiskott99]

Feature Localization

• Initial alignment: eye locations known a-priori• Overlay bunch graph with average edge

lengths• Take Gabor jets; pick best match in each

bunch• Localize based on displacement estimation

(local optimization problem)


The Idea: Automatic Fiducial Point Placement

• Bunch graph training requires manual fiducial point placement– 70 images, 25 points

• Why not statistically determine optimal features to match on?

• We can align/normalize all faces and take some statistical measure at each point in “face space” to determine goodness

• Replaces training step; back-end algorithm is identical


Related Work

• Gokberk et al.: choosing fiducial points with genetic algorithms– But their chosen points are global– Same goal as our system, excluding prelocalization

• Salient Points– Wavelet-based approach to image retrieval– Choras et al., 2006: similar approach with

goodness function, but no EBGM


Information Content: Variance Model

• Compute goodness function over face-space

• Inter-subject variance over intra-subject variance

• Self-normalizing unitless measure

• Requires multiple images per subject


Computing the goodness function

• FRGC: 5404 images – 700 MB, 128x128 grayscale (7 GB before normalization)

• Each pixel: 12 seconds, on fast Athlon 64• Split into 128 Condor jobs

– Each pixel is independent: easy• Pre-normalize image set, dump to fast-loading

binary format (single file)• Run Condor jobs: three hours• Post-processing to reassemble results


Fiducial Point Placement• Random placement with

probability density• Compute gradient of

goodness function• Probability is product of

gradient and goodness• Place points sequentially,

decay probability around points exponentially

• Mirror-point constraint: mirror placements across centerline, or snap to center


Prelocalization: Pseudo-Bunches

• Displacement estimation requires canonical feature jet from bunch

• We can’t provide this if we have no knowledge of feature

• Solution: fake a jet bunch– Make educated guess with K-means clustering on

jets from all images at given point• Then, run displacement estimation to

prelocalize points on each image


Results

• Competitive with original, manual points– In both cases, automatic

training points yield only ~1% performance drop

– With no human training!

• Prelocalization did not work as intended

• Success without this suggested by Gokberk’s results


FERET

FRGC

ROC curves

FERET FRGC


(EER = 11.4%)(orig: 11.1%)

(EER = 31.4%)(orig: 34.8%)

Prelocalization: causes for failure

• Poor pseudo-bunch clustering: K-means often found optimal clustering at self-imposed cap of N/10 clusters– Likely because initial jets are too far off

• Naïve localization: single-step– Bolme thesis compares several optimization

algorithms• Average displacement of 2.628 pixels: larger

than 2.021 pixels in manual points


Future Work

• More sophisticated prelocalization• Look at pseudo-bunch statistics to determine

failure mode in more detail• Look at per-fiducial point statistics to

determine where performance is weak• Investigate: are manual pts a theoretical limit,

or can we exceed them?• Try new image classes – test claim of

genericism


Questions?

• Email [email protected]• Full thesis and source code will be posted

online: http://c1f.net/research/mark5/


mailto:[email protected]

http://c1f.net/research/mark5/

Distance Metrics on Jets

• Phase-insensitive: magnitude only– Selects best jet in bunch

• Phase-sensitive– Can solve for

displacement vector: basis of localization

• Displacement estimation


Documents

Automatic Face Feature Localization for Face Recognition Christopher I. Fallin honors thesis defense: May 1, 2009 advisor: Dr. Patrick J. Flynn May 1,