Upload
myrrh
View
69
Download
0
Tags:
Embed Size (px)
DESCRIPTION
MITRE Corporation. Pose Correction for Automatic Facial Recognition. Team : Elliot Godzich , Dylan Marriner , Emily Myers-Stanhope, Emma Taborsky (PM), Heather Williams Liaisons : Josh Klontz ’10 and Mark Burge Advisor : Zachary Dodds. Automated Facial Recognition. - PowerPoint PPT Presentation
Citation preview
MITRE Corporation
Pose Correction for Automatic Facial Recognition
Team: Elliot Godzich, Dylan Marriner, Emily Myers-Stanhope, Emma Taborsky (PM), Heather Williams
Liaisons: Josh Klontz ’10 and Mark BurgeAdvisor: Zachary Dodds
• Fraud detection• Aid distribution• Law enforcement• National security
• Algorithmic identification of faces from images• Commercial systems exist; MITRE is building a
U.S. system for flexibility and security• Unobtrusive relative to other biometric
techniques, but with similar applications:
Automated Facial Recognition
Privacy Concerns
• Off-pose images are a significant challenge for automated facial recognition
• Many current algorithms, including MITRE's, do not include pose correction
Pose Correction
Pose Correction• Our approach to pose-correction involves finding
and matching facial features in different images
• Feature-finding and shape transformation, are also useful for other image-processing tasks
research: use and extend existing approaches
implement: within MITRE's existing codebase
test: using MITRE's test scaffolding and databases
Problem Statement
Our goal is to research, implement, and test a pose correction library that
improves MITRE's existing facial recognition system.
Average of Synthetic Exact
Filters
Active Shape Model
Pose-correction pipeline
Pixels Features Shape
ASEF ASM
• Facial features, or landmarks, can support both recognition and pose-correction
• Features are based on spatial geometry and/or appearance
Features
ASEF filter creation
training image (with known right-
eye location)
human-designed synthetic output
For each training image we create a synthetic output with the correct position of the feature, e.g., the right eye.
ASEF filter creation
training image (with known right-
eye location)
human-designed synthetic output
filter transforming the image at left into
the image at right
We want to create a filter that exactly transforms a training image into the desired synthetic output
* =
ASEF filter creation
In the Fourier domain, we want
where Synthetic, Image, and Filter are the 2D Fourier transforms of the synthetic output, image, and filter.
Complex division thus provides the filter:
ASEF filter creationWe take the average of all of the synthetic exact
filters to define, here, a final right-eye filter
We average 517 filters like this…
ASEF filter creationWe take the average of all of the synthetic exact
filters to define, here, a final right-eye filter
We average 517 filters like this…
…to obtain the final filter?
ASEF filter creationWe take the average of all of the synthetic exact
filters to define, here, a final right-eye filter
We average 517 filters like this…
…to obtain the final filter.
ASEF filter application
The filter’s strongest response is most right-eye-ey location in the image
Unfiltered image Filtered image
We apply the filter in the Fourier domain; the peak in the spatial domain is a first estimate of the feature location
Final output
Gallery
Error Images within that error
< .01 26.3 %
< .02 63.9 %
< .05 86.1 %
< .1 87.7 %
ASEF resultsMany images' eyes are found quite accurately,
but there are also some dramatic outliers:
Units are fraction of interocular distance
Percentage of pictures
Influence of ASEF’s Gaussian s
Radius, s = 2px Radius, s = 25pxRadius, s = 15px
synt
heti
c o
utpu
tsA
SEF
filt
ers
Radius, s = 20px
ASEF tradeoffsTesting changes in Gaussian radii (s)
the opposite tradeoff
more accurate localization – and
more outliers
left eye error (units of interocular distance)
Radius, s = 5px
left eye error (units of interocular distance)
ASEF improvementsUsing spatial heuristics as weights
Unweighted filtered image
Spatially weighted filtered image
1.0 * original 0.5 * originaloriginal
Without weighting With weighting
ASEF improvementsUsing spatial heuristics as weights
right eye error (units of interocular distance)
right eye error (units of interocular distance)
left
eye
err
or these clusters show mis-identifying the
left or right eye
Average of Synthetic Exact
Filters
Active Shape Model
Pose-correction pipeline
Pixels Features Shape
ASEF ASM
Active Shape Models (ASM)
• Describe classes of objects with varying shapes
geometric arrangement of facial features: eyes, nose, …
• Each shape is a set of points
• ASM trains on a training set of shapes, creating a statistical model of the variation within that shape-family.
ASM, step 1: Procrustes fitting
Procrustes analysis determines a scaling, rotation, and translation that best align a family of shapes.
training data (hundreds of faces) mean face (not necessarily angry)
We use this approach to align all of the training faces and extract the mean face.
ASM, step 2: Estimating face space
We use the most descriptive eigenvectors to describe the allowable shape domain.
ASM uses principal components analysis to build a model of representative transformations of a face
s = 0 (mean face)
-3s +3s
Independent face-shape axes
ASM, step 3: Transforming facesWe can apply realistic transformations to the
mean face along face space’s eigenvectors.
Second semester plans1) Multi-resolution and weighted ASEF feature finding
2) Adding pixel appearance to the ASM shape models
3) Implementing pose-correction techniques (for pixels)
shape space: yaw
First approach: apply ASM's transformations to generate poses at desired values of pitch and yaw.
Project Work Clinic Deliverables Due DateJanuary Winter break
Spring Semester Begins: 1/17
Phase III Presentation 1/17/2012Implement AAM, continue improving ASEF, research and select pose correction methods
Final Report & Poster
February Begin implementing selected pose correction methods, combine ASEF and ASM
March
Spring Break: 3/9-18Spring Break
Continue work on pose correction
AprilFinal Report Draft of Poster Design 4/2/2012Revise FR, Final Pres Draft 1 of Final Report 4/10/2012
Final Touches Final Report Review 4/12/2012Feature Freeze 4/13/2012Draft of Final Report 4/18/2012Draft of Final Presentation
4/23/2012
May
Finals: 5/3-4Projects Day 5/1/2012Final Report 5/4/2012
Spring ScheduleMITRE clinic, spring 2012 schedule
Questions?
Average of Synthetic Exact
Filters
Active Shape Model
Pixels Features Shape
ASEF ASM
Gallery
Gallery
Second semester plans
The spring term will focus on researching and implementing landmark-based pose correction techniques.
First approach: apply transformations given by ASM to generate poses at varying degrees of pitch and yaw.
yaw
pitch
Error Without log transform< .01 26.3 %
< .02 63.9 %
< .05 86.1 %
< .1 87.7 %
ASEF results
Comparing image pre-processing techniques
Error With log transform< .01 25.4 %
< .02 61.3 %
< .05 83.7 %
< .1 85.6 %
Fraction of interocular distance
Percentage of pictures
AAM adds color or grayscale information to ASM’s model. AAM can generate photorealistic faces, not just geometrically realistic ones.
Active Appearance Models (AAM)
Shown here are faces generated by varying the central face’s
appearance parameters by ±3 s along two appearance axes.
from T.F. Cootes, G.J. Edwards, and C.J. Taylor, Active Appearance Models
old pipeline
new pipeline
Face-recognition pipeline
Face detection
Recognition
Landmarking
Pose correction
Input image
Output ID
Fall term’s focus
Spring term’s focus
Next Steps
Improving ASEF:We will experiment with image processing techniques and weighting based on expected pose and image complexity
Extending ASM: We will implement Active Appearance Models to extend face pose-generation to face image-generation.
Implementing Pose Correction: ASEF and ASM provide a baseline approach: namely, transforming a query image to a standard face pose
Pixels
Features
Shape
Automated Facial Recognition• Use of computers to identify faces from images• Commercial systems exist, but MITRE is developing a
system specifically for the US for flexibility and security• Unobtrusive relative to other biometric techniques, but
with similar applications:
Motivation: Uses for Biometrics
• Law enforcement and national security • Fraud detection• Aid distribution• Social networking
Error Percent of Identifications< .01 0.263610315186246
< .02 0.638968481375358
< .05 0.861031518624642
< .1 0.876790830945559
Error Percent of Identifications< .01 0.253581661891117
< .02 0.613180515759312
< .05 0.836676217765043
< .1 0.859598853868195
Without cosine window
With cosine window
ASEF improvementsMapping
Last semester
This semester
Face-recognition pipeline
old pipeline
new pipeline
Face-recognition pipeline
Face detection
Recognition
Landmarking
Pose correction
Input image
Output ID
Training Data Average Face
ASM, step 2: Mean-face finding
We use this approach to align all of the training faces and thus find the mean face.
We got this… ?
Centered! ASEF’s right-eye filter in the spatial domain
Face-recognition pipeline
Face-recognition pipeline
Pixels
Landmarks
Shape Model
Average of Synthetic Exact Filters (ASEF)
Active Appearance Model (AAM)
Landmarking algorithms
ASEF filter creation
For each training image we create a synthetic output with the correct position of the feature, e.g., the right eye.
training image (with known right-eye location)
human-designed synthetic output
• Our approach to pose-correction involves finding and matching facial features in different images
Pose Correction
With dots.
Average of Synthetic Exact Filters (ASEF)
Active Shape Model (ASM)
Landmarking algorithms
Pixels
Features
Shape
old pipeline
Face-recognition pipeline
Face detection
Recognition
Input image
Output ID
Maybe we
don't use
this slide
at all?
old pipeline
new pipeline
Face-recognition pipeline
Face detection
Recognition
Landmarking
Pose correction
Input image
Output ID
Maybe we
don't use
this slide
at all?
MITRE Corporation
Pose Correction for Automatic Facial Recognition
Team: Elliot Godzich, Dylan Marriner, Emily Myers-Stanhope, Emma Taborsky (PM), Heather Williams
Liaisons: Josh Klontz ’10 and Mark BurgeAdvisor: Zachary Dodds
No dots at
all?