Upload
anabel-evans
View
215
Download
0
Embed Size (px)
Citation preview
Computer Science Department
Detection, Alignment and Recognitionof Real World FacesErik Learned-Miller
with Vidit Jain, Gary Huang, Andras Ferencz, et al.
Faces in the Wild
2Computer Science
Is Face Recognition Solved?
3Computer Science
Is Face Recognition Solved?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
“100% Accuracy in Automatic Face Recognition” [!!!]
Science 25 January 2008
4Computer Science
Is Face Recognition Solved?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
“100% Accuracy in Automatic Face Recognition” [!!!]
Science 25 January 2008
A history of overstated results.
5Computer Science
The Truth
Many different face recognition problems• Out of context, accuracy is meaningless!
Many problems are REALLY HARD!• For some problems
state of the art is 70% or worse!
We have a long way to go!
6Computer Science
Face Recognition at UMass
Labeled Faces in the Wild The Detection-Alignment-Recognition pipeline Congealing and automatic face alignment Hyper-features for face recognition New directions in recognition
7Computer Science
Labeled Faces in the Wild
http://vis-www.cs.umass.edu/lfw/
8Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
9Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
10Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
11Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
Labeled Faces in the Wild
12Computer Science
The Many Faces of Face Recognition
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Labeled Faces in the Wild
13Computer Science
Labeled Faces in the Wild
13,233 images, with name of each person 5749 people 1680 people with 2 or more images
Designed for the “unseen pair matching problem”.• Train on matched or mismatched pairs.• Test on never-before-seen pairs.
Distinct from problems with “galleries” or training data for each target image.
Best accuracy: currently about 73%!
14Computer Science
Detection-Alignment-Recognition Pipeline
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
DetectionRecognitionAlignment
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
“Same”
15Computer Science
Detection-Alignment-Recognition Pipeline
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
DetectionRecognitionAlignment
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
“Same”
Parts should work together.
16Computer Science
Labeled Faces in the Wild
All images are output of a standardface detector.
Also provides aligned images. Consequence: any face recognition algorithm
that works well on LFW can easily be turned into a complete system.
17Computer Science
Congealing (CVPR 2000)
QuickTime™ and a decompressor
are needed to see this picture.
18Computer Science
Criterion of Joint Alignment
Minimize sum of pixel stack entropies by transforming each image.
A pixel stack
19Computer Science
Congealing Complex Images
Window around pixel SIFT vector and clusters
SIFT clusters
vector representingprobability of each cluster,or “mixture” of clusters
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
21Computer Science
Crash Course on Martian Identification
?
Test: Find Bob after one meeting
Martian training set
=
=
=
Bob
22Computer Science
Training Data
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
“same”
“different”
23Computer Science
General Approach to Hyper-feature method
Carefully align objects Develop a patch-based model of
image differences. Score match/mismatch based on patch
differences.
24Computer Science
Three Models
1. Universal patch model:P(patchDistance|same)P(patchDistance|different)
2. Spatially dependent patch model:P(patchDistance |same,x,y)P(patchDistance |different,x,y)
3. Hyper-feature dependent model:1. P(patchDistance |same,x,y,appearance)2. P(patchDistance |different,x,y,appearance)
25Computer Science
Universal Patch Model
A single P(dist | same) for all patches
Different blue patches are evidence against a match!
26Computer Science
Spatial Patch Model
P(dist|same,x1,y1) estimated separately from P(dist|same,x2,y2)
Greatly increases discriminativeness of model.
27Computer Science
Hyper-Feature Patch Model
Is the patch from a matching face going tomatch this patch?
28Computer Science
Hyper-Feature Patch Model
Is the patch from a matching face going tomatch this patch? Probably yes
29Computer Science
Hyper-Feature Patch Model
What about this patch?
30Computer Science
Hyper-Feature Patch Model
What about this patch?Probably not.
31Computer Science
Ridiculous Errors from the World’s Best Unconstrained Face Recognition System
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
32Computer Science
Ridiculous Errors from the World’s Best Unconstrained Face Recognition System
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
33Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
34Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we guesspose?
35Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we guessgender?
36Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we guessdegree of balding,
beardedness,moustache?
37Computer Science
The New Mission: Estimate Higher Level Features
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Can we say thatnone of these individuals are
the same person?
38Computer Science
What can we do with a good segmentation?
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
39Computer Science
CRF Segmentations
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
40Computer Science
CRF Segmentations
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
41Computer Science
Who’s This?
42Computer Science
Who’s This?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
43Computer Science
Who’s This?
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
from www.coolopticalillusions.com
Computer Science Department
Thanks