Upload
halle
View
26
Download
0
Embed Size (px)
DESCRIPTION
RECOGNIZING FACIAL EXPRESSIONS THROUGH TRACKING. Salih Burak Gokturk. OVERVIEW. PROBLEM DESCRIPTION TRAINING STAGE TESTING STAGE EXPERIMENTS CONCLUSION. Components of the recognition system. Training with stereo. Data. Classifier. New Data. Testing with mono. Output. - PowerPoint PPT Presentation
Citation preview
RECOGNIZING FACIAL EXPRESSIONS
THROUGH TRACKING
Salih Burak Gokturk
OVERVIEW• PROBLEM DESCRIPTION
• TRAINING STAGE
• TESTING STAGE
• EXPERIMENTS
• CONCLUSION
Components of the recognition system
Analysis -Face Tracking
Intelligence-Support Vector Machine
Classifier
Shape Parameters
Training with stereoData Classifier
Testing with mono
New Data
Output
PROBLEM DESCRIPTION(Tracking )
?
PROBLEM DESCRIPTION (Recognition)
X(t)[ Rigid, Open Mouth, Smile]
?[ Rigid, Open Mouth, Smile]
TrainingData Classifier TestingNew Data Output
OVERVIEW• PROBLEM DESCRIPTION
• TRAINING STAGE
• TESTING STAGE
• EXPERIMENTS
• CONCLUSION
p - degrees of freedom
Stereo TrackingData Monocular TrackingAnd Classification
Learn Shape
)1(uuo XX
uo
uu XnXnX )()(
p
iii
u XXX1
0
uoTp XXXXX 21
Support Vector Machines (SVM)
- Best discriminating hypersurface between two class of objects
- Map the data to high dimension using a map function - The hypersurface in the feature space corresponds to a hyperplane in the mapped space
TrainingData ClassifierTesting
(Classifier)New Data Output
OVERVIEW• PROBLEM DESCRIPTION
• TRAINING STAGE
• TESTING STAGE
• EXPERIMENTS
• CONCLUSION
LUKAS TOMASI KANADE OPTICAL FLOW TRACKER EXTENDED TO 3D
X(t)
I(x(t)) I(t+1)
TIME t+1
?X(t+1) tyx I
v
uII
P
iiiXXX
10 ),,( TRX
d
dT
dR
d
v
dT
v
dR
vd
u
dT
u
dR
u
v
u
J
tyx I
d
dT
dR
JII
One to Many Application of Support Vector Machines (SVM)
- One hypersurface per class is calculated
- A new data is tested for each hypersurface
k
z
z
k
i
e
eiP )(
- A different probability is assigned to ith class
OVERVIEW• PROBLEM DESCRIPTION
• TRAINING STAGE
• TESTING STAGE
• EXPERIMENTS
• CONCLUSION
-Training (Stereo) with 2 people, totally 240 frames - Testing with 3 people - 5 expressions: neutral, open mouth, close mouth, smile, raise eyebrow- velocity term is added to the shape vector:
3nn
nnewn
- Two other classifiers were tested: 1 - Clustering 2 – N-Nearest Neighbor
MOVIE (1)
MOVIE (2)
Decision of the system
Input
Neutral Open mouth
Close mouth
Smile Raise eyebrow
Neutral (44) 32 6 3 0 3
Open mouth (80) 0 76 4 0 0
Close Mouth (50) 0 1 49 0 0
Smile (87) 2 0 0 81 4
Raise Eyebrow (21) 3 0 0 0 18
Performance of the system for different expressions
Table 1
Comparison Between Different Methods
SVM with kernel erbf
SVM with kernel rbf
Clustering N-Nearest with N=9
N-Nearest with N=5
Same person
176/182 170/182 161/182 173/182 173/182
Total 256/282 253/282 242/283 255/282 253/282
Table 2
-Training (Stereo) with 1 person, totally 130 frames
- Testing with 3 people
- 5 expressions: neutral, open mouth, close mouth,
smile, raise eyebrow
Comparison Between Different Methods with only one person training set
SVM with kernel erbf
SVM with kernel rbf
Clustering N-Nearest with N=9
N-Nearest with N=5
Same person 98/110 99/110 109/110 109/110 110/110
Total 216/282 207/282 233/282 231/282 229/282
Table 3
-Training (Stereo) with 2 people, totally 240 frames
- Testing with 3 people
- 3 emotional expressions: neutral, happy, surprise
- Transition between expressions are separated
Comparison Between Different Methods with three emotional expressions
SVM with kernel erbf
SVM with kernel rbf
Clustering
N-Nearest with N=9
N-Nearest with N=5
N-Nearest with N=3
N-Nearest with N=1
Same person
164/165 165/165 152/165 163/165 164/165 164/165 164/165
Total 222/228 223/228 213/228 225/228 224/228 223/228 223/228
Table 4
Performance Comparison Between Previous Expression Recognition Work
Recognition Rate
Pose Change
Number of Expressions
Test/Train Subject
Number of Data
Comments
Chen et.al, ICME 2000
%89 Direct camera view
7 Different subject
470 images
Problem with different people
Wang et.al, AFGR 1998
%96 Direct camera view
3 Different subject
29 image sequence
Sequence classification
(easier) Lien et.al,
AFGR 1998 %85-%93 ~10
degrees rotation
4 Different subject
~130 images
Only upper part of the face is
classified Hiroshi et.al, ICPR 1996
%70 ~45-60 degrees rotation
5 Same subject
900 images
Permits for rotations, but
rates are not as good Chang et.al,
IJCNN 1999 %92 Direct
camera view
3 Different subject
38 images Small test and training set
Matsuno et.al, ICCV 1995
%80 Direct camera view
4 Different subject
45 images Small test and training set
Hong et.al, AFGR 1998
%65-%85 Direct camera view
7 Same and different subject
~250 images
%85 with known person % 65 with unknown person
Hong et.al, AFGR 1998
%81-%97 Direct camera view
3 Same and different subject
~250 images
%97 with known person % 81 with unknown person
Sakaguchi et.al, ICPR
1996
%84 Direct camera view
6 Same subject
- The test and training set not
mentioned Our Work %91 ~70-80
degrees rotation
5 Different subject
282 images
Table 2
Our Work %98 ~70-80 degrees rotation
3 Different subject
228 images
Table 4 - Emotional
Expressions
OVERVIEW• PROBLEM DESCRIPTION
• TRAINING STAGE
• TESTING STAGE
• EXPERIMENTS
• CONCLUSION
Future Work
Conclusions
- Breakthrough facial expression recognition rates .
- 3-D is the right way to go…
- Test with more subjects and expressions.
- further application to face recognition (?)