Embed Size (px)
Citation preview
Computational Model for Body Expression of Emotion
(BEE)
byMarina Ousov-Fridin
Tamar Flash
Faculty of Mathematics and Computer ScienceThe Weizmann Institute of Science, Israel
Statement of the problem
1.Main Aim: To provide o provide compactcompact math. math.
representation of BEE for 4 basic representation of BEE for 4 basic
emotions: emotions: Sadness, Joy, Fear, AngerSadness, Joy, Fear, Anger
and build computational model. Based and build computational model. Based
on it to define on it to define primitives and primitives and
synergismssynergisms..
2.2.Correlation between Correlation between PrimitivesPrimitives and and
Perception.Perception.
Theory of organization of the motor system:
Primitives and Synergies
Motor primitivesMotor primitives: the entire repertoire of man actions could be constructed from limited set of building blocks (Mussa-Ivaldi [2004]):
universal
defined in terms of different state variables, coordinate frames and
may exist at different levels of representation
static, kinematical, dynamic or combined
Examples: Troje - PCAPCA ; Mataric - static set of joint angles;….static set of joint angles;….
SynergiesSynergies: the coordinated control over several limb segments or multiple effectors.
coordination between different leg angles
coordination between hand and leg trajectories.
The BEE input stream (1)
BEE Gesture Symbolic
I.I. TypeType:
Single person ,Context-Free Simulations, Static Photos Naturalness
Low Computational Complexity
II.Genuine range of subjects : Ordinary people and actors;
Genders, ages, social backgrounds and cultural
influences;
Build inBuild in
The BEE input stream (2)Main ProblemMain Problem: Uncertainty and uniqueness
ObserverPerformer
RecognitionRatepicture
IntensityRatepicture
Performers present the apexapex of the expression by their opinion in the portrayed BEE
Subjects:Subjects: 27 (R.V. Lab)
18 (~20 y. old)
24 (14-17 y. old)
72 (artists) Subjects:Subjects: 21 (R.V. Lab)
Vision Processing
TrackingTracking
Human Vision-based analysis Human Vision-based analysis Human Vision-based analysis Human Vision-based analysis
Tracking Heads/FacesTracking Heads/Faces
Tracking HandsTracking Hands
Tracking Body (Segmentation)Tracking Body (Segmentation)
Human Body Parts Human Body Parts Position EstimationPosition Estimation
Tracking Body Parts (Labeling)Tracking Body Parts (Labeling)
Head Position EstimationHead Position Estimation
Gesture RecognitionGesture Recognition
Body model estimationBody model estimation
Extract candidate features :: nF ...1
Information measurement criteria:
I(C; F) = H(C) – H (C|F)is amount of information delivered by a candidate primitive about
the class of emotion
Primitive definition (F)
Bank of primitives (B)
Select important features/primitives
0
1C
data set belong to class
otherwiseClass Non Class
Similarity Function
Similarity measurement (S)
Normalized Cross Correlation
Distance between turning
function of polygon
Primitive as binary variables (Fi)
i y
i x
i y x
m i y m i x
m i y m i x S
2 2 ) ) ( ( ) ) ( (
) ) ( ( * ) ) ( [(
PBAp BAd ),(
otherwise 0
),( if 1),( iiii
fISIf
MI and associated threshold
The mutual information between the primitiveprimitive and the classclass of a emotion (I)
Similarity Threshold ( )
Max-Min Algorithm
First Primitive
K-primitive: max additional information
Mutual Information Algorithm
Cii
Fi
ii
ffFfP
f
))|)Log(P(C|P(C)(
C))P(C)Log(P(-))(I(C;
1,0
C
)),((maxarg CfI iiii
i
FFCIF );(maxarg1
)|;(maxarg 11 BFCIF kk )|;(minmaxarg1 iiF
k FFCIF
General Scheme
Anger Fear SadnessJoy
i iI
nIn
.
.
.
.
.
.
i iI
nIn
.
.
.
.
.
.
i iI
nIn
.
.
.
.
.
.
i iI
nIn
.
.
.
.
.
.
S ),( ii If ))(I(C; iif
)|;(minmaxarg iiF
FFCI
. . .
C C: : : :CC
Max - Min Max - Min
FFCI );(maxarg
Results: Body
0.1408
0.1755 0.1484
0.1467 0.1431
0.1408
Joy (Join Angles)Joy (Join Angles)
0.1755 0.0108
0.0827 0.0960
0.1215 0.0781
MIMI Max-MinMax-MinS
im.
Mea
sur.
Sim
. M
easu
r.
AngerAnger FearFear JoyJoy SadnesSadnesss
JAJA
TATA
0.53 0.47 0.78 0.77
0.6 0.56 0.8 0.83
Recognition Recognition RateRate
Results: Gesture
Sadness: Right GestureSadness: Right Gesture
Max-MinMax-Min
Extract EdgeExtract Edge : primitives are gesture polygon
Similarity MeasurementSimilarity Measurement : distance between turning function
AngerAnger FearFear JoyJoy SadnesSadnesss
Right Right HandHand
Recognition Recognition RateRate
Left Left HandHand
0.78 0.87 0.76 0.84
0.95 0.7 0.5 0.6
Body Feature
Body boundary box normalized by silhouette
Body tendency
Vision processing Segmentation: background subtraction
Find Body Silhouette: morphology and edge detection
Head Feature
Pith
Roll
Vision processing Tracking head: Skin detection Approximation to ellipse
Additional Feature (1)
Length
Area
etteBodySilhou
ryBoxBodyBounda
hemisphere-lower0
hemisphere-upper1
)()(2
1
2
1
1
2
1)|(
ssT
s cc
s
eskincp
on HSVHSV color space
Additional Feature (2)AngeAnge
rrFearFear JoyJoy SadnessSadness
How to combine all p
ossible
How to combine all p
ossible
feature?
feature?
Head
Body
BEE Perception and correlation to Computational model
Rating database; influence of rating on computational model.
Class differences: Gender
Actors
Culture influences Contributes to emotion theory Response time – Wrong/Right Recognition
Response Time - Emotion
Conclusion
exists computational model, describes exists computational model, describes static static BEEBEE (for 4 basic emotion) by (for 4 basic emotion) by
compact representation, when features compact representation, when features (primitives)(primitives) are selected by computational are selected by computational
measurement measurement (MI)(MI)
Thank you for
Thank you for
your attention!
your attention!
