Upload
adonia
View
41
Download
0
Tags:
Embed Size (px)
DESCRIPTION
PROGRESS ON EMOTION RECOGNITION. J G Taylor & N Fragopanagos King’s College London. KCL WORK IN ERMIS. Analysis of emotion v cognition in human brain ( →simulations of emotion/attention paradigms) → emotion recognition architecture ANNA - PowerPoint PPT Presentation
Citation preview
ERMIS / IST-2000-29319
PROGRESS ON EMOTION
RECOGNITION J G Taylor & N Fragopanagos King’s College London
ERMIS / IST-2000-29319KCL WORK IN ERMIS
• Analysis of emotion v cognition in human brain (→simulations of emotion/attention paradigms)
• → emotion recognition architecture ANNA• ANNA hidden layer = emotion state, +
feedback control for attention (= IMC)• Learning laws for ANNA developed• ANNA fuses all modalities or only one• HUMAINE: WP3 +WP4
ERMIS / IST-2000-29319
BASIC BRAIN EMOTION CIRCUIT
• Valence in amygdala & OBFC• Attention in parietal & PFC • Interaction in ACG
SC
Parietal
A
Thal
ACG
SFG
NBM
ERMIS / IST-2000-29319
• SIMPLIFIED ARCHITECTURE OF EMOTIONAL/COGNITIVE PROCESSING IN THE BRAIN:
ERMIS / IST-2000-29319
DETAILED ARCHITECTURE FOR
FACES CLASSIFICATION
gender
ERMIS / IST-2000-29319
BASIC ERMIS EMOTION
RECOGNITION ARCHITECTURE:
Attention control system:
Feature vector Inputs:
Emotion state ashidden layer
Output as recognised emotional state
↑
→
ERMIS / IST-2000-29319
ANNA:Assume linear output:
Hidden layer response:
IMC node response:
Then solve self-consistent
equations for (y, z) for each
training input by relaxation
i
ii yaOUT
j k
kijkjiji zAxwfy ])1[(
i
ikik yBfz )(
ERMIS / IST-2000-29319NATURE OF ANNA
• Handles both unimodal and multi-modal data (input vector x of arbitrary dimension, not too large)
• Needs consistent input and output data {x(t), OUT(t)}, with t specified for both x & OUT=(activat, evaluat)
• Uses SALAS date-base (450 tunes)from QUB (Roddie/Ellie/Cate)
ERMIS / IST-2000-29319UNIMODAL RESULTS
• Can use numerous representations of emotion: extreme, continuous in n dimensions, …
• ANNA → FEELTRACE output (continuous 2-D)
• Trained on unimodal for prosody• First look at word content
ERMIS / IST-2000-29319
Text Post-Processing Module
• Prof. Whissell compiled ‘Dictionary of Affect in Language (DAL)’
• Mapping of ~9000 words → (activation-evaluation), based on students’ assessment
• Take words from meaningful segments obtained by pause detection → (activation-evaluation) space
• But humans use context to assign emotional content to words
ERMIS / IST-2000-29319
Text Post-Processing Module (SALAS data)
Table 1. Quadrant match for normal text (full DAL).Participant P1 P2 P3 P4 P9 P12 AllQuadrant match (%) 21.4 12.5 21.4 30.4 25.0 19.6 16.1
Table 2. Quadrant match for scrambled text (full DAL).Participant P5 P6 P7 P8 P10 P11 AllQuadrant match (%) 07.1 23.2 25.0 32.1 23.2 21.4 21.4
Table 3. Standard deviation of participants’ assessments for normal and scrambled text (average over all passages assessed).
Normal ScrambledEvaluation 1.24 1.45Activation 1.55 1.73
Table 4. Quadrant match averaged over participants’ groups for normal text and scrambled text when threshold for DAL range* is varied.
Threshold 0.0 0.25 0.5 0.75Normal text 16.1 16.0 12.5 16.4Scrambled text 21.4 21.4 19.6 21.8
*The higher the threshold the higher emotionally rated words are spotted only.
Conclude: need further context/semantics
ERMIS / IST-2000-29319
Correlational analysis of ASSESS
features• Correlational analysis between ~450
ASSESS features and FeelTrace =>– ASSESS features correlate more
highly with activation– Similar top ranking features for 3 out
of 4 FeelTracers (but still differences)– Different top ranking features for
different SALAS subjects->Is there a male/female trend? Difficult to say - insufficient data
ERMIS / IST-2000-29319
ANNA on top correlated ASSESS
features• Quadrant match using top 10 activation
features + top 10 evaluation features and activation – evaluation output space: Feeltracer jd cc dr em
Avg Quad Match
0.42 0.39 0.37 0.45
Std Dev 0.03 0.02 0.02 0.04
ERMIS / IST-2000-29319
ANNA on top correlated ASSESS
features• Half-plane match using top 10
activation features and activation only output space: Feeltracer jd cc dr em
Avg Quad Match
0.75 0.66 0.64 0.74
Std Dev 0.02 0.02 0.02 0.03
ERMIS / IST-2000-29319
PRESENT SITUATION OF ANNA:
MULTIMODAL
• Time-stamped data now becoming available for lexical (ILSP) & face streams (NTUA)
• Expect to have results in about 1 month for recognition for fused modalities (faces/prosody/words)
ERMIS / IST-2000-29319CONCLUSIONS
• UNIMODAL: ANNA on prosody OK (especially activation)
• MULTIMODAL: Soon to be done• On semi-realistic data (SALAS QUB)• Future work: 1) analysis of detailed results 2) insert temporality in ANNA
ERMIS / IST-2000-29319
QUESTIONS
• How to handle variations across experiencers and across FEELTRACERS?
• How to incorporate expert knowledge?• How combine recognition across
models? • Coding of emotions: as dimensional
reps or as dissociated states (sad AMYG v angry OBFC)?
• Nature of emotions as goal/reward assessment (frustration → anger; impossible →sadness, etc: brain-based)?