Systemizing Emotions: Using Interactive Multimedia as a Teaching Tool
by Ofer Golan and Simon Baron-Cohen
Chapter 10
Nonverbal Communication
Fluent social and communication skills are dependent on the ability to infer emotional and other mental states based on: Facial expressionsVocal intonationContext
Facial Emotion Recognition
Individuals with autism have more difficulty than neurotypicals or those with cognitive delays in recognizing the six basic emotions expressed in the face:
•Happiness•Sadness•Fear
•Anger•Surprise•Disgust
Facial Emotion Recognition
Individuals with Asperger Syndrome or HFA may recognize the basic emotions, but have trouble with more complex emotions such as:EmbarrassmentInsincerityIntimacy
Processing Faces Differently
Individuals with ASDs tend to process faces by looking at individual features (mouth, nose, or eyes)
Neurotypicals tend to process faces as a whole
Processing Facial Emotions Differently When individuals with ASDs process
facial emotions, there is less activation (than with neurotypicals) in regions of brain that processFaces (fusiform gyrus)Emotions (amygdala)
Vocal Recognition of Emotion Less research in this area, but several
studies have reported a deficit in high-functioning adults with ASDs, compared to neurotypicals
One neuroimaging study showed less intensive/extensive activation in the medial frontal area of the brain (used for judging others’ mental states) during an vocal emotion recognition task
Contextual Recognition of Emotion Individuals w/ ASDs have more difficulty
inferring emotional/mental state based on contextual information (i.e., “reading between the lines”)
Neuroimaging study showed reduced activation of the left medial prefrontal cortex during the Strange Stories Test*
*assesses ability to interpret nonliteral statements made by story characters, based on context
Multimodal Recognition of Emotion
In real life, we have to interpret and integrate facial, vocal, body language, and contextual information simultaneously
Individuals with ASDs show deficit in this area, as evidenced in studies that use life-like social situations (e.g., movies)
Multimodal Recognition of Emotion
Individuals with ASDs may have difficulty because they don’t pick up the correct cues, AND/OR
They don’t integrate the cues into a whole (i.e., weak central coherence), and focus too much on the detailsThis could be due to under-connectivity
among frontal brain regions
Systemizing in Autism
Individuals with ASDs show good, and sometimes superior, skills in systemizingThe drive to analyze or build systems
in order to understand and predict the behavior of rule-based events
Systemizing in Autism
Individuals with ASDs show superior abilities in tasks that requireSearching for detailAnalyzing and manipulating systems
Could high-functioning individuals w/ ASDs use their systemizing skills to compensate for their difficulties recognizing emotions?
Early Attempts to Teach Emotion Recognition
Either focused on just the basic emotions, or
Were included as part of social skills group curriculum, but didn’t focus on systemizing training
Advantages of Using Computers to Teach Emotion Recognition Computerized environment is predictable,
consistent, and free from social demands Many individuals w/ ASDs enjoy using
computers Lessons can be repeated over and over
until mastery is achieved Interest and motivation can be maintained
through different and individually selected computerized rewards
Mind Reading Software
Interactive guide to emotions and mental states Designed by Baron-Cohen, Golan,
Wheelwright & Hill (2004) Teaching tool for emotion recognition
for learners with ASDs
Mind Reading’s Systemizing Approach 412 emotions and mental states, each
defined and demonstrated by: 6 silent films of faces 6 voice recordings 6 written examples 18 x 412 = 7,416 units of emotion info
• Grouped by: 24 emotion groups 6 developmental levels (ages 4-adult)
Emotions Database Applications Emotions Library
Allows users to freely browse, compare, and add their own notes
Learning Center Structured approach to lessons, quizzes,
rewards Game Zone
5 educational games to play while studying emotions
Controlled Study of Adults Using Mind Reading Software 3 groups matched on age, verbal and
performance IQ, handedness, and gender Intervention Group: 19 adults with AS or
HFA each used software at home for 10-20 hours over 10 weeks
AS/HFA Control Group: 22 adults participated only in pre- and post- assessments, with no intervention
Typical Control Group: 28 participants
Intervention Group
Allowed to use Emotions Library and Learning Center as they wished, but told to limit games to no more than 1/3 of usage time
Encouraged to analyze facial and vocal stimuli systematicallye.g., Distinguish “insincere” from
“interested” by looking not just at smiling mouth but at eyes as well
Findings
Intervention group significantly improved their ability to recognize complex emotions/mental states from both faces and voicesImprovement was limited to taught
stimuli, (i.e., faces and voices taken from Mind Reading)
Generalization Issues
Participants found it difficult to generalize to other tasks of emotion recognition from voices and eyes
No improvement on integration of facial, vocal, and contextual cues
Software usage time was correlated with higher film task scores, so perhaps using software longer would increase generalization
Qualitative Data
Participants reported paying greater attention to faces and emotions, as well as improved eye contact
A year later, participants who continued to use Mind Reading reported having improved friendship skills
Weak Generalizing or Strong Systemizing?
Authors’ view is that reduced generalization is not so much a deficit as a reflection of the strong drive to systemize in people with ASDs
But socio-emotional world is flexible, context-dependent, and unpredictable, making it difficult--if not impossible--to systemize
Potential Problem
Attempting to systematically teach about emotions to individuals with ASDs may lead to rigid application of the system, and eventually to limited generalization
Mind Reading’s Role in an Intervention Plan
Authors recommend that Mind Reading be viewed as a first step in a training programExamples demonstrate what emotions
look like and sound like Next steps should deal with context
and integration of socio-emotional cues into one (flexible) picture
Question for Future Research
When participants improve their emotion recognition by using Mind Reading software, are regions of their brains activating in patterns more like those of neurotypicals?
Or are they using compensatory strategies in other parts of the brain?
Conclusions
Complex emotion recognition can be improved over a relatively brief length of time using Mind Reading’s systemizing approach, but …
Additional methods are required to improve generalization.