Recognition of

Nonverbal Facial and

Vocal Affect After TBI Barbra Zupan, Ph.D., Brock University, Canada

Duncan R. Babbage, Ph.D., Massey University, NZ

Dawn Neumann, Ph.D., Indiana University, USA

Barry Willer, Ph.D., University at Buffalo, USA

Affect Recognition Difficulties

Research has shown that people with TBI have perceptual

deficits in the recognition of facial affect (Babbage et al., 2011;

Green et al., 2004; Hopkins et al., 2002) and vocal affect (Dimoska et al.,

2010; Pell, 2006; Pell & Baum, 1997; Pell et al., 2011)

BUT few studies have simultaneously examined recognition

of facial and vocal affect in the same group of people

Do these deficits co-occur?

Is one modality impacted more greatly than the other?

Spell & Frank,

2000

Milders et al.,

2003

McDonald &

Saunders,

2005

Ietswaart et

al., 2008

Participants 24 TBI

24 Controls

17 TBI

17 Controls

34 TBI

28 Controls

37 TBI

34 Controls

Severity Not specified Moderate to

Severe

Severe Only Mild to Severe

Test DANVA2

(4 emotions)

E-60-FT

(6 emotions)

FAB-8A

(5 emotions)

TASIT-EET

(7 emotions)

E-60-FT

(6 emotions)

FAB-8A

(5 emotions)

Deficits Impairment in

both modalities

Impairment in

both modalities

No group

difference for

static/dynamic

faces

Poorer on voices

only

Impairment in

both modalities

Face vs Voice Higher accuracy

for faces

N/A

Higher accuracy

for faces

Higher accuracy

for voices

Other Findings

Had more difficulty

with high intensity

faces than Controls

but not low

The Current Study

Data collected as part of a Randomized Clinical Trial

Funded by the National Institute on Disability and Rehabilitation

Research (H133G080043)

Three sites

Canada, United States, New Zealand

Comparing two treatment programs to one control group

Facial Affect Recognition

Social Emotional Inferencing

Extensive testing done to evaluate eligibility and changes due

to treatment

Participation Criteria

People with moderate to severe TBI

Injury sustained after the age of 8

All were between 18 and 65 years of age at time of testing

Able to respond to basic oral and written language

No current substance dependence, major psychiatric disorder

Screened: 202

Ontario: 41 (19)

US: 108 (36)

NZ: 53 (18)

Cause

MVA: 124 (62%)

Fall: 28 (14%)

Assault: 14 (7%)

Gender

Males: 149

Females: 53

Age

Mean: 39.9 yrs

Range: 21-65 yrs

Time Since Injury

Mean: 10 yrs

Range: 1 - 42 yrs

Diagnostic Analysis of Nonverbal

Affect-2 (Nowicki, 2008)

24 coloured photographs

6 of each emotion

Happy, sad, angry, fearful

3 high intensity expressions

3 low intensity expressions

Displayed 15 seconds

Adult Faces

Diagnostic Analysis of Nonverbal

Affect-2 (Nowicki, 2008)

Adult Paralanguage

24 repetitions portrayed by one male and one female actor

“I’m going out of the room now and I’ll be back later”

6 of each emotion

Happy, sad, angry, fearful

3 high and 3 low intensity exemplars within each category

Heard each sentence one time only

Results

Not impaired Faces Only Voices Only Both Modalities

48% 17% 14% 21%

n=92 n=32 n=26 n=40

Over half of the participants showed impaired

recognition in one or both modalities (n=98)

Facial stimuli recognized significantly better than vocal

stimuli, 2 (2)= 22.16, p < .001

Supports previous work by Spell and Frank (2000)

and McDonald and Saunders (2005)

Rasch Analysis

Need to evaluate psychometric properties of DANVA2 when

considering the effect of intensity and emotion category on

affect recognition

Items should be sensitive to different skills levels as indicated

by Item Characteristic Curves

Happy

Fearful

Angry

Sad

DANVA-AF

High intensity

Happy had low

difficulty

Not sensitive

for

differentiation

Other items

showed

appropriate

properties and

adequate range

of difficulty

Low intensity

items more

difficult across

categories

Happy

Fearful

Angry

Sad

DANVA-AP

Low intensity

items more

difficult across

categories

Voice stimuli

less well

differentiated in

difficulty and

intensity

Conclusion

More than half of the 202 participants screened showed deficits in

the recognition of one or both modalities

Significantly more participants had difficulty with faces

Happy faces easier to identify than other emotions

No differentiation by emotion category for voices

High intensity expressions easier than low for both modalities

With the exception of high intensity Happy faces, the DANVA2

appears to provide a suitable range of difficulty to assess facial and

vocal affect recognition in people with moderate to severe TBI

References Babbage, D.R., Yim, J., Zupan, B., Neumann, D., Tomita, M.R., & Willer, B. (2011). Meta-analysis of facial affect recognition difficulties after traumatic

brain injury, Neuropsychology, 25(3), 277-285.

Dimoska, A., McDonald, S., Pell, M.C., Tate, R.L., & James, C.M. (2010). Recognizing vocal expressions of emotion in patients with social skills deficits following traumatic brain injury. Journal of International Neuropsychological Society, 16, 369-382.

Green, R.E.A., Turner, G.R., & Thompson, W.F. (2004). Deficits in facial emotion perception in adults with recent traumatic brain injury. Neuropsychologia, 42, 133-141.

Hopkins, M.J., Dywan, J., & Segalowitz, S.J. (2002). Altered electrodermal response to facial expression after closed head injury. Brain Injury, 16, 245-257.

Ietswaart, M., Milders, M., Crawford, J.R., Currie, D., Scott, C.L. (2008). Longitudianl aspects of emotion recognition in patients with traumatic brain injury. Neuropsychologica, 46, 148159.

McDonald, S., & Saunders, J.C. (2005). Differential impairment in recognition of emotion across different media in people with severe traumatic brain injury. Journal of the International Neuropsychological Society, 11, 392-299.

Milders, M., Fuchs, S., & Crawford, J.R. (2003). Neuropsychological impairments and changes in emotional and social behavior following severe traumatic brain injury. Journal of Clinical and Experimental Neuropsychology, 25(2), 157-172.

Nowicki, S. (2008). The Manual for the Receptive Tests of the Diagnostic Analysis of Nonverbal Accuracy 2 (DANVA 2). Atlanta, GA: Department of Psychology, Emory University.

Pell, M.D. (2006). Cerebral mechanisms for understanding emotional prosody in speech. Brain and Language, 96, 221-234.

Pell, M.D., & Baum, S.R. (1997). The ability to perceive and comprehend intonation in linguistic and affective contexts by brain-damaged adults. Brain and Language, 57, 80-99.

Pell, M.D., Jaywant, A., Monetta, L., & Kotz, S.A. (2011). Emotional speech processing: Disentangling the effects of prosody and semantic cues, Cognition and Emotion, 25(5), 834-853.

Spell, L.A., & Frank, E. (2000). Recognition of nonverbal communication of affect following traumatic brain injury. Journal of Nonverbal Behavior, 24(4), 285-300.