Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Music Processing in Deaf Adults
with Cochlear Implants
by
Mathieu R. Saindon
A thesis submitted in conformity with the requirements for the degree of Master of Arts
Graduate Department of Psychology University of Toronto
© Copyright by Mathieu R. Saindon 2010
ii
Music Processing in Deaf Adults with Cochlear Implants
Mathieu R. Saindon
Master of Arts
Graduate Department of Psychology University of Toronto
2010
Abstract
Cochlear implants (CIs) provide coarse representations of pitch, which are adequate for speech
but not for music. Despite increasing interest in music processing by CI users, the available
information is fragmentary. The present experiment attempted to fill this void by conducting a
comprehensive assessment of music processing in adult CI users. CI users (n =6) and normally
hearing (NH) controls (n = 12) were tested on several tasks involving melody and rhythm
perception, recognition of familiar music, and emotion of recognition in speech and music. CI
performance was substantially poorer than NH performance and at chance levels on pitch
processing tasks. Performance was highly variable, however, with one individual achieving NH
performance levels on some tasks, probably because of low-frequency residual hearing in his
unimplanted ear. Future research with a larger sample of CI users can shed light on factors
associated with good and poor music processing in this population.
iii
Acknowledgments
This thesis would have not been possible without the constant efforts, guidance and dedication of
my supervisors Dr. Sandra Trehub and Dr. Glenn Schellenberg. I am very grateful for all of their
help with this research project.
I would also like to thank my parents and sister for their long-distance support, and my lovely
wife Lauren for baking all of those muffins.
Lastly, I would like to thank the health professionals and patients of the Sunnybrook Cochlear
Implant Program. Without them, this project would not have been possible.
iv
Table of Contents
Acknowledgments .......................................................................................................................... iii
Table of Contents ........................................................................................................................... iv
List of Tables ................................................................................................................................. vi
List of Figures ............................................................................................................................... vii
List of Appendices ....................................................................................................................... viii
1 Introduction ................................................................................................................................ 1
2 Method ....................................................................................................................................... 4
2.1 Participants .......................................................................................................................... 4
2.2 Apparatus ............................................................................................................................ 6
2.3 Test Battery ......................................................................................................................... 6
2.3.1 Metric task .............................................................................................................. 7
2.3.2 Rhythm task ............................................................................................................ 7
2.3.3 Distorted Tunes Test ............................................................................................... 8
2.3.4 Musical emotion test ............................................................................................... 8
2.3.5 Diagnostic Analysis of Nonverbal Accuracy 2 ....................................................... 9
2.3.6 Open-set word recognition ...................................................................................... 9
2.3.7 CAMP test ............................................................................................................... 9
2.3.8 Familiar music task ............................................................................................... 10
2.3.9 Pitch- and interval matching ................................................................................. 11
2.4 Procedure .......................................................................................................................... 11
3 Results and Discussion ............................................................................................................. 12
3.1 Open-Set Word Recognition ............................................................................................. 12
3.2 CAMP ............................................................................................................................... 12
v
3.3 Distorted Tunes Test ......................................................................................................... 15
3.4 Familiar Music Task ......................................................................................................... 17
3.5 Metric Task & Modified MBEA Rhythm Task ................................................................ 20
3.6 Music Emotion & DANVA2 ............................................................................................ 22
3.7 Pitch- and Interval-Matching Task ................................................................................... 25
3.8 Conclusion ........................................................................................................................ 27
References ..................................................................................................................................... 29
Appendix ....................................................................................................................................... 36
vi
List of Tables
Table 1. Participant Characteristics……………………………………………………………….5
Table 2. List of CVC Words………………………………………………………………………9
Table 3. Music Emotion Arousal Scores……………………………………………………...…24
vii
List of Figures
Figure 1. Syllable (CVC) Recognition…………………………………………………………...13
Figure 2. CAMP Pitch Threshold (semitones)………………………………….....……………..13
Figure 3. CAMP Melody Recognition (Percent Correct)….…………………………………….14
Figure 4. CAMP Timbre Recognition (Percent Correct)..……………………………………….14
Figure 5. Distorted Tunes Test…………………………….……………………………………..16
Figure 6. Familiar Music – No-Rhythm Condition (Percent Correct)…………………………...17
Figure 7. Familiar Music – Melody Condition (Percent Correct)………………………………..18
Figure 8. Familiar Music – Instrumental Condition (Percent Correct)…………………………..19
Figure 9. Metric Task…………………………………...………………………………………..21
Figure 10. Modified MBEA Rhythm Task………………………………………………………22
Figure 11. Music Emotion Task.…………………………………………………………………24
Figure 12. DANVA 2: Adult Vocal Emotion Task…...…………………………………………25
Figure 13. Average Deviations in Pitch (semitones)…….………………………………………26
Figure 14. Deviations in Interval Matching (semitones)……………...…………………………26
viii
List of Appendices
Appendix A. Music and Cochlear Implants Questionnaire…………………………………...…36
Appendix B. Music Background Information Questionnaire (Adults)…………………………..42
Appendix C. Music and Cochlear Implants Interview…………………………………………...44
Appendix D. Semi-Structured Interview……………………………………………………..….45
1
1 Introduction
A cochlear implant (CI) is a prosthetic device designed to provide hearing sensations to
deaf individuals. Unquestionably, it is the most successful neural prosthesis to date, as viewed by
the number of individuals who have received it worldwide and derived great benefit from it
(Wilson, 2004). Its external microphone and signal processor receive incoming sound, transform
it into an electrical signal, and extract features that are important for speech perception. This
information is then transmitted to electrodes implanted in the cochlea, and, in turn, to the
auditory nerve.
Modern devices provide relatively coarse representations of spectral information, which
are adequate for perceiving speech in ideal listening conditions (Shannon, Zeng, Kamath,
Wygonski & Ekelid, 1995; Wilson, 2000), but they are inadequate for perceiving speech in noise
(Fetterman & Domico, 2002; Firszt et al., 2004), identifying emotion from speech prosody
(Hopyan-Misakyan, Gordon, Dennis & Papsin, 2009; Meister, Landwehr, Pyschny, Walger &
Von Wedel, 2009), differentiating one speaker from another (Meister et al., 2009), identifying
musical timbres or instruments (McDermott & Looi, 2004), and recognizing melodies from pitch
cues alone (Kang et al., 2009; Kong, Cruz, Ackland-Jones & Zeng, 2004).
Music perception is especially challenging for CI users. Coding strategies in implant
processors extract the temporal envelope, discarding the temporal fine structure that is critical for
music perception (Galvin, Fu & Shannon, 2009). Consequently, the music perceived by CI users
is considerably degraded in sound quality and detail, especially as it pertains to pitch patterning.
In fact, implant users often describe music as unpleasant, mechanical, and difficult to follow
(Gfeller, Christ, Knutson, Woodworth, Witt & DeBus, 1998; Gfeller, Witt, Stordahl, Mehr &
Woodworth, 2000; Lassaletta et al., 2007). It comes as no surprise, then, that postlingually
deafened adult CI users, who had access to rich auditory representations of music before their
hearing loss, are often disappointed with music heard via their implant (Gfeller, Christ, Knutson,
Witt, Murray & Tyler, 2000; Lassaletta et al., 2007; Looi & She, 2010; Veekmans, Ressel,
Mueller, Vischer & Brockmeier, 2009). This is unfortunate because music is an important source
of pleasure for many, if not most, hearing individuals (Laukka, 2006). Even for postlingually
2
deafened implant users, quality of music perception is associated positively with quality-of-life
ratings (Lassaletta et al., 2007).
As noted, limited temporal fine structure or spectral detail provides limited access to
pitch patterning. Cooper, Tobey, and Loizou (2008) used a test battery designed for the diagnosis
of amusia, or tone deafness, in individuals with normal audiological profiles. They found that CI
users failed to discriminate two melodies that differed in pitch patterning even when the
difference involved a change in pitch contour or key. In that sense, CI users performed much like
amusic individuals who are typically deficient in the perception of pitch patterns but not
temporal patterns (Foxton, Nandy & Griffiths, 2006). In other research, CI users have exhibited
difficulty determining which of two sounds is higher in pitch (also referred to as pitch ranking –
see Kang et al., 2009; Looi, McDermott, McKay & Hickson, 2004, 2008), detecting the direction
(higher or lower) of a pitch change in a melody (Gfeller et al., 2007; Leal et al., 2003), and
differentiating melodies in the absence of rhythmic cues (Kang et al., 2009; Kong et al., 2004).
In the context of these pitch perception difficulties, it is not surprising to find deficient pitch
production as well. For example, child CI users preserve the rhythms but not the pitch contours
(i.e., patterns of rising and falling pitches) when they sing familiar songs (Nakata, Trehub,
Mitani & Kanda, 2006; Xu et al., 2009). Although this pattern is mirrored, to some extent, in the
song production of amusic individuals, some individuals with severe pitch perception deficits
manage to produce accurate contours and intervals when singing familiar songs with words,
which reveals an unexpected dissociation between perception and action (Dalla Bella, Giguère &
Peretz, 2009).
By contrast, tempo and rhythm perception in CI users are reportedly comparable to
normally hearing (NH) listeners except when the stimuli or tasks are complex (Cooper et al.,
2008; Gfeller, Woodworth, Robin, Witt & Knutson, 1997; Kong et al., 2004). Although we have
learned much in recent years about the music perception skills of CI users, much remains to be
learned. For example, the perceptual demands of differentiating simple rhythm or pitch patterns
differ drastically from the demands of perceiving conventional music on the radio, on iPods, or
in concert halls. Rhythm, pitch, and timbre are typically blended into a coherent whole.
Discriminating two rhythms in isolation does not mean that a CI user would be able to hear a
guitar solo when it is accompanined by a drum kit, bass, guitar, and vocals. He or she might also
be unable to pick out the recurring cello melody in a Beethoven symphony. In short, there is little
3
understanding of CI users’ ability to perceive music as they might hear it on a recording or at a
concert.
In addition to providing pleasure and contributing to quality of life, music perception
skills underlie the perception of emotion in speech as well as music (Juslin & Laukka, 2003).
Emotion in speech is conveyed primarily by musically relevant cues such as loudness, tempo or
rate, rhythm, pitch height, pitch range, and pitch contours. For example, expressions of anger in
speech and music typically involve rapid tempo and increased amplitude or loudness in contrast
to expressions of sadness, which typically involve slow tempo, low pitch, and decreased
loudness. Although word recognition is obviously crucial for successful verbal communication,
it is difficult to discern a speaker’s true emotions and communicative intentions without access to
paralinguistic and prosodic cues. To date, however, there has been little research on CI users’
perception of emotion in speech and none on their perception of emotion in music.
The goal of the present study was to provide a comprehensive assessment of the music
perception skills of adult CI users who became deaf postlingually. The perception of rhythm and
pitch perception was assessed. The perception of emotion conveyed through speech and music
and pitch production were also assessed. Rhythm perception was assessed in the context of
simple rhythmic patterns as well as melodies with accompaniment. Adding accompaniment to a
simple rhythm test used previously with adult CI users (Cooper et al., 2008) made it possible to
determine whether “normal” rhythm perception skills remained evident in ecologically valid
musical contexts. Melody perception was assessed by means of tasks that required comparisons
of the musical input with long-term representations of music. The perception of emotion in
speech was assessed with a task that has been used with child CI users (Hopyan-Misakyan et al.,
2009). Although child CI users were unsuccessful at differentiating vocal emotions, it is possible
that adult CI users, by virtue of their previous access to acoustic information and their greater
understanding of communicative conventions, might be more successful than children at this
task. Finally, we tested open-set word recognition, using monosyllabic consonant-vowel-
consonant words, as a check on CI users’ use of bottom-up cues in speech.
Large individual differences are pervasive in CI outcomes. Factors influencing outcomes
among postlingually deafened adults include duration of near-total deafness (i.e., little or no
benefit from hearing aids) before implantation, with shorter durations having more favorable
4
outcomes (Van Dijk, Van Olphen, Langereis, Mens, Brokx & Smoorenburg, 1999); cognitive
abilities (Pisoni & Cleary, 2004); integrity of the auditory nerve and central auditory system
(Hartman & Kral, 2004; Leake & Rebscher, 2004); and relevant experience or training (Fu,
Nogaki, & Galvin, 2005; Galvin, Fu & Nogaki, 2007). Adults with residual hearing immediately
prior to implantation perform better on subsequent recognition of speech and environmental
sounds than those without usable residual hearing even though implantation destroys the residual
hearing (Van Dijk et al., 1999). Moreover, CI users with music training in high school, college,
or later exhibit better music perception (Gfeller et al., 2008). Based on these findings and our
own specific goals, we designed a questionnaire that could potentially shed light on individual
differences in performance. Information was solicited about education, history of hearing loss
and implantation, implant characteristics, music listening and music-making habits, and music
training. We expected CI users to perform poorly compared to NH listeners except on the test of
simple rhythm discrimination. We also expected performance to be affected by duration of
deafness before implantation, musical exposure and training, and residual hearing, if any, in the
unimplanted ear. Finally, we expected CI users to perform better on musical materials that were
highly familiar to them than on those that were less familiar or unfamiliar.
2 Method
2.1 Participants
The target participants were adult CI users (n = 6) 46-76 years of age (M = 62.2, SD = 13.0; see
Table 1) who were recruited from the Cochlear Implant Program of Sunnybrook hospital in
Toronto. All of them were postlingually deafened, they communicated solely by auditory-oral
means, and they expressed some interest in music. Additionally, they all reported progressive
hearing losses that were gradual, except for one participant. Although she experienced
substantial hearing loss when she was very young, her bilateral hearing aids were very helpful
until 6 years ago when she experienced a precipitous loss of most of her residual hearing. One
participant used a hearing aid in his unimplanted ear to amplify his residual hearing selectively at
500 and 250 Hz (90 and 70 dB thresholds, respectively). With respect to musical background,
three CI users had taken music lessons in the past, but only two were still playing music.
Table 1. Participant Characteristics
Participant M/F Age Device(s) Type of CI
Hearing loss
onset (age)
Progressive loss
(yes/no)
Hearing aid use (years)
Implant use
(years)
Music lessons (years)
Current instrument
Weekly music listening (hours)
CI-1 F 47 2 CIs Advanced
Bionics 1 yes
sudden 40 6 0 No 7 – 10
CI-2 M 46 CI + HA Cochlear 5 yes
gradual 35 5 5 Yes 10 or more
CI-3 F 67 CI Advanced Bionics 57 yes
sudden 5 5 7 No 4 – 7
CI-4 F 74 CI Advanced Bionics 35 yes
gradual 30 4 23 Yes 1 – 4
CI-5 M 76 CI +HA Med-El 58 yes gradual 12 2 0 No 1 – 4
CI-6 F 63 2 CIs Cochlear 10 yes
gradual 35 17 0 No 1 – 4
5
6
The control group consisted of normally hearing (NH) listeners (n = 12) 19-58 years of
age (M = 29.0, SD = 13.8) with no personal or family history of hearing problems. A few
participants in the control group had received music lessons as children, but only two had
substantial musical training. One of these was a professional musician.
2.2 Apparatus
Testing was conducted in a double-wall sound-attenuating chamber (Industrial Acoustics Co.,
Bronx, NY). A computer workstation and amplifier (Harmon-Kardon 3380, Stamford, CT)
outside of the booth interfaced with a 17-in touch-screen monitor (Elo LCD TouchSystems,
Berwyn, PA) and two wall-mounted loudspeakers (Electro-Medical Instrument Co., Mississauga,
ON) inside the booth. The touch-screen monitor was used for presenting instructions for all tasks
and for recording participants’ responses. The loudspeakers were mounted at the corners of the
sound booth, each located at 45 degrees azimuth to the participant, and the touch-screen monitor
was placed at the midpoint. Sound files were presented between 60 and 65 dB, according to the
preferences of each participant. One CI user (CI-2) requested sound levels up to 75 dB. CI
participants were free to alter the settings on their processor in the course of the test session.
2.3 Test Battery
Trials for the Metric Task (from Hébert & Cuddy, 2002), the Rhythmic subtest of the Montreal
Battery for Evaluation of Amusia (MBEA; Peretz, Champod & Hyde, 2003), the Distorted Tunes
Test (DTT; Drayna, Manichaikul, de Lange, Snieder & Spector, 2001), the Music Emotion Task
(Veillard, Peretz, Gosselin, Khalfa, Gagnon & Bouchard, 2007), the Diagnostic Analysis of
Nonverbal Accuracy Scale 2 (DANVA2; Nowicki & Duke, 1994; Baum & Nowicki, 1998), and
the individualized Familiar Music Task were presented via a customized program created with
Affect 4.0 (Hermans, Clarysse, Baeyens & Spruyt, 2005; Spruyt, Clarysse, Vansteenwegen,
Baeyens & Hermans, 2010). FLXLab 2.3 software (Haskell, 2009) was used to arrange the
presentation of the Word Recognition, Pitch-Matching, and Interval-Matching tasks. The entire
Clinical Assessment of Music Perception (CAMP) test, which was designed for cochlear implant
users (Kang et al., 2009), was also adminstered.
7
2.3.1 Metric task
The rhythms comprising this task were the strong-meter rhythms from Hébert and Cuddy (2002).
These rhythms were created with SoundEdit 16, version 2.0. A temporal interval was defined as
the onset-to-onset time (IOI) of successive events, with all events consisting of the sound of a
snare drum. The basic IOI was 200 ms, and IOIs varied in a 1:2:3:4 ratio, with IOIs of 200, 400,
600, and 800 ms. Each standard rhythm consisted of a different permutation of nine IOIs (five
IOIs of 200 ms, two of 400 ms, one of 600, and one of 800 ms). All tones were of equal intensity
(i.e., no amplitude accents) and duration (100 ms). To create strong metric patterns, longer IOIs
occurred on the beat. There were 4 practice trials (2 same, 2 different) with visual feedback
(correct, incorrect) provided on the monitor followed by 20 test trials (10 same, 10 different)
presented in random order with no feedback. On each trial, participants received a standard and
comparison drum pattern, and they were required to judge whether they were the same or
different. On “same” trials, the standard and comparison patterns were identical. On “different”
trials, one 400-ms IOI from the standard pattern was replaced by an 800-ms IOI. Participants
responded by touching “same” or “different” on the touch-sensitive monitor. They also touched
the monitor to proceed to the following trial, at their own pace.
2.3.2 Rhythm task
The principal modification to the Rhythmic subtest of the MBEA (Peretz et al., 2003) was the
addition of accompaniment, as described below. The test consisted of 31 trials without feedback
preceded by training trials consisting of two examples with feedback. Participants listened to
two tonal melodies and judged whether they were the same or different. Differences consisted of
alterations in the duration of two adjacent tones, which changed the rhythmic grouping but not
the meter or number of tones. Rhythmical patterns varied across melodies. The melodies spanned
a total frequency range of 247 (B3) to 988 Hz (B5), with the smallest range being 247 to 311 (E-
flat-4) Hz, and the largest range 247 to 784 (G5) Hz. Melodies had 7 to 21 notes and were 3.8 to
6.4 s in duration (M = 5.1 s), depending on the tempo (100, 120, 150, 180, and 200 bpm). Tone
durations varied from 150 to 1800 ms depending on the rhythm and tempo of each melody.
Synthesized piano versions of the melodies were used.
8
For the present purposes, accompaniment consisting of sampled bass, guitar (strummed
chords), and drum kit sounds created by means of Cakewalk Music Creator (Version 5.0.4.23;
Roland, Hamamatsu, Japan) was added to all of the melodies. Amplitude was standardized for
each instrumental track across all melodies. Participants were told that accompaniment had been
added to increase the difficulty of the task. They were asked to base their judgments of similarity
or difference entirely on the piano melody. Participants called for trials by touching the monitor
and entered their responses (same or different) on the monitor.
2.3.3 Distorted Tunes Test
This test (Drayna et al., 2001) required participants to judge whether synthesized piano
performances of 26 short melodies (12-26 notes) that are well-known in the U.K. and North
America were correct (no pitch errors) or distorted (one or more pitch errors). Of the 26 tunes, 9
were played correctly, and 17 were distorted by pitch changes (i.e., errors) in 2-9 notes, within
one or two semitones of the correct note but maintaining the melodic contour (rise and fall) of
the normal melody. The errors in the melodies resulted in out-of-key notes in all but one melody
(stimulus no. 13). All melodies in the DTT were unaltered in rhythm. The majority of tunes (17
out of 26) were played incorrectly, but there is no indication of performance differences on intact
or distorted versions (Drayna et al., 2001).
2.3.4 Musical emotion test
This task, from Veillard et al. (2007), required participants to identify the predominant emotion
conveyed by short musical excerpts as happy, sad, angry, or scary. The excerpts, representing
five of the most readily identified excerpts from each emotion category, as determined in a
preliminary study (Hunter, Schellenberg, & Stalinski, submitted), were MIDI files set to piano
timbre. The happy excerpts were in the major mode with a mean tempo of 137 beats per minute
(bpm) and the melodic line in a medium-to-high pitch range. The sad excerpts were in the minor
mode, with a mean tempo of 44 bpm, medium pitch range, and sustain pedal. The peaceful
excerpts were in the major mode, with an intermediate tempo of 69 bpm, a medium pitch range,
and also the sustain pedal. The scary excerpts had minor chords on the third and sixth degree, a
mean tempo of 95 bpm, and a low-medium pitch range. Mean stimulus duration was 13.3 s for
all emotional categories.
9
2.3.5 Diagnostic Analysis of Nonverbal Accuracy 2
The Adult Paralanguage subtest of the DANVA2 (Baum & Nowicki, 1998) assessed the ability
to perceive emotion through non-verbal speech cues. In this test, a semantically neutral sentence
(“I’m going out of the room now, but I’ll be back later”) was spoken with happy, sad, angry, or
fearful intentions at two levels of emotional intensity by a male and female actor.
2.3.6 Open-set word recognition
As a check on basic speech perception skills, CI users and NH listeners were required to repeat
20 isolated consonant-vowel-consonant (CVC) words (see Table 2) produced by a female
speaker. This task, like others in the battery, was self-administered and self-paced. Each stimulus
word was preceded by a visual warning signal on the monitor (+), and participants’ responses
were recorded.
Table 2. List of CVC words
back beach chain cup doll
fan food gum jar leg
love map meat nut pen
pig run sit sun talk
2.3.7 CAMP test
This music perception test (Kang et al., 2009) had subtests of pitch direction discrimination,
melody recognition, and timbre recognition. The pitch subtest used an adaptive procedure (1-up
1-down) to determine the threshold for pitch direction discrimination within the range of 1 to 12
semitones. On each trial, listeners indicated whether the first or second of two tones was higher
in pitch. The melody subtest assessed recognition of widely known melodies presented without
rhythmic cues (i.e., all tones of equal duration). On each trial, listeners identified the melody
10
from a set of 12 alternatives. In the timbre subtest, listeners heard a five-note sequence (the same
one on all trials) and were required to identify the instrument from a set of eight alternatives.
Stimuli for the pitch direction and melody subtests consisted of synthesized, complex tones with
uniform spectral envelopes to preclude temporal envelope cues. Stimuli for the timbre subtest
consisted of recordings of professional musicians playing real instruments. The pitch subtest was
preceded by four practice trials and the melody and rhythm subtests were preceded by training
sessions, in which participants were required to listen to each stimulus twice before beginning
the test phase.
2.3.8 Familiar music task
Stimuli for this task were personalized for CI users and NH listeners based on music that was
most familiar to them. Prior to their laboratory visit, participants provided a list of up to 10
musical selections (title, album and recording artist) that they heard regularly. Five selections
were included in the test, along with five unfamiliar selections from the same genre (as listed on
ITunes) and with similar tempi. The familiar music task had three conditions: (1) no rhythm, (2)
melody only, and (3) original instrumental. The original instrumental versions consisted of 10-s
excerpts with salient melodic content from each selection, which were extracted with Audacity
software (Version 1.3.11 Beta). If the musical selection did not have 10 s without vocal content,
the vocals were removed with Vocal Remover Version 2 plugin from Audacity. Melodic content
from all selections was transcribed to produce two monophonic WAV files per selection – a
melody version and a no-rhythm version. These excerpts were produced with a synthesized flute
timbre from Cakewalk Music Creator. In contrast to the melody version, which maintained the
rhythm, the no-rhythm version was isochronous (i.e., all tones of equal duration). The original
pitch durations were maintained in the no-rhythm version by means of repeated tones at the
pitches in question. On each trial, participants listened to the selection and identified it from a set
of six alternatives, which consisted of the five familiar musical pieces and “none of the above.”
The conditions were administered in fixed order from most to least difficult: (1) no rhythm; (2)
melody; and (3) original instrumental.
11
2.3.9 Pitch- and interval matching
The stimuli for this task consisted of eight pitches 1-2 s in duration sung by a man and woman
and eight ascending intervals sung by the same individuals in a legato (continuous or
uninterrupted) manner. The male stimuli ranged from B3 (246.942 Hz) to B4 (493.883 Hz), and
the female stimuli ranged from B4 to B5 (987.767 Hz). Each pitch and interval stimulus was
presented twice in a predetermined order, with the pitch-matching task presented first.
Participants were required to sing back what they heard, and their responses were recorded by
means of FLXLab. The intervals always began on the first degree of the scale (B3 for male
stimuli and B4 for female stimuli). Only pitches from the key of B major were used, which
resulted in the following intervals: unison, octave, major 2nd, 3rd, 6th and 7th, and perfect 4th
and 5th. Pitches and intervals of the imitations were calculated by means of Praat software
(Version 5.1.43; Boersma & Weenink, 2010).
2.4 Procedure
Prior to their laboratory visit, implant users completed a questionnaire (see Appendix A) that
included information about demographic background (e.g., history of hearing loss, implant
experience, education, languages spoken), musical background (e.g., musical training, music
listening habits before and after their hearing loss, music enjoyment) and familiar musical
selections. NH participants completed a questionnaire about their musical background and
subjective experience of music (see Appendices B and C) just before the test session.
Test sessions with CI users began with a semi-structured interview designed to elicit
information about their subjective experience of music (see Appendix D). All interviews were
recorded with a Sony Net MD Walkman (MZ-N707 model) and a Sony electret condenser
microphone (ECM-DS70P model). Once the interview was completed, participants were
escorted to the sound-attenuating booth for administration of the test battery. The experimenter
provided instructions before each component of the battery. These instructions were repeated on
the touch-screen monitor prior to each task. Participants were also told that the sounds could be
made louder or softer, according to their preference. Tasks were presented in fixed order.
Participants were told that the pitch- and interval-matching tasks, which were the in the test
battery, were strictly optional.
12
3 Results and Discussion
Due to the small sample size and large individual differences among CI users, we examined
performance individually for each task, noting the CI users who performed within one SD of the
mean for NH listeners, those who performed within two SDs, and so on. On the basis of previous
research, CI users were expected to perform much better on tests of rhythm and meter and on
other tasks based on timing cues than on those based on pitch cues (Cooper et al., 2008; Kang et
al., 2009; Kong et al., 2004).
3.1 Open-Set Word Recognition
As one would expect, performance of the NH group was at ceiling (see Figure 1) such that there
was no variance in the data. CI users also performed well on this task, with scores ranging from
100% correct (CI-2) to a low of 20% correct (CI-4). Performance on these isolated monosyllabic
words sheds light on how well each CI user was using bottom-up cues that are relevant to speech
perception. It should be emphasized, however, that the CI participants were uniformly excellent
at perceiving speech in quiet backgrounds when contextual cues were available, as confirmed in
lengthy, individual interviews. Variability on open-set recognition tasks has been reported in a
number of other studies (Loizou, Poroy, & Dormann, 2000; Vandali, Whitford, Plant, & Clark,
2000). CI-4, who had the poorest performance on this task, had the longest delay from the time
her hearing aids became ineffectual (i.e., no usable residual hearing) until implant surgery. The
top performer, CI-2, had a number of advantages, including professional knowledge of hearing
and assistive technologies as well as residual low-frequency hearing (at 250 and 500 Hz) in his
unimplanted ear, which was selectively amplified. Zhang, Dorman, and Spahr (2010) have
documented the contribution of low-frequency acoustic hearing to the recognition of
monosyllabic words.
3.2 CAMP
Pitch-detection thresholds are illustrated in Figure 2, whereas melody recognition and timbre
recognition are illustrated in Figures 3 and 4, respectively. The mean threshold for pitch-
direction identification for NH listeners was 1.3 semitones (SD = 0.8), whereas their average on
the melody-recognition task was 88.0% (SD = 10.0%) and their average result on the timbre-
recognition task was 85.6% (SD = 16.0%). The means for the CI users group were 4.6 semitones
13
Figure 1. Mean score and standard deviation on the Speech Perception Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users.
Figure 2. Mean score and standard deviation on the CAMP Pitch Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users.
14
Figure 3. Mean score and standard deviation on the CAMP Melody Recognition Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
Figure 4. Mean score and standard deviation on the CAMP Timbre Recognition Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
15
(SD = 2.7) for the pitch-ranking task, 25.7% correct for the melody task (SD = 25.5), and 45.8%
for the timbre task (SD = 19.5). Results for both groups were very similar to those reported
previously by the developers of the test (Kang et al., 2009). Because the CAMP tests examine
the ability to perceive pitch and timbre cues, it is not surprising that most of the CI users did not
do well on these tasks.
Two CI users (CI-1 and CI-2) managed to perform particularly well on pitch-direction
identification, falling within one SD of the mean of the NH group. The Melody task, which
excluded all timing cues, proved to be more difficult. In fact, two CI users (CI-5 and CI-6) opted
to discontinue the task because of its extreme difficulty. Moreover, no CI user was able to obtain
a score within two SDs of the NH mean, although CI-2’s performance was substantially better
than that of the other CI users. His score was 63.9%, whereas the average of the scores of the
three other CI users was 13.0%. CI-2 also scored much higher than other CI users on the timbre
identification task, obtaining a score of 83.3% correct, which was near the NH mean. None of the
other CI users had a score within two SDs of the NH mean, although CI-3 came close. The
amplified residual hearing of CI-2 undoubtedly accounts for his success and for his ability to
play in a musical ensemble. The contribution of hearing aids in the unimplanted ear to music
perception has been noted previously (Looi, McDermott, McKay, & Hickson, 2007; Turner,
Reiss & Gantz, 2008).
3.3 Distorted Tunes Test
The DTT comprised 26 questions and two response options on each trial, such that chance
performance was a score of 13. As in the original study that used the DTT (Drayna et al., 2001)
and an additional study by the same research team (Jones et al., 2009), the scores of NH listeners
were near ceiling (M = 24.7, SD = 1.3; see Figure 5). Because the DTT comprises traditional
North-American folk melodies, our CI users, who were on average much older than our control
group (mean age of 62.7 vs 29.0), would have been more familiar with these melodies before
they became deaf. Nonetheless, CI users had extreme difficulty on this task. Their mean score
was 11.8 correct (SD = 2.8), and scores for all of the CI users were more than two SDs below the
NH mean and near chance levels. In fact, the highest score was 16 correct (CI-2). Because
mistunings on the DTT (except for one) are created by using pitches outside the key of each
melody, the findings indicate that CI users are unable to use tonality-related cues when
16
perceiving music. This interpretation is consistent with previous performance of CI users on the
Scale subtest of the MBEA, which was at chance levels (Cooper et al., 2008).
It is notable that CI-2, despite his good pitch resolution and reasonable performance on
other tasks, was unable to do this task, which involved comparing current melodies with long-
term representations of those melodies or making judgments based on tonality. As noted, the
pitch errors in this test are relatively small (one or two semitones). Considering the mean score
of CI users on the CAMP pitch-ranking task (threshold of 4.6 semitones), it is not surprising that
CI users were unable to perceive mistunings in the DTT melodies. The authors of the DTT
created these errors to be salient by virtue of their violations of tonality. Thus, it is not surprising
that these violations are not salient to CI users.
Figure 5. Mean score and standard deviation on the Distorted Tunes Test for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
17
3.4 Familiar Music Task
Scores for this task were the number of correct answers out of 10, converted into percent correct
scores. It was possible to generate individualized materials for only five participants from the
NH group. Mean scores were 74.0% (SD = 15.2%) in the No-rhythm condition (Figure 6),
92.0% (SD = 4.5%) in the Melody (with timing cues) condition (Figure 7), and 94.0% (SD =
5.5%) in the Instrumental condition (Figure 8), which featured all or most cues from the original
recordings, except for the lyrics for selections involving songs. CI users’ scores were
exceedingly low. Moreover, they were lowest in the No-Rhythm condition (M = 26.7%, SD =
25.2%), slightly higher in the Melody condition (M = 35.0%, SD = 19.1%), and highest in the
Instrumental condition (M = 70.0%, SD = 18.3%). Two CI users were excluded from
consideration because they provided artists with whom they were familiar (e.g., Louis
Armstrong, Frank Sinatra) but no specific musical selections. Of the four remaining CI users,
one discontinued the No-Rhythm condition because of its difficulty. CI-1 and CI-2 scored below
two SDs of the NH mean for this condition. Although CI-4 managed to score within two SDs of
Figure 6. Mean score and standard deviation on the No-Rhythm Condition of the Familiar Music Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users.
18
the NH mean, she did so only by responding “none of the above” for all of the trials. Obviously,
she was unable to recognize any melodies without rhythmic cues. Although CI users fared better
in the Melody condition than in the No-Rhythm condition, all four failed to score within two SDs
of the mean for NH listeners. In the Instrumental condition, CI-2 obtained a score similar to the
NH mean (90.0% versus 94.0%, respectively). Although CI-4 and CI-1 obtained higher scores in
the Instrumental condition than in the other two conditions, they were still more than two SDs
below the NH mean.
Figure 7. Mean score and standard deviation on the Melody Condition of the Familiar Music Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users.
19
The Familiar Music Task was created specifically for this study. The expectation was that
the use of highly familiar music would generate better performance than one would predict based
on the available literature. In fact, CI children have shown some success in the recognition of
specific recordings that they hear regularly (Vongpaisal, Trehub & Schellenberg, 2006 & 2009)
even though such children are generally unsuccessful at recognizing generic versions of
culturally familiar tunes (Olszweski, Gfeller, Froman, Stordahl, & Tomblin, 2005; Stordahl,
2002). However, this was not the case for the current group of adult CI users. Of the six CI users
in the present study, three (CI-1, CI-5 and CI-6) reported in their interview that the lyrics were
the most salient part of their music listening experiences. However, lyrics were excluded from
the test materials, even when recordings with vocals were selected as familiar music, because
they provided obvious cues to the identity of the music.
CI-4, who listened to classical music and attended concerts frequently, was unable to
recognize the original recordings (same music and performers) that she heard regularly. CI-6 was
also unable to recognize the four instrumental pieces that were among her 10 selections, which
suggests that her at-home listening experiences are guided and enriched by knowledge of what
Figure 8. Mean score and standard deviation on the Instrumental Condition of the Familiar Music Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users.
20
she is playing. CI-2, the “star” performer in the present study, indicated that he listens especially
closely to the bass line in music. This follows, perhaps, from programming his hearing aid to
capitalize on his residual low-frequency hearing. CI-2 is also a bass player who performs with an
amateur blues/rock group. It is nevertheless impressive that this participant was as proficient as
NH listeners at identifying the familiar instrumental excerpts.
During her interview, participant CI-6 shed light on factors contributing to her musical
preferences. She stated that, in order to enjoy music, it had to have meaning, such as a narrative.
For example, she very much enjoyed the lyrics in a number of the selections she submitted.
Although she also selected instrumental pieces, some of them were orchestral works with
underlying narratives. For example, Symphony No. 11 by Dmitri Shostakovich, entitled “In The
Year 1905,” depicts the Russian revolution. Another of her selections, the orchestral work
“Finlandia” by Jean Sibelius, depicts the Finnish struggle to break free from the Russian empire.
Because CI users do not have access to the acoustic details available to NH listeners, they may
find other ways of enjoying music. The enjoyment of CI-6 was enriched by a narrative linked to
the overall structure of the musical work rather than its melodies or harmonies. CI-6 described
hearing the Cossacks charging on their horses in the work by Shostakovich, and the struggles and
the triumph of the Finnish people in the Sibelius piece.
Another factor that may have contributed to CI users’ difficulty at identifying the
material was the 10-second duration of the excerpts, which posed no problem for NH listeners. It
is possible that they would be somewhat more successful with longer samples of the music.
3.5 Metric Task & Modified MBEA Rhythm Task
Because the Metric task comprised 20 questions and each trial had two response options,
chance responding was a score of 10. The mean of the NH group was 17.1 (SD = 3.4; see Figure
9), which is similar to the mean of a NH group tested on the same task in a previous study
(Hopyan, Schellenberg & Dennis, 2009). CI-1 received a perfect score on this task. CI-2 and CI-
3 scored within one SD below the NH mean, and CI-4 and CI-5 scored within two SDs. CI-6 was
below two SDs and also below chance levels for this task. In short, the majority of CI users (5 of
6) were within two SDs of the mean for NH listeners, which is in line with CI users’ previous
success in discriminating simple rhythmic patterns (Gfeller et al., 1997; Kong et al., 2004).
21
The modified subtest of the MBEA had 31 trials and two response options on each trial,
such that chance performance was a score of 15.5. The NH group mean was 26.9 (SD = 3.6, see
Figure 10), which is virtually identical to a sample of individuals with normal music perception
skills reported by Peretz et al. (2003) who were tested on a similar task without accompaniment
(M = 27.0, SD = 2.1). Similar performance across studies indicates that the additional
instrumentation created for the purpose of this study did not impair the performance of NH
listeners. By contrast, the average performance of CI users on our modified task was only 17.0
(SD = 2.1), which is substantially lower than the mean obtained by CI users tested by Cooper et
al. (2008; approximately 24 correct) on the original MBEA rhythm task. Although CI-2 was
slightly less than two SDs below the NH mean, the other CI users were near or below chance
levels, which confirms that the additional instrumentation impeded their ability to perceive the
rhythm of the melodic line.
Figure 9. Mean score and standard deviation on the Metric Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
22
Although CI users fared as well as NH listeners on the original version of this rhythm
discrimination task, which involved monophonic piano melodies (Cooper et al., 2008), their
rhythm discrimination was impaired when there were multiple streams of auditory information.
In fact, almost all of the CI users performed near chance level. This finding suggests that CI
users would have difficulty discerning the rhythms encountered in their everyday experience of
music.
3.6 Music Emotion & DANVA2
The Music Emotion task comprised 20 questions and 4 response options on each trial, such that a
score of five correct responses corresponded to chance responding. Once again, NH listeners
were near ceiling on this task (M = 19.2, SD = 1.4; see Figure 11), which is slightly higher than
the results reported by Hunter et al. (submitted) for adult listeners, who had an average of 16.7
correct. All CI users were more than two SDs below the NH mean, with a mean of 12.7 (SD =
3.6).
Figure 10. Mean score and standard deviation on the Modified MBEA Rhythm Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
23
Because CI users are better able to perceive timing cues than pitch cues, we examined the
possibility that CI users could interpret arousal, which is based largely on tempo cues, better than
valence, which is based on mode (major/minor) and consonance/dissonance cues. Thus, we
combined the response options based on arousal: happy or scary vs. sad or peaceful (see Table 3
for arousal scores). For three of the CI users (CI-2, CI-4 and CI-5), a majority of the errors (over
50%) on this task involved confusions between stimuli that contrasted in valence but were
similar in arousal. These findings suggest that tempo cues play a substantially greater role than
mode cues in CI users’ perception of emotion in music. This interpretation is consistent with
reports of adequate tempo perception in CI users (Kong et al., 2004). Tempo cues are also more
important than mode cues for young children (Dalla Bella, Peretz, Rousseau, & Gosselin, 2001),
not because of pitch resolution difficulties but because they have not yet learned Western
musical conventions about mode.
The DANVA2 comprised 24 trials and four response options on each trial (happy, angry,
sad, and fearful) such that a score of six correct corresponded to chance responding. The mean
for the NH listeners was 19.3 (SD = 2.3; see Figure 12), which is similar to the mean reported by
Nowicki (2006; M = 18.0, SD = 2.9). The average score for the CI users was only 10.8 (SD =
3.3). Only CI-2 and CI-6 performed within two SDs of the NH mean, with the remaining CI
users having lower scores and three performing at close to chance levels (CI-3, CI-4, CI-5).
Performance on the DANVA2 by child CI implant users in the study by Hopyan-Misakyan et al.
(2009) was similar to adult CI users in the present study in that both groups were unsuccessful in
differentiating the vocal emotions.
The DANVA2, which has been used widely (Nowicki, 2006), is intended to be a
challenging test, with average NH scores ranging from 14 to 18.5 out of 24. Among its
advantages is that it allows specially gifted individuals to achieve higher scores than the
population mean. However, this test may not be the most appropriate means of assessing CI
users’ access to emotion cues in speech. A test involving a greater range of emotional
expressiveness would enable us to learn more about this skill in CI users.
24
Table 3. Music Emotion Arousal Scores Participant Test score (20) Modified arousal score (20) Valence errors
CI-1 17 18 33% CI-2 13 18 71% CI-3 15 17 40% CI-4 14 18 67% CI-5 7 12 38% CI-6 10 16 60%
Figure 11. Mean score and standard deviation on the Music Emotion Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
25
3.7 Pitch- and Interval-Matching Task
Only CI users were asked to complete the pitch- and interval-matching tasks, which were
described as strictly optional. With the exception of one CI user, who was short of time, all
agreed to complete the matching tasks. The overwhelming majority of NH individuals can match
pitches within one semitone (Moore, Estis, Gordon-Hickey, & Watts, 2008). For CI users, the
mean error in pitch matching (Figure 13) was 3.9 semitones (SD = 3.1). Only CI-2 performed
within the expected range of NH listeners, with a mean pitch error of 1.1 semitones. Performance
in the interval-matching task (Figure 14) was similar. Errors on interval matching (Figure 14)
were comparable to those on pitch matching (M = 3.1 semitones, SD = 2.0). Again, CI-2
performed surprisingly well, with a mean error of 1.0 semitones on interval matching, which is in
line with his low pitch-ranking threshold on the CAMP test (1.6 semitones).
Figure 12. Mean score and standard deviation on the DANVA2 Adult Vocal Emotion Task for normally hearing (NH) listeners and individual scores for cochlear implant (CI) users. The dotted line represents chance performance.
26
Figure 13. Individual average pitch deviations in semitones on the Pitch-Matching Task for cochlear implant (CI) users.
Figure 14. Individual average pitch deviations in semitones on the Interval-Matching Task for cochlear implant (CI) users.
27
3.8 Conclusion
In sum, postlingually deafened adult CI users performed well below the level of the NH control
group on most tasks in the present study. Their performance was especially poor on tasks that
relied strongly on pitch cues, such as the DTT, isochronous melody tasks, familiar melody task,
pitch ranking, and pitch matching. They had more success on the simple rhythm discrimination
task but not on the more complex rhythm discrimination task. They also had poor results on the
emotion discrimination tasks, which required the joint use of pitch and timing cues.
As in most studies of CI users, there were large individual differences in performance.
CI-2 performed considerably better than other CI users, especially on the pitch-ranking and
pitch-matching tasks. Although his musical background may have played some role, it is likely
that amplified residual hearing in his unimplanted ear made the most important contributions to
his success on the tasks involving pitch. Along with musical training and residual hearing, CI-2
had the further advantage of formal training in audiology and familiarity with hearing aid
technology. As he put it, he programmed his own hearing aid to “act like a subwoofer,” which
enables him to maximize his perception of music and speech. In his interview, CI-2 indicated
that neither his implant nor his hearing aid alone provided a satisfactory representation of sound
but together they provided a credible and highly enjoyable rendition of music. In short, the whole
was a lot better than the sum of its parts. Plans for re-testing CI-2 with his implant alone will
provide a clearer picture of the independent contributions of implant and hearing aid.
CI-4 had extensive musical training (piano) and even considered a career as a musician
when she was a young woman with normal hearing. Her progressive hearing loss over the years
and a long period of very poor auditory reception with hearing aids seemed to erase any potential
benefit from her training and knowledge of music. For CI-2, by contrast, gradual hearing loss
began at about 5 years of age and his hearing aids functioned effectively for music listening until
approximately five years before receiving his implant.
Plans to enlarge the sample will make it possible to identify links between various
background variables and performance on music processing tasks such as these. It would be of
interest to determine whether limited training enhances music processing in CI users and their
28
ability to derive pleasure from music. Such training may also have favorable consequences for
other auditory but non-musical tasks. These questions can be addressed in future research.
29
References
Baum, K., & Nowicki, S., Jr. (1998). Perception of emotion: Measuring decoding accuracy of
adult prosodic cues varying in intensity. Journal of Nonverbal Behavior, 22, 89 – 109.
Boersma, P., & Weenink, D. (2010). Praat: Doing phonetics by computer [computer software].
Retrieved August 17, 2010, from http://www.fon.hum.uva.nl/praat/download_win.html.
University of Amsterdam, The Netherlands.
Cooper, W.B., Tobey, E., & Loizou, P.C. (2008). Music perception by cochlear implant and
normal hearing listeners as measured by the Montreal Battery for Evaluation of Amusia.
Ear & Hearing, 29, 618 – 626.
Dalla Bella, S., Giguère, J.-F., & Peretz, I. (2009). Singing in congenital amusia. Journal of the
Acoustical Society of America, 126, 414 – 424.
Dalla Bella, S., Peretz, I., Rousseau, L. & Gosselin, N. (2001). A developmental study of the
affective value of tempo and mode in music. Cognition, 80, B1 – B10.
Drayna, D., Manichaikul, A., de Lange, M., Snieder, H., & Spector, T. (2001). Genetic
correlates of musical pitch recognition in humans. Science, 291, 1969 – 1972.
Fetterman, B.L., & Domico, E.H. (2002). Speech recognition in background noise of cochlear
implant patients. Otolaryngology – Head and Neck Surgery, 126, 257 – 263.
Firszt, J.B., Holden, L.K., Skinner, M.W., Tobey, E.A., Peterson, A., Gaggl, W., et al. (2004).
Recognition of speech presented at soft to loud levels by adult cochlear implant recipients
of three cochlear implant systems. Ear & Hearing, 25, 375 – 387.
Foxton, J.M., Nandy, R.K., & Griffiths, T.D. (2006). Rhythm deficits in ‘tone deafness.’ Brain
and Cognition, 62, 24 – 29.
Fu, Q.-J., Nogaki, G., & Galvin, J.J., III (2005). Auditory training with spectrally shifted speech:
Implications for cochlear implant patient auditory rehabilitation. Journal for the
Association of Research in Otolaryngology, 6, 180 – 189.
30
Galvin, J.J., III, Fu, Q.-J., & Nogaki, G. (2007). Melodic contour identification by cochlear
implant listeners. Ear & Hearing, 28, 302 – 319.
Galvin, J.J., III, Fu, Q.-J., & Shannon, R.V. (2009). Melodic contour identification and music
perception by cochlear implant users. Annals of the New York Academy of Sciences,
1169, 518 – 533.
Gfeller, K., Christ, A., Knutson, J.F., Witt, S., Murray, K.T., & Tyler, R.S. (2000). Musical
backgrounds, listening habits and aesthetic enjoyment of adult cochlear implant
recipients. Journal of the American Academy of Audiology. 11, 390 – 406.
Gfeller, K., Christ, A., Knutson, J.F., Woodworth, G., Witt, S. & DeBus, B. (1998). Timbral
recognition and appraisal by adult cochlear implant users and normal-hearing adults.
Journal of the American Academy of Audiology, 9, 1 – 19.
Gfeller, K., Turner, C., Oleson, J., Zhang, X., Gantz, B., Froman, R., et al. (2007). Accuracy of
cochlear implant recipients on pitch perception, melody recognition, and speech reception
in noise. Ear & Hearing, 28, 412 – 423.
Gfeller, K., Witt, S., Stordahl, J., Merh, M., & Woodworth, G. (2000). The effects of training on
melody recognition and appraisal by adult cochlear implant recipients. Journal of the
Academy of Rehabilitative Audiology, 33, 115 – 138.
Gfeller, K., Woodworth, G., Robin, D. A., Witt, S., Knutson, J.F. (1997). Perception of rhythmic
and sequential pitch patterns by normally hearing adults and adult cochlear implant
users. Ear & Hearing, 18, 252 – 260.
Hartmann, R., & Kral, A. (2004). Central Responses to Electrical Stimulation. In F.-G. Zeng,
A.N. Popper, & R.R. Fay (Eds.), Cochlear Implants: Auditory Prostheses and Electric
Hearing (pp. 213 – 285). New York: Springer.
Haskell, T.R. (2009). FLXLab 2.3 [computer software and manual]. Retrieved February 28,
2010, from http://xlab.sourceforge.net
31
Hébert, S., & Cuddy, L.L. (2002). Detection of metric structure in auditory figural patterns.
Perception & Psychophysics, 64, 909 – 918.
Hermans, D., Clarysse, J., Baeyens, F., & Spruyt, A. (2005). Affect Version 4.0 [Computer
software]. Retrieved February 4, 2010, from http://www.psy.kuleuven.ac.be/leerpsy/
affect4. University of Leuven, Belgium.
Hopyan, T., Schellenberg, E.G., & Dennis, M. (2009). Perception of strong-meter and weak-
meter rhythms in children with spina bifida meningomyelocele. Journal of the
International Neuropsychological Society, 15, 521 – 528.
Hopyan-Misakyan, T. M., Gordon, K. A., Dennis, M., & Papsin, B.C. (2009). Recognition
of affective speech prosody and facial affect in deaf children with unilateral right
cochlear implants. Child Neuropsychology, 15, 136 – 146.
Hunter, P.G., Schellenberg, E.G., & Stalinski, S.M. (submitted). Emotions expressed by music:
Age and gender difference in identification accuracy and preferences.
Jones, J.L., Lucker, J., Zalewski, C., Brewer, C., & Drayna, D. (2009). Phonological processing
in adults with deficits in musical pitch recognition. Journal of Communication Disorders,
42, 226 – 234.
Juslin, P.N., & Laukka, P. (2003). Communication of emotions in vocal expression and music
performance: Different channels, same code? Psychological Bulletin, 129, 770 –814.
Kang, R., Nimmons, G.L., Drennan, W., Lognion, J., Ruffin, C., Nie, K., et al. (2009).
Development and validation of the University of Washington Clinical Assessment
of Music Perception test. Ear & Hearing, 30, 411–418.
Kong,Y.-Y., Cruz, R., Ackland-Jones, J., & Zeng, F.-G. (2004). Music perception with temporal
cues in acoustic and electric hearing. Ear & Hearing, 25, 173 – 185.
McDermott, H.J., & Looi, V. (2004). Perception of complex signals, including musical sounds,
with cochlear implants. International Congress Series, 1273, 201–204.
32
Lassaletta, L., Castro, A., Bastarrica, M., Pe´Rez-Mora, R., Madero, R., De Sarria, J., et al.
(2007). Does music perception have an impact on quality of life following cochlear
implantation? Acta Oto-Laryngologica, 127, 682 – 686.
Laukka, P. (2006). Uses of music and psychological well-being among the elderly. Journal of
Happiness Studies, 8, 215 – 241.
Leake, P.A., & Rebscher, S.J. (2004). Anatomical Considerations and Long-Term Effects of
Electrical Stimulation. In F.-G. Zeng, A.N. Popper, & R.R. Fay (Eds.), Cochlear
Implants: Auditory Prostheses and Electric Hearing (pp. 101 – 148). New York:
Springer.
Leal, M.C., Shin, Y.J., Laborde, M.-L., Calmels, M.-N., Verges, S., Lugardon, S., et al. (2003).
Music Perception in Adult Cochlear Implant Recipients. Acta Oto- Laryngologica, 123,
826 – 835.
Loizou, P.C., Poroy, O., & Dormann, M. (2000). The effect of parametric variations of cochlear
implant processors on speech understanding. Journal of the Acoustical Society of
America, 108, 790 – 802.
Looi, V., McDermott, H., McKay, C., & Hickson, L. (2004). Pitch discrimination and melody
recognition by cochlear implant users. International Congress Series, 1273, 197 – 200.
Looi, V., McDermott, H., McKay, C., & Hickson, L. (2007). Comparisons of quality ratings for
music by cochlear implant and hearing aid users. Ear & Hearing, 28, 59S – 61S.
Looi, V., McDermott, H., McKay, C., & Hickson, L. (2008). The effect of cochlear implantation
on music perception by adults with usable pre-operative acoustic hearing. International
Journal of Audiology, 47, 257 – 268.
Looi, V., & She, J. (2010). Music perception of cochlear implant users: A questionnaire, and its
implications for a music training program. International Journal of Audiology, 49, 116–
128.
33
Meister, H., Landwehr, M., Pyschny, V., Walger, M., & Von Wedel, H. (2009). The perception
of prosody and speaker gender in normal-hearing listeners and cochlear implant
recipients. International Journal of Audiology, 48, 38 – 48.
Moore, R.E., Estis, J., Gordon-Hickey, S., & Watts,C. (2008). Pitch discrimination and pitch
matching abilities with vocal and nonvocal Stimuli. Journal of Voice, 22, 399 – 407.
Nakata, T., Trehub, S.E., Mitani, C. and Kanda, Y. (2006). Pitch and timing in the songs of deaf
children with cochlear implants. Music Perception, 24, 147– 154.
Nowicki, S., Jr. (2006). Manual for the receptive tests of the Diagnostic Analysis of Nonverbal
Accuracy 2 (DANVA2). Retrieved on March 28, 2010 from
http://psychology.emory.edu/ clinical/interpersonal/DANVAmanual03.doc
Nowicki, S., Jr., & Duke, M. P. (1994). Individual differences in the nonverbal communication
of affect: The Diagnostic Analysis of Nonverbal Accuracy Scale. Journal of Nonverbal
Behavior, 18, 9 – 35.
Olszweski, C., Gfeller, K., Froman, R., Stordahl, J., & Tomblin, B. (2005). Familiar melody
recognition by children and adults using cochlear implants and normal hearing children.
Cochlear Implants International, 6, 123 – 140.
Peretz, I., Champod, A.S., & Hyde, K. (2003). Varieties of Musical Disorders: The Montreal
Battery of Evaluation of Amusia. New York Academy of Sciences, 999, 58 – 75.
Pisoni, D.B., & Cleary, M. (2004). Learning, Memory, and Cognitive Processes in Deaf Children
Following Cochlear Implantation. In F.-G. Zeng, A.N. Popper, & R.R. Fay (Eds.),
Cochlear Implants: Auditory Prostheses and Electric Hearing (pp. 334 – 376). New
York: Springer.
Shannon, R.V., Zeng, F.-G., Wygonski, J., Kamath, V., & Ekelid, M. (1995). Speech recognition
with primarily temporal cues. Science, 270, 303 – 304.
34
Spruyt, A., Clarysse, J., Vansteenwegen, D., Baeyens, F., & Hermans, D. (2010). Affect 4.0: A
free software package for implementing psychological and psychophysiological
experiments. Experimental Psychology, 57, 36 – 45.
Stordahl, J. (2002). Song recognition and appraisal: A comparison of children who use cochlear
implants and normally hearing children. Journal of Music Therapy, 39, 2 – 19.
Turner, C.W., Reiss, L. & Gantz, B.J. (2008). Combined acoustic and electric hearing:
Preserving residual acoustic hearing. Hearing Research, 242, 164 – 171.
Van Dijk, J.E., Van Olphen, A.F., Langereis, M.C., Mens, L.H., Brokx, J.P., & Smoorenburg,
G.F. (1999). Predictors of cochlear implant performance. Audiology, 38, 109 – 116.
Vandali, A. E., Whitford, L. A., Plant, K. L., and Clark, G. M. (2000). Speech perception as a
function of electrical stimulation rate: Using the Nucleus 24 cochlear implant system. Ear
& Hearing, 21, 608 – 624.
Veekmans, K., Ressel, L., Mueller, J., Vischer, M., & Brockmeier, S.J. (2009). Comparison of
music perception in bilateral and unilateral cochlear implant users and normal-hearing
subjects. Audiololgy and Neurotology, 14, 315 – 326.
Veillard, S., Peretz, I., Gosselin, N., Khalfa, S., Gagnon, L., & Bouchard, B. (2007). Happy, sad,
scary and peaceful musical excerpts for research on emotions. Cognition and Emotion,
22, 720 – 752.
Vongpaisal, T., Trehub, S.E., & Schellenberg, E.G. (2006). Song recognition by children and
adolescents with cochlear implants. Journal of Speech, Language, and Hearing Research,
49, 1091–1103.
Vongpaisal, T., Trehub, S.E., & Schellenberg, E.G. (2009). Identification of TV tunes by
children with cochlear implants. Music Perception, 27, 17–24.
Wilson, B. S. (2000). Cochlear Implant Technology. In J.K. Niparko, K.I. Kirk, N.K. Mellon,
A.M. Robbins, D.L. Tucci, & B.S. Wilson (Eds.), Cochlear implants: Principles &
Practices (pp. 109 – 118). New York: Lippincott, Williams & Wilkins.
35
Wilson, B.S. (2004). Engineering Design of Cochlear Implants. In F.-G. Zeng, A.N. Popper, &
R.R. Fay (Eds.), Cochlear Implants: Auditory Prostheses and Electric Hearing (pp. 14 –
52). New York: Springer.
Xu, L., Zhou, N., Chen, X., Li, Y., Schultz, H.M., Zhao, X., et al. (2009). Vocal singing by
prelingually-deafened children with cochlear implants. Hearing Research, 255, 129 –134.
Zhang, T., Dorman, M.F, & Spahr, A.J. (2010). Information from the voice fundamental
frequency (F0) region accounts for the majority of the benefit when acoustic stimulation
is added to electric stimulation. Ear & Hearing, 31, 63 – 69.
36
Appendix
Appendix A
MUSIC & COCHLEAR IMPLANTS QUESTIONNAIRE NAME: DATE: GENDER (M/F): AGE: E-MAIL: TEL: COUNTRY OF BIRTH: FIRST LANGUAGE: HIGHEST EDUCATION ACHIEVED: CAUSE OF HEARING LOSS: AGE AT DIAGNOSIS: DID YOU USE HEARING AIDS PRIOR TO IMPLANT SURGERY? IF SO, FOR HOW LONG? (YEARS): PROGRESSIVE LOSS? (YES/NO): TYPE OF COCHLEAR IMPLANT: PROCESSING STRATEGY: AGE AT SURGERY: ONE IMPLANT OR TWO: IF ONE, HEARING AID IN OPPOSITE EAR? (YES/NO):
37
MUSIC & COCHLEAR IMPLANTS
A. HOURS OF MUSIC LISTENING PER WEEK PRIOR TO HEARING LOSS: (1) 0 hours; (2) 1 – 4 hours; (3) 4 – 7 hours; (4) 7 – 10 hours; (5) 10 hours or more. ANSWER: B. HOURS OF MUSIC LISTENING PER WEEK AFTER HEARING LOSS BUT PRIOR TO IMPLANT SURGERY: (1) 0 hours; (2) 1 – 4 hours; (3) 4 – 7 hours; (4) 7 – 10 hours; (5) 10 hours or more. ANSWER: C. HOURS OF MUSIC LISTENING PER WEEK SINCE IMPLANT SURGERY: (1) 0 hours; (2) 1 – 4 hours; (3) 4 – 7 hours; (4) 7 – 10 hours; (5) 10 hours or more. ANSWER: D1. HAVE YOU EVER PLAYED A MUSICAL INSTRUMENT AND / OR SUNG REGULARLY? (YES/NO): D2. IF YES, WHICH ONE(S), AND FOR HOW MANY YEARS DID YOU PLAY AND / OR SING REGULARLY? (e.g., piano, 3 years; voice, 5 years)
Instrument Years
D3. IF YOU ARE NO LONGER PLAYING AN INSTRUMENT AND/OR SINGING, HOW LONG HAS IT BEEN SINCE YOU LAST PLAYED AN INSTRUMENT AND/OR SANG REGULARLY? (YEARS):
38
E1. HAVE YOU EVER TAKEN MUSIC LESSONS? (YES/NO): E2. IF YES, FOR HOW MANY YEARS? (e.g., piano, 3 years; guitar, 5 years):
Instrument Years E3. IF YOU ARE NO LONGER TAKING LESSONS, HOW LONG HAS IT BEEN SINCE YOUR LAST LESSON? (YEARS): F1. IF YOU PLAYED AN INSTRUMENT AND/OR SANG IN THE PAST, WHAT WAS THE AVERAGE NUMBER OF HOURS A WEEK THAT YOU PLAYED AND/OR SANG PRIOR TO HEARING LOSS? (1) 1 – 3 hours; (2) 3 – 6 hours; (3) 6 – 9 hours; (4) 9 – 12 hours; (5) 12 hours or more. ANSWER: F2. HOW LONG DID YOU MAINTAIN THIS INTENSITY OF PLAYING/SINGING? (YEARS): G1. IF YOU PLAYED AN INSTRUMENT AND/OR SANG IN THE PAST, ON AVERAGE, HOW MANY HOURS A WEEK DID YOU PLAY AND/OR SING AFTER HEARING LOSS BUT PRIOR TO IMPLANT SURGERY? (1) 1 – 3 hours; (2) 3 – 6 hours; (3) 6 – 9 hours; (4) 9 – 12 hours; (5) 12 hours or more. ANSWER: G2. HOW LONG DID YOU MAINTAIN THIS RATE OF PLAYING/SINGING? (YEARS): H1. IF YOU CURRENTLY PLAY AN INSTRUMENT AND/OR SING, HOW MANY HOURS A WEEK DO YOU PLAY OR SING? (1) 1 – 3 hours; (2) 3 – 6 hours; (3) 6 – 9 hours; (4) 9 – 12 hours; (5) 12 hours or more. ANSWER: H2. HOW LONG HAVE YOU MAINTAINED THIS RATE? (YEARS):
39
I. DID YOU PARTICIPATE IN MUSIC ENSEMBLES (CHOIR, BAND, INFORMAL GROUP, ETC) PRIOR TO YOUR HEARING LOSS? (YES/NO): J. DID YOU PARTICIPATE IN MUSIC ENSEMBLES (CHOIR, BAND, ETC) AFTER HEARING LOSS BUT PRIOR TO IMPLANT SURGERY? (YES/NO): K. DO YOU PARTICIPATE IN MUSIC ENSEMBLES (CHOIR, BAND, ETC) SINCE IMPLANT SURGERY? (YES/NO): L. TYPES OF MUSIC ENJOYED PRIOR TO HEARING LOSS: (e.g., classical, pop, country) M. TYPES OF MUSIC ENJOYED AFTER HEARING LOSS BUT PRIOR TO IMPLANT SURGERY: (e.g., classical, pop, country) N. TYPES OF MUSIC ENJOYED SINCE IMPLANT SURGERY: (e.g., classical, pop, country)
40
FOR THE FOLLOWING QUESTIONS, PLEASE INDICATE HOW MUCH THE FOLLOWING STATEMENTS APPLY TO YOU. O. I ENJOYED LISTENING TO MUSIC PRIOR TO MY HEARING LOSS. 1. Strongly disagree 2. Disagree 3. Neither agree nor disagree 4. Agree 5. Strongly agree ANSWER: P. I ENJOYED LISTENING TO MUSIC WITH MY HEARING AID(S). 1. Strongly disagree 2. Disagree 3. Neither agree nor disagree 4. Agree 5. Strongly agree ANSWER: Q. I ENJOY LISTENING TO MUSIC WITH MY COCHLEAR IMPLANT(S). 1. Strongly disagree 2. Disagree 3. Neither agree nor disagree 4. Agree 5. Strongly agree ANSWER: R. THE SOUND OF MUSIC AS HEARD WITH MY COCHLEAR IMPLANT(S) IS PLEASANT. 1. Strongly disagree 2. Disagree 3. Neither agree nor disagree 4. Agree 5. Strongly agree ANSWER:
41
MUSIC & COCHLEAR IMPLANTS List of Familiar Musical Excerpts
• Please indicate in the table below a list of 10 musical selections that you listen to
regularly. If possible, select only one track from each album and/or each artist. • If you listen to fewer than 10 musical tracks from different artists or different
albums, provide whatever information you can in the table below.
TRACK NAME ARTIST/PERFORMER ALBUM HOW LONG HAVE YOU BEEN FAMILIAR WITH THIS
SELECTION? EXAMPLE: Clair de lune (from Suite bergamasque)
Claude Debussy / François-Joel Thiollier
Debussy: Piano Works Vol. 1
3 years
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
42
Appendix B
MUSIC BACKGROUND INFORMATION QUESTIONNAIRE (ADULTS) Name: Gender: F M Student # Date of Birth : Age: Phone # Country of Birth First language Musical Background Have you ever taken private music lessons (instrumental or voice)? Yes No (Circle one) Years since Years playing Years since last Instrument Years of lessons last lessons regularly played regularly
Have you ever taken music in elementary or high school? Yes No (Circle one) Years since Years since last Instrument Years of lessons last lessons Years playing played regularly
Have you ever studied music theory? If so, how extensively? (i.e., levels achieved or courses taken). How much ear training have you had? Would you describe yourself as being "tone deaf"? Yes No (Circle one) Do
you have absolute pitch (perfect pitch)? Yes No (Circle one)
43
Did you grow up listening to music primarily from Canada, the United States, or England?
Yes No (Circle one) If not, which country's music did you grow up listening to?
Please describe your musical activities (singing, playing, dancing, listening, etc.) Regardless of whether you've ever taken music lessons, please rate how "musical" you think you are in relation to the average person.
extremely unmusical average extremely musical
1 2 3 4 5 6 7 Please describe your music listening: Type of Music Hours per week listening
44
Appendix C
Music and Cochlear Implants Interview
Please answer these questions with a few sentences or a short paragraph. You can answer in prose or in point-form. You are not obligated to fill the entire space, but you can do so if you wish.
1. How would you describe the role of music in your life (i.e. is it important, not important, why, etc)? 2. Please describe to us the way you enjoy music on a day-to-day basis (i.e. when and where you listen to and / or play music, how it makes you feel, etc) 3. What do you generally like (or not like) about music? Please specify whether you are referring to playing or listening to music. Feel free to refer to both of these activities in your answer.
45
Appendix D
SEMI-STRUCTURED Interview open-ended questions (prompts)
1. Perhaps you can tell me about the role of music in your life to before your hearing loss. 2. Can you describe your experience with music during the period when you used hearing aids? 3. How has your experience of music changed since getting your cochlear implant? 4. Are you still able to enjoy music? 5. How would you describe the sound of music as heard through your implant? How could you compare it to what you think is heard by someone with normal hearing? 6. Are you still able to enjoy the specific musical pieces that you liked before your hearing loss? 7. Have your musical preferences changed since receiving your implant? 8. When you listen to music, to which part do you primarily listen to? Was this different before receiving your implant?