Dept. for Speech, Music and Hearing

Quarterly Progress andStatus Report

Training in the use of theTactaid VII: A case study

Plant, G.

journal: STL-QPSRvolume: 35number: 1year: 1994pages: 091-102

http://www.speech.kth.se/qpsr

STL-QPSR 111 994

Training in the use of the Tactaid VII: A case study GeofS Plant

Abstract A normally-hearing subject was provided with a training program aimed at introducing the tactile information provided by the Tactaid VII. The program consisted of 21 one hour sessions and included both analytic and synthetic training materials. On-going testing revealed that, following training, the subject was able to correctly diferentiate between a number of vowel and consonant contrasts presented via the tactile aid alone. Aided and unaided lipreading testing using the KTH speech tracking procedure showed an advantage when lipreading was supported by the Tactaid VII. An overview of the training program is provided and the significance of the results obtained is discussed in detail. Suggestions are made for further research in this area.

Introduction In a paper presented at a workshop on tactile and visual aids for deaf people, Elliott and Sherrick (1976) characterised training and evaluative procedures as the "orphans of sensory aid development" (Elliott and Sherrick, 1976; p. 488). Although much work remains to be done on the development of optimal approaches to training in the use of tactile aids, there have been a number of significant advances in this area over the intervening eighteen years.

One area of considerable work has been the evaluation of approaches and materials for use in training subjects to use various tactile aids. This has coincided with growing debate in the general area of aural rehabilitation over the value of the various approaches to training of hearing-impaired adults. Traditionally, training approaches have been categorised as either analytic or synthetic. Analytic may be seen as attempting to optimise an individual's ability to perform "bottom-up" speech processing. The training concentrates on maximising the hearing-impaired person's ability to discriminate between and identify the individual sound patterns of speech. An example of this approach is that developed for use at the Walter Reed Hospital in Washington DC by Walden et al. (1981). Synthetic approaches, on the other hand, attempt to optimise the use of "top-down" speech processing. Synthetic approaches, such as that advocated by Rubinstein and Boothroyd (1987), aim at maximising the hearing-impaired person's unconscious knowledge of the rules and conventions of hisher language in order to develop an ability to "fill-in" any gaps in the perceived speech signal. Although this dichotomy has been traditionally accepted, most training programs in fact combine elements of both an analytic and a synthetic approach. Even the training programs of an out-spoken advocate of synthetic training such as Edward Nitchie (1913) contain some analytic materials.

There appears to be some evidence supporting the use of a combined analyticlsynthetic approach in training subjects to use tactile aids, especially more complex multi-channel displays. The combined analytic and synthetic training

STL-QPSR 111 994

approach developed and evaluated for the "Tickle-Talker" by Alcantara et al. (1990) has shown the value of such training. They concluded that individuals who have had limited exposure to tactile stimulation are "more likely to respond to both analytic and synthetic training" (Alcantara et al., 1990; p. 203). Similarly, results reported for the Queen's vocoder (Brooks et al., 1985; Brooks et al., 1986a & b), and the Tactaid I1 (Plant, 1988) support the use of a training approach which include both analytic and synthetic approaches and materials.

A study indirectly supporting the use of a combined analyticlsynthetic approach is that of Osberger et al. (1986). In this study, four profoundly hearing-impaired subjects were fitted with a single-channel tactile aid (the Tactaid I) and provided with extensive training in aided and unaided lipreading using De Filippo and Scott's (1978) speech tracking procedure. Although speech tracking can be used to provide analytic training,

t

it is usually considered to be a synthetic approach to training. The one subject whose lipreading performance was enhanced in the aided condition, was congenitally hearing- 1 impaired, whereas the other three subjects had acquired hearing losses. The authors speculated that "an adult who has acquired language via the auditory channel may require extensive training to learn to comprehend spoken language via a new sensory modality" (Osberger et al., 1986; p. 4).

More recently, Plant et al. (1994) used only the KTH speech tracking procedure (Gnosspelius & Spens, 1992) to provide one subject with extensive training in the use of a number of different vibrotactile aids. The results of this study indicated that tracking alone may not provide sufficient training for the effective use of tactile aids. The study presented in this paper represents a follow-up to Plant et al. (1994) and investigates the use of a combined analyticlsynthetic approach to tactile aid training.

Method Subject The subject of this paper was a 26 year-old normally-hearing male native speaker of Swedish who spoke English as a second language. The subject had had limited I experience acting as a subject in lipreading studies in Swedish. He had had no previous experience in lipreading English or in the use of tactile aids. ,

Aid The aid used in this study was the Tactaid VII - a seven channel vibrotactile aid. The speech processing system used in this aid divides the incoming signal into seven bands with the following channel frequencies: Channel 1 - 200-400 Hz; Channel 2 - 400-600 Hz; Channel 3 - 600 - 800 Hz; Channel 4 - 800- 1,200 Hz; Channel 5 - 1,200- 1,600 Hz; Channel 6 - 1,600-3,000 Hz; and Channel 7 - 3,000-7,000 Hz. Four channels are each assigned to cover the range of the first (Channels 1 - 4) and second (Channels 4 - 7) formants of speech, Channel 4 being shared so that only seven output channels are generated. The output from the speech processor is presented via an array of seven vibrators which is designed to be worn on the back of the neck, the sternum or the abdomen. In the present study, however, the vibrators were set into a purpose-built hand piece which was arranged so that Channel 1 was transmitted to the thumb,

STL-QPSR 111 994

Channel 2 to the fore-finger, Channel 3 to the middle finger, Channel 4 to the ring finger and Channel 5 to the little finger. Channels 6 and 7 stimulated the palm of the hand.

Procedure All testing and training was carried out by the author, a male, native speaker of Australian English. The subject was seated in an audiometric test booth and observed the speaker through a glass window. The subject wore headphones providing sufficient masking noise to ensure that he could not hear the speaker's voice nor any acoustic output from the tactile transducers.

Twenty-one training sessions of approximately one hour each were provided over a period of approximately 6 weeks. In each session the subject completed at least two 5- minute speech tracking sessions for both the aided and unaided lipreading conditions using the KTH speech tracking procedure (see Gnosspelius & Spens, 1992, and Plant et al., 1994, for a detailed description of this procedure). In the first five training sessions speech tracking in both the aided and unaided conditions was supplemented by auditorily present speech low-pass-filtered (LPF) at 250 Hz. When it became apparent that the subject's lipreading alone skills in English were sufficient to allow a reasonable tracking rate (around 15 - 20 words per minute) the use of the LPF speech was discontinued. All remaining traclung sessions were administered via lipreading alone or via lipreading supplemented by the Tactaid VII. In a number of training sessions the subject was provided with additional synthetic practice involving a simple conversation conducted via lipreading plus the tactile aid.

Analytic training exercises were also provided in each training session. These were presented without lipreading cues via the tactile aid alone or via the tactile aid plus speech LPF at 250 Hz.

Tactile testing was conducted via the tactile aid alone. The subject was provided with no lipreading or auditory cues for this testing.

Training materials A series of short stories was used to introduce the long vowels [i], [a] and [3] and the consonants [n], [s], [t], [dl and [J]. These phonemes were selected as [i], [a] and [3] represent the "point" vowels of Australian English while the consonants chosen provide examples of the following contrasts - voiced/voiceless stops, nasalloral, stoplcontinuant and, alveolarlpost alveolar sibilants. Although they form a relatively small subset of the total inventory of English vowels and consonants it was felt they were a representative sample and gave a good introduction to a number of feature contrasts of English.

The short stories were written to provide multiple presentations of a specific phoneme in a variety of meaningful word contexts. The subject was presented with a written version of the story and told the phoneme to which he should attend. In the text all words containing the phoneme being introduced were bolded. The following example is taken from the story introducing [t].

STL-QPSR 1 / 1 994

Test materials and presentation Following the introduction of a specific phoneme, discrimination tests were used to determine whether the subject could perceive a particular contrast. Each test consisted of twenty pairs of words which differed only by the phonemes being contrasted. The items were presented using an ABX format. That is, the two alternatives were presented, and then one was repeated as the test stimulus. The subject's task was to indicate which of the two words had been repeated. The tests were presented via the tactile aid alone and no feedback as to the correctness of response was given. The following consonant contrasts were tested:

1. Presencelabsence of initial [s] - sitlit, sortlought 2. Initial [s] vs [n] - sightlnight, solno 3. Final [s] vs [n] - spicelspine, kisskin 4. Initial [s] vs [t] - sightltight, soltoe 5. Final [s] vs [t] - ceaselseat, micelmight 6. Final [t] vs [n] - seatlseen, mightlmine 7. Initial [dl vs [t] - doughhoe, dolto 8. Final [dl vs [t] - hardheart, heedheat 9. Presencelabsence of initial u] - sheetleat, shalelale

10. Initial [s] vs [J] - seatlsheet, sortlshort.

The vowel contrasts tested using the ABX format were:

1. [i] vs [a] - heedhard, keeplcarp 2. [i] vs [3] - heedhorde, feetlfort 3. [a] vs [3] - farrn/form, parchlporch

A list of 60 monosyllabic words was prepared to test the subject's ability to discriminate between the three long point vowels. The list contained 20 words with [a], 20 with [i] and, 20 with [3] presented in a random order. The subject was instructed that he was to feel a number of words containing either [a], [i] or [3]. His task was to identify which of the three vowels was in each word.

Following completion of this testing the consonants It, d, n, s, J/ were presented in the. VCV syllables IaCaI, /iCi/ and /3C3/. A number of 60 item lists were prepared, presenting each of the 15 possible syllables four times in a random order. The subject was given a list of the possible responses, and two lists were presented via lipreading alone, the Tactaid VII alone and lipreading plus the Tactaid VII.

An additional series of 20 item test lists were prepared contrasting the long and short "point" vowels of Australian English. The contrasts tested were:

STL-QPSR 11 1994

1. [i] 2. [31 3. [a1 4. [a1 5. [31 6. [i] 7. [i] 8- [a1 9. b l

vs A ] - seedsun, beamuck vs [A] - talkltuck, fawnlfun vs [A] - hearthut, marchlmuch vs [I] - harmhim, lardllid vs [I] - sortlsit, ballhill vs [I] - peachlpitch, read/rid vs [u] - steed/stood, she'd/should vs [u] - partlput, hardhood vs [u] - portlput, wall/wool

The words introduced in sessions 11 - 21 were tested in simple sentences. The subject was presented with the set of alternatives arranged in the following way.

E house home

I bought new school YOU saw a/an old door she hit big car it had small man

i book

He was then instructed that he would be presented with a sentence such as "I SAW A SMALL DOOR". His task was to identify each word in the sentence. The sentence was presented at a slow rate with a pause inserted between each word. The subject could ask for as many repeats of a sentence as he wished. Three lists of 10 sentences each were prepared and presented as test materials on three consecutive days. Each sentence required the subject to identify four words resulting in a possible score for each list of 40.

Results The results for the ABX testing of the vowel and consonant contrasts are presented in Table I. A score of 75% (15120) or better is significantly above chance at p ~ 0 . 0 5 . The subject's score for the 60-item test requiring identification of the vowel in a monosyllabic word was 54/60 correct (90%). The subject's overall scores for the task requiring identification of the consonants It, d, n, s, J/ in /aCa/, /iCi/ and /3C3/ presented via lipreading alone, the Tactaid VII alone and lipreading plus the Tactaid VII were 56.7%, 46.7%, and 66.7%, respectively. The scores were also calculated for correct vowel identification. Not surprisingly the subject scored 100% correct when the materials were presented either lipreading alone or lipreading plus the Tactaid VII. His vowel identification score for this task presented via the Tactaid VII alone was 93.3% correct. The scores for correct consonant identification were lipreading alone - 56.7%, Tactaid VII alone - 50.0% and lipreading plus the Tactaid VII - 66.7%.

Table I. Results for the testing of consonant and vowel contrasts presented tactually only via the Tactaid VII. A score of 75% or more is significantly above chance at p ~0.05.

STL-QPSR 111 994

Consonant contrasts 1. Presencelabsence of initial 2. Initial [s] vs [n] 3. Final [s] vs [n] 4. Initial [s] vs [t] 5. Final [s] vs [t] 6. Final [t] vs [n] 7. Initial [dl vs [t] 8. Final [dl vs [t] 9. Presencelabsence of initial

10. Initial [s] vs

[s] -100% - 95% - 90% - 95% - 75% - 75% - 70% - 100%

[J] -100% [J] - 65%

Long vowel contrasts 1. [i] vs [a] - 100% 2. [i] vs [3] - 100% 3. [a] vs [3] - 80%

Long vs short vowel contrasts 1. [i] vs [A] 2. [3] VS [A] 3. [a] vs [A] 4. [a] vs [I] 5. [3] vs [I] 6. [i] vs [I] 7. [i] vs [u] 8. [a] vs [u] 9. [3] vs [u]

The scores obtained for the three tactual presentations of the closed set words in sentences task were 36/40, 36/40 and 31/40 (mean = 85.8% correct). The mean scores over the three lists for the subsets of words were; pronouns (n = 4) 10110, verbs (n = 4) 7.7110, adjectives (n = 4) 9.7110 and nouns (n = 7) 7/10.

The subject's performance over 50 x 5 minute tracking sessions lipreading alone and lipreading plus the Tactaid VII are summarised in Fig. 1. The subject's mean scores for lipreading alone and lipreading plus the Tactaid VII over the entire study were 33.5 words-per-minute (wpm) and 39.5 wpm, respectively. This represents an improvement of 18% in traclung rate for the lipreading plus the Tactaid VII condition. The subject's mean tracking rates for the final five tracking sessions (10 x 5 minutes tracking) were

STL-QPSR 1 / 1994

50 O L (lipreading + Tactaid

40 vii) 9- LC (lipreading)

30

20 6 LC (lipreading + Tactaid vii)

10

0

1-5 6-10 11-15 16-20 21-25

Session no.

Proportion of blocked words

Session no.

,,illiT

S n i o n no. Srsion no. I I S r s i r nu.

Fig. 1. The tracking results for the subject for materials presented via lipreading alone (filled bars) and lipreading plus the Tactaid VII (unfilled bars). Each datum point represents ten 5-minute tracking sessions. The top panel presents the subject's tracking rates (L) and ceiling rates for the two conditions (LC) in words- per-minute. The middle panel shows the proportion of blocked words. The bottom three panels showing the number of words repeated once, twice, and three times. Note that in Sessions 1 - 5 the subject also received speech low-pass-filtered at 250 Hz via audition.

STL-QPSR 11 1994

spectral cues. For example, he reported that it was easy to distinguish between "I" and "you" as their tactile patterns moved in opposite directions across the tactile array. It should be noted that these materials were presented at a slow rate of articulation. Presentation of the sentences at a normal rate of articulation would almost certainly have lowered the subject's performance level quite markedly. Given his relative lack of experience with the tactile aid, however, these present results were very encouraging.

The tracking rates for aided and unaided lipreading show a clear advantage for the combined condition. The only sessions where the scores were equivalent were those where the subject was also receiving auditorily presented LPF speech. Once this was removed tracking rates in the aided condition were always higher than those obtained lipreading alone. It was only in the final block of tracking sessions, however, that the subject reported benefit in the aided condition. Prior to this point he felt that the aid provided little, if any, assistance in lipreading connected discourse.

It should be emphasised that at the end of this study the subject's total exposure to the aid was around 14 hours and considerably more than half of this time was devoted to tactile alone training. A total of 50 x 5-minute tracking sessions were presented in the lipreading plus the tactile aid condition. A further one hour was spent in conversational practice. In total the subject had just over five hours practice in integrating the information provided via the tactual and visual modalities. It is, therefore, reasonable to expect that with increased training and exposure to the aid the subject's performance will continue to improve. For example, Sherrick (1984) believes that "it is altogether unrealistic for us to expect that a recoding system involving speech patterns can be well learned in a period of less than two or three years" (Sherrick, 1984; p. 1336).

The results of this study provide at least some support for the contention that a tactile training program should include both synthetic and analytic approaches. The subject of this study provided anecdotal support for the use of a combined syntheticlanalytic approach. He felt that the analytic exercises had provided him with a working knowledge of the aid's display patterns which could then be utilised at the level of tracking. He commented that the phoneme loaded stories had been very beneficial in allowing him to discover the characteristics of the tactual display. It is hoped that this investigation will continue using a greater number of subjects and involving both profoundly deaf and artificially deafened subjects.

References Alcantara, J.I., Cowan, R.S.C., Blarney, P.J., & Clark, G.M. (1990). "A comparison of two training strategies for speech recognition with an electrotactile speech processor". J. Speech & Hear. Res. 33 , pp. 195 - 204.

Bernard, J.R. (1989). "Quantitative aspects of the sounds of Australian English", in P. Collins & D. Blair (Eds), Australian English. Queensland University Press, Brisbane.

Brooks, P.L., Frost, B.J., Mason, J.L. & Chung. K.(1985). "Acquisition of a 250-word vocabulary through a tactile vocoder", J. Acoust. Soc. Am. 77, pp. 1576 - 1579.

STL-QPSR 1 / 1 994 I Brooks, P.L., Frost, B.J., Mason, J.L. & Gibson, D.M. (1986a). "Continuing evaluation of the Queen's University Tactile Vocoder. I. Identification of open set words", J. Rehab. Res. & Dev. 23, pp. 119 - 128.

Brooks, P.L., Frost, B.J., Mason, J.L. & Gibson, D.M. (1986b). "Continuing evaluation of the Queen's University Tactile Vocoder. 11. Identification of open set sentences and tracking narrative", J.Rehab.Res. & Dev. 23, pp. 129 - 138.

De Filippo, C.L. & Scott, B.L. (1978). "A method for training the reception of ongoing speech", J. Acoust. Soc. Am. 63, pp. 1 186 - 1192.

Elliott, L.L. and Sherrick, C.E. (1976). "NINCDS workshop on tactile and visual aids for the 1 deaf. J. Acoust. Soc. Am. 59,486 - 489.

t

Gnosspelius, J. & Spens, K.-E. (1992). "A computer-based speech tracking procedure", STL- I QPSR 1/92., pp. 131 - 137.

Lisker, L. & Abramson, A.S. (1964). "A cross-language study of voicing in initial stops: Acoustical measurements", Word. 20, pp. 384 - 422.

Nitchie, E.B. (1913). "Lipreading, an art", Volta Review. 15, pp. 276 - 278. I Osberger, M.J., Rines-Weiss, D. and Kalberer, A. (1986, November). "Speech tracking performance of deaf adults using a vibrotactile aid", Paper presented at the Convention of the American Speech-Language-Hearing Association, Detroit, MI.

Peterson, G.E. & Lehiste, I. (1961). "Duration of syllable nucleii in English", J. Acoust. Soc. Am. 33, pp. 268 - 277.

I Plant, G. (1988). "Lipreading with tactile supplements", Volta Rev. 90, pp. 149 - 160. I Plant, G. (1984). "A diagnostic speech test for severely and profoundly hearing-impaired children", Aust. J. Audiol. 6, pp. 1 - 9.

Plant, G., Gnosspelius, J., and Spens, K.-E. (1994). "Three studies using the KTH speech tracking procedure", STLlQPSR 1/94, pp. .... - .... Plant, G, and Macrae, J.H. (1977). "Visual identification of Australian consonants, vowels and diphthongs", Aust. Teacher Deaj 18, pp. 45 - 50.

Rubinstein, A. and Boothroyd, A. (1987). "Effect of two approaches to auditory training on speech recognition by hearing-impaired adults", J. Speech Hear. Res. 30, pp. 153 - 160. I Rudmin, F. (1983). "The why and how of hearing Is/", Volta Review. 85, pp. 263 - 269. I Sherrick, C.E. (1984). "Basic and applied research on tactile aids for deaf people: Progress and prospects. J. Acoust. Soc. Am. 75, pp. 1325 - 1342.

Summerfield, A.Q. (1987). "Some preliminaries to a comprehensive account of audio-visual speech perception", in B. Dodd & R. Campbell (Eds.) Hearing by Eye, London: Erlbaum.

Walden, B.E., Erdman, S., Montgomery, A.A., Schartz, D.M., and Prosek, R.A. (1981). "Some effects of training on speech recognition by hearing-impaired adults", J. Speech Hear. Res. 24, pp. 207 - 2 16.

Walden, B.E., Prosek, R.A., Montgomery, A.A., Scherr, C.K. and Jones, C.J. (1977). "Effect of visual training on the visual recognition of consonants", J. Speech Hear. Res. 20, pp. 130 - 145.