Speech training aids for hearing-impaired individuals: III

\.4yt.," VeteransAdministration

Journal of Rehabilitation Researchand Development Vol . 25 No . 4Pages 69–82

Speech training aids for hearing-impaired individuals:III® Preliminary observations in the clinic and childrens' homes

JAMES J . MAHSHIE, PhD, DIANE VARI-ALQUIST, MEd, BETTY WADDY-SMITH, MS, andLYNNE E. BERNSTEIN, PhDDepartment of Audiology, Gallaudet University, Washington, DC 20002

Abstract—Preliminary evaluation of 2 related computer-based speech training and practice aids for profoundlydeaf children was conducted . The Speech Training Sta-tion (STS) uses both acoustic and physiological transduc-ers for assessment and training in a school or clinic . TheSpeech Practice Station (SPS) uses the acoustic signal,and was designed primarily for use in the home . A seriesof games and activities was implemented on the 2systems . Use of the STS was evaluated by 2 speechclinicians during a 15-month period . Fifteen children weresubjects in the evaluation . The aid was found to be easilyincorporated into clinic activities and useful for diagnosisand therapy . The SPS was evaluated during a 1-to-2-weekperiod, during which it was placed in the homes of 5profoundly deaf children. Using an activity log andquestionnaire completed by the childrens' parents, usagestatistics and impressions were obtained . Potential valueand problems with such aids are discussed.

Key words : acoustic and physiological transducers, com-puter-based speech training aids, deaf children, electro-glottograph, pneumotachograph.

Address all correspondence and reprint requests to Dr . James J.Mahshie, Department of Audiology and Speech-Language Pathology,Gallaudet University, 800 Florida Avenue NE, Washington, DC 20002.Diane Vari-Alquist, MEd, Speech Processing Laboratory, JohnsHopkins University, Baltimore, MD 21218.Betty Waddy-Smith, MS, Kendall Demonstration Elementary School,Gallaudet University, Washington, DC 20002.

Lynne E . Bernstein, PhD, Center for Auditory and Speech Sciences,Gallaudet University, Washington, DC 20002 .

INTRODUCTION

Hearing loss affects the ability to both receive andproduce spoken language . When severe or profoundhearing impairment occurs early in life, the progno-sis for acquisition of intelligible speech is poor.Recent technological developments, especially incomputers, have the potential for helping deafindividuals achieve improved speech production (6).Presented here is a report of initial clinical andhome evaluations done in conjunction with develop-ment of 2 new computer-based speech training aids.An overview of the aids and a technical descriptionare given in 2 companion papers in this volume(1,2) . Two interrelated aids were developed, onedesigned to facilitate assessment and training inclinical settings (a Speech Training Station or STS),and the other for use in the home (a Speech PracticeStation or SPS) . Each of the speech training gamesis described following a brief review of the system.

STS and SPSSpeech signals for the STS are obtained from 3

transducers : a microphone (to obtain the acousticwaveform); an electroglottograph (to obtain degreeof vocal fold contact) ; and a pneumotachograph (toobtain average oral airflow) . Software for analyzingand displaying acoustic parameters is implementedon both the STS and SPS . Software for physiologi-cal parameters, obtained from the electroglotto-

69

70

Journal of Rehabilitation Research and Development Vol . 25 No . 4 Fall 1988

graph and pneumotachograph, is implemented onlyfor the STS . The decision not to implement physio-logical measures on the SPS was an outcome ofconsideration of transducer cost and complexitywith regard to calibration, and proper physicalplacement on the child.

STS and SPS softwareThe goals for software design were to facilitate : 1)

assessment of skills through provision of objectivemeasures ; and, 2) practice through drill in a gameformat . These goals were derived from consider-ations about the needs of therapists ' and childreninvolved in the process of speech training . It wasposited that assessment and therapy can be facili-tated by providing therapists with information aboutthe underlying physiologic mechanisms of speech,which they cannot obtain from listening alone.Further, the computer can provide the opportunityfor the child and therapist to have equal access toobjective information about speech production, andtherapy may proceed more efficiently, with less timespent in attempting to convey the desired targets tothe child.

The need for repeated drill is clear to all involvedin speech training, and the computer is ideally suitedfor this activity . The goal of providing drill in agame format derives directly from the concept thatspeech training will proceed best when the child'sinterest is aroused and he/she is motivated toproduce a large quantity of directed and monitoredpractice. Game development was made somewhatmore demanding because of the desire to implementgames that would be appealing to very young deafchildren. This objective is consistent, however, withthe idea that speech training of prelingually, pro-foundly deaf children should begin as soon afteridentification as possible, in order to promote morenormal speech and language patterns.

Access to software on both systems is throughmenus . The speech therapist can quickly changefrom one game to another, and can easily changeparameters for controlling games, depending on thechild's level of speech/language skills.

Software was written to teach speech attributessuch as sustained vocalization, production of re-peated syllables, and control of voice intensity and

'we will use the terms "clinicians" or "therapists" interchangeably torefer to all individuals, regardless of training, whose responsibility is toprovide speech therapy to deaf children .

fundamental frequency (Fo). Timing and intensityinformation is obtained primarily from the acousticsignal . Fo is obtained from the electroglottograph(EGG) (3) or from acoustic signals . The extent ofvocal fold abduction is also obtained from the EGG.The pneumotachograph signal provides informationconcerning initiation and termination of phonationas well as information about appropriate vocal foldgestures for production of voiceless versus voicedsegmental distinctions.

The Sustained Duration Game was developed toteach the child to produce sustained vocalizationsfor 1 to 7 seconds . The ability to sustain voicing (oreven produce voicing on demand) is frequentlylacking in young, prelingually deaf children, and isconsidered a prerequisite to production of connectedspeech (4) . The display for this game consists of aseries of rectangles, with lengths proportionate tothe target duration of each vocalization (Figure 1).Using menus, the therapist can easily set parametersfor the target duration, criteria for success (i .e .,percentage of time during which phonation mustoccur to receive a reward), and voice intensity . Theoption to set these parameters facilitates individual-ized training . Once variables have been set by thetherapist, 5 rectangles are outlined on the left side ofthe screen. Voice initiates the graphic display . Whilethe child sustains vocalizations, the rectangles arefilled with a color from left to right . Whenever thechild fails to vocalize, or his/her vocalizations fallbelow the criterion intensity level, the rectangle isleft blank in proportion to the unsatisfactory periodof time, and no reward is given . A successfulvocalization results in a graphic reward to the rightof the filled rectangle . Following each attempt, theprogram is ready for the next trial.

The graphic reward consists of animated colorfigures such as a bunny that dances, a bear thatwaves and wiggles its ears, a star that flashes color,a happy face that winks, and a chick that pops froman egg . The child can choose a specific reward, orrewards can be selected randomly by the computer.The set of rewards is used in all the games thatinvolve intensity.

The Repeated Short Vocalizations Game wasdesigned to facilitate production of a series ofdistinct syllables (for example, [ba, ba, ba, ba, ba] ),such as CV (consonant-vowel) or CVC, within aspecified period of time . This vocal activity isintended to facilitate production of basic motor-

71

MAHSHIE et al . Speech Training Aids: III . Preliminary Observations

I figure 1.A display from the Sustained Vocalization Game . The horizontal dimension of the rectangles corresponds to a 2-second vocalization

duration (c) . Vocalizing causes the rectangle to fill with color from left-to-right, with interruptions in vocalization resulting in gaps in

the fill (b) . Rewards are administered when the criteria set by the clinician for the vocalization have been met (a).

Figure 2.A display from the Repeated Short Vocalizations Game . Each footprint corresponds to a target syllable production . Three trials of 8

syllables each are shown on the displayed screen . At the onset of vocalization, a worm appears below the footprint and begins tomove across the screen at a speed proportional to the selected speaking rate . The bird jumps from one footprint to the next after each

distinct vocalization . If the bird beats the worm in the race across the screen, it picks up the worm and flies off the screen .

72


Figure 3.

A display from the Intensity Game with Immediate Feedback. This game is designed to teach the child an association betweenacoustic intensity and the height of the balloon on the display . As the child vocalizes, the balloon rises and falls next to the bar, inproportion to the child's vocal intensity . The 3 color regions on the bar correspond to low, moderate, and high intensity levels.

kinesthetic patterns required for correct syllabifica-tion (4) . The clinician can specify independently thenumber of syllables and the rate of production . Thetotal number of syllables can be varied between 1and 15, with the rate varied from 1 to 3 syllables persecond. The system checks to make sure that thechild does not produce voicing between syllables,and considers multiple syllables with continuousvoicing to be a single production (for example, whenthe vocal folds continue to vibrate between 2 "ba"segments, such that there is minimal reduction inacoustic energy during the consonant segment).

The initial display for this program is a series ofbird's footprints, each footprint corresponding to atarget syllable production (Figure 2) . At the onset ofvocalization, a worm appears below the footprintsand begins to move across the screen at a speedproportional to the selected speaking rate . The birdjumps from one footprint to the next after eachdistinct vocalization . If the bird beats the worm inthe "race" across the screen, it picks up the worm

and flies off the screen . If the child fails to producethe desired number of syllables at the desired rate,no animation follows, and the bird is simplypositioned at the next set of footprints.

The Intensity Game with Immediate Feedbackprovides real time visual feedback for acousticintensity, which is used to familiarize the child witha loudness scale, and to train the child to controlacoustic intensity prior to attempting subsequentintensity games.

This game is designed to help the child learn anassociation between color and height on the monitorscreen, and the motor-kinesthetic patterns requiredfor controlling intensity . This is accomplished byhaving the speaker's acoustic intensity, converted toa perceptual loudness scale, control the verticalposition of a balloon along a vertical bar (Figure 3).The bar is divided into blocks of 3 different colors.Colors on the bar correspond to 3 different intensitylevels . Blue, at the bottom, corresponds to low-intensity speech; green, in the middle, corresponds

73

MAHSHIE et al . Speech Training Aids: III . Preliminary Observations

Figure 4.A display from the Intensity Game with Delayed Feedback Game . A target intensity level is shown by the pointing hand, and thechild is required to produce voice at that level . The vocalization is then compared to a criterion and a reward issued if the productionis judged to be correct . Feedback is also provided by movement of the balloon to the level corresponding to the vocalizationproduced.

to conversational speech levels; and red, at the top,corresponds to loud speech . As the child vocalizes,the balloon rises and falls next to the bar inproportion to the child's vocal intensity . The thera-pist can also select specific target intensity for thechild to attempt.

The Intensity Game with Delayed Feedback wasdeveloped to facilitate carryover of intensity controlskills . This game employs postponed feedback,intended to nurture independence from the aid.During the game, a target intensity level is chosenrandomly and is indicated by a picture of a handpointing to various positions on the intensity bar(Figure 4). When the child attempts to produce avocalization at the designated intensity level, thevocalization is sampled and compared to the crite-rion level . Correct vocalizations can be followed bya display of one of the previously described ani-mated graphic characters . Numerical scores can alsobe given to motivate 2 or more children in thecontext of a competitive game . When scoring is

chosen, the game begins by requesting that thechild's name be typed, so that it can later appear onthe screen with the child's score.

The Intensity Game with Limited Feedback uti-lizes a graphic of a clown holding balloons whosecolors correspond to target intensity levels (Figure5) . The game does not make use of the verticaldimension to indicate loudness, only the color-loudness association . During the game, each balloonrepresents a trial . A trial is initiated by a blinkingballoon. The child must then produce a vocalizationat the intensity corresponding to the color of theballoon . The balloon blinks for 2 seconds whilespeech is sampled. Success results in a graphicreward appearing over the target balloon . Incorrectvocalizations result in no action by the game. Theoption of offering a numeric tally of correctproductions is also available for this game.

The Fo Puzzle Game was designed to providefeedback for Fo . This game has been used to teachchildren to produce speech in a prescribed F o range

74


Figure 5.A display from the Intensity with Limited Feedback Game. Each balloon represents a separate trial, with the color of each ballooncorresponding to the intensity levels used in the earlier intensity training games . Successful production of a target intensity levelresults in a graphic reward appearing over the target balloon.

and to produce F o contour patterns associated withintonation.

Clinical experience to date has been exclusivelywith the Fo information obtained from the EGG,although the current implementation of the STS andSPS is capable of Fo extraction from either EGG ormicrophone signals . Cycle-to-cycle pitch periods arescaled and displayed as a function of time, with timealong the abscissa and Fo along the ordinate (Figure6) . The clinician can select the total duration to bedisplayed on the time axis, as well as the total F orange to be displayed.

To play the game, the therapist draws a box onthe Fo display indicating the target frequency rangeand duration for vocalizations . If the child's Foremains within the borders of the box, a small,randomly-selected piece of a puzzle appears on theupper right area of the monitor . The therapist mayselect from a menu the number of pieces comprisingthe picture (i .e ., 2 to 16) and the picture the puzzlewill form. When the child has produced the correctnumber of vocalizations to accumulate all the puzzle

pieces, the scrambled puzzle pieces are enlarged tofill the entire screen . Using a cursor, the child movesthe pieces to unscramble the puzzle . For youngerchildren, for whom the puzzle is too difficult, theoption of pressing a single key, (F2), was providedto assemble the puzzle.

Physiological measuresA measure of vocal fold abduction has also been

implemented, making use of the EGG signal, buthas not yet been incorporated into a game . Theabduction measure has been shown to reflect degreeof vocal fold abduction during phonation, anadjustment required for production of unvoicedsegments and shown to be associated with excessivebreathiness during voice production (9,10).

The abduction measure is based on the relativedecreased duration of vocal fold contact, during avibratory cycle of the vocal folds, associated withbreathy voice production . To obtain the index ofvocal fold abduction, cycle-to-cycle measures of the

75

MAHSHIE et al . Speech Training Aids : III. Preliminary Observations

Figure 6.A display from the Fundamental Frequency (F,,) Puzzle Game . Production of vocalization with F o that occurs within the borders ofthe box results in the display of a small randomly-selected piece of a puzzle on the screen . Correct production of 6 vocalizations willresult in the presentation of all puzzle pieces . The child then has the opportunity to unscramble the pieces, using either arrows or amouse . The total number of pieces, criteria for rewards, etc ., are under the control of the clinician.

duration of vocal fold approximation, and theoverall duration of a glottal cycle are extracted (2).This information is used to compute an abductionmeasure for each glottal cycle, using the proceduredescribed by Rothenberg and Mahshie (9) . Thecomputer displays the abduction measure on ascaled screen as a function of time . The display givestime along the abscissa and degree of vocal foldapproximation along the ordinate . The clinician isprovided control of the total duration displayed onthe time axis.

Software has also been developed to provide asimple display of airflow (from the pneumo-tachograph) for monitoring accurate initiation andtermination of phonation as well as appropriatevocal fold gestures for production of a voicelessversus voiced distinction . Thus far, only oscillo-graphic type displays (airflow versus time) have beendeveloped and utilized (Figure 7), and only verypreliminary clinical experience has been obtained . Asimilar display has been shown to be useful in

modifying atypical devoicing behavior in deaf adults(6).

Clinical evaluation of the STS and SPSSixteen children between 3 and 11 years of age

worked regularly with 2 clinicians who were part ofthe development team . As discussed in Bernstein,Goldstein, and Mahshie (1), the clinical work wasintended primarily to provide human factors infor-mation to the developers, although assessment ofthe clinical benefit was also conducted on a limitedbasis. Both the SPS and STS were used by theclinicians during therapy for approximately 15months, from September, 1985, through December,1986 . During this period, the therapists (DV-A andBW-S) kept records regarding : 1) ease of use andpracticality of the aid ; and, 2) benefits and/orlimitations to the children and therapists . In addi-tion, video recordings of training sessions wereobtained during a 2-week period of use .

76

Journal of Rehabilitation Research and Development Vol . 25 No. 4 Fall 1988

Figure 7.

A display for monitoring average airflow associated with speech production . Shown is average airflow (cc/s .) along the ordinate andtime along the abscissa for production of a series of intervocalic consonant segments by a normal hearing speaker.

SubjectsFourteen prelingually, profoundly deaf children

ranging in age from 3 to 11 years, and 1 adventi-tiously deaf child (age 11 years), participated in theproject . Each child's audiological report documentsa profound bilateral sensory-neural hearing loss . Allchildren were fitted with, and routinely used, ear-level hearing aids in school . No other handicappingconditions were noted . One of the 15 childrenwithdrew from the project before its completion dueto geographical distance and lack of interest afterseveral months of participation . The reported etiolo-gies of the hearing loss of the remaining 15 childrenwere: 2 from cytomegalovirus, 2 by meningitis, 6from hereditary factors, and 5 from unknownetiologies.

Total communication (using pidgin signed En-glish) was used in all the children's educationalsettings, except for the 11-year-old adventitiouslydeaf child. This child was educated orally until she

was 10 years old, at which time she was enrolled in atotal communication program. She is still in theprocess of developing her signing skills . Both signsand speech were used during her therapy.

Clinical evaluation sitesSTS aids were used at the Kendall Demonstration

Elementary School, Gallaudet University, Washing-ton, DC, and the Speech Processing Laboratory ofThe Johns Hopkins University in Baltimore, MD.The speech pathologist (BW-S) at the KendallSchool used a STS with 7 of the preschool children,ages 3 to 5 years, as part of their regular in-schoolspeech therapy program . The children were seentwice a week for 20 minutes in small groups ; usually2 to 3 children per group . For 2 weeks in thesummer of 1986, BW-S used the STS with 3 primaryschool children (ages 8 to 9).

A STS was used clinically in the Speech Process-ing Laboratory at Johns Hopkins University, where

77

MAHSHIE et al . Speech Training Aids : HI . Preliminary observations

hardware and software development was ongoing.Five children ranging in age from 7 to 11 years cameto the laboratory for speech therapy sessions aftertheir normal school day . Both individual and smallgroup therapy was conducted, with therapy sessionslasting for approximately 1 hour . DV-A conductedall the therapy sessions in the laboratory.

EvaluationEase of use . Both clinicians reported that the

system was easy to learn . The function and arrowkeys on the standard 1BM-PC keyboard, used forprogram control, minimized the time needed to trainthe children to independently operate many of thegames . Both therapists agreed that the simplifiedkey arrangement permitted fairly rapid movementthrough the menus.

The therapists also reported that control ofparameters, such as the threshold level for theintensity games, is a desirable feature that permitstailoring of games to the needs of the individualchild. However, the default settings were adequatefor most parameters, and saved set-up time.

The software was intended to meet the needs ofchildren between the ages of 3 and 11 years . Amongthose needs are 2 systems that are easy to operateand control, even by the youngest children . BW-Sobserved that children as young as 2-and-a-halfyears of age had no difficulty understanding andcontrolling the games . We do not know whethereven younger children would be able to attend toand utilize the aid.

Reliability. Reliability was considered in terms ofdependability of the hardware/software system as awhole, and in terms of the signal processing anddisplays . Because evaluation took place continu-ously with development, signal processing and dis-play problems did occur in the context of therapy.The clinicians felt, however, that such problemswere solved expeditiously as they were discovered.Programs processed signals and generated displaysas expected . That is, the clinicians usually agreedwith the judgments made by the system.

Some occasions did arise when there were dis-agreements between the computer and the clinician.Some of these disagreements may have been due to amore lenient or different application of criteria bythe clinicians than by the computer . For example,the goal of the repeated short vocalizations program

is to facilitate production of syllable sequences withclear interruptions in phonation between each sylla-ble . However, the clinicians were either less able todetect continuous voicing or were less rigorous intheir application of criteria . As a result, productionsin which there was some reduction, but not cessationof voicing, were considered acceptable by the clini-cian but not by the computer. In this case, theclinician told the student that the computer hadmade a mistake, and that the production was a goodone.

A different class of computer-clinician disagree-ment was due to the ability of the clinician tomonitor several attributes, while the computer wasattending to only a single parameter . For instance,the clinicians often considered production of a loudvoice with aberrant vocal quality as unacceptable,even though the goal of the game being used wascontrol of loudness . Current work is directed towardsignal processing and games that involve multiplelevels of processing.

The most serious engineering problem concernedthe collection of intensity trials using the acousticsignal . Initially, the microphone was placed on agooseneck so that it could be passed from child tochild. The children and therapists were required tohold the microphone close to their mouths in orderto reduce the effects of ambient room noise.However, at close range, small movements inmouth-to-microphone distance can result in signifi-cant differences in estimates of loudness . Therefore,a child might produce a very loud utterance at aconsiderable distance from the microphone, and theresultant feedback would indicate production of aquiet voice . This created confusion for some of thechildren, until the therapist explained that themouth-to-microphone distance should be kept con-stant . Interestingly, children whose speech skillswere more advanced appeared to be aware that thecomputer was "wrong." When the children pho-nated, and the aid failed to respond in the predictedmanner, they would look to the therapist for anexplanation . The therapist's response in these casesappeared to satisfy the children, and they generallyaccepted that the computer was wrong in theseinstances . The problem was corrected with a fixedplacement of the microphone at some distance fromthe speaker, and with some changes in software toaccommodate ambient room noise .

78


Childrens' reactions to the software. Many at-tributes of the aid were attractive to the children.They were reported to enjoy the colorful anddynamic displays . It was observed that all thechildren genuinely enjoyed playing the games . Fre-quently, children at the Kendall School would enterthe therapy room and use the computer, even whenthey were not scheduled for therapy.

The aid motivated the children to practice withoutthe direct supervision of therapists . The availabilityof feedback encouraged increased attempts to cor-rectly produce the desired target . DV-A reportedthat the older children maintained their interest inworking on speech for a longer time than they mightotherwise, with the computer-based sessions lastingwell beyond the half-hour duration of a typicalschool session . BW-S reported a similar trendamong the younger children . Both clinicians agreedthat the children were willing to work on speechgoals more frequently with the aid than they wouldhave using more conventional therapy methods.Similar findings have been reported by Osberger,Moeller, and Kroese (8).

This increased interest may be due to the challeng-ing tasks offered the children, and the ability toindividualize the games and activities . Since it waspossible to customize the criteria for each child, theclinician was able to alter the difficulty level in smallincrements whenever the child demonstrated suffi-cient consistency at the previous criterion level . As aresult, children continued to be challenged, whilefrustration was minimized.

The element of competition was especially moti-vating to the older children . After working for sometime with the graphic rewards, the older childrencontinued to work when the numerical score wasprovided. Frequently, competition among childrenresulted in sustained attention to the speech tasks.Numerical scores motivated children as young as 5to 6 years of age.

Across the age range, increased quantity ofvocalization resulted from use of the F o PuzzleGame . The clinicians reported that a number of thechildren exhibited greater control over pitch varia-tions while watching the F o display . The therapistsfurther noted that this game provided an efficientmeans of eliciting a broader range of vocalizedfrequencies than would likely be obtained usingmore conventional methods .

These results are similar to those reported in theliterature for older children (7) and with youngadults (6), suggesting that the use of such displays,in conjunction with therapy, results in improvedability to control Fo both with and without thepresence of the feedback.

Clinical benefits. Experience suggests that the aidcan be used in therapy with children from 3 yearsold to early adolescence. The clinicians reported thatthe aid provided more clinical benefits than they hadexpected prior to using it . They were enthusiasticabout the reduction in time required for them todevelop creative materials intended to motivatechildren, and the increase in the time available fortherapy . They also commented favorably about theease with which the aid could be integrated withmore traditional therapy techniques . The traditionalstrategy of indicating production of a sustainedvocalization is by moving the finger along the child'sarm or pulling a toy across the table : the SustainedVocalization Game provides an analogous display.

When 2 or more children worked together on theaid, turn-taking and friendly competition resultedand was encouraged . Increased language output wasreported among the children, as well as between thechildren and the clinician . The children often dem-onstrated an understanding of the desired speechgoals by attempting to explain to each other how toproduce the desired targets.

Therapy activities sometimes involved the creativeuse of the software . The older children created theirown game with the Fo display by drawing "maps"on transparencies that could be placed on themonitor screen . The locations on the map served asthe fundamental frequency targets for subsequentactivities whose goal was to increase the childrens'ability to control and modify Fo . DV-A reportedthat this activity allowed the therapist to observe thechildren's self-assessment of Fo goals.

The therapists reported that with increases invocalization, notable phonatory changes also oc-curred in vocal characteristics other than thoseaddressed directly during the therapy session . Onechild who exhibited a weak, breathy voice improvedthe quality of her productions while working onincreasing duration of sustained voicing.

The objective measures supplied by the aid werealso reported to be positive clinical attributes.Quantification of Fo, duration, and intensity levels

79

MAHSHIE et at, Speech Training Aids : III . Preliminary Observations

enabled the therapists and children to judge therelative correctness of production across sessions.

The clinicians also agreed that use of the systemhelped them to attend to single speech parameters.Because displays were based on a single parameter(Fo , intensity, etc .), it was easier to "filter out"attributes of the children's speech other than thosebeing worked on.

The ability to view individual speech parameterswas also useful in training other clinicians . Tengraduate students in speech and language pathologyused the aid in the school during a 2-week period.The clinical supervisors commented that the clini-cians used the computer aids to establish criteria foracceptable versus unacceptable production of indi-vidual speech attributes.

When both child and therapist have equal accessto objective information about speech production,rewards for incorrect production are less likely . Thiswas found to be true with the F o display . Also, byminimizing dependence on auditory skills alone tomonitor correctness of the speech target, hearing-impaired teachers can take a more active role in thespeech training of the children in their classes duringdrill and practice.

Evaluation of the home-based speech training aidOne of the goals of this project was to develop a

home training aid. The rationale for this goal is thatby extending accessibility of monitoring and feed-back capability outside of the therapy room, in-creased speech-practice time will be achieved, andconsequently, better speech . Accordingly, SPS'swere placed in the homes of 5 children for periodsof 1 to 2 weeks. Four of the children were 3 to 5years old, and 1 was 10 years old.

Parent orientation and record-keeping . Before theaid was placed in a child's home, a brief orientationwas provided to the parents by the project engineerand the speech/language pathologist . During theorientation, project goals were described, systemoperation was explained and demonstrated, andprocedures were described for solving any problemsthat might arise . During the orientation, parentswere given an opportunity to "play" the games andask questions.

To obtain information regarding use of the SPS, 2types of questionnaires were prepared : a sessionform, to be completed after each practice session ;

and a general impression form, to be completed atthe end of the home trial period . The session formsrequired "yes/no" and multiple choice responses,and some brief descriptions . Space was provided forthe parent/reporter to make any additional com-ments they felt to be useful . A parent completed theforms for the younger children. A 10-year-oldfemale subject completed the session forms withoutparental assistance. The impression forms werecompleted by the parents at the end of the hometrial period. The forms consisted of questionsrequiring responses on a 5-point scale (from easy todifficult), "yes/no" questions, and general com-ments. Procedures for completing the forms wereexplained during the orientation, and each familywas given a packet containing instructions forcompleting the forms (approximately 15 sessionforms, and 1 impression form).

All of the games that used acoustic input parame-ters (i .e ., Sustained Duration, Repeated Short Vo-calizations, and Intensity with Immediate, Delayed,and Limited Feedback) were installed on the homeaid. Home sessions were developed by the therapistfor 4 of the children, using an interactive programthat permitted selection of games to be presented,the number of times an exercise was presented, thesequence of games, display parameters, and criterialevels . Individualized games for home practice wereconsistent with the child's activities in school.Development of a home game disk typically requiredapproximately 5 to 7 minutes . At the end of eachtherapy session during which the school device wasused, a child was given a home disk . In addition tostoring game parameters for home practice, this diskwas used to log the time the child spent using thedevice. (Complete records were not obtained for allthe children, so those data are not reported here .)

The 10-year-old child (Subject 5) brought thedevice home for a 1-week period, during which therewas no simultaneous work being done in school.This child was given all the options for use at homeand was allowed to select among the various gamesin any order desired. Certain game parameters wereestablished by the clinician, based on the child's useof the STS during the school year . The child wasalso able to select other options, such as theduration of the Sustained Vocalization Game andthe number of vocalizations required for the Re-peated Short Vocalization Game .

80


Parent observations of the home aid . Parents'questionnaire responses indicated that orientation tothe computer and directions given before taking theaid home were clear and helped them in using thesystem . They were able to start the system and runthe appropriate software . The parents further indi-cated that the aid and software were helping theirchildren toward the acquisition of desired speechgoals, and that use of the aid at home was a positiveexperience. Three of the 4 children reportedly wereable to operate the games independently . One par-ent, however, noted that the fixed order of the pres-entation of games was a limitation, presumably be-cause the child was sometimes unable to continuework on the activities in which he was most inter-ested.

One exception to the generally favorable resultswas that the family of S4 experienced difficultyusing the home system . While the system was in thehome for 2 weeks, the parents were unable to get itto operate until day 3, and it was not until day 6that all the games were working correctly . Thisfamily was the first to receive the home system, andsome "bugs" were worked out during this 2-weekperiod. This experience appeared to have had anegative effect on S4's enthusiasm, as well as hisparents' . Four families reported some instances inwhich the software malfunctioned . However, de-scribed failures were not replicated in the labora-tory.

The difficulty level of activities sent home ap-peared appropriate, and rewards were reported as"helpful ." Two parents suggested, however, thatrewards could have been more exciting ("likefireworks") . Perhaps the parents were commentingon their own impression of the rewards, not theirchildren's interests, since the children have shownconsistent enthusiasm for the graphic displays whenused in a clinical or laboratory setting . Three of thefamilies reported that the games seemed appropri-ate, but the other 2 families thought the games weretoo easy. Three families wanted a larger selection ofgames . These comments may be attributed to theextensive experience these children had with the aidat school, the relatively small number of differentactivities currently available, or to the difficulty theparents had making clinical judgments . All parentsreported that the durations of the games seemedappropriate, and that each child had a favoritegame .

Table 1 summarizes the number of sessions, totaltime, and mean time the children used the aid intheir homes, based on logs kept by the parents . TheSPS was used at home for a total of 679 minutes.Times ranged from 51 minutes to 185 minutes perweek . Mean time per session for individual childrenranged from 8 .5 minutes to 34 minutes . It should benoted that the child who used the aid the least wasthe youngest child (S4), whose parents had initiallyexperienced difficulties with the system.

Table 1.Childrens' use of the Speech Practice Station (SPS).

Subject Total time(mins .)

Meantime/session

(mins .)

Totalnumber of

sessions

Days inhome

(Si) 140 23 .3 7 7

(S2) 185 20 .5 9 7

(S3) 170 34 . 5 7

(S4) 102 8 .5 12 14

(S5) 82 10 .25 8 7

Based on logs kept by the parents.

Parents commented that sessions included adultsand siblings . The device served as a focal point forspeech activity . Since increased interest by thefamily is important for realization of spoken lan-guage (5), this result suggests a very positive benefitfrom using the system.

SUMMARY AND CONCLUSIONS

The project goal was to develop and assess 2related computer-based speech training aids used tofacilitate speech development in deaf children . Theaids appear to have promise as both an aid to, andextension of, speech activity in the therapy room.Results of assessing certain human factors, such asease of use by clinician and children, acceptance ofthe aid and displays by the children, and ability toincorporate the aids into existing therapy modelswere promising . The general impression of theclinicians was that the aid resulted in increasedwillingness by the children to work on speech skills,

81

MAHSHIE et al . Speech Training Aids : III . Preliminary Observations

and thus reduced the time and effort spent attempt-ing to motivate the children to practice speechactivity . The speech parameters obtained by thesystems (intensity, durational, and Fo attributes ofspeech) represent the most elementary aspects ofspeech production that are necessary for a youngdeaf child to develop speech (4) . Selection of suchbasic aspects of production resulted in incorporationof the aid into daily clinical practice with minimaleffort.

Assessment of the accuracy and consistency of theaids reveals that, although certain differences dooccur between computer and clinician judgments,there is reasonable agreement . Discrepancies be-tween clinician and machine judgments may havebeen due to the clinicians having somewhat morelenient or altogether different criteria for thechildrens' productions than the computer . Otherdiscrepancies appeared due to the aids' provision offeedback based on individual speech parameters,while clinicians often simultaneously monitor severaldifferent attributes.

Results of home use as an extended therapy aidsuggest that the device resulted in speech activity athome that would not likely have occurred otherwise.By permitting the clinician to control the perfor-mance parameters associated with the games athome, it was possible to maintain consistencybetween home and school speech activities thatwould have been difficult to establish . While use of

the SPS in the home appears promising, a greatervariety of games and activities was requested by theparents . Because such observations are somewhat atvariance with clinical observations of the childrenwith the same games, these comments raise apotential issue associated with bringing a speechtraining aid into the home . The parents' remarksmay actually reflect an inability on their part toadequately judge the speech skills that childrenshould practice . While new games are currentlybeing developed for the STS and SPS, additionalconsideration of the implications of bringing ther-apy home is clearly needed.

The development of speech training aids for usewith prelingually deaf children is feasible . While thepresent system has achieved many desirable features,further work remains . Additional features, such assimultaneous feedback for prosody and vowel iden-tity, are being developed . An expanded ability tomonitor attributes of consonant production featuresis also desirable . Clearly, future evaluation mustinclude more formal and larger scale designs forassessment of the benefit to speech of those usingthe aids. Such an assessment is currently beingplanned.

ACKNOWLEDGMENTS

The work reported here was supported by NIH/NINDCS(NS-4-2372).

REFERENCES

1. Bernstein LE, Goldstein Jr . MH, Mahshie JJ : Speechtraining aids for hearing-impaired individuals : I . Over-view and aims . J Rehabil Res Dev 25(4), 1988 (this issue).

2. Ferguson J, Bernstein L, Goldstein M: Speech trainingaids for hearing-impaired individuals : II . Configurationof the Johns Hopkins aids . J Rehabil Res Dev 25(4), 1988(this issue).

3. Fourcin A, Abberton E : First application of a newlaryngograph . Med Biol Illus 21 :172-183, 1971.

4. Ling D : Speech and the Hearing-impaired Child : Theoryand Practice . Washington, DC : The Alexander GrahamBell Association for the Deaf, Inc ., 1976.

5. Ling D : Integration of diagnostic information : Implica-

tions for speech training in school-aged children . InSpeech Assessment and Speech Improvement for theHearing-Impaired, J . Subtelney (Ed .) . Washington, DC:The Alexander Graham Bell Association for the Deaf,Inc ., 1980.

6. Mahshie J, Hasegawa A, Mars M, Herbert E, Brandt F:A computer-based system for modifying deaf speakers'phonatory behavior . In Proceedings of the InternationalCongress on Rehabilitation Engineering, 1984.

7. McGarr N, Head J, Friedman M, Behrman AM,Youdelman K: The use of visual and tactile sensory aidsin speech production training : A preliminary report . JRehabil Res Dev 23(1) :101-110, 1986 .

82


8. Osberger MJ, Moeller MP, Kroese JM : Computer-

the meeting of the American Speech-Language-Hearingassisted speech training for the hearing impaired . Acad

Association, Atlanta, GA, 1979.Rehabil Aud 14 :145-158, 1981 .

10 . Rothenberg M, Mahshie JJ : Monitoring vocal fold abduc-9. Rothenberg M, Mahshie JJ : Monitoring vocal fold abduc-

tion through vocal fold contact area . J Speech Hear Restion through vocal fold contact area. Paper presented at

(in press) .

Documents

Speech training aids for hearing-impaired individuals: III