Speech Based Optimization of Hearing Devices

Alice E. Holmes, Rahul Shrivastav, Hannah W. Siburt & Lee Krause

The Problem

Programming is based on electrically measured dynamic ranges of pulsed stimuli (non-speech)Current programming methods have numerous options

PurposeThe goal is to understand speech, the tuning of the device should be based on speech and not tones. Development of a standard metric to understand the strengths and weaknesses of the individual device user. The complexity of problem requires an automated and intelligent process to optimize the device programming.

Overview Hearing

Device, DBrain, B

Input Signal, Sinp

Output Signal, Sout

Intermediate Signal, Sint

inp int

int out

inp out

D S S

B S S

B D S S

inp outWe want :

. . . .

S S

i e B D I

Almost nothing is known about the function B

What to optimize?Acoustic contrasts essential for speech intelligibility-- Minimize error functionFrom patient experiments, we can get data for different values of the parameters and the corresponding errors– The dimensionality of this data is related to the

number of independent programmable parameters– Many parameters, hence very high dimensionality

leading to the “curse of dimensionality”

How to reduce the complexity of the problem?

Artificial Intelligence Algorithms-- Patient-independent knowledge should be available (e.g. as “rules”)-- Patient-specific knowledge should be statistically extracted from the performance of each patient-- “Model field theory” approach to model relationships

CI user speech feature battery test Develop Optimized

Map for CI device Using:

- Fuzzy logic - Genetic Algorithms - Model Field Theory

Audiologist updates CI Map

Re-evaluate the CI user with adjusted map repeat until map is optimized

Speech feature to CI device map parameter knowledge base

Speech based Optimization of Cochlear Implant Processor patient Study

Benefits: -Optimized Map for CI user -Improved hearing performance -Improved quality of life -Reduced cost of tuning procedure

Perform standard CI user evaluation: -HINT -CNC Monitor

Session results

Confusion Error Matrix

Monitor Optimization results

CI user with optimized map

Initial Clinical Trial20 adults with

– N24 or New Freedom implants– Freedom ProcessorsAdjusted the following parameters

– Rate– Loudness growth– Frequency allocation tables Outcome measures

– CNC lists in quiet – BKB-SIN– Subjective questionnaire

Subject DemographicsGender Male Female

N=7N=13

Age (Years)

Mean S.D. Range

57.319.9

24-82

Length of CI Use (months)

Mean S.D. Range

25.628.975-115

Type of CI N24 New Freedom

N=3N=17

Initial Clinical TrialThe Optimization program (Clarujust™) was designed to interface with a customized version of Cochlear Corp. Custom Sound so that programming changes recommended by the algorithm could be tested seamlessly. All stimuli were presented through a direct connection to the speech processor and at a constant level across all test sessions (approximately 60 dBA). 3 Sessions – two weeks apart

Clarujust™ A series of VCV syllables were presented & verbal responses were recorded by the researcher. NWE for the processor setting was calculatedThe next combination of FAT, PR & LG was automatically recommended & tested.Procedure was repeated for 30 minutes

AI Algorithm

Loudness Growth

Rate

FAT

Inputs from CI

software

DF Error Matrix

Learning feedback

Optimization Models

ProceduresOutcome measures Clarujust™ routineMap with lowest net weighted error (NWE) was selected and programmed in to processer for use until next session

Subject Map Parameters

250 500 720 900 1200 18000

2

4

6

8

RATE

NU

MB

ER O

F SU

BJE

CTS

10 15 20 25 300

5

10

15

20

LOUDNESS GROWTH

NU

MB

ER O

F SU

BJE

CTS

188-7938

188-7438

188-6938

188-6563

188-6063

188-5938

188-5813

0

5

10

15

20

Baseline Opt 1 Opt 2

FREQUENCY ALLOCATION TABLE (Hz)

NU

MB

ER O

F SU

BJE

CTS

CNC Word Scores

-15

-5

5

15

25

35

45

55

65

75

85

95

-15

-5

5

15

25

35

45

55

65

75

85

95

Perc

ent C

orre

ct

Subject Number

CNC Word Scores

Opt 1 gain

Opt 2 gain

Series3

Baseline

Opt 1

Opt 2

• Significant difference among the three conditions using Greenhouse-Geisser analysis (p < 0.004).

• Further trend analyses indicated a significant ascending omnibus trend from baseline (p < 0.004)

• Pairwise comparisons significant differences between baseline and Opt 1 (p < 0.025) and between Baseline and Opt 2 (p < 0.015).

CNC Phoneme Scores

-30

-10

10

30

50

70

90

110

130

150

170

190

210

230

250

270

290

-30

-10

10

30

50

70

90

110

130

150

170

190

210

230

250

270

290

118 106 110 114 101 103 116 102 120 121 104 119 113 112 105 109 108 107 117 115

Phon

emes

cor

rect

Subject Number

CNC Phoneme Scores

Opt 1 gainOpt 2 gainSeries3

Baseline

Opt 1

Opt 2

•Significant difference among the three conditions using Greenhouse-Geisser analysis (p < 0.008).

•Further trend analyses indicated a significant ascending trend from baseline (p < 0.015)

•Pairwise comparisons showed significant differences between base line and Opt1 (p < 0.003) and between Baseline and Opt 2 (p < 0.04).

BKB-SIN Scores

-10

-8

-6

-4

-2

0

2

4

6

8

10

12

14

16

18

20

-10

-8

-6

-4

-2

0

2

4

6

8

10

12

14

16

18

20

114 106 120 118 110 101 102 103 104 121 119 113 108 105 107 116 112 109 117 115Si

gnal

-to-N

oise

Rat

io (

SNR

) in

dB

Subject Number

BKB-SIN Scores

Opt 1 gainOpt 2 gainSeries3

Baseline

Opt 1

Opt 2

• Significant difference among the three conditions using Greenhouse-Geisser analysis (p < 0.03).

• Further trend analyses indicated a significant ascending quadratic trend from baseline (p < 0.009).

• Pairwise comparisons showed significant differences between baseline and Opt 1 (p < 0.03)

Subjective Results

At the end of this clinical trial, 17 out of 20 patients preferred to continue using one of their optimized maps. Subjective ratings in various situations were also obtained from each subject (Holden, et al, J Am Acad Audiol 18:777–793, 2007)

Subjective Performance in 19 Listening Situations

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50

Conversation on the Telephone

Message on the Answering Machine

News on TV

Movies/Dramas/Sitcoms on TV

Radio in the Car

Radio at Home

Lyrics to Music

Conversation at Dinner Table

Conversation in Quiet with One

Conversation in Quiet with Several

Conversation in a Car

Conversation at Social Gathering

Conversation at Restaurant

Conversation with Cashier

Conversation with a Child

Conversation Outside

Someone in the Distance

Church Service

Meeting in a Large Room

Subjective Performance

Optimization 2 Optimization 1Baseline

Summary• The optimization method used in this study

resulted in improved subject performance in all outcome measures.

• Speech perception was significantly better in word and phoneme identification with optimized maps.

• In addition, subjects performed better in noise using the optimized maps.

• Subjective tests suggest that patients preferred the optimized maps in their daily lives.

What is Next?Continue to refine process with CI technologyCurrently doing clinical trials with two hearing aid manufacturers– Three pilot subjects have been fitted with bilateral

hearing aids using the optimization protocolFuture applications– Hybrids– Audiologic rehabilitation– Cell phones– ????

Thank you to the students involved

Hannah SiburtKevin StillElyse SchwartzBekah Gathercole

Acknowledgments

This project is funded by Audigence, Inc. and the Florida High Tech Corridor Council.We wish to thank Cochlear Corporation for supplying the fitting software platform and for their extensive and timely technical support. We also want to thank our subjects for their willingness to participate in the experiment.