Making ICU Alarms Smarter

Dr. Ido Schoenberg iMDsoftDr. Ido Schoenberg iMDsoft

Dr. Roy Schoenberg, Dr. Charles SafranDr. Roy Schoenberg, Dr. Charles Safran

Center for Clinical ComputingCenter for Clinical Computing

Beth Israel Deaconess Medical CenterBeth Israel Deaconess Medical Center

Harvard Medical SchoolHarvard Medical School

2

Purpose of alarms

• Alert ICU staff to changes in patient status that are Alert ICU staff to changes in patient status that are clinically significant*clinically significant* and require immediate and require immediate

response or interventionresponse or intervention

*Significance - as judged by the ICU staff member responding to the alarm*Significance - as judged by the ICU staff member responding to the alarm

3

The needle in the haystack(what alarms are looking for)

• Physiological events (e.g. pulmonary edema)Physiological events (e.g. pulmonary edema)

• Gradual and continuous over timeGradual and continuous over time

• Affect multiple physiological parametersAffect multiple physiological parameters

• Variable Variable

In a single patientIn a single patient

Between patientsBetween patients

Relative in timeRelative in time

4

Alarm systems that are used today..

• Follow a single parameter (e.g. heart rate)Follow a single parameter (e.g. heart rate)

• Are boundary basedAre boundary based

• Have fixed, audiovisual outputHave fixed, audiovisual output

• Are discreteAre discrete

• Are unaware of interventionsAre unaware of interventions

5

Alarms...

• 86% 86% are false readingsare false readings

• 6% are true but clinically insignificant6% are true but clinically insignificant

• 8% are significant*8% are significant*

For some devices - only 2% are significantFor some devices - only 2% are significant

*Tsien CL, Fackler JC; Crit Care Med; 1997 Apr 25:4,614-9

6

The effect of a 1:50 ratio( the cry wolf effect )

• Alarms are disregarded Alarms are disregarded 1

• Alarms are silenced for a period of timeAlarms are silenced for a period of time

• Alarm ranges are “adjusted” Alarm ranges are “adjusted” 2

• Alarms are turned off Alarms are turned off 3

• Monitoring is turned offMonitoring is turned off

• Murphy’s law kicks in...Murphy’s law kicks in...

7

Alarm systems we wish for

• SensitiveSensitive

• Specific Specific

• SensibleSensible

Smart alarms !

8

• If [HR]>120 or [HR]<60

Traditional monitors - using boundaries

91,92,89,10,81,85,88,90,92,90,92,95,93,98,25,121,118,112,115,119,114,123,280,128,122,117,114,112,110

91,92,89,1010,81,85,88,90,92,90,92,95,93,98,2525,121121,118,112,115,119,114,123123,280280,128128,122122,117,114,112,110

9

The study system - using algorithms

2. Average [HR] of 1 minute > 110 bpm

3. Moving value-delta over 20 seconds > 10 bpm

Filter values below 30 bpm and over 250 bpm

4. Two minute delta of 1-minute-average > 5 bpm

1. [HR] > 120 bpm

91,92,89,10,81,85,88,90,92,90,92,95,93,98,25,120,118,112,115,119,114,123,280,128,120,117,114,112,110

10

3. if true=0.3 if false= 0

4. If true=0.4 if false= -0.2

1. if true=0.2 if false= 0.1

2. If true=0.8 if false= -0.3

91,92,89,10,81,85,88,90,92,90,92,95,93,98,25,120,118,112,115,119,114,123,280,128,120,117,114,112,110

91,92,89,10,81,85,88,90,92,90,92,95,93,98,25,120,118,112,115,119,114,123123,280,128,120,117,114,112,110

> trigger value

0.9

The study system - using algorithms

11

The study system

• Interface build in Visual Basic

• Core & drivers in C++

• MetaVision™ from iMDSoft©

• MS Windows™ NT 4.0

• MS SQL Server™ 7.0

12

*ICU staff classified alarms as they occurred.

Study design

• Six Six patients, five days in a university hospital ICUpatients, five days in a university hospital ICU

• Manually documented monitor alarms for heart rate, Manually documented monitor alarms for heart rate, blood pressure and oxygen saturationblood pressure and oxygen saturation

• ICU staff identifies subset of significant alarmsICU staff identifies subset of significant alarms

• Study system produces its own set of alarmsStudy system produces its own set of alarms

• Measure the ability of both systems to detect Measure the ability of both systems to detect significant alarmssignificant alarms

13

PPVPPV

Tim

e

Monitor alarms Algorithm alarmsSignificant alarms

14

Results - monitor alarms

6,872 6,872 alarms recorded during 337 hours of study timealarms recorded during 337 hours of study time

False (3,912)57%

Significant (297)

4%True, non-significant

(2,663)39%

15

Results - algorithm system

• Missed 55 significant alarmsMissed 55 significant alarms (18.6%) (18.6%)

Significant(Matched)

24244%

Insignificant (Not-Matched)

30256%

544 alarms generated

16

Alarm distribution(By parameter)

0

500

1000

1500

2000

2500

HR SBP DBP SaO2%

Ala

rms

Monitor TotalMonitor Signif icantSeries3Algorithm TotalAlgorithm Signif icant

17

Positive predictive values(By parameter)

0

0.1

0.2

0.3

0.4

0.5

0.6

HR SBP DBP SaO2%

PP

V

Monitors Algorithms

P=0.011

18

Alarms - overall

6575

297302242

0

1000

2000

3000

4000

5000

6000

7000

Insignificant Significant

Ala

rms

Monitors Algorithm

19

Positive predictive values(Overall)

3.8

35.7

0

5

10

15

20

25

30

35

40

45

50

Pos

itiv

e P

red

icti

ve V

alu

e (%

)

Monitors Algorithms

20

The increased PPV can be attributed to ..

• Averaging values Averaging values

• Attaching high weight to persistent changesAttaching high weight to persistent changes

• Excluding outliers in calculationsExcluding outliers in calculations

• Slow sampling rate Slow sampling rate

• The use of “delta” rather than absolute valuesThe use of “delta” rather than absolute values

• Negative valued componentsNegative valued components

21

The missing 18%

• Missing dataMissing data

• Bad algorithmsBad algorithms

• Time rangesTime ranges

• Delta ratiosDelta ratios

• Incorrect weight assignment to componentsIncorrect weight assignment to components

• Missing alarms were insignificant ?Missing alarms were insignificant ?

• ...Bugs in code...Bugs in code

22

Discussion

• High specificity and predictive valueHigh specificity and predictive value

• Sensitivity at 81% ( Good enough ? )Sensitivity at 81% ( Good enough ? )

• Further research & development:Further research & development:

• Better algorithmsBetter algorithms

• Multi parameter algorithmMulti parameter algorithm

• Cascading alarmsCascading alarms

23

Building a septic shock alarm

• Developing tachycardiaDeveloping tachycardia

• Developing hypotensionDeveloping hypotension

• Dropping body temperatureDropping body temperature

• Dropping urine output Dropping urine output

• Positive blood culturePositive blood culture

TachycardiaTachycardiaTachycardiaTachycardia

HypotensionHypotensionHypotensionHypotension TemperatureTemperatureTemperatureTemperature

Urine outputUrine outputUrine outputUrine output

Positive blood culturePositive blood culture

24

Making ICU Alarms Smarter

Dr. Ido Schoenberg iMDsoft Dr. Ido Schoenberg iMDsoft

Dr. Roy Schoenberg, Dr. Charles SafranDr. Roy Schoenberg, Dr. Charles Safran

Center for Clinical ComputingCenter for Clinical Computing

Beth Israel Deaconess Medical CenterBeth Israel Deaconess Medical Center

Harvard Medical SchoolHarvard Medical School

25

Sensitivity & PPV of a system

Detected

Not detected

Significant Insignificant

a b

c d

• Sensitivity = a / ( a + c )

• Specificity = d / ( b + d )

• Positive Predictive Value = a / ( a + b )

26

Analogy to neural networks

27

Customizing formulas

28

Customizing statements

29

Customizing alarms

30

Running the alarm

31

Alarm Sounds

Patient under intervention ?

T=T= 00

S=S= 00

I=I= 11

T=T= 11

S=S= 00

I=I= 00

T=T= 00

S=S= 00

I=I= 00

Attended by nurse ?

Defined as Defined as true alarm ?true alarm ?

Defined as Defined as significant ?significant ?

T=T= 11

S=S= 00

I=I= 00

T=T= 11

S=S= 11

I=I= 00

Yes

No

Yes

Yes

Yes

No

No

No

Collector decision

Nurse input

LEGEND:

T - true alarm

S - Significant

I - Interventional

Categorizing AlarmsCategorizing Alarms

Categorizing alarms

32

HRHR

SBPSBP

DBPDBP

MBPMBP

SaO2%SaO2%

RRRR

Avg(xtime)[ ]

Delta(xtime)[ ]

Last Value Delta [ ]

Delta Avg(xtime)[ ]

Abs value [ ]

++ ++

Parameters

Functions

Logic Operand & Reference value

Component Example

truefalse

0.80.2

Missingvalues

0.5

Component Score

Building a component

>

<

=

LVD [ HR ] >5

AND

Delta_Avg(1,3)[HR]>10

And

Or

Not

33

Running an alarm

Component 1

Status at 10:57AM

True

False

True

True

Score

0.5

0.1

0.8

- 0.2

Alarm trigger set to 1.0

1.2 > 1.0

Alarm will sound

Alarm components

1.2

True = 0.8

False = 0.2

Component 2True = -0.2

False = 0.3

Component 3True = 0.4

False = 0.1

Component 4True = 0.5

False = 0

34

Conclusion

• Holds promise mainly in filtering false alarmsHolds promise mainly in filtering false alarms

• As is -- May serve as an “emphesizer” to traditional monitor As is -- May serve as an “emphesizer” to traditional monitor

alarmsalarms

• Further Research & Development :Further Research & Development :

• Better Algorithms ( Sharing knowledge )Better Algorithms ( Sharing knowledge )

• Multi Parameter AlgorithmsMulti Parameter Algorithms

• Linkage to other clinical resourcesLinkage to other clinical resources

• Decision SupportDecision Support

35

Distribution of recorded alarms

False,Technical 51%

False, Interv entional

6%

True, Non Significant

39%

True & Significant

4%

Total of 6872 Alarms

36

Evolving tachycardia ?

• Is there a continuous increase in heart rate ?

• Are these values likely to be erroneous readings ?

• Is the actual heart rate high ?

• Does this look like a steady rhythm or an arrhythmia ?