MIMIC III-ICU Predictive Modeling

Cynthia Alexander, John Huynh, Allen Ko, Sameh Saleh August 2016

We built predictive classification models to determine if ICU patients were at risk for mortality based on features related to their care during their hospital stay.

Our primary resource was the MIMIC III ICU database, which contains all recorded anonymized information pertaining to patient admissions (Approximately 40,000 total patients) at Beth Israel Hospital in Boston from 2001-2012.

From this set, we created a subset of ~18,000 patients. The model has been trained on patients who have been hospitalized within the ICU for at least 48 hours. By having at least a day’s worth of information, we can predict the outcome of patients within 24 hours.

Problem Statement

18,097 patients | 18.9% mortality

That’s a lot of patients

106 curated featuresDemographics, Vitals, Labs

All ICU records from 2001-2012 And a lot of time series

● Patient Level○ Is he/she decompensating or stable?○ When to start having the conversation about comfort

measures?● ICU/Provider Level

○ Who has high mortality in my ICU and who is stable?○ Who can be transferred out of the ICU?

● Hospital/Administrative Level○ How to allocate medical resources○ Optimizing ICU logistical flow and bed placement

Use Cases

Sample Features

Interactive display of the effect of top feature values on ICU mortality

Day 0 Day 1

Admission

24 hours beforeOutcome

48 hours BeforeOutcome

Discharge from ICU/Death

24 hours After Admission

Day 7 Day 9Day 8

Beyond

Sample Patient Timeline in the ICU

XGBoost consistently outcompeted other models

PR curve predicting 24-Hr mortality ROC curve predicting 24-Hr mortality

FPR (1-specificity)Precision

TPR

(se

nsiti

vity

)

Rec

all

AUC = 0.956AUC = 0.944

AUC = 0.933

XGBoost Feature Importances for 24-Hour ICU mortality prediction

AgeLength of stay in ICU

Length of stay in hospitalSpO2**

Respiratory Rate**Temperature F*

O2 FlowGlucose

White blood cell countHeart Rate*

Temperature*Glucose*

Heart Rate Temperature F**

PlateletsSystolic BP

Systolic BP*Diastolic BP

Mean BP*Mean BP

Respiratory Rate*Heart Rate **

Blood urea NitrogenPlatelets*

SpO2

F1 score

* 24hr median**24hr std.

Day

s fr

om O

utco

me

(Tes

t)

1 2 3 7 14

1

2

3

7

14

Test

Days from Outcome (Train)

Predicting Mortality N Days Out

Trai

n

PR curve predicting N-hr mortality

Rec

all

Precision

TPR

(se

nsiti

vity

)

FPR (1-specificity)

ROC curve predicting N-hr mortality

Predicting Mortality N Days Out

◎ After 24 hours in the ICU - Will you live?◎ Uses another model to predict immediate mortality as a feature.

Predicting ICU MortalityAPACHE V (v. 0.1)

SubmodelPredicts immediate

mortality

Vitals, Labs

Demographics

~100 Features

Surviving Entire ICU Stay

AUC: 0.83

PR curve predicting 24-Hr mortality ROC curve predicting 24-Hr mortality

Predicting ICU MortalityAPACHE V (v. 0.1)

Interactive Scatter Plot

Age vs. Probability of Mortality

● We do great in predicting mortality 24-hours out.

● We still do really well at 48 and 72 hours:○ If you know the patient is going to die at 24

hours, you can likely predict death at 48 and 72 hours.

● Understandably, signal (and sample size) is lost over time.

● Predicting mortality 1 day from admission is comparable to gold standards WITHOUT using any prior patient information (like diagnoses)

Conclusions:

● Data Improvements○ Incorporate more curated data: microbiology labs,

patient diagnoses, ins and outs, medications, etc.○ Transform data to better identify high risk patients.○ External cross-validation on different population.

● Model Improvements○ Refine/remove unimportant features.○ Create models for individual body systems.

● D3.js improvements○ Change X-Axis on more features (scatter plot).○ Create a dashboard versus tooltips (scatter plot).○ Fully functional sliders EDA.○ Visualize Time Series Data (both).○ Host publicly.

Improvements

Thanks!

Any questions?