View
221
Download
4
Tags:
Embed Size (px)
Citation preview
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
Optimal Organ Allocation Policies:An application of discrete event simulation
Mark S. Roberts, MD, MPP
Associate Professor of Medicine, Health Policy and Management and Industrial Engineering
Chief, Section of Decision Sciences and Clinical Systems ModelingUniversity of Pittsburgh School of Medicine
University of Pittsburgh
SECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
2
The general problem of Organ Allocation
• Organs are a scare resource, and waiting lists are increasing
• The debate (in the US) surrounds– Who gets transplanted? (or retransplanted)
– What determines selection priority and site?• UNOS (Has changed allocation rules 3 times)
• HCFA (rules about success rates required for sites)
– What level of regional preference is appropriate?
– Organs to the sickest or to those who would benefit most?
• What are the appropriate methods to analyze this problem?
• As much a talk about the value of simulation as a talk about transplantation allocation
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
3
What is the Clinical and Policy Problem?
• There are two very distinct questions that can be asked regarding transplantation:– CLINICAL Question: given a specific patient with a specific
disease and a set of clinical characteristics, what is the optimal timing in the declining course of disease to transplant that specific patient?
– POLICY Question: What set of selection, listing criteria and allocation rules maximize the utility of the limited supply of organs? (What is the optimal timing from the point of view of the society?)
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
4
Allocation of organs to Patients
• In the US, organs are allocated (in theory) with a balance of efficiency and fairness
• The United Network for Organ Sharing (UNOS) is responsible for implementing and setting allocation policy
• Current rules are essentially a combination of two concepts:– PRIORITIZATION: where a patient falls in the waiting list
– ALLOCATION: how many lists there are in the country• For example, there are different lists for each region, and acute liver
disease is treated differently from chronic liver disease
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
5
US Organ Procurement Organizations (OPOs)
OPOs are aggregated into 11 regions
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
6
Prioritization
• As changed several times over the past decade:
• Prior to 2002– 4 “Status” groups
• 1 (acute, fulminate liver failure)
• 2a (chronic liver failure, need ICU care for survival)
• 2b (chronic liver failure, need hospitalization)
• 3 (chronic liver failure, not in hospital)
– Several other “status” levels for special circumstances• Status 7 (“too sick” at the moment)
– Order within status mainly determined by waiting time
– Allows for gaming the system
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
7
Prioritization (MELD score since 2002)
• Prioritization was changed to rank people based on level of illness (transplant the sickest first)
• Model for End-stage Liver Disease (MELD score)– Predicts the probability of survival for the next three months
– Scaled to an integer between 6 (lowest probability of death) and 40 (highest probability of death)
• Status 1’s are the same (fulminate, with p(death in 7 days) > 50%)
• The remainder are grouped by MELD, rank within MELD is where wait time, blood type compatibility matter
MELD Score = 10*(0.957 x ln(creatinine) + 0. 378 x ln(bilirubin) + 1.120 x ln(INR) + 0.643)
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
8
Allocation Hierarchy: OPO to Region to Nation
• Livers are divided into two major groups:– STATUS 1: patients with acute
liver failure with a LE of <7 days
– CHRONIC patients, allocated by MELD Score (a statistical score representing probability of dying in 3 months) that varies between 6 (healthiest) and 40 (sickest)
OPO
Region
Nation
Status 1
MELD Score
1
2
3
4
5
6
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
9
Goal of overall modeling effort
• Goal is to build a model that represents the clinical natural history of ESLD and then superimpose selection, timing and allocation policies on top of that model
• Requires a clinically robust, detailed model of the progression and natural history of liver disease
Waiting list
Organ
Different rules will imply that different patients receive organs at different times in their disease: post transplant success is a function of clinical characteristics of the recipient and the donor
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
10
ESLD Clinical Model: Chronic Disease"L
IVE
R F
UN
CT
ION
"
TIME
THERAPY REQUIRED
DEATH
CHRONIC DISEASE
SYMPTOMDEVELOPMENT “Natural History”
COMPLICATIONS
Imagine there was a single marker of “liver function” that could be tracked over time. As liver function declines, various clinical events begin to occur
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
11
Average Natural History:LI
VE
R F
UN
CT
ION
TIME
Therapyrequired
Death
symptomdevelopment
“Natural History”
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
12
Effect of Natural HX on Transplant Success
• As liver disease progresses,the success of transplantation changes:– Increase operative death
– Decreased post-op survival
100%
80%
60%
40%
20%
0%
OP
ER
AT
IVE
MO
RT
AL
ITY
OPERATIVE MORTALITY
POST TRANSPLANT SURVIVAL
DECLINING LIVER FUNCTION(PROGRESSION OF DISEASE)
PO
ST
-TR
AN
SP
LA
NT
SU
RV
IVA
L (
YR
S)
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
13
Post Transplant Survival by Severity of Disease
• If transplanted early, there is little operative death
• As disease progresses, operation carries higher mortality risk, and post TX survival declines
• Eventually patients become extremely high risk
Early (Asymptomatic)
Intermediate(sick)
Late (very sick)
TIME
SU
RV
IVA
L
Transplantation Survival by stage of disease
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
14
Natural History and Post TX Survival
• Transplantation early may provide less post-tx survival than Nat Hx
• Transplanting too late may provide post tx survival that is to short
Natural History(no transplant)
Early transplantIntermediateTransplant
Late Transplant
SU
RV
IVA
L
TIME
Transplantation Survival vs. Natural History
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
15
Markov Model: Initial attempts to modelSTABLEWITHESLD
AWAITTRANSPLANT
RECEIVEGRAFT
STABLEAFTER
TRANSPLANT
GRAFTFAILURE
DEATH
TRANSPLANTSELECTION
TRANSPLANT
(2) ELECTIVELY (1) EMERGENTLY(3) WAIT
NO
YES
YES
NO
NO
YES
YES
NO
NO YES
DIE 1st 24 HOURS
p(die)=Logit(Age, bilirubin, creatinine ..., etc ...)
p=BOOLEAN(EMERGENT TX CRITERIA)
p=BOOLEAN(ELECTIVE TX CRITERIA)
COMPLICATION
p(die)=f(covariate vector)
NO YES
REPLICA OFSUBTREE # 1
COMPLICATION(Subtree #1)
FATALNOYES
YES
YES
NO
NO
YES
NO
COMA
SEPSIS
COMA
MEDICAL
THERAPY
SUCCEEDSFAILS
RENALFAILURE
p=f(COMPLICATIONS, SEVERITY, TIME IN QUE)
UPDATE TIME-DEPENDENT PORTIONSOF COVARIATE VECTOR
EMERGENT DUMMY SET = TO 1
COMPLICATION
YES
NO
TO STABLE
WITH ESLD
SEPSIS
SEPSIS
SEPSIS
(using various non-parametric and semi-parametric models of survivalanalysis that allow for time-varying covariate effects)
How do we do this part?
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
16
Optimal Timing: asking the wrong question
• We (society) doesn’t chose a “time”, we choose a strategy
• When different people are transplanted is a function of the system
• This question is much more relevant in living donor transplants
• You have seen the work by Oguzhan Alagoz, PhD (a former student) on optimizing this problem
• So, we wanted to look at the societal question: what are the consequences of various allocation rules
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
17
Discrete Event Simulation
• Methodology directly applicable to the problem
• Can model the queues formed, and the other characteristics of the natural history, survival, etc.
• DES simulation allows for competition between resources
• DES models the specifics of the situation – Actual number of people on the list
– Number of transplants
– Number on waiting list
• These are questions that CANNOT be addressed by RCTs or standard statistical methods
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
18
Discrete Event Simulation: The Liver Transplant Model
• Model individual patients presenting with liver disease
• Model individual organs generated by donors
• Model individual transplant centers
• Model pre and post-transplant survival
• Model natural history
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
19
Basic Model Structure
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
20
Patient Generator
• Disease (10 Groups)
• Gender
• Age
• Race
• Blood Type
• Laboratory values– Bilirubin– Creatinine– PT– Albumin
Organ Procurement Organizations (OPOs)(which are clustered into
Regions)
OPO1
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
21
Organ Generator
Organ Procurement Organizations (OPOs)
OPO1
• Gender• Age• Race• Blood Type
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
22
Data Dependencies
PBC
Female
Home
AlbuminBilirubin
CreatinineProthrombin timeEncephalopathy
PSC Hepatitis Other
Male
Hospital ICU
Age (category)
Caucasian
Arrival rate
Year 1990 1991 1992 1993 1996
Hepatitis
Disease (10 groups)
Age
Location
AA Hispanic
Race
Gender
LaboratoryValues
Other
Type A
Caucasian
Type O
Blood Type
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
23
Regional/Geographic Overlay
Transplant Center
Transplant Center
Transplant CenterRegion 2
Region 1
DONOR
DONORPOOL
National Waiting List
Center Waiting List
Center Waiting List
Center Waiting List
AllocationAlgorithm
DONOR
Region 1 Waiting List
Region 2 Waiting List
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
24
Model incorporates current regional preference
• Model could arbitrarily change to any level of regional prioritization or not
OPO
Region
Nation
Status 1
MELD Score
1
2
3
4
5
6
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
25
Allocation Mechanism: Generic structure
1
2
3
4
Arbitrary number ofPRIORITY LEVELS
Arbitrary Number of LISTS (Regional/OPO, National)
Criteria for Membership in LEVEL
RANKING WITHIN PRIORITY LEVEL
N
PRIORITY LEVEL
•••
OR
DE
R O
F S
EA
RH
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
26
UNOS Algorithm (1999-2002)
Priority
1
2
3
4
NOT USED
NOT USED
1
2a
2b
3
Regions: 10 OPOsStatus Levels: 4
Ranked by points (ABO; relative time on list)
In ICULE < 7 days….
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
27
MELD Algorithm (post 2002)
Priority
1
MELD 40
MELD 39
MELD 38
Ranked by points (relative time on list at that score or worse)
In ICULE < 7 days….
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
•••
MELD 6
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
28
Disease Progression Module
• Modeling natural history– Discrete event simulation requires the ability to predict
(quantitatively) the changes in clinical parameters over time
• Traditional statistical methods are not suited to do this concurrently
where
• Data available is likely biased
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
X(t+1) = f(X(t))or
X(t+t) = f(X(t),t)
X = (x1x2,x3, ••• xn)
Clinical covariates of interest
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
29
Natural History Estimating Problem
Time
Bili
rub
in
Observed values of variable
“Average Natural History”
Evaluation
Transplant
Natural history according to NIDDK
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Patients are “over-sampled” when they are sick, under sampled when they are healthy
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
30
Natural History: Prior Simulation Efforts
Status 1
Status 2a
Status 2b
Status 3
12a2b3
1 2a 2b 3
Time 2
Tim
e 1
In earlier simulation (ULAM), by Pritzker and UNOS, the natural history model is directly tied to the allocation/selection model
p11 p12a p12b p13
p2a1 p2a2a p2b2b p2a3
p2b1 p2b2a p2b2b p2b3
p31 p32a p32b p33
Cannot modify this to assess the effect of the change to the MELD score, for example
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
31
Natural History Modeling
• Laboratory data does not come at regular intervals:– More dense when patient is sick (over-sampled)
– Less dense when patient is healthy (under-sampled)
• Actual laboratory data is interpolated using cubic splines
Observed bilirubin
Cubic spline estimated bilirubinEstimated cubic spline
Time
t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15
Bil
iru
bin
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
32
Maintaining correlations in the data
• All of the laboratories are sampled at the same time, keeping relationships between laboratories
• Each persons laboratory history is decomposed into a series of overlapping triplets– Each triplet characterizes
a short time interval for that patient: labs yesterday, today, and tomorrow
Time
t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15
Bili
rub
in L
ev
el (
mg
/dL
)
Bilirubin Albumin
t 1 t 2 t 3 t 1 t 2 t 3
t 2 t 3 t 4 t 2 t 3 t 4
t 3 t 4 t 5 t 3 t 4 t 5. . . . . .. . . . . .. . . . . .t n-2 t n-1 t n t n-2 t n-1 t n
Time
t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15
Alb
um
in L
ev
el (
mg
/dL
)
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
33
Natural History: Stratification
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
1) Create splines
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
2) Cut into sequential triplets
Patients with Primary Biliary Cirrhosis
3) Shuffle
PBC
1) Create spline
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
Time 1 Time 2 Time 3 t1 t2 t3 t2 t3 t4 t3 t4 t5
t13 t14 t15
2) Cut into sequential triplets
3) Shuffle
Patients with Hepatitis B
Hep B
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
34
Natural History, Stratification
DZ 1 DZ 2 DZ 3 DZ 4 DZ 5
DZ 1 DZ 2 DZ 3 DZ 4 DZ 5
DZ 1 DZ 2 DZ 3 DZ 4 DZ 5
Five Disease Groups
Out of Hospital
In Hospital
In Intensive Care
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
35
Disease Progression Mechanism
t-10.93.91.21728
t1.03.91.31829
t+11.23.61.82135
A 42 year old male with hepatitis C
Among all 40-50 old male patients with hepatitis C, find one with “similar” laboratory profile
The “similar” patient’s time t+1 values become the current patient’s time t+1
values.
CreatALBtBILIPTALT
t+1?????
t-11.24.21.01530
t1.34.01.81845
1
2
3
CreatALBtBILIPTALT
CreatALBtBILIPTALT
t-11.24.21.01530
t1.34.01.81845
t+11.23.61.82135
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
36
Determining “similarity”
• Assessed TX surgeons, gastroenterologists
• Determined how different each lab had to be to be “clinically important”
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
37
Modeling Natural History
50 year old Hispanic female with Alcoholic Liver disease
Patient #1
0
5
10
15
20
25
Days
La
bo
rato
ry v
alu
es
0 200 400 600 800
Prothrombin time (sec)Bilirubin (mg/dL)Albumin (g/dL)Creatinine (mg/dL)
1000 1200
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
38
Modeling Natural HistoryPatient #2
0
5
10
15
20
25
0 200 400 600 800 1000 1200Days
Lab
ora
tory
val
ues
50 year old Hispanic female with Alcoholic Liver disease
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
39
Modeling Natural History
0
5
10
15
20
25
0 2 4 6 8 10 12 14 16 18Days
La
bo
rato
ry v
alu
es
ICU Hospital ICU
Death
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
40
Modeling Natural History Results:
* Shown only for one group of ESLD diagnoses (primary biliary cirrhosis, primary sclerosing cholangitis, alcoholic liver disease, and autoimmune disorders). Differences between actual and simulated change are within levels considered “clinically insignificant” by clinical advisory group.
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Laboratory ValueActual
ChangeModel
ChangeBilirubin -2.638 -2.515Creatinine -0.197 -0.528Albumin 0.334 0.221PT -0.838 -0.796
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
41
Modeling Natural History Results:
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Correlations between Clinical Covariates
Bilirubin Creatinine Albumin PT
Bilirubin 1
Creatinine 0.329 1
Albumin -0.100 -0.006 1
PT 0.389 0.119 -0.273 1
Bilirubin Creatinine Albumin PT
Bilirubin 1
Creatinine 0.329 1
Albumin -0.100 -0.006 1
PT 0.389 0.119 -0.273 1
BilirubinBilirubin CreatinineCreatinine AlbuminAlbumin PTPT
BilirubinBilirubin 11
CreatinineCreatinine 0.3290.329 11
AlbuminAlbumin -0.100-0.100 -0.006-0.006 11
PTPT 0.3890.389 0.1190.119 -0.273-0.273 11
Actual Data
Bilirubin Creatinine Albumin PT
Bilirubin 1
Creatinine 0.332 1
Albumin -0.231 -0.024 1
PT 0.494 0.139 -0.431 1
Bilirubin Creatinine Albumin PT
Bilirubin 1
Creatinine 0.332 1
Albumin -0.231 -0.024 1
PT 0.494 0.139 -0.431 1
BilirubinBilirubin CreatinineCreatinine AlbuminAlbumin PTPT
BilirubinBilirubin 11
CreatinineCreatinine 0.3320.332 11
AlbuminAlbumin -0.231-0.231 -0.024-0.024 11
PTPT 0.4940.494 0.1390.139 -0.431-0.431 11
Simulated Data
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
42
Pre-transplant survival
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Pro
po
rtio
n S
urv
ivin
g
0 100 200 300 400
Days
1.0
0.9
0.8
0.7
0.6
0.5
ActualModel p=0.26
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
43
Post Transplant Survival
• Estimated disease-specific post transplant survival curves from sample of ~17,000 transplants from UNOS 1991-1996 w/ follow-up to 1999
• Cox proportional Hazards models• Model transplants the patient at a given time, and “knows”
the clinical covariate vector at that time– age, gender, bilirubin, creatinine, PT, albumin, encephalopathy
• Model generates a covariate-adjusted cumulative hazard• Hazard function is randomly samples to arrive at a specific
survival time• Re-estimated for Cox model predicting graft survival
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
44
Post Transplant Survival
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
0.9
0.7
0.5
0.3
0.1
1.0
0.8
0.6
0.4
0.2
1 2 3 4 5 6 7 8 9
Years
Po
rtio
n s
urv
ivin
g
0
(1596)
(1487)
(4460)
(3150)
(784)
PBC
PSC
HEP C
ALD
HEP B
Autoimmune
Cancers
Metabolic
Fulminant
Other
(2839)
(538)
(583)
(954)
(653)
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
45
Post Transplant Survival
• Cox proportional Hazard models by disease
Patient Survival
Graft Survival
If Patient Survival > Graft Survival, patient is RELISTED at time of graft failure
If Patient Survival < Graft Survival, patient dies at survival time
Organ characteristics
Patient characteristics
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
46
Cost Module
• Not yet implemented
• Extract of ~2000 patients from UNOS matched to CMS claims
• Costs of care– Pre-transplant/transplant/post-transplant
• Disease-specific, location (in hospital out of hospital) specific
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
47
Quality of Life module
• Prospective evaluation of patients awaiting transplant
• Formal Utility Assessments– Standard Gamble
– Time Trade off
– Visual Analog Scale
– NIDDK Quality of Life Questionnaire
• Hoped to predict utility from QOL question responses
• Entered ~ 130 patients
Out of HospitalIn HospitalIn ICUPost Transplant
TTO.74.54.40.84
SG.67.41.40.64
From literature
Discrete EventSimulation Model
Patient Generator
Organ Generator
Survival Module
Disease Progression
Module
Resource Use
Module
Quality of Life
Module
Selection and Allocation
Rules
Model OutputsUser-defined
Inputs SurvivalQuality-Adjusted Survival Costs spent on ESLDNumber of deaths waitingAverage waiting timeNumber of wasted organs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
48
PatientGenerator
Alive on Waiting List
Improved
“Too sick”
RefusedTransplant
Alive Post Transplant
Dead
Organ MatchOrgan
Generator
Unused
Die Post Transplant
OrganFailure
Die WhileWaiting
Post TransplantSurvival
Quantitative Natural History
Waiting List
Removals
Distribution by:• Region/OPO• Disease• Age• Gender• Race• ABO
•Location• CMV• Prior TX• Clinical Hx• laboratories
Distribution by:• Region/OPO • Age• Race• Gender• CMV • ABO
Post Transplant Patient Survival
Post Transplant Graft Survival
# of organs wasted
# of organs transplanted
# died priorto transplant
# removed from list
• Bilirubin• Albumin
• Creatinine• INR
Arbitrary priority scheme
Arbitrary allocation
rules
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
49
Calibration and Validation
Variable# of Transplants
# Waiting
Deaths while waiting
Waiting Time (median)
199226142599
19031880
516473
124142
199329472946
28092548
567514
175193
199431293124
38383544
671589
242217
199534703460
53655072
835754
345316
199635673583
72036795
1000919
n/an/a
ModelUNOS
ModelUNOS
ModelUNOS
ModelUNOS
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
50
Allocation Rule Predictions
• We have examined 4 alternative strategies– Original UNOS ranking, local preference
– Original UNOS ranking, national List
– Current MELD ranking, local preference
– Current MELD ranking, national List
• Compare several outcome between multiple scenario runs under each set of conditions
• Use the model to develop (calculate) EMERGENT PROPERTIES – these are properties that are measurable in real world but are
calculated by the model, not used as inputs
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
51
Results:
Outcome measure
Patients relistedDeaths while waiting1 year patient survival1 year graft survivalmedian wait time (days)mean survival (years)Mean survival (QALYS)
UNOS Regional
162235890.840.782529.516.65
UNOS National
194731690.820.753469.636.56
MELD Regional
167036120.840.781819.326.67
MELDNational
198531490.820.752849.466.65
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
52
Geographic Variability
• Model captures the remarkable geographic variability in waiting times, which is eliminated with move to national list
0.00
100.00
200.00
300.00
400.00
500.00
600.00
700.00
800.00
900.00
1 2 3 4 5 6 7 8 9 10 11
Region
Med
ian
Wai
tin
g t
ime
Regional National
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
53
SRTR efforts: Simulation Allocation Models (SAMs)
Simulation Allocation Model (SAM)
Donor Organs
Transplant Candidates
Outcomes Under Policy A
Outcomes Under Policy B
DiseaseProgression
Waiting List
Unused Organs
ComparePolicies
Post Transplant Events
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
54
SRTR Natural History Model
time
time
time
Bili
rub
inA
lbu
min
Pro
thro
mb
in t
ime
Pick one individual, use that person’s actual history
•What do you do when model actual history?•How to interpolate?
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
55
SRTR evaluation of transplant policy: Lung
• Lung transplant rules used waiting time as major prioritization
• Recently (2005) changed from longest wait first to sickest first
• Results have dramatically changed the survival in chronic progressive lung disease
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
56
SRTR evaluation of transplant policy: Lung
September 24, 2006
Lung Patients See a New Era of Transplants
By Denise Grady
A quiet revolution in the world of lung transplants is saving the lives of people who, just two years ago, would have died on the waiting list.
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
57
Current allocation question
14 15 17 19 21 24 25 26 28 31 34 36
14 15 17 19 21 24 25 26 28 31 34 36
Sickest first
Largest Net Benefit First
14
Survival with THIS organ (Stx)
Survival with NO organ (Sno-tx)
NET Benefit = Stx – Sno-tx
6.9 7.2 6.7 6.3 7.2 7.6 7.3 8.1 8.2 7.4 7.8 7.5NB:
Patients ranked by MELD score (probability of death in 3 months)
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
58
Summary:
• Simulation methods match the problem in this context
• DES allows for queues, waiting times, etc to be emergent properties of the model
• Example of biological modeling with a policy overlay
• So, why is it so accepted?
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
59
Simulation model acceptance by Transplant community
• Clear standard research methods won’t work– Impractical (and likely illegal) to randomize
• Model was built with clinical oversight and assistance
• Model demonstrates “predictive validity”– Model predicts the effects of rules change
– Rules are changed
– Observe the actual results
Section of Decision Sciences and Clinical Systems ModelingSECTION OFDECISION SCIENCESAND CLINICAL SYSTEMSMODELING
SDSCSM
60