Examples of multi-state models Models for intensities Estimating transition and state probabilities
Multi-state models
Per Kragh Andersen
Section of Biostatistics, University of Copenhagen
DSBS CourseSurvival Analysis in Clinical Trials
January 2018
1 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Overview
Examples of multi-state models
Models for intensities
Estimating transition and state probabilities
www.biostat.ku.dk/˜pka/SACT18-part2
2 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Examples of multi-state models
3 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
The two-state model for survival data
Alive Dead-λ(t)
0 1
λ(t) ≈ P(state 1 time t + dt | state 0 time t)/dt
S(t) = P(state 0 time t)
F (t) = 1− S(t) = P(state 1 time t) is the cumulative
probability (”risk”) of death over the interval from 0 to t
F (t) = 1− exp(−∫ t
0λ(u)du).
4 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
The competing risks multi-state model
Alive0
Dead, cause kk
QQQQQQs
Dead, cause 11
������3
λ1(t)
λk(t)
ppp
5 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Basic parameters
Cause-specific hazards j = 1, ..., k (“transition intensities”):
λj(t) ≈ P(state j time t + dt | state 0 time t)/dt.
State occupation probabilities:
1 Overall survival function:
S(t) = P(alive time t)
= exp(−∫ t
0
∑j
λj(u)du).
2 Cumulative incidences j = 1, ..., k:
Fj(t) = P(dead from cause j before time t)
=
∫ t
0S(u)λj(u)du.
6 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
More types of event of interest
1 Time from BMT to either relapse or death in remission
2 Time from randomization to the occurrence of an adverse event or to withdrawal
3 Time from entering a wait list until heart transplantation, or to death while onwait list, combined with time to death after transplantation
4 Time from BMT to either relapse or death in remission, combined with time todeath after relapse
5 Time(s) from randomization to the occurrence of first, second, ... hypoglycemicevent
6 Time(s) from first diagnosis to first, second, ... re-admission to hospital
7 Time(s) from randomization to the occurrence of first, second, ... hypoglycemicevent, combined with time to an adverse event or to withdrawal
8 Time(s) from first diagnosis to first, second, ... re-admission to hospital,combined with time to death
The first two are covered by the competing risks model.
7 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Stanford Heart Transplant Data
Crowley and Hu (J. Amer. Statist. Assoc., 1977) presenteddata from the Stanford Heart Transplantation Study
Patients identified as being eligible (n=103) for a hearttransplant were followed until death or censoring
In total, 69 received a transplant during follow-up, and 34 didnot
75 patients died during follow-up - some on the wait list,some after transplantation
The purpose was to study whether transplanted patients livelonger
8 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
An illness-death model
Waiting Transplant
Dead
-
SSSSSSw
������/
λ01(t)
λ02(t) λ12(t | past)
0 1
2
9 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Illness-death model: parameters
Transition intensities:
λ01(t) ≈ P(state 1 time t + dt | state 0 time t)/dt,
λ02(t) ≈ P(state 2 time t + dt | state 0 time t)/dt,
λ12(t) ≈ P(state 2 time t+dt | state 1 time t, past at time t)/dt.
Note that the interpretation of these is identical to those in thesimple (two-state or competing risks) multi-state models but thatthe intensity out of the ‘transient’ state 1 involves ‘the past’ attime t.This could include
Time spent in state 1 at time t (time-dependent)Time of entry into state 1 (time-fixed)
10 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Another ‘illness-death’ example
In the EBMT or the ‘bmt1715’ data sets, no information on thedisease course after relapse was available.Some times, that could be the case (Andersen and Pohar Perme,LIDA, 2008):
2009 patients had data extracted from the Center for International Blood andMarrow Transplant Research
Patients suffered from AML (n=1406) or ALL (n= 603) and got a transplant(bone marrow and/or peripheral blood stem cells) in first complete remissionfrom an HLA-identical sibling between 1995 and 2009
The mean age was 31.9 years (SD=15.4 years) and n=896 patients (44.6%)were females. No patients were younger than 2 years old
259 relapses of whom 232 (89.6%) died during follow-up
737 deaths in remission
989 experienced (acute or chronic) Graft-versus-host disease (GvHD)
As in the other BMT studies, the composite end-point ‘relapse-free survival’ wasalso of interest
Here, the mortality rate (λ12) after relapse could depend on timeof/time since relapse.
11 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: the PROVA trial
(PROVA study group, Hepatology, 1991; Andersen, Esbjerg andSørensen, Stat. in Med., 2000).
286 patients with liver cirrhosis and endoscope-verifiedoesophageal varicesrandomized in a 2 by 2 design to combinations ofsclerotherapy (yes/no) and propranolol (yes/no)Primary outcomes: bleeding or death without bleeding, butdeath after bleeding was also of interest
Treatment Patients Bleedings Deaths Deathswithout bleeding total
Sclerotherapy only 73 13 13 18Propranolol only 68 12 5 11Both 73 12 20 30Neither 72 13 8 16
Total 286 50 46 7512 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
An extended illness-death model for BMT data
Remission
10
GvHD
Relapse Dead
-
-
2
HHHH
HHHHHHHj
����
��������
? ?
3
λ01(t)
λ03(t) λ12(t | past)
λ23(t | past)
λ02(t) λ13(t | past)
13 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events: mortality negligible, ‘no duration’
No event 1 event-λ01(t)
- 2 eventsλ12(t)
-λ23(t)0 1 2
Again, transition intensities could depend on the past. This leadsto considering a single transition intensity function:
λ(t) ≈ P(event in (t, t + dt) | past)/dt,
where the past would include information on previous events (i.e.,in (0, t)), e.g., number and/or times of events.
14 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: tumors in rats
Data from Cook and Lawless 2007 Springer-book “The StatisticalAnalysis of Recurrent Events” (“orginally” from Gail et al., 1980,Biometrics).76 female rats were exposed to a carcinogen and then given retinylacetate to prevent cancer for 60 days. 48 rats, still tumor-free,were randomized to either continued treatment (23) or control(25) and followed for another 122 days. They were examined fortumors twice weekly and times of tumors were noted. The data setincludes the variables:
id
start, stop, status (tumor or not)
num (record no.)
trt (treatment indicator)
15 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: clinical trial in stage I bladder cancer
Trial conducted by the Veterans Administration CooperativeUrological Research Group (Byar, 1980) - famous text bookexample.
118 (86) patients with stage I bladder cancer
randomized to pyridoxine (32, NB: not included in this data set), thiotepa (38),or placebo (48)
followed for the occurrence of superficial bladder tumors
Data set includes:
subject: person-idenum: record number per subjectstart: start time in monthsstop: stop time in monthsstatus (at time stop): 0 = alive, no new tumor, 1 = alive,new tumor, 2 = dead, no new tumor, 3 = dead and new tumornumber: number of tumors at time 0size: largest tumor at time 0trt: 0 = placebo, 1 = thiotepa
16 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events ‘with duration’, mortality negligible
Event-free10
Event-
�
λ01(t | past)
λ10(t | past)
Here, there are ‘gaps’ after occurrence of the event where thesubject is not at risk for a new event. In both states there may bea past to consider when modeling the intensities.
17 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Pulmonary exacerbations
Data from Cook and Lawless (2007) book(orginally from Fuchs et al., NEJM, 1994; also used by Therneauand Hamilton, 1997, Statist. in Med.).645 patients with cystic fibrosis randomized to rhDNase (321) orplacebo (324) followed from randomization and about 169 days.The data set includes the variables:
id
trt (treatment indicator), fev, fev2 (baseline measurements)
start, stop, status
etype (1 if ‘at risk’, 2 if ‘under treatment’)
enum (record no.), enum1 (gap time no.), enum2 (treatmentperiod no.)
18 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Psychiatric admissions (1)
Kessing, Olsen and Andersen, Amer. J. Epidemiol. (1999) studieddata from the Danish Central Psychiatric Registry DCPR:
All psychiatric admissions 1 April 1970-
Discharge diagnoses
ICD-8 until 31 december 1993
ICD-10 from 1 January 1994
and linked to the Danish Civil Registration System CPR to obtaininformation on vital status and emigration.
19 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Sample from register data.
All patients identified in DCPR before 1 January 1994 with abipolar (‘manic’) diagnosis at first discharge
Restrict attention to patients younger than 52 years atdiagnosis, 1307 men and 1655 women
Followed to 31 December 1999 w.r.t. later psychiatricadmissions, death, diagnosis of schizophrenia or emigration
Purpose of study: Evaluate theory of ‘sensitization’ (or ‘kindling’).According to this theory, mood episodes themselves stress thebrain so that its sensitivity to biologic and psychosocial stressorsincreases. This leads to shorter and shorter intervals betweensuccessive episodes, e.g. (Post, 1992, Amer. J. Psych.)
20 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Psychiatric admissions (2)
Clinical data collected by Swiss psychiatrist Jules Angst (!) inZurich - also interest in sensitization.Here, we restrict attention to prospectively collected data onpatients with a first diagnosis after 1958:
133 patients with unipolar (‘depression’) or bipolar (‘manic’)disorder
dates on admission to and discharge from psychiatric hospital
covariates includesexage at first diagnosisyear at first diagnosis
Some times, ‘cycles’ (time from admission to re-admission) areconsidered instead of times from discharge to re-admission. Thatbrings the example into the framework of recurrent events with ‘noduration’.
21 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events and mortality, ‘no duration’
No event 1 event-λ01(t)
- 2 eventsλ12(t)
-λ23(t)0 1 2
DeadD
?
λ1D(t)
@@@@@@@R
λ0D(t)
��
��
���
λ2D(t)
22 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: hypoglycemic events in clinical trial
Data on time to drug discontinuation for different reasons in a1-year RCT of active drug vs. placebo, n=559 patients.A treatment emergent hypoglycemic event is an event with onsetdate at or after the first day of treatment and no later than 14 daysefter the last day of treatment. The event process is interrupted bydrop-out (e.g., drug discontinuation) before week 52.
id subject id
enum record number per id
start in weeks
stop in weeksstatus at time stop
0: complete on drug1: hypo event2: drop-out
drug 0: placebo, 1: active
23 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: clinical trial in stage I bladder cancer
Trial conducted by the Veterans Administration CooperativeUrological Research Group (Byar, 1980) - famous text bookexample. The fact that 30 patients died during follow-up has beenneglected by many authors who have used the data as illustration!
118 (86) patients with stage I bladder cancerrandomized to pyridoxine (32/7, NB: not included in this data set), thiotepa(38/12), or placebo (48/11)followed for the occurrence of superficial bladder tumors
Data set includes:
subject: person-idenum: record number per subjectstart: start time in monthsstop: stop time in monthsstatus: 0 = alive, no new tumor, 1 = alive, new tumor, 2 =dead, no new tumor, 3 = dead and new tumornumber: number of tumors at time 0size: largest tumor at time 0trt: 0 = placebo, 1 = thiotepa
24 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Psychiatric admissions (2)
Clinical data collected by Swiss psychiatrist Jules Angst in Zurich.Here, we restrict attention to prospectively collected data onpatients with a first diagnosis after 1958:To be honest, 85 patients died during follow-up. If that is takeninto account and if we study ‘cycles’ then this is an example ofrecurrent events with mortality and no duration.
133 patients with unipolar (‘depression’) or bipolar(‘manic-depression’) disorder
dates on admission to and discharge from psychiatric hospital
covariates include
sexage at first diagnosisyear at first diagnosis
25 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events ‘with duration’ and mortality
Event-free10
Event
Dead
-
�
D
SSSSw
����/
λ01(t)
λ10(t)
λ0D(t) λ1D(t)
Again, all intensities may depend on the past.
26 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Psychiatric admissions
Both in the Danish registry data and in the Swiss clinical data,patients may die (or get a diagnosis of schizophrenia).When these ‘final’ events are taken into account we are in thesituation of recurrent events with duration and mortality.
27 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Models for intensities
28 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Likelihood
A multi-state model, say X (t) involves different states: h = 1, ..., kand types of direct transition, h→ j , h, j = 1, ..., k ; h 6= j .The counting process Nhj(t) counts the number of direct h→ jtransitions in [0, t]. Let Yh(t) be the number of subjects observedin state h at time t−. Then the intensity process for Nhj(t) is:λhj(t)Yh(t) ≈ P(X (t + dt) = j | X (t−) = h, past)/dt,
for some function λhj(·) of time and past.According to Jacod’s formula, the likelihood for the λ’s based onobservation of (Nhj(t),Yh(t), 0 ≤ t ≤ τ) (censoring allowed) is
L =∏t
∏h,j ;h 6=j
(λhj(t)Yh(t)dt
)dNhj (t)×exp(−∑h,j
∫ τ
0λhj(u)Yh(u)du)
=∏h,j ;h 6=j
(∏t
(λhj(t)Yh(t)dt
)dNhj (t)exp(−
∫ τ0 λhj(u)Yh(u)du)
).
29 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Models for intensities
As we saw it for competing risks (which is of course a specialcase), this likelihood factorizes into a product over transitions.
As a consequence, if no λ’s have parameters in common thentransition intensities may be modelled separately.However, in contrast to the competing risks situation, models withshared parameters may be relevant in some of the ‘new’ models,e.g. the mortality rates in the illness-death model may beproportional and/or risk factors may have same effects on 0→ 2and 1→ 2 transitions.
We may therefore conclude that:1 hazard models from simple survival analysis (and competing
risks) are applicable, e.g. Cox models2 modelling transitions out of non-initial transient states poses
new challenges30 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Markov models
An important simple situation is when the multi-state process X (t)is Markov:
P(X (t+dt) = j | X (t−) = h, past) = P(X (t+dt) = j | X (t−) = h),
that is, the only dependence on the past at time t is via thecurrent state h = X (t−).In this case, as we shall see, both state occupation probabilitiesQh(t) = P(X (t) = h) and transition probabilities
Phj(s, t) = P(X (t) = j | X (s) = h), h, j = 1, ..., k , s ≤ t
may be derived from the intensities.
31 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Stanford heart transplantations
Waiting Transplant
Dead
-
SSSSSSw
������/
λ01(t)
λ02(t) λ12(t | past)
0 1
2
32 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Stanford heart transplantations
The main emphasis is on a comparison of the mortality rates with(λ12(t)) and without (λ02(t)) transplantation, possibly afteradjusting for age at entry. Possible models include:
1 λh2(t), h = 0, 1 are completely unspecified. This corresponds to atime-dependent stratification according to transplantation status. A comparisonbetween the two rates may be performed using the logrank test (with delayedentry for λ12(t)).
2 λh2(t), h = 0, 1 are proportional. This corresponds to treating transplantationstatus as a time-dependent covariate and comparison between the two rates issimply a test for the effect of this covariate. Adjustment for other variables canalso be done easily.
3 Both of these models are Markov. A non-Markov model could be one whereλ12(t) depends on wait, the time from entry to transplantation. Comparisonbetween the two rates is not obvious, since they are not adjusted for the samevariables. Another non-Markov model could be one where λ12(·) depends ontime since transplantation (t-wait).
4 In all models, adjustment for age at entry could be made, either with the sameeffect or with different effects (interaction) before and after transplantation.
33 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Stanford heart transplantations
Other possible models:
1 Instead of using time, t since entry as the time variable andadjusting for age at entry, one could model baseline hazards interms of current age (= t+ age at entry).
2 This could be done with or without adjustment for t.
3 The transition intensity λ01(t) of receiving a transplant couldalso be modelled though this parameter was not of primaryinterest.
34 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events, the PWP model
In all of the recurrent events scenarios studied (i.e., with or withoutduration and with or without mortality) intensity models similar tothose discussed for the heart transplant data may be used.One option is to use separate models (possibly with sharedcovariate effects) for λh,h+1(t), h = 0, 1, ....
This is known as the PWP model (Prentice, Williams andPeterson, Biometrika, 1981):
λh,h+1(t | X ) = λh0(t) exp(βTX ), h = 0, 1, ...
This is a Cox model with time-dependent strata.
The effect of X might vary with the number of previous events, i.e.βh instead of β.
35 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events, the AG model
Assuming proportionality between the different λh0(t), a specialcase of the AG model (Andersen and Gill, Ann. Statist, 1982) isobtained.In this model there is a single event intensity:
λ(t) = λ0(t) exp(βTX (t))
that is allowed to depend on the past (e.g., number of previousevents N(t−)) via time-dependent covariates which may alsointeract with other covariates.The model may be fitted using delayed entry for later events.If there are only time-fixed covariates, this is an inhomogeneousPoisson process which is Markov.
36 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
AG model in SAS
We look at the simple rats data:
PROC PHREG DATA = rats;
MODEL stop*status (0)=trt/ ENTRY =start RL;
RUN ;
or, equivalently:
PROC PHREG DATA = rats;
MODEL (start ,stop)*status (0)=trt/ RL;
RUN ;
37 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Recurrent events, gap time models
Gap time models are models where the baseline intensity depends,not on t, but on time t − TN(t−) since last event, e.g.:
λh,h+1(t) = λh0(t − Th) exp(βTh X ).
If there are no covariates and successive gap times are i.i.d., this isa renewal process.These models are some times called semi-Markov.To fit the model, no delayed entry is needed: for each transitionh→ h + 1, subjects are at risk from the ‘new time zero’.
Putter et al. (Statist. in Med., 2007), in their tutorial onmulti-state models, called such models ‘clock reset’ (in contrast toMarkov models which were called ‘clock forward’).
38 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: Pulmonary exacerbations
AG-type Cox models:
Covariate β SD β SD
Treatment -0.29 0.11FEV -0.017 0.002 -0.017 0.002Treatment (t ≤ 80) -0.42 0.16Treatment (t > 80) -0.14 0.14I (Ni (t−) > 0) 0.81 0.23
39 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Example: psychiatric admissions
A gap time model was used because of the ‘renewal features’ ofthe data:When evaluating the tendency to re-admission it is more relevantto consider time since last event than time since diagnosis.A simple such model (without covariates) would be:
λi (t) = λh0(t − Tih)
where h = Ni (t−) is the number of episodes before time t sincediagnosis.For each patient, we then have a process allowing for different gaptime distributions for different numbers of previous episodes (samedistribution across patients).Estimate the survival function:
Sh(w) = exp(−∫ w
0λh0(u)du)
by the Kaplan-Meier estimator? 40 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Kaplan-Meier estimators
Kaplan-Meier estimates of the survivor functions for 1st, 2nd andlater gap times.
0 1000 2000 3000 4000
0.00.2
0.40.6
0.81.0
Time(days)
Survi
val F
uncti
on
Men 1wt
2wt
3wt
4wt
5wt
0 1000 2000 3000 4000
0.00.2
0.40.6
0.81.0
Time(days)
Survi
val F
uncti
on
Women 1wt
2wt
3wt
4wt
5wt
41 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Using Kaplan-Meier curves
The Kaplan-Meier curves do not properly address the problem ofsensitization due to selection/heterogeneity – those with 2 episodesare a select subgroup of those with 1 etc.Another way of stating the problem is via dependent censoring:e.g., censoring of the second gap time depends on the first gaptime and if successive gap times are correlated then censoring ofthe second gap time depends on the second gap time:Ui : censoring time for subject i , Wi1, Wi2: first and second gaptime for subject iUi2: censoring time for Wi2 is Ui2 = Ui − wi1.If Wi1 and Wi2 are dependent, then Wi2 and Ui2 are alsodependent and the Kaplan-Meier estimator does not work forS2(w).
42 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Random effects (‘frailty’) models
Possible model for intensity of re-admission λi (t):
λi (t | Zi = zi ) = zi · λ0(t − Tih) exp(βTXi (t))
w = t − Tih gap time at time t, since diagnosis
Xi (t): explanatory variables including gender, age at firstepisode, calendar time and number of previous episodes
Zi : random, unobserved frailty assumed to follow somedistribution with mean 1 and SD σ across the patientpopulation, e.g. the gamma distribution.The frailty accounts for dependence between successive gaptimes in each patient.
43 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Results for younger bipolar patients
(Kessing et al., 1999):
Men WomenEpisode (h) Rate ratio 95% c.i. Rate ratio 95% c.i.
1 1.00 ref. 1.00 ref.2 1.18 1.03-1.34 1.22 1.08-1.373 1.46 1.27-1.69 1.47 1.29-1.674 1.72 1.49-2.02 1.61 1.40-1.85
5+ 2.35 2.07-2.67 2.19 1.07-2.45σ2 0 (no frailty) 0 (no frailty)
1 1.00 ref. 1.00 ref.2 0.99 0.84-1.15 1.07 0.93-1.223 1.10 0.91-1.33 1.16 0.99-1.364 1.16 0.93-1.45 1.17 0.98-1.39
5+ 1.30 1.04-1.64 1.25 1.04-1.50σ2 0.45 0.26-0.63 0.41 0.28-0.54
44 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Pulmonary exacerbations
AG-type Cox and frailty models:
Covariate β SD β SD
FEV -0.019 0.003 -0.017 0.002Treatment (t ≤ 80) -0.51 0.18 -0.42 0.16Treatment (t > 80) -0.16 0.16 -0.14 0.14Frailty variance 0.94I (Ni (t−) > 0) 0.81 0.23
Note that the results from a frailty model have subject-specificinterpretations, i.e., hazard ratios for given frailty.
This may or may not be what you want!
45 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Doing it is SAS
We look again at the simple rats data:
PROC PHREG DATA = rats;
CLASS id;
MODEL stop*status (0)=trt/ ENTRY =start RL;
RANDOM id/ DIST = GAMMA ;
RUN ;
However, in this example the subject-specific interpretation is notreally appealing.
46 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Joint frailty models
The frailty accounts for unobserved risk factors in the recurrentevents process, and inference builds on integrating the frailty outof the likelihood.
If there is a non-negligible mortality and if mortality rates dependon the same frailty then a joint frailty model for both the recurrentevents process and the mortality rate is needed.
Rondeau et al. (Biostatistics, 2007) developed such a model whichis implemented in the R package frailtypack.
See also Liu et al. (Biometrics, 2004) and Huang and Wang(JASA, 2004).
47 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Joint frailty models
The frailties Z1, ...,Zn are assumed to be i.i.d. gamma variateswith mean 1 and SD σ. The model is then:
λi (t | Zi = zi ) = zi · λ0(t) exp(βT1 Xi (t))
for the recurrent events intensity and
λDi (t | Zi = zi ) = zαi λD0(t) exp(βT2 Xi )
for the mortality rate. Thus, the frailty is shared but its effect onrecurrent events and mortality could be different, modelled by thepower α. The sets of covariates could differ between the twomodels.
The likelihood is not extremely nice and the authors approximatethe baseline intensities by cubic splines and use a penalizedlikelihood.
48 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Joint frailty models
If there are ‘gaps’ between successive episodes then one must either
1 assume that their distribution is independent of the frailty Zor
2 model the dependence, e.g., by using 1Z as the frailty for that
transition
The second option is not available in frailtypack.
49 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Estimating transition and state probabilities
50 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Probabilities in multi-state models
We have seen that hazard models from survival and competingrisks may also form the basis for modelling intensities in multi-statemodels (albeit with some new challenges).
When studying survival analysis and competing risks we also sawthat getting state occupation probabilities Qh(t) = P(X (t) = h),which for these simple models were equal to transition probabilities
Phj(s, t) = P(X (t) = j | X (s) = h), h, j = 1, ..., k , s ≤ t,
depended on the structure of the model (competing risks or not).
This is still the case for general multi-state models and eventhough a specification of all intensities specifies the likelihood and,thereby, the whole probability distribution for the multi-stateprocess, there are some such models where no simple plug-intechniques are available.
51 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Markov processes
Recall the Markov property:
P(X (t+dt) = j | X (t−) = h, past) = P(X (t+dt) = j | X (t−) = h),
i.e. at any time t the transition intensity λhj(t) only depends onthe current state h (and on time t).Assume that there are k states and that the transition intensitiesand cumulative intensities are, respectively, λhj(t) and Λhj(t) forh, j = 1, ...k . (Some of these may be zero everywhere, e.g., λ10(t)in the two-state model.) The matrix P(s, t) of transitionprobabilities Phj(s, t) = P(X (t) = j | X (s) = h) is then given bya product-integral:
P(s, t) =∏(s,t]
(I + dΛ(u)
),
where Λhh(t) = −∑
j 6=h Λhj(t).52 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Markov processes: The Aalen-Johansen estimator
We can estimate the cumulative transition intensities using theNelson-Aalen estimator:
Λhj(t)
∫ t
0
I (Yh(u) > 0)
Yh(u)dNhj(u).
This means that we can also easily estimate P(s, t) by plug-in:
P(s, t) =∏(s,t]
(I + dΛ(u)
)where, again, Λhh(t) = −
∑j 6=h Λhj(t) (Aalen and Johansen,
Scand. J. Statist., 1978).In fact, both the Kaplan-Meier estimator for the two-state modeland the Aalen-Johansen estimator for the competing riskscumulative incidence have this form and (slightly confusingly) theterm ‘Aalen-Johansen’ is used for both the general estimator andfor the special case of competing risks.
53 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
The Aalen-Johansen estimator, simplest cases
For the simple survival model the 2× 2 matrix at a death time, Tis: (
1− dN(T )Y (T )
dN(T )Y (T )
0 1
)For the competing risks model with two causes of failure, the 3× 3matrix at time T is: 1− dN1(T )+dN2(T )
Y (T )dN1(T )Y (T )
dN2(T )Y (T )
0 1 00 0 1
For a general Markov process, the k × k transition matrix at timeT can be set up similarly.
54 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
The Aalen-Johansen estimator
The calculations have been implemented in the R package mstate
and in Stata one can msset the data and then use standardsurvival routines. SAS does not (yet) provide procedures for doingthis.
The mstate package can also take results, Λhj(· | X ), e.g. from afitted Cox model, as the input and compute the product-integralsfor given covariates.
The package works, in particular, for the competing risks model.This gives an alternative (in R) that is more flexible than the SAS
MACRO %CUMINC.
55 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
State occupation probabilities
The ‘marginal’ state occupation probabilities Qh(t) = P(X (t) = h)equal the transition probabilities P0h(0, t) if every one begins instate 0 at time 0 (otherwise it is a mixture of such transitionprobabilities over the initial state distribution).
This means that state occupation probabilities can also beestimated in Markov processes using the Aalen-Johansen estimator.
The good news is that, as shown by Datta and Satten (Stat. Prob.Letters, 2001), even for non-Markov processes the Aalen-Johansenestimator consistently estimates Qh(t).
56 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
State occupation probabilities
For certain transient states, an alternative and simple estimator forthe state occupation probability is available (Pepe, JASA, 1991):‘the difference between Kaplan-Meier’s estimator’.For the illness-death model:
1 Compute the Kaplan-Meier estimator, say S0(t) for the initialstate 0. (It estimates Q0(t).)
2 Compute the Kaplan-Meier estimator, say S01(t) for survival.(It estimates 1− Q2(t) = Q0(t) + Q1(t).)
3 Compute the difference S01(t)− S0(t). (It estimates Q1(t).)
This idea can be used more generally for transient states in othermulti-state models and often it gives curves similar toAalen-Johansen estimates (e.g., BMT data):
57 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Probability of being alive with relapse
58 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Transition probabilities in non-Markov processes
Methods are developing for estimating transition probabilities innon-Markov processes:
For the illness-death model, Meira-Machado et al. (LIDA,2006) proposed an estimator based on ‘weighted Kaplan-Meierintegrals’. This is implemented in an R package TPmsm.
Titman (Biometrics, 2015) proposed estimators for generalnon-Markov processes based on either the Pepe idea or on anidea of Allignol et al. (LIDA, 2014) using ‘competing risksmethods for the illness-death model’.
Putter and Spitoni (LIDA, in press) used the Datta-Sattenidea to those observed in state h at time s to estimatePhj(s, t), for t > s.
However, we are now on the edge of what is in general use andimplemented.
59 / 60
Examples of multi-state models Models for intensities Estimating transition and state probabilities
Micro-simulation
A technique that is used in practice, e.g. in demography, ismicro-simulation.
If all intensities in the multi-state model are specified then it ispossible to generate a large number of realizations of the processand to estimate, e.g. transition probabilities as simple averagestaken over the repeated realizations (e.g., Iacobelli and Carstensen,Stat. in Med., 2013).
60 / 60