View
83
Download
0
Category
Tags:
Preview:
DESCRIPTION
Using routinely-collected data to estimate patient retention in care and loss to follow-up ICAP Methodology Webinar January 19, 2012. Matthew Lamb mrl2013@columbia.edu ICAP-M&E NY. Upcoming methodology webinars. February 9 Overview of ICAP Geographic Information System Resources - PowerPoint PPT Presentation
Citation preview
Matthew Lamb mrl2013@columbia.edu
ICAP-M&E NY
Using routinely-collected data to estimate patient retention in care and loss to follow-up
ICAP Methodology WebinarJanuary 19, 2012
Upcoming methodology webinars
February 9
Overview of ICAP Geographic Information System Resources
Charon Gwynn, Yingfeng Wu, and Mark Becker
Future methodology webinar ideas?
email the methodology webinar coordinator, Bill Reidy: wr2205@columbia.edu
Matthew Lamb mrl2013@columbia.edu
ICAP-M&E NY
Using routinely-collected data to estimate patient retention in care and loss to follow-up
ICAP Methodology WebinarJanuary 19, 2012
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
Enrollment into HIV care and treatment clinic
Already on ARTART ineligible at
enrollment (window)
Unknown eligibility at enrollment
(window)
ART eligible at enrollment
Initiates ART
Becomes ART eligible
Follow-up after ART initiationFollow-up after ART initiation
LTF
Death
LTFDeath
LTFDeath
LTFDeath
LTFDeath
Transfer
Transfer
Transfer
Transfer
Transfer
Transfer
Working definitions
• Retained• Known to be alive and engaged in care
• Retained on ART• Known to be alive, engaged in care, and on ART
• Retained in pre-ART care• Known to be alive and engaged in care, but not yet on ART
• Known dead• Death known to clinic and documented
• Transferred out• Patient transfer to another clinic known and documented
• Lost to follow-up• Patient not known to be dead or transferred, treatment
status and whereabouts unknown
Known dead
Lost to follow-up
+
Non-retained =
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
Non-retention combines “bad” outcomes of death and loss to follow-up
LTF Death
Treatment interruption/stoppageLack of monitoringLack of services
Death LTF
Unmeasured death Non-retained
Loss to follow-up results in underestimates of patient mortality
Measured death
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
LTF influences our measures of survival
(example)
• Suppose you live in a universe where HIV clinics have perfect documentation, and all patients who enroll into HIV care attend every one of their scheduled visits and take all of their medication.
Measuring risk of death in a cohort with no LTF
Time (months) sinceART initiation
Incidence proportion of death:
4/20(20%)
Retention proportion:16/20(80%)
1 2 3 4 5 6 7 8 9 10 11 12
Incidence proportion of death:
4/20(20%)
Retention proportion:16/20(80%)
Incidence proportion of death:
3/20(15%)
Retention proportion:12/20(60%)
1 2 3 4 5 6 7 8 9 10 11 12
Time (months) sinceART initiation
LTF underestimating the risk of death
Introducing incidence rates
• To account for unequal follow-up time in the presence of loss to follow-up, we measure incidence using incidence rates instead of incidence proportions
Incidence proportion Incidence rate
number of eventspopulation at risk
number of eventsperson−time at risk
Estimating mortality rates in the presence of LTF
Incidence proportion of death:
4/20(20%)
Retention proportion:16/20(80%)
Incidence proportion of death:
3/20(15%)
Retention proportion:12/20(60%)
1 2 3 4 5 6 7 8 9 10 11 12
125.51212
7.7512121212121212129
1212129
1212
223.25 person-months
Incidence rateof death
4/223.25 pm21.5 per 100 py
Non-retention rate is the same here
122.251212
7.7512121212129
1249
128
1296
12
199 person-months
Incidence rateof death
3/199 pm18.1 per 100 py
Non-retention rate
8/199 pm48.2 per 100 py
Time (months) sinceART initiation
Months of observation for each patient
Total person-time
Incidence rates give us closer estimates to “the truth” in populations with loss to follow-up
No LTF LTF % Difference
Incidence Proportion
20% 15% 20%
Incidence Rate
21.5 per 100 py
18.1 per 100 py
16%
Using incidence proportions when follow-up time is unequal biases our interpretation of outcome occurrence
3m 6m 9m 12mIncidence proportion: 4/(4+10) = 4/ 14 = 29%Incidence rate:4/((4*1m) + (10*12m)) =
4/124 person-months = 39 per 100 person-years
3m 6m 9m 12mIncidence proportion: 4/(4+10) = 4/ 14 = 29%Incidence rate:4/((4*11m) + (10*12m)) =
4/164 person-months= 29 per 100 person-years
Risk Ratio = 29%/29% = 1
Rate Ratio = 39/29 = 1.3
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
Comparing retention between clinics allows us to assess factors that may improve retention
Age group Proportion retained 1 year after ART initiation
5-10y 12%
11-14y 15%
15-24y 30%
25-54y 19%
55 and over 17%
*Among patients initiating ART April 2008-March 2010. Loss to follow-up is defined as patients not known to have died or transferred without a visit in the last 6 months of data collection (ART). Patients LTF are censored 15 days after their last visit (ART).
Comparing retention between clinics allows us to assess factors that may improve retention
Source: Lambdin et al. JAIDS. Volume 57(3), 1 July 2011, pp e33-e39
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
Transforming patient-level data into follow-up cohorts
Working example: Non-retention one year after ART initiation• Select study population and time period for enrollment
and follow-up of patients• Define loss to follow-up
• No visit in the last 6 months of data collection• Therefore need to extend follow-up time for 6
months after enrollment period• Define “zero time”• Calculate person-time for each patient• Calculate the incidence rate for the outcome of interest
Q1 Q2 Q3 Q4 FU1 FU2
13 Non-retained within 1 year
153.5 person-months
Non-retention rate = 13/153.5 person-monthsNon-retention rate = 102 per 100 person-years
Calculating retention from patient-level data
• Study population: patients initiating ART between Q1 and Q4• Exclude patients initiating prior to Q1
• Outcome = non-retention (LTF or death)• Make sure all patients have sufficient opportunity to meet definition of LTF
• Extend follow-up period to 6 months after Q4 • Start following patients from their ART start date until they become non-retained or the study period ends• Calculate person-time• Calculate non-retention rates
7.54
4.57.5127.512127.5125
7.57.57.57.52.597
6.57
Limit follow-up to1 year
Slight diversion: what to do about transfers
Transfers prevent us from knowing the true retention status of the patient after they transfer.
We say that patients who transfer are “censored,” meaning that we do not have complete information on their retention status, had they remained at their initiating clinic
We allow transfers to contribute person-time to the denominator until their transfer date
Summary: calculating non-retention from patient-level data
• Clearly define• Study population• Study period of enrollment• Follow-up time• Loss to follow-up• Outcome of interest• Zero time
• Calculate each person’s follow-up time from zero time until reaching outcome of interest, censoring, or end of study
• Calculate incidence rate
Patient-level data is awesome, but…
• Not everyone has it• It requires some “work” to analyze
Thankfully, there is other information that we routinely collect
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
12 month aggregate ART cohort data (URS)
Quarter of ART initiation:May-July 2010
Number initiating ART inMay-July 2010:
269
Number on ART July-Sep 2011
214
Proportion retained = 214/269 = 80%
Q1 Q2 Q3 Q4 FU1 FU2
12 month aggregate ART cohort data
28
Count patients initiating within the same quarter
5
Count how many of theseare on ART 11-16 months later
1
Proportion retained = 1/520%
Weighted average retention measures from aggregate cohort data
ART initiation quarter
Number initiating ART during quarter
Number alive and on ART 12 months later
Retention proportion
Jan-Mar 5 1 20%
Apr-Jun 100 96 96%
Jul-Sep 150 146 97%
Oct-Dec 200 196 98%
Total 455 439 96%
Weighted-average 12-month retention incidence proportion
(5∗0.2 )+ (100∗0.96 )+(150∗0.97 )+(200∗0.98)5+100+150+200
¿96%
12 month aggregate ART cohort data: caveats
• Transfers should be excluded from numerator and denominator
• Does not separate non-retention into LTF, death• Can not directly calculate incidence rates• Only collected at 12 months• Can combine several cohort measures of
retention to obtain a clinic-specific average 12-month retention
Outline
• Defining retention• Why is retention a useful outcome, compared to
mortality?• How does loss to follow-up impact our measures of
patient survival?• How can we use retention to assess patient and
programmatic outcomes?• How can we measure retention using routinely-collected
data?• Patient-level• ART cohort• Aggregate
HIV care and treatment clinics routinely report the following
• Cumulative number of patients on ART at the end of the previous quarter• (which equals the cumulative number of patients on
ART at the beginning of the current quarter)
• Number newly initiating ART during the quarter• Cumulative number of Deaths, Transfer, LTF,
ART discontinuation through the end of the quarter
• From these, we can calculate overall retention estimates
Q1 Q2 Q3 Q4 FU1 FU2
Aggregate retention
2
5
7
5 5 5
9 13 14
3 1 4
Q1 Q2 Q3 Q4
Aggregate retention
2
5
7
5 5 5
9 13 14
3 1 4
Assume 3 months follow-up timefor each individual on ART at beginning of the quarter
Assume 1.5 monthsfollow-up time for each individual initiating during the quarter
Subtract 1.5 months follow-up time for each individual exiting during the quarter
2*3 + 5*1.5
=13.5 pm
7*3 + 5*1.5 –3*1.5
=24 pm
9*3 + 5*1.5 –1*1.5
=33 pm
13*3 + 5*1.5 –4*1.5
=40.5 pm
Q1 Q2 Q3 Q4
Aggregate retention
2
5
7
5 5 5
9 13 14
3 1 4
Overall one-year non-retention rate:
7/(13.5+24+33+40.5)
7 per 111 person-months
76 per 100 person-years
2*3 + 5*1.5
=13.5 pm
7*3 + 5*1.5 –3*1.5
=24 pm
9*3 + 5*1.5 –1*1.5
=33 pm
13*3 + 5*1.5 –4*1.5
=40.5 pm
Aggregate retention: important caveats
• Aggregate retention does not estimate a patient’s risk of being non-retained in one year• It is a clinic-level estimate of the flow of
patients into and out of the clinic• We can use the same procedure to calculate
aggregate LTF and mortality rates• Comparing aggregate retention rates between
clinics is very useful
Summary
• Loss to follow-up results in underestimates of survival• In situations with high loss to follow-up, non-retention
provides us with a more conservative estimate of undesirable patient outcomes
• Incidence rates are the preferred measure of non-retention incidence, but are not always available
• Routinely-collected Patient-level, ART cohort, and cumulative aggregate data can be used to estimate non-retention incidence
• Comparing non-retention between populations and clinics can help us identify areas that affect patient outcomes…
Using non-retention: CROI 2012 Abstracts
• Lamb et al. Factors Associated With High Loss To Follow-Up Among sub-Saharan African Youth 15-24 Years of Age Enrolled in HIV Care
• Elul et al. Six- and 12-month Non-retention Over Time among 5,690 Cohorts with 316,762 Patients Initiating Antiretroviral Therapy (ART) in 9 Countries in Sub-Saharan Africa
• Mcnairy et al. Retention of HIV-infected Children on ART in ICAP-supported HIV Care and Treatment Programs
Recommended