Maximizing Retention in an Urban Prospective Cohort Study

Elaina Murray, BS1; Kate Beatty, MPH1; Louise F. Flick, DrPH1; Michael Elliott, PhD1; Lisa V. John, PhD2; Vetta Sanders Thompson, PhD3; Allison King, MD, MPH3; Laura Bernaix, PhD, RN4; Candi LeDuc, RN4; Elizabeth Lacy, RN4; Kristi Helmkamp, RN2; Amanda S. Harrod, MPH1; Nikki Weinstein, MSW2

1Saint Louis University, 2Battelle Memorial Institute, 3Washington University in Saint Louis, and 4Southern Illinois University-Edwardsville

Presenter Disclosures

The following personal financial relationships with commercial interests relevant to this presentation

existed during the past 12 months:

Elaina Murray

• No relationships to disclose

National Children’s Study (NCS)St. Louis City LocationBackground

• NCS will examine the effects of the environment on the growth, development, and health of children across the US, following them from before birth until age 21

• St. Louis City: Recruiting door-to-door in randomly selected segments (neighborhoods)• Segments, groups of 8 to 25 city blocks distributed around city• Eligible women were pregnant or trying to conceive

• Retaining participants in multi-year prospective cohort studies presents challenges, especially in urban settings• Early identification of participants at risk for attrition may enhance

retention

3

Research Questions

4

• Can we predict missed appointments from SES characteristics?

• Is the use of a subjective risk ranking better at predicting risk than SES characteristics alone?

Methods

5

Subjective Risk Ranking of participants• Data collectors assigned participants to

either low, medium, or high risk of loss to follow-up

• High risk characteristics: •Chaotic family life•Very busy•Disengaged from the Study•Financial or housing crisis

Methods

Data Set• SES variables pulled from larger NCS database (N =97)

• Race- White- Black

• Age- 18-24- 25-34- 35-44

• Education- Less than HS or HS grad- Some college or more

• Risk Ranking• Low/Medium• High

• # missed appointments• None• One or more

6

Statistical Analysis

7

• Used IBM SPSS Statistics 20• Preliminarily used Chi-square analysis to look for

associations• Binary logistic regression to develop propensity scores

to summarize SES variables (DV=risk ranking, IV = race, age, & education)

• Binary logistic regression to develop 4 models (DV = missed appointments):• Model 1: IV = Race, age, and education• Model 2: IV = Risk only• Model 3: IV = Race, age, education, and risk• Model 4: IV = Propensity score and risk

Statistical Analysis

• Propensity score•Typically used to match cases/controls•Also used to control for demographic variables by creating one score to account for all the SES variables-Used when not interested in the effects of each SES variable alone-Good for small sample sizes, increases power

8

Statistical Analysis

9

• Propensity score (cont.)•Ran logistic regression using risk ranking as outcome-Included all SES/demographic variables as predictors of risk ranking

•Saved predicted probabilities to use as propensity score for each participant

Results

Total (%) Missed Appointments χ2

None (%) One or more (%)

Race, N=67

White 50.7 40.8 33.3 χ2=4.307df=1*

Black 49.3 59.2 66.7

Age, N=67 χ2=.572df=2

18-24 34.5 32.2 40.0

25-34 50.0 52.5 44.0

35-44 15.5 15.3 16.0

Education, N=67 χ2 =.116df=1

<HS or HS Graduate 44.6 45.8 41.7

Some college or more

55.4 54.2 58.3

Risk Ranking, N=72 χ2=8.698df=1**

Low/Medium 75.9 84.5 52.4

High 24.1 15.5 47.6

Table 1. χ2 Analysis of Demographics & Risk with Number of Missed Appointments

*p<.05; **p<.01; ***p<.001

Table 2. Logistic regression: Predicting risk ranking to develop propensity score (N = 67)

*p<.05; **p<.01; ***p<.001

Results

Covariates Model Propensity Score

aO.R. (95% CI)

N 67

Intercept .354

Race

White Reference

Black 4.738 (.406 – 5.221)

Age

18-24 Reference

25-34 .393 (.96 – 1.611)

35-44 1.915 (.265 – 13.856)

Education

<HS or HS Graduate Reference

Some college or more .282 (.070 – 1.142)

-2 Log likelihood 61.163

Model Χ2 Χ2=15.885df=4**

LR Stat

Nagelkerke R2 .306

Results

Covariates Model 1 Demographics Only

aO.R. (95% CI)

Model 2 Risk only

cO.R. (95% CI)

Model 3 Demographics and Risk

aO.R. (95% CI)

Model 4Risk and propensity

score aO.R. (95% CI)

N 67 72 67 67

Intercept .295 .224 .121 .224

Race

White Reference --- Reference ---

Black 2.787 (.913 - 8.508) --- 1.456 (.406 – 5.221) ---

Age

18-24 Reference --- Reference ---

25-34 .796 (.231 – 2.739) --- 1.076 (.257 – 4.507) ---

35-44 1.309 (.242 – 7.087) --- 1.105 (.144 – 8.483) ---

Education

<HS or HS Graduate Reference --- Reference ---

Some college or more .973 (.289 – 3.276) --- 1.912 (.426 – 8.574) ---

Propensity Score --- --- --- 1.566 (.066 – 37.334)

Risk Ranking

Low/Medium --- Reference Reference Reference

High --- 4.949 (1.626 – 15.062)** 6.109 (1.495, 24.955)* 4.782 (1.2 – 19.051)*

-2 Log likelihood 85.955 83.456 71.302 72.336

Model Χ2 Χ2=4.254df=4 Χ2=8.035df=1** Χ2=8.603df=5 Χ2=7.569df=2*

Nagelkerke R2 .080 .141 .173 .153

*p<.05; **p<.01; ***p<.001

Table 3. Hierarchical logistic regression of DV of missed appointments

Limitations

• Rating of risk done at one point in time, based on all experiences

• Length of time considered varied

• Based on a relatively low total number of visits (1 to 7)

• Ratings done by 3 nurses, do not have measure of inter-rater reliability

13

Conclusions

• Using the propensity score to control for demographics worked

• Using the subjective risk ranking is a better predictor of missed appointments than using demographics alone

14