A Phase 4 replication of MET/CBT5 in 36 sites to examine how findings vary by site, client characteristics, and implementation

fidelity

Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Normal, IL

Richard D. Lennox, Ph.D.Psychometric Technologies, Hillsborough, NC

Randy Muck, M.Ed.Substance Abuse and Mental Health Services Administration (SAMHSA)

Center for Substance Abuse Treatment (CSAT), Rockville, MD

College of Problems on Drug Dependence (CPDD) & Society of Adolescent Substance Abuse Treatment Effectiveness (SASATE),

Reno, NV. June 23, 2009

Background

• In 1997 the third wave of cannabis use was the largest and youngest cohort to date, double the number of adolescents presenting to publicly funded treatment

• There were no publicly available manual guided evidenced based practices targeting this population

• The Cannabis Youth Treatment (CYT) experiments (n=600) were designed to manualize and field test five promising intervention for short term outpatient treatment of adolescent with cannabis (and other) substance use disorders

• Adapted from earlier studies with adult alcohol and cannabis users, Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) was the briefest, one of the least expensive, similar in clinical outcomes, and hence one of the most cost-effective approaches evaluated (Dennis et al 2004; French et al 2003).

Effective Adolescent Treatment (EAT)

• From 2003 to 2008 SAMHSA’s Center for Substance Abuse Treatment (CSAT) conducted a phase IV (i.e., post randomization) replication of MET/CBT5 in 36 sites.

• All sites received standardized training, quality assurance and monitoring on their implementation of MET/CBT5, as well as the collection of data with the Global Appraisal of Individual Needs (GAIN) to facilitate comparison with the original CYT study in terms of implementation and outcome.

• The objectives of this presentation are to : 1. Demonstrate that EAT used MET/CBT5 with a more

diverse population2. Replicate the implementation and outcomes of MET/CBT53. Identify participant characteristics moderators and

intervention mediators that are associated with outcomes

Sample Selection

• The Target Population Inclusion Criteria for including cases from the EAT data set were adolescents who: – Were assigned to MET/CBT in Outpatient and– Reported lifetime abuse or dependence symptoms and– Reported substance use in the last 90 days they were in the

community and– Who were due for 6 month follow-up

• Of 36 sites, 12 were dropped because they did not collect treatment received data at 3 months or because they less than 50% with BOTH 3 and 6 month interview

• Of the remaining 3556 clients from 24 sites, – 2540 (71%) have outcome data at 6 months– 2540 (86%) have outcome data at 3 or 6 months

• This group was compared using GAIN data to a cohort of 199 (98% of 202) adolescents from CYT that met the same criteria

EAT More Geographically Diverse

AK

AL

ARAZ

CA CO

CT

DC

DE

FL

GA

HI

IA

ID

IL IN

KS KY

LA

MA

MD

ME

MI

MN

MO

MS

MT

NC

ND

NE

NH

NJ

NM

NV

NY

OH

OK

OR

PARI

SC

SD

TN

TX

UTVA

VTWA

WI

WV

WY

CYT: 4 Sites

IncludedEAT: 24 Sites

Excluded EAT: 12 Sites

Demographics

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Male*

AA

White

Hispanic

Mixed /Other

11-14

15-17

18-22

CYT MET/CBT5(n=199)EAT MET/CBT5(n=2756)

Rac

e gr

oups

*A

ge g

roup

s*

*p<.01

EAT Clients were more likely to be female, non-white, and

have a wider age range

Clinical Characteristics

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Alcohol

Marijuana

Amphetamines

Cocaine, Opioids, Other

None

Internalizing Only

Both

Externalizing Only

None

Unofficial

Arrest/police contact

Court/Probation/Parole

Correctional Institution

CYT MET/CBT5(n=199)

EAT MET/CBT5(n=2756)

Pri

mar

y S

ubst

ance

Com

obid

ity

Del

inqu

ency

Lev

el*

*p<.01

EAT Clients less likely to have cannabis as primary substance, similar in their comorbidity, and to have

more justice system involvement.

EAT did as well or Better as CYT on Service Engagement

39%

63%

19%

90%

74%

40%

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Initiation within14 days

Engagement for42 days

Continuing Careafter 90 days

CYT MET/CBT5(n=201)EAT MET/CBT5(n=3355)

*p<.01

Days of Treatment in the First 3 Months

4.2

1.4

4.7

1.1

0 1 2 3 4 5

Outpatient

OtherTreatment

Days of Treatment

CYT MET/CBT5(n=201)EAT MET/CBT5(n=3355)

*p<.01

84%

94%

Content of CYT’s 5 Treatments Varied

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Direct ServicesReceived

Family ServicesReceived

ExternalServicesReceived

TreatmentReceived Scale

MET/CBT5

MET/CBT12

FSNM

ACRA

MDFT

*p<.01

Little related to family services

Low end for external or wrap around services

Similar on direct services

MET/CBT5 in CYT and EAT had a Similar Range of Content

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Direct ServicesReceived (0-8)

Family ServicesReceived (0-4)

ExternalServices

Received (0-8)

TreatmentReceived Scale

(0-20)

CYT MET/CBT5(n=201)EAT MET/CBT5(n=3355)

*p<.01

EAT received less external

services

Comparison of Outcomes

• Evaluate the Increase in the Days of Abstinence from Intake to Last Observation.

• For all time periods days abstinent are adjusted by subtracting any days in a controlled environment during the period (average is less than 5 days).

• Change scores are calculated as last observation minus intake.

• The large sample sizes involved make even trivial differences statistically significant. Thus this comparison focuses as much on clinical significance by using effect sizes.

Within and Between Group Effect Size Calculations

• Effect sizes for within group change reported as: Cohen’s d = (MLast -MIntake )/ Std Dev.Intake

where small=.2, moderate=.4 and large=.8

• Effect sizes for group (G) differences in change scores (Last-Intake) are reported relative to the grand mean (GM) as:

Cohen’s f = (ABS(G(change) –GM(change)))/ Std Dev.(GM Change)

where small=.1, moderate=.2 and large=.4

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

CYT (d=0.36)

EAT (d=0.38)

Change in Days Abstinent by Study (f=.02)

Slopes are NOT

significantly different

EAT more severe

Effect Size by Site

-0.2

0

0.2

0.4

0.6

0.8

140

0000

EA

T L

ansi

ng, M

I

7000

00 E

AT

2 R

ed B

ank,

NJ

3400

00 E

AT

Hou

ston

, TX

3300

00 E

AT

Tho

rnto

n, C

O

2040

0 C

YT

Phi

lade

lphi

a, P

A (

CH

OP

)

2800

00 E

AT

Den

ver,

CO

6300

00 E

AT

2 La

redo

, T

X

5900

00 E

AT

Dur

ham

, NC

6000

00 E

AT

Orla

ndo,

FL

3800

00 E

AT

Bur

lingt

on, V

T

3000

00 E

AT

St L

ouis

, MO

3700

00 E

AT

Tuc

son-

PIM

A, A

Z

8700

00 E

AT

2 A

von

Par

k, F

L

2030

0 C

YT

Mar

yvill

e, IL

(C

HS

)

8600

00 E

AT

2 D

aly

City

, CA

2010

0 C

YT

Har

tford

, CT

(U

CH

C)

EA

T

8200

00 E

AT

2 S

ilver

City

, NM

Tot

al

CY

T

6700

00 E

AT

2 F

itchb

urg,

MA

2900

00 E

AT

Por

tland

, OR

3900

00 E

AT

Pon

tiac,

MI

8500

00 E

AT

2 S

anta

Bar

bara

, CA

6600

00 E

AT

2 P

awtu

cket

, RI

3500

00 E

AT

Oly

mpi

a, W

A

6800

00 E

AT

2 R

ockv

ille,

MD

8400

00 E

AT

2 Lo

s A

ngel

es, C

A

2020

0 C

YT

St.

Pet

ersb

urg,

FL

(PA

R)

5800

00 E

AT

Pin

ella

s P

ark,

FL

8800

00 E

AT

2 A

kron

, OH

Co

hen

's E

ffec

t S

ize

d

Intake Predictors of Change in Days Abstinent Between Intake and Last Follow-up

Variable Cohen’s f β P

Days Abstinent 0.48 -0.63 0.0000

Controlled Environment 0.21 -0.09 0.0000

Count of Clinical Problems 0.32 -0.08 0.0000

Cocaine Problem Severity 0.27 -0.63 0.0000

Any Opioid Use 0.16 -0.04 0.0248

Other Drug Severity\a 0.10 -0.05 0.0016

Age 0.05 -0.05 0.0035

Gender 0.04 -0.06 0.0006

R2 .33 -- 0.0000\a Other than alcohol or marijuana

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

79-90 Days (d=-.27)70-78 Days (d=-.03)56-69 Days (d=.21)40-55 Days (d=.32)23-39 Days (d=.36)0-22 Days (d=.57)

Change in Days Abstinent by Days of Abstinence at Intake (f=.48)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

<13 Days(d=0.33)

13+ Days(d=1.02)

Change in Days Abstinent by Days in a Controlled Environment at Intake (f=. 21)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

0-1(d=0.29)

2-7(d=0.37)

8-12(d=0.65)

Change in Days Abstinent by Count of Major Problems at Intake (f=.32)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

No cocaine use(d=0.33)

Some cocaineuse/problems(d=0.58)

Weekly CocaineUse orDependence(d=1.1)

Change in Days Abstinent by Cocaine/Crack Problem Severity at Intake (f=.27)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

None (d=0.35)

Any (d=0.79)

Change in Days Abstinent by Any Opioid Use in community at Intake (f=.16)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

No other use(d=0.26)

Some otheruse/problems(d=0.45)

Weekly other useor dependence(d=0.49)

Change in Days Abstinent by Other Drug Problem Severity at Intake (excluding alcohol/marijuana) (f=.10)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

11-14 (d=0.29)

15-17 (d=0.41)

18-21 (d=0.33)

Change in Days Abstinent by Age group at intake (f=.05)

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

Female (d=0.44)

Male (d=0.35)

Change in Days Abstinent by Gender (f=.04)*

* P<.05

Other Client Characteristics that did NOT Predict the Amount of Change

• Race

• Single Parent

• Metropolitan size

• Primary drug

• Days of use or problem group for alcohol, cannabis, amphetamine

• Victimization

• Psychopathology

• Delinquency levels

Baseline + Mediators of Change in Days Abstinent Between Intake and Last Follow-up

Variable Cohen’s f β P

Baseline Risk\a -- .65 .0000

Days in a Controlled Environment Mons 1-3

0.22 -.17 .0000

High Treatment Cost 0.04 -.07 .0008

Any incarceration 0.17 -.06 .0075

Any other Treatment 0.04 .03 .0431

Site (6+Other) 0.14 -- .0000

R2 .42 -- .0000

\a Predicted from intake only

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

0-15 days(d=0.40)

16-90 days(d=0.22)

Change in Days Abstinent by Days in a Controlled Environment Mons 1-3 (f=.22)*

* P<.0001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

Below band(d=0.29)Within band(d=0.38)Above band(d=0.41)

Change in Days Abstinent by Costs within SAMHSA cost bands (f=.04)

* P<.001

0

10

20

30

40

50

60

70

80

90

Intake Last FU (3,6m)

No (d=0.40)

Yes (d=0.18)

Change in Days Abstinent by Any Incarceration in Mons 1-3 (f=.17)*

* P<.001

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

No (d=0.38)

Yes (d=0.40)

Change in Days Abstinent by Any Other SA Treatment (f=.04)*

* P<.05

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up

EAT-Houston (d=0.78)

EAT-Red Bank (d=0.63)

EAT-Lansing (d=0.59)

EAT-Burlington (d=0.52)

EAT-Orlando (d=0.40)

EAT-Tucson-PIMA (d=0.36)

Other 20 sites (d=0.34)

EAT-St Louis (d=0.31)

EAT-Pontiac (d=0.25)

Change in Days Abstinent by Sites (f=.14)*

* P<.0001

6 sites had more

change than average

2 sites had less change

than average(all low severity)

Other Moderators that did NOT Predict the Amount of Change

• Initiation & Engagement in but p>.05• Length of stay and continuing care• Treatment Received Scales (direct, family, wrap

around)• SA Days of residential, IOP, OP, Medication, ER

or Urine test/breathalyzer or summary index • MH Days of Inpatient, OP, Medication, ER or

summary index • PH Days of Inpatient, OP, Medication, ER or

summary index• Study or other sites• Months from intake to last follow-up

Limitations

• Primarily relied on adolescent self report (plus some records on implementation). It would have been useful to have collateral or urine test results.

• First cut only examined days of abstinence, it is likely that different variables impact other outcomes

• Could have used other ways of adjusting for time in a controlled environment

• All variation by site not explained yet.

• May need to look at environment and peer risk to explain differences

Conclusions

• EAT grantees were more diverse geographically, demographically and clinically

• EAT grantees implementation was better than CYT in terms of engagement, similar in dosage, and only slightly less in content

• Baseline severity was the primary factor explaining differences in the amount of change observed

• Engagement, dosage and content were not the major mediator of change – environmental variables were

• Further investigation is needed to understand why some sites did better than average even after controlling for above factors

This presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following CSAT grantees:

(CYT: TI-11320, TI-11317, TI-11321, TI-11323, TI-11324,

EAT: TI-15413, TI-15415, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15469, TI-15478, TI-15479, TI-15481, TI-15483, TI-15489, TI-15514, TI-15524, TI-15545, TI-15562, TI-15577, TI-15586, TI-15672, TI-15677, TI-15682, TI-15686). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Rod Funk, Mark Lipsey, Barth Riley, Michelle White and Ken Winters for their suggestions. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 448 Wylie Dr., Normal, IL 61761, mdennis@chestnut.org .

Acknowledgements