Estimating and Understanding Therapist Effects

Empirical Evidence

Bruce E. WampoldUniversity of Wisconsin--Madison

History of Omission Historically, provider effects

ignored Education Agriculture Medicine

Ignoring Therapists Therapists unimportant

Therapist effects not estimated Focus on treatment

Methodological issues Differences among treatments may be due to

therapists Increases Type I error rate and effect size for

Tx effects Confounds within and between group

relationships

Goals How important is the provider relative

to the treatment? Estimate the variability among

therapists (within treatments) Therapist variability v. treatment

variability Understand the characteristics and

actions of effective therapists Discriminate between patient and

therapist contributions to outcomes

Estimating Therapist Effects Sources of variability:

Treatment differences (fixed effect) αj

Therapist variability (random effect) σ2ther

Error or patient variability σ2error

Therapist Effects: intraclass correlation coefficient (Therapist variability) / (Total Variability) ρ = σ2

ther/(σ2ther + σ2

error)

Effects as Percentage of Variability of Termination Score

Pretest – 40% to 50% Tx v. No Treatment– about 13% Treatment A v. Treatment B– at

most 1% Alliance– 5% to 9% Therapist….

9% (Chrits-Christoph et al., 1991)

NIMH TDCRP reanalysis Nested Design (CBT and IPT) Well trained therapists, adherence monitored,

supervision Elkin:

The treatment conditions being compared in this study are, in actuality, “packages” of particular therapeutic approaches and the therapists who choose to and are chosen to administer them…. The central question… is whether the outcome findings for each of the treatments, and especially for differences between them, might be attributable to the particular therapists participating in the study.

Random Effects Modeling Therapists considered a random factor Therapists nested within treatments

(multilevel model) Final observations, controlling for pretest at

patient and therapist level Kim, Wampold, & Bolt, Psychotherapy Research, 2006

Random Effects Modeling Therapists considered a random factor Therapists nested within treatments

(multilevel model) Final observations, controlling for pretest at

patient and therapist level Therapist slope fixed and random

Kim, Wampold, & Bolt, Psychotherapy Research, 2006

Greater Severity

Greater Severity

Variance due to Tx and Therapists

Variable Treatment

Therapist

BDI 0%

HRSD 0%

HSCL-90 0%

GAS 0%

Variance due to Tx and Therapists

Variable Treatment

Therapist

BDI 0% 5% - 12%

HRSD 0% 7% - 12%

HSCL-90 0% 4% - 10%

GAS 0% 8% - 10%

Note: Elkin et al. (2006) found negligible therapist effects in the same data

Variance due to therapists in practice 581 Therapists, 6146 patients More heterogeneous patients Outcome Questionnaire 30 Diagnosis, degree, experience, 0

percent Medication, 1 percent (but dependent

on psychotherapist) 5 percent Wampold & Brown, JCCP, 2005

Cross-validation: year 1 to year 2

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

Re

sid

ua

lize

dch

an

ge

Pro

po

rtio

nre

liab

lych

an

ge

d

Effe

ct s

ize

Best Q1

Worst Q4

At least 9 cases in yr 1

73 Therapists

Therapist Effects in Psychopharmacology (NIMH)

Antidepressants: Imipramine v. Placebo 3% due to treatment 9% due to therapist Best therapists get better outcome with

placebo that worst therapists with imipramine

McKay, Imel & Wamold, 2006

Therapists– Psychopharm

BDI_MEAN_BEST_PROV

-8

-6

-4

-2

0

2

4

6

8

10

12

6 23 3 11 1 28 20 15 19

Provider

RE

S G

AIN

SC

OR

E

BDIPL

BDITX

Conclusions

Therapists make a difference Size of therapists effects at least

an order of magnitude greater than treatment effects

What are the characteristics or actions of effective therapists?

Characteristics and Actions of Effective Therapists

Consult Buetler (Handbook of Psychotherapy and Behavior Change)

We don’t know And we don’t care

Alliance? Alliance measured early in therapy related to

outcome Therapist contribution? Patient contribution? Interaction?

Alliance: Patient v. Therapist Contribution to Alliance Counseling center consortium data OQ pre and post, Alliance 4th session 188 patients, 22 therapists 5% of variance due to therapists What is correlation of alliance with

outcome Within therapists? Between therapists?

And the results….

Within or between?

Size of Effect

Conclusions Method is vital to proper conclusions Improper models affect results Therapist effects are sizable,

especially compared to treatment differences

Every process and outcome study MUST include therapists in the model

Multilevel modeling can answer the fundamental question