Volker Tschuschke, Aureliano Crameri, Margit Koemeda, Peter Schulthess, Agnes von Wyl, Rainer Weber, Miriam Köhler, Katharina Muth, Jessica

Mallmann, Pia Pulte, Antonia Roth-Ehrang

Treatment Adherence andTreatment Adherence andOutcome in Psychotherapy -Outcome in Psychotherapy -

First Results From the PAP-S-StudyFirst Results From the PAP-S-Study

FMPP-IFP Congress Luzern 2010

Treatment integrity in psychotherapyTreatment integrity in psychotherapy• ... is essential for psychotherapy research

• ... it is obvious that no control of treatment delivery by therapists in research studies leads to questionnable results

• ... therefore process research is indispensable

• ... there is a need to examine and determine that treatments said to have been provided were indeed the treatments provided (Kendall et al., 2004)

• ... and its implications for treatment outcomes is still unknown. A review of relevant psychological journals revealed that systematically addressed treatment integrity procedures are virtually absent in the literature (Perepletchikova et al., 2007), only 3.5% of the studies or less than 6% (Waltz et al., 1993)

FMPP-IFP Congress Luzern 2010

Process Study of Outpatient Psychotherapy Process Study of Outpatient Psychotherapy Treatment Switzerland (PAP-S)Treatment Switzerland (PAP-S)

• process-outcome study (start in 2006, end in 2012)• large outcome battery (pre, post, follow-up)• diagnostic assessments by trained raters (ICD, OPD, SKID-II) in major Swiss cities (Basel,

Berne, Lausanne, Luzern, St. Gallen, Zurich)• several process measures administered each 5th session

• 11 different therapy schools• objective ratings of treatment adherence

• each session being audiotaped• three sessions (beginning, middle, and end of treatment) will be depicted for integrity ratings• raters are blind toward school affiliation of therapists and patient data• 4 out of 5 raters are trained psychotherapists• so far the average interrater reliability (Kappa) lies between .50 and .60 for three raters

(codings of event-by-event, each therapist intervention, approx. 30 - 50 ratings per session)• a detailed rating manual including 101 intervention categories was being developed• rating manual‘s categories comprise 7 - 13 intervention categories from each school including

22 nonspecific intervention categories• each therapist intervention is being rated taking into account context (contents of patient‘s

utterances)

• therapists code their school specific behavior after each session using a check-list of their used particular school categories including nonspecific interventions (subjective ratings)

FMPP-IFP Congress Luzern 2010

RatingsRatings

• concept true intervention ratings in relationship to nonspecific intervention ratings

• example Gestalt Therapy session:

• raw score interventions true to Gestalt therapy = 35• 13 intervention categories• raw score nonspecific interventions = 28• 22 intervention categories

• scores > 1 mean more specific codings compared to nonspecific, and scores < 1 mean less specific codings compared to nonspecific ones

35 / 13

28 / 22= 2.12

FMPP-IFP Congress Luzern 2010

Treatment Adherence - Self- vs. ObjectiveRatings (Gestalt)

0

0,2

0,4

0,6

0,8

1

1,2

Session 101 Session 103

Self-RatingObjective Rating

Score

FMPP-IFP Congress Luzern 2010

OQ-Total - Therapy 101 (Gestalt)

0,00

20,00

40,00

60,00

80,00

100,00

120,00

1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 137 145 153

oqTOTAL6 Per. Gleitender Durchschnitt (oqTOTAL)Polynomisch (oqTOTAL)

Score

Sessions

FMPP-IFP Congress Luzern 2010

Treatment Adherence - Subjective vs. Objective Ratings - Integrative Body Psychotherapy

0

0,2

0,4

0,6

0,8

1

1,2

Session 42 Session 43 Session 44 Session 46

Subjective RatingsObjective Ratings

Score

FMPP-IFP Congress Luzern 2010

OQ-Total Therapy 2A-8 (IBP)

0,00

20,00

40,00

60,00

80,00

100,00

120,00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Reihe16 Per. Gleitender Durchschnitt (Reihe1)Polynomisch (Reihe1)

Score

Sessions

FMPP-IFP Congress Luzern 2010

• subjective ratings by therapists tend to estimate treatment integrity significantly higher compared to observer ratings as is found in other studies (Martino et al., 2009)

• even therapists consider a fairly high amount of nonspecific ingedients in their intervention style

• observers‘ ratings suggest that most of the intervention activity of psychotherapists is nonspecific, thus leaving therapists‘ ratings as probably not valid (Waltz et al., 1993)

• although both therapists seem to have little treatment integrity, their treatments seam to be efficient (at least the given two cases)

• both therapists used mainly unique and essential, essential but not unique and acceptable but not necessary interventions and very little proscribed interventions

• so far the question is unanswered whether higher treatment integrities are correlated with better outcomes (the PAP-S Study will address this question)

• it may also be the case that more severely disturbed patients - complex interventions, demanding, tedious, extended in time etc - generate serious problems in maintaining treatment integrity (Yeaton and Sechrest, 1992)

• the PAP-S-study will be able to look at different disturbance levels and therapists‘ techniques in these treatments

ConclusionsConclusions

FMPP-IFP Congress Luzern 2010

References

• Kendall et al. (2004): Methodology, design, and evaluation in psychotherapy research. In Lambert (ed.): Handbook of Psychotherapy and Behavior Change. 5th ed., John Wiley & Sons, New York, pp 16-43

• Martino et al. (2009): Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers.Psychotherapy Research 19: 181-193

• Perepletchikova et al. (2009): Barriers to implementing treatment integrity procedures: survey of treatment outcome researchers.Journal of Consulting and Clinical Psychology 77: 212-218

• Waltz et al. (1993): Testing the integrity of a psychotherapy protocol: assessment of adherence abd competence. Journal of Consulting and Clinical Psychology 61: 620-630

• Yeaton / Sechrest (1992): Critical dimensions in the choice and maintainance of successful treatments: strength, integrity, and effectiveness. In Kazdin (ed.): Methodolgical Issues & Strategies in Clinical Research. American Psychological Association, Washington D.C., pp. 137-156