Assessing Change in Clinical Practice

Richard Bost, Ph.D., ABPPFrances Wen, Ph.D.Michael Basso, Ph.D

Reliable Change Index (RCI)Jacobsen & Truax (1991)

• Appropriate for measures not subject to practice effects

• Examples: measures of attitude, psychological disorders, life satisfaction, health habits, coping styles, marital adjustment

Reliable Change Index (RCI)Jacobsen & Truax (1991)

• Not appropriate for measures affected by practice effects

• Examples: measures of memory, intelligence

Reliable Change Index (RCI)Jacobsen & Truax (1991)

• Appropriate for measures with underlying normal distributions or frequency spreads

• Examples: Dyadic Adjustment Scale, Geriatric Depression Scale, SCL-90-R, BDI, SF-36, etc.

Reliable Change Index (RCI)Jacobsen & Truax (1991)

• Not appropriate for measures which do not have underlying normal distributions or frequency spreads

• Examples: the MCMI (I-III), and the MCMD (or MBHI)

Reliable Change Index (RCI)Jacobsen & Truax (1991)

• Does change exceed what would be expected based on measurement error alone?

• If “yes”, then change is considered to be “statistically reliable”

Reliable Change Index (RCI)5 Steps to Assessing Reliable Change

1. Administer appropriate measure at pre-test

2. Re-administer measure at a post-test period

3. Calculate change score

4. Calculate the RCI

5. Compare change score to the RCI

Reliable Change Index (RCI)

The Reliable Change Generator 2.0

©Grant J. Devilly 2004

Web: www.swin.edu.au/victims

Reliable Change Index (RCI) The Reliable Change Generator

Devilly, G.J. (2004). The Reliable Change Generator for Windows: Version 2.0 (computer programme). The Centre for Neuropsychology, Swinburne University, Australia.

Reliable Change Index (RCI) The Reliable Change Generator

Jacobson, N.S., and Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.

Reliable Change Index (RCI)Data for the Reliable Change Generator

• Test-retest reliability of the measure

• Sample• Time period

• Standard deviation for the sample (at Time 1) which was used to estimate the test-retest reliability

Reliable Change Index (RCI)Assessing Clinical Meaningfulness

• Assess after a change in scores is found to be reliable

• Consider various criteria:

• diagnostic• classification• degree of change

Reliable Change Index (RCI)Assessing Clinical Meaningfulness

Causation• Reliable change

implies lack of causation due to measurement error

• Variables unrelated to intervention may account for reliable change

Reliable Change Index (RCI)Assessing Clinical Meaningfulness

Direction of Change

• Improvement?

• Deterioration?

Reliable Change Index (RCI)A Case Example: Jane D.

• 19 y.o., single, unemployed

• Wilson’s disease

• Candidate for liver transplant

• Pre-transplant assessment: clinical interview, BDI-II, SF-36, STAI, SWLS, WAIS subtests

Reliable Change Index (RCI)A Case Example: Jane D.

• Post-transplant psychotherapy

• Post-transplant/Pre-therapy re-assessment:– clinical interview– BDI-II– SF-36– STAI– SWLS

Reliable Change Index (RCI)A Case Example: Jane D.

Pre-transplant Post-transplant/

Pre-therapy

95% RCI

BDI-II 28 23 5.78

SF-36 PCS 30 51 7.47

STAI-Trait 68

87

54 70

12.41

SWLS 11 15 6.99

Reliable Change Index (RCI)A Case Example: Jane D.

0

10

20

30

40

50

60

70

80

90

1st Testing 2nd Testing

BDI-II STAI Trait SWLS

Reliable Change Index (RCI)A Case Example: Jane D.

• Diagnosis:– Adjustment Disorder

with Mixed Anxiety and Depression

• Therapy Issues:– Physical, familial,

social, occupational

Reliable Change Index (RCI)A Case Example: Jane D.

Post-therapy • Changed occupational

and academic goals• Changed social

network• Changed residence

Reliable Change Index (RCI)A Case Example: Jane D.

Pre-therapy Post- therapy 95% RCI

BDI-II 23 13 5.78

SF-36 PCS 51 53 7.47

STAI-Trait 54

(70)

40

(54)

12.41

SWLS 15 26 6.99

Reliable Change Index (RCI)A Case Example: Jane D.

0

10

20

30

40

50

60

70

80

90

1st Testing 2nd Testing 3rd Testing

BDI-II STAI Trait SWLS

Additional References

• Ferguson RJ, Robinson AB, Splaine M. (2002). Use of the reliable change index to evaluate clinical significance in SF-36 outcomes. Quality of Life Research, 11, 509-516.

• Pavot W, Diener E. (1993). Review of the Satisfaction With Life Scale. Psychological Assessment, 5, 164-172.

Additional References

• Sprinkle SD, Lurie D, Insko SL, Atkinson G, Jones GL, Logan AR, Bissada NN. (2002). Criterion validity, severity cut scores, and test-retest reliability of the Beck Depression Inventory-II in a university counseling center sample. Journal of Counseling Psychology, 49, 381-385.

Additional References

• Spielberger CD, Gorsuch RL, Lushene RE (1970). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA; Consulting Psychologists Press.

• Ware JE, Kosinski M, Keller SK. SF-36® Physical and Mental Health Summary Scales: A User's Manual. Boston, MA: The Health Institute, 1994.