Patient-oriented Outcome Measures

ACSM 2014 Lecture

The “DASH” to theperfect clinical tool.

James MacDonald, M.D., M.P.H., FACSM

1

2

What We’ll Talk About

• Background.• History and Context.• What’s the Purpose?• Language Matters.• Specific Examples of Measures.• Selected Survey of Studies Using

Measures.

3

“Doing It” matters

• We “do it”• Researchers “do it”• Payors “do it”• Patients want to “do it” too

In modern medicine, it’s how you prove your patients are getting better.

4

In medicalese……

• “Outcomes management is a technology of patient experience designed to help patients, payers, and providers make rational medical care-related choices based on better insight into the effect of these choices on the patient’s life.”

Paul Ellwood, 1988 Shattuck Lecture

5

It’s Really a Part of EBM

• Integration of 3 elements: – High quality clinical research

• Pooled data, readily accesisble • Greater reliance on standards/guidelines

– Clinician’s own expertise • Ability to interpret when to use above

– Patient input and values• Measure and incorporate

6

A bit of history• Epidemiologic shift: 20th

century• 1st valid/reproducible

questionnaires: 1949• EBM (Prof. Cochrane): 1972• RAND SF-36: 1988• “A Technology of Patient

Experience”: 1988– Shattuck Lecture, Paul Ellwood

• PCORI established: 2010

7

Patient-Centered Outcomes Research

Institute

8

Important Functions of Scales

• Comparison of functional status– Of one patient over time– Inter-patient comparison

• Allowing patients to self-evaluate• Evaluating effectiveness of various treatment

modalities

9

“Simple Stuff”

10

Graded Symptom Scale in Concussions

J Sport Rehabil. 2008 Feb;17(1):11-20.

Healthy youth are reliable in reporting symptoms on a graded symptom scale.• Found excellent reliability in middle

school athletes on the PCSS • However, difference in patient/parent

Steiner and Micheli Bracing in Spondylolysis 1985

11

“Old School”

“New School”

0 - 10

Micheli Functional Scale

0 - 5

0 - 4

0 - 3

0 - 3

13

A ‘hierarchy’ of numbers• Scales can be

–Nominal• e.g. 1 = male, 2 = female

–Ordinal• e.g. 1 = mild pain, 2 = moderate pain, 3 =

severe pain

– Interval*• e.g. temperature (10°C, 20°C, etc.)

–Ratio*• e.g. weights

*only scales that can be summated

14

Words Matter• RELIABILITY• VALIDITY• RESPONSIVENESS• GENERALIZABILITY

“An Ideal Rating System has proven reliability, validity, and responsiveness and can be applied with the knowledge that the results will be precise, accurate, and detect meaningful change.” It should also be generalizable.

15

Reliability

• What is it?• Intra-observer Reliability• Inter-observer Reliability• How can it be measured

– Correlation coefficients• Pearson’s• Spearman’s• ICC

– Cronbach’s α

16

Validity

• What is it?• ‘Face’ Validity• Criterion Validity• Construct Validity• How can it be measured?

– Correlation coefficients

17

Responsiveness

• Also known as “sensitivity to change”• The ability of a rating scale to detect

clinical change.

18

Minimal Clinically Important Difference

(MCID)• a.k.a. “The minimal clinically significant

difference”• Smallest difference in score in

domain of interest which patients perceive as beneficial

• Anterior Knee Pain Scale (AKPS) or “Kujala Scale” 3 studies variously found 7 – 14 is MCID (scale 0 to 100)

• VAS (0 to 100mm) is 18mm

Micheli Functional Scale 19

“Ceiling” and “Floor” effects

20

Generalizability

• One cannot assume that instruments can be used in every population

• or……“Moderate” pain may be different in Sweden than in Ethiopia

• Need to do testing in different populations– Cross-cultural differences– Gender differences– Age differences (think PEDI SCAT3)

21

Button and Pinney 2004

22

Which one to choose?

23

or……….What is the ideal test?

• Reliable, valid, responsive• Pertinent to and tested on your

population • Culturally and linguistically

appropriate• What are you looking at?

– ‘Health Related Quality of Life’ (HRQOL)– Functional status

• Disease state (e.g. osteoarthritis)• Body part (e.g. hip)

• Who fills it out?• and…….

24

Public vs. Private ($)

Public• SPADI• AIMS2• Oswestry Disability

Index• DASH

Private• SF-36• POSNA Pediatric MSK

Functional Health Questionnaire

• Pediatric Quality of Life Inventory*

25

1. USER AGREEMENT

User is required to complete and sign a User Agreement in which specific conditions requested by the Authors are detailed.

2. ACCESS FEES

AUTHORS’ ROYALTY FEES:

Authors' royalty fees are requested according to the study design and context of use of the questionnaire:- In academic research and individual clinical practice: free of charge

- In commercial studies (1) involving "for-profit" organizations and academic studies funded by commercial companies: 500 €* per protocol/application plus an additional

- l 500 €* per available language version (original version and translated versions)

Other specific conditions requested by the Authors are detailed in the User Agreement.

Mapi Research Institute Lyon, France

26

From the American College of Rheumatology• Measures should

– Be relevant to and/or used in rheumatology research and/or clinical practice.

– Be in the public domain (do not require purchase)

– Require very little or easily attainable equipment to administer.

– Not require special training or certification to administer.

– Not be not biologically based (e.g., radiographic grading systems).

27

General Types of Scales

• Quality of Life– e.g. SF 36

• Region Specific– e.g. DASH

• Disease Specific– e.g. AIMS2, WOMAC

• Mixed

28

“Mixed” Instruments

• Harris Hip Score used in Hip OA, outcomes of total hip arthroplasty, etc.

• Mix of subjective and objective criteria

• http://www.orthopaedicscore.com/scorepages/harris_hip_score.html

29

Journal of Sport Rehabilitation 2011 20 46-60

30

Clinical Outcomes In Sports Related

Concussions

31

Specific Scales

HRQOL• Pediatric Quality of

Life Inventory• SF-36

FHS• HIT-6 (Headache

Impact Test)• POSNA Pediatric MSK

Functional Health• WOMAC• Oswestry Low Back

Pain Scale• DASH• Micheli Functional

Scale

32

Health Related Qualify of Life

• HRQoL: what is it?• Who does it?

– http://www.healthypeople.gov/2020/about/QoLWBabout.aspx (CDC)

– http://www.nihpromis.org/ (NIH)

33

HRQL: defining it

• “Health-related quality of life is the value assigned to duration of life as modified by the impairments, functional states, perceptions, and social opportunities that are influenced by disease, injury, treatment or policy.”

• “…..the ultimate goal of all health care is to improve, restore, and preserve HRQL”

34

SF 36

35

Pediatric Quality of Life Inventory

• Used increasingly in HRQOL assessments in a variety of pedi studies

• “Parent reports cannot be substituted for child self-reports”

• 65 translations• Different modules• Validated on children and

adolescents: Cronbach’s alpha 0.83

36

DASH (Disability Arm, Shoulder, Hand )

• DASH has 30 questions• QuickDASH has 11 questions • Both offer a “Sports/Performing Arts

Module” which is 4 questions• Free! Find it at

http://www.dash.iwh.on.ca/index.htm

• Evaluates symptoms and patient’s ability to perform ADLs/Sport-specific activity

37

ODI

38

ODI: other versions

39

Oswestry Disability Index

JAMA 2006 40

Spine Patient Outcomes Research Trial (SPORT)

• To assess efficacy of surgery for lumbar intervertebral disk herniation

41

HRQOL in collegiate athletes

with history of concussion

42

SF-36

• Two indices used– SF-36 and HIT-6

Kuehl et al. CJSM 2010

43

SF-36

44

HIT-6

45

Quality of Life in Adolescents with Mild

Asthma• 13-18 year olds• “HRQoL lower in adolescent athletes

with EIB in comparison with peers without this disorder”

• Previous studies have demonstrated treatment of moderate to severe asthma improves HRQoL

• Argues for doing the same in EIB

46

47

JAMA 5/21/14 • 102 community volunteers, age > 50

with radiographic evidence of hip OA and VAS pain > 4cm at baseline

• 12 weeks of PT vs. sham• Outcome measures

– VAS pain– WOMAC OA Index (physical function, hip

specific)– Hip OA Outsome scale

• No difference!!!

d'Hemecourt et al. CJSM 2012 48

Micheli Functional Scale

49

Validity testing of MFS

Spearman’s correlation Coefficient showing high correlation between MFS and ODI

(rs = 0.821, p < 0.0001)

50

Summary• History and Current Use• Why it matters• Psychometric

Terminology• Reliability, validity,

responsiveness, generalizability, floor/ceiling effects

• Survey of Instruments • Examples in MSK/

Sports Med Literature

• Surv

51

References• “A Meta-Analysis of Outcome Rating Scales in Foot and Ankle

Surgery: Is there a Valid, Reliable, and Responsive System?” Button and Pinney, 2004 Foot and Ankle International 25 (8): 521 – 525.

• “Measuring Clinically Important Changes With Patient-Oriented Questionnaires,” Liang et al. 2002 Med Care 40 (suppl): II-45-II-51

• “A Systematic Review of Faces Scales for the Self-report of Pain Intensity in Children” Tomlinson et al., 2010 Pediatrics 126: e1168-e1198

• “Clinical Outcomes Assessment for the Management of Sport-Related Concussion,” Mcleod and Register-Mihalik 2011 Journal of Sports Rehabilitation 2011 20 46 – 20.

• “The New Subjective Medicine: Taking the Patient’s Point of View on Health Care and Health,” Sullivan 2003 Social Science and Medicine 1595 – 1604

• http://moon.ouhsc.edu University of Oklahoma • “Effect of Physical Therapy on Pain and Function in Patients with

Hip Osteoarthritis: A Randomized Clinical Trial,” Bennell et al. 2014. JAMA 311:

This talk will be available at cjsmblog.com

52

Thanks!