How Can We How Can We Improve the Improve the Accuracy of Accuracy of

Routine Pain Routine Pain Screening?Screening?L.R. ShugarmanL.R. Shugarman

RAND CorporationRAND Corporation

VA HSR&D Center for the Study of VA HSR&D Center for the Study of Healthcare Provider Behavior, VA of Healthcare Provider Behavior, VA of

Greater Los AngelesGreater Los Angeles

ColleaguesColleagues

K.A. Lorenz, Study PIK.A. Lorenz, Study PI C.D. SherbourneC.D. Sherbourne L.V. RubensteinL.V. Rubenstein L. WenL. Wen A. CohenA. Cohen J. GoebelJ. Goebel A. LantoA. Lanto S. Asch, Study Co-PIS. Asch, Study Co-PI

BackgroundBackground

Despite available, effective Despite available, effective treatment, clinicians and patients treatment, clinicians and patients often have trouble achieving often have trouble achieving adequate pain controladequate pain control

Although there is widespread use of Although there is widespread use of the `5th vital sign´ in the VA, its the `5th vital sign´ in the VA, its accuracy is unclear. accuracy is unclear.

ObjectivesObjectives

Compare routine pain rating during Compare routine pain rating during vital sign intake to the same pain vital sign intake to the same pain scale applied under ideal research scale applied under ideal research conditions and to a gold standard conditions and to a gold standard measure, the Brief Pain Inventory measure, the Brief Pain Inventory (BPI)(BPI)

Distinguish variation associated with Distinguish variation associated with instrumentation from that associated instrumentation from that associated with routine measurement processeswith routine measurement processes

MethodsMethods

Randomly sampled (March 2006-April 2007) Randomly sampled (March 2006-April 2007) veterans in primary care, urgent care, women’s veterans in primary care, urgent care, women’s health, oncology and cardiology clinicshealth, oncology and cardiology clinics

19 clinics from 2 hospitals, 6 affiliated sites in 19 clinics from 2 hospitals, 6 affiliated sites in 3 large urban counties (Los Angeles, Ventura, 3 large urban counties (Los Angeles, Ventura, and Orange) and Orange)

Surveyed patients, their nurses, & treating Surveyed patients, their nurses, & treating providersproviders

Patients surveyed immediately following clinic Patients surveyed immediately following clinic visitvisit

Additional measures derived from chart reviewAdditional measures derived from chart review

MeasuresMeasures Numeric Rating Scale (NRS) – 0-10 pain ratingNumeric Rating Scale (NRS) – 0-10 pain rating

Most commonly used method for assessing painMost commonly used method for assessing pain NRS of 5+ = moderate/severe painNRS of 5+ = moderate/severe pain

Brief Pain Inventory (BPI) – developed by the Brief Pain Inventory (BPI) – developed by the WHOWHO Measures pain intensity and pain interference with Measures pain intensity and pain interference with

various activitiesvarious activities Demonstrated reliability across patient populations Demonstrated reliability across patient populations

and settingsand settings Study measures:Study measures:

NRS gathered during vital sign intake (Nurse-NRS) – NRS gathered during vital sign intake (Nurse-NRS) – from chart reviewfrom chart review

NRS gathered under research conditions (Research-NRS gathered under research conditions (Research-NRS) – from patient surveyNRS) – from patient survey

BPI-24 hours, BPI-one week, BPI-interference – from BPI-24 hours, BPI-one week, BPI-interference – from patient surveypatient survey

AnalysisAnalysis Intraclass correlation used to assess Intraclass correlation used to assess

agreement between Research-NRS and agreement between Research-NRS and Nurse-NRS and Nurse-NRS and BPINurse-NRS and Nurse-NRS and BPI

Determined sensitivity/specificity of Determined sensitivity/specificity of cutpoints on the Nurse-NRS to BPI (gold cutpoints on the Nurse-NRS to BPI (gold standard) by fitting ROC curves and standard) by fitting ROC curves and calculated the area under the curve (AUC)calculated the area under the curve (AUC)

Variation in agreement further evaluated:Variation in agreement further evaluated: Did RN ask patient to rate pain on 0-10 scale?Did RN ask patient to rate pain on 0-10 scale? Has patient’s pain changed since arrival at Has patient’s pain changed since arrival at

clinic?clinic?

Results – Sample Results – Sample Characteristics (N=627)Characteristics (N=627)

Mean Age (SD)Mean Age (SD) 62.5 62.5 (12.9)(12.9)

% Male% Male 94.794.7

% White% White 49.149.1

Mean SRHS (1=Poor, 5=Excellent) Mean SRHS (1=Poor, 5=Excellent) (SD)(SD)

3.4 (1.1)3.4 (1.1)

% Mental Health Problem% Mental Health Problem 4444

% Cancer% Cancer 1313

% Cardiovascular Disease% Cardiovascular Disease 3838

% Musculoskeletal Condition% Musculoskeletal Condition 4545

Mean Research-NRS (SD)Mean Research-NRS (SD) 3.1 (3.2)3.1 (3.2)

Mean Nurse-NRS (SD)Mean Nurse-NRS (SD) 2.1 (3.2)2.1 (3.2)

Direction of Difference in Ratings Direction of Difference in Ratings Between Nurse-NRS and Between Nurse-NRS and Research-NRS RatingsResearch-NRS Ratings

Difference in RatingsDifference in Ratings N(%)N(%)

Nurse > Research 3+ pointsNurse > Research 3+ points 24 (3.8)24 (3.8)

Nurse > Research 2 pointsNurse > Research 2 points 19 (3.0)19 (3.0)

Nurse > Research 1 pointNurse > Research 1 point 24 (3.8)24 (3.8)

Nurse = ResearchNurse = Research 347 (55.3)347 (55.3)

Nurse < Research 1 pointNurse < Research 1 point 51 (8.1)51 (8.1)

Nurse < Research 2 pointsNurse < Research 2 points 39 (6.2)39 (6.2)

Nurse < Research 3+ pointsNurse < Research 3+ points 123 (19.6)123 (19.6)

Intraclass Correlations Intraclass Correlations Among Pain Rating Among Pain Rating

MeasuresMeasuresIntraclass Intraclass CorrelationsCorrelations

Nurse-Nurse-NRSNRS

Research-Research-NRSNRS

BPI-BPI-Sev/24 Sev/24 hrhr

BPI-BPI-Sev/last Sev/last weekweek

BPI-IntBPI-Int

Nurse-NRSNurse-NRS --

Research-Research-NRSNRS

.627.627 --

BPI-Sev/24 hrBPI-Sev/24 hr .613.613 .866.866 --

BPI-Sev/last BPI-Sev/last weekweek

.588.588 .845.845 .947.947 --

BPI-IntBPI-Int .409.409 .597.597 .693.693 .753.753 --

ResultsResults

AUC for Nurse-NRS compared to AUC for Nurse-NRS compared to Research-NRS was 0.78 for a cutoff of Research-NRS was 0.78 for a cutoff of 5 and 0.77 for a cutoff of 75 and 0.77 for a cutoff of 7

Similar AUC results found for Nurse-Similar AUC results found for Nurse-NRS compared to BPI-24 hour and NRS compared to BPI-24 hour and BPI-last weekBPI-last week

Nurses more likely to underestimate Nurses more likely to underestimate pain if they did not use 0-10 scalepain if they did not use 0-10 scale

Agreement attenuated when patient Agreement attenuated when patient pain changedpain changed

LimitationsLimitations

VA system institutionalized routine VA system institutionalized routine pain screening in the last decade; as pain screening in the last decade; as such, findings may conservatively such, findings may conservatively estimate the challenges of consistent estimate the challenges of consistent implementation in more diverse, implementation in more diverse, non-VA settingsnon-VA settings

Study limited to outpatient Study limited to outpatient evaluation of the 5th vital signevaluation of the 5th vital sign

ConclusionsConclusions

Accuracy of the 5th vital sign is Accuracy of the 5th vital sign is moderatemoderate

Nurses may not always use the 0-10 Nurses may not always use the 0-10 NRS to properly quantify pain levelsNRS to properly quantify pain levels

Nursing staff training in pain Nursing staff training in pain measurement may be warrantedmeasurement may be warranted