998 Journal of Pain and Symptom Management Vol. 26 No. 5 November 2003

Original Article

The Problem of Assessment Bias WhenMeasuring the Hospice Effect on NursingHome Residents’ PainNing Wu, MD, MS, Susan C. Miller, PhD, Kate Lapane, PhD,and Pedro Gozalo, PhDDepartment of Community Health (N.W., S.C.M., K.L., P.G.), and Center for Gerontologyand Health Care Research (S.C.M., K.L., P.G.), Brown University School of Medicine,Providence, Rhode Island, USA

AbstractThis study examined the observed differential documentation of pain on nursing home(NH) resident assessments (minimum data sets [MDS]) when dying residents were andwere not enrolled in hospice. We studied 9,613 NH residents who died in 6 states in 1999and 2000. Documented pain was compared among three groups of residents who werecategorized by their hospice exposure. At the time of their last MDS completion, residents inhospice were more likely to receive opioids for their moderate to severe pain than were non-hospice residents and residents enrolled in hospice after the last MDS assessments.However, hospice residents were twice as likely as non-hospice residents and 1.3 times aslikely as residents who eventually enrolled in hospice to have pain documented. Thesecounterintuitive findings suggest that there is differential documentation of pain on theMDS when hospice is involved in care, perhaps because of superior pain assessment byhospice. J Pain Symptom Manage 2003;26:998–1009. � 2003 U.S. Cancer PainRelief Committee. Published by Elsevier Inc. All rights reserved.

Key WordsHospice, MDS, ascertainment bias, pain, end-of-life care, nursing homes

IntroductionNursing homes (NHs) are an increasingly im-

portant site of death. Twenty-three percent ofthe 1.8 million persons who were 65 or olderand who died in 1999 died in NHs.1 Pain iscommon in dying NH residents.2–4 On average,

Address reprint requests to: Ning Wu, MD, MS, Depart-ment of Community Health, Brown UniversitySchool of Medicine, 169 Angell St., Providence, RI02912, USA.Accepted for publication: March 20, 2003.

� 2003 U.S. Cancer Pain Relief CommitteePublished by Elsevier Inc. All rights reserved.

13–35% of NH residents experience daily painin the months before death.5 Furthermore, asdeath approaches, pain intensity increases.6

Hospice aims to alleviate dying patients’ symp-toms and improve their quality of life at the endof life. One of the most important componentsof hospice care is symptom management, partic-ularly pain management.7,8 Hospice care hasbeen provided in NHs in United States since1985.9

Studies are needed to demonstrate thehospice effect on pain management relative totraditional care. To date, no study has demon-strated that NH residents receiving hospice ser-vices experienced less pain than those not

0885-3924/03/$–see front matterdoi:10.1016/S0885-3924(03)00328-2

Vol. 26 No. 5 November 2003 999The Problem of Assessment Bias

receiving hospice services. One study, using sec-ondary data, has shown that hospice is associ-ated with better analgesic use compared totraditional analgesic use in NHs.10 Due to thepotential for ascertainment bias, using second-ary data to determine pain status and its man-agement might complicate the comparativestudy of pain status and management for dyingresidents enrolled and not enrolled in hospice.Ascertainment bias is probable since knowl-edge of pain assessment is likely to vary betweenhospice and non-hospice staff.11 However, dueto ethical and methodological reasons,12,13 ob-servational studies using secondary data havebecome an increasingly important tool to testthe effect of hospice.

In NHs, a major data source for secondarydata analysis on pain prevalence and manage-ment is the mandated and computerized resi-dent assessment instrument, the minimum dataset (MDS). The MDS is collected by all Medi-care- or Medicaid-certified NHs to fulfill federalgovernment requirements. Minimum data setinformation is used by nursing homes to mon-itor residents’ conditions and quality of care,and by the government to assess NH perfor-mance and to determine payment for Medicareskilled nursing home care. Over 350 itemsare recorded in the MDS, including resi-dents’ physical and mental function, psycholog-ical well-being, clinical signs and symptoms,diagnoses, and medications.14 Each year, over10 million MDS assessment records are submit-ted to the Center for Medicare and MedicaidServices (CMS) and are made available forresearchers (when an approved data useagreement exists). The major concern regard-ing the use of MDS data is that the quality ofdata is not controlled through standardized re-search protocols. The accuracy of MDS data hasbeen questioned,15–17 but many other studieshave documented MDS data to be reliable andvalid.18–21 However, most of the validity and re-liability studies have used trained nurses as as-sessors, not NH staff or NH and hospice staff.

The aim of our study was to examine nursinghome pain assessment and management data,as obtained from MDS data, and to determinewhether comparisons of data support thenotion that there is differential assessment anddocumentation of pain when hospice is in-cluded in a resident’s care. Because hospice’sfocus is on symptom management, we reasoned

that hospice staff are more likely to perform painassessments, and are more likely to performhigh quality pain assessments. Therefore, wehypothesized that residents enrolled in hos-pice would be more likely to have pain docu-mented in their MDS compared to residentsnot enrolled in hospice.

MethodsStudy Population and Data Sources

We studied all the 13,534 residents who diedin nursing homes between January 1, 1999,and December 31, 2000, in six states (Illinois,Maine, Mississippi, New York, Ohio, and SouthDakota) and had valid drug information fromMDS. Our sample was derived from the Sys-tematic Assessment of Geriatric Drug Use viaEpidemiology (SAGE) database, which linksdata from nursing home resident assessments,CMS Medicare claims data, and organizationaldata on nursing home providers.3,22 A data useagreement for MDS and CMS claims data wasin place.

For each decedent, the last full MDS assess-ment before death was identified. Of the 13,534residents, 9,613 met the inclusion criteria andhad complete data. They composed the finalstudy sample (Figure 1). After an initial admis-sion time period (30 days), MDS assessments arescheduled to be conducted every 90 days.Therefore, we did not include residents withgreater than 180 days between their last MDSand their date of death as these residents eitherhad incomplete data, did not remain in theNH, or the NH did not fulfill the federalrequirement.

Hospice StatusTo identify hospice enrollment and death,

we linked MDS data with CMS claims data. NHresidents can choose to enroll in hospice if theyare certified by their attending physicians and ahospice medical director as terminally ill, whichis defined as having a life expectancy of 6months or less under normal disease progres-sion.9,23 These terminally ill residents can disen-roll and then re-elect the Medicare hospicebenefit; the number of hospice episodes isnot limited.

In order to compare the pain documented inthe MDS by hospice enrollment, we included

1000 Vol. 26 No. 5 November 2003Wu et al.

Fig. 1. Creation of the final study sample.

NH residents who did not enroll in hospiceduring their NH stay and those enrolled in hos-pice before the last MDS. In the first group ofresidents, the MDS assessments were notaffected by hospice, whereas in the secondgroup the MDS documentation may have beeninfluenced by hospice. Because we reasonedthat residents with more severe symptomschoose to enroll in hospice, which may cause avalid difference in documented pain by hospiceenrollment, we included a third group of resi-dents who enrolled in hospice but after theirlast documented MDS assessments. For theseresidents, the last MDS was not influenced byhospice, but because they were later referredto hospice we reasoned that their severity ofsymptoms would be more similar to residents

enrolled in hospice before the last MDS than tonon-hospice residents. Therefore, based on thedate of a resident’s hospice enrollment andthe date of the last MDS before death, we cate-gorized residents into three groups:

1. Residents who never enrolled in hospice(non-hospice residents).

2. Residents who enrolled in hospice beforetheir last MDS (hospice-pre-MDS residents).

3. Residents who enrolled in hospice aftertheir last MDS (hospice-post-MDS residents).

For residents in Group Two, we included notonly those who were on hospice when theirlast MDS were conducted (n � 342), but alsoresidents whose hospice enrollment ended

Vol. 26 No. 5 November 2003 1001The Problem of Assessment Bias

within 180 days before their last MDS (n � 73).This inclusion was based on an intention to treatreasoning—if a resident enrolled in hospice andwithdrew afterwards, his/her treatment and ser-vice strategieswere believedto bestill influencedby the prior hospice enrollment. Of the 9,613residents, 82.2% were non-hospice residents,4.3% were hospice-pre-MDS, and 13.5% werehospice-post-MDS.

Pain InformationThe MDS version 2.0 collects information re-

garding any experienced pain in the seven daysprior to the assessment date. Data can be fromresident report, observation by interviewers,physician records, or medical charts. For resi-dents who are not able to communicate, NHstaff are instructed in the MDS manual to lookfor non-verbal cues of body pain such as moan-ing or wincing. The MDS collects two measuresof pain—frequency (0 � no pain, 1 � pain lessthan daily, 2 � pain daily) and intensity (1 �mild pain, 2 � moderate pain, 3 � times whenpain is horrible or excruciating).14 As done byothers,4 we used these two measures to create ascale torepresent theseverityofpain.Sevenscoreswere given to indicate the increasing severity ofpain: 0 � no pain; 1 � less than daily mild pain;2 � daily mild pain; 3 � less than daily moderatepain; 4 � daily moderate pain; 5 � less than dailyexcruciating pain; and 6 � daily excruciatingpain. Residents with a pain score of greater orequal to 4 were considered to be in severe pain.4

Opioid UseThe MDS collects information on up to 18

medications that a resident uses during the sev-en days prior to the assessment date. Detailedinformation for each medication includes a Na-tional Drug Coding (NDC) code, the number ofdoses administered per day (per standing order),the number of doses administered for an asneeded order, and the route of administration.Based on the NDC code, we identified whetherresidents received opioids during the seven dayspreceding the assessment date.

Residents were classified as opioid users ifthey received at least one opioid prescription,either standard or as needed order. Regularopioid users were residents who received eitheran immediate-release opioid greater than orequal to twice a day or equivalent sustained-release opioid.

Analytic ApproachWe calculated and compared the prevalence

of any documented pain and severe pain forthe three hospice status groups. Additionally, be-cause previous research has shown care processesand outcomes for non-hospice NH residents todiffer when residing in NHs where there is ahospice presence,10,24 we examined documentedpain prevalence by hospice enrollment statusand by whether a resident was in a NH that hadany hospice enrollees. For this analysis, residentswere considered as hospice enrollees if they everreceived or enrolled in hospice services duringtheir NH stay. Otherwise, residents were consid-ered non-hospice. We categorized NHs into twotypes—those with at least one decedent who wasenrolled in hospice and those with no hospiceenrollees. The prevalence of documented painwas stratified by resident hospice status andthen by type of NH.

Pain is subjective and very hard to measurein NH residents who are frequently cognitivelyimpaired. Analgesic use has been considered lesssubjective and used as a surrogate measure toreflect residents’ experience of pain. We alsocompared opioid use in these residents withregard to their hospice status. For residentswithout documented pain in the last MDS, eitherthey did not experience pain, their pain was al-ready controlled, or the NH staff did not noticethe pain. We did compare analgesic use amongresidents with any documented pain by hospicestatus, but findings were not informative (datanot shown). Therefore, we restricted our analysisof analgesic use among those with documentedsevere pain. For these residents, we evaluatedwhether they were opioid users and regularopioid users (see above).

The hospice effect on documented pain preva-lence and analgesic use was analyzed with sep-arate logistic regression models with generalizedestimating equations (GEE) in SAS GENMOD(SAS Institute, Inc., Cary, NC). Two indicatorvariables were created for the two hospicegroups, with non-hospice serving as the referent.The use of GEE adjusted for the correlation oc-curring because residents resided in the samenursing home. The odds ratios derived from ourmodels were interpreted as relative risks.

In each of the logistic models, we consideredvariables as confounders if they altered any oneof the two estimates of the effect of hospice bymore than 10%. In our analysis we controlled

1002 Vol. 26 No. 5 November 2003Wu et al.

for four diagnosis groups found to be informa-tive in previous analyses (i.e., cancer without de-mentia, cancer with dementia, dementia onlyand diagnoses other than cancer or dementia)as well as for osteoporosis and arthritis. Diag-noses were obtained from the MDS. We alsoadjusted for the effect of age, sex, ethnicity (non-Hispanic White vs. all other ethnic groups com-bined), a six-point activities of daily living index(ADL), the seven-point Cognitive PerformanceScale (CPS), “do not resuscitate” (DNR) orders,number of days between last MDS and death,state where the NH was located, and type ofMDS assessments (admission vs. all other typescombined).

We calculated the predicted probabilities ofhaving an outcome for each of the four diagnosisgroups by hospice status. Predicted probabili-ties were calculated from the following formula:P � 1/(1 + e�(α + Σ β x)). We estimated the pre-dicted probabilities by evaluating this formula atthe average of the covariates and 0 or 1 at theindicator variables for diagnoses and hospicestatus. Ninety-five percent confidence intervals(CI) for the predicted probabilities were esti-mated by the formula Y � 1.96X′Cov(B)X.25

ResultsThe demographic characteristics were similar

across the three hospice status groups (Table1). Compared to non-hospice residents, cancer(with or without dementia) was more prevalentand dementia (without cancer) less prevalent inresidents with hospice pre- or post-MDS. On theother hand, the percentages of residents with“Other” diagnoses (i.e., without cancer or de-mentia) were similar between non-hospice andhospice-post-MDS residents (51% and 40%, re-spectively), whereas the percentage was muchlower in hospice-pre-MDS ones (27%). The per-centages of residents who had DNR orders werealso similar between non-hospice and hospice-post-MDS residents (57% and 58%, respectively),both of which were much lower than that ofhospice-pre-MDS residents (84%). The averagenumber of days between last MDS and death wasthe lowest for residents with hospice-pre-MDS,and highest for those with hospice-post-MDS(Table 1).

Of the 9,613 NH residents, 4,722 (49%) hadany pain and 2,147 (22%) had severe pain docu-mented in the last MDS assessments. The preva-lence of any documented pain was 46% for the

non-hospice group, 59% for the hospice-post-MDS group and 71% for the hospice-pre-MDSgroup; severe pain was 20%, 29% and 43%,respectively.

Compared to residents who lived in NHs withno hospice enrollees, the prevalence of severepain was higher among the residents who livedin NHs with at least one resident enrolled inhospice (18% versus 24%) (Table 2). Becausethe residents may differ systematically with regardto hospice status, we stratified the analysis byresidents’ hospice status. Among the non-hospiceresidents, prevalence of severe pain was stillhigher for those who lived in a NH with at leastone decedent enrolled in hospice (Table 2).

After controlling for confounders, residentswith hospice-pre-MDS were twice as likely to havesevere pain documented than were non-hos-pice residents (adjusted odds ratio [AOR] � 2.0,95% CI � 1.6–2.5). Residents with hospice-post-MDS were also more likely to have severe paindocumented, but to a much lesser extent thanhospice-pre-MDS residents (AOR � 1.3, 95%CI � 1.1–1.5) (Table 3). Factors that were associ-ated with an increased probability of recordedsevere pain were having a diagnosis of cancer,being female, having osteoporosis or arthritis,having a higher ADL score, living in SouthDakota, and the MDS assessment being an admis-sion assessment. Factors that were associated withdecreased probability of recorded severe painwere having a diagnosis of dementia and havingthe last MDS assessment performed further intime from death (Table 3).

The predicted probabilities of having severepain documented were highest in hospice-pre-MDS residents across the four diagnosiscategories (Table 4). Although the predictedprobability of severe pain in hospice-post-MDSresidents was higher than that for non-hospiceresidents, the predicted value is much closer tonon-hospice residents than it was to hospice-pre-MDS residents.

Based on data of 2,147 residents with docu-mented severe pain, 66% received at least onedose of opioid and 40% received regular opioidintervention. In multivariate analysis, hospice-pre-MDS residents, compared to non-hospiceresidents, had a 2.1 (95% CI � 1.3–3.2) timesgreater likelihood of receiving any opioid, and2.1 (95% CI � 1.4–3.0) times greater likelihoodof receiving regular opioid intervention (Table3). Hospice-post-MDS residents were less likely

Vol. 26 No. 5 November 2003 1003The Problem of Assessment Bias

Table 1Comparisons of Residents Included in Analysis by Hospice Status (n � 9,613)

Hospice Status

All Hospice pre-MDS Hospice post-MDS Non-Hospice(n � 9,613) (n � 415) (n � 1,299) (n � 7,899)

Age categories, %�65 3.4 3.6 2.5 3.665 � age �75 15.2 19.5 21.1 14.075 � age �85 36.3 38.8 41.2 35.4�85 45.1 38.1 35.2 47.1

Sex, %Female 62.4 63.1 60.3 62.7

Ethnicity, %Non-White 9.4 6.8 8.6 9.7

Diagnoses, %Cancer with dementia 4.1 11.8 4.5 3.6Cancer, no dementia 18.4 47.0 35.9 14.0Dementia 29.4 14.7 19.5 31.8Other 48.2 26.5 40.2 50.6

Osteoporosis, % 10.0 8.4 6.9 10.6Arthritis, % 23.5 16.9 19.0 24.6Advance directives

Do not resuscitate, % 57.9 83.9 57.7 56.6Cognitive performance 2.6 � 2.0 2.7 � 1.8 2.1 � 1.9 2.7 � 2.0

scale,a mean � SDActivity of daily living,b mean � SD 3.4 � 1.2 3.7 � 1.2 3.1 � 1.2 3.4 � 1.2Number of days between 48.5 � 39.6 34.4 � 29.8 58.1 � 43.5 47.7 � 39.1

last MDS and death, mean � SDStates, %

Illinois 26.9 28.7 32.0 25.9Maine 6.8 2.2 2.3 7.8Mississippi 5.7 3.4 5.8 5.8New York 12.7 5.5 5.0 14.4Ohio 44.7 56.6 53.6 42.7South Dakota 3.2 3.6 1.3 3.5

Type of MDS assessmentAdmission, % 37.1 47.0 43.6 35.1

SD � standard deviation.a 0 (intact) to 6 (very severe impairment).b 0 (minimal oversight) to 5 (highly dependent).

than hospice-pre-MDS residents, but more likelythan non-hospice residents to have received atleast one opioid (AOR � 1.3; 95% CI � 1.0–1.7)and to have received regular opioid intervention(AOR � 1.2; 95% CI � 0.9–1.5) (Table 3). Hos-pice-pre-MDS residents had the highest predictedprobability of receiving opioid among the threehospice status groups, regardless of their diagno-ses (Table 4).

DiscussionOur crude analyses show that the prevalence

of pain documented in the last MDS prior todeath increases gradually across non-hospice,hospice-post-MDS, and hospice-pre-MDS resi-dents. After controlling for sociodemographicand clinical factors, hospice enrollees are more

likely to have severe pain documented, com-pared to non-hospice residents; the predictedprobabilityofhavingseverepaindocumented wasconsistently highest among residents who hadbeen served by hospice prior to the MDS assess-ments (hospice-pre-MDS). Although a higherprevalence of pain was recorded for them, theyreceived better pain management than did resi-dents who were not enrolled in hospice at thetime of their last MDS assessments. They were sig-nificantly more likely to receive opioids and morelikely to receive an immediate-release opioidgreater than or equal to twice a day or equivalentsustained-release opioid.

These apparently contradictory findings areresolved if we assume that there is ascertainmentbias by hospice enrollment status in the measure-ment of pain. The prevalence of documentedpain is affected by a NH staff’s ability to identify

1004 Vol. 26 No. 5 November 2003Wu et al.

Tab

le2

Pre

vale

nce

ofSe

vere

Pai

nin

Nur

sing

Hom

esby

Whe

ther

Nur

sing

Hom

eH

asR

esid

ents

Enr

olle

din

Hos

pice

and

byIn

divi

dual

Res

iden

tE

nrol

lmen

tSt

atus

(n�

9,61

3)

Hos

pice

Stat

us

All

Res

iden

tsN

on-H

ospi

ceR

esid

ents

Hos

pice

Res

iden

ts

NH

wit

hA

llN

Hw

ith

atL

east

NH

wit

hA

llN

Hw

ith

atL

east

NH

wit

hat

Lea

stN

on-H

ospi

ceR

esid

ents

On

eH

ospi

ceE

nro

llee

Non

-Hos

pice

Res

iden

tsO

ne

Hos

pice

En

rolle

eO

ne

Hos

pice

En

rolle

e

Tot

alN

o.%

inSe

vere

Pain

Tot

alN

o.%

inSe

vere

Pain

Tot

alN

o.%

inSe

vere

Pain

Tot

alN

o.%

inSe

vere

Pain

Tot

alN

o.%

inSe

vere

Pain

All

3,21

118

6,40

224

3,21

118

4,68

821

1,71

433

Illin

ois

616

191,

965

2661

619

1,43

023

535

34M

ain

e47

123

187

3347

123

148

3139

41M

issi

ssip

pi19

312

350

2719

312

261

2689

30N

ewYo

rk59

99

625

1259

99

537

1188

18O

hio

1,18

021

3,12

025

1,18

021

2,18

922

931

32So

uth

Dak

ota

152

2815

526

152

2812

320

3253

and record pain. The MDS data for hospice-pre-MDS residents were most likely influenced byhospice enrollment. On the other hand, theMDSs of hospice-post-MDS and non-hospice resi-dents were not influenced by hospice enroll-ment, and thus, should be of similar quality.Previous studies have suggested that hospice isassociated with better assessment of symp-toms.11,24 In our study, the pain prevalence inhospice-post-MDS residents was more similar tothat of non-hospice residents than it was to hos-pice-pre-MDS residents; this similarity suggeststhat hospice enrollment at or before completionof an MDS assessment plays an important rolein how pain is assessed and documented inthe MDS. The prevalence of documented pain isalso affected by the incidence and/or durationof pain. Residents may choose to enroll in hos-pice due to unbearable symptoms, for example,severe pain. In our study population, cancer wasthe most prevalent diagnosis for hospice-pre-MDS residents and dementia is the most preva-lent diagnosis for non-hospice residents. Canceris known to be associated with frequent severepain, whereas dementia was associated with alesser frequency of pain.3,17 This fact might ex-plain the gradual increment in the crude preva-lence of documented pain across the threehospice status groups. However, after controllingfor diagnoses, the likelihood of having docu-mented pain was still the highest for hospice-pre-MDS residents, and lowest for non-hospiceresidents.

Hospice affects the quality of symptom assess-ment not only at the individual level, but also atfacility level. We found that among non-hospiceresidents, there was higher prevalence of docu-mented pain in NHs with any hospice enrolleesthan in NHs with no hospice enrollees. In NHswith hospice care, staff may improve their painassessment and documentation skills, perhapsbecause of a diffusion of hospice knowledge andpractices. Such a “spill over” effect of hospiceservice in NHs has been found by other studies.Miller et al. found that the higher the hospiceconcentration in a NH, the lower the rate ofhospitalization of NH residents during the last 30days of life.26 Alternatively, staff in NHs thatcontract with hospice may have superior painassessment and documentation skills prior to theNH’s decision to contract with hospice.

As described below, there are at least threemechanisms to explain why pain ascertainment

Vol. 26 No. 5 November 2003 1005The Problem of Assessment Bias

Table 3Effect of Hospice Status on Documented Pain and Opioid Use

Among Residents with Severe Pain Documentedin the Last MDS (n � 2,147)

Moderate Daily or Received Any Opioids Received Regular OpioidWorst Pain Recorded in During the 7 Days Intervention During the

the Last MDS (n � 9,613) Before Last MDS 7 Days Before Last MDS

OR (95% CI) OR (95% CI) OR (95% CI)

Hospice statusa

Hospice-pre-MDS 2.0 (1.6–2.5) 2.1 (1.3–3.2) 2.1 (1.4–3.0)Hospice-post-MDS 1.3 (1.1–1.5) 1.3 (1.0–1.7) 1.2 (0.9–1.5)

Diagnosis groupsb

Cancer, no dementia 2.0 (1.8–2.3) 2.2 (1.7–2.7) 2.7 (2.1–3.3)Cancer with dementia 1.3 (1.0–1.7) 1.7 (1.0–2.8) 1.6 (1.0–2.5)Dementia only 0.6 (0.5–0.7) 0.6 (0.5–0.9) 0.8 (0.6–1.2)

Age (per year increase) 1.0 (1.0–1.0) 1.0 (1.0–1.0) 1.0 (1.0–1.0)Ethnicity: non-White 0.8 (0.7–1.0) 1.1 (0.7–1.5) 1.4 (1.0–1.9)Sex: female 1.5 (1.3–1.7) 0.9 (0.8–1.2) 1.1 (0.9–1.3)Activities of daily living 1.2 (1.1–1.2) 1.0 (0.9–1.1) 0.9 (0.9–1.0)

(per unit increase)Cognitive performance scale 0.8 (0.8–0.9) 1.0 (0.9–1.0) 1.0 (0.9–1.1)

(per unit increase)Diagnosis of arthritis 1.6 (1.4–1.8) 1.2 (1.0–1.5) 1.1 (0.9–1.4)Diagnosis of osteoporosis 1.6 (1.4–1.9) 1.4 (1.1–1.8) 1.6 (1.2–2.2)Interval between last 1.0 (0.9–1.0) 1.0 (0.9–1.0) 1.0 (0.9–1.0)

MDS and death(per 10-day increase)

Advance directives: 1.1 (1.0–1.3) 1.1 (0.9–1.4) 1.4 (1.2–1.8)Do-not-resuscitate order

Statesc

Illinois 0.9 (0.8–1.1) 0.8 (0.6–1.0) 0.8 (0.6–1.0)Mississippi 0.8 (0.6–1.1) 1.2 (0.8–2.0) 1.1 (0.7–1.6)Maine 1.2 (1.0–1.5) 1.0 (0.7–1.4) 1.0 (0.7–1.5)New York 0.6 (0.5–0.7) 0.5 (0.4–0.8) 1.0 (0.6–1.5)South Dakota 1.4 (1.1–1.8) 1.0 (0.6–1.6) 1.0 (0.6–1.6)

Type of MDS assessment 1.3 (1.2–1.5) 1.0 (0.8–1.3) 1.0 (0.8–1.2)Admission assessment

CI � Confidence Interval.aReference group is non-hospice.bReference group is ‘Other.’cReference state is Ohio.

bias may occur. First, to justify regular opioidprescription (more common for hospice resi-dents), NH staff may feel obliged to record pain;second, there may be differential reports ofpain by residents; and third, there may be differ-ential assessment of pain by NH staff and hos-pice staff.

The Recording of Pain to JustifyOpioid Prescription

During the past decades, there have been greatadvances in medical management of pain. Guide-lines on analgesic use in the elderly are also devel-opedandpublished.27 Therecommendeddosageof analgesics to adequately manage pain at theend of life is increasingly high. Such doses mayhasten or accidentally cause death. Conceptu-ally, accidental death due to high dose of analge-sics with initial intention of symptom relief will

not be considered murder or assisted suicide.However, in real life there is no clear-cut distinc-tion between the two. Nurses may face criminalcharges if such accidents happen.28 In order toavoid the risk, nurses may feel obliged to recordpain although the symptom is already con-trolled with opioids the residents receive.

Differential Reports by ResidentsPeople have differing attitudes regarding pain,

and these attitudes vary dramatically betweengenders and across cultures.2,29 Such attitudesmay cause differential reporting of pain. For ex-ample, women are more likely than men toreport and to complain about pain. Many peopleconsider pain as a normal component of dis-ease and aging.30 Older people are reluctant toreport it. In fact, in one study, even when insubstantial pain, most of the NH residents were

1006 Vol. 26 No. 5 November 2003Wu et al.

willing to tolerate pain rather than to report it.Few residents asked for medication change al-though they felt the control was inadequate.31

We found in our study population that women,Whites, and younger residents were more likelyto have documented pain. The attitudes towardspain may also differ between hospice and non-hospice residents. In hospice, the emphasis ofintervention switches from cure to symptom con-trol. When residents choose to enroll in hospicethey accept the care philosophy to someextent; therefore, they might be more willing toreport pain to caregivers. Cognitive impairmentis another barrier of pain reporting. Normally,caregivers ask residents about their pain, but resi-dents with cognitive impairment are less likelyand able to report pain. Instead, pain assessmentis often dependent on non-verbal cues. In ourstudy, the prevalence of dementia was higherfor non-hospice residents. Although the MDSmanual gives detailed instructions on how toassess pain in non-verbal residents, NH staff maynot be that experienced in assessing these non-verbal cues.

Differential Assessment and DocumentationAssessment and documentation of pain

may be biased by NH staff due to their priorknowledge of residents’ diagnoses. Although the

perception of pain varies from person to personand is associated with psychological factors,2 NHstaff may ignore this variation and develop theirown judgments unconsciously based on resi-dents’ diagnoses. During data collection theymay probe for answers they anticipate. Ohrbachet al. and Chibnall et al. found that observers’expectations of pain level can bias the assessmentof pain.32,33 Chibnall et al. found that medicalstudents tended to overestimate the severity ofpain for patients with cancer and physical disabil-ities compared to patients without cancer orphysical disabilities.

Considering the above, we cannot fully explainwhy, after controlling for diagnoses and othercase-mix variables, hospice residents were morelikely than were non-hospice residents to havesevere pain documented. A likely factor contrib-uting to this probable ascertainment bias is thatNH hospice and non-hospice residents are as-sessed by different staff. When a residentchooses to enroll in hospice, a hospice care teamassumes the responsibility for designing the treat-ment strategy and for providing palliative careservices to the NH resident. Hospice staff are ex-perts in assessing and managing physical andpsychosocial symptoms. On the other hand, perMedicare and Medicaid regulations, NH staff spe-cializes in intervention and rehabilitation. Pain

Table 4Predicted Probabilities of Having Severe Pain Documented and of Opioid Use

When Severe Pain is Documented

Among Residents with Severe Pain Documentedin the Last MDS (n � 2,147)

Moderate Daily or Received Any Received RegularWorst Pain Recorded Opioids During the Opioid Intervention During

in the Last MDS 7 Days Before the 7 Days Before(n � 9,613) Last MDS Last MDS

Diagnoses Hospice status P (95% CI) P (95% CI) P (95% CI)

Cancer, no dementiaHospice-pre-MDS 0.47 (0.42–0.53) 0.87 (0.82–0.91) 0.70 (0.62–0.77)Hospice-post-MDS 0.37 (0.34–0.41) 0.82 (0.77–0.85) 0.56 (0.50–0.62)Non-hospice 0.31 (0.29–0.34) 0.77 (0.73–0.80) 0.52 (0.48–0.57)

Cancer, with dementiaHospice-pre-MDS 0.37 (0.30–0.44) 0.84 (0.74–0.91) 0.58 (0.44–0.70)Hospice-post-MDS 0.28 (0.22–0.34) 0.77 (0.66–0.85) 0.43 (0.32–0.56)Non-hospice 0.23 (0.18–0.27) 0.72 (0.60–0.81) 0.39 (0.29–0.51)

DementiaHospice-pre-MDS 0.21 (0.17–0.25) 0.67 (0.55–0.77) 0.42 (0.31–0.54)Hospice-post-MDS 0.15 (0.13–0.17) 0.57 (0.48–0.65) 0.29 (0.22–0.37)Non-hospice 0.12 (0.11–0.13) 0.50 (0.43–0.56) 0.26 (0.20–0.32)

OtherHospice-pre-MDS 0.31 (0.26–0.36) 0.76 (0.67–0.83) 0.46 (0.38–0.55)Hospice-post-MDS 0.23 (0.20–0.25) 0.67 (0.61–0.73) 0.33 (0.27–0.39)Non-hospice 0.18 (0.17–0.20) 0.61 (0.57–0.64) 0.29 (0.26–0.32)

P � Predicted probability; CI � Confidence Interval.

Vol. 26 No. 5 November 2003 1007The Problem of Assessment Bias

assessment and management are not the focusof NH staff, and many staff have not had trainingin such. Considering this, it is probable that hos-pice staff would be more likely than NH staff toelicit complaints of pain and to notice signs ofpain.11

Importance of Our FindingsHospice has been introduced into NH since

1985.9 Many studies have tried to assess the hos-pice effect on pain management. Because proxyoutcomes of pain management (e.g., analgesicuse) are easier to measure and less subject tobias, one would expect consistent results in stud-ies using such outcomes. Indeed, studies havefound that hospice is associated with better anal-gesic use.10,34 Our results are consistent withthese previous findings. However, results fromcomparative studies on prevalence and/or sever-ity of pain, the direct measurement of outcomeof interest, are inconsistent.8,35 One reason forthis inconsistency appears to be the variationin the quality of the data used. We believe thatMDS data is subject to ascertainment bias in rela-tion to hospice/non-hospice care. Therefore,hospice/non-hospice comparative studies basedon MDS data may lead to biased results.

Ascertainment bias of MDS pain data has beennoted in other research. A study by Miller et al.found the differential documentation of symp-toms on the MDS by hospice enrollment.36 Theyalso observed that pain assessment is morelikely to be documented for non-hospice resi-dents when these residents are in nursing homeswith a higher proportion of residents enrolledin hospice (vs. in nursing homes with no or littlehospice).37 In our study, we used data based ona newer version of the MDS, in which painintensity was collected. We were able to comparethe prevalence of residents with severe pain andthe use of analgesics controlling for severity ofpain. Our results provide additional support thatpain ascertainment bias by hospice enrollment ispresent in MDS data. Studies have suggested thatascertainment bias exists for other healthoutcomes documented in MDS (e.g., depressionand pressure ulcer).15,38

Prevalence of uncontrolled pain is now one ofthe quality indicators (QI) that are used by CMSto monitor NH quality of care. The indicatorconsiders the number of residents in moderatedaily or worse pain divided by number of all

residents in the NH.38 Using similar method-ology, we calculated the prevalence of severe pain(Table 2), but included only decedents in ournumerators and denominators. In a national vali-dation study, Mor et al. compared the MDS col-lected by study nurses against the MDS collectedby NH staff.19 They demonstrated that the painQI is valid and reliable and suitable for monitor-ing quality of care in NH by CMS. They alsofound that the higher the pain QI score a NHhas, the more likely its NH staff are to detectpain as detected by the study nurses. We foundconsistent differences in pain prevalence by aresident’s hospice status and by whether a NHcontracted with hospice. With introduction ofpain QI, NHs will probably be less likely to recordpain to justify medications given. However, differ-ential pain assessment by hospice and for NHcontracting with hospice in all likelihood will stillbe present. As NHs become more skilled at painassessment, the MDS data can be used for hos-pice/non-hospice for comparative analysis, butthis will have to be monitored. Until improve-ment is observed, such comparative analysis isprobability unduly influenced by ascertainmentbias and results/conclusions are likely to be in-valid because of this. At present, comparativeanalysis of pain prevalence in NHs by hospice/non-hospice status, is best performed by usingexternally trained assessors for both hospice andnon-hospice residents.

LimitationsThere are several limitations in our study. For

each resident, the MDS documents any pain ex-perienced and medication received for the sevendays before assessment. Temporal sequencingcan be a problem. It is possible that during theseven days, opioids were prescribed after resi-dents complained about pain or NH staff noticedsigns of pain, but not for the full seven days. Toavoid such problems, we presented the preva-lence of having severe pain in NH residents.Severe pain is defined as daily moderate, less thandaily excruciating and daily excruciating pain. Ifsevere pain is assessed, physicians should havebeen able to give some opioid to control it.Since sometimes it takes days to titrate the opioiddosage to the optimal level, residents may stillhave some pain. But, it is not likely that residentsreceiving regular opioid will still have severe pain,especially daily excruciating pain.

1008 Vol. 26 No. 5 November 2003Wu et al.

The pain scale we used has not been validated.MDS 2.0 is only officially mandated since 2000.It is the first time that MDS included questionsabout both intensity and frequency of pain. Fewstudies have developed and validated pain scalesbased on MDS 2.0. In 2000, Fries et al. developeda pain scale that is a little different from what weused.39 It categorized pain into four levels: 1) nopain, 2) mild pain—less than daily no matterhow severe it is; 3) moderate pain—daily mildor daily moderate; and 4) excruciating pain—daily excruciating. They validated the scaleagainst Visual Analogue Scale. Their analysisshowed good agreement between the twoscales. However, we were uncomfortable cate-gorizing residents with less than daily excruciat-ing pain as mild. Therefore, we used the scaledeveloped and used in other NH research.4 Also,we wanted to use a pain scale more similar to whatCMS uses to monitor a NH’s quality.38

We were not able to prove directly that as-certainment bias exists in MDS data, or to deter-mine to what extent such ascertainment biaswould affect estimates. Such analysis would re-quire validation data to be collected with strin-gent protocols by trained staff. Although thereare new statistical methods for adjusting ascer-tainment bias,40 they have not been tested inMDS data. Future research, however, will applythese advanced statistical methods to validate thepresence of pain ascertainment bias and itsextent.

SummaryUsing MDS assessment data to examine the

effect of hospice on end-of-life pain prevalenceresulted in counterintuitive findings. Residentsin hospice at their last MDS were more likelythan were non-hospice residents and residentseventually enrolled in hospice to receive opi-oids for their moderate to severe pain. However,hospice residents were twice as likely as non-hos-pice residents and 1.3 times as likely as resi-dents eventually enrolled in hospice to have painassessment documented. These contrary findingsprovide support for the notion that there is differ-ential assessment and documentation of pain inthe MDS when hospice is involved in care.

AcknowledgmentsThis studywas funded by theAgency for Health-

care Research and Quality, Department of Health

andHumanServices(1R01HS10549-01).Theau-thors would like to thank Dr. Vincent Mor ofBrown University for his thoughtful reviews of thisarticle and computer programmer Jefferey Hirisof Brown University who helped to compile thedata file.

References1. Available from www.cdc.gov/nchs/datawh/statab

/unpubd/mortabs/gmwk309_10.htm.

2. Won A, Lapane K, Gambassi G, et al. Correlatesand management of nonmalignant pain in the nursinghome. SAGE Study Group. Systematic Assessment ofGeriatric drug use via Epidemiology. J Am Geriatr Soc1999;47(8):936–942.

3. Bernabei R, Gambassi G, Lapane K, et al. Manage-ment of pain in elderly patients with cancer. SAGEStudy Group. Systematic Assessment of Geriatric druguse via Epidemiology. JAMA 1998;279(23):1877–1882.

4. Teno JM, Weitzen S, Wetle T, et al. Persistent painin nursing home residents. JAMA 2001;285(16):2081.

5. Available from www.chcr.brown.edu/COMMSTA-TE/PAINMONOGRAPHWEBVERSION.PDF.

6. Morris JN, Suissa S, Sherwood S, et al. Last days:a study of the quality of life of terminally ill cancerpatients. J Chronic Dis 1986;39(1):47–62.

7. Butler RN, Burt R, Foley KM, et al. A peacefuldeath: how to manage pain and provide quality care. Aroundtable discussion: Part 2. Geriatrics 1996;51(6):32–42.

8. Greer DS, Mor V, Morris JN, et al. An alternativein terminal care: results of the National Hospice Study.J Chronic Dis 1986;39(1):9–26.

9. Consolidated Omnibus Budget Reconciliation Actof 1985 (P.L. 99–272). U.S. Congress, 1985.

10. Miller SC, Mor V, Wu N, et al. Does receipt of hos-pice care in nursing homes improve the managementof pain at the end of life? J Am Geriatr Soc 2002;50(3):507–515.

11. Hollen CJ, Hollen CW, Stolte K. Hospice andhospital oncology unit nurses: a comparative surveyof knowledge and attitudes about cancer pain. OncolNurs Forum 2000;27(10):1593–1599.

12. McWhinney IR, Bass MJ, Donner A. Evaluationof a palliative care service: problems and pitfalls. BMJ1994;309(6965):1340–1342.

13. Jordhoy MS, Kaasa S, Fayers P, et al. Challengesin palliative care research; recruitment, attrition andcompliance: experience from a randomized con-trolled trial. Palliat Med 1999;13(4):299–310.

14. Morris JN, Murphy K, Nonemaker S. Long-TermCare Facility Resident Assessment Instrument (RAI)User’s Manual. October, 1995.

Vol. 26 No. 5 November 2003 1009The Problem of Assessment Bias

15. Schnelle JF, Wood S, Schnelle ER, et al. Measure-ment sensitivity and the Minimum Data Set depres-sion quality indicator. Gerontologist 2001;41(3):401–405.

16. Snowden M, McCormick W, Russo J, et al. Valid-ity and responsiveness of the Minimum Data Set. JAm Geriatr Soc 1999;47(8):1000–1004.

17. Fisher SE, Burgio LD, Thorn BE, et al. Pain as-sessment and management in cognitively impairednursing home residents: association of certified nurs-ing assistant pain report, Minimum Data Set painreport, and analgesic medication use. J Am GeriatrSoc 2002;50(1):152–156.

18. Hawes C, Morris JN, Phillips CD, et al. Reliabilityestimates for the Minimum Data Set for nursinghome resident assessment and care screening (MDS).Gerontologist 1995;35(2):172–178.

19. Available from www.cms.gov/providers/nursing-homes/nhi/validation_fd_082002.pdf.

20. Phillips CD, Morris JN. The potential for usingadministrative and clinical data to analyze outcomesfor the cognitively impaired: an assessment of theminimum data set for nursing homes. Alzheimer DisAssoc Disord 1997;11(Suppl 6):162–167.

21. Lawton MP, Casten R, Parmelee PA, et al. Psycho-metric characteristics of the minimum data set II:validity. J Am Geriatr Soc 1998;46(6):736–744.

22. Gambassi G, Landi F, Peng L, et al. Validity ofdiagnostic and drug data in standardized nursinghome resident assessments: potential for geriatricpharmacoepidemiology. SAGE Study Group. System-atic Assessment of Geriatric drug use via Epidemiol-ogy. Med Care 1998;36(2):167–179.

23. Tax Equity and Fiscal Responsibility Act of 1982(TEFRA-82), P.L. 97–248.

24. Miller SC, Gozalo P, Mor V. Outcome and utiliza-tion for hospice and non-hospice nursing facility de-cedents. In: Synthesis and analysis of Medicare’sHospice Benefit. Washington, DC: Office of Disabil-ity, Aging, and Long Term Care Policy in the Officeof the Assistant Secretary for Planning and Evalua-tion, U.S. Department of Health and Human Service,2000. Contract number: 100-97-0010. Available fromwww.aspe.hhs.gov/daltcp/reports/samhbes.htm.

25. Kleinbaum DG, Kupper LL, Morgenstern H. Epi-demiologic research: principles and quantitativemethods. New York: Van Nostrand Reinhold, 1982.

26. Miller SC, Gozalo P, Mor V. Hospice enrollmentand hospitalization of dying nursing home patients.Am J Med 2001;111(1):38–44.

27. American Geriatrics Society. Clinical PracticalGuidelines. The management of chronic pain inolder persons: AGS Panel on Chronic Pain in OlderPersons. J Am Geriatr Soc 1998;46(5):635–651.

28. Alpers A. Criminal act or palliative care? Prose-cutions involving the care of the dying. J Law MedEthics 1998;26(4):308–331.

29. Edwards CL, Fillingim RB, Keefe F. Race, eth-nicity and pain. Pain 2001;94(2):133–137.

30. Cramer GW, Galer BS, Mendelson MA, ThompsonGD. A drug use evaluation of selected opioid andnonopioid analgesics in the nursing facility setting.J Am Geriatr Soc 2000;48(4):398–404.

31. Yates PM, Edwards HE, Nash RE, et al. Barriersto effective cancer pain management: a survey ofhospitalized cancer patients in Australia. J Pain Symp-tom Manage 2002;23(5):393–405.

32. Ohrbach R, Crow H, Kamer A. Examiner expec-tancy effects in the measurement of pressure painthresholds. Pain 1998;74(2–3):163–170.

33. Chibnall JT, Tait RC, Ross LR. The effects ofmedical evidence and pain intensity on medical stu-dent judgments of chronic pain patients. J BehavMed 1997;20(3):257–271.

34. Goldberg RJ, Mor V, Wiemann M, et al. Analgesicuse in terminal cancer patients: report from theNational Hospice Study. J Chronic Dis 1986;39(1):37–45.

35. Kane RL, Klein SJ, Bernstein L, et al. The roleof hospice in reducing the impact of bereavement.J Chronic Dis 1986;39(9):735–742.

36. Miller SC, Mor V, Teno J. Hospice enrollmentand pain assessment and management in nursinghomes. J Pain Symptom Manage 2003;26:791–799.

37. Miller SC, Mor V, Teno J. Hospice in nursinghomes: an empirical examination of its scope andquality outcomes. Final Report. The Retirement Re-search Foundation (529043), Chicago, Il. 2000.

38. Harris Y, Clauser SB. Achieving improvementthrough nursing home quality measurement. HealthCare Financ Rev 2002;23(4):5–18.

39. Fries BE, Simon SE, Morris JN, et al. Pain inU.S. nursing homes: validating a pain scale for theminimum data set. Gerontologist 2001;41(2):173–179.

40. Gozalo P, Linton O. Local nonlinear leastsquares: using parametric information in nonpara-metric regression. J Econometrics 2000;99(1):63–106.