Topics in Health Economics – class 3 Matilde P. Machado [email protected]

Topics in Health Economics – class 3

Matilde P. Machado

[email protected]

Gautam Gowrisankaran and Robert Town (JHE, 1999) Title: Estimating the Quality of Care in Hospitals

using Instrumental Variables Typically the literature on Hospital Quality has used

hospital-specific output measures such as mortality rates to compare hospital performance.

Potential Problem with these measures is that they suffer from ENDOGENEITY which can bias estimates in the wrong direction i.e. the best hospitals may show worse mortality data simply because they treat the most difficult patients.

Gautam Gowrisankaran and Robert Town (JHE, 1999) In an equation such as:

It could be that the best hospitals receive the most difficult cases i.e. attract a sicker population of patients and have higher mortality rates. Therefore i has two components (quality and severity)

Researchers try to solve this selection bias by including case-mix variables, i.e. variables that capture the severity of the patients received at each hospital (DRGs, main diagnosis, secondary diagnosis, demographic variables (age, gender, comorbidities))

1

where hospital, mortality rate

N

it i i it iti

y H u i y

1

where

N

it it i i iti

y X H u i hospital

Gautam Gowrisankaran and Robert Town (JHE, 1999)

Still the problem remains if there is unobserved severity that is not captured by X but is correlated with the H dummies, in which case the best hospitals may look worse or not much better than the rest of hospitals (u’s are not equally distributed across the H’s, the H’s are endogenous):

To avoid this bias, Gowrisankaran and Town (1999): Suggest to use IV that are correlated with H dummies but not to these

unobservables. This paper shows evidence of remaining selection even after controlling

for observed patient characteristics.

1

where

N

it it i i iti

y X H u i hospital


The authors use mortality due to pneumonia from Medicare patients’ data (patients older than 65 years of age) because: Pneumonia is a common disease among this age group It may lead to death, it is the 4th leading cause of death among the elderly. In

their sample in-hospital mortality due to pneumonia in 1989 was 17.9%. The outcome depends in a great deal on the hospital’s procedures, therefore,

an important part of the variance in outcomes can be explained by differences in hospital quality.

Note: It is important to notice that the paper can estimate only the quality of hospitals in pneumonia treatment (which may or may not be correlated with other aspects of hospital quality)

Other researchers have used pneumonia mortality as a benchmark for hospital quality (Keeler et al. JAMA , 1990, and McGarvey and Harper, 1993).

When diagnosed with pneumonia the patient/doctor have time to choose the “right” hospital, they do not need to hurry to the closest one (as with heart attacks). This allows to test whether selection is important.

Medicare patients pay the same out-of-pocket expenditures at every hospital

Gautam Gowrisankaran and Robert Town (JHE, 1999) The Model of mortality and hospital choice

The Choice Equation: Once the patient becomes ill with pneumonia he (his physician) chooses the hospital based on: the perceived quality of treatment, the patient’s severity of illness, the cost to the patient, and the distance to the hospital.

1 if patient chooses hospital 1,...J

0 otherwise

( , , ) there are J equations, one for each hospital

z set of hospital characteristics (location, services, ownership)

set of indi

ij

ij i ij

i

i jc j

c f z u

vidual characteristics, include location or distance

characteristics unobserved to the researchers, not to the patient/physician including severityiju


The mortality Equation: They model it as a hazard function that gives the probability of a patient dying in a given day conditional on being alive at the beginning of the day. The hazard depends on the observed and unobserved severity of the patient’s illness, the number of days since admission and the quality of treatment received at the hospital chosen.

1

1 if the patient dies on day t

0 otherwise

( ,...., ) vector of dummy variables

vector of patient characteristics that affect the patient's mortality

age,

it i i t it it

it

i i iJ

i

m c x d s

im

c c c

x

gender, disease stage, income, diagnosis, comorbidities

dummy variables for days since admission

unobserved severity t

it

d

s

i are interpreted as hospital quality. Parameters of interest

Linear probability model


Notice that sit will affect the decision of which hospital to go to, more severe patients may choose higher quality hospitals, i.e. sit may be part of the error term uit ,which implies that the error terms in both equations are correlated, implying that ci is correlated with the error term in the mortality equation.

Deaths after discharge are not observed. Important!: Assume sit (residual health status after controlling for all

the Xs) and it are identically and independently distributed in the population and orthogonal to the regressors xi and dt.

error term

it i i t it itm c x d s1442443

Gautam Gowrisankaran and Robert Town (JHE, 1999) Instruments:

There are J-1 endogenous variables (c1,….cJ-1), we need at least as many instruments. There is a reference hospital, example for J=2:

Distance of each patient to hospitals as IVs for ci. Comments: A good IV must be correlated with ci and not with sit or it. A patient’s distance to hospital j is negatively correlated to the choice of

that hospital so should be a good candidate. A patient distance to a given hospital should not be correlated with the error

term sit+it under the assumption of equal distribution of sit

However, more severe patients may be willing to travel larger distances to be treated at a particular hospital. So distance of a given patient to the chosen hospital is correlated with severity.

1 1 2 2 1 2 2 2 1 2 1 2(1 ) ( )i i i i i i i i i i i iy c c X v c c X v c X v


Imagine same distribution of severity in town A as in B. For simplicity, assume each town has a hospital in downtown i.e. symmetric distance of patients to A and B (dA and dB is independent of severity distribution)

Most severe patients (shown in red) are in the same proportion in both towns.

Hospital A is of higher quality – so all patients in region A go to A and all severe patients (in red) in region B go to A. So the average distance of patients that choose hospital A is higher than the average distance of patients that chose B. So distance to chosen hospital is correlated with severity (invalid instrument) a simple code

A B


But is the assumption that severity is equally distributed a plausible one? Presumably more sick patients may prefer to leave closer to a good hospital. If this is so then their instrument is not valid.

A B

Patients in red are severe while patients in green are less severe. In this case, severe patients locate close to the best hospital, say A. So distance to A would be negatively correlated with severity, those with less distance to A are the most severe. The opposite for B. Distance to A and B would not be valid instruments

Note: One instance in which this is so, is nursing homes which tend to locate near high quality hospitals. For these reason they discard patients admitted from nursing homes.

Gautam Gowrisankaran and Robert Town (JHE, 1999) Instruments (cont.):

Use 2J instruments: dij ≡ distance of patient i to each j hospital

exp(-dij) ≡ a non-linear function of distance. The reason is that most likely in the choice equation the effect of

distance is non-linear so they rely on more instruments (any function of distance should do) but they rather estimate

They estimate the following equation for each hospital separately:

1 2 3 exp( )ij j j ij j ij ijc d d u

maximum likelihood would do. T


For <1 the function exp(-d) looks as follows:

distance

For each variable cij they obtain an estimate of j. They set equal to the average value of 0.25

Gautam Gowrisankaran and Robert Town (JHE, 1999) They estimate the mortality equation with a linear probability

models. Linear probability models: There is heteroskedasticity, can be solved via GLS. Predicted values lie outside the [0,1] range, cannot be

solved. The coefficients have nice interpretations, i.e. j is the

incremental probability of death on any day at hospital j. Technical:

In a linear regression, F is linear then: ( )

where 1 ,

E y x

y x x x

Gautam Gowrisankaran and Robert Town (JHE, 1999) Then the variance of the error term is given by:

Then if the error term is itsit+it the variance of the error

term is var(it)= it(1-it).

0

0

2 2 20 0

2 20 0

2 20 0 0 0 0 0

if 0

1 1 if 1

For 0, variance is:

var( ) ( ) ( ) Pr( 0) (1 ) Pr( 1)

( ) (1 ) (1 ) ( )

( ) (1 ) (1 ) ( ) (1 ) (1 )

x yy x

x y

E y y

x x

x x

Gautam Gowrisankaran and Robert Town (JHE, 1999)Two step procedure:1. IV regression (not controlling for heteroskedasticity) consistent (but

not efficient) estimates.

2. Construct for each observation based on the IV estimates from step 1

3. Because the var-cov matrix is diagonal with elements pre-multiply the mortality equation by .Notice they use a slightly different formula to correct for (rare) cases where the estimated residuals are negative or greater than 1:

apply IV again to the transformed variables. Estimates will be consistent and asymptotically efficient (it is not necessary to transform the instruments).

1

ˆ ˆmax (1 ),0.01it it

ît

ˆ ˆ1it it

ˆ ˆ1 1it it

Gautam Gowrisankaran and Robert Town (JHE, 1999) Endogeneity Test (Hausman,1978) on the c’s: Compares IV-

GLS and GLS. H0: No endogeneity (IV-GLS≈GLS). Need a consistent and efficient estimator under the null and a consistent estimator under the alternative.

Overidentification test (Bowden and Turkington, 1984), the idea is that the Z and the ’s should not be correlated:

^ ^

21

1 1

ˆ ˆ ˆ ˆ ˆ ˆvar( ) var( ) ~IV GLS IV GLs IV GLS J

K KK K

H

12 2

1

( ) 11 ( ) ( ) ( )

ˆ ˆˆ ~ where J+1 is the number of overidentifying restrictions

(2J instruments f

J

cols Zcols Z cols Z cols Z

O s Z Z Z Z

or J-1 endogeneous variables) S2 is the estimated variance of the residuals in the second stage.

Full vector of estimates

J-1 number of endogeneous variables

Gautam Gowrisankaran and Robert Town (JHE, 1999)Data Selection They keep patients ≥65 years old that entered into a hospital in one of 4

of the California counties with a diagnosis of pneumonia (they specify carefully their selection based on the diagnosis)

They eliminate all patients that were transferred from nursing homes and other medical facilities since these facilities hold the most severe cases and are usually close to the highest quality hospital so that the assumption that sit is equally distributed in the population would be violated.

Remove small hospitals from the sample because the FE for these hospitals would be very imprecise.

Drop patients where no matching with income data is possible due to “unknown race” for example.

Final data has 178,972 observations

Gautam Gowrisankaran and Robert Town (JHE, 1999)Results:

1. Their main result is that the Hausman test rejects (at 1%) the null hypothesis of exogeneity. So c’s are endogenous, i.e. there is selection.

2. Moreover they find that the spread of IV coefficients of hospital quality is larger than the spread of the GLS coefficients. Because the severity is positively correlated with quality (i.e. negatively correlated with the beta’s since lower betas means high quality) this is evidence of this correlation. Why? For the low true betas (high quality hospitals) the non-iv estimate is higher because these hospitals receive more serious cases i.e. it underestimates the true quality, and the opposite for the low quality hospitals. Therefore the non-iv estimates of beta have less variance than the unbiased estimates.


True betas

Estimated betas

Unbiased estimates 45º line

Take beta=0 as the benchmark. So beta below 0 is better than the benchmark hospital. The more negative the better (less mortality).

- - GLS estimates

GLS estimates

GLS estimatesHigh quality

Overestimates the quality hereUnderestimates

the quality here

Low quality

Gautam Gowrisankaran and Robert Town (JHE, 1999)Results (cont.):

3. They also find that the correlation between the GLS and IV was negligible which implies that the GLS completely misses the correct hospital ranking.

4. The standard deviations of the IV estimates are high which implies that the quality of most hospitals is not statistically significantly different from the quality of the average hospital.

What explains hospital quality? Why are some hospitals better than others: The authors cannot include in their mortality regression hospital characteristics (in-

variant with time) because these are colinear with hospital dummies. What is possible is once we obtain FE estimates, to regress them against a bunch of

hospital characteristics to see what characteristics better explain the differences in quality. {type of organization (public, not-for-profit, for-profit), teaching hospital dummy, dummy for operating long-term-facility, geriatric care unit, size (usually measured as number of beds) interacted with hospital type, average length of stay for medicare pneumonia patients and occupancy rates.

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” What are report cards? Report cards constitute a public disclosure

of information on the performance of physicians or hospitals. Interesting web-page (report cards for several states):

http://www.consumerhealthratings.com/index.php?action=showSubCats&cat_id=301

Example for NY 2004 (hospitals and surgeons) for coronary artery bypass graft (CABG):http://www.health.state.ny.us/statistics/diseases/cardiovascular/docs/pci_2005-2007.pdf

Net effects of report cards are unclear: positive because they decrease asymmetries of information and may give

incentives to increase quality Possibly negative: give physicians and hospitals incentives to decline the

most difficult cases in order to avoid worsening their mortality score.

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” What may be the problems with report cards:

1. Mortality rates must be adjusted for patients’ risk factors (“risk adjustment”) otherwise providers treating more serious cases would fare worse.

2. But analysts can only adjust for characteristics in databases, providers always have better information capacity of selecting patients and case-mix unable of capturing it.

3. Even if risk adjustment was correct there may still be incentives to select patients, why? Suppose the matrix of mortality by hospital quality and patients’ risks is the following:

High Low

A (High) 5 1

B (Low) 10 2

5 1

Quality

Risks

hospitals

Patients’s severity

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards”

Patients’ risk ={0,1}

Mortality

1

10

5

21

Regression for the Low quality hospitals

Regression for the high quality hospital

If we could observe hospital quality we would estimate two different regressions. Key: The difference in outcomes (between High and Low quality hospitals) is higher for high risk patients! Low quality providers look more similar to High quality for less sick patients.

0

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” i.e. if we observe the quality of the hospital, we could run two regressions:

If we do not observe the quality of the hospital and we run a single regression

for High Quality Hospitals where R= 0,1 for patients' risk

for Low Quality Hospitals

5 1 10 2ˆ ˆwe observe that 4 8

1 0 1 0

H H H H

L L L L

H L

M c R

M c R

ˆ4 8

ând the exact value of c and depends on the mixture of patients.

If both hospitals have the same proportion of H and L risk patients

ˆthen 6 but if the high quality hospital has a hi

M c R

gher proportion

ôf high risks than 6 and closer to , because the identification

of this parameter comes from more observations of the high quality hospitalH


Patients’ risk ={0,1}

Mortality

1

10

5

2

1

With selection the constant (i.e. the mortality of the low risk patients, R=0) is identified by the low quality hospital and slope (the mortality of the high risk patients, R=1) by the high quality hospital. The residuals from both the low quality and the high quality hospital are now smaller and therefore the adjusted mortalities more similar among the two type of hospital.

Regression without selection (equal proportion of high and low risks in both hospitals); c=1.5; =6

Regression with selection (higher proportion of high risks at the best hospital) c=1.8; =4.2


But what does that imply? Suppose the high quality hospital has a higher proportion of high risks then = 4.2, and the adjusted mortalities:

H L Average

Mortality

Predicted Mortality

Performance (residuals)

A (H) 80% 20% 4.2 5.16 -0.96

B (L) 20% 80% 3.6 2.64** 0.96

|Diff. with patient Selection| 0.6 2.52 1.92

A (H) 50% 50% 3 4.5* -1.5 (below the mean)

B (L) 50% 50% 6 4.5 1.5

|Diff. without patient Selection|

3 0 3

*0.5×1.5+0.5×(1.5+6)

** 0.8*1.8+0.2*(1.8+4.2)

The Low quality fares better when it has a lower proportion of high risk patients


4. Finally, even if risk adjustment was correct (on average) but noisy and if providers were risk averse then again the providers may shun away high risks patients if the variance of outcomes is higher for the high risks.

mortalityRisk Adjusted

Mortality

EULow risk

EUHigh risk Variance for the high risks

Variance for the low risks


Mandatory CABG (coronary artery bypass graft) report cards were introduced in the beginning of the 90’s in NY and Pennsylvania.

They use Medicare data from 1987- 1994 and estimate the short-run effects of the introduction of report cards in New York and Pennsylvania.

In particular they are interested in the effects of report cards on: The matching of patients to providers – may improve if sicker patients have more

to gain from being treated at higher quality hospitals and report cards gives correct information about high quality hospitals.

The incidence and quantity of CABG surgeries – The incidence of surgeries may have shifted from sicker to healthier patients and the quantity of surgeries may have gone up or down. If sicker patients have more to gain from the surgeries then social welfare may decrease.

The incidence and quantity of substitute treatments – substitute treatments may have increased for sicker patients, however if hospitals avoid patients altogether then we would observe a decrease in these as well.

Health care expenditures and patients health outcomes -

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Methodology: they use a differences-in-differences approach where they

compare the trends before and after the report cards for the NY and Penn states with the trends for control states (where there was no introduction of report cards).

Findings: Report cards improve the matching of patients to hospitals + CABG surgery increased and changed the incidence from sicker to healthier

patients ± ? Higher health costs – Deterioration of health outcomes, specially amongst the most ill patients – Conclusion: report cards were welfare-reducing

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Identifying assumptions (in order to be able to conduct the differences-in-

differences approach): Assumption 1: The adoption of report cards must be exogenous i.e. uncorrelated

with state level unobservables in trends in costs, health outcomes, and treatments.

Assumption 2: AMI patients (acute myocardial infarction) patients are a relevant at-risk population for CABG surgery and its composition does not change by the introduction of report cards (means that people do not have more or less hearth attacks because of the introduction of report cards).

In December 1990 NY released report cards on raw and risk-adjusted hospitals’ mortality on patients receiving CABG surgery

In 1992 NY released surgeon specific mortality November 1992, Pennsylvania published hospital+surgeon report cards on

risk adjusted CABG mortality. NY and Penn are treatment states, the rest are control states


1st: Show evidence that report cards did not alter the distribution of AMI patients (that was assumption 2). Note that SEVERITY is measured by the expenditures in the year previous to admission.

Nationwide increase in intensity of treatment for people with heart problems, not different at treatment states, consistent with assumption 2.


2nd: Trends in the people that undergo CABG surgery look very different

Smaller expenditures of people undergoing CABG surgery in treatment states consistent with a shift towards healthier patients in these states.

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Empirical Model

They study cohorts of AMI patients – all those that suffered at least one heart attack, they may or may not go through CABG.

They study cohorts of CABG patients - who may or may not have had an AMI.

The idea here is that (at least in the short run) the CABG report cards did NOT affect the AMI population (this is an emergency event) but in contrast may affect the composition of those receiving CABG because it is an elective procedure.

They estimate two models Hospital is the unit of analysis Patient as the unit of analysis

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Empirical Model – in the hospital-level analysis, the effects of

report cards are measured by comparing the evolution of treatment states with the evolution of control states. Matching – check whether report cards led to greater homogeneity (in

health status) of patients within the hospital. Incidence – check whether the health status of patients going through

CABG changed due to report cards. If the health status has changed there may be two explanations: 1. report cards had to do with it 2. the underlying health of the population has changed – to dismiss this

possibility they do the same exercise with AMI patients (which by assumption are not affected in the short run by report cards).


Hospital Level analysis :

ln( ) A B Z L N

index for hospital

index for state

index for time = 1987,...1994

lst s t lst st st lsth g p q e

l

s

t

mean of the illness severity before treatment of Medicare CABG's patients

(they use hospital expenditures and days in the hospital in the year before admission)

A state fixed effects

B 8 a

lst

s

t

h

nnual dummies 1987,...1994

Z hospital characteristics (rural location, size, ownership categories, teaching status)

L =1 if the hospital is in NY in or after 1991 or in Penn in or after 1993, 0 olst

st

therwise

N number of hospital (its square and cube) in state s in year t (competition)

each observation is weighted by the number of CABG patients addmitted.st

p is the difference-in-difference estimate coefficient of the effect of report cards on the severity of CABG patients. If p<0 then report cards caused a shift in incidence of the surgeries to an average healthier patient.


A negative p could be caused by an effect of report cards on a better health status of all patients (in risk of CABG surgery). To discard this explanation they examine the effects of report cards on AMI patients (there should be no effect since these patients are not subject to selection). Re-estimate the model using the mean severity of AMI patients as the dependent variable. Compare the estimated p’s of both models.

Also, instead of using mean severity as dependent variable they use the coefficient of variation (s.d./ mean) before treatment within each hospital. Improved matching of patients would cause the coefficient of variation to decline in NY and Penn relative to other states, again a negative p.

Finally, with improved sorting we should observe more ill patients to be at higher quality hospitals. Since quality is not observed they use teaching status as a quality proxy and re-estimate the model with an interaction term ZTEACH

lst×Lst if the coefficient is positive it means report cards lead more sick patients to be treated at teaching hospitals.


3rd: hospital level regressions

Decline in illness severity => shift in incidence towards healthier patients

Better matching

Difficult to interpret because the increase in cv is also due to decrease mean severity

Sample with selection Sample without selection

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards”3rd: hospital level regressions (cont.) they also find that the more serious cases

go more to the teaching hospitals after the report cards.

Increase severity of patients at teaching hospitals: evidence of better matching

Constant severity at teaching hospitals


Patient Level Analysis – similar regressions:

To test for a quantity effect of report cards on CABG surgery:

A B Z L (2)

index for patient

s index for state

t index for time (1987,...1994)

1 if patient k has CABG surgery within 1 year after admission for AMI (heart attack)

0

kst s t kst st kst

kst

C g p e

k

C

otherwise

Z vector of patient characteristics (rural residency, gender, race, age, and interactions

between gender, age and race

L 1 if patient k's residence is in NY in or after 1991 or

kst

st

Penn state in or after 1993,

0 otherwise

A positive p means that report cards increase the probability of an AMI patient having a CABG surgery. To test for the effect on substitute treatments (PCTA and cath) replace C by these treatments and re-estimate (2).


Patient Level Analysis (cont.):

To assess the effects of report cards on sick versus healthier patients they include an interaction term:

A B Z L (3)

measures patient k's illness severity

if r 0 then report cards altered the incidence of CABG

if r<0 healthier patients have higher probability of C

kst s t kst st kst st kst kst

kst

C g p qw rL w e

w

ABG

if r>0 it is the sicker that have now higher probability

where C is the probability of CABG for example

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards”Patient level regressions: 1) increase in CABG quantity; 2) that increase was based

on healthier patients; 3) decrease in PCTA; 4) delay in treatment

(1)

(2)

(3)

The second part of table 4 has PTCA within one day of admission and cath

Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards”Patient Level Analysis (cont.):For health outcomes and health expenditures follow similar procedure.

If report cards increase the incidence of adverse health outcomes and increase costs decrease welfare

If report cards decrease the incidence of adverse health outcomes and decrease costs increase welfare

Otherwise they can compute the cost-effectiveness of report cards

1 if patient k from state s, at time t experienced an adverse health effect

(e.g. heart failure), 0 otherwise

ln total hospital expenditures of patient k in the year after admission w

kst

kst

O

y

ith AMI

reestimate (2) if 0 then report cards increase the incidence of adverse outcomes

and hospital expenditures.

p

(1)

(1) Increase in costs because now the average patient is more likely to undergo CABG; Also more costs to the severely ill although these were less likely to undergo treatment, which means they became even sicker and therefore eventually demanded more expensive treatment. Other results show health status declined with RC, specially among the sicker (2).

(2)


Patient level regressions (cont): 1. increase in expenditures for all patients (surprisingly since the more severe

were now less likely to receive CABG and less likely to receive PTCA now, the problem is that they will have more complications and that is costly too)

2. worse health outcomes one year after admission for sicker patients episodes of AMI and heart failure and slightly better outcomes for healthier patients (due to higher CABG procedures).

Overall they conclude that the welfare was reduced. They claim that report cards should focus on the population at risk, say the

AMI population and not only report information on those that actually received the procedure

They claim that sorting i.e. the apparent better matching of patients to providers is positive

The short run analysis cannot say anything about the long run impact on providers’ quality

Documents

Topics in Health Economics – class 3 Matilde P. Machado [email protected]