Cost-effectiveness analysis of capitation vs. fee-for-service for Medicaid patients with severe

mental illness

Richard GrieveVisiting scholar UC Berkeley

London School of Hygiene and Tropical Medicine

Joint work-Acknowledgments

• Jasjeet Sekhon– Dept Political Science, UC Berkeley

• Tei-wei Hu, Joan Bloom– School of Public Health UC Berkeley

Content of talk• Applied study

– Context– Methods– Results– Discussion

• Key Methodological issue

Context to the study

• Worldwide concerns about cost of health care• How best to deploy scare resources to improve

population health?• Cost-effectiveness analysis (CEA) technique

for guiding resource use towards efficient allocation (Gold et al, 1997)

• Some governments routinely use CEA to set priorities (UK, Australia, Canada)

• Its use is controversial, and its uptake variable (Sheldon et al 2002)

Context to the study

• Lack of explicit use of CEA in US (Neuman et al 2006)

• Concerns about rationing of resources (Aaron et al 2005)

• Historically concern about methodological standards (Prosser et al 1996, Gold et al 1997)– How to value outcomes (QALYs)– How to represent uncertainty

• Published evaluations tend to report costs and outcomes separately, rather than present full CEA

Context: Case study• FFS vs capitation for Medicaid cases with severe

mental illness• Studies found capitation associated with lower

costs– e.g. Bloom et al 2002; Lurie et al 1992– Clinical outcomes similar (Cuffel et al 2002)

• Studies report costs and outcomes separately• Unclear whether capitation is more cost-effective?• Specific Aim of this study: to demonstrate use of

CEA for assessing relative cost-effectiveness • Illustrate by conducting CEA of capitation vs FFS

for Medicaid cases with severe mental illness

Methods: case study overview

see Bloom et al (2002)• Legislation passed in Colorado in 1995• Medicaid services for patients with mental illness• Counties divided into 3 groups

– Group 1:FFS– Group 2: Direct Capitation (DC)– Group 3: Managed care Behavioral Organizations

(MBHO)• Observational study focused on severe mental illness• Compared costs and outcomes across 3 groups• Before and after introduction of capitation

Methodsdefinition of ‘interventions’

• FFS: providers reimbursed retrospectively

• DC: Community mental health services (CMHC) contract with state to organize/provide services– Not for profit

• MBHO– Joint venture between for profit private firm

commissioning services– CMHCs provided some services

• State regulation/ audit• Contracts every 2 yrs

Methodsselection of groups

• In population– CMHCs in each county bid for capitation contract– State selected those bids perceived to be ready

• In Evaluation – those counties most comparable – A stratified random sample of cases – Data for analysis for 522 cases across 3 groups

Methodscost and outcome measurement

G roup 1: FFSP ost 2: costs and health status

G roup 1: FFSPost 1: costs and health status

G roup 1: FFSB aseline costs and health status

G roup 2: DCP ost 2: costs and health status

G roup 2: DCPost 1: costs and health status

G roup 2: DCB aseline costs and health status

G roup 3: M B HOP ost 2: costs and health status

G roup 3: M B HOPost 1: costs and health status

G roup 3: M B HOBaseline costs and health status

Methodscost and outcome measurement

• Costs and outcomes measured over 3 nine month periods

• Cost measurement– Medicaid costs claims data, shadow billing– Considered substitution of health services

• Outcome measurement– Short form 36 (SF-36); global functioning (GAF)

• Outcome valuation– Brazier et al (2002)

• QALY calculation: utility score* life years

MethodsAnalytical strategy

• Baseline differences in casemix, cost and health status

• Used non-parametric method to match cases• Genetic matching algorithm (Diamond and

Sekhon 2006, Sekhon 2006, Mebane and Sekhon 1998)

• Applied Multivariate matching across 3 groups using– Previous costs; baseline costs, baseline outcomes,

casemix

Analytical strategy

• Reported incremental QALYs and costs separately• Reported incremental cost-effectiveness

– Reported incremental net benefits (INB)• INB (A vs B)=λ(ΔE)-ΔTC

– λ-societal willingness to pay health gain – ΔEi and ΔTCi mean difference in effects and

costs• if INB>0 then ‘accept’ A in preference to B• Test whether conclusions vary according to λ

– Plot cost-effectiveness acceptability curves– Probability ‘intervention’ is cost-effective

Results: baseline imbalanceMean costs ($) FFS vs MBHO

before and after matching (9 month period)

FFS MBHO p value

(KS test)Before matching 4820 6822 0.04

After matching 4820 4581 0.42

For FFS n=151 before and after matching,

MBHO=195 before and n=151 after matching

KS test: Bootstrap Kolomogorov-Smirnov test

Results: Mean utility and mean QALYs

FFS DC MBHO

Pre 0.63 0.63 0.63

Post 1 0.64 0.62 0.64

Post 2 0.63 0.61 0.65

QALYs (18 months) 0.934 0.919 0.954

QALYS: FFS vs DC p=0.48; FFS vs MBHO p=0.30

Resultsutilization % any service over each 9 month

period

FFS DC MBHO

Pre 89.4 89.4 89.4

Post 1 88.8 83.4 77.6

Post 2 83.6 78.8 71.0

ResultsMean Cost ($) per user over each 9 month period

FFS DC MBHO

Pre 5374 5375 5095

Post 1 4856 7116 3837

Post 2 4777 9002 4714

ResultsMean Cost per case ($) over each 9 month period

FFS DC MBHO

Pre 4808 4805 4581

Post 1 4313 5938 2979

Post 2 3991 7094 3349

costs: FFS vs DC p=0.06; FFS vs MBHO p=0.32

‘conclusions’ from cost-consequence

• DC significant increase in cost vs FFS

• MBHO non significant decrease vs FFS

• No significant outcomes differences

• Can we make use of evidence?

• CEA

Cost-effectiveness analysis (over 18 months)means (95% CI)*

DC-FFS MBHO-FFS

Incremental cost 4729(-70 to 9596) -1976(-5929 to 1808)

Incremental QALY

-0.016(-0.065 to 0.027) 0.020(-0.015 to 0.059)

Incremental net benefit

-5497(-10591 to -200) 2959(-1203 to 7250)

λ =$50,000 per QALY gained; * bootstrapped bias corrected version

Role of cost-effectiveness analysis

DC vs FFS

MBHO vs FFS

Cost-effectiveness acceptability curves

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 25000 50000 75000 100000 125000 150000 175000 200000

value of ceiling ratio (Rc)

prob

abili

ty c

ost-

effe

ctiv

e

MBHO vs DCMBHO vs FFSDC vs FFS

p=0.92

Preliminary conclusions from CEA• Cost effectiveness analysis useful in extending

traditional cost-consequence approach• Allows for potential tradeoffs between costs and

outcomes• In this case study/context CEA found that:

– MBHO model more cost-effective than FFS– DC model less cost-effective than FFS (or MBHO)

• Observational study but results based on appropriate methods adjusting imbalance

• Sensitivity analysis applied 2 part model to adjust for any outstanding differences: findings unchanged

Why the difference?• MBHO model- for profit stronger incentive

to cost minimise• Interviews (Bloom et al 2000) suggested MBHO

emphasised maintaining access but reducing costs per user

• DC areas had less targeted utilisation review• Different capitation models targeted different

patient groups

CEACS MBHO vs FFS; by diagnosis

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 25000 50000 75000 100000 125000 150000 175000 200000

value of ceiling ratio (Rc)

pro

bab

ilit

y c

ost-

eff

ecti

ve

MBHO vs FFS-schizophrenia

MBHO vs FFS- bipolar

CEACS DC vs FFS; by diagnosis

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 25000 50000 75000 100000 125000 150000 175000 200000

value of ceiling ratio (Rc)

pro

bab

ilit

y c

ost-

eff

ecti

ve

DC vs FFS-bipolar

DC vs FFS schizophrenia

Conclusions in context of other findings

• Caveats– small nos, Short followup– selection centres– Only sub sample severe mental illness– limited to morbidity costs

• Other analyses found capitation associated with cost reductions without selection (Coffman et al 2006, wallace et al 2006)

• Addition to literature suggesting form of capitation matters

Methodological extensionsuse of appropriate methods correcting for

imbalance• Lack of use of appropriate matching methods in

health services research• Deeks et al (2003) highly critical of current methods

of bias adjustment in observational studies• Seriously limits their use in policy making• Economic evaluations often dependant on data from

observational studies• Rely on ‘traditional’ methods of casemix adjustment• Recent interest in use of propensity score methods in

cost-effectiveness analysis (Mitra et al 2005) • More work in other areas e.g. labour economics

Methods to adjust for baseline differences

• Genetic matching• Improves on alternative methods such as multiple

linear regression and propensity scores• More efficient even when propensity score is known• When propensity score is unknown• Genetic matching minimizes bias even where

– distribution of baseline measures are skewed,– covariates have non linear relationship with

outcomes

A more general solutionGenetic matching (GM) algorithm

Diamond and Sekhon (2006)

• Uses search algorithm (Mebane and Sekhon 1998) • On basis of stringent non-parametric tests of balance

searches for ‘best’ match between treatment and controls across baseline covariates

• Previous work demonstrated that when applied to observational data can replicate the results of RCTs

• Software now available- Sekhon 2006

Baseline cost imbalance for FFS vs MBHO comparison:pre and post matching

FFS MBHO p value

(KS test)

Before matching

Mean costs

Max costs

SD

4820

75000

10111

6822

95000

10552

0.04

After matching

Mean costs

Max costs

SD

4820

75000

10111

4558

95550

10427

0.42

Comparison of method for adjusting for baseline imbalances• Compare cost-effectiveness estimates

– from unmatched data with parametric model adjusts using mean differences (1)

– from unmatched data with parametric model adjusts using linear adjustment (2)

– From matched data with parametric model adjusts using linear adjustment (3)

• Parametric model uses 2 stage and log transform

CEACs with different approaches to adjusting for imbalance DC vs FFS

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 25000 50000 75000 100000 125000 150000 175000 200000

value of ceiling ratio (Rc)

prob

abili

ty c

ost-

effe

ctiv

e

No matching parametric model mean adjustment

CEACs with different approaches to adjusting for imbalance DC vs FFS

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 25000 50000 75000 100000 125000 150000 175000 200000

value of ceiling ratio (Rc)

prob

abili

ty c

ost-

effe

ctiv

e

No matching parametric model mean adjustment

No matching linear parametric model

CEACs with different approaches to adjusting for imbalance DC vs FFS

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 25000 50000 75000 100000 125000 150000 175000 200000

value of ceiling ratio (Rc)

prob

abili

ty c

ost-

effe

ctiv

e

No matching parametric model mean adjustment

No matching linear parametric model

matching; linear parametric model

Preliminary conclusion for methodological section

• unbiased estimates of cost-effectiveness important• Choice of method can make a difference• Inappropriate method of adjustment overstated

probability intervention was cost-effective• Genetic matching preferable does not rely on

assumptions routinely violated • Skewed cost data, non linear relationships• Method works well even for this smallish case study

because of baseline data on costs, outcomes and casemix

• Further applications are required