Quality and Variation in Medical Practice: Why are

Doctors so Different?

Mark W. Shen, M.D.November 19, 2010M k W Sh M D

Objectives

• List 3 examples of significant variation in pediatric practice

• Describe the relationship between variation and quality of a process

• Describe one method of improving quality of care by addressing variation

A Mad Lib

1. Pick a pediatric practitioner• Generalist, pulmonologist, rheumatologist,

hematologist, neonatologist, ID, GI, ENT…2. Pick a management scenario:

• ITP, post-op T&A, HSP, bronchiolitis, protein-losing enteropathy, post-op cardiac surgery, bacterial meningitis…

3. Pick a word pair: • Given & stop OR Not given & start

Your Mad LibYou are a pediatric [insert type of practitioner] and begin

your busy Monday by seeing a patient cared for by your partner over the weekend

The patient has been receiving care for [insert disease]and was [given/not given] steroids.

You completely disagree with this approach and [stop/start] the medication.

What a frustrating start to the day. Sometimes you wonder how 2 physicians could practice such different medicine.

As you leave the room, you notice a look of puzzlement on the family’s faces…

Encountering Variation:The 5 Stages of Grief

• Denial– Is that person board-certified?

• Anger– It’s my patient, I can do what I want

• Bargaining– Let me try to use the family to get my way (I’ll tell

them my side)• Depression

– I’m an accomplice in providing poor care• Acceptance

– Just do whatever the other MD wants

The History of (the study of) Medical

Variation

Int J Epidemiol 2008;37:9–19

Variation in Incidence of Tonsillectomy: J Alison Glover

“Puzzling as is the geographical distribution, the social distribution is yet more of an enigma.Tonsillectomy is at least three times as common in the well-to-do classes.”

Sci Am 1982;246:120-34

Tonsillectomy Variation: Back Across the Pond

• 1934, American Child Health Association• 1000 New York City School Children • 40% had not yet undergone tonsillectomy

– School physicians: 45% needed an operation• Of those not selected, another group of physicians

recommended that 46% receive tonsillectomy– Of the twice-rejected children, a third group of physicians

recommended operation in 44%

After 3 exams, only 65 children remained

The Beginnings of Modern Day Variations Research

John Wennberg’sHouse

Science 1973;182:1102-1108

Extreme Variation in Tonsillectomy Rates

Sci Am 1982;246:120-34

Probability of Having Surgery in 11 Vermont Hospitals

Int J Epidemiol 2008;37:26–29

Surgical Rates for the Most Populous Hospital Areas: Maine

Int J Epidemiol 2008;37:26–29

The Surgical Signature

NEJM 1982;307:1310-14

International Differences in Surgical Rates

Sci Am 1982;246:120-34

Proof of Preference-Sensitive Care

Preference-Sensitive Careaka:

Medical Variation:The Present

Terminology

• Unwarranted Variation:– Care that is not consistent with a patient’s preference

or related to their underlying illness• Preference-Sensitive Care

– No right rate (T&A)– Misuse

• Effective Care– Evidence-based care not provided– Underuse

Evidence for Underuse of Effective Care: Adults

Evidence for Underuse of Effective Care: Children

Terminology

• Unwarranted Variation:– Care that is not consistent with a patient’s preference

or related to their underlying illness• Preference-Sensitive Care (Misuse)

– No right rate• Effective Care (Underuse)

– Evidence-based care not provided• Supply-Sensitive Care (Overuse)

– Systems supply creates demand

http://www.dartmouthatlas.org/downloads/reports/Spending_Brief_022709.pdf

The Dartmouth Atlas: Medicare Spending Varies Dramatically

http://www.ahrq.gov/about/annualconf09/brownlee.htm

63%12%

25% 63%1212%12%12%12%21

25%

Preference Sensitive Care

Effective Care

Supply Sensitive Care

Unwarranted Variationin Medicare Spending

Source: John E. Wennberg and Dartmouth Atlas

www.nejm.org

A Trending Topic

www.pediatrics.org

Trending in Pediatrics

Examples of Data Sources in Pediatrics

• Databases– PHIS (Pediatric Health Information Systems)

• Propietary administrative database• Maintained by Child Health Corporation of America

(CHCA)– Business Alliance of 42 children’s hospitals

• Collaborative Networks– VIP (Value in Inpatient Pediatrics)

• Grassroots collaborative improvement network• Data: administrative and chart review

Pediatrics 2009;123;636-642

Osteomyelitis: Variability in Early Conversion to Oral Therapy

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Hospital

Con

vert

ed to

ora

l the

rapy

(%)

Pediatrics 2010;126;196-203

UTI in Infants: Variability in Length of IV Therapy

Circulation. 2010 Nov 8. [Epub ahead of print]

Corticosteroid Use After Congenital Heart Surgery

Source: VIP Network

Bronchodilator Doses Per Patient in Acute Bronchiolitis

Center

I See Variation

Isn’t Variety the Spice of Life?

Unwarranted Variation:The Losers

• Patients• Learners• System (everyone loses)

Preference-Sensitive Losers:The Patients

• Patients lose when not involved– Recent Dell Children’s patient comments:

• “Doctors, deliver a consistent message. We heard different plans from different doctors.”

• “Lack of communication between doctors”• “I was given conflicting info, on which I had to

make a judgment call.” I didn’t know who to talk to.• “Too many doctors involved”

Preference-Sensitive Losers:The Learners

“What do you want to do?”(everyone does things differently so just tell me

what you want to do)

“I don’t care”

Adverse Effects of Unmeasured Variation

Adverse Systems Effects of Unwarranted Variation

Medicare: Cost vs Quality The Value Equation

Quality• Value = ---------------------

Cost

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

$- $10,000 $20,000 $30,000 $40,000 $50,000 $60,000 $70,000 $80,000 $90,000

Average Annual Reimbursement per Beneficiary (Wage-Index Adjusted)

Aver

age

Qua

lity

of C

are

Scor

e

* Based on percent of beneficiaries with three conditions (diabetes, chronic obstructive pulmonary disease, and congestive heart failure) who had a doctor’s visit four weeks after hospitalization, a doctor’s visit every six months, annual cholesterol test, annual flu shot, annual eye exam, annual HbA1C test, and annual nephrology test. Source: G. Anderson and R. Herbert for The Commonwealth Fund, Medicare Standard Analytical File 5% 2001 data.

Best Practice CurveA

Greenville, NC

B C

DDNewark, NJ

CCMelrose Park, IL

BSaginaw, MI

Manhattan, NY

Orange County, CAEast Long Island, NY

t Practice CEaEE

Ft. Lauderdale, FL

Boston, MA

Variation in Annual Total Cost and Quality for Chronic Disease Beyond Just The Numbers

McAllen vs El Paso: Medicare Spending

http://www.whitehouse.gov/omb/blog/09/06/04/McAllenRedux/ http://www.newyorker.com/reporting/2009/06/01/090601fa_fact_gawande?currentPage=all

To Vary is Human

http://www.managedcaremag.com/archives/0311/0311.variation.html

CABG Rates in California

Public Perception:CABG Rates, Redding vs CA

Doctors’ Decisions and Impact on Medical Care

Lenses Under Which to Analyze Doctors’ Decisions

• Uncertainty & limits of the human brain– Medical decision-making, clinical problem-

solving• Different Disciplines

– Clinical, economic, sociological, psychological• Components

– Patient, physician, system

Health Affairs 1984;3:74-89

We Are Surrounded By Uncertainty

• Defining a Disease• Making a Diagnosis• Selecting a Procedure (e.g., test or

intervention)• Observing Outcomes• Assessing Preferences

Health Affairs 1984;3:74

Colorectal Experts:Consensus???

Question: What is the effect of screening annual fecal occult blood and flexible scope on colorectal cancer?

Doctors Decisions and the Cost of Medical Care. Michigan, 1986

Eisenberg: Determinants of Medical Decision-Making

• Physician as a self-fulfilling practitioner

• Physician as patient’s agent

• Physician as guarantor of social good

Doctors Decisions and the Cost of Medical Care. Michigan, 1986

Eisenberg: Determinants of Medical Decision-Making

• Physician as a self-fulfilling practitioner1. Desire for income2. Desire for a style of practice3. Personal characteristics4. Practice setting5. Standards established by clinical leadership

Medical Care 1981;19:297-309

Older Doctors Use Fewer Laboratory Tests

Doctors Decisions and the Cost of Medical Care. Michigan, 1986

Eisenberg: Determinants of Medical Decision-Making

• Physician as a self-fulfilling practitioner1. Desire for income2. Desire for a style of practice3. Personal characteristics4. Practice setting5. Standards established by clinical leadership

Health Affairs 1984;3:74

David Eddy on Practice Setting’s Impact on Variation

This tendency to follow the pack is the most important single explanation of regional variations in medical practice.

If uncertainty caused individual physicians to practice at random, or to follow their personal interpretations and values, without any attempts to match the actions of their neighbors, the variations in practice patterns would average out, and no significant differences would be observed at the regional level.

Differences between regions are observed because individual physicians tend to follow what is considered standard and accepted in the community.

Doctors Decisions and the Cost of Medical Care. Michigan, 1986

Eisenberg: Determinants of Medical Decision-Making

• Physician as patient’s agent1. Economic agent2. Clinical agent3. Patient demand4. Defensive medicine5. Patient characteristics6. Convenience

Doctors Decisions and the Cost of Medical Care. Michigan, 1986

Eisenberg: Determinants of Medical Decision-Making

• Physician as guarantor of social good– Duty to the patient vs steward of resources– Tension between “the prisoner’s dilemma”

and “the tragedy of the commons”• Classic scenario: end-of-life care (flat of the

curve medicine)

Does Genotype Determine Medical Decision-Making? Knowledge is Paralyzing

How Do We Improve?

Learn from Patients

• Paternalistic Craft-based Silos are archaic

Learn from Improvement Science

1. Measure the process2. Analyze the data3. Intervene: Control the process

(Research is a Slightly Different Order)

1. Measure the process2. Control everything3. Intervene4. Analyze the data

Learn from Improvement Science

1. Measure the process2. Analyze the data3. Intervene: Control the process

• Control unwarranted variation through standardization

• Continue to measure and analyze

Quality Improvement in Action: Control Chart

0%10%20%30%40%50%60%70%80%90%

100%

11/15

/2009

11/22

/2009

11/29

/2009

12/6/

2009

12/13

/2009

12/20

/2009

12/27

/2009

1/3/20

10

1/10/2

010

1/17/2

010

1/24/2

010

1/31/2

010

2/7/20

10

2/14/2

010

2/21/2

010

2/28/2

010

3/7/20

10

3/14/2

010

3/21/2

010

3/28/2

010

4/4/20

10

%

Step 1: Measure

Step 3: Intervene - Standardize

Step 2: Analyze

Learn fromHigh Performers

• Pediatric Oncology– Minimal unwarranted variation– All variation is measured & patient-level– Enormous success

• Pediatric cancer transformed from uniformly fatal disease in 1950s to 78% five-year survival for all types

– Better outcomes than adult groups for adolescents and young adults (AYA)

Blood 2008;112:1646-1654

Pediatric vs Adult Trials in AYA with ALL

• Reasons– Better compliance on

pediatric protocols– Better enrollment in

pediatric trials• Next Steps

– Enrolling adults in trials with pediatric protocols

If to vary is human,then only through

collaboration will we truly divine