Updated validation of AHRQ Prevention Quality Indicators in the USA

Patrick S. Romano, MD MPH

UC Davis Center for Healthcare Policy and Research

Organization for Economic Cooperation and Development

October 22, 2009

PQIs: Potentially Avoidable Hospitalizations

Admissions for diagnoses that may have been prevented or ameliorated with access to high-quality outpatient care

Two independently developed measure sets described in the 1990s literature– John Billings

– Joel Weissman

Strong independent negative correlations between self-rated access and avoidable hospitalization

Correlations between avoidable hospitalization and:– household income at zip code level (neg)

– uninsured or Medicaid enrolled (pos)

– maternal education (neg)

– Primary care physician to population ratio (neg)

– Weaker associations for Medicare populations

Current uses of the PQIs in the USA:Four examples

National Healthcare Quality Report, Commonwealth Fund, California:

Public health agencies tracking and comparing health system performance across counties, states

Wisconsin Medicaid program:

Comparing performance of managed care plans for low-income persons

Dallas-Fort Worth Hospital Council:

Exploring how community health affects hospitals

General Motors:

Estimating potential “return on investment” with improving primary care access in communities with GM employees.

Measures recommended for state Medicaid programs by the Foundation for Accountability

Ambulatory care sensitive conditions (“potentially avoidable hospitalizations”) Angina

Adult asthma/pediatric asthma

Chronic obstructive pulmonary disease

Congestive heart failure

Diabetes (short-term and long-term complications, uncontrolled)

Lower extremity amputation with diabetes

Hypertension

Other potentially avoidable conditions Perforated appendix

Low birth weight

Evaluating Medicaid managed care programs in Wisconsin

% ICare Enrollees with CHF Hospitalized for CHF

20.8

15.1

0

5

10

15

20

25

1998 2000

Pe

rce

nt

% ICare Enrollees with Asthma Hospitalized for

Asthma: 1998 & 2000

3.9

2.8

0

2

4

6

8

10

12

1998 2000

Pe

rce

nt

% ICare Enrollees with COPD Hospitalized for COPD

5.84.7

0

2

4

6

8

10

12

1998 2000

Pe

rce

nt

% ICare Enrollees with Diabetes Hospitalized for

Diabetes: 1998 & 2000

2.9

2.5

0

0.5

1

1.5

2

2.5

3

3.5

1998 2000

Perc

en

t

Comparison of total E.R. visits and ambulatory care sensitive condition (ACSC) E.R.

visits for I-Care and matched FFS recipients, 1999

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

I-Care (n=1722) Comparable FFS (n=1722)

No ER visit

ER, not for ACSC

ER for ACSC

Reducing ED visits for PQIs in Medicaid managed care

Community Health Assessments using PQIs to understand role of hospitals

Risk Adjusted Rates per

100,000 PopulationNamed counties without shading have a Risk

Adjusted rate of zero.1 to 117.0

117.1 to 283.0

283.3 to 399.2

399.3 to 565.2

> 565.2

DI Hospitals

Congestive Heart Failure Admission Rate

Congestive heart failure (CHF) can be

controlled in an outpatient setting for the most

part; however, the disease is a chronic

progressive disorder for which some

hospitalizations are appropriate.

AHRQ Prevention Quality Indicators

Congestive Heart Failure Admission Rate - 2003

+ = County’s RA rate significantly lower than State RA rate

- = County’s RA rate significantly higher

o = No statistical difference

Texas Hospital Inpatient Discharge Public Use Date File, FY2002. Texas

Health Care Information Council, Austin, Texas. December, 2003.

© 2004, Dallas-Fort Worth Hospital Council - Data Initiative For Hospital Internal Use Only – April, 2005

2003 (PQI 08) Rates per 100,000 Cases

County

Numerator

(Outcome)

Denominator

(Population) Observed

Risk

Adjusted

Confidence Interval

(95%)

Stat.

Sig.

State of Texas 66,822 15,882,253 420.7 504.5

BOSQUE 21 13,486 155.7 0.0 ( 0.0, 0.0 )

CAMP 80 8,567 933.8 860.3 ( 664.7, 1055.9 ) -

COLLIN 911 429,184 212.3 410.7 ( 391.6, 429.8 ) +

COMANCHE 117 10,233 1,143.3 954.4 ( 766.0, 1142.8 ) -

COOKE 60 28,036 214.0 142.7 ( 98.5, 186.9 ) +

DALLAS 6,412 1,631,345 393.0 527.8 ( 516.7, 538.9 ) -

DELTA 32 4,172 767.1 622.7 ( 384.0, 861.4 ) o

DENTON 876 369,935 236.8 456.8 ( 435.1, 478.5 ) +

EASTLAND 26 14,031 185.3 0.0 ( 0.0, 0.0 )

ELLIS 367 88,785 413.4 518.2 ( 471.0, 565.4 ) o

ERATH 97 24,993 388.1 376.7 ( 300.8, 452.6 ) +

FANNIN 171 25,024 683.3 610.2 ( 513.7, 706.7 ) -

FRANKLIN 48 7,616 630.3 505.7 ( 346.4, 665.0 ) o

GRAYSON 568 86,204 658.9 612.4 ( 560.3, 664.5 ) -

HAMILTON 19 6,241 304.4 23.7 ( 0.0, 61.9 ) +

HENDERSON 386 58,796 656.5 560.5 ( 500.2, 620.8 ) o

HILL 220 25,615 858.9 776.5 ( 669.0, 884.0 ) -

HOOD 149 34,883 427.1 330.1 ( 269.9, 390.3 ) +

HOPKINS 37 24,315 152.2 84.1 ( 47.7, 120.5 ) +

HUNT 362 59,959 603.7 630.8 ( 567.4, 694.2 ) -

JACK 12 6,916 173.5 135.9 ( 49.1, 222.7 ) +

JOHNSON 564 100,860 559.2 664.9 ( 614.7, 715.1 ) -

KAUFMAN 390 59,401 656.6 750.8 ( 681.4, 820.2 ) -

LAMAR 260 36,763 707.2 619.4 ( 539.2, 699.6 ) -

MONTAGUE 21 14,900 140.9 0.0 ( 0.0, 0.0 )

MORRIS 64 10,014 639.1 507.3 ( 368.2, 646.4 ) o

NAVARRO 216 34,415 627.6 596.6 ( 515.2, 678.0 ) -

PALO PINTO 22 20,368 108.0 2.9 ( 0.0, 10.3 ) +

PARKER 210 72,541 289.5 361.6 ( 317.9, 405.3 ) +

RAINS 38 8,472 448.5 396.8 ( 262.9, 530.7 ) o

ROCKWALL 106 39,433 268.8 394.3 ( 332.4, 456.2 ) +

SOMERVELL 9 5,416 166.2 139.0 ( 39.8, 238.2 ) +

STEPHENS 6 7,147 84.0 0.0 ( 0.0, 0.0 )

TARRANT 3,688 1,118,382 329.8 456.1 ( 443.6, 468.6 ) +

TITUS 40 19,796 202.1 198.5 ( 136.5, 260.5 ) +

VAN ZANDT 209 38,329 545.3 458.6 ( 391.0, 526.2 ) o

WISE 58 39,967 145.1 209.0 ( 164.2, 253.8 ) +

WOOD 235 30,845 761.9 599.1 ( 513.0, 685.2 ) -

YOUNG 15 13,604 110.3 0.0 ( 0.0, 0.0 )

General Motors: Impact of PQI admissions on employer-financed health care

QI Name

Potential cost savings if

number of admissions

were reduced by specified

percentage

County name (all counties in MI listed),

average cost of admission for QI

specified, total number of cases, and

total cost

General Motors mapped PQI data

Name of Indicator

and Data Year in

Map Title

Data quintiles.

Green is the lowest

20% or the lowest

rates. Red is the

highest 20% or the

highest rates.

Symbol indicating

number of GM covered

beneficiaries, number

below is average in the

group.

Counties with

high indicator

rates and

higher number

of beneficiaries

Using GM-mapped data

Integrate action plans with other Community Initiatives projects

Pay for Performance for providers in specific counties to reduce admission rates

Coordination with other Community Stakeholders to achieve desired improvement

Funding to implement projects at a community level

Focus on indicators with highest potential cost savings

Potential uses of PQIs

QI

Comparative

Reporting

Pay for

Performance

Area X

Payor X X

Provider X X X

1 We initially assessed the internal quality improvement application for large provider groups. Following our initial rating

period, panelists expressed interest in applying select indicators to the long term care setting and these applications were

added to our panel questionnaire.

Current application

Extended applications

Revalidation methods

Clinical Panel review using new hybrid Delphi/Nominal Group technique

Two groups: Core and Specialist

– Core assesses all; Specialist only those applicable to their specialty

Three indicator groups: Acute, Chronic, Diabetes

Two panels:

– Delphi

– Nominal Group

Delphi vs. Nominal

Delphi group

– Advantages: Larger,

hence better reliability,

more points of view,

less chance for one

panelist to pull the group

– Disadvantage: Less

communication and

cross-pollination across

panelists, less ability to

discuss and refine

details of

indicators/evaluation

Nominal group

– Advantages: Can

discuss details, facilitate

sharing of ideas

– Disadvantages: Limited

in size and therefore in

representation, one

strong panelist can

flavor group and

therefore poorer

reliability

Composition of panels

Characteristic Delphi Group (n = 42)

Nominal Group (n = 23)

Gender

Male 62.8 73.9

Female 37.2 26.1

Urban/Rural1

Urban 32.6 30.4

Suburban 14.0 13.0

Rural 7.0 8.7

Multiple/All areas served 16.3 30.4

Academic Affiliation1

Academic practice 27.9 47.8

Non-academic practice 34.9 30.4

Any academic affiliation 69.8 87.0

Underserved population

in practice1 46.5 69.6

Funding1

Public 27.9 34.8

Private and/or Non-profit 20.9 39.1

Multiple sources 7.0 0

Specialties representedSpecialty Delphi Panel Nominal

Panel

Internal Medicine 5 3

Family Medicine 4 1

Geriatric Medicine 2 2

Public Health Physician 4 0

Emergency Medicine 3 2

General Nurse Practitioner 2 1

Endocrinology 4 2

Vascular Surgery 2 1

Diabetes Outpatient Management

1 1

Nephrology 0 1

Cardiology 4 3

Pulmonology 3 2

Asthma Specialist 1 0

Pulmonary Rehabilitation 1 0

Infectious Disease 2 2

General Surgery 3 2

Urology 1 0

Delphi Dephi rating

Results: initial rating

Delphi comments

Nominal comment

Nominal Nominal rating

Results: Initial rating

1st round results to panelists prior to call

Diabetes call

Acute call

Chronic call

Nominal panel re-rates

Call summaries to panels

Final ratings

Delphi panel re-rates

Panel Process: Exchange of Information

Quality Improvement Applications

Indicator Provider

COPD and Asthma (40 yrs +) ▲▲ +

Asthma ( < 39 yrs) ▲▲▲▲

Hypertension ▲▲ +

Angina ▲▲

CHF ▲▲▲▲

Perforated Appendix ▲+

Diabetes Short Term Complications ▲▲▲▲

Diabetes Long-Term Complications ▲▲+

Lower Extremity Amputation ▲▲+

Bacterial Pneumonia ▲▲

UTI ▲▲

Dehydration ▲+

▲ Major Concern

Regarding Use ,

▲▲Some Concern

▲▲▲General Support

▲▲▲▲Full Support

+ Either Delphi or

Nominal Panel

reported higher level of

support for measure

than shown

Comparative Reporting Applications

Indicator Area Provider

COPD ▲▲ ▲▲+

Asthma ( < 39 yrs) ▲▲+ ▲▲+

Hypertension ▲▲+ ▲▲

Angina ▲▲ ▲

CHF ▲▲+ ▲▲▲▲

Perforated Appendix ▲+ ▲+

Diabetes Short Term ▲▲ ▲▲+

Diabetes Long-Term ▲▲+ ▲▲

LE Amputation ▲▲▲▲ ▲▲

Bacterial Pneumonia ▲▲ ▲▲

UTI ▲▲ ▲▲

Dehydration ▲▲ ▲

▲ Major Concern

Regarding Use ,

▲▲Some Concern

▲▲▲General Support

▲▲▲▲Full,Support

+ Either Delphi or

Nominal Panel

reported higher level of

support for measure

than shown

Potential interventions to reduce hospitalizations

Acute Chronic

Area Access to primary

care/urgent care

Access to care

Lifestyle modifications

Payor Coverage of medications

Coverage of auxiliary

health services (e.g. at

home nursing)

Access to primary

care/urgent care

Coverage of medications

Coverage of comprehensive care

programs

Coverage of auxiliary health

services (e.g. at home nursing)

Disease management programs

Lifestyle modification incentives

Provider Quality nursing triage

Patient education

Accurate/rapid diagnosis

and treatment

Appointment availability

Outpatient treatment of

complications

Education, disease management

Lifestyle medication interventions

Comprehensive care programs,

care coordination, auxiliary health

services

Potential improvements

Defining the numerator– One admission per patient per year?

– Using related principal diagnoses with target secondary diagnoses (i.e., pneumonia with COPD)

– Consider excluding first hospitalization before chronic condition diagnosed

Defining the denominator– Identifying patients with chronic diseases (mulitple

diagnoses from outpatient claims, population survey rates, pharmaceutical data

– Requiring minimum tenure with payor or provider or minimum duration of residence in an area

Risk adjustment

Demographics– Age and gender highly rated as important

– Race/ethnicity depending on indicator (i.e., diabetes)

Disease severity– Historical vs. current data

Comorbidity

Lifestyle associated risk and compliance– Smoking, obesity

– Pharmacy records

Socioeconomic status– May mask true disparities in access to care

– Panel felt benefits of inclusion outweighed problems

Policy implications

Ensuring true quality improvement

– Shifting risk (adverse selection)

– Manipulating coding

Cost/burden of data collection to support better risk adjustment

Different perspectives of different stakeholders

Does avoiding hospitalization really reflect the best quality and value?

Acknowledgments

Project team:

Sheryl Davies, MA (Stanford)

Kathryn McDonald, MM (Stanford)

Eric Schmidt, BA (Stanford)

Ellen Schultz, MS (Stanford)

Olga Saynina, MS (Stanford)

Jeffrey Geppert JD (Battelle)

Patrick Romano, MS, MD (UC Davis)

This project was funded by a contract from the Agency for Healthcare Research and Quality (#290-04-0020)