Next Steps in Measuring Next Steps in Measuring Clinical Quality Clinical Quality

Joe V. Selby, MDJoe V. Selby, MD

Division of ResearchDivision of Research

Kaiser Permanente Northern CaliforniaKaiser Permanente Northern California

Differences in Clinical Quality – Differences in Clinical Quality –

Diabetes CareDiabetes Care

Plan A Plan B

Retinal Screening (P) 41.4 45.3

Hb A1c Testing (P) 72.7 75.1

Hb A1c Control (O) 61.9 55.2

Monitoring Nephropathy (P) 28.5 36.1

LDL-C Testing (P) 60.7 69.1

LDL-C Control (O) 29.1 36.7

*HEDIS Health Plan Summary Data

What We KnowWhat We Know WhatWhat We MeasureWe Measure

The Chasm in Clinical Quality AssessmentThe Chasm in Clinical Quality Assessment

Quantitative effects of many process measures and of differences in outcomes on survival and on non-fatal complications in populations – from clinical trials

Processes of care not known to be related to outcomes or effectiveness

Semi-quantitative outcomes (Hb A1c >9.5%, LDL-C < 100) that hide more effectiveness differences than they reveal

Population Rates for Simple Process Population Rates for Simple Process Measures do NOT Consistently Reflect Measures do NOT Consistently Reflect

Clinical Benefit in those PopulationsClinical Benefit in those Populations

Point #1Point #1

PacifiCare Texas

Kaiser Permanente No. California

Pacific Health Research Institute

U. Michigan

Indiana U.

UCLA

UMDNJ

CDC

Centers for Disease Control - Sponsor and Data Coordinating Center

TTranslating ranslating RResearch esearch IInto nto AAction for ction for DDiabetesiabetes

A multi-center cohort study of diabetes in managed care settingsA multi-center cohort study of diabetes in managed care settings

10 health plans 10 health plans (n=500 to 2000(n=500 to 2000per plan)per plan)

67 physician67 physiciangroups with groups with > 50 > 50 membersmembersin sampling in sampling frameframe

The TRIAD Sampling SchemeThe TRIAD Sampling Scheme

(Sampling scheme: Aimed for equal numbers from each physician group within health plan, so from 50 - 1500 per physician group)

TRIAD Data (2000-2001)TRIAD Data (2000-2001)

Patient Surveys (telephone or mailed) – 11,928 respondents

Chart Reviews – 8,757 patients

Medical Director Surveys – health plan and provider group directors

Four Measures of Disease Management Four Measures of Disease Management Intensity – from Health Plan and Provider Intensity – from Health Plan and Provider

Group Director SurveysGroup Director Surveys

Use of diabetes registries

Use of clinician reminders

Performance feedback to physicians

Diabetes care management:Guideline use

Patient reminders,

Patient education

Use of care/case managers

Provider Group Performance Difference (%) Provider Group Performance Difference (%) (80(80thth – 20 – 20thth Percentile of Dis Mgmt Intensity) Percentile of Dis Mgmt Intensity)

PROCESS MEASURESPROCESS MEASURES

Care Management

Performance Feedback

Diabetes Registry

MD Reminders

Hb A1c Test 11 0.001 9 0.0001 9 0.01 4 0.07

LDL-C Test 13 0.0001 8 0.001 11 0.01 2 0.59

Retinal Exam 7 0.01 8 0.001 4 0.13 7 0.001

Urine Albumin

16 0.0001 11 0.0001 13 0.01 10 0.01

Foot Exam 8 0.01 6 0.01 3 0.45 5 0.05

Aspirin Advised

0 0.99 1 0.74 3 0.30 3 0.38

adjusted for patient age, sex, race, education/income, diabetes treatment andduration, comorbidities, SF-12 (PCS), health plan disease mgmt intensity

Provider Group Performance DifferencesProvider Group Performance Differences (80 (80thth – 20 – 20thth Percentile of Dis Mgmt Intensity) Percentile of Dis Mgmt Intensity)

INTERMEDIATE OUTCOMES INTERMEDIATE OUTCOMES

Care Management

Performance Feedback

Diabetes Registry

MD Reminders

Hb A1c (%) 0.1 0.71 -0.1 0.74 -0.1 0.55 0 0.74

Syst. Blood Pressure (mmHg)

2 0.01 1 0.22 3 0.01 1 0.22

LDL-cholesterol (mg/dL)

2 0.06 2 0.70 2 0.46 0 0.70

adjusted for patient age, sex, race, education/income, diabetes treatment and duration, comorbidities, health plan disease mgmt intensity

Moreover,

Provider Group intensity of disease management also unrelated to the appropriateness* of treatment for each condition

Provider Group Quality Scores based on process measures were unrelated to provider group levels of control of blood pressure, LDL-C or Hb A1c

*Proportion in control or on appropriately aggressive pharmacotherapy

Point #2

Even if we measure evidence-based Even if we measure evidence-based processes or outcomes, the potpourri of processes or outcomes, the potpourri of indicators within and across diseases indicators within and across diseases

don’t readily yield a measure of overall don’t readily yield a measure of overall clinical benefitclinical benefit

Differences in Clinical Quality Differences in Clinical Quality (hypothetical) based on evidence-based (hypothetical) based on evidence-based

processes/ outcomesprocesses/ outcomes

Plan A Plan B

Patients Using Aspirin (%) 41 54

Mean Hb A1c (%) 8.1 7.5

Mean LDL-C (mg/dL) 106 131

Mean SBP (mmHg) 141 136

Flu Shot Past 12 mos (%) 67 54

How Do We Quantify the Net Benefit?How Do We Quantify the Net Benefit?

Each of these differences represents a predictable change in expected survival and complications (i.e., each measures a clinical benefit )

But practical questions remain:

Which is more important, the difference in Hb A1c levels or the difference in BP control?

Should plans, providers work to improve multiple measures modestly, or drive one indicator toward the optimal for all patients?

We need a composite, quantitative measure of net clinical benefit that can be compared across plans, provider groups, systems.

Quality-adjusted life-year

The QALY

A common metric for measuring clinical quality (both survival and quality of life)

Across interventions (using aspirin, BP lowering)

Across perspectives (patient, provider, purchaser)

Across diseases (diabetes, CHF, CAD, asthma)

Across activities (e.g., chronic disease care, prevention)

Where Do QALY’s Come From?Where Do QALY’s Come From?

Creating a Quantitative Metric for Creating a Quantitative Metric for DiabetesDiabetes

NaturalHistoryModel

SystolicBlood

Pressure

HemoglobinA1C

AspirinUse

LDL-Cholesterol

Expected Survival &

Complicatons

Adjusted Life-expectancy

RiskAdjusters

Potential Advantages of ModelPotential Advantages of Model

Expresses quality in familiar metric – life expectancy

Requires clinical trial evidence clearly evidence-based

Allows exploration to explain differences, which emphasizes population importance of various indicators

Potential Disadvantages/Questions Potential Disadvantages/Questions

Will require extensive explanation and transparency of the model to gain acceptance

New evidence will have to be incorporated over time, potentially altering metrics across years

Because it takes a population or public health perspective, will not capture quality of care well for rare conditions (because prevalence too small)

Questions of whether and how to adjust for case-mix differences between population will have to be addressed