Risk-Adjustment Methodologies and

Applications in the VA

Amy K. Rosen, Ph.D.

Director, Risk Assessment and Patient Safety

Center for Health Quality, Outcomes and Economic Research

A VA HSR&D Center of Excellence (Bedford, MA)

&

Professor, Health Policy and Management

Boston University School of Public Health

Purpose of Talk

Introduce concept of risk adjustment Describe two well-known diagnosis-based

risk-adjustment tools: Diagnostic Cost Groups (DCGs) and Adjusted Clinical Groups (ACGs)

Discuss applications in the VA Development of psychiatric risk-adjustment

measure for the VA

Why is Risk Adjustment Necessary?

Health status of population can vary significantly

Goal is to provide equitable compensation and make appropriate comparisons

Allocations based on efficiency and quality, not selection

Risk Adjustment

The process by which the health status of a population is taken into account when

evaluating patterns or outcomes of care or setting capitation rates

Applications of Risk-Adjustment Measures

Payment management (Prospective) Provider profiling (Concurrent) Disease/Case management (Prospective) Quality and outcomes (Concurrent) Resource allocation (Prospective)

Types of Risk-Adjustment Measures

Evaluation Criteria

Predictive validity Subgroup fit Administrative feasibility Incentives for efficiency Resistance to gaming

Diagnosis-Based Risk Adjustment

Increasing use of risk adjustment based on diagnosis codes from administrative data

Persisting concerns with reliability and validity of diagnosis codes Outpatient data - not much known about reliability Variability in coding practices across providers and facilities Upcoding and diagnostic creep Tentative coding

Risk-adjustment measures minimize some of these (e.g., excluding ill-defined codes)

What are diagnosis-based risk-adjustment measures?

Diagnosis-based measures use demographics/diagnostic information from claims/encounters to:

Classify patients into clinically homogeneous groupsbased on expected need for resource utilization Create clinical profile Identify clinical needs Evaluate clinical management programs

Predict relative resource use Predict expenditures Same year as diagnosis (Concurrent Models) Subsequent year to diagnoses (Prospective Models)

Data Requirements

Defined population of patients Claims/encounter data available for all

members of the population (12 months) Unique patient identifiers (i.e., social security

numbers) Age and gender ICD-9-CM diagnosis codes from face-to-face

clinical encountersOptional: payer, DOD, sociodemographics

Family of Diagnostic Cost Group (DCG) Models

Type of population Commercial, Medicare, Medicaid

Clinical data available All-encounter, inpatient only, age/sex and

pharmacy Year of prediction

Concurrent or prospective Recognizes cumulative effect of multiple

conditions in predicting costs

Relative Risk Scores

Relative Risk Scores

DCGCategories

DCGCategories

ICD-9-CMCodes

ClinicalGroups

ClinicalGroups

ResourceUse

Predictor

RelativeHealthStatus

ClinicalProfilesClinicalProfiles

DCG Model Overview

Hierarchies imposed for predictions

Condition Categories (CCs)(n= 184)

Aggregated Condition Categories (ACCs)(n= 30)

ICD-9-CM codes(n = 15,000+)

DxGroups(n =781)

PIP-DCGClinical Classification

PIP=Principal InpatientBased on Inpatient Dx only

Single-condition model

Relative risk

score

DCG Clinical Classifications

410.91 AMI of unspecified site, initial episode of care

491.2 obstructive chronic bronchitis

586 renal failure nos

585 chronic renal failure

72.01 AMI, initial episode of care

96.01 emphysema/chronic bronchitis

106.04 renal failure,unspecified

106.03 chronic renal failure

50 AMI

64 COPD

78 Renal Failure

ICD-9-CM DxGroup CC

518.1 Interstitial emphysema

Clinical Vignette:59 year old woman AMI, COPD, renal insufficiency (Release 5.0)

Hierarchical Condition Categories (HCCs)

31 Hierarchies are imposed on the CCs to produce HCCs. The clinical hierarchies: Identify the most costly manifestation of each

distinct disease Decrease the model’s sensitivity to coding

idiosyncrasies Examples: Diabetes, Cancer, Heart, Mental

Health

Metastatic Cancer

High Cost Cancer

Moderate Cost Cancer

Low Cost Cancer

Carcinoma in Situ

Uncertain Neoplasm

Skin Cancer Except Melanoma

Benign Neoplasm

Cancer Hierarchy (Release 5.0)

Current and Prospective Predictions

How Do All-Encounter DCG Models Predict?

Linear additive formulas (OLS regressions) combine predictions based on HCCs and age/sex cells subject to: Hierarchical restrictions Exclusions of CCs in prospective models

that are not useful for predicting costs (minor injuries) vague and discretionary CCs based on concerns

about gaming in payment models

DCG Predictions:Relative Risk Score (RRS)

Illustrate annual resource use as determined from DCG cost weights

RRS calculated by adding cost weights of an individual’s HCCs and dividing by benchmark (i.e., Medicare) mean dollar amount

RRS normalized so that population mean = 1.00

Prospective Relative Risk Score Calculated

0.45 54 year old maleHCC

5.71 Diabetes with renal manifestation0.95 Type 1 diabetes1.84 Congestive heart failure0.90 Acute myocardial infarction0.89 Vascular disease with complication0 Vascular disease18.09 Dialysis status … …..0.46 Diabetes & congestive heart failure

43.30 Relative Risk Score

Health Score for Year 2

Which Providers are “More Efficient”?

Adjusted Clinical Groups (ACGs)

Clustering of morbidity is a better predictor of health care resource use than presence of specific diseases

Level of resources necessary for delivering health care services is correlated with the morbidity of that population

15,000 ICD-9-CM Diagnosis Codes

Step. 1: Adjusted Diagnosis Groups

Step 2: Collapsed ADGs

Step 3: CADGs combined into Major Adjusted Categories (MACs)

Step 4: Adjusted Clinical Groups

(32 ADGs)

(12 CADGs)

(26 MACs)

(106 ACGs)

AGE, GENDER

Generating ACG Output(Version 4.5)

Examples of ADGs and Their Common ICD-9-CM Codes

ADG Common Diagnosis (ICD-9-CM Code)

1 Time Limited: Minor Noninfectious Gastroenteritis (558.9)3 Time Limited: Major Phlebitis of Lower Extremities(451.2)9 Likely to Recur: Progressive mpaction of Intestine (560.3)

Malignant Hypertensive Renal Disease WithRenal Failure (403.01)Cerebral Thrombosis (434.0)Adult Onset Type II Diabetes w/ Ketoacidosis250.10)

10 Chronic Medical: Stable Essential Hypertension (401.9)Adult-Onset Type I Diabetes (250.00)

11 Chronic Medical: Unstable Malignant Hypertensive Heart Disease (402.0)Sickle-Cell Anemia (282.6)Diabetes Mellitus Without Complication250.03)

23 Psychosocial: Time Limited,Minor

Cannabis Abuse, Unspecified (305.20)

24 Panic Disorder (300.01)Psychosocial: Recurrent orPersistent, Stable Bulimia (307.51)

25 Catatonic Schizophrenia (295.2)Psychosocial: Recurrent orPersistent, Unstable Alcohol Withdrawal Delirium Tremens (291.0)

Clinical Vignette:40 year old woman: diabetes, hypertension (Release 4.5)

250.41, Diabetes with renal manifestations

401.9, Essential Hypertension

250.00, Adult Onset Diabetes, without complications

V70.0, Adult Routine Exam

ICD-9-CM ADG CADG MAC ACG

31: Preventative Administrative

10: Chronic Medical: Stable

9: Likely to recur: Progressive

5: Chronic Medical: Unstable

6: Chronic Medical: Stable

24: Multiple ADG Categories

4100: 2-3 other ADG combinations Age >34

Applying DCGs/ACGs in VA

Explore the feasibility of adapting diagnosis-based measures to the VA population

Examine how well each measure explains concurrent resource utilization and predicts future resource utilization in the VA

Evaluate their performance in clinically meaningful groups

Profile networks on their efficiency after adjustment for case-mix

ADG Categories in the VA and a Fee for Service Managed Care Population

ACC Categories in the VA and Medicare

Predictive Ratios for Patients with MH/SA Disorders

0.80 0.85 0.90 0.95 1.00 1.05 1.10

HCC 35 (LowerCost MentalDisorders)

HCC 33(Depression)

HCC 32(Psychosis)

DCG/HCC model

DCG/HCC model + dummy markers

Predictive Ratios For Subgroups of Veterans: Concurrent Models

Actual and Predicted Ambulatory Provider Encounters: Concurrent Models

ACG, DCG, and Unadjusted Efficiency Indices By Network

0.75

0.85

0.95

1.05

1.15

1.25

1.35

A C P B O J Q H L I N M D U E R S F G V T K

NETWORKS SORTED BY DCG PREDICTED RESOURCE USE

EF

FIC

IEN

CY

IN

DIC

ES

ACG DCG UNADJUSTED

Improved Special Population Data

*Note: A value greater than 1 means that the actual cost exceeds the predicted cost (or price).

What Weaknesses Remained?

Did not predict mental health costs well Did not explain long-term care costs Did not predict special population costs

Patient Safety Indicators (PSIs)

Developed by Agency for Healthcare Research and Quality (AHRQ)

Screen for potential safety events in the inpatient setting

Risk adjustment based on age, sex, age/sex interactions, DRGs, 27 comorbidities (AHRQ comorbidity software)

Examine observed and risk-adjusted PSI rates in VA 16 medical/surgical PSIs relevant to VA

AHRQ Comorbidities for “Decubitus Ulcer”

Congestive heart failure

Valvular disease

Pulmonary circulation disorders

Peripheral vascular disorders

Hypertension (combine uncomplicated and complicated)

Other neurological disorders

Chronic pulmonary disease

Diabetes, uncomplicated

Diabetes, complicated

Hypothyroidism

Renal failure

Peptic ulcer disease excluding bleeding

AIDS: Acquired immune deficiency syndrome I

Lymphoma

Metastatic cancer

Solid tumor without metastasis

Rheumatoid arthritis/collagen vascular diseases

Obesity

Weight loss

Blood loss anemia

Deficiency anemias

Alcohol abuse

Drug abuse

Depression

Additional VA comorbidities Paralysis

Liver disease

Psychoses

Characteristics of VA and NIS Samples: Discharges and Patients

PSI 3

FacilityVA Observed

AHRQ Expected

VA Expected

VA Obs / AHRQ Expt

VA Obs / VA Expt

A 19.45 22.92 17.60 0.85 1.11

B 18.16 24.10 17.99 0.75 1.01

C 19.42 24.21 20.50 0.80 0.95

“Decubitus Ulcer”

VA does well in non-VA comparison Within VA comparison changes direction

Conclusions

Despite different ways of evaluating model performance, model-based resource allocation for subgroups of veterans would not be adequate

Existing methods (ACGs/DCGs) generally underestimate health care costs of individuals with mental health/substance abuse (MH/SA) disorders

Non-VA based risk adjustment can be misleading in VA facility comparisons

Adequate Risk Adjustment: Important for Veterans with MH/ SA Disorders

The VA is the largest mental health service delivery system in the United States

Prevalence of mental disorders in VA: 30% Goal: develop and validate a psychiatric

diagnosis-based risk-adjustment measure (the “PsyCMS”) for veterans with MH/SA disorders

Guiding Principles

Incorporate all 526 adult MH/SA codes Develop clinically homogeneous categories

based on resource utilization Demonstrate face validity Include “manageable” # of categories Minimize “gaming” Predict concurrent/prospective utilization and

costs

Methods

Sample All veterans who received any health care in the

VA during Fiscal Year 1999 (October 1, 1998 through September 1, 1999) and had a MH or SA diagnosis (ICD-9-CM codes 290-312.9 or 316) (n=914,225)

Methods

Data Diagnostic and utilization data from VA inpatient

and outpatient administrative data Costs obtained from VA Health Economics and

Resource Center (HERC) FY99 data used for concurrent modeling; data

split into 60% development sample (n=548,535) and 40% validation sample (n=365,690)

FY00 data used for prospective modeling

Variables

Dependent Variables Total MH/SA costs: sum of costs associated with all

outpatient and inpatient MH/SA utilization Outpatient MH/SA encounters: sum of all visits associated

with any MH/SA diagnosis code, plus all visits in MH/SA specialty clinics

Inpatient MH/SA utilization: number of days a patient resided in any inpatient setting for MH or SA treatment

Independent Variables Age, gender, diagnostic information (all MH/SA primary

and secondary diagnoses)

Methods

Data Analysis (Four major steps):1. Classification and categorization of all MH/SA

codes into diagnostic classification system

2. Examined distribution of MH/SA disorders using PsyCMS

3. Assessed predictive validity of the PsyCMS using concurrent and prospective modeling

4. Compared performance of PsyCMS with ACGs and DCGs

PsyCMS Mood/Psychosis Hierarchy

PsyCMS Anxiety Hierarchy

PsyCMS Alcohol Hierarchy

PsyCMS Drug Hierarchy

DxGroups Condition Categories Hierarchical Condition Categories (HCCs)

HCC 31

HCC 32

HCC 33

HCC 34

HCC 35

No

No

No

No

Yes

Yes

Yes

Yes

Yes

53.01 alcoholic psychoses53.02 drug psychoses59.01 alcohol dependence59.02 drug dependence

54.01 delirium/delusions/hallucinations54.02 hallucinations, symptomatic55.01 schizophrenic disorders56.01 manic & depressive (bipolar) disorder56.02 major depressive disorders57.01 paranoid states57.02 other nonorganic psychoses60.01 personality disorders, including dissociative identity disorder134.04 attempted suicide/self-inflicted injury

60.06 nonpsychotic organic brain syndrome60.07 depression, excluding depressive psychosis60.11 autism, other childhood psychoses60.12 anorexia/bulimia nervosa60.19 prolonged posttraumatic stress disorder

58.01 panic disorders/attacks58.02 generalized anxiety disorder58.04 somatoform/dissociative disorders58.05 phobic disorders58.06 obsessive-compulsive disorders

58.03 other & unspecified anxiety states58.07 other & unspecified neurotic disorders59.03 non-dependent abuse of alcohol59.04 tobacco use disorder59.05 other nondependent drug abuse60.02 sexual deviations & disorders60.03 psychosomatic illness60.04 acute reaction to stress60.05 adjustment reaction, excluding prolonged depressive60.08 behavior disorder60.09 emotional disorders of childhood/adolescence60.10 other mental disorders60.13 attention deficit disorder, other hyperkinetic syndrome60.14 learning/development learning disorder

Drug/Alcohol Dependence/ Psychoses

Psychosis & Other Higher Cost mental

Disorders

Depression & Other Moderate

Cost Mental Disorders

Lower Cost Mental Disorders

Anxiety Disorders

Diagnostic Cost Group (DCG) Mental Health Groupings

ICD-9-CM Psychiatric Codes

ADG 23

Psycho-social: Time Limited, Minor

ADG 24

Psycho-social: Recurrent or Persistent,

Stable

ADG 25

Psycho-social: Recurrent or Persistent,

Unstable

CADG 10

Psycho-Social

MAC 10

Psychosocial

MAC 17

Acute: Minor and Psychosocial

MAC 24

All Other Combinations Not

Listed Above

•CADG 10 & CADG 1 & CADG 3

•CADG 10 & CADG 1 •CADG 10 & CADG

1 & CADG 2 & CADG 3

•CADG 10 & All Other Remaining CADG Combinations

ACG 1500

ACG 1400

ACG 2500

ACG 2700

Refer to MAC 24 Decision Tree

No

Yes

Yes

No

No

Yes

No

•CADG 10 Only

•CADG 10 & CADG 12 &

Anything Else

Yes

ACG 3700

MAC 21

Acute: Minor & Likely to Recur &

Psychosocial

ADG 25?

ADG 24?

ACG 1300

ADG 25?

ADG 24?

MAC 12

Pregnancy

Refer to MAC 12 Decision TreeACG 3500 ACG 2600

MAC 23

Acute: Minor & Acute: Major &

Likely to Recur & Psychosocial

Adjusted Clinical Group (ACG) Psycho-social Groupings

Results: Prevalence of Selected PsyCMS Categories

Table 1: Total MH/SA Costs for Selected Categories

Table 2: Model Goodness of Fit for Concurrent (FY99) Validation Samples

Table 3: Model Goodness of Fit for Prospective (FY00) Validation Samples

Conclusions

PsyCMS appears to be valid and reliable measure for MH/SA risk adjustment

PsyCMS performs better than other systems in predicting concurrent and prospective MH/SA costs/utilization 

It can serve as risk-adjustment system for describing MH/SA populations, profiling MH/SA services, and budgeting future MH/SA resources

Any Questions?

Amy Rosen, Ph.D.E-mail akrosen@bu.edu

CHQOER Bedford VAMC

200 Springs Road (152)Bedford, MA 01730 U.S.A

Phone (781) 687-2960Fax (781) 687-3106

http://www.va.gov/chqoer/RAPS/RAPS.htm

References

Rosen AK, Loveland S, Anderson J, Rothendler J, Hankin C, Moskowitz M, Berlowitz DR. Evaluating diagnosis-based case-mix measures: how well do they apply to the VA population? Medical Care 2001; 39(7): 692-704.

Rosen AK, Loveland S, Anderson J. Applying DCGs to Examine the Disease Burden of VA Facilities: Comparing the Six “Evaluating VA Costs” Study Sites to Other VA Sites and Medicare. Medical Care, June 2003: 41(6 suppl): II-91-II-102.

Rosen AK, Loveland S, Anderson J, Hankin C, Breckenridge J, Berlowitz DR. Diagnostic Cost Groups (DCGs) and concurrent utilization among patients with substance abuse disorders. Health Services Research, 2002: 37(4): 1079-1102.

Rakovski C, Rosen AK, Loveland S, Anderson JJ, Berlowitz DR, Ash A. Evaluation of diagnosis-based risk adjustment measures among specific subgroups: can existing measures be improved by simple modifications?" Health Services and Outcomes Research Methodology, 2002: 3(1): 57-74.

Rosen AK, Rakovski C, Loveland S, Anderson JJ, Berlowitz DR. Profiling resource use across providers: do different outcomes affect assessments of provider efficiency after case-mix adjustment? American Journal of Managed Care, 2002: 8(12): 1105-1115.

Rosen AK, Loveland S, Rakovski C, Christiansen C, Berlowitz DR. Do different case-mix measures affect assessments of provider efficiency? Lessons from the VA. The Journal of Ambulatory Care Management 2003: 26(3): 229-242.

Rosen AK, Reid R, Broemeling AM, Rakovski C. Applying a risk adjustment framework to primary care: can we improve on existing measures? Annals of Family Medicine, 2003: 1(1): 44-51.

References (cont’d)

Rosen AK, Trivedi P, Amuan M, & Montez M. The John Hopkins Adjusted Clinical Groups (ACGs) case-mix system: A risk-adjustment methodology currently available at the VA Austin Automation Center. VIReC Insights Vol. 4, No. 1. Hines, IL: VA Information Resource Center, 2003. Available at http://virec.research.med.va.gov.

Liu CF, Sales AE, Sharp ND, Fishman P, Sloan KL, Todd-Stenberg J, Nichol WP, Rosen AK, Loveland S. Case-mix adjusting performance measures in a VA population: pharmacy- and diagnosis- based approaches. Health Services Research, 2003: 38 (5): 1319-1338.

Warner G, Hoenig, H, Montez M, Wang F, Rosen AK. Evaluating diagnosis-based risk-adjustment methods in the spinal cord dysfunction population. Archives of Physical Medicine and Rehabilitation, 2004: 85(2): 218-226.

Rosen AK, Christiansen CL, Montez ME, Loveland S, Shokeen P, Sloan KL, and Ettner SL. Evaluating risk-adjustment methodologies for patients with mental health and substance abuse disorders in the Veterans Health Administration. International Journal of Healthcare Technology and Management, 2006: 7 (1/2): 43-81.

Sloan KL, Montez ME, Spiro A III, Christiansen CL, Loveland S, Shokeen P, Herz L, Eisen S, Breckenridge, JN, Rosen AK. Development and validation of a psychiatric case-mix system. Medical Care 2006: 44:568-580.

Montez ME, Christiansen CL, Ettner SL, Loveland S, Shokeen P, and Rosen AK. Performance of statistical models to predict mental health and substance abuse cost. BMC Medical Research Methodology, October 2006: 6:53.

Rosen AK, Zhaos S, Rivard P, Loveland S, Montez M, Elixhauser A, and Romano P. Tracking Rates of Patient Safety Indicators over Time: Lessons from the VA. Medical Care 2006: 44(9): 850-861.

www.dxcg.com

www.acg.jhsph.edu

Risk Adjustment for Measuring Health Care Outcomes, edited by Lisa Iezzoni, Health Administration Press, 3rd edition, 2003.