Measure Calculations - CMSMeasure Calculations There are multiple ways to calculate quality measures . This document provides information about the calculations of proportion measures

Supplemental Material to the CMS MMS Blueprint

September 2020 Page 1

Measure Calculations

There are multiple ways to calculate quality measures. This document provides information about the calculations of proportion measures, continuous variable (CV) measures, and ratio measures, and supplements the information found in the Blueprint, Chapter 5, Measure Specification.

1 PROPORTION MEASURES

This section provides guidance on the calculation and interpretation of proportion measures.

A proportion is a measure in which the numerator is a subset (or part) of the denominator and can be written as a/(a+b) (Figure 1). A proportion measure differs from a ratio measure because in a ratio measure the numerator is not a subset of the denominator population (refer to the Blueprint Chapter 5.1, Measure Specifications by Measure Category). Although the numerator and denominator populations may be related or may overlap for ratio measures, these populations do not have a superset/subset relationship. Figure 2 provides a flow from initial population to numerator and denominator.

1 Proportion Measures .......................................1

1.1 Elements of a Proportion Measure.............4

1.2 Performance Rates ..................................4

1.3 Proportion Measure Examples ..................4

2 Continuous Variable Calculations.......................7

2.1 Elements of Continuous Variable Measures ...............................................7

2.2 Continuous Variable Aggregate Calculations ............................................9

2.3 CV Measure Example ...............................9

3 Ratio Measure Calculations............................. 10

3.1 Elements of a Ratio Measure .................. 10

3.2 Ratio Measure Aggregate Calculations...... 12

3.3 Ratio Measure Example.......................... 12

4 Key Points .................................................... 14

References........................................................ 15

https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/Downloads/Blueprint.pdf

Supplemental Material to the CMS MMS Blueprint Measure Calculations


Figure 1. Proportion Measure Populations

Population Criteria Definitions for Proportion Measures (Health Level Seven International® [HL7], n.d.)




Figure 2. Population Criteria Definitions for Proportion Measures (Health Level Seven International® [HL7], n.d.)




1.1 ELEMENTS OF A PROPORTION MEASURE

From these relationships and definitions, the measure developer should use these sequential steps to determine whether a patient falls into a given population.

1. Initial population (IP): Identify those patients who meet the IP criteria.

2. Denominator (DENOM): Identify the subset of the IP that meet the DENOM criteria.

3. Denominator exclusion (DENEX): Identify the subset of the DENOM that meet the DENEX

criteria. Remove those that are patients from the denominator as exclusions. After removal of

these patients, the remaining subset reflects the group of patients evaluated for the

numerator criteria.

4. Numerator (NUMER): Identify those in the DENOM and not in the DENEX that meet the NUMER

criteria. In proportion measures, the numerator criteria are the processes or outcomes

expected for each patient, procedure, or other unit of measurement defined in the

denominator.

5. Numerator exclusion (NUMEX): Identify the subset of the NUMER that meet the NUMEX

criteria. Use NUMEX to define instances not included in the numerator data.

6. Denominator exception (DENEXCEP): Identify those in the DENOM and not in the DENEX and

NOT in the NUMER that meet the DENEXCEP criteria. Remove these patients from the IP and/or

DENOM.

1.2 PERFORMANCE RATES

Specific programs may require reporting of performance rates. The performance rate is the number of patients meeting NUMER criteria (accounting for exceptions), divided by patients in the DENOM (accounting for exclusion and exception). Calculate the performance rate using the formula

Performance rate1 = (NUMER – NUMEX) / (DENOM – DENEX – DENEXCEP)

Exception: 0% Performance Rates

• Inverse measures: For inverse measures, a lower rate indicates better performance; consider a 0% performance rate as satisfactorily reporting (e.g., 30-day mortality rate for acute myocardial infarction).

• Null Scores: If the measure is not applicable for all patients within the sample, the performance rate would be 0/0 (null) and would be considered satisfactorily reporting. Do not count performance exclusion Quality Data Codes (QDCs) in the performance denominator.

1.3 PROPORTION MEASURE EXAMPLES

Examples of the mathematical relationships between populations in proportion measures

Example #1: Proportion Measure

A fictitious proportion measure defines the population criteria.

1 Also referred to as the raw rate.




• IP: all patients aged 65 years and older with an active diagnosis of diabetes mellitus

• DENOM: equals IP

• DENEX: bilateral blindness

• NUMER: dilated eye exam for diabetic retinopathy

• NUMEX: none

• DENEXCEP: bed confinement status in a community where mobile eye exam imaging is

unavailable

Mr. Jones is 75 years old and has an active diagnosis of diabetes. There is no mention of blindness in his chart. He has a documented dilated eye exam for diabetic retinopathy.

• (IP = YES) Mr. Jones meets the IP criteria.

• (DENOM = YES) Mr. Jones meets the DENOM criteria.

• (DENEX = NO) By the positive evidence principle, Mr. Jones does not meet the DENEX criteria.

• (NUMER = YES) Mr. Jones meets the NUMER criteria.

• (NUMEX = NO)

• (DENEXCEP = NO) no evaluation of the DENEXCEP because the patient met the NUMER criteria.

Mr. Smith is 75 years old and has an active diagnosis of diabetes. There is no mention of blindness in his chart. There is no mention of a dilated eye exam in his chart. There is no mention in his chart that he is bed bound.

• (IP = YES) Mr. Smith meets the IP criteria.

• (DENOM = YES) Mr. Smith meets the DENOM criteria.

• (DENEX = NO) By the positive evidence principle, Mr. Smith does not meet the DENEX criteria.

• (NUMER = NO) By the positive evidence principle, Mr. Smith does not meet the NUMER criteria.

• (NUMEX = NO)

• (DENEXCEP = NO) By the positive evidence principle, Mr. Smith does not meet the DENEXCEP

criteria.

Mr. Johnson is 85 years old and has an active diagnosis of diabetes. There is no mention of blindness in his chart. He has a documented dilated eye exam for diabetic retinopathy and is confined to bed in a community where mobile eye exam imaging is unavailable.

• (IP = YES) Mr. Johnson meets the IP criteria.

• (DENOM = YES) Mr. Johnson meets the DENOM criteria. • (DENEX = NO) By the positive evidence principle, Mr. Johnson does not meet the DENEX criteria.

• (NUMER = YES) Mr. Johnson meets the NUMER criteria. • (NUMEX = NO)

• (DENEXCEP = NO) By definition, Mr. Johnson does not meet the DENEXCEP criteria because DENEXCEP criteria are not applicable to those meeting the NUMER criteria.

Example #2: Inverse Proportion Measure

A fictitious inverse proportion measure (i.e., where improvement is a decrease in the rate) defines the population criteria.

• IP: patients aged 8 to 65 who gave birth • DENOM: equals IP




• DENEX: patients with gestational age < 37 weeks • NUMER: patients with medical induction of labor or C-section

• NUMEX: patients in active labor or with spontaneous rupture of membranes before induction of labor or C-section

• DENEXCEP: none

Mrs. Jones is a 31-year-old woman who gave birth at 37 weeks’ gestation. She has a medical induction of labor with no prior evidence of active labor or spontaneous rupture of membranes.

• (IP = YES) Mrs. Jones meets the IP criteria. • (DENOM = YES) Mrs. Jones meets the DENOM criteria.

• (DENEX = NO) Mrs. Jones does not meet the DENEX criteria. • (NUMER = YES) Mrs. Jones meets the NUMER criteria.

• (NUMEX = NO) By the positive evidence principle, Mrs. Jones does not meet the NUMEX criteria.

• (DENEXCEP = NO)

Mrs. Thompson is 31 years old and had a C-section at 38 weeks after a spontaneous rupture of membranes.

• (IP = YES) Mrs. Thompson meets the IP criteria.

• (DENOM = YES) Mrs. Thompson meets the DENOM criteria.

• (DENEX = NO) Mrs. Thompson does not meet the DENEX criteria.

• (NUMER = YES) Mrs. Thompson meets the NUMER criteria.

• (NUMEX = YES) Mrs. Thompson meets the NUMEX criteria.

• (DENEXCEP = NO)

Mrs. Hill is 31 years old and gave birth at 36 weeks with induction after a spontaneous rupture of membranes.

• (IP = YES) Mrs. Hill meets the IP criteria.

• (DENOM = YES) Mrs. Hill meets the DENOM criteria. • (DENEX = YES) Mrs. Hill meets the DENEX criteria.

• (NUMER = NO) By definition, Mrs. Hill does not meet the NUMER criteria because NUMER criteria are not applicable to those meeting the DENEX criteria.

• (NUMEX = NO) By definition, Mrs. Hill does not meet the NUMEX criteria because NUMEX criteria are only applicable to those meeting the NUMER criteria.

• (DENEXCEP = NO)

Example #3: Measure Aggregate Calculations

Aggregate scores are simply the counts of individuals in each population. Thus, the aggregate IP is the count of individuals meeting the IP criteria.

Building on Example #1, counting all individuals within the population, the aggregate counts are determined.

• Initial population: N=150 (e.g., 150 patients meet the IP criteria)

• Denominator: N=150 • Denominator exclusion: N=20 (meet DENOM and meet DENEX)

• Numerator: N=75 (meet DENOM, not in DENEX, and meet NUMER criteria)




• NUMEX: N=0 • Denominator exception: N=5 (meet DENOM, not in DENEX, not in NUMER, and meet the

DENEXCEP criteria) • Performance rate = (NUMER – NUMEX) / (DENOM – DENEX - DENEXCEP) = (75-0) / (150-20-

5) = 0.6

2 CONTINUOUS VARIABLE CALCULATIONS

This section provides further guidance on the precise mathematical relationships between populations in a CV measure and the process to be used to determine individual and aggregate scores.

In a CV measure, each individual value for the measure can fall anywhere along a continuous scale, for example, mean number of minutes between presentation of chest pain to the time of administration of thrombolytic medications.

2.1 ELEMENTS OF CONTINUOUS VARIABLE MEASURES

There is a fixed mathematical relationship between the populations in a CV measure, as shown in Figure 3:

Figure 3. CV Measure Populations




Figure 4. Population Criteria for Continuous Variable Measures (HL7, n.d.)

Figure 4 provides a flow from initial population to measure observation. Note the figure references electronic clinical quality measures (eCQMs), but is valid for other measure types.




We define the CV measure query process.

• IP: Identify those cases that meet the IP criteria.

• Measure population (MSRPOPL): Identify the subset of the IP that meet the MSRPOPL criteria.

• Measure population exclusion (MSRPOPLEX): Identify the subset of the MSRPOPL that meet

the MSRPOPLEX criteria.

For a CV measure, an individual observation is determined for each case in the MSRPOPL and not in the MSRPOPLEX.

eCQM Clinical Quality Language (CQL) logic should not explicitly remove the MSRPOPLEX cases because the execution environment for the eCQM is responsible for excluding the MSRPOPLEX cases and calling the Measure Observation function for each of the remaining cases.

Per Requirement 13 of the HL7® Version 3 Implementation Guide: CQL-based Health Quality Measure Format (HQMF) Release 1, Standard for Trial Use (STU) 3 – US Realm , aggregation rules are mandatory: “methodCode shall be populated to indicate the aggregation method for the measure.”

2.2 CONTINUOUS VARIABLE AGGREGATE CALCULATIONS

Aggregate scores for CV measures are more complex than for proportion measures in that they are more than just the counts of cases in each population. In addition to the identification of measure population(s), CV measures define observations made on cases falling into various populations. These individual observations, after removing the exclusions, are then aggregated according to aggregation rules specific to each measure.

In CV measures, the measure developer should make an individual observation for each case falling into the measure population and not in the measure population exclusion. The remaining individual observations are then used to calculate the aggregate CV, which constitutes the measure “score.”

It is important to note that observations on the individual can be sent in Quality Reporting Document Architecture (QRDA) Category I, whereas aggregate calculations can be sent in QRDA Category III.

2.3 CV MEASURE EXAMPLE

A fictitious CV measure defines the population criteria.

• IP: All inpatient encounters ending during the measurement period with length of stay fewer than or equal to 120 days

• MSRPOPL: Inpatient encounters preceded by an emergency department (ED) visit

• MSRPOPLEX: none • Individual observations: Time in minutes from ED admission to ED discharge for patients

admitted to the facility from the ED

• MSRPOPL observation: Number of minutes in the ED

Aggregation rules

For eCQMs, HQMF methodCode specifies median.

Mr. Jones is a 75-year-old man admitted to the hospital from the ED. He spent 90 minutes in the ED.


http://www.hl7.org/dstucomments/showdetail.cfm?dstuid=230




• (IP = YES) Mr. Jones has inpatient encounters that meet the IP criteria. • (MSRPOPL = YES) Mr. Jones meets the MSRPOPL criteria.

• (MSRPOPLEX = NO) • Individual observations for Mr. Jones: 90 minutes

• MSRPOPL observation: 90 minutes

Mr. James is 75 years old, admitted directly to the hospital from an outside facility.

• (IP = YES) Mr. James has inpatient encounters that meet the IP criteria. • (MSRPOPL = NO) By the positive evidence principle, Mr. James does not meet the MSRPOPL

criteria.

• (MSRPOPLEX = NO) • Individual observations for Mr. James: none

• MSRPOPL observation: N/A (Mr. James has no inpatient encounters that meet the MSRPOPL

criteria).

Measure aggregate calculations

Building upon the previous examples, but now considering all encounters within the population, the aggregate counts are determined.

• IP: N = 150 (i.e., 150 encounters meet the IP criteria)

• MSRPOPL: N =120

• MSRPOPLEX: N = 0

CV Score: 96 minutes (median of all individual Measure Population observations of time spent in the ED).

3 RATIO MEASURE CALCULATIONS

This section provides further guidance on the precise mathematical relationships between populations in a ratio measure. The measure developer can use this process to determine individual and aggregate scores.

Derive a ratio measure by dividing a count of one type of data by a count of another type of data (e.g., the number of patients with central lines who develop infection divided by the number of central line days). A ratio measure differs from a proportion measure because in a ratio measure the numerator is not a subset of the denominator population. Although the numerator and denominator populations may be related or may overlap, these populations do not have a superset/subset relationship.

3.1 ELEMENTS OF A RATIO MEASURE

There is a fixed mathematical relationship between the populations in a ratio measure, as shown in Figure 5. Ratio measures are often two CV calculations for related populations (e.g., median ED waiting time for the index hospital, median ED waiting time for the region in which the hospital is located as the numerator).




Figure 5. Ratio Measure Populations

Rate ratio measure – numerator flow

Figure 6. Population Criteria for Ratio Measures - Numerator (HL7, n.d.)




Figure 6 provides a flow from initial population to the numerator and numerator exclusion.

From these relationships and definitions, define the ratio measure query process.

• IP: Identify those cases that meet the IP criteria. (Some ratio measures will require multiple IPs: one for the numerator and one for the denominator).

• DENOM: Identify the subset of the IP that meet the DENOM criteria.

• DENEX: Identify the subset of the DENOM that meet the DENEX criteria. • NUMER: Identify the subset of the IP that meet the NUMER criteria.

• NUMEX: Identify the subset of the NUMER that meet the NUMEX criteria.

Compute individual observations for ratio measures both for the denominator and numerator populations.

• For each case in the DENOM and not in the DENEX, determine the individual DENOM observations.

• For each case in the NUMER and not in the NUMEX, determine the individual NUMER observations.

3.2 RATIO MEASURE AGGREGATE CALCULATIONS

As with CV measures, aggregate scores for ratio measures are more than just the counts of cases in each population. In addition to the identification of measure population(s), the measure developer should make ratio measures define observations on cases falling into various populations and then aggregate these individual observations according to aggregation rules specific to each measure.

In ratio measures, for each population, make individual observations, e.g., measurements or calculations, for denominator and numerator cases, then use them to calculate the aggregate ratio

Ratio = aggregate NUMER / aggregate DENOM

where the aggregate DENOM is calculated using individual observations for all cases in the DENOM and not in the DENEX, and the aggregate NUMER is calculated using individual observations for all cases in the NUMER and not in the NUMEX, calculate the aggregate NUMER.

3.3 RATIO MEASURE EXAMPLE

A fictitious ratio measure looking at the rate of central line blood stream infections (i.e., the ratio of number of central line infections per total central line days) defines the population criteria.

• IP: All hospitalized patients with a laboratory confirmed bloodstream infection.

• DENOM: Presence of central line or umbilical catheter for >2 calendar days • DENEX: Patient is immunosuppressed.

• NUMER: Central line blood stream infection

• NUMEX: Deem infection to be a contaminant.

Individual Observations

• DENOM observation: Number of hospital days with central line • NUMER observation: Fully define the observation in this case by the population criteria (i.e., the

observation is simply whether the case met the population criteria or not).




Aggregation Rules

• DENOM aggregation: Sum of number of hospital days with central line – DENEX

• NUMER aggregation: Count of NUMER – NUMEX

Mr. Jones is 75 years old and was in the hospital for 7 days. He had a central line in place for 5 days. There is no mention of immunosuppression in his chart. There is no mention of a central line blood stream infection in his chart.

• (IP = YES) Mr. Jones meets the IP criteria.

• (DENOM = YES) Mr. Jones meets the DENOM criteria.

• (DENEX = NO) By the positive evidence principle, Mr. Jones does not meet the DENEX criteria.

• (NUMER = NO) Mr. Jones does not meet the NUMER criteria.

• (NUMEX = NO) By definition, Mr. Jones does not meet the NUMEX criteria because NUMEX

criteria are only applicable to those meeting the NUMER criteria.

Individual observations for Mr. Jones

• DENOM observation: 5

• NUMER observation: N/A (Mr. Jones does not meet the NUMER criteria)

Mr. James is 75 years old and was in the hospital for 24 days. He had a central line in place for 17 days. There is no mention of immunosuppression in his chart. He has a documented central line blood stream infection while the central line was in place. There is no mention that the infection is a contaminant.

• (IP = YES) Mr. James meets the IP criteria. • (DENOM = YES) Mr. James meets the DENOM criteria.

• (DENEX = NO) By the positive evidence principle, Mr. James does not meet the DENEX criteria. • (NUMER = YES) Mr. James does meet the NUMER criteria.

• (NUMEX = NO) By the positive evidence principle, Mr. James does not meet the NUMEX criteria.

Individual observations for Mr. James

• DENOM observation: 17

• NUMER observation: Mr. James meets the NUMER criteria.

Measure aggregate calculations

Building from the examples above, considering all cases within the population, the aggregate counts are determined.

• IP: N = 150 (i.e., 150 cases meet the IP criteria)

• DENOM: N = 20 of the 150 had laboratory confirmed blood stream infection with central lines. • DENEX: N = 2

• NUMER: N = 6 (Mr. James was one of the six numerator cases.)

• NUMEX: N = 1

Aggregate DENOM: 108 (In this example, 108 is an assumed sum of central line days across all cases in the DENOM and not in the DENEX).

Aggregate NUMER: 5 (total number of central line blood stream infections, excluding those deemed to be contaminants).




Ratio = aggregate NUMER / aggregate DENOM = 5 / 108 = 0.046

Additional information about eCQMs and proportion, ratio, and CV measure calculation can be found at the Electronic Clinical Quality Improvement (eCQI) Resource Center .

4 KEY POINTS

The measure calculation is an important part of measure specification: it defines the mathematical relationships among measure components (numerator, denominator, exclusions, and exceptions) and dictates how the measure score should be interpreted. Although there are many ways to calculate a measure score, the most common are proportions, CVs, and ratios. Proportion measures are those in which the numerator is a subset of the denominator. They differ from ratio measures because, although numerator and denominator populations may be related for ratio measures, they do not have a superset/subset relationship. CV measures, on the other hand, allow for individual scores that can fall anywhere along a continuous scale (e.g., mean number of minutes between arrival and admission). Each of these calculation methods is easily depicted with a population flow diagram, or calculation algorithm, that illustrates the relationships among measure components to ensure that measure scores are calculated in a standardized way.


https://ecqi.healthit.gov/



REFERENCES

Electronic Clinical Quality Improvement Resource Center. (n.d.). eCQI resource center. Retrieved July 29, 2020, from https://ecqi.healthit.gov/

HL7 International. (2018). Health Level Seven International (HL7) version 3 implementation guide: CQL-based health quality measure format (HQMF) release 1, standard for trial use (STU) 3 - US realm. Retrieved July 29, 2020, from http://www.hl7.org/dstucomments/showdetail.cfm?dstuid=230

HL7 International. (n.d.). Quality measure implementation guide. Retrieved July 29, 2020, from http://build.fhir.org/ig/HL7/cqf-measures/measure-conformance.html#ratio-measures


https://ecqi.healthit.gov/




http://build.fhir.org/ig/HL7/cqf-measures/measure-conformance.html#ratio-measures

Documents

Measure Calculations - CMSMeasure Calculations There are multiple ways to calculate quality measures . This document provides information about the calculations of proportion measures