1 NQF 101: Developing Consensus March 25, 2009 NQF Spring Membership Meeting Renaissance Cleveland...

1

NQF 101: Developing Consensus

March 25, 2009

NQF Spring Membership MeetingRenaissance Cleveland Hotel

Cleveland, OH

2

NQF’s Mission

The National Quality Forum is a private, not-for-profit voluntary consensus standards-setting organization.

The mission of NQF is: • to improve the quality of American

healthcare by setting national priorities and goals for performance improvement;

• endorsing national consensus standards for measuring and publicly reporting on performance; and

• promoting the attainment of national goals through education and outreach programs. 

3

Developing Consensus

• Formal consensus development process (CDP)• Attention to overall strategy for measuring and reporting

healthcare quality, including establishing national goals• Multi-stakeholder membership including consumers,

purchasers, plans, professionals, providers, quality and researchers, community and public health and supplier and industry organizations

• Public and private sector representation on governing board

• Focus is on the entire continuum of healthcare• Formal endorsement as “voluntary consensus standards”

4

Consensus Development Process

Draft recommendations

National Priorities andNQF program

priorities

Specific projecttopics

Project Steering/Review Committee

Draft consensus standards

NQF-endorsedconsensus standards

Technical Advisors or Panels; Workgroups

Review (Member and public)

CSAC/Board action

Appeals

Update aswarranted

Member voting

5

Roles of the Steering Committee

• Act as a proxy for the NQF multi-stakeholder membership for a specific project

• Work with NQF staff to achieve the goals of the project

• Evaluate candidate measures against the formal measure evaluation criteria

• Make recommendations to the NQF membership for endorsement

• Respond to comments submitted during the review period

• Co-chair(s) represent the Steering Committee when the CSAC meets

• Respond to any directions from the CSAC

6

Role of NQF Staff

• NQF project staff works with Steering Committee to achieve the goals of the project and ensure adherence to the consensus development process: – Organize and staff SC meetings and

conference calls– Guide the SC through the steps of the CDP and

advises on NQF policy and procedures – Draft measure evaluations and reports for SC

review– Ensure communication among all project

participants (including SC and measure developers)

7

Roles of NQF Staff

– Respond to NQF member or public queries about the project

– Maintain documentation of project activities– Post project information to NQF website– Work with measure developers to provide

necessary information and communication for the SC to fairly and adequately evaluate measures for endorsement

– NQF project staff works with communications department to publish final report

8

Strategic Measurement Issues for NQF

• Driving toward high performance• Shifting toward composite measures• Moving toward outcomes

measurement• Measure disparities in all we do

9

NQF Endorsement Criteria

• New NQF measure evaluation criteria were approved by Board of Directors in August 2008

• Clarify, strengthen and recommend changes to endorsement criteria in order to achieve:– a stronger link to national priorities and higher-

level performance measures;– greater measure harmonization;– greater emphasis on outcome measures; and– for process measures, a tighter outcomes-

process

10

NQF Endorsement Criteria

Old criteria New criteria (August 2008)

Conditions for consideration

Importance Importance to measure and report (must pass criterion)

Scientific acceptability Scientific acceptability of measure properties

Feasibility Feasibility (greater emphasis on Health IT)

Usability Usability (greater emphasis on harmonization)

11

Conditions for Consideration

1. The measure is in the public domain or an intellectual property agreement is signed.

2. The measure owner verifies there is an identified responsible entity and process to maintain and update the measure.

3. The intended use of the measure includes both public reporting and quality improvement.

4. The requested measure submission information is complete. • Generally, measures should be fully developed and tested

so that all information needed to evaluate the measure is available.

• Measures that have not been tested are only potentially eligible for a time-limited endorsement

• Measure owners must verify that testing will be completed within 24 months of endorsement.

12

Importance to Measure and Report

• The specific focus of what is measured should be considered important enough to expend resources for measurement and reporting, not only that it is related to an important broad topic area.

• In the modified criteria, these concepts are addressed in separate sub-criteria for – Relation to an NQF priority or high impact

aspect of healthcare– Evidence to support the measure focus– Opportunity for improvement.

13

Importance to Measure and Report

• The recommendation that the importance criterion should be considered “must pass.”

• If a measure is not judged to be sufficiently important, then regardless of how well it might be assessed on the other criteria the measure would not be recommended for consideration as a voluntary consensus standard.

14

Scientific Acceptability of Measure Properties

• The label clearly indicates this criterion applies to measure properties: – Precise specifications– Reliability, validity, and discrimination (testing is

expected to demonstrate reliability and validity)– Demonstration of comparability if more than one

data source/method is allowed – Specifications should allow for identification of

disparities. – Risk-adjustment– Exclusions

15

Exclusions

• Exclusions increase complexity and measurement burden, limit the use of electronic sources, and often create a barrier to measure harmonization.

• Evaluation criteria requires that evidence is presented to demonstrate that measure results would be distorted without the specified exclusions.

• If patient preference is a consideration in numerator or denominator exclusions, the measure should be specified so that the effect of patient preference on the measure is transparent

• Exclusions should not require additional data sources beyond what is required for scoring the measure (e.g., numerator and denominator) unless justified as supporting measure validity.

16

Usability

• Requires evidence that the measure results are meaningful and understandable to intended audiences and useful for both public reporting and informing quality improvement. – This is consistent with NQF policy of not

endorsing measures solely for quality improvement.

– New criteria requiring that measures are harmonized and provide a distinctive or additive value to existing endorsed measures.

17

Feasibility

• Extent to which the required data are readily available, retrievable without undue burden, and can be implemented for performance measurement.– Required data are routinely generated

concurrent with and as a byproduct of care processes during care delivery.

– The required data elements are available in electronic sources. If the required data are not in existing electronic sources, a credible, near-term path to electronic collection by most providers is specified and the data elements are specified for transition to the electronic health record.

18

Staff

• Helen Burstin, MD, MPHSr. Vice President, Performance Measures

hburstin@qualityforum.org

• Marybeth Farquhar, PhD, MSN, RNManaging Director, Performance Measures

mfarquhar@qualityforum.org

• Del Conyers, MPHAssistant Managing Director, Performance Measures

dconyers@qualityforum.org