Workshop on Standards for Clinical Practice Workshop on Standards for Clinical Practice GuidelinesGuidelines

Institute of MedicineInstitute of MedicineJanuary 11, 2010January 11, 2010

Vivian H. Coates, Vice President, ECRIVivian H. Coates, Vice President, ECRIProject Director, NGC and NQMCProject Director, NGC and NQMC

National Guideline Clearinghouse National Guideline Clearinghouse (NGC)(NGC)

NGC and its sister National Quality Measures NGC and its sister National Quality Measures Clearinghouse (NQMC) sponsored by U.S. Agency for Clearinghouse (NQMC) sponsored by U.S. Agency for Healthcare Research and Quality (AHRQ)Healthcare Research and Quality (AHRQ)

Mary Nix, Clearinghouse AdministratorMary Nix, Clearinghouse Administrator Since 1997, ECRI contractor to AHRQ to create and Since 1997, ECRI contractor to AHRQ to create and

maintain NGC; since 2001, contractor for NQMCmaintain NGC; since 2001, contractor for NQMC ECRI EPC also provides methodology support to federal ECRI EPC also provides methodology support to federal

and private sector CPG developersand private sector CPG developers

NGC Perspective - Overview of all NGC Perspective - Overview of all CPGsCPGs

Maintain relationships with hundreds of guideline developers from Maintain relationships with hundreds of guideline developers from many countriesmany countries

Produce structured abstracts of thousands of guidelines, taking Produce structured abstracts of thousands of guidelines, taking note of underlying methodology used to develop each guidelinenote of underlying methodology used to develop each guideline

Author guideline syntheses – in depth comparisons of areas of Author guideline syntheses – in depth comparisons of areas of agreement and difference across multiple guidelines on same topicagreement and difference across multiple guidelines on same topic

Maintain relational database and website (Maintain relational database and website (www.guideline.gov)) Currently, NGC contains more than 2500 guidelines from more Currently, NGC contains more than 2500 guidelines from more

than 200 developers, and has reviewed nearly 8000 guidelinesthan 200 developers, and has reviewed nearly 8000 guidelines

NGC Perspective - Overview of all NGC Perspective - Overview of all CPGsCPGs

NGC template of attributes a widely used tool for NGC template of attributes a widely used tool for understanding important guideline characteristicsunderstanding important guideline characteristics

Facilitates understanding and critical appraisal of Facilitates understanding and critical appraisal of guideline quality and rigor guideline quality and rigor

Important attributes include: guideline scope, methods Important attributes include: guideline scope, methods used to collect and analyze evidence, formulate and used to collect and analyze evidence, formulate and validate recommendations, panel characteristics, validate recommendations, panel characteristics, financial and other disclosuresfinancial and other disclosures

NGC Perspective - Overview of all NGC Perspective - Overview of all CPGsCPGs

New in 2009New in 2009: research project underway by ECRI’s NGC team to explore : research project underway by ECRI’s NGC team to explore feasibility of using Guideline Elements Model (GEM) to process clinical feasibility of using Guideline Elements Model (GEM) to process clinical guidelines to facilitate their use in clinical decision support (CDS) guidelines to facilitate their use in clinical decision support (CDS) information systems at the point of careinformation systems at the point of care

Many guideline recommendations statements are vague and not Many guideline recommendations statements are vague and not “actionable” (decidable and executable), and therefore cannot readily be “actionable” (decidable and executable), and therefore cannot readily be implemented into CDS or electronic health record (EHR) systemsimplemented into CDS or electronic health record (EHR) systems

In general, the 2500 guidelines in NGC vary widely in methodologic rigor:In general, the 2500 guidelines in NGC vary widely in methodologic rigor:

• Comprehensive, transparent, well-documentedComprehensive, transparent, well-documented

• ““Evidence-light”, poorly documentedEvidence-light”, poorly documented

Challenges for CPG Developers and Challenges for CPG Developers and UsersUsers

Recommendation statements that are vague or ambiguous may be due to Recommendation statements that are vague or ambiguous may be due to deficiencies in underlying evidence – Developer may not be able to formulate deficiencies in underlying evidence – Developer may not be able to formulate explicit recommendations.explicit recommendations.

Guidelines often conflict: differences in evidence interpretation, but also Guidelines often conflict: differences in evidence interpretation, but also because of differences in approaches to evidence synthesis ( eg, narrative because of differences in approaches to evidence synthesis ( eg, narrative review vs full systematic review),different developer perspectives (eg. primary review vs full systematic review),different developer perspectives (eg. primary care vs specialty; health plan vs provider). NGC identifies at least 25 different care vs specialty; health plan vs provider). NGC identifies at least 25 different conditions for which conflicting guidelines exist.conditions for which conflicting guidelines exist.

Complex patients with multiple co-morbidities not well addressed by most Complex patients with multiple co-morbidities not well addressed by most guidelines.guidelines.

Other Challenges for CPG DevelopersOther Challenges for CPG Developers

Resource and time constraints Resource and time constraints • many orgs cannot afford to undertake an indepth evidence synthesis (eg, using many orgs cannot afford to undertake an indepth evidence synthesis (eg, using

full GRADE approach)full GRADE approach)• updating existing guidelines: many developers unable to do this in a timely wayupdating existing guidelines: many developers unable to do this in a timely way Need for education of less sophisticated developers in how best to conduct Need for education of less sophisticated developers in how best to conduct

evidence reviews evidence reviews Available funding sources may lead to actual bias or perception of bias Available funding sources may lead to actual bias or perception of bias Guideline committees may be subject to undue influence by individuals Guideline committees may be subject to undue influence by individuals

(“eminence-based medicine”)(“eminence-based medicine”)

Additional Challenges for Users of Additional Challenges for Users of Guidelines Represented in NGCGuidelines Represented in NGC

Lack of disclosure in CPGs about financial and other conflicts of Lack of disclosure in CPGs about financial and other conflicts of interest is commoninterest is common

Lack of documentation in CPG about the details of methodologic Lack of documentation in CPG about the details of methodologic approaches used is commonapproaches used is common

Lack of any detail about composition of guideline panel is commonLack of any detail about composition of guideline panel is common Lack of awareness on part of guideline developers of what is needed Lack of awareness on part of guideline developers of what is needed

by quality measure developers (eg, what data sources will document by quality measure developers (eg, what data sources will document whether a guideline recommendation was used for a specific patient)whether a guideline recommendation was used for a specific patient)

Additional Challenges for Users of Additional Challenges for Users of Guidelines Represented in NGCGuidelines Represented in NGC

Lack of awareness by CPG developers of how to Lack of awareness by CPG developers of how to facilitate uptake of their guidelines into clinical facilitate uptake of their guidelines into clinical decision support systems (CDS) and EHRsdecision support systems (CDS) and EHRs

Delay/long time frames for release/publication of many Delay/long time frames for release/publication of many guidelines means that the evidence on which they are guidelines means that the evidence on which they are based is outdated before implementation phase based is outdated before implementation phase beginsbegins

Suggestions for Composition of PanelSuggestions for Composition of Panel

Guidelines for conditions requiring multidisciplinary approaches Guidelines for conditions requiring multidisciplinary approaches should have multidisciplinary panel compositionshould have multidisciplinary panel composition

Nonphysician perspectives (esp. nursing) often helpfulNonphysician perspectives (esp. nursing) often helpful Include individual trained in EBM from a patient advocacy Include individual trained in EBM from a patient advocacy

perspective, rather than simply a consumer repperspective, rather than simply a consumer rep Include a quality improvement expert, if QM are to be developed Include a quality improvement expert, if QM are to be developed

from the guidelinefrom the guideline Include a clinical informatics expert, if CPG recommendations are Include a clinical informatics expert, if CPG recommendations are

to be actionable and implementable in CDS and EHR systemsto be actionable and implementable in CDS and EHR systems

Other Issues to Consider:Other Issues to Consider:

There is currently no consensus on what constitutes a “good” There is currently no consensus on what constitutes a “good” guideline. A guideline may recommend interventions and practices guideline. A guideline may recommend interventions and practices that are based on good evidence, but be poorly worded so that the that are based on good evidence, but be poorly worded so that the recommendations are unclear or not implementable in a CDS. The recommendations are unclear or not implementable in a CDS. The reverse is also true.reverse is also true.

NGC identifies more than 40 attributes of guidelines. Guidelines for NGC identifies more than 40 attributes of guidelines. Guidelines for which key information is “Not stated” may contain evidence-based, which key information is “Not stated” may contain evidence-based, implementable recommendations, but lack transparency about the implementable recommendations, but lack transparency about the underlying methodology and processes.underlying methodology and processes.

Other Issues to Consider:Other Issues to Consider:

Most of the developers whose guidelines are represented in NGC (158 of Most of the developers whose guidelines are represented in NGC (158 of 204; 77%) use some sort of rating scheme to grade the underlying 204; 77%) use some sort of rating scheme to grade the underlying evidence and/or strength of the recommendations. evidence and/or strength of the recommendations.

Of these, 10 developers report using GRADE or modified GRADEOf these, 10 developers report using GRADE or modified GRADE 6 report using the USPSTF approach, either as is, or modified6 report using the USPSTF approach, either as is, or modified The great majority (142 developers) do not identify the origin of their rating The great majority (142 developers) do not identify the origin of their rating

schemes, and appear to be using schemes unique to their organizations.schemes, and appear to be using schemes unique to their organizations. A more consistent approach implemented by CPG developers would be A more consistent approach implemented by CPG developers would be

very helpful.very helpful.