This sessions will be This sessions will be interactive. So please interactive. So please sit with people with sit with people with whom you will enjoy a whom you will enjoy a conversationconversation

PlanPlan

• GRADE backgroundGRADE background

• two stepstwo steps– confidence in estimates (quality of confidence in estimates (quality of

evidence)evidence)– strength of recommendationstrength of recommendation

• evidence profilesevidence profiles

Grading good idea, but which Grading good idea, but which grading system to use?grading system to use?

• many availablemany available– Australian National and MRCAustralian National and MRC– Oxford Center for Evidence-based Oxford Center for Evidence-based

MedicineMedicine– Scottish Intercollegiate Guidelines (SIGN)Scottish Intercollegiate Guidelines (SIGN)– US Preventative Services Task ForceUS Preventative Services Task Force– American professional organizationsAmerican professional organizations

• AHA/ACC, ACCP, AAP, Endocrine society, etc....AHA/ACC, ACCP, AAP, Endocrine society, etc....

• cause of confusion, dismaycause of confusion, dismay

Dilemma: proliferation of systemsDilemma: proliferation of systemsSolution: common international Solution: common international

grading system?grading system?

• GRADE (GRADE (GGrades of rades of rrecommendation, ecommendation, aassessment, ssessment, ddevelopment and evelopment and eevaluation)valuation)

• international groupinternational group– Australian NMRC, SIGN, USPSTF, WHO, NICE, Australian NMRC, SIGN, USPSTF, WHO, NICE,

Oxford CEBM, CDC, CCOxford CEBM, CDC, CC

• ~ 30 meetings over last eleven years~ 30 meetings over last eleven years• (~10 – 50 attendants)(~10 – 50 attendants)

Grading system – for what?Grading system – for what?

• interventionsinterventions– management strategy 1 versus 2management strategy 1 versus 2

• what grade is what grade is notnot about about– individual studies (body of eidence) individual studies (body of eidence)

– prognostic questionsprognostic questions• in patients with heart failure is high uric acid in patients with heart failure is high uric acid

associated with increased riskassociated with increased risk

– diagnostic accuracy questionsdiagnostic accuracy questions• in lung cancer, what is the accuracy of CT scanning of in lung cancer, what is the accuracy of CT scanning of

the mediastinumthe mediastinum

GRADE UptakeGRADE UptakeAgencia sanitaria regionale, Bologna, Italia Agency for Health Care Research and Quality (AHRQ)Allergic Rhinitis and Group - Independent Expert PanelAmerican Association for the study of liver diseasesAmerican College of Cardiology FoundationAmerican College of Chest PhysiciansAmerican College of Emergency PhysiciansAmerican College of PhysiciansAmerican Endocrine Society American Gastroenterology AssociationAmerican Society for Colposcopy and Cervical Pathology American Society of Gastrointestinal EndoscopyAmerican Society of Interventional Pain PhysiciansAmerican Thoracic Society (ATS)Ärztliches Zentrum für Qualität in der Medizin - Germany Austrian Ludwig Boltzmann Institute for HTA BMJ Clinical Evidence British Medical Journal Canadian Agency for Drugs and Technology in HealthCanadian Cardiovascular SocietyCanadian Society of NephrologyCanadian Task Force on Preventive Health CareCentre for Disease Control Committee on Immunization Practices COMPUS at CADTH - Canada Centers for Disease ControlCochrane Collaboration Critical Ultrasound JournalDutch Institute for Healthcare Improvement CBOEBM Guidelines Finland Emergency Medical Services for Children National Resource

Center European Association for the Study of the LiverEuropean Monitoring Centre for Drugs and Drug

AddicatonEuropean Respiratory SocietyEuropean Society of Thoracic SurgeonsEvidence-based Nursing Sudtirol, Alta Adiga, ItalyEvidence-Based Tuberculosis Diagnosis (tbevidence.org) -

Canada

Finnish Office of Health Technology Assessment German Agency for Quality in MedicineGerman Center for Evidence-based Nursing "sapere aude" - Germany Heelth Inspectorate for ScotlandInfectious Disease Society of America Institute for Clinical Systems ImprovementJapanese Society of Oral and Maxillofacial Radiology Japanese Society for Temporomandibular JointJoslin Diabetes CenterJournal of Infection in Developing CountriesKaiser PermanenteKidney Disease International Guidelines Organization Ministry of Health and Long-term Care, OntarioNational and Gulf Centre for Evidence-based MedicineNational Board of Health and Welfare - Sweden National Institute for Clinical Excellence (NICE)National Kidney FoundationNHS Quality Improvement Scotland - UKNorwegian Knowledge Centre for the Health ServicesOntario MOH Medical Advisory SecretariatPanama and Costa Rica National Clinical Guidelines ProgramPolish Institute for EBMScottish Intercollegiate Guideline Network (SIGN)Society of Critical Care MedicineSociety of Pediatric Endocrinology Society of Vascular SurgerySpanish Society of Family Practice (SEMFYC) Stop TB Diagnostic Working GroupSurviving sepsis campaign Swedish Council on Technology Assessment in Health CareSwedish National Board of Health and Welfare University of Pennsylvania Health System for EB Practice UpToDate WINFOCUSWorld Allergy OrganizationWorld Health Organization (WHO)

GRADE uptake

What are we grading?What are we grading?

• two componentstwo components

• confidence in estimate of effect confidence in estimate of effect adequate to support decision adequate to support decision (quality of body of evidence)(quality of body of evidence)

• high, moderate, low, very lowhigh, moderate, low, very low

• strength of recommendationstrength of recommendation• strong and weakstrong and weak

Confidence in estimateConfidence in estimate(quality of evidence) (quality of evidence)

no confidence

totally confidentHigh

Moderate

Low

Very Low

Determinants of Determinants of confidenceconfidence

• RCTs start highRCTs start high

• observational studies start low observational studies start low

• what can lower confidence?what can lower confidence?

• risk of biasrisk of bias• inconsistencyinconsistency• indirectnessindirectness• imprecisionimprecision• publication biaspublication bias

What can raise What can raise confidence?confidence?

• large magnitude can rate up one levellarge magnitude can rate up one level– very large two levelsvery large two levels

• common criteriacommon criteria– everyone used to do badlyeveryone used to do badly– almost everyone does wellalmost everyone does well– quick actionquick action

• hip replacement for hip osteoarthritiship replacement for hip osteoarthritis

Dose-response gradientDose-response gradient

• childhood lymphoblastic leukemiachildhood lymphoblastic leukemia

• risk for CNS malignancies 15 years risk for CNS malignancies 15 years after cranial irradiationafter cranial irradiation

• no radiation: 1% (95% CI 0% to 2.1%) • 12 Gy: 1.6% (95% CI 0% to 3.4%) • 18 Gy: 3.3% (95% CI 0.9% to 5.6%).

Confidence assessment criteriaConfidence assessment criteria

Quality Assessment

Summary of Findings

QualityRelative Effect

(95% CI)

Absolute risk difference

OutcomeNumber of

participants(studies)

Risk of Bias

Consistency Directness PrecisionPublication

Bias

Myocardial infarction

10,125(9)

No serious limitations

No serious imitations

No serious limitations

No serious limitations

Not detected

High0.71

(0.57 to 0.86)1.5% fewer

(0.7% fewer to 2.1% fewer)

Mortality10,205

(7)No serious limitations

Possiblly inconsistent

No serious limitations

ImpreciseNot

detectedModerate

or low1.23

(0.98 – 1.55)

0.5% more(0.1% fewer

to 1.3% more)

Stroke10,889

(5)No serious limitaions

No serious limitations

No serious limitations

No serious limitations

Not detected

High2.21

(1.37 – 3.55)0.5% more

(0.2% more to 1.3% more0

Beta blockers in non-cardiac surgery

Strength of RecommendationStrength of Recommendation

• strong recommendationstrong recommendation– benefits clearly outweigh risks/hassle/costbenefits clearly outweigh risks/hassle/cost– risk/hassle/cost clearly outweighs benefitrisk/hassle/cost clearly outweighs benefit

• what can downgrade strength?what can downgrade strength?

• low confidence in estimates low confidence in estimates

• close balance between up and downsidesclose balance between up and downsides

Significance of strong vs Significance of strong vs weakweak

• variability in patient preferencevariability in patient preference– strong, almost all same choice (> 90%)strong, almost all same choice (> 90%)– weak, choice varies appreciablyweak, choice varies appreciably

• interaction with patientinteraction with patient– strong, just inform patientstrong, just inform patient– weak, ensure choice reflects valuesweak, ensure choice reflects values

• use of decision aiduse of decision aid– strong, don’t botherstrong, don’t bother– weak, use the aidweak, use the aid

• quality of care criterionquality of care criterion– strong, considerstrong, consider– weak, don’t considerweak, don’t consider

ConclusionConclusion

• clinicians, policy makers need summariesclinicians, policy makers need summaries– confidence in estimatesconfidence in estimates– strength of recommendationsstrength of recommendations

• explicit rulesexplicit rules– transparent, informativetransparent, informative

• GRADEGRADE– transparent, systematictransparent, systematic– increasing wide adoptionincreasing wide adoption