Less is more: Guidelines - esim2014.org is more IV Primiano Iannone.… · ESIM Summer School ....

ESIM Summer School June 20 Friday, 2014

Less is more: Guidelines

Primiano Iannone, MD Head of Emergency Department Ospedali del Tigullio, Lavagna (GE) Italy

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

layout

2011

Farquhar CM, et al. Med J Aus 2002

How clinical guidelines are percieved

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

Physicians take decisions about (with) their patients

We take clinical decisions every day, several times per day. Often, these decisions are semiautomatic choices dictated by consolidated practices, previous professional experiences and knowledge background, local clinical habits and policies, with an heuristic approach typical of type 1 thinking («fast thinking») according to Kahneman. Often we consider also what our patients say and think about their illness. Sometimes we don’t.

However, we face often serious uncertainty about the quality of evidences on which to base our decisions, as well as to what extent individual patients’ conditions related to age, gender, morbidity, personal preferences and beliefs could modify the picture.

In these cases a sound, slow and complex rational approach («type 2 thinking», referring again to Kahneman’s terminology) is required.

So, we need searching, appraising and staying up-to-date with the best evidence, integrating it with our personal knowledge and experiences, as well aswith cost considerations, weighting risksand benefits carefeully, patientspreferences, with a clever clinicalreasoning

but can we do this efficiently ?

the exponential growth of randomized controlled trials

…we need leaner and moreefficient methods of staying up-to-datewith the evidence. Using current methods,the Cochrane Collaboration has not beenable to keep even half of its reviews up-to-date…

RCTs indexed on PubMED

1978-2013 355.272

1978: 1787 RCTs

2013: 16944 RCTsHeart failure [MeSH] RCTs 2010-2013: 1104

So, what do we need ?

• Raise the right questions in an answerable manner(PICO)

• Search for evidences efficiently• Appraising critically evidences and rating them• Integrating evidence with our experiences and previous

knowledge• Adapting evidences and deciding whehter it is worth

applying them to individual patients

JAMA, 1992; 268: 2420-25

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise withthe best available external clinical evidence from systematic research.

Evidence based medicine: what it is and what it isn't

David L Sackett, William M C Rosenberg, J A Muir Gray, R Brian Haynes, W Scott RichardsonBMJ 1996;312:71-72 (13 January)

Evidence based medicine

Evidence Based Clinical Guidelines

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

Systematic review of literature

Studies have shown that the balance of disciplines within a guideline development group has considerable influence on theguideline recommendations

Multidisciplinary development

Guidelines based on a consensus of expert opinion or on unsystematic literature surveys have been widely criticised as not reflecting current medical knowledge and being liable to bias.

Graded recommendations

Guideline recommendations are graded to differentiate between those based on strong evidence and those based on weak evidence

Miller J, Petrie J. Development of a practice guideline. Lancet 2000; 355:82–3.

What does it mean systematic review of literature?

To minimise potential sources of bias in the guideline recommendations, the literature should be identified according to an explicit search strategy, selectedaccording to defined inclusion criteria,and assessed againstconsistent methodological standards

Graded recommendations

Certainty(Level of evidence)

Strenghtof recommendations

strong

weaklow

high

Many guidelines derive(d) level of evidence almost exclusively from study type

Moreover classification of level of evidences with letters, numbers, or symbols was chaotic

ESC/AHA

SIGN

ERC 2010 guidelines

Type of study

Quality of evidenceStrength

of recommendation

And so, no RCT, no strong recommendation?

?

Sometimes trials are unethical or impossible

Sometimes trials are unethical or impossible

yet some treatments are quite effective

DC shock for ventricular fibrillation

Insulin for diabetic coma

Blood trasfusion for haemorrhagic shock

Type of study

Quality of evidence Strenghtof recommendation

Other factors ?

1.Relevance of outcomesImportance of the outcome that treatment prevents

Deep vein thrombosis : Postflebitic syndrome vs death from PulmonaryEmbolism

Atrial Fibrillation: Palpitations Vs stroke

2. Magnitude of treatment effectthe lower the NNT (=1/ARR) , more effective the treatment is

Relative risk reduction overestimates effect of treatment

3.Risk of BiasSystematic error leading to overestimate or

underestimateof true treatment effect

Also RCTs may be affected by several biases that weaken their quality

Selection biasDetection biasAttrition bias

Reporting bias…..

4. Precision

Reliable measurement of the effect size of the treatment

95 %Confidence intervals

ASA vs Placebo for stroke prevention in Atrial Fibrillation has wider 95% CIthan in Transient Ischemic Attacks

5. inconsistency

Conflicting results across trials

6. directness

differences between studied and target populationas regard of

Interventions

Patients (applicability)

Outcomes (hard vs surrogate)

Absence of head to head comparisons

type of study

Quality of evidence Strenght of recommendation

magnitude of effectrisk of bias

precision

consistency

directness

relevance

Balance of all favorable /unfavorable outcomes

patients values & preferences

resources ?

a more complex approach is needed

According to GRADE Quality of evidence must be summarized in a table

Iannone et Al,JAMA Intern Med, 2014

a more complex approach

http://www.gradeworkinggroup.org/

There are goodguidelines

However many medicalspecialty societies

haven’t adopted GRADE yet

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

Grilli et al : Lancet, 2000

Low quality of early guidelines

We have also another problem

Conflict of interests

Bias almost alwaysresults in an overestimation of benefit

and an underestimation ofharm

Is not a source of a random error

COI generates BIAS

A COI is a set of conditions in which professional judgment concerning a primary interest (such as the health and well being of a patient or the validity of research), is unduly influenced by a secondary interest -The secondary interests may be financial or nonfinancial.

Thompson DF (1993) Understanding financial conflicts of interest. NEJM 329: 573–576.

Managing COI within a guideline panel is of

paramount importanceto warrant trustworthy

recommendations

Lenzer et Al, BMJ 2013

Lenzer et Al, BMJ 2013

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

• How to decide whether a guideline istrustworthy

• Evaluation frameworks (AGREE, GIN, IOM standards)• Concordance between guidelines

Traditional approach

2011

1. Establishing Transparency

2. Management of Conflict of Interest (COI)

3. Guideline Development Group Composition

4. Clinical Practice Guideline–Systematic Review Intersection

5. Establishing Evidence Foundations for and Rating Strength ofRecommendations

6. Articulation of Recommendations

7. External Review

8. Updating

Evaluation frameworks explore the qualityof producing and reporting guidelines

NOT the trustworthiness of their recommendations

Concordance of recommendationsbetween (among) differentguidelines

Proxy of trustiworthiness

Adherence to quality standards

Or not ?

how many guidelinesfor a disease ?

A case study

• Three renowned medical specialty societies• Three guidelines on the same disease (why ?)• Same evidence base about a given drug X• One of three guideline declares to comply with GRADE• Full disclosures of conflict of interests• Another guideline declares to comply with AGREE criteria• Substantial agreement among them about the effectiveness

of drug X

Who could doubt it?

Can dronedarone be recommended for preventing recurrencesof Atrial Fibrillation ?

• Three renowned medical specialty societies (AHA, ESC, CCS)

• Three guidelines on the same disease (why ?)• Same evidence base (6 RCTs) about dronedarone• One guideline declared to comply with GRADE• Full disclosures of conflict of interests• Another guideline declared to comply with AGREE

criteria• Substantial agreement among them about the

effectiveness of dronedarone

However applying GRADE methods to the same evidencebase considered by these three guidelines….

We didn’t find relevant favorable outcomes, we found unexplained heterogeneity of results, and we could not exclude an unfavorable effect of dronedarone on mortality, with an excess of 13 (95%CI, −15 to 61) deaths per 1000 patients treated with it

Iannone et Al, JAMA Internal Medicine, 2014

Can dronedarone be recommended for AtrialFibrillation ?

• Three renowned medical specialty societies (AHA, ESC, CCS)• Three guidelines on the same disease (why ?)• Same evidence base (6 RCTs) about dronedarone• One guideline declares to comply with GRADE• Full disclosures of conflict of interests• Another guideline declares to comply with AGREE criteria• Substantial agreement among them about the effectiveness of dronedarone

NO

in presence of

• flawed methods (no GRADE guidelines)• uncontrolled conflict of interests• restricted panel compositions

Concordance of recommendations betweenguidelines and declared adherence to qualitystandards do not warrant their trustworthiness

A roadmap I would suggest…

(a very modest & weak recommendation…)

Have You a clinical problem ?

Search whether a guideline addressing relevant outcomes does exsist

Sound methodology ?(GRADE fully exploited)

No/Negligible conflict of Interest ?

Multidisciplinary involvement ?

Low risk of untrustworthiness

PICO conceptualisation

YES

YES

IOM criteriahelpful

Evaluate primary evidences carefully in case of any doubt

YES

NO

NO

NO

NO

Search for other

evidences

YES

• Overall quality of evidences• Relevance of outcomes• Type of studies• Precision• Consistency• Directness• Risk of bias• Modifiers

• Balance across all favourable and unfavourableoutcomes

• Patients’ values and preferences• Resources’ use

Follow GRADE conceptualisation

assessing their trustworthiness, too….

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

If a recommendation is trustworthy, the main issue is deciding whetherIt can be applied to our patient(s)

It is a matter of clinical judgement consideringdifferences beween ideal study conditions whereevidences were produced and real life settings in terms of

• Patients• Type of intervention• Outcomes considered• absence or presence of head-to-head comparisons

There is often some uncertainty about this

If a recommendation is untrustworthy

• It should be openly and widely presented and discussed to avoid unnecessary harm to the patients and resources’ wasting

• Reasons of untrustworthiness should be clarifiedand addressed

• An in depth GRADE based, multidisciplinar, unconflicted reassessment of flawedrecommendations should be urgently carried out to produce more firm guidelines

clinical research agenda should be prioritizedto fulfill these gaps, if relevant for our patients

problem driven research vs

curiosity driven research

When guidelines highlight the absence of firm evidences

EBM helps ethical integrity of biomedical research

• What is a clinical guideline• Why do we need clinical guidelines• How guidelines are (and should be) produced• Quality of current guidelines• How to decide whether a guideline is trustworthy• How to use a clinical guideline• The future of clinical guidelines

Clinical guidelines at their crossroad

Evolution or extinction

More trustworthy, more evidence based,unconflicted, balanced tools to inform wiseclinical decisions and manage uncertainty

Their transformation into

I didn’t mean to confuse You

But Evidence Based Medicine isan eminently creative

methodology which emphasizescritical reasoning

and not the robotic application of rules and recommendations…