Metaanalysis qjbc

[email protected] @ljhopp @PurdueNorthwest @icebnp www.ebnp.org

Meta-Analysis – Translating the “So What” for Practice

[email protected]

mailto:[email protected]


Workshop Goals Explain how meta-analysis estimates

magnitude of effect Translate pooled effect for clinical

practice



Clinical Significance and Magnitude of Effect

Pooling studies of effect and harm

Weigh the effect with cost/resource of change

Determine precision of estimate



An example-Intensive Insulin Tx

Leuven Trial-2001 Large RCT 1548 surg ICU pts blindly

allocated to conventional tx (IV insulin if glc > 215 mg/dL) and intensive (IV insulin to maintain glc 80-110 mg/dL)

Findings: IIT reduced mortality, morbidity in critically ill surgery patients

Van den Berghe, G. et al (2001). NEJM, 345, 1359-1367





Practice changed



Hold on-Meta-analysis (2010) 7 RCTs pooled with 11,425 pts IIT did not:

Reduce 28-day mortality (OR=.95 [CI, .87-1.05]

Reduce BSI (OR=1.04 [CI, .93-1.17] Reduce renal replacement tx (OR=1.01

[CI, .89-1.13] IIT did:

Increase hypoglycemic incidents (OR=7.7 [CI, 6.0-9.9]

Marik, P.E. & Preiser, J. (2010). Chest, 137, (3)



Hold on: Meta-analysis (2010) Meta-regression revealed:

Relationship between proportion of parenteral calories and 28-day mortality

Leuven trials tx effect related to parenteral feeding

Harm? Mortality lower in control (glc 150 mg/dl) OR=.9

[CI, .81-.99] when Leuven trials removed No evidence to support IIT in general med-

surg ICU pts fed according to current guidelines (ie, enteral)

Mar

ik, P

.E. &

Pre

iser,

J. (2

010)

. Che

st,

137,

(3)



“Tight glycemic control is associated with a high incidence of hypoglycemia and an increased risk of death in patients who do not receive parenteral nutrition”.

Marik, P.E. & Preiser, J. (2010). Chest, 137, (3)

Hold on-Meta-analysis (2010)



So what does this mean? Single studies alone, rarely should

direct practice change Pooled data increases power and

precision and illuminates sources of variability

We may need multivariate analyses to uncover sources of variability



The Quality Teeter Totter:

Internal Validity

External Validity

Relationship between IV

and DV?

Used locally?

NICE

, 200

5



Apply How big is the effect? How quickly should I rush to use

evidence?



Clinical Significance and Magnitude of Effect

Pooling of homogeneous studies of effect or harm

Weigh the effect with cost/resource of change

Determine precision of estimate



Types of Outcome Data: Effect of Tx or Exposure

Dichotomous Effect/no effect Present/absent

Continuous Interval or ratio

level data B/P, HR, weight,

etc



Effect Measures-Dichotomous Outcomes

Exposure (tx, c)

Outcome

TotalPresent Absent

Tx a b a + b

Control c d c + d

Total a + c b + d a + b + c + d

Dice

nso,

Guy

att &

Cilis

ka (2

005)



Odds vs Risk What are the odds of a mother

birthing a baby boy? What is the risk of having a baby

boy? It’s about the denominator



Risk Risk

# times something happens

# opportunities for it to happen “Risk” of birthing baby boy?

One boy is born for every 2 opportunities: 1/2 = .5That is: 50% probability (risk) of having a

boy One of every 100 persons treated, has a

side-effect, 1/100 = .01



Odds Odds =

# times something happens# times it does not happen

What are the odds of birthing a boy? For every 2 births, one is a boy and one isn’t

1/1 = 1That is: odds are even

One of every 100 persons treated, has a side-effect,

1/99 = .0101



For example: Nicotine Gum and Smoking Cessation

Exposure (tx, c)

Outcome

TotalQuit No

QuitGum 1149 5179 6328

Placebo 893 7487 8380

Total 2042 12666 14708Tx event ratio=1149/6328 = 18.2 %Control event ratio = 893/8380 = 10.7 %



Effect Measures

Exposure (tx, c)

Outcome

TotalQuit No

QuitGum 1149 5179 6328

Placebo 893 7487 8380

Total 2042 12666 14708

Tx event ratio=18.2% Control event ratio = 10.7%

RR= 18.2/10.7 = 1.7% ARR= 18.7-10.7 = 7.5%**NNT = 1/.075 = 13.3



Confidence Intervals-Why? Confidence intervals are an

indication of how precise the findings are

Sample size greatly impacts the CI-i.e., the larger the sample size the smaller the CI, the greater the power and confidence of the estimate



CIs indicate: When calculated for Odds Ratio, the

CI provides the upper and lower limit of the odds that a treatment may or may not work



Interpret these CIs:Exp event ratio: 1149/5179 = .2219Control event ratio: 893/7487 = .1193Odds ratio: .2219/.1193 = 1.86

Interpret 95% CI for the OR at -1.14 to 4.86 Interpret 95% CI for the OR at 1.56 to 2.16



What about infrequent outcomes?

CER-EER/CER |CER-EER| 1/ARR

CER EER RRR ARR NNT

MRC trial 5.7% 4.3% 25% 1.4% 72 (100/1.4%)

Hypothetical trivial trial

.000057% .000043% 25% .000014% 7142857 (100/.000014%)

2 hypothetical RCTs: statins v. control on stroke in pts at risk

Straus, Glasziou, Richardson, & Haynes (2011) Great site and trial data:http://www.thennt.com/the-nnt-explained



Try one?CER EER NNT

18% 29% ?•RCT•123 smokers wishing to quit •Nicotine inhaler•Placebo inhaler•Lab verified abstinence•Followed 1 year

Example from: http://ktclearinghouse.ca/cebm/glossary/nnt/respiratoryCitation: Hjalmarson, Nisson, Sjostrom & Wiklund 91997). Arch Intern Med 157, 1721-8


http://ktclearinghouse.ca/cebm/glossary/nnt/respiratory



Try one?CER EER NNT

18% 29%•RCT•123 smokers wishing to quit •Nicotine inhaler•Placebo inhaler•Lab verified abstinence•Followed 1 year


29-18=11%1/.11=9.09





Try one?CER EER NNT

18 29 *10•RCT•123 smokers wishing to quit •Nicotine inhaler•Placebo inhaler•Lab verified abstinence•Followed 1 year


*CI for NNT: 5-483





It depends on:

What is a good NNT?

HarmBenefit

CostBenefit

TrivialSerious



What is the take-home?

http://www.medicine.ox.ac.uk/bandolier/band59/NNT1.html#Heading11

“An NNT is easy to calculate on the back of an envelope… You don't need a supercomputer, but you do need a pencil and calculator, a few neurons in active mode, and a pinch of salt.”

Small NNT: Big Effect&

It’s hard to lie with NNT-but consider the

precision



Cautions Infrequent outcomes:

Risk and odds ratios are similar However, when considering relative changes

(like relative risk reduction), you may inflate the significance. Use absolute risk reduction and NNT

CI help determine clinical significance, but are dependent on inter-subject variability and precision of measurement



Heterogeneity What model?

Fixed Effects Random Effects

Estimating heterogeneity Chi2 I2--% variability due to heterogeneity

rather than sampling error



Interpreting Forest Plots

Ram FSF et al. (2004). Non-invasive positive pressure ventilation for treatment of respiratory failure due to exacerbations of chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews, Issue 3. Art. No.: CD004104. DOI: 10.1002/14651858.CD004104.pub3.



Interrogate the Forest Plot What is the level of measurement for this

outcome? Which 2 studies contributed most to the

pooled result? Which trial had the greatest variability? Was there significant heterogeneity

amongst the trials? (p<.1) Does NPPV work?



Translate for Practice?NPPV vs placebo effect on intubation:

RR=.42 (95% CI .33-.53)NNT=4



Summary of Findings Tables (JBI 2016): What to include

Question (all elements of PICO)

RR, OR or WMD Samples size # studies GRADE quality of

evidence of each finding

Comments (including why GRADE ranking assigned)

The Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party*. Summary of Findings Tables for Joanna Briggs Institute Systematic Reviews. The Joanna Briggs Institute. 2016. http://joannabriggs.org/assets/docs/sumari/Summary_of_Findings_Tables_for_Joanna_Briggs_Institute_Systematic_Reviews-V3.pdf

JBI, 2016



Magnitude of Effect and GRADE Upgrade or downgrade level of

evidence Risk of bias Magnitude of effect

Levels High Moderate Low Very low

JBI, 2016



Magnitude of Effect and GRADELevel InterpretationHigh Confident true effect lies close to the estimate of the effectModerate Moderately confident; true effect is like to be close to the

estimate, but there is possibility that it is substantially different

Low Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect

Very low We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect

JBI, 2016



Risk of Bias and Heterogeneity ROB:

Serious: -1 Very serious: -2

Heterogeneity Serious inconsistency: -1 Very serious inconsistency: -2

JBI, 2016



Indirectness, Precision, Publication Bias

Indirectness Serious: -1 Very serious: -2

Imprecision Wide CI: -1 Very wide CI: -2

Publication Bias Likely: -1 Very likely: -2



Upgrading Magnitude of effect

Large: +1 Very large: +2

Dose response Gradient: +1

All plausible confounding factors would reduce the demonstrated effect: +1

Or create a spurious effect where results suggest no effect: +1



Translation needs to include both quality and magnitude of effect in order for clinicians to

make sense of synthesized data


Health & Medicine

Metaanalysis qjbc