Methods for searching the literature: why and how? · Methods for searching the literature: why and...

Methods for searching the

literature:

why and how?

Dr. Evi NaglerGhent University Hospital

Conflict of interest

No ties with industryNo interest in systems discussed

BUT

Professional ties with guideline organisation

N° Questions

Q every 2 to 3 patients

≈10 questions

every day!

Green ML et al. AJM 2000; 109: 218-233.

N° Questions

Ramsey PG et al. JAMA 1991; 266(8):1103-1107.

If it works, we're right

If he dies, it was something else

Experience = Knowledge?

40

44

48

52

56

60

64

5 7 9 11 13 15

Bo

ard

sco

res

Years since initial certification

Board exam results for 300 internists

Ramsey PG et al. JAMA 1991; 266(8):1103-1107.

Experience = Knowledge?

0

20

40

60

80

100

Knowledge Diagnosis Therapy

Studies (%)

Poorer

No relation

Better

Systematic Review59 studies

Choudry NK et al. Ann Intern Med 2005; 142(4): 260-273.

Experience = Knowledge?

0

20

40

60

80

100

Knowledge Diagnosis Therapy

Studies (%)

Poorer

No relation

Better

Systematic Review59 studies

Choudry NK et al. Ann Intern Med 2005; 142(4): 260-273.

N° Questions

Ramsey PG et al. JAMA 1991; 26(8):1103-1107.

10 questions?

maybe more

N° Answers?

In 4 /10 questions Time Inability to navigate available

data Forget the question...

Ely J et al. J Am Med Inform Assoc 2005; 12(2): 217-224.

Shariff SZ et al. J Nephrol 2011; 24(6): 723-732.

Difference between physicians? Trainee ≈ Specialists Nephrologists ≈ Other internists Academic hospitals ≈ Community hospitals

Where do we look?

Shariff SZ et al. J Nephrol 2011; 24(6): 723-732.

SearchPD and peritonitis

First 40 citations 2 minutes

Evidence pyramid

StudiesUnfiltered

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Evidence pyramid

Systematic Reviews

StudiesUnfiltered

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Systematic Review Methods

Question definition

Systematic search

Study selection

Data extraction

Critical appraisal

Data synthesis

≈ 6 months

Usefulness equation

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Shaughnessy AF et al. J Fam Pract 1994; 39: 489-499.

Little workHighly relevant Higly valid

Usefulness equation

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Shaughnessy AF et al. J Fam Pract 1994; 39: 489-499.

Little workHighly relevant Higly valid

Evidence pyramid

Studies

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Evidence pyramid

Studies

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Evidence pyramid

Summaries

Systematic Reviews

Studies

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Where do we look?

SearchPD and peritonitis

Where do we look?

SearchPD and peritonitis

15 seconds !

Where do we look?

SearchPD and peritonitis

Where do we look?

SearchPD and peritonitis

15 seconds !

Usefulness equation

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Shaughnessy AF et al. J Fam Pract 1994; 39: 489-499.

Little workHighly relevant Highly valid

Usefulness equation

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Shaughnessy AF et al. J Fam Pract 1994; 39: 489-499.

Little workHighly relevant Highly valid

Relevance - coverage

0

20

40

60

80

100

ACP Pier Clinical Evidence Dynamed UpToDate

Banzi 2010

Prorok 2011

Banzi R et al. J Med Int Res 2010; 12(3): e26.

Prorok JC et al. J Clin Epi 2012; 65: 1289-1295.

Relevance - coverage

0

20

40

60

80

100

ACP Pier Clinical Evidence Dynamed UpToDate

Banzi 2010

Prorok 2011

Banzi R et al. J Med Int Res 2010; 12(3): e26.

Prorok JC et al. J Clin Epi 2012; 65: 1289-1295.

No tool covers every question

Relevance - Timeliness

0 5 10 15 20 25

ACP Pier

Clinical Evidence

Dynamed

UpToDate

Time (months)

No tool is to the date

Prorok JC et al. J Clin Epi 2012; 65: 1289-1295

Usefulness equation

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Shaughnessy AF et al. J Fam Pract 1994; 39: 489-499.

Little workHighly relevant Highly valid

Usefulness equation

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Shaughnessy AF et al. J Fam Pract 1994; 39: 489-499.

Little workHighly relevant Highly valid

Validity

0

20

40

60

80

100

ACP Pier Clinical Evidence Dynamed UpToDate

Banzi 2010

Prorok 2011

Banzi R et al. J Med Int Res 2010; 12(3): e26.

Prorok JC et al. J Clin Epi 2012; 65: 1289-1295.

Validity

0

20

40

60

80

100

ACP Pier Clinical Evidence Dynamed UpToDate

Banzi 2010

Prorok 2011

Banzi R et al. J Med Int Res 2010; 12(3): e26.

Prorok JC et al. J Clin Epi 2012; 65: 1289-1295.

No tool has perfect validity

Evidence pyramid

Summaries

Systematic Reviews

Studies

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Less workLess detail

More workMore detail

Evidence pyramid

Summaries

Systematic Reviews

Studies

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Where do we look?

PD and peritonitis

15 seconds !

Evidence pyramid

Summaries

Systematic Reviews

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Randomised trials

Where do we look?

PD and peritonitis

10 minutes

Evidence pyramid

Summaries

Systematic Reviews

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Randomised trials Cohort studies Case-control studies Case-series Case reports

Where do we look?

Shariff SZ et al. J Nephrol 2011; 24(6): 723-732.

SearchPD and peritonitis

Hours

General advise

Adapted from Haynes RB. EBM 2001; 6: 36-38.

Criticims

Some questions I KNOW are not covered!

If you’re right you’ll lose 45 seconds

If you’re wrong you could spare yourself embarrasment

Criticims

Not detailed enough!

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠 =

𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑐𝑒 ∗ 𝑉𝑎𝑙𝑖𝑑𝑖𝑡𝑦

𝑊𝑜𝑟𝑘

Criticims

Dangerous if you ask me!!!

For > 50% of questions, we don’t look for an

answer, but we ALWAYS take a decision

Oh, and stupidity is always dangerous...

What to remember

Search strategically: top-to-bottom

Don’t be ashamed of using point-of-care tools

Information you can search, but no-one can do the thinking for you!