CRITICAL APPRAISAL OF LITERATURE

Why do we need to appraise the available

literature?

> Volume of information available for patient management

> Variety of evidence available on one topic (e.g., books, journals)

> Want to give the best possible treatment to our patients

OBJECTIVES

1. To understand what is evidence-based medicine (EBM)

2. To apply the rule of

evidence in order to decide on the validity of an article on therapy / diagnostic test

EVIDENCE-BASED MEDICINE (EBM)

new paradigm in clinical decision-making OLD PARADIGM: OLD PARADIGM:

> observations from clinical practice> observations from clinical practice> basic disease mechanisms and > basic disease mechanisms and pathophysiologic principlespathophysiologic principles> traditional medical training & common > traditional medical training & common sensesense> content expertise & clinical experience: > content expertise & clinical experience: guidelines for clinical practiceguidelines for clinical practice

> NEW PARADIGM:> NEW PARADIGM: > clinical experience: important & > clinical experience: important & necessary, but not the only elements of necessary, but not the only elements of becoming a competent physicianbecoming a competent physician > understanding of basic disease > understanding of basic disease mechanisms are necessary, but mechanisms are necessary, but insufficientinsufficient > understanding rules of evidence: > understanding rules of evidence: correct interpretation of available correct interpretation of available literatureliterature

> application of the rules of evidence to > application of the rules of evidence to evaluate the available clinical literature evaluate the available clinical literature to answer a clinical dilemmato answer a clinical dilemma

CLINICAL SCENARIOCLINICAL SCENARIO

45/M, smoker45/M, smoker government employeegovernment employee would like to stop smoking, but does would like to stop smoking, but does

not want to use any pharmacologic not want to use any pharmacologic interventionintervention

given health education and advised to given health education and advised to follow-up follow-up

DILEMMA: Among adult smokers, what DILEMMA: Among adult smokers, what is the effectiveness of counseling in is the effectiveness of counseling in smoking cessation using a randomized smoking cessation using a randomized controlled trial?controlled trial?

APPRAISING AN ARTICLE ON APPRAISING AN ARTICLE ON THERAPEUTIC THERAPEUTIC

INTERVENTIONSINTERVENTIONSA.A. RELEVANCE QUESTIONRELEVANCE QUESTION

1. Is the objective of the article 1. Is the objective of the article comparing therapeutic comparing therapeutic interventions similar to your interventions similar to your clinical dilemma?clinical dilemma?

a. population of the studya. population of the studyb. intervention & comparative b. intervention & comparative interventionsinterventionsc. outcome of the studyc. outcome of the study

B. VALIDITY GUIDESB. VALIDITY GUIDES 1. Was the assignment to the different 1. Was the assignment to the different treatment groups randomized?treatment groups randomized?

2. Was follow-up complete?2. Was follow-up complete?

If with If with drop-outsdrop-outs, for results to be , for results to be acceptable:acceptable: a. Intention to Treat analysis a. Intention to Treat analysis b. < 20% of subjects b. < 20% of subjects c. explanation of why subjects dropped out c. explanation of why subjects dropped out

3. Was the study blinded?3. Was the study blinded?

a. Single-blind a. Single-blind b. Double-blind b. Double-blind

4. Were the outcome measurements 4. Were the outcome measurements clearly described and determined in clearly described and determined in the same way between the same way between interventions?interventions?

Can the study be replicated using the Can the study be replicated using the same methodology & measurements?same methodology & measurements? Are these the outcomes you are Are these the outcomes you are aiming for?aiming for?

5. Over-all, is the study valid?5. Over-all, is the study valid?

C. WHAT ARE THE RESULTS?C. WHAT ARE THE RESULTS? 1. What are the results?1. What are the results?

Risk in treatment (Rt) Risk in treatment (Rt) = # of pxs who did = # of pxs who did not get well in the treatment group/total # of not get well in the treatment group/total # of pxs in the treatment grouppxs in the treatment group

Risk in control (Rc) Risk in control (Rc) = # of pxs who did not = # of pxs who did not get well in the control group/total # of pxs in get well in the control group/total # of pxs in the control groupthe control group

Relative Risk (RR) Relative Risk (RR) = Rt/Rc= Rt/Rc

Relative Risk Reduction (RRR)Relative Risk Reduction (RRR) = ( 1 – RR = ( 1 – RR x 100%)x 100%)

Absolute Risk Reduction (ARR)Absolute Risk Reduction (ARR) = Rc – Rt = Rc – Rt

Number Needed to Treat (NNT) = Number Needed to Treat (NNT) = 1/ARR1/ARR

Cost Effectiveness Cost Effectiveness = NNT x dose x days x = NNT x dose x days x costcost

* RR = 1 : * RR = 1 : no difference between treatment no difference between treatment & control& control RR > 1 : RR > 1 : treatment is harmfultreatment is harmful RR < 1 : RR < 1 : treatment is beneficialtreatment is beneficial

2. Are the results reliable?2. Are the results reliable?

Confidence Interval (CI) Confidence Interval (CI) : > or = 95% : > or = 95% p-valuep-value : < or = 0.05 : < or = 0.05

D. APPLICABILITY GUIDESD. APPLICABILITY GUIDES 1. Are the medical, social, & economic 1. Are the medical, social, & economic resources needed to administer the resources needed to administer the treatment available in your setting?treatment available in your setting?

2. In your perception, were the treatment 2. In your perception, were the treatment and its outcome as measured in the and its outcome as measured in the article preferred by the patient & his article preferred by the patient & his family?family?

3. Do you think that the patient, family, 3. Do you think that the patient, family, and/or community will be willing to accept and/or community will be willing to accept and pay for the treatment in question?and pay for the treatment in question?

E. RESOLUTION OF THE THERAPEUTIC DILEMMAE. RESOLUTION OF THE THERAPEUTIC DILEMMA

CLINICAL SCENARIOCLINICAL SCENARIO

35/M, employee35/M, employee showed hematuria during a routine showed hematuria during a routine annual company PEannual company PE with known history of recurrent UTI in with known history of recurrent UTI in the past yearthe past year PWI: UrolithiasisPWI: Urolithiasis KUB-IVP or KUB-UTZ?KUB-IVP or KUB-UTZ?

DILEMMA: Among patients with DILEMMA: Among patients with hematuria, hematuria, will a KUB-UTZ detect urolithiasis vs. will a KUB-UTZ detect urolithiasis vs. KUB-IVP?KUB-IVP?

APPRAISING AN ARTICLE ON APPRAISING AN ARTICLE ON A DIAGNOSTIC TESTA DIAGNOSTIC TEST

SCREENING QUESTIONSCREENING QUESTION1.1. Was the objective of the paper Was the objective of the paper

relevant to your clinical relevant to your clinical question?question?

VALIDITYVALIDITY1. Was a representative sample of 1. Was a representative sample of

subjects to whom the diagnostic subjects to whom the diagnostic test is supposed to be test is supposed to be administered gathered?administered gathered?

2. Was the diagnostic test and reference 2. Was the diagnostic test and reference standard described in detail to permit standard described in detail to permit reliable replication?reliable replication?

Methodology must be detailed enough (e.g., Methodology must be detailed enough (e.g., patient preparation, ideal conditions to carry patient preparation, ideal conditions to carry out test) for reliable replicationout test) for reliable replication

3. Was there an independent comparison 3. Was there an independent comparison with the reference standard? with the reference standard?

Verification bias: increases the accuracy of Verification bias: increases the accuracy of the testthe test

4. Was the reading or interpretation of 4. Was the reading or interpretation of the diagnostic test or reference standard the diagnostic test or reference standard done independently?done independently?

RESULTSRESULTS1. What are the likelihood ratios for the 1. What are the likelihood ratios for the different test results?different test results?

DISEASEDISEASE TESTTEST

Sensitivity Sensitivity = TP / TP + FN= TP / TP + FN Specificity Specificity = TN / TN + FP= TN / TN + FP

(+) Likelihood Ratio (+) Likelihood Ratio = Sensitivity / 1 – = Sensitivity / 1 – SpecificitySpecificity

(--) Likelihood Ratio (--) Likelihood Ratio = 1 – Sensitivity / = 1 – Sensitivity / SpecificitySpecificity

Positive Predictive Value (PPV) Positive Predictive Value (PPV) = TP / TP + = TP / TP + FPFP

Negative Predictive Value (NPV) Negative Predictive Value (NPV) = TN / TN = TN / TN + FN+ FN

APPLICABILITYAPPLICABILITY1. Are the medical, social, & 1. Are the medical, social, & economic resources needed to economic resources needed to perform the diagnostic test perform the diagnostic test available in your setting?available in your setting?

2. Was the diagnostic test & 2. Was the diagnostic test & accuracy in the evidence preferred accuracy in the evidence preferred by the patient & the family?by the patient & the family?

3. If the physician gives the 3. If the physician gives the diagnostic test to a similar patient diagnostic test to a similar patient in the community, will he get the in the community, will he get the same result to most of them?same result to most of them?

4. Is the patient, family, and 4. Is the patient, family, and community willing to accept and community willing to accept and pay for the diagnostic test?pay for the diagnostic test?

RESOLUTION OF THE SCENARIORESOLUTION OF THE SCENARIO

THANK YOUTHANK YOU