Upload
filchibuff
View
222
Download
0
Embed Size (px)
Citation preview
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
1/33
CriticallyAppraised
Topics:
An OverviewMarishiel Mejia-Samonte, MD, DFM
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
2/33
Evidence-Based Medicine
Systematic approach
Acquisition
Appraisal
Application
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
3/33
Evidence-Based Medicine
more than 12,000 new articles, includingpapers, are added to the MEDLINEdatabase each week
'information overload
develop efficient skills in critical appraisal
Young, JM & Solomon, MJ. How to Critically Appraise an Article
Nat Clin Pract Gastroenterol Hepatol. 2009;6(2):82-91.
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
4/33
Evidence-Based Medicine
focus only on the highest-quality studiesthat will guide their clinical practice
extrapolate information when necessaryfrom studies of less rigorous design if high-quality trials are unavailable
Young, JM & Solomon, MJ. How to Critically Appraise an Article
Nat Clin Pract Gastroenterol Hepatol. 2009;6(2):82-91.
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
5/33
Critical Appraisal
Systematic process used to identify thestrengthsand weaknessesof a research
article in order to assess the usefulnessand validityof research findings
The rules of evidence vary with
circumstances
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
6/33
Critical Appraisal
NO 'gold-standard' instrument for criticalappraisal exists
criteria used are not static evolve with improvements in
understanding of the important sources ofbias inherent in different study designsand increased awareness of the potentialinfluence of other nonmethodologicalfactors, such as conflicts of interest
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
7/33
Most Important Componentsof a Critical Appraisal
Evaluation of the appropriateness of the studydesign for the research question
Careful assessment of the keymethodological features of this design
Other factors that also should be considered:
Suitability of the statistical methods used andtheir subsequent interpretation
Potential conflicts of interest Relevance of the research to one's own
practice
Young, J.M. and Solomon, M.J., 2008
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
8/33
10 Basic Questions Asked
1. Is the research question relevant?
Always the first question asked
Subjective opinion
What might be crucial to some will beirrelevant to others
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
9/33
10 Basic Questions Asked
2. Does it add something new?
New ideas or knowledge are usually
based on previous work Breakthrough researches are rare
Researches that make incrementaladvances are also valuable
Repeating a study can increase its validity Extending the original article to a new
population
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
10/33
10 Basic Questions Asked
3. What type of research question does thestudy pose?
Components of a well-developedresearch question:
P = the group or population of patients
I/C = the studied parameter (e.g. a therapy
or clinical intervention)
O = the outcomes of interest
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
11/33
10 Basic Questions Asked4. Was the Study Design Appropriate for theResearch Question?
Heirarchy of study designs
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
12/33
10 Basic Questions Asked4. Was the Study Design Appropriate for theResearch Question?
However, in some circumstances, RCTs are
either not feasible or considered ethicallyinappropriate. These issues are morecommon in nonpharmaceutical trials, such asthose of surgical procedures.
When specific design is not feasible, thereasons that preclude its use will determinethe type of alternate study design that canbe used
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
13/33
10 Basic Questions Asked5. Did the Study Methods Address the KeyPotential Sources of Bias?
Presence of bias means that the results of astudy have deviated from the truth
chance (e.g. a random error) = Random errordoes not influence the results in any particulardirection, but it will affect the precision of thestudy
study methods (systematic bias) = Systematic
bias has a direction and results in theoverestimation or underestimation of the'truth'.
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
14/33
10 Basic Questions Asked
6. Was the Study Performed in Line with the OriginalProtocol?
Deviations affect the validity or relevance of a study
Failure to recruit the planned number of participants potentially reduces the extent to which the results of
the study can be generalized
reduces the power of the study
Changes to the inclusion and exclusion criteria Variation in the provided treatments or interventions
Changes to the employed techniques or technologies,
and changes to the duration of follow-up.
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
15/33
10 Basic Questions Asked
7. Does the Study Test a Stated Hypothesis?
A hypothesis is a clear statement of what
the investigators expect the study to find Hypothesis states the research question in
a form that can be tested and refuted
Null hypothesis states that the findings of a
study are no different to those that wouldhave been expected to occur bychance.
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
16/33
10 Basic Questions Asked8. Were the Statistical Analyses PerformedCorrectly?
Difficult for nonstatisticians
All quantitative research articles shouldinclude a segment within their 'Method'section that explains the tools used in thestatistical analysis and the rationale for thisapproach
Original data should be presented in such away that readers can check the statisticalaccuracy of the paper.
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
17/33
10 Basic Questions Asked
9. Do the Data Justify the Conclusions?
Are conclusions reasonable on the basis of
the accumulated data
Issues to be wary of are whether the authors
generalized their findings to broader groups of
patients or contexts than was reasonable
given their study sample, and whether
statistically significant associations have been
misinterpreted to imply a cause and effect
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
18/33
10 Basic Questions Asked10. Are There any Conflicts of Interest?
Researchers are in a position to decide which
studies will be conducted in their unit, whichpatients will be invited to participate in astudy and whether certain clinicaloccurrences should be reported as adverseevents
Decisions require researchers to act withintegrity and not for personal or institutionalgain
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
19/33
Parts of a Critical Appraisal
Internal Validity
What are the Results?
External Validity / Applicability
Th
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
20/33
Therapy
Was the assignment of patients totreatment randomised?
Were the groups similar at the start of the
trial?
Aside from the allocated treatment, weregroups treated equally?
Were all patients who entered the trial
accounted for? And were they analysedin the groups to which they wererandomised
Internal Validity
Th
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
21/33
How large was thetreatment effect?
RR ARR
RRR
NNT
How precise wasthe estimate of thetreatment effect?
Will the results helpme in caring for my
patients?
Results
Therapy
ExternalValidity
Or
Applicability
Di ti
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
22/33
Internal Validity
Was the diagnostic test evaluated in arepresentative spectrum of patients (likethose in whom it would be used in
practice)?Was the reference standard applied
regardless of the index test result?
Was there an independent, blind
comparison between the index test andon appropriate reference (gold)standard of diagnosis?
Diagnostics
Di ti
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
23/33
Are testcharacteristics
presented? Sensitivity
Specificity
Positive Predictive
Value Negative
Predictive Value
Diagnostics
Were the methodsfor performing thetest described in
sufficient detail topermit replication?
Results ExternalValidityOr
Applicability
P i
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
24/33
Internal ValidityWas the defined representative sample of
patients assembled at a common (usuallyearly) point in the course of their disease?
Was patient follow-up sufficiently long andcomplete?
Were outcome criteria either objective orapplied in a blind fashion?
If subgroups with different prognoses wereidentified, did adjustment for importantfactors take place?
Prognosis
P i
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
25/33
How likely are theoutcomes over
time? How precise are
the prognosticestimates?
Prognosis
Can I apply thisvalid, importantevidence about
prognosis to mypatient?
Results ExternalValidityOr
Applicability
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
26/33
Internal Validity
What question (PICO) did the systematicreview address?
Is it unlikely that important, relevant studieswere missed?
Were the criteria used to select articles forinclusion appropriate?
Were the included studies sufficiently validfor the type of question asked?
Were the results similar from study to study?
Systematic Review
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
27/33
How are the resultspresented?
Will the results helpme in caring for mypatients?
Systematic Review
Results
External Validity
OrApplicability
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
28/33
What do we do with theappraised articles?
discuss the published informationgathered from these appraised articles
regarding a particular subject matter
APPLY IT TO YOUR PATIENTS
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
29/33
WORKSHOP1. Divide yourselves into 8 groups composed of
10 to 11 members per group (this will be yourpermanent grouping for Research 1).
2. Articles will be provided to the differentgroups.
3. Determine if the appraisal questions can beanswered or not using the abstract alone,
using full text, not at all
Therapy
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
30/33
Therapy
Diagnostics
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
31/33
Diagnostics
Prognosis
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
32/33
Prognosis
7/21/2019 Critically Appraised Topics 23June2014 (1) (2)
33/33
Systematic Review