Upload
yus-ani
View
212
Download
0
Embed Size (px)
Citation preview
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 1/102
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 2/102
RESEARCH QUESTION
This is the essential question the study is setup to answer. Most studies are concernedwith assessing one of four types ofobjectives:
to assess the magnitude of a health problemor health factor;
to assess the efficacy of an intervention; to asses the causal relation between one
factor or set of factors and the disease oroutcome of interest; or to asses the natural history of disease
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 3/102
HYPOTHESIS
A proportion about the relationship between two or
more factors or variables of interest. The purpose of the study is to collect data which
will allow the researcher to test the hypothesis.
In some studies the hypothesis may not be statedbut it is implicit. (However, not all studies testhypotheses. Some times descriptive studies arecalled hypothesis generating).
Hypotheses are often started in a null form, so asto allow them to be refuted.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 4/102
STUDY FACTOR
The exposure or variable of interest that ishypothesized to be related to the healthproblem, disease or outcome of interest.
As the independent variable. For example, in astudy of salt intake and its relationship toblood pressure, the study factor is salt intake.
Must be quantifiable, but it may be assessed
by a variety of means both direct and indirect,objective and subjective
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 5/102
OUTCOME FACTOR
The event or occurrence that is supposed tohave happened as a result of the influenceof the study factor.
The example: the outcome factor is bloodpressure, it being seen to be influenced bythe study factor, salt intake. The outcomefactor is also know as the dependent
variable May also be measured by a variety of
methods direct and indirect, objective andsubjective
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 6/102
POPULATION
The whole collection of units(individuals, cases, events) fromwhich a sample may be drawn; andwhich must be defined geographically
and temporally.
REFERENCE POPULATION
This is an abstract concept of thepeople to whom the researcher wishesto refer results. Also known as thetarget population.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 7/102
SOURCE POPULATION
In setting up a study it is frequentlyimpossible or impractical to include all ofthe reference population and the
researcher will have to resort to a subsetknown as the source population .
The population that is included in the study
actually comes from and its specificationswill be more limited than the referencepopulation
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 8/102
SAMPLING FRAME
To access the source population the researcherrequires a sampling frame or study frame which will represent the source population.
The simplest sampling frame would be completelisting of all the population.
From this sampling frame a sample is collectedwhich may consist of all or part of the frame. As
not all of those identified in the sample will beavailable or willing to participate in a study,those that do will comprise the final studysubjects .
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 9/102
STUDY SAMPLE
This is the subject selected byrandom or non-random means fromthe sampling frame to represent the
source population.
In some studies it is possible to studythe entire source population for the
period of the study and there is nonecessity to draw a sample
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 10/102
SAMPLING
A process for selecting a study samplefrom the sampling frame when it isimpractical to use the complete sourcepopulation.
The usual preferred method is randomsampling whereby inclusion in the studysample is decided by chance.
The objective of sampling is to select anon biased sample which can be studied.
The hierarchy of sampling can beillustrated in the sample given below:
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 11/102
Source Population- the broad group of people from
whom the subjects will be obtained, i.e., schoolchildren in the Hunter Region. Sampling Frame – the list of potential subjects
from which a sample will be drawn, i.e., class listsof all Public Schools in the greater Newcastle area
Reference Population – theabstract concept of peopleto whom the results wouldapply-in these case- Australian school children
Study Subjects Sample
Sampling Frame
Source Population
Reference Population
A researcher wants to study the prevalence ofhearing defects in Australian school children.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 12/102
Sampling Frame – the list of potentialsubjects from whom which a sample will
be drawn, i.e. class lists of all PublicSchools in the greater Newcastle area.
Sample- the subjects who are selected to
take part in the study, e.g. a randomselection of 25% of each class in each ofthe 8 schools which have been randomlyselected from the 40 in the greater
Newcastle area Sample Subjects- these provide the
data, e.g. 90% of children had parentalapproval to take part in the study.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 13/102
BIAS
Any effect at any stage of investigation (orinference) tending to produce results thatdepart systematically (i.e., not randomly)
from the truth.
Many varieties of bias have beendescribed including measurement bias,
selection bias and confounding bias
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 14/102
CONFOUNDER
A factor that distorts the apparent magnitude ofthe effect of the study factor on the outcomefactor.
A confounder is it self a determinant of theoutcome of interest, and is unequally distributedamong the subjects that are exposed or notexposed to the study factor.
For example, in a study of the influence ofmaternal nutrition on the birth weight ofnewborns (where it is hypothesised that women
with a poor diet will have low birth weightinfants), smoking is a confounding factor ifwomen with poor dietary habits are also morelikely to smoke (and it is known that smoking isof itself a risk factor for low birth weight infants)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 15/102
RISK
Probability that an event will occur, e.g.,that an individual will die or become ill within a
stated period of time or age
ATTRIBUTABLE RISK The difference between the rate of disease
in the exposed population and the rate in thenon-exposed population.
It is a measure of the amount of diseasethat might be reasonably attributed
to the exposure in question.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 16/102
RISK FACTOR An attribute or exposure that increases the
probability occurrence of disease or a specified
outcome. This defines risk factor as a determinantof the outcome, though risk factor can be used todenote association with a specified outcome thatis not necessarily causal (i.e., a risk marker or riskindicator, (e.g., smoking and positive parental
history are considered risk factors for coronaryartery disease.
PREVALENCE The total number of all individuals who have the
disease or factor or interest at a particular time orduring a particular period ( the numerator)divided by the population at risk of having thedisease at this time ( the denominator)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 17/102
INCIDENCE
The number of new cases of a diseaseoccurring in a defined population within aspecified period of time.
Usually expressed as the incidence ratewhich has the number of new cases of adisease in a defined period of time as the
numerator and the number of persons inthe stated population in which the casesoccurred as the denominator.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 18/102
RELATIVE RISK
It is one measure of the strength of effect of anexposure on the likelihood of a particularoutcome and is the ratio of risk of disease ordeath among those exposed to the study factor,to the risk of disease or death among those notexposed to the study factor.
Note, that to calculate relative risk, it isnecessary to use incidence figures for disease ordeath. As some study designs preclude thispossibility, an approximation to relative risk,called the odds ratio is used instead.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 19/102
(STATISTICAL) POWER
This is the probability that a study willdetect a statistically significant differencewhen a difference really exists.
COHORT STUDY The study population is classified according
to exposure to the study factor prior to the
development of the outcome of interest.The incidence of the outcome is thenascertained in groups with differentexposure status
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 20/102
CASE CONTROL (OR CASE REFERENT) STUDY The study sample consists of a group with the
outcome of interest (cases) and a sample of
controls representative of the same sourcepopulation from which the cases arose. Theprevalence of exposure to the study factor in eachgroup at a time prior to the development of theoutcome in the cases is compared
RANDOMIZED CONTROLLED TRIAL An epidemiological experiment whereby the
incidence of the study outcome(s) is compared ingroups assigned by chance to different
interventions and usually one group (the controlgroup) who are not exposed. The aim ofrandomization is to achieve an equal or at leastunbiased distribution of confounders, particularlyunmeasured or potential confounders in exposedand non exposed groups
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 21/102
STUDY VALIDITY
“The degree to which the inference drawnfrom a study, especially generalisationsextending beyond the study sample, are
warranted when account is taken of thestudy methods, the representativeness ofthe study sample, and the nature of the
population from which it is drawn. Twovarieties of study validity aredistinguished” (Last):
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 22/102
1. Internal Validity. The groups being compared (treatedvs. non-treated, exposed vs. non-exposed or cases vs.controls) have been selected and measurements
made in such a way that the results can be considereda good approximation to truth.
2. External Validity. The subjects in the study areselected and described in such a way that the results,
given internal validity, can be applied or generalizedoutside of the study sample.
Threats to internal validity come from systematic error orbias. Threats to external validity come from the selection
and sampling of subjects for a study . In order to seewhether a study has external or generalizability it isimportant to have a clear idea of where the subjects havecome from
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 23/102
MEASUREMENT ISSUES
Epidemiology is very much about making measurements
and about being critical of the measurements that aremade by ourselves and other. There are two issues withall measurements which will be defined here. They areValidity and repeatability .
1. Measurement validity. An expression of the degree towhich a measurement measures what it purports tomeasure, e.g., an autopsy has more validity than abiopsy or x-ray diagnosis
2. Repeatability. Critical to the validity of test is itsrepeatability, i.e., the degree of stability exhibitedwhen the measurement is repeated under identicalconditions. Most biological measurements show somevariation when repeated.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 24/102
STUDY EFFICIENCY
This addresses the issues of the cost of obtaininginformation. “ the effects or end-results achievedin relation to the effort expended in terms ofmoney, resources, and time” (Last). Case control
studies for example, are considered to be moreefficient than longitudinal studies at obtaininginformation about the effect of an exposure or anoutcome. This is because less effort needs to gointo acquiring data by comparing diseased and
non-diseased people for their exposure versuscomparing exposed and non-exposed people forwhether they develop an outcome
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 25/102
Critical Appraisal
using
Epidemiology a Basic Methods. Critical
Appraisal ( Including StudyTerminology).
University of Newcastle; 1994.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 26/102
CRITICAL APPRAISAL WORKSHEET (CAW) A General approach to critical appraisal of scientific literature
The worksheet is a matrix composed of nine rows of
questions about the paper, and three columns in which you
are:
firstly whether the information to answer the question isavailable in the paper;
secondly, whether they was a problem with the way inwhich it was handled and;
thirdly, whether the problem identified was sufficient toaffect the validity of the study
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 27/102
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
(1) (2) (3)
Can you find thisinformation in the
paper?
Is the way this was done aproblem?
Does this problemthreaten the
validity of thestudy
1. What is theresearch questionand/or hypothesis?
2. What is the studytype?
3. What is thereferencepopulation? Whatare the samplingframe andsampling method?
Is it concerned with theimpact on anintervention, causality, ordetermining the
magnitude of a healthproblem?
Is the study typeappropriate to theresearch question ?
Is the sampling frameappropriate for thereference population
Is there selection bias?
If not, how usefulare the resultsproduced by thistype of study?
Do these threatenthe externalvalidity of thestudy
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 28/102
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
(1) (2) (3)
Can you find this
information in thepaper?
Is the way this was
done a problem?
Does this problem
threaten the validity ofthe study
4. What are the study
factors and how arethey measured?
5.What are the outcome
factors and how
are they
measured?
Is there measurement
error?
a. Are all relevant
outcomes assessed?
b.Is there measurement
error?
Is measurement error
an important source ofbias?
a.How important are
omitted outcomes?
b.Is measurement error
an important source
of bias?
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 29/102
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
(1) (2) (3)
Can you find this
information in thepaper?
Is the way this was
done a problem?
Does this problem
threaten the validity ofthe study
6. Are these sources of
bias relevant to thestudy? Selection
bias, recall bias,
ascertainment
bias, confounding
bias, non-random
assignment,
incomplete follow-
up
Have steps been taken
to avoid thesebiases?
Is the internal validity
of the study threatenedby bias?
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 30/102
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
(1) (2) (3)
Can you find this
information in the
paper?
Is the way this was done a
problem?
Does this problem
threaten the
validity of thestudy
7. Are sample size
issues considered?
Is the power of the
study indicated?
8. Are statistical
methods
described?
Is the sample size sufficient
to detect clinically/
socially meaningful
differences
Are the proposed statistical
methods appropriate for
addressing the stated
research question and/orhypothesis? Are the
proposed statistical
methods appropriate for
the data?
Do the conclusions
drawn follow
logically from the
results of theanalyses?
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 31/102
CRITICAL APPRAISAL WORKSHEET (CAW) (1991)
(1) (2) (3)
Can you find this
information in the
paper?
Is the way this was done a
problem?
Does this problem
threaten the
validity of thestudy
9. What conclusions
did the authors
reach about the
research question?Did they generate
new hypotheses?
Do you agree with
the conclusions?
Do the results apply to the
population in which you
would be interested
Do you accept the
results of this
study?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 32/102
Critical Appraisal
using
The Pocket Guide to Critical AppraisalIain K. Crombie
BMJ
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 33/102
THE STANDARD APPRAISAL QUESTIONS
ARE THE AIMS CLEARLY STATED? WAS THE SAMPLE SIZE JUSTIFIED?
ARE THE MEASURMENTS LIKELY TO BE VALID ANDRELIABLE?
ARE THE STATISTICAL METHODS DESCRIBED?
DID UNTOWARD EVENTS OCCUR DURING THE STUDY? WERE THE BASIC DATA ADEQUATELY DESCRIBED?
DO THE NUMBERS ADD UP
WAS THE STATISTICAL SIGNIFICANCE ASSESSED?
WHAT DO THE MAIN FINDINGS MEAN?
HOW IS NULL FINDINS INTERPRETED? ARE IMPORTANT EFFECTS OVERLOOKED?
HOW DO THE RESULTS COMPARE WITH PREVIOUS REPORTS?
WHAT IMPLICATIONS DOES THE STUDY HAVE FOR YOURPRACTICE?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 34/102
THE COMPLETE LIST FOR APPRAISALOF SURVEYS
The essential questionsWho was studied?How was the sample obtained?What was the response rate?.
The detailed questions* Design
Are the aims clearly stated? Is the design appropriate to the stated objectives? Was the sample size justified?
Are the measurements likely to be valid and reliable? Are the statistical methods described? Is there a suggestion of haste?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 35/102
Conduct Did untoward events occur during the study?
Analysis Where the basic data adequately described?Do the numbers add up?Was the statistical significance assessed? Were the findings serendipitous?
Interpretation What do the main findings mean? How could selection bias arise?
How are null findings interpreted? Are important effects overlooked? Can the results compare with previous reports? How do the results compare with previous reports?
What implications does the study have for your practice?
THE COMPLETE LIST FOR THE
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 36/102
THE COMPLETE LIST FOR THE APPRAISAL OF COHORT STUDIES
The essential questions Who exactly has been studied? Was a control group used? Should one have been used? How adequate was the follow-up?
The detailed questions* Design Are the aims clearly stated? Is the design appropriate to the stated aims? Was the sample size justified? Are the measurements likely to be valid and reliable? Were relevant outcome measures ignored? Are the statistical methods described?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 37/102
Conduct Did untoward events occur during the study? Analysis Did the analysis allow for the passage of time? Do the numbers add up?
Were the basic data adequately described? Was statistical significance assessed? Interpretation What do the main findings mean? What else might influence the observed outcome?
How are null findings interpreted? Are important effects overlooked? How do the results compare with previous reports? What implications does the study have for your practice?
THE COMPLETE LIST FOR THE
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 38/102
THE COMPLETE LIST FOR THE APPRAISAL OF CLINICAL TRIALS
The essential questions Were treatment randomly allocated? Were all the patients accounted for? Were outcomes assessed bind? The detailed questions*
Design Are the aims clearly stated? Was the sample size justified? Are the measurements likely to be valid and reliable? Could the choice of subjects influence the size of treatment effect?
Were the ambiguities in the description of the treatment and itsadministration?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 39/102
Are the statistical methods described? Could lack of blinding introduce bias? Are the outcomes clinically relevant? Conduct How was the randomization carried out?
Did untoward events occur during the study? Analysis Were the treatment groups comparable at baseline? Were results analyzed by intention to treat? Was the statistical significance assessed?
Were the basic data adequately described? Do the numbers add up? Were side-effect reported?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 40/102
Interpretation
What do the main findings mean?
How are null findings interpreted?
Are important effects overlooked? How do the results compare with previous
reports?
What implications does the study have foryour practice?
THE COMPLETE LIST FOR THE
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 41/102
THE COMPLETE LIST FOR THE APPRAISAL OF CASE-CONTROL STUDIES
The essential questions How were the cases obtained? Is the control group appropriate? Were data collected the same way for cases and
controls? The detailed questions*Design Are the aims clearly stated? Is the method appropriate to the aims? Was the sample size justified? Are the measurements likely to be valid and reliable? Are the statistical methods described?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 42/102
Conduct Did untoward events occur during the study? Analysis Were the basic data adequately described? Do the numbers add up? Was there data-dredging? Was the statistical significance assessed?
Interpretation What do the main findings mean? Where are the biases? Could there be confounding?
How are null findings interpreted? Are important effects overlooked? How do the results compare with previous
reports? What implications does the study have for your
practice?
THE COMPLETE LIST FOR
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 43/102
THE COMPLETE LIST FOR APPRAISAL OF REVIEW PAPERS
The essential questions
How were the papers identified?
How was the quality of papers assessed?
How were the results summarized?
The detailed questions*
Design
Is the topic well defined?
Are the statistical methods described?
Conduct
Were the detailed study designs reviewed?
Was missing information sought?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 44/102
Analysis
Were the basic data adequately described?
Was publication bias taken into account?
Was heterogeneity of effect investigated?
Interpretation
What do the main findings mean? Are there other findings which merit attention?
Are the conclusions justified?
How do the findings compare with previousreports?
What implications does the study have for yourpractice?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 45/102
Evidence Based Medicine
Evidence-based medicine (EBM) requires the integration of the best
research evidence with our clinical expertise and our patient‟s
unique values and circumstances
By best research evidence we mean valid and clinically relevant
research, often from the basic sciences of medicine, but
especially from patient-centered clinical research into the
accuracy of diagnostic tests (including the clinical examination),
the power of prognostic markers, and the efficacy and safety of
therapeutic, rehabilitative, and preventive regimens. New
evidence from clinical research both invalidates previouslyaccepted diagnostic tests and treatments and replaces them
with new ones that are more accurate, more efficacious, and
safer
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 46/102
Evidence Based Medicine
By clinical expertise we mean the ability to use our clinical skills and
past experience to rapidly identify each patient‟s unique health state
and prognosis, their individual risks and benefits of potential
interventions, and their personal circumstances and expectations
By patient values we mean the unique preferences, concerns and
expectations each patient brings to a clinical encounter and which
must be integrated into clinical decisions if they are to serve the
patient.
By patient circumstances we mean their individual clinical state and
the clinical setting.
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 47/102
Critical Appraisal
using
Evidence-Based Medicine.
How to Practice and Teach EBM.Sharon E. Straus, W. Scott Richardson, Paul
Glasziou, R. Brian Haynes.
Third edition (CD room); Elseiver Churchill Living stone
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 48/102
Critical Appraisal using EBM Version needs worksheets:
1. Diagnosis and screening worksheet
2. Harm / Etiology worksheet
3. Prognosis worksheet
4. Systematic Review worksheet
5. Therapy or Prevention worksheet
6. Etc
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 49/102
1. DIAGNOSIS WORKSHEET
Are the results of this diagnostic study valid
Was there an independent , blind comparison
with a reference (“gold”) Standard of diagnosis
Was the diagnostic test evaluated in an
appropriate spectrum of patients (like those inwhom it would be used in practice)?
Was the reference standard applied regardless
of the diagnostic test result?
Was the test (or cluster of tests) validated in a
second, independent group of patients?
A th lt f thi di ti t d lid?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 50/102
Are the results of this diagnostic study valid?
Sample Calculation
Target disorder
(iron deficiency anemia)
Totals
Present Absent
Diagnostictest result(serumferritin)
Positive
(<65 mmol/L)
731
a
270
B
1001
a+b
Negative
(>65 mmol/L)
78
c
1500
d
1578
c+d
Totals 809
a+c
1770
b+d
2579
a+b+c+d
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 51/102
Sensitivity = a/(a+c) = 731/809 = 90%
Specificity = d/(b+d) = 1500/1770 = 85%
Likelihood ratio for a positive test result =LR+=sensitivity/(1-specifity)=90%/15%=6
Likehood ratio for a negative test result =LR-=(1-Sensitivity)/specificity=10%/85%=0.12
Positive predictive value = a/(a+b)=731/1001=73%Negative predictive value = d/(c+d)=1500/1578=95%
Pre-test probability (prevalence) = (a+c) / (a+b+c+d)= 809/2579=32%
Pre-test odds=prevalence/(1-prevalence)=31%/69%=0.45
Post-test odds = ( pre-test odds) x LR
Post-test probability=(post-test odds)/post-test odds+1)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 52/102
YOUR CALCULATIONS
Target disorder Totals
Present Absent
Diagnostictest result
Positive a b a+b
Negative c d c+d
Totals a+c b+d a+b+c+d
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 53/102
Can You apply this valid, important evidence abouta diagnostic test in caring for your patient?
Is the diagnostic test available,affordable, accurate, and precise in yoursetting?
Can you generate a clinically sensible
estimate of your patient‟s pre-testprobability (from personal experience,prevalence statistics, practice database,or primary studies)?
Are the study patients similar to your
own? Is it unlikely that the disease
possibilities or probabilities have
changed since the evidence was
gathered?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 54/102
Can You apply this valid, important evidence abouta diagnostic test in caring for your patient?
Will the resulting post-test probabilitiesaffect your management and help yourpatient?
Could it move you across a test-
treatment threshold ?
Would your patient be a willing partner
in carrying it out?
Would the consequences of the test helpyour patient?
Additional Notes:
2 HARM WORKSHEET
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 55/102
2. HARM WORKSHEET
Are the results of this harm study valid?
Were there clearly defined groups ofpatients, similar in all important waysother than exposure to the treatment orother cause?
Were treatments/exposures and clinicaloutcomes measured in the same ways inboth groups (was the assessment ofoutcomes either objective or blinded toexposure)?
Was the follow-up of study patientssufficiently long and complete?
Do the results satisfy some “diagnostictests for caution”?
Are the results of this harm study valid?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 56/102
Are the results of this harm study valid?
Is it clear that the exposure preceded theonset of the outcome?
Is there a dose-response gradient?Is there positive evidence from a “dechellenge-rechallenge” study?
Is the association make biological sense?
Does the association make biologicalsense?
What is the magnitude of the associationbetween the exposure and outcome?
What is the precision of the estimate ofthe association between exposure andoutcome
Are the valid results from this harm study important?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 57/102
Adverse outcome Totals
Present(Case)
Absent(control
Exposed tothetreatment
Yes
(cohort)
a b a+b
No
(cohort)
c d c+d
Totals a+c b+d a+b+c+d
In randomized trial or cohort study: relativerisk=RR=[a/(a+b)]/[c/(c+d)]
In case-control study : odds ratio (or relative odds)=OR=ad/bc
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 58/102
Should this valid, potentially importantresults changes the treatment of yourpatient?1. Do the results apply to our patient?
2. Is our patient so different from those in the study that its
results don‟t apply?
3. What are the patient‟s risks of the adverse event?
4. To calculate the NNH ( number of patients we need to
treat to harm one of them) for any odds ratio (OR) and
our patient‟s expected event rate for this adverse eventif they were not exposed to this treatment (PEER):
PEER (OR-1)+1NNH = ------------------------------
PEER(OR-1)x(1-PEER)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 59/102
5. What are our patient‟s preferences, concerns andexpectations from this treatment?
6. What alternative treatments are available?
Additional notes:
3. PROGNOSIS WORKSHEET
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 60/102
3. PROGNOSIS WORKSHEET
Are the results of this prognosis study valid ?
1. Was a defined, representative sample of patients
assembled at a common ( usually early) point in the
course of their disease?
2. Was patient follow-up sufficiently long and complete?
3. Were objective outcome criteria applied in a “blind”fashion?
4. If subgroups with different prognoses are identified:- Was there adjustment for important prognostic factors?
- Was there validation in an independent group („test
set‟) of patients?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 61/102
Are the valid results of this prognosis studyimportant?
1. How likely are the outcome over time?
2. How precise are the prognostic estimates?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 62/102
IF YOU WANT CALCULATE A CONFIDENCE INTERVAL AROUND THE MEASURE OF PROGNOSIS
Clinical measure Standard error (SE) Typical calculation ofCI
Proportion ( as in therate of someprognostic event, etc.)where:
n=the number ofpatients
P=the proportion ofthese patients who
experience the event
__________√[p x (1-p)/n]
Where p is proportionand n is number ofpatients
If p=24/60=0.4(or40%) and n 60:
_____________
SE =√[0.4x(1-0.4)/60]= 0.063 (or 6.3%)
95%CI is
40%±1.96x6.3% or
27.6% to 52.4 %
n from your evidence:
p from your evidence:
Our calculation:
SE: ___________
95% CI:
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 63/102
Can you apply this valid, important evidenceabout prognosis in caring for your patient?
1. Do the results apply to our patient?
2. Is our patient so different from those in the study that its
results cannot apply?
3. Will this evidence make a clinically important impact on
our conclusions about what to offer or tell our patient?
Additional notes:
4 SYSTEMATIC REVIEW (OF THERAPY) WORKSHEET
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 64/102
4. SYSTEMATIC REVIEW (OF THERAPY) WORKSHEET
Are the results of this systematic review valid?
1. Is this a systematic review of randomized trials?
2. Does it describe a comprehensive and detailed search forrelevant trials?
3. Were the individual studies assessed for validity?
4. Were the individual patient data used in the analysis ( oraggregate data)?
Are the valid results of this systematic review
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 65/102
The numbers in the body of the tables are the NNTs for thecorresponding odds ratio (OR) at that particular patient‟sexpected event rate (PEER)
Are the valid results of this systematic reviewimportant?
1. Are the results consistent across studies?
2. What is the magnitude of the treatment effect?
3. How precise is the treatment effect?
Translating odd ratio to NNTs
When the odds ratio (OR)<1 This table applies when a bad outcome is prevented by therapy
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 66/102
This table applies when a bad outcome is prevented by therapy
OR < 1
Patient‟sexpectedevent
rate(PEER)
0.9 0.8 0.7 0.6 0.5
0.05 2.09a 104 69 52 41b
0.10 110 54 36 27 21
0.20 61 30 20 14 11
0.30
0.40
46
40
22
19
14
12
10
9
8
7
0.50 38 18 11 8 6
0.70 44 20 13 9 6
0.90 101c 46 27 18 12d
a The relative risk reduction (RRR) here is 10%b The RRR here is 49%c For any OR, NTT is lowest when PEER=0.50d The RRR here is 9%
When the odds ratio (OR)>1
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 67/102
When the odds ratio (OR)>1This table applies both when a good outcome is increased by therapyand when a side effect is caused by therapy
OR > 1
Patient‟s
expectedeventrate(PEER)
1.1 1.2 1.3 1.4 1.5
0.05 212 106 71 54 43
0.10 112 57 38 29 23
0.20 64 33 22 17 14
0.30 49 25 17 13 11
0.40
0.50
43
42
23
22
16
15
12
12
10
10
0.70 51 27 19 15 13
0.90 121 66 47 38 32
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 68/102
Can you apply this valid, important evidence fromsystematic in caring for your patient ?
1. Do these results apply to our patient ?
2. Is our patient so different from those in the study that itsresults cannot apply?
3. Is the treatment feasible in our setting?
4. What are our patient‟s potential benefits harms from thetherapy?
5. Method I : In the OR tables above, find the intersectionof the closest OR from the systematic review and ourpatient‟s expected event rate (PEER)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 69/102
6. Method II : To calculate the NNT from any OR and PEER
1-[PEERx(1-OR)]NNT= ----------------------------
(1-PEER)xPEERx(1-OR)
7. Are our patient‟s values and preferences satisfied by theregimen and its consequences ?
8. Do we and our patient's have clear assessment of theirvalues and preferences?
9. Are they met by this regimen and its consequences?
Should you believe apparent qualitative differences
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 70/102
Should you believe apparent qualitative differencesin the efficacy of therapy in some subgroups ofpatients? Only if you can say “yes” to all of the
following ?1. Do they really make biologic and clinical sense?
2. Is the qualitative difference both clinically (beneficial forsome but useless or harmful for others) and statistically
significant?3. Was this difference hypothesized before the study began
(rather than the product of dredging the data)?
4. Was this one of just a few subgroup analyses carried outin this study?
5. Has the result been confirmed in other independentstudies?
Additional Notes:
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 71/102
5. THERAPY WORKSHEET Are the results of this single preventive ortherapeutic trial valid?
1. Was the assignment of patients to treatmentsrandomized?
2. Was the randomization list concealed?
3. Was follow-up of patients sufficiently long and complete?
4. Were all patients analyzed in the groups to which theywere randomized?
5. Were patients, clinicians, and study personal kept" blind”to treatment?
6. Were the groups treated equally, apart from theexperimental treatment?
7. Were the groups similar at the start of the trial apart
from the experimental therapy?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 72/102
Are the valid results of this randomized trialimportant?
1. What is the magnitude of the treatment effect?
2. How precise is the estimate of the treatment effect?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 73/102
SAMPLE CALCULATIONS
Occurrence of diabetic
neuropathy at 5 years amonginsulin-dependent diabetics in
the DCCT trial
Relative risk
reduction(RRR)
Absolute risk
reduction(ARR)
Number
needed totreat (NNT)
Usual insulinregimen
control eventrate (CER)
Intensiveinsulin
regimenexperimentalevent rate
(EER)
CER – EER
CER
CER - EER 1/ARR
9.6% 2.8% 9.6% - 2.8%
9.6%=71%
9.6% - 2.8%
= 6.8 %
1/6.8%
= 15 patients
95% CIa 4.4% to 9.2% 11 to 23
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 74/102
Are the valid results of this randomized trialimportant?
a95% confidence interval (CI) on an NNT
= 1/(limits on the CI of its ARR)
CER x ( 1-CER) EER x (1-EER) = ±1.96 √ ------------------------- + -----------------------------
number of control patients number of experimental patients
0.96 x 0.904 0.028 x 0.972
= ±1.96 √ ----------------------- + ---------------------730 711
= ±2.4%
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 75/102
YOUR CALCULATIONS
Relative risk
reduction(RRR)
Absolute risk
reduction(ARR)
Number
needed totreat (NNT)
CER EER CER – EER
CER
CER - EER 1/ARR
95% CI
Can you apply this valid, important evidence aboutth i i f ti t ?
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 76/102
therapy in caring for your patient ?
1. Do these results apply to our patient ?
2. Is our patient so different from those in the study that itsresults cannot apply?
3. Is the treatment feasible in our setting?
4. What are our patient‟s potential benefits harms from thetherapy?
5. Method I : f
Risk of the outcome in our patient, relative to patients inthe trial
Expressed as a decimal: __________
NNT/f = ---------/--------------=
(NNT for patients like ours)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 77/102
6. Method II : 1/(PEER xRRR)
Our patient‟s expected event rate if they received the
control treatment (PEER)= ___1/(PEER x RRR) = 1/ =
(NNT for patients like ours)
7. Are our patient‟s values and preferences satisfied by theregimen and its consequences ?
8. Do we and our patient‟s have a clear assessment of theirvalues and preferences?
9. Are they met by this regimen and its consequences?
Additional notes:
RINGKASAN
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 78/102
RINGKASAN
Setelah membaca artikel dan belum jelas
desain penelitiannya (study design dasar),maka “worksheet” yang digunakan adalah:
I.1. Critical appraisal worksheet 1991,
Epidemiology A Basic Methods
RINGKASAN
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 79/102
Setelah membaca artikel dan telah jelas desainpenelitiannya (study design dasar), maka menggunakan
The Pocket Guide to Critical Appraisal By IAIN K.Crombiedengan worksheet:II.1. The complete list for the appraisal of surveys (p:35)II.2. The complete list for the appraisal of cohort studies
(p:42)II.3. The complete list for the appraisal of clinical trials(p:49)
II.4. The complete list for the appraisal of case-controlstudies (p:55)
II.5. The complete list for the appraisal of review papers(p:62)
II.6. Baik point II.1 s/d I.5. perlu jawaban pertanyaan&……..dari I.3.1 – I.3.3
RINGKASAN
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 80/102
Setelah membaca artikel dan telah jelas desain dan jenis
penelitiannya maka menggunakan format EBM yangterdiri dari:III.1 Diagnosis and screeningIII.2 Harm / EtiologyIII.3 Prognosis
III.4 Therapy or PreventionIII.5 Etc
The conclusions have to answer these questions:
1. Is the evidence from this study valid?2. If valid, is this evidence important?3. If valid and important, can you apply this evidence
in caring for your patients
DAFTAR PUSTAKA
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 81/102
DAFTAR PUSTAKA
1. Critical Appraisal ( Including Study Terminology). Epidemiology aBasic Methods. Critical Appraisal ( Including Study Terminology).
Newcastle: University of Newcastle; 1994. p.12
2. Crombie IK. The Pocket Guide to Critical Appraisal: A Handbook
for Health Care Professionals. London: BMJ Publising group;
1996. p.23, 35, 42, 49, 62
3. Straus SE, Richardson WS,Glaziou P, Haynes RB. Evidence Based
Medicine: How to Practice and Teach EBM 3rd ed CD room;
Elseiver Churchill Living stone
GLOSSARY
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 82/102
Terms you are likely to encounter in your clinicalreading
Absolute risk reduction (ARR). See treatment effects Allocation concealment. Occurs when the person who is enrolling aparticipant into a clinical trial is unaware whether the nextparticipant to be enrolling will be allocated to the intervention orcontrol group
Case-control study. A study which involves identifying patients
who have the outcome of interest (cases) and control patientswithout the same outcome, and looking back to see if they had theexposure of interest
Case series. A report on a series of patients with an outcome ofinterest. No control group is involved
Clinical practice guideline. A systematically developed statementdesigned to assist clinician and patient decisions about appropriatehealth care for specific clinical circumstances
Cohort study. Involves identification of two groups (cohort) ofpatients, one that received the exposure of interest, and one thatdid not, and following these cohorts forward for they outcome of
interest.
Confidence interval (CI). Quantifies the uncertainly in
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 83/102
Confidence interval (CI). Quantifies the uncertainly inmeasurement. It is usually reported as 95% CI, which isthe range of values within which we can be 95% sure
that the true value for the whole population lies. Forexample, for an NNT of 10 with a 95% CI or 5 to 15, wewould have 95% confidence that the true NNT valueslies between 5 and 15
Control event rate (CER). See treatment effects
Cost- benefit analysis. Assesses whether the cost of an intervention as worththe benefit measuring both in the same units; monetaryunits are usually used
Cost-minimization analysis. If health effects areknown to be equal, only costs are analyzed and the leastcostly alternative is chosen
Cost-utility analysis. Converts health effects into
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 84/102
Cost utility analysis. Converts health effects intopersonal preferences (or sterilities) and describes howmuch it costs for some additional quality gain(e.q. cost
per additional quality gain (e.q. cost per additionalquality-adjusted life-year, or QALY)
Crossover study design. The administration of two ormore experimental therapies one after the other in aspecified or random order to the same group of patients
Cross-sectional study, The observation of definedpopulation at a single point in time or time interval.Exposure and outcome are determined simultaneously
Decision analysis ( or clinical decision analysis). The application of explicit, quantitative methods thatquantity prognoses, treatment effects, and patientvalues in order to analyze a decision under conditions ofuncertainty
Event rate. The proportion of patients in a group inwhom the event is observed Thus if of 100 patients
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 85/102
whom the event is observed. Thus, if of 100 patients,the event is observed in 27, the event rate is 0.27.Control event rate(CER) and experimental event rate(EER) are used to refer to this in control andexperimental groups of patients, respectively. Thepatient expected event rate (PEER) refers to the rate ofevents we'd expect in a patient who received notreatment or conventional treatment. See treatmenteffects.
Evidence-based health care. Extends the applicationof the principles of evidence-based medicine (seebelow) to all professions associated with health care,including purchasing management.
Evidence-based medicine (EBM). The conscientious,
explicit, and judicious use of current best evidence inmaking decisions about the care of individual patients.The practice of evidence-based medicine requires theintegration of individual clinical expertise with the bestavailable external clinical evidence from systematicresearch and our patient‟s unique values andcircumstances.
Experimental event rate (EER) See treatment
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 86/102
Experimental event rate (EER). See treatmenteffects.
Inception cohort. A group of patients who are
assembled near the onset of the target disorder Incidence The proportion of new cases of the target
disorder in the population at risk a specified timeinterval.
Intention-to-treat analysis. A method of analysis for
randomized trials in which all randomly assigned to oneof the treatments are analyzed together, regardless of ornot they completed or received that treatment, in orderto preserve randomization.
Likelihood ratio (LR). The likelihood that a given testresult would be expected in a patient with the targetdisorder compared with the likelihood that the sameresult would be expected in a patient without the targetdisorder. See Table A2.2 for calculations.
Meta-analysis. A systematic review that uses
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 87/102
Meta analysis. A systematic review that usesquantitative methods to synthesize and e the results.
n-of-1-trials. In such trials, the patient undergoes pairsof treatment periods organized so that one periodinvolves the use of the experimental treatment and theother involves the use of an alternate or placebotherapy. The patient and physician are blinded, ifpossible, and outcomes are monitored. Treatmentperiods are replicated until the clinician and patient are
convinced that the treatments are definitely different ordefinitely not different. Negative predictive value. Proportion of people with
a negative test who are free of the target disorder. Seealso likelihood ratio.
needed to treat (NNT). The inverse of the absoluterisk reduction and the number of patients that need tobe treated to prevent one bad outcome. See treatmenteffects
Odds. A ratio of the number of people incurring an
event to the number of people who have an event.
Odds ratio (OR). The ratio of the odds of having the
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 88/102
Odds ratio (OR). The ratio of the odds of having thetarget disorder in the experimental group relative to theodds in favor of having the target disorder in the control
group (in cohort studies or systematic reviews), or theodds in favor of being exposed in participants with targetdisorder divided by the odds in favor of being exposed incontrol participants (without the target disorder). SeeTable A2A for calculations.
Overview. See systematic review. Patient expected event rate (PEER). See treatment
effects.
Positive predictive value. Proportion of people with a
positive test who have the targ disorder. See alsolikelihood ratio.
Post-test odds. The odds that the patient has thetarget disorder after the test is carried out [(pre-testodds) x (likelihood )]
Post-test probability The proportion of patients with
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 89/102
Post-test probability. The proportion of patients withthat particular test result who have the target disorder[(post-test odds) / (1 + post-test odds)].
Pre-test odds. The odds that the patient has the targetdisorder before the test is carried out [(pre-testprobability) / (1 - pre-test probability)].
Pre-test probability/prevalence. The proportion of
people with the target disorder in the population at riskat a specific time (point prevalence) or time interval(period prevalence). See also likelihood ratio.
Randomization (or random allocation). Method
analogous to tossing a coin to assign patients totreatment groups (the experimental treatment isassigned if the coin lands "heads"; a conventional,"control" or "placebo" treatment is given if the coin lands"tails”
Randomized controlled clinical trial (RCT)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 90/102
Randomized controlled clinical trial (RCT).
Participants are randomly allocated into an- experimental
group or a control group and followed over time for the
variables/outcomes of interest.
Relative risk reduction (RRR). See treatment effects.
Risk ratio (RR). The ratio of the risk in the treated
group (EER) to the risk in the control group (CER); usedin randomized trials and cohort studies. RR=ERR/CER.
Also called relative risk.
Sensitivity. Proportion of people with the target
disorder who have a positive test result. It is used toassist in assessing and selecting a diagnostic
test/sign/symptom. See also likehood ratio.
SnNout When a sign/test/symptom has a high
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 91/102
SnNout. When a sign/test/symptom has a highsensitivity (5n), a negative result (N) can help rule out(out) the diagnosis. For example, the sensitivity of a
history of ankle swelling for diagnosing ascites is 93%;therefore, if person does not have a history of ankleswelling, it is highly unlikely that the person has ascites.
Specificity. Proportion of people without the target
disorder who have a negative test result. It is used toassist in assessing and selecting a diagnostictest/sign/symptom, See also likelihood ratio.
SpPin. When a sign/test/symptom has a high specificity
(Sp), a positive result (P) can help to rule in (in) thediagnosis. For example, the specificity of a fluid wave fordiagnosing ascites is 92%; therefore, if a person doeshave a fluid wave, he/she may have ascites.
Systematic review. A summary of the medical literature that usesexplicit methods to perform a comprehensive literature search and
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 92/102
explicit methods to perform a comprehensive literature search andcritical appraisal of individual studies, and that uses appropriatestatistical techniques to combine these valid studies.
Treatment effects. The evidence-based journals (Evidence BasedMedicine and ACP journal Club ) have achieved consensus on someof the terms they use to describe both the good and the bad effectsof therapy. We will bring them to life with a synthesis of threerandomized trials in diabetes which individually showed that several
years of intensive insulin therapy reduced the proportion of patientswith worsening retinopathy to 13% from 38, raised the proportionof patients with satisfactory hemoglobin A 1c levels to 60% fromabout 30%, and increased the proportion of patients with at leastone episode of symptomatic hypoglycemia to 57% from 23%. Notethat, in each case, the first number constitutes the "experimentalevent rate" (EER), and the second number the "control event rate"(CER). We will use the following terms and calculations to describethese effects treatment.
When the experimental treatment reduces the probability of a badoutcome (worsening - „diabetic retinopathy)
RRR (relative risk reduction). The proportionalreduction in rates of bad outcomes between
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 93/102
reduction in rates of bad outcomes betweenexperimental and control participants in a trial,calculated as [EER – CER]/CER accompanied by a 95%confidence interval (CI). In the case of worseningdiabetic retinopathy, [EER – CER]/CER = [13% -38%]/38% = 66%.
AAR (absolute risk reduction). The absolutearithmetic difference in rates of bad outcomes betweenexperimental and control participants in a trial,
calculated as [EER – CER], and accompanied by a 95%CI. In this case, [EER – CER] = [13% - 38%] = 25°(Thissometimes called the risk difference.)
NNT (number needed to treat). The number ofpatients who need to be treated to achieve one
additional favorable outcome, calculated as l/ARR andaccompanied by a 95% CI. In this case, 1/ARR = 1/25%= 4.
when the experimental treatment increases theprobability of a good outcome (satisfactory hemoglobin
A 1c levels)
RBI (relative benefit increase). The proportionalincrease in rates of good outcomes between experimental
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 94/102
increase in rates of good outcomes between experimentaland control patients in a trial, calculated as [EER – CER]/CER, and accompanied by a 95% confidence interval
(CI). In the case of satisfactory hemoglobin A 1c levels,[EER – CER]/CER = [60% - 30%]/30% = 100%.
ABI (absolute benefit increase). The absolutearithmetic difference in rates of good outcomes betweenexperimental and control patients in a trial, calculated as[EER – CER], and accompanied by a 95% CI. In the caseof satisfactory hemoglobin A 1c levels, [EER – CER]= [60%- 30%]= 30%
NNT (number needed to treat). The number of
patients who need to be treated to achieve one additionalgood outcome, calculated as 1/ARR and accompanied by a95% C is case, 1/ARR = 1/30% = 3.
When the experimental treatment increases theprobability of a bad outcome (episodes of hypoglycemia) )
RRI (relative risk increase). The proportionalincrease in rates of bad outcomes between experimental
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 95/102
increase in rates of bad outcomes between experimentaland control patients in a trial, calculated as [EER – CER]/CER, and accompanied by a 95% confidence interval(CI). In the case of hypoglycemic episodes, [EER-CER]= [57% - 23% ]/ 23% = 148%. (RRI is also used inassessing the impact of "risk factors" for disease.)
ARI (absolute risk increase). The absolute arithmeticdifference in rates of bad outcomes betweenexperimental and control patients in a trial, calculated as
[EER – CER], and accompanied by a 95% C1. In thecase of hypoglycemic episodes, [EER – CER] = [57% -23%] = 34%. (ARI is also used in assessing the impactof "risk factors" for disease.)
NNH (number needed to harm). The number of patients
who, if they received the experimental treatment, wouldresult in one additional patient being harmed, comparedwith patients who received the control treatment;calculated as I/AAR and accompanied by a 95% CI. Inthe case, 1/ARR=1/34% =3
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 96/102
Table A2.1
How to calculate LRs
Table A2.2
We can assume that there are four possible groups ofpatients as indicated ( a to d ) in the table From these
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 97/102
patients, as indicated ( a to d ) in the table. From thesewe can determine the “sensitivity” and specificity” asfollows:
– Sensitivity = a/(a + c) – Specificity = d/ (b + d)
We can now use these to calculate the likelihoodratio for a positive test result (LR+):
LR+ = sensitivity/ (1 – specificity)= [a/(a +c)]/[b/(b+d)] Similarly, we can calculate the likelihood ratio for a
negative test result (LR-):LR - = (1-sensitivity)/specificity
= [c/(a+c)]/[d/(b+d)] Positive predictive value = a/(a+b) Negative predictive value = d/(c+d)
Pre-test probability = (a+c)/(a+b+c+d)
SAMPLE CALCULATION
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 98/102
Suppose you have a patient with anemia and a
serum feritin of 60 mmol/L You come across asystematic review* of serum ferritin as a
diagnostictest for iron deficiency anemia, with the results
summarized in the following table.
Table A2.3
The results indicate that 90% or the patients with iron deficiencyanemia have a positive test result (serum ferritin < 65 mmol/L)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 99/102
anemia have a positive test result (serum ferritin < 65 mmol/L).This is known as the “sensitivity” and is calculated as:
Sensitivity = a/(a+c) = 731/809 = 90%
The results also show that 85% of patients who do not have irondeficiency anemia have a negative test result. This is referred to asthe “specificity”, calculated as:
Spesitivity = d/(b+d) = 1500/1770 = 85%
From the sensitivity and specificity, the positive (LR+) and negative(LR-) likelhood ratios can be determinated
LR+ = sensitivity/(1-specificity) = 90%/15% = 6LR- = (1 – sensitivity/specificity) = 10%/85% = 0.12
The results indicate that 90% or the patients with iron deficiencyanemia have a positive test result (serum ferritin < 65 mmol/L)
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 100/102
anemia have a positive test result (serum ferritin < 65 mmol/L).This is known as the “sensitivity” and is calculated as:
Sensitivity = a/(a+c) = 731/809 = 90%
The results also show that 85% of patients who do not have irondeficiency anemia have a negative test result. This is referred to asthe “specificity”, calculated as:
Spesitivity = d/(b+d) = 1500/1770 = 85%
From the sensitivity and specificity, the positive (LR+) and negative(LR-) likelhood ratios can be determinated
LR+ = sensitivity/(1-specificity) = 90%/15% = 6LR- = (1 – sensitivity/specificity) = 10%/85% = 0.12
Thus from your calculation of LR+, you determine that your patient‟spositive test result would be about six times more likely to be seenin someone with iron deficiency anemia than in someone withoutthe disorder.
Calculation of odds ratio/relative risk
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 101/102
The table below can be used to calculate the odds
ratio/relative risk for the use of trimethoprim-sulfamethoxazole prophylaxis in cirrhosis:
Table A2.4
7/21/2019 critical apraisal.ppt
http://slidepdf.com/reader/full/critical-apraisalppt 102/102