Diagnosis: Highlights

Cita Rosita Sigit PrakoeswaDepartment of Dermato Venereology Dr Soetomo Hospital Faculty of Medicine, Airlangga University, SurabayaTropical Disease Center, Airlangga University, Surabaya

What is diagnosis ?Increase certainty about presence/absence of diseaseDisease severityMonitor clinical courseAssess prognosis risk/stage within diagnosisPlan treatmentScreeningEpidemiology

Knottnerus, BMJ 2002

By the end of this session,you should be able to.describe and illustrate key measures of diagnostic test performancerepresent diagnostic test performance

EBM Process(Lang, 2000)*

What should I do about this condition or problem?What causethe problem?Does this personhave the condition or problem?Who will getthe condition or problem?How commonis the problem?What are thetype of problem?INTERVENTIONPROGNOSIS/RISK FACTORSDIAGNOSISPROGNOSIS FACTORSFREQUENCY & RATEPHENOMENA / THOUGHTSCLINICALQUESTION*

ACQ Diagnosis (PICO)

Patient / Problem / PopulationIntervention(Index)ComparisonOutcomeIn an otherwise healthy 7-year-old boy with sore throathow does the clinical examcompare to throat culturein diagnosing GAS infection?

ResearcherInvolvementLongitudinalCross-sectionalResearchGoalResearchApproachControlled?Randomized?ResearchFocusClinical Manifestation / Diagnosis / Prognosis / Therapy / Review1324experimentalobservasionalControl (+)Control (-)*

Hierarchy of study designs*

Basic Principles (1)Ideal diagnostic tests right answers:(+) results in everyone with the disease and( - ) results in everyone elseUsual clinical practice:The test be studied in the same way it would be used in the clinical settingObservational study, and consists of:Predictor variable (test result)Outcome variable (presence / absence of the disease)

Basic Principles (2)Sensitivity, specificityPrevalence, prior probability, predictive valuesLikelihood ratiosDichotomous scale, cutoff points (continuous scale)Positive (true and false), negative (true & false)ROC (receiver operator characteristic) curve

What is the reason that there are many parameters in diagnostic test?PrevalenceSensitivity (%)Specificity (%)LR+LR-PPV (%)NPV (%)Pre-test OddsPost-test OddsPre-test Probability (%)Post-test Probability (%)

Disease(+)Disease(-)TotalTest (+)True posaFalse posba+bTest (-)False negcTrue negdc+dTotala+cb+da+b+c+d

METHOD 1: NATURAL FREQUENCIES TREE

Population1.000

IN EVERY 1.000 PEOPLE, 200 WILL HAVE THE DISEASEIf these 1000 people are representative of the population at risk, the assessed rate of those with the disease (20%) represents the PREVALENCE of the disease it can also be considered the PRE-TEST PROBABILITY of having the disease

Disease +200

Disease -800

Population1.000

Sensitivity

The proportion of people who truly have a designated disorder who are so identified by the test.Sensitive tests have few false negatives. When a test with a high Sensitivity is Negative, it effectively rules out the diagnosis of disease. SnNout

In other words, the sensitivity is 190/200=95%Test Alergi dengan Uji KulitSensitivitas 95 %, artinya:SnNout: bila hasil uji kulitnya (-): 95% out (dia bukan penderita alergi )

Sensitivity

Disease +200

Disease -800

Test +190

Test -10

Population1.000

The proportion of people who are truly free of a designated disorder who are so identified by the test. Specific tests have few false positivesWhen a test is highly specific, a positive result can rule in the diagnosis. SpPin

Specificity

Test Alergi dengan Uji KulitSpesifitas 96 % artinya: SpPin: bila hasil uji kulitnya (+): 96% in (dia penderita alergi)

In other words, the specificity is 768/800= 96%

Specificity

Disease +200

Disease -800

Test +190

Test -10

Population1000

Test +32

Test -768

CASESNON-CASESSensitivity & SpecificityNegativePositiveDegree of positivity on test% of GroupDISEASEDNON-DISEASEDTest cut-offFALSE NEGATIVESFALSE POSITIVES

Numeric? (complex)

Sensitivity & SpecificitySensitivity and Specificity are usually considered properties of the test rather than the setting, and are therefore usually considered to remain constant.

However, sensitivity and specificity are likely to be influenced by complexity of differential diagnoses and a multitude of other factors (spectrum bias).

Sensitivity & Specificity Positive & Negative Predictive Value For sensitivity and specificity, the reference variable (denominator) is the DISEASEFor predictive value, the reference variable (denominator) is the TEST

Pre Test & Post Test ProbabilityPre-test ProbabilityThe probability of the target condition being present before the results of a diagnostic test are available. (prevalence)Post-test ProbabilityThe probability of the target condition being present after the results of a diagnostic test are available. (Positive Predictive Value)

POSITIVEPREDICTIVEVALUE = 190/222=86 %This is also the POST-TEST PROBABILITY of having the diseasePositive Predictive ValueTest Alergi dengan Uji KulitPPV 86 % artinya bila hasil uji kulitnya (+): kemungkinan dia menderita alergi adalah 86%

Disease +200

Disease -800

Test +32

Test -768

Test +190

Test -10

Population1000

Negative Predictive ValueNEGATIVEPREDICTIVEVALUE = 768/778=99%Test Alergi dengan Uji KulitNPV 99 % artinya bila hasil uji kulitnya (-): kemungkinan dia tidak menderita alergi adalah 99 %

Disease +200

Disease -800

Test +32

Test -768

Test +190

Test -10

Population1000

Positive & Negative Predictive ValueThe Positive Predictive Value of a test will vary (according to the prevalence of the condition in the chosen setting)

Predictive value & changing prevalencePrevalence reduced by an order of magnitude from 20% to 2%

Disease +200

Disease -9.800

Population10.000

Sensitivity and Specificity unchangedPredictive value & changing prevalence

Disease +200

Disease -9.800

Test +392

Test -9.408

Test +190

Test -10

Population10.000

POSITIVEPREDICTIVEVALUE = 33%Positive predictive value at low prevalencePreviously, PPV was 86%

Disease +200

Disease -9.800

Test +392

Test -9.408

Test +190

Test -10

Population10.000

NEGATIVEPREDICTIVEVALUE >99%Previously, NPV was 99%Negative predictive value at low prevalence

Disease +200

Disease -9.800

Test +392

Test -9.408

Test +190

Test -10

Population10.000

Prediction of low prevalence eventsEven highly specific tests, when applied to low prevalence events, yield a high number of false positive resultsBecause of this, under such circumstances, the Positive Predictive Value of a test is lowHowever, this has much less influence on the Negative Predictive Value

Likelihood Ratio

Relative likelihood that a given test would be expected in a patient with (as opposed to one without) a disorder of interest. probability (%) of the test result in patients without diseaseLR=probability (%) of a test result in patients with disease

LikelihoodThe likelihood that someone with the disease will have a positive test is 190/200 or 95%This is the same as the sensitivity

Disease +200

Test +190

Test -10

Population1000

The likelihood that someone without the disease will have a positive test is 32/800 or 4%This is the same as the (1-specificity)Likelihood

Disease -800

Test +32

Test -768

Population1000

LIKELIHOOD OF POSITIVE TEST IN THE ABSENCE OF THE DISEASESENSITIVITY1- SPECIFICITY== 23.8LIKELIHOOD OF POSITIVE TEST GIVEN THE DISEASE=LIKELIHOOD RATIO + (LR+)A Likelihood Ratio (LR) of 1.0 indicates an uninformative test (occurs when sensitivity and specificity are both 50%)The higher the Likelihood Ratiothe better the test (other factors being equal) 0.950.04=Test Alergi dengan Uji KulitLR+=23,8, artinya bila hasil uji kulitnya (+): hasil (+) ini dapat terjadi 23,8 kali lebih besar terjadi pada penderita alergi dibandingkan dengan yang bukan penderita alergi Likelihood Ratio

RECEIVER OPERATING CHARACTERISTIC CURVEOverall shape is predicted by the reciprocal relationship between sensitivity and specificityThe closer the curve gets to Sensitivity=1 and Specificity=1, the better the overall performance of the testThe diagonal line (representing Sensitivity=0.5 and Specificity=0.5) represents performance no better than chanceHence the area under the curve gives a measure of the tests performanceFALSE POSITIVE RATE (1-Specificity)TRUE POSITIVE RATE (Sensitivity)

AREA UNDER ROC CURVESSensitivity and specificity both 100% - TEST PERFECTSensitivity and specificity both 50% - TEST USELESSAREA=1.0AREA=0.5The area under a ROC curve will be between 0.5 and 1.0

Area = 0.7 (between 0.5 and 1.0)Consider (hypothetically) two patients drawn randomly from the DISEASE+ and DISEASE- groups respectivelyIf the test is used to guess which patient is from the DISEASE+ group, it will be right 70% of the timeAREA UNDER ROC CURVES

RECEIVER OPERATING CHARACTERISTIC (ROC) CURVEThis study compared the performance of a dementia screening test in a community sample (ACAT) and a memory clinic sample (MC)Flicker L, Loguidice D, Carlin JB, Ames D. The predictive value of dementia screening instruments in clinical populations. International Journal of Geriatric Psychiatry 1997 ; 12 : 203-209

Diagnostic tests Is not about finding absolute truth, but about limiting uncertaintyestablishes both the necessity and the logical base for introducing probabilities, pragmatic test-treatment thresholds ..\Start thinking about what youre going to do with the results of the diagnostic test, and whether doing the test will help your patients

Interpreting Diagnostic Studies VIA - RaMMbo

Validity

ParticipantsIndex group (IG) & Gold standard Comparison Group (CG)Outcome I GCGRepresentative?Selection?VALIDITYReproducible Maintain? Measurements blind subjective? OR objective? QUESTION:

Diagnostic Accuracy Study: Basic DesignSeries of patientsIndex testReference standardBlinded cross-classification

Recruitment:Was diagnostic test evaluated is representative spectrum of patient?Series of patientsIndex testReference standardBlinded cross-classification

Maintenance:Was the endpoint of the reference standard obtained for all subjects?Series of patientsIndex testReference standardBlinded cross-classification

Measurement:Were the assesors kept blind to the results of each test and/or were the reference standard endpoint objectiveSeries of patientsIndex testReference standardBlinded cross-classification

Selected PatientsIndex testReference standardBlinded cross-classificationSpectrum Bias

Series of patientsIndex testReference standardBlinded cross-classificationVerification Bias

Series of patientsIndex testBlinded cross-classificationRef. Std ARef. Std. BDifferential Reference Bias

Series of patientsIndex testReference standardUnblinded cross-classificationObserver Bias

Importance

INTERVENTIONETIOLOGY/RISK FACTORSDIAGNOSISPROGNOSIS & PREDICTIONFREQUENCY & RATEPHENOMENA / THOUGHTSIMPORTANCEWhat should I do about this condition or problem?What causethe problem?Does this personhave the condition or problem?Who will getthe condition or problem?How commonis the problem?What are thetype of problem?*

CLINICAL TRIALPROGNOSISDIAGNOSTIC RRR, ARR, NNT p & CISurvival curveRR / ORp & CISn,Sp,LH,PPV,NPVp & CIIMPORTANCE*

Applicability

PICO & ApplicabilityYour question(PICO)StudyWhat do theResult mean?How well wasstudy done?ValidityImportanceApplicability*

CRITICALAPPRAISALDIAGNOSTIC TEST

Critical appraisal diagnostic testUse worksheet (VIA; RAMMbo)STARD Use supporting softwaresCAT Maker

Validity (1)Apakah penelitian uji diagnostik dilakukan secara tersamar dengan baku emas yang benar ?Validity (2) Apakah uji diagnostik dilakukan terhadap pasien dengan spektrum penyakit atau kelainan yang memadai sehingga dapat diterapkan dalam praktek sehari-hari?Validity (3) Apakah pemeriksaan dengan baku emas dilakukan tanpa memandang hasil pemeriksaan dengan uji diagnostik ?

ImportantBerapa Sn, Sp, LR+, LR-, PPV, NPV, Pre-test probability, Post-test probability, Pre-test Odds, Post-test Odds ?

Applicable (1)Apakah uji diagnostik tersebut tersedia, terjangkau dan akurat?Applicable (2) Apakah kita bisa memperkirakan pre-test probability (prevalens) penyakit pada pasien kita ?Applicable (3) Apakah post-test probability yang dihitung akan mengubah tatalaksana pasien kita?Applicable (4) Apakah secara keseluruhan uji diagnostik tersebut bermanfaat bagi pasien ?

Section and and topicTitle, abstract, and keywords IntroductionMethodsParticipantsTest methodsStatistical methodsResultsParticipantsTest resultsEstimatesDiscussions

STARD initiative (25 items)Standards for Reporting of Diagnostic AccuracyBossuyt PM, Reitsma JB, Bruns, DE, Gatsonis CA, Glasziou PP et al. BMJ 2003,326:41-6

1st component of STARD

2nd component of STARD

Does early diagnosis really lead to improved survival, or quality of life, or both? Are the early diagnosed patients willing partners in the treatment strategy? Is the time and energy it will take us to confirm the diagnosis and provide (lifelong) care well spent? Do the frequency and severity of the target disorder warrant this degree of effort and expenditure?Guides for deciding whether a screening or early diagnostic maneuver does more good than harm:

**************************************