Challenges for Evidence-based Diagnosis&

Course Review

Tom Newman

11/29/12

Challenges for EBD 2012-1127

Lecture Outline

Announcements Criticisms of EBM Challenges for EBM & rational decision

making Course review

Announcements We hope you read chapter 12. Optional reading on the website Take-home final will be posted after

section today– Due ON PAPER in class Thursday,

12/6/12 at 8:45 AM, no exceptions– If you will not be here, email to

Olivia@epi.ucsf.edu by 8:45 AM 12/6.– Your own work only, no collaboration

Announcements 2

Take home final, continued– Feel free to look up cited articles for

clarification– ½ of problems based on problems submitted

by students.– Thank you to everyone for writing problems!– Special thanks to Patti Curl, Miriam Laker,

Natasha Din, Jesus Valdez, Sharon Orbach, Felicia Chow, Priya Prasad, Cecily Miller, Eugenie Poirot, Mona Mehta, Kara Saperston & Helen Byakwaga and Ashish Aggarwal (2011)

– We hope you enjoy it! Next week: exam review and course evaluation

Historical background for EBM Started out as a revolutionary questioning of

authority Changes that made EBM possible

– Development of science of clinical research– Increased ease of access to evidence via

computers and the Internet– Multiple examples of interventions with strong

basic science rationale that turned out to be harmful

– More interventions offering only slight marginal benefit over alternatives; need to quantify and understand effect size

Criticisms of Evidence-Based Medicine -1

EBM over-values randomized blinded trials and denigrates other forms of evidence, including clinical experience.

– No systematic reviews of parachutes*– Oversimplification of problems– Excessive faith in evidence hierarchy– Reliance on checklists

*Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ  2003;327:1459-1461

Criticisms of Evidence-Based Medicine -2 Evidence-based screening and

treatment recommendations tend towards the nihilistic

– May not recommend tests or treatments that some clinicians, professional societies or disease-specific advocacy groups believe are effective.

– Adoption of US Preventive Services Task Force evidence grading for other clinical situations

Criticisms of Evidence-Based Medicine -3

EBM has been or might be used by payers as an excuse to deny payment and limit clinician autonomy.

– Perceived need to recommend tests and treatments for fear they will not be reimbursed if optional

– Prostate cancer screening: Winners and Losers

– Sometimes real issue is probably costs, but e.g., Medicare cannot legally consider costs in coverage decisions

– What is the alternative to EBM?

Stories and Statistics

People are more moved by individual stories than by statistics

"A single death is a tragedy; a million deaths is a statistic." *

Infant safety seats on airplanes– Requirement recommended by the Americam

Academy of Pediatrics – FAA projected net increase in deaths due to

diversion to cars– TBN et al estimated if this did NOT occur, cost to

save 1 life ~$1.3 billion (@ $200/ticket)

*Joseph Stalin

Congressional Testimony

"The question, I think, Mr. Chairman, comes down to how many more children must die, how many more have to be hurt before we reach the threshold of FAA's ghoulish cost/benefit ratio?"

--Congressman Jim Lightfoot, Iowa

“Real” vs “Theoretical” Children

“The argument in support of the FAA’s resistance to the NTSB...is unreasonable on its face and ridiculous in its justification. It protects theoretical children driving in cars at the expense of real flesh-and blood infants whose safety is unquestionably compromised when flown as a lap-baby.”

Nader R, Smith WJ. Collision course: the truth about airline safety. Blue Ridge Summit, PA: TAB Books, 1994. Cited by Beshai D. Arch Ped Adol med 2003;157:953-4

Challenges for evidence-based diagnosis

Difficulty estimating pre-test probabilities– Possible for common presentations fo common

conditions Difficulty finding applicable likelihood ratios

– Systematic reviews may combine apples and oranges Uncertainty about treatment thresholds

– Requires “utility assessment” – very hard

Clinicians are better at making decisions than at estimating any of the above

Heuristics Used in Probability Estimation

Representativeness Availability Adjustment from an anchor

These heuristics can lead to biased estimates. See Chapter 12 for details.

Representativeness

If patient has typical features of a disease, we assign the disease high probability, even if prior probability was very low

Examples:– Chest pain radiating to the back aortic

dissection– “Worst headache of my life”

subarachnoid hemorrhage

Availability

The more easily you can imagine something happening, the higher the probability you assign it

Most recent and worst cases stand out Examples:

– Distorted view of risk of rare, serious illnesses from training in tertiary settings

– TN distorted view of risk of kernicterus (Consequences of making an error

factor into the probability estimation)

Anchoring

People tend to estimate probabilities by starting some place and revising probability up or down

This happens even if anchor is irrelevant Probability of pulmonary embolism study*

– “Do you think it is > or <1%?” OR “Do you think it is > or < 90%?”

– What do you think it is?– Average estimate 23% vs 53%

Importance of triage room

*Brewer et al. Med Decis Making 2007;27:203-11

Another bias Example: I have a rule that allows me to

say whether any sequence of 3 numbers is golden

The series 2, 4, 6 is golden Class write down other series of 3 numbers

to test hypotheses about my rule See how quickly you can guess my rule Confirmation bias and premature closure

* Mlodinow L. The Drunkard's Walk. Random House, 2008

More on cognitive biases

System 1: Fast, effortless, intuitive, emotional, subconscious

System 2: Slow, deliberate, rational, effortful, lazy

Sample experiment in Thinking Fast and Slow

Jar in coffee break room to put donations for milk

RQ: will donations vary depending on photo posted?

Results

Future of EBM

More expensive tests and treatments Greater competition for resources More guidelines, algorithms, computer-

aided decision making Need for leaders to understand how these

work, evaluate them critically, and help produce them!

Eventually need to acknowledge some tests or treatments cost too much

Need people who can explain these things to the public

Course Review (interactive)