Course Overview:Medical Information for Decision Making

HuBio 590, Fall 2008Peter Tarczy-Hornoch MD

Head and Professor, Division of Biomedical & Health InformaticsProfessor, Division of Neonatology

Adjunct Professor, Computer Science and Engineering

faculty.washington.edu/pth

October 21, 2008

Course Objectives1. Describe the value of high quality medical information

for clinical care

2. Describe the range of factors that influence the clinical decision making process

3. Translate a clinical scenario into a searchable question

4. Describe advantages and limitations of various medical information resources and types of documents

5. Find documents from one or more medical information resource(s) that may address the clinical situation

6. Assess systematically the relevance and validity of a given document with respect to the clinical situation

7. Compare relevance and validity across two documents

Course Overview Session 1, Tuesday October 21

“Medical Information and Medical Decision Making” & “Statistics 101” (1:30-2:30; Learning Objectives 1,2,3)

Small Group: Introductions (2:40-3:20) Session 2, Wednesday October 22

Small Group: Translating a clinical question into a searchable one (2:00-2:50; Learning Objective 3)

“Finding Medical Information in a Clinical Context” (3-3:50; Learning Objectives 4,5; L. St. Anna)

Session 3, Monday October 27Small Group: Hands on session searching on-line databases

(2:00-2:50; Learning Objectives 4,5; Librarians join groups)“Assessing a Document on Treatment” (3:00-3:50; Learning

Objective 6 with focus on treatment) Session 4, Wednesday October 29

Small Group: Practice assessing document on treatment

Course Overview Session 4, Wednesday October 29

Small Group: Review of sample problems and discussion of real world examples of interpreting articles on treatment (2:00-2:50; Learning Objective 6 with focus on treatment)

“Assessing a Document on Diagnosis”(3:00-3:50; Learning Objective 6 with focus on diagnosis)

Session 5, Monday November 3Small Group: Review of sample problems and discussion of

real world examples of interpreting articles on diagnosis (2:00-2:50; Learning Objective 6 with focus on diagnosis)

“Assessing Multiple Studies” (3:00-3:50; Learning Objectives 6,7 with focus on systematic reviews)

Session 6, Wednesday November 5Small Group: Review of sample problems (2:00-2:50)“Applying MIDM Concepts in the Real World” (3:00-3:50)

Course Logistics Lectures: T-435

Lectures cover key content in the syllabus (see courses.washington.edu/midm “Syllabus” for content outline for each session). Assignments permit practice of material presented prior to small group session.

Small Groups: T538, T540, T543, T546, T548, T547, T549 Focus on application and discussion of lecture material See web page “Small Group Assignments” (note user

name/password information e-mailed earlier) Office Hours

Peter Tarczy-Hornoch, I264A By arrangement – e-mail pth@u.washington.edu

Grading & Class Attendance To pass the course the following is required:

Attendance at all six small group sessions If you miss a small group session then a makeup assignment

will be required. Assignments will be given after Lectures 1-5 to prepare

for small group sessions but these assignments will not be graded

Passing the final exam70% is a passMultiple choice and fill in the blanksTake-home, open book, web administeredAvailable at 5P Wed Nov 5th; due 5P Wed Nov 12th

courses.washington.edu/midm

Questions? The complete syllabus (PDF) contains the

following sections: Course Description WWAMI Course Chairs Seattle Course Chair and Small Group Leads Learning Objectives Course Organization Grading and Class Attendance Required Textbook/Readings Schedule for 2008-9 Content outline for each of the six sessions

Session 1a: Medical Information and Medical Decision Making

Peter Tarczy-Hornoch MDHead and Professor, Division of BHIProfessor, Division of Neonatology

Adjunct Professor, Computer Science and Engineering

faculty.washington.edu/pth

October 21, 2008

Medical Information and Medical Decision Making

Medical Decision Making Nature of Medical Information Reducing Errors & Improving Quality Finding Knowledge and Evidence

Medical Decision Making Requires Integrating Information

Patient Data & Information (ICM)

General Information & Knowledge (MIDM – finding/assessing)

Case specificdecision making

Diagnostic Testing(What is it?)(Session 4)

Therapy/Treatment(What do I do for it?)(Session 3)

Patient Information Comes From Diverse Sources

Current Visit Info:- Symptoms- History- Findings

Past Visit Info:- Paper chart(s)- History - Physical- Family history- Problem lists- etc.

RadiologySystem (X-rays)

PharmacySystem (drugs)

LabSystem(test results)

TranscriptionSystem

Billing System- Stay / Visit / Cost- Diagnoses / Treatments

Medical Knowledge Acquired From Diverse Sources

Books School

Journals

CD ROM books, CME, etc.

Networked InformationSources

Medical Decision Making Requires Integrating Information

Patient Specific Knowledge: 6 year old boy History of chicken pox exposure Currently on steroids for asthma Exam showing “dewdrop on a rose petal”

General Medical Knowledge: Diagnosis and management of chicken pox Management of asthma Risk of steroids and chicken pox

Therapy Decision: Stop steroids, treat with Acyclovir

Clinical Encounters Generate Questions “…conservative to conclude that every interaction

between a patient and a doctor is likely on average to generate at least one question” R. Smith, BMJ, 1996

Types of information needed (data to knowledge) Patient specific (laboratory, radiology, immunization) Guidelines, policies, standards (national or local) Drug information (dosage, interactions, side effects) Medical literature (textbooks, reference books, journals)

Information needed in the context of a specific encounter (e.g. immunization)

Known/unknown & met/unmet information needs

Medical Information and Medical Decision Making

Medical Decision Making Nature of Medical Information Reducing Errors & Improving Quality Finding Knowledge and Evidence

Managing Medical Information a Longstanding Challenge

“The Art is long, life is short, opportunity fleeting, experience delusive, judgment difficult”

Hippocrates ~400 B.C. “While the continuing gains in medical knowledge and

the accompanying ability of doctors to treat the sick have been real, the passage of time has too often proven the espoused remedies of one era to be of limited value or frankly harmful in the next…How much of what we embrace as truth today will suffer this fate over the ensuing decades”

LC Epstein 1997

“the Art is long”: Information Overload “if the most conscientious physician were to attempt to keep up

with the literature by reading two articles per day in one year this individual would be 800 years behind” O. Barnett, 1990

2/3 of primary care practitioners surveyed: “the current volume of scientific literature is unmanageable” J.W.Williamson, 1989

“Although well over 1 million clinical trials have been conducted, hundreds of thousands remain unpublished or are hard to find and may be in various languages. In the unlikely event that the physician finds all the relevant trials of a treatment, these are rarely accompanied by any comprehensive systematic review attempting to assess and make sense of the evidence” Bero & Rennie, JAMA 1995

Best evidence may not be as good as you’d wish n=2500 treatments in BMJ Clinical Evidence

See http://www.clinicalevidence.com/ceweb/about/knowledge.jsp

“experience delusive”: unproven treatments

“experience delusive”: data vs. opinion

“Types” of medical knowledge Compiled formal/scientific knowledge (best)

E.g. systematic reviews, evidence based medicine, some Up To Date entries

Uncompiled formal/scientific knowledge (good) E.g. “raw” PubMed search, a single randomized trial

Compiled informal/experiential knowledge (ok) E.g. consensus statements, “standard of care”, books,

“Spiral” manuals, some Up To Date entries Uncompiled informal/experiential knowledge (ok)

E.g. opinions/customs of experts and consultants

“judgment difficult”Uncertainty Impacts Decision Making Diagnostic uncertainty

Horses vs. Zebras (infection vs. genetic problem) Availability bias (last patient I saw with this had X)

Therapeutic uncertainty Attributes of patient vs. study population Study drug vs. class of drugs Population vs. individual (genes + drugs)

“Islands of certainty in seas of uncertainty” Studies vs. experience/judgment

“judgment difficult”Biases Impact Decision Making

Bias: 2a. A preference or an inclination, especially one that inhibits impartial judgment. 3. A statistical sampling or testing error caused by systematically favoring some outcomes over others. (American Heritage Dictionary)

Recall bias: inaccurate recollection of information Availability bias: recent or memorable

information/decisions easier to remember Sampling bias: personal experience around

information/decision making not representative Publication bias: “negative” studies hard to publish

Biomedical Informatics Studies the Use and Nature of Medical Information

Doctors use medical knowledge for clinical problem solving and decision making

Biomedical informatics focuses on the general issues of biomedical knowledge capture, retrieval, and application

“the scientific field that deals with biomedical information, data, and knowledge – their storage, retrieval, and optimal use for problem solving and decision making” Shortliffe, E.H., 2006

Important discipline in the context of medical information for decision making Division of Biomedical and Health Informatics at UW www.bhi.washington.edu

“Just in time information”: unmet need

“At the bedside or in the office, physicians should have instantaneous, up-to-date assistance from an affordable, universally available database of systematic reviews of the best evidence from clinical trials” Bero and Rennie, JAMA, 1995

“New information tools are needed: they are likely to be electronic, portable, fast, easy to use, connected to both a large valid database of medical knowledge and the patient record” R. Smith, BMJ, 1996

Medical Information and Medical Decision Making

Medical Decision Making Nature of Medical Information Reducing Errors & Improving Quality Finding Knowledge and Evidence

Do these challenges around medical information and decision making matter?

Hippocratic Oath Hippocratic Oath has evolved since 400 BC as

societal values and standards have changed Two aspects particularly relevant to HuBio 590

Medical Information for Decision Making: “To practice and prescribe to the best of my ability for

the good of my patients, and to try to avoid harming them.” => finding and applying the best information

“To keep the good of the patient as the highest priority” => integrating patient specific information

How successful are we at this today?

2000 Press release “…medical errors kill some

44,000 people in U.S. hospitals each year. Another study puts the number much higher, at 98,000 ”

“Even using the lower estimate, more people die from medical mistakes each year than from highway accidents, breast cancer, or AIDS.”

Some errors are unavoidable mistakes, some errors are due to missing or incorrect data or knowledge

2001 Press release “The nation's health care

industry has foundered in its ability to provide safe, high-quality care consistently to all Americans”

Studies estimate 3-4% of hospitalizations result in adverse events

Recommendation 2 “…six major aims: specifically, health care should be safe, effective, patient-centered, timely, efficient, and equitable”

2007 Press release “Medication errors are among the most

common medical errors, harming at least 1.5 million people every year, says a new report from the Institute of Medicine of the National Academies.  The extra medical costs of treating drug-related injuries occurring in hospitals alone conservatively amount to $3.5 billion a year”

Recommendation 3: “All health care organizations should immediately make complete patient-information and decision-support tools available to clinicians and patients. Health care systems should capture information on medication safety and use this information to improve the safety of their care delivery systems.”

Economic & Legal Context Rising healthcare costs

2005: $2 Trillion, 16% GDP, 6.9% (2 x inflation) Each diagnostic & therapeutic decision has a cost If 47% of treatments are “of unknown effectiveness”

then what is the cost implication of this?

Malpractice Malpractice “the provider failed to conform to the

relevant standard of care.” Historically standard of care = “reasonable person” Evolution of standard of care = “what is the evidence”

Medical Information and Medical Decision Making

Medical Decision Making Nature of Medical Information Reducing Errors & Improving Quality Finding Knowledge and Evidence

First Randomized Controlled Trial - 1948

Evidence Based Medicine – 1992-2008 “Evidence-Based Medicine: A New Approach to Teaching

the Practice of Medicine”, JAMA 1992 (classic article, see MIDM website)

“Evidence-based medicine (EBM) requires the integration of the best research evidence with our clinical expertise and our patient’s unique values and circumstances” in Evidence Based Medicine: How to Practice and Teach EBM, Straus et al 2005 (the definitive textbook)

“Progress in Evidence-Based Medicine”, JAMA 2008 (reviews 1992 article, see MIDM website)

Evidence Based Medicine - Caveat Parachute use to prevent

death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. Smith and Pell, BMJ, Dec 2003

Where to get “best research evidence”?

Ideal: Synthesized Authoritative Current Searchable

Books School

Journals

CD ROM books, CME, etc.

Networked InformationSources

No single source for “best evidence” Ideal: continually updated, synthesized, expert

authored, peer reviewed, electronic knowledge base

Cochrane collaboration (www.cochrane.org) and similar databases are closest we have but… Restricted to areas with sufficient literature Updated episodically, not real time

Developing tools for finding evidence is an active area of biomedical informatics research nationally

So What Do You Do? Use Evidence Based Medicine Resources Learn General Standard of Care

Manuals/textbooks National policy statements, e.g. American Academy of Pediatrics

Learn Local Standards of Care Policies/guidelines, e.g. UW Prevailing practice – conferences/grand rounds

Keep Up to Date on New Clinical Studies Journals Journal abstracting/summarizing services Conferences

Learn to search databases of medical knowledge…

Steps to Finding & Assessing Information

1. Translate your clinical situation into a formal framework to get a searchable question (today)

2. Choose source(s) to search (Session 2)

3. Search your source(s) (Session 2)

4. Assess the resulting articles (documents) Therapy documents (Session 3) Diagnosis documents (Session 4) Systematic reviews/comparing documents (Session 5)

5. Decide if you have enough information to make a decision, repeat 1-4 as needed (ICM, clinical rotations, internship, residency) (Session 6)

Step 1: Frame the question (I) Translate clinical question to searchable question

(PPICONSS framework for assessing a document, PPICOS framework for formulating a search/finding information) P P : Problem P P : Patient/Population I I : Intervention C C : Comparison O O : Outcome N : Number of Subjects S S : Study Design/Type/Statistics S : Sponsor

Step 1: Frame the question (II) Translate clinical question to searchable question (PPICOS)

P: Problem What is the question of interest? E.g. “How to treat athletes foot?”

P: Patient Demographics (e.g. gender/age range), condition, disease E.g. “Healthy female college athlete with skin/nails affected”

I: Intervention Diagnosis/treatment, which one is of primary interest/preferred a priori E.g. “Treatment with over the counter ointment”

C: Comparison Alternative diagnosis(es)/treatment(s) (of secondary interest) E.g. “Over the counter ointment vs. prescription ointment vs. pill”

O: Outcome Diagnostic accuracy, complication, death, cost, etc. E.g. “Cheapest and safest cure since no insurance” => Cost, how often does

each alternative cure it, what are side effects of each treatment S: Study Design/Type

Ideally what type of study/document are you looking for E.g. “Systematic Review”

Compare Results to Search (I)Search Result Comparison

Problem at hand Problem studied Are they really the same?

Patient characteristics Population characteristics Is patient similar enough to population studied?

Intervention most relevant to patient/provider

Intervention studied (primary one)

Are they the same?

Comparison – other alternatives considered

Comparison – alternatives studied

Are alternatives studied those of interest to you?

Outcomes – those important to pat/prov

Outcomes – those looked at by study

Are outcomes studied those of interest to you?

Number of subjects Does study have enough subjects to trust results?

Study design hoped for Statistics – study design and statistical results

Is study design good? What do results mean?

Sponsor – who paid for study

Is there potential bias?

Compare Results to Search (II)Search Result Comparison

Problem: Athletes foot of skin/nails

Problem studied: Athletes foot of skin/nails

They are the same

Patient: young adult, female, healthy

Population characteristics: healthy elderly males

May not be similar enough

Intervention relevant: treatment with over counter cream

Intervention studied: treatment with new pill

Comparison: over counter cream, prescription cream, pills

Comparisons studied: prescription cream vs. old pill vs. new pill

Pretty close but no over counter comparison

Outcomes: desire cheapest & safest cure

Outcomes: cure rate, side effects

Ok since can look up cost elsewhere but side effects in patient vs. population?

Study design hoped for: “Systematic Review”

Statistics – a single clinical study

We wanted a review of all available studies

Conclusion: probably not what we want, look for another document

Session 1b: Statistics 101

Peter Tarczy-Hornoch MDHead and Professor, Division of BHIProfessor, Division of Neonatology

Adjunct Professor, Computer Science and Engineering

faculty.washington.edu/pth

October 21, 2008

Statistics 101: Mean, Standard Deviation

*Population: weights of all medical students in the class *Sample: weights of 10 randomly chosen students:

50, 53, 56, 60, 65, 67, 70, 73, 73, 75 kg

Sample Mean: Mean=sum{x1..xn}/n Mean=sum{50,53,…75}/10=64.2 kg

Sample Variance s2=sum{(x1-mean)2,...(xn-mean)2}/(n-1) s2=sum{(50-64.2)2,...(75-64.2)2}/(10-1) =80.6

Sample Standard Deviation s=sqrt(s2) s=sqrt(80.6)=8.97 kg

Statistics 101: Normal Distribution

68% of values are +/- one from the mean 95.4% of values are +/- two from the mean 99.6% of values are +/- three from the mean Sample standard deviation and mean estimate

population and meanhttp://en.wikipedia.org/wiki/Normal_distribution

*Population*Sample 1=X

*Sample 2=O

OX XX

OO

Statistics 101 – Sample Sizes Mean:

Mean=sum{x1..xn}/n Increasing n does not impact mean

Sample Variance s2=sum{(x1-mean)2,...(xn-mean)2}/(n-1) Increasing n decreases sample variance

Sample sizes Larger sample sizes decrease variance and allow you to

see smaller differences between groups Rule of thumb for sample size for a strong study n=400

Statistics 101: p values

“Treatment 1 was better than treatment 2 (P<0.05)” P<0.05 roughly means less than 5% (0.05) chance treatment 1 and 2

are the same “Treatment 1 was better than treatment 2, P=0.001”

P=0.001 roughly means 1/1000 (0.001) chance treatment 1 and 2 are the same

p=0.04 vs. p=0.05 vs. p=0.06 All roughly the same, choice of “p<0.05” as “statistically significant”

is arbitrary Study 1: mortality cut by 50% with p=0.04 vs Study 2:

mortality cut by 1% with p=0.01 Cutting mortality by 50% clinically more significant than by 1% P=0.01 is statistically more significant than P=0.04

See http://www.acponline.org/journals/ecp/julaug01/primer.pdf

Also http://www.acponline.org/journals/ecp/primers.htm

Statistical Significance: are the treatments the same? Clinical Significance: if they are different then do we

care about the difference? Examples:

Duration of pharyngitis: 8.1 days to 7.4 days

Weight: 279 lbs to 266 lbs after 3 months

Survival increased from 4.5 mos to 5.2 mos with 100%

mortality at 12 months

Claudication: Increase in walking distance by 34 ft.

Statistical Clinical Significance

Small Group Sessions 1 & 2 S1: Small group leads to introduce themselves

Name, where they are from, where they went to medical school, their clinical practice, interesting fact about their background

S1: Students to introduce themselves Name, where they are from, interesting fact about their

background, what they hope to get from small group sessions S1: Small group leads to give examples of translating

clinical situations/scenarios into a searchable question (using PPICOS framework)

S2: Assignment for 10/22 (tomorrow) for students Come up with one clinical “situation” you have wondered about or

been asked about and translate it into PPICOS framework Work through 4 scenarios “Formulating a searchable question” (

http://courses.washington.edu/midm/schedule.htm) Bring paper or electronic copy of your completed assignment Small groups to identify two PPICOS scenarios for assignment for

small group on 10/27 * Reminder: students please sign in, group leads please turn

in sign in sheets to Donna Rowe, Box 357240

QUESTIONS? Medical Decision Making Nature of Medical Information Reducing Errors & Improving Quality Finding Knowledge and Evidence Statistics 101 Small Group Session

When done with Q/A we will all go to our assigned small groups