Quality Improvement Science and Patient Safety Research

Dan France, Ph.D., MPH

Center for Clinical Improvement

Vanderbilt University Medical Center

Outline

• Quality Improvement – Need for the engineering mentality/systems

thinking in healthcare

• Patient Safety

• Student Project

Engineering

• Design/Analysis

• Systems Engineering

• Engineering Management

• Quality Management

• Quality Engineer

• Industrial Engineer

• Health System Engineer

Part I. Quality Improvement

Background

• Institute of Medicine (IOM) reports– Nov 1999: To Err is Human – March 2001: Crossing the Quality Chasm

• Brief History of QI– Scientific Management (Taylor, 1911)

• Assembly lines

– Statistical Process Control (Shewhart, 1931)– Quality Improvement (Deming, 1955)– Lean Production (Womack, 1990)

• Mass customization

Expert knowledgeContent knowledge

System ThinkingStatistical VariationScientific MethodPsychology of Change

Traditional Improvement Continuous Quality Improvement

Improvement in Healthcare

Paul Batalden MD

What is Quality

Quality is the degree to which we meet or exceed customer expectations

Quality Assurance versus Quality Improvement

• Quality Assurance– meet a specification or standard– Take sample measurements to measure

performance

• Quality Improvement– continual process to improve current

performance– Continual measurement and data feedback

The Relation Between Quality Inspection, Regulation, Management, and Improvement

Design

RedesignManagement & Improvement

Numberof

Providers

0

Inspection & Regulationfor Public Safety

Level of Quality

SanctionsResearch &Development

IOM Definition of Quality

• Six Dimensions of Quality in Healthcare– Safe– Effective– Timely– Patient centered– Efficient– Equitable

QI is a ScienceDefined Methodology

• Focus on systems (Systems theory)• Develop ideas for change and test them (Scientific

method)• Understand the variation of data measured

continuously over time (SPC)• Understand reasons and motivation of people to

act on data (Common cause, special cause variation, diffusion of innovation)

• Use a balanced set of measures (Value compass)

QI is a Discipline • QI research is funded by AHRQ and NIH• QI research is published in peer review

journals such as NEJM and JAMA• QI science is taught in schools of public

health, business schools, graduate programs in engineering, management and education, medical schools in health services research, biostatistics, public health

• There is a national Quality Scholars program in healthcare

Variation in PracticeInstitute of Medicine

• Overuse (eg. Antibiotics, C-Section)

• Underuse (eg. Mammography, Beta-Blockers)

• Misuse (eg. Medical errors)

The issue is unnecessary variation

i.e., appropriateness of care

Six Sigma

• Domestic Airline Fatality– 6– 99.99966% “Right”

• Mammography Screening– 1.7 – 56% “Right”

QI is a Science: Statistical ApproachVariation and Improvement

Lessons about Variation

• Once we begin to measure important quality characteristics and outcomes, we notice variation.

• We question measurements that display no variation.

• Often, single data points alone are uninformative, but data displayed over time can provide information for improvement.

• The primary purpose of understanding variation is to enable prediction.

• Interaction among process variables produces sources of variation: materials, methods, procedures, people, equipment, information, measurement, and environment.

A process

... a series of linked steps, often but not necessarily sequential, designed to ...

cause some set of outcomes to occur transform inputs into outputs generate useful information add value

Walter Shewhart: a system of causes

Constant (convergent) systems

follow the laws of mathematical probability:

How the process behaved in the pastpredicts how it should behave in the future

non-constant (divergent) systems follow the laws of chaos theory:

How the process behaved in the pastdoes not predict how it should behave in the future

Random variation

represents the sum of many small variations, arising from real but small causes that are inherent in—and part of—any real process

follows the laws of probability— behaves statistically as a random probability function

because random variation represents the sum of many small causes, it cannot be traced back to a root cause

represents " appropriate " variation

different processes have different levels of random variation random variation is a matter of measurement, not goal setting

is a physical attribute of the process

Assignable variation

represents variation arising from a single cause that is not part of the process (system of causes)

represents " inappropriate " variation

therefore can be traced, identified, and eliminated (or implemented)

Registration Times

– These are actual times it took triage level 2 patients to register in the Emergency Department of a hospital:

15 67 4 14 10

12 54 3 7 11

14 83 54 17 20

10 53

Parametric frequency distribution

Value observed

Nu

mb

er

of

tim

es

ob

serv

ed

(Num

ber,

ra

te,

perc

enta

ge,

prop

ortio

n)

Parameters: mean and variance

Value observed

Nu

mb

er

of

tim

es

ob

serv

ed

(Num

ber,

ra

te,

perc

enta

ge,

prop

ortio

n)

center (mean, median)

spread (variance, standarddeviation, range)

Probability-based boundaries

Frequency Distribution

Value observed

Nu

mb

er

of

tim

es

ob

serv

ed

(Num

ber,

ra

te,

perc

enta

ge,

prop

ortio

n)

2.575 std. devs. 2.575 std. devs.

0.5% 0.5%

99%

Statistical Process Control Chart

Time

Ob

serve

d

valu

e

Random variation

Process Control Chart

Time

(How the process behaves over time)

T1 T2 T3 T4 T5 T6 T7 T8 T9

Ob

serve

d

valu

e

Assignable variation

Process Control Chart(How the process behaves over time)

Time

T1 T2 T3 T4 T5 T6 T7 T8 T9

Ob

serve

d

valu

e

Managing assignable variation

track it to root causes

Find a data point that probably represents assignable variation (usually a statistical outlier)

(React to individual fluctuations in the data)

eliminate (or implement) the assignable cause

UNDESIRABLEOUTCOMES

ENVIRONMENTPEOPLE

POLICIESPROCEDURES

EQUIPMENT

Communication (8) MD to MD MD to family RN to family

Judgement accuracy of diagnosis (10) timing of treatment (5)

Central Lines (2)

Use of non-VUMC equipment (2)

Staffing (6)

Access to MD & treatmentafter regular hours (7)

Technical issues (7) mgmt of cultures from lab placing/monitoring central lines giving or not giving meds fluid management DKA

Following IV policies (4)

Accepting test results from referring hospitals (3)

Frequency of tests/x-rays (3)

On-Call system forsub-specialties (4)

SUMMARY

Number patient encounters: 281,000Total number of claims: 25Total Cost Incurred: $7,426,815Median Case: $ 100,000

RISK MANAGEMENT PROJECT

d:\RiskMgtProjFishbone

Tampering:

Shewhart proved that tamperingdoes not just waste time and effort --

it seriously harms process performance

Using assignable methodsin an attempt to manage

random variation

Statistical process control charts

Show the probability that an observation arose from the underlying process — that is,

the probability that a particular point's deviation from the center represents only "random" variation arising from the system of causes that make up the process, as opposed to "assignable" variation representing an identifiable, intruding cause.

separate random from assignable variationbased on statistical probabilityusing control limits, runs, trends, and other patterns

in longitudinal data.

They

A trend

55.000000

UCL 76.56000

LCL 33.44000

8.500000

UCL 27.77571

LCL 0.00000

Psych Inpatient Admits / Month# patients

# patients

QI is a Science: Statistical ApproachOverall Improvement Strategy

Outcome

Remove special causes Process change Process change

Unstable processSpecial causes presentAverage is too high

Stable processCommon causevariation is highAverage is too high

Stable processCommon causevariation reducedAverage too high

Stable processCommon causevariation lowAverage reduced

CAP protocol compliance

Implementation Group -- Loose Abx Compliance

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

-23 -21 -19 -17 -15 -13 -11 -9 -7 -5 -3 -1 1 3 5 7 9 11 13 15 17

Month relative to CPM implementation

Pro

po

rtio

n c

om

pli

ant

P chart - 0.01 control limits

Baseline Implementation

Using data to improve

The minimum standard: an annotated time series

Start with a run chart (80% of total value)1.

Add center and goal lines (anchors the eye - now 95% of total value)2.

Add control limits (in appropriate zones)3.

"Teen use turns upward"

1992 17.3%1993 19.0%

% high school seniorswho smoke daily

USA Today, June 21, 1994

"Teen use turns upward"

USA Today, June 21, 1994

1984 18.8%1985 19.6%1986 18.7%1987 18.6%1988 18.1%1989 18.9%1990 19.2%1991 18.2%1992 17.3%1993 19.0%

% high school seniorswho smoke daily

(average moving range = 0.778)

% high school seniors smoking

0

5

10

15

20

25

84 85 86 87 88 89 90 91 92 93

Year

% h

igh

sch

oo

l se

nio

rs w

ho

sm

oke

dai

ly

% high school seniors smoking

0

5

10

15

20

25

84 85 86 87 88 89 90 91 92 93

Year

% h

igh

sch

oo

l se

nio

rs w

ho

sm

oke

dai

ly

Mean = 18.64%

% high school seniors smoking

0

5

10

15

20

25

84 85 86 87 88 89 90 91 92 93

Year

% h

igh

sch

oo

l se

nio

rs w

ho

sm

oke

dai

ly

Mean = 18.64%Avrg Moving Range = 0.778%Upper Process Limit = 20.71%Lower Process Limit = 16.57%

Part II. Patient Safety

Heinrich Triangle

Major Error

Minor Errors

Intercepted Errors(Near Misses)

1

29

300

Information on Major Errors

Information onMinor Errors

Information onNear Misses

Error

Knowledge

Parallel Universe

Essential System Characteristics

• Uses available technologies

• Real-time data

• Feedback providing (closing the loop)

• Designed to succeed (safe)

ALCOA

“At ALCOA I have a real fine data system so that I knew every minute of every day the health and safety condition of 140,000 people.  We shared the information across the whole place so that we had real-time learning among the people.  The information was not there for me.  It was for 140,000 people to learn from shared experiences. Without information having to travel up through some appointment process and maybe some day gets distributed so you can learn something.  It was there every day.  If we had an incident in Sumatra, the people in Jamaica knew it tomorrow morning and they did something about it to avoid the same kind of circumstances.  When I asked for the data at Treasury, it took them a long time to get it for me and when they did, it turned out that their lost workday rate in the Treasury, that has about the same number of employees, was 20 times higher than ALCOA’s.”

Paul O’Neill, Treasury Secretary

J.T. Reason

“major residual safety problems do not belong exclusively to either the technical or the human domains. Rather, they emerge from as yet little understood interactions between the technical and social aspects of the system”

J.T. Reasons,

Safety at Sea and in the Air-Taking Stock Together

Symp., Nautical Institute, 1991

Disney

“But, ultimately, even the most conscientious Cast Members cannot do it alone. Guests, too, have an essential role to play in making every visit to our parks safe.”

Paul S. Pressler,

Chairman, Walt Disney Parks and Resorts

Aviation Safety Network

“Without a doubt 2001 was the year with the highest aviation caused fatalities ever. However, when we take a closer look at the figures we can see that 34 fatal multi-engined airliner accidents were recorded, which was an all-time low since 1946.”

Learning Objectives

• Implement a blame-free reporting culture• Improve or expand chemotherapy taxonomy/definitions

– Preventable adverse drug events and near misses– Operational barriers (i.e., delays) as errors?

• Evaluate wireless technologies as an electronic resources and reporting tool– Integrate into daily workflow– Extend to bedside

• Apply Computerized Order Entry/Decision support• Quality Improvement via multidisciplinary teamwork

based on data feedback

Intelligent Chemo Delivery System

Clinical Improvement(generate hypotheses, tests of change)

Chemotherapy Registry(tracks metrics over time)

Blame-Free ADE Reporting(Process focused)

Decision Support System(Inbedded safety logic)

Perfect Chemotherapy Delivery

Chemo Events – Data Capture

Standardized Reporting

Lesson Learned• Leadership and organizational culture are as critical for

patient safety as structure• Vertical and horizontal organization communication

are essential components of surveillance, prioritization• Team communication (chatter) is key to developing

safe culture and trust; Foundation for safety “pattern recognition”

• Timely data feedback drives safety improvement• Healthcare can learn much about systems thinking

from other industries and cultures• Tightly coupled systems are more prone to failure than

highly adaptive systems

Student Project

1. Develop and test taxonomy for systems errors in Emergency Medicine

– What system factors in the ER increase the likelihood that care providers will commit errors?

– How to measure these factors?

2. Clinic Redesign - Orthopedics– Room Utilization tracking program