Workshop for Commissioners - 21 January 2016 - Using Data to Support System Improvement

7/25/2019 Workshop for Commissioners - 21 January 2016 - Using Data to Support System Improvement

http://slidepdf.com/reader/full/workshop-for-commissioners-21-january-2016-using-data-to-support-system 1/161

Using Data to Support

System Improvement

21 January 2016

0900 – 1330

London Law Society

Learning Event for Commissioners

Robert Lloyd, PhD

Vice President

Institute for Healthcare Improvement



©2015 Institute for Healthcare Improvement/R. C. Lloyd

2

Consider the following issues…

• The focus on measurement will only increase in health and social

services.

• The role of measurement: Is it for the patient, the family or the care giver?

For staff? For the public, politicians or for researchers? Who is thecustomer of the measurement system?

• Ultimately, measurement should be for those receiving the output of our

processes.

• Financial measures, for example, usually have been for someone else notthe patient or family.

• How do we open a new mind set and dialogue on measurement since

historically much of the measurement for health and social services has

been required and done by external groups and used for passing

judgement?

So, why do we need a dialogue on

Using Data to Support Heal th Systems Improvement?




3

A few more things to think about…

• If we trust the data but it is lagged by several quarters or a year or

more, how do we use it for improvement?

• How can we develop measurement systems that reflect currentperformance rather than being aggregate by quarter or year?

• During the last 5 years we have seen a new perspective emerging.

The data collected nationally are expected to “drive” improvement

at the sites of care. How do we make this happen? Can it

happen?

• Improvement can only happen if the people who produce the

actual work own the measures and the data not someone

removed from the work.

So, why do we need a dialogue on

Using Data to Support Heal th Systems Improvement?



Discussion Questions for Today

Question 1What is the difference between a Commissioningprocess that is focused on QA and one that is focused onQI? How do we strike a balance between assurance andimprovement?

Question 2How do analyse data from a QI perspective and whatquestions should we ask about the results?

Question 3How can Commissioners support providers in buildingcapacity and capability for improvement?

4



• We want to know what you think is thedefinition of quality.

• Use the sticky notes on your table.

• Fill in the following statement:

Qual ity is ___________________.

• Place your note(s) on the designated

flipchart.

What is Quality?




Quality is… a combination of value and outcome in the eyes of the consumer

a product or service delivered with 100% satisfaction the first time, every time

a product or service that provides an expected value a product that lasts, for the best price

a satisfied customer

a very good product or service - one you would want again

above standard results or outcomes

an excellent product or service delivered by professional, friendly,knowledgeable people in a timely manner at the appropriate time

an unending struggle for excellence

accurate results to health care consumers

anticipation and fulfillment of needs

A vision which provides growth and satisfaction for the customer or consumer of

our service

attentive and excellent patient care

attention to detail, timeliness, competence

being the best, best of the best!

being present for every experience

best result possible in a given category



“Quality is meeting and

exceeding the customer’sneeds and expectations and

then continuing to improve.” W. Edwards Deming

What is Quality?



On the use of Statistical Analysis

in assessing Quality in Health Care

“These statistics will enable usto ascertain what diseases andages press most heavily on theresources of particularhospitals.”

“They (i.e., the statistics) willshow subscribers how theirmoney is being spent, whatamount of good is really being

done with it, or whether themoney is doing mischief ratherthan good.” Florence Nightingale

(1820-1910)

8



Health Care Quality Improvement

“A broad range of activities of varying degrees of

complexity and methodological and statistical

rigor through which health care providers

develop, implement, and assess small-scale

interventions and identify those that work well

and implement them more broadly in order toimprove clinical practice.”

The Ethics of Improving Health Care Quality & Safety: A Hastings Center/AHRQ

Project , Mary Ann Baily, PhD, Associate for Ethics & Health Policy, The HastingsCenter, Garrison, New York, October, 2004

9



X

Is life this simple?

Patient encounter

with physician

A healthy and productive

member of society

Let’s start by thinking about the

Messiness of Life

Y

If it was this simple we wouldn’t need to be

here discussing improvement!



Life looks more like this…

X 3

X 2

X 1

X 5

X 4 Y

There are numerous direct effects between the independent

variables (the Xs) and the dependent variable (Y).

Time 1 Time 3Time 2

Patient Assessment

Score (could be

health outcomes,

functional status or

satisfaction)

I n d e p e n d e n t V a r i a b l e s

Coordination of Care

Current

health

status

Age

Gender

Communication



In this case, there are numerous direct and indirect effects between the

independent variables and the dependent variable. For example, X1 and X4both have direct effects on Y plus there is an indirect effect due to the

interaction of X1 and X4 conjointly on Y.

Y

Well, actually, it looks like this!

X 3

X 2

X 1

X 5

X 4

Time 1 Time 3Time 2

R 3

R 2

R 1

R 5

R 4

R Y

R = residuals or error terms representing the

effects of variables not included in the model.

Coordination of care

Age

Gender

CommunicationCurrent health

status

Patient Assessment

Score (could be

health outcomes,

functional status orsatisfaction)



Quality is about improving

Complex Problems! But… 13

“Some problems are so

complex that you have to be

h igh ly intell igen t and wel lin formed jus t to be undecided

about them.” --Laurence J. Peter



Walter

Shewhart(1891 – 1967) Joseph Juran

(1904 - 2008)W. Edwards

Deming

(1900 - 1993)

The Quality Pioneers



" Both pure and app l ied science have

gradual ly pushed fur ther and fur ther the

requ irements fo r accuracy and precis ion .

However, app l ied science , is even more

exact ing than pure science in certain

matters of accuracy and prec is ion ."

Dr. Walter Shewhart

A li d S i i t



Applied Science requires two

types of knowledge

SOI

Knowledge

Subject Matter

Knowledge

Science of Improvement (SOI)

Knowledge: The interplay of thetheories of systems, variation,

knowledge, and psychology.

Subject Matter Knowledge: Knowledge basic to the things wedo in life. Professional knowledge.Knowledge of work processes.



Knowledge for Improvement

SOI

Knowledge

Subject Matter

Knowledge

Improvement: Learn to combine subject matterknowledge and SOI knowledge in creative ways todevelop effective changes for improvement.

Improvement



Y

Improving the messiness of

life requires applied science.

X 3

X 2

X 1

X 5

X 4

Time 1

Time 3

Time 2

R 3

R 2

R 1

R 5

R 4

R Y



Exercise

Assessing the Messiness of Life!

Do you think Commissioners and providers regularly view issues as

being rather messy and complex or do they see them as simple problems

that should be resolved quickly and easily (i.e., X causes Y)?

List a few of these messy problems that you are currently addressing and

why they are this way.

On a scale of 1-10, how messy is each of these problems? (1 = not very

messy to 10 = extremely messy).

Do you have current measures for these messy problems that allow you

to determine just how complex and challenging each problem is?

If you have measures, do you feel that they are valid, reliable and

appropriate?



Exercise

Assessing the Messiness of Life!

What is the topic of thisMessy Problem?

How Messy is this

Problem? Select anumber 1 -10 with1 = not very messy

10 = extremely messy

List the current measures

you have for this MessyProblem?

Do you have baseline data on

these measures?

Do you feel that these

measures are valid,reliable and appropriate?




21

The Challenge…

QA QI



QualityBetter

Old Way

(Quality Assurance)

QualityBetter Worse

New Way

(Quality Improvement)

Action taken

on all

occurrences

Reject

defectives

The Challenge:

Moving from the Old Way to the New Way

Source: Robert Lloyd, Ph.D., 2012

Requirement,Specification or

Threshold

Noaction

taken

here

Worse

Th S i tifi M th d id th f d ti f ll



23

Theoretical

Concepts

(ideas & hypotheses)

Interpretation

of the Results

(asking why?)

Information

for DecisionMaking

Data

Analysis and

Output

Select &

DefineIndicators

Data

Collection(plans & methods)

Deductive Phase

(general to specific)

Inductive Phase

(specific to general)

Source: R. Lloyd Quality Health Care, 2004, p. 153.

Theory

and

Prediction

The Scientific Method provides the foundation for all

Quality Improvement models and approaches



Source: Moen, R. and Norman, C. “Circling Back: Clearing up Myths about the Deming

Cycle and Seeing How it Keeps Evolving,” Quality Progress November, 2010:22-28.

Understanding the Timeline is Critical

API Model for

Improvement

(1996)




Quality Models & Approaches

Across the Years

Human Factors/Ergonomics (Ancient Greece initially thenrefined in 1857 and then again in 1949)

The International Federation of the National Standardizing

Associations (ISA) (1926)

International Organization for Standardization (ISO) (1947)

Toyota Production System (1950s)

Six Sigma (Motorola, 1980s)

Baldrige Criteria (1987)

European Foundation for Quality Management (EFQM)

(1988)

Model for Improvement (1996)



Adding Six Sigma & Lean to the Timeline

Bill Smith (1986)

Motorola

Six SigmaMikel Harry (1988)

Motorola- MAIC

Forrest Breyfogle 111

(1992)- Integration

Michael George

(1991)- Integration

F.Taylor-The Principles of

Scientific Management

(1911)

Toyoda Family

Kiichiro Toyoda

Sakichi Tooda

Taiichi Ohno 1950-1980

Toyota Production System

Reference: Wortman 2001

Womack & Jones

Scoville & Little Comparing

Lean and Quality

Improvement (2014)

S th A di f f th



See the Appendices for further

details on the history of QI

• Evolution of Quality Management over time

– Age of Craftsman

– Age of Mass Production

– Age of Quality Management

• Evolution of Quality Management (1850-1974)


• Fourth Generation Management (Dr. Brian Joiner)

• Evolution of Quality Management in Healthcare

• What is Lean?



Institute for Healthcare Improvement, 2004

The choice of a quality system, approachor model should be driven by the

objectives of the organization, its culture

and its products or services!

The decision should NOT be driven by

how popular a particular approach is or

even if it has been used successfully inother settings.

In short…

The Key: Constancy of Purpose!




29The Quality Improvement Journey for IHI(blending Juran’s and Deming’s approaches)

Juran’s

Quality

Trilogy

QualityPlanning

Quality

Improvement

Quality

ControlDeming’s System

of Profound

Knowledge



The Juran Trilogy

The Juran Trilogy consists of three types ofactivities:

– Quality Planning,

– Quality Control (or Quality Assurance)

– Quality Improvement

Quali ty Planning : – Setting aims

– Selecting improvement projects

– Selecting team and providing resources

30



Juran on Quality Control

Quality Control (QC): “Quality control is theregulatory process through which we measure

actual quality performance, compare it with

quality goals, and act on the difference”

(Juran, 1988)

This is usually done by operations (e.g.,

clinicians and managers) with support from a QCDepartment.

31



The Juran Trilogy Journey32

Deming’s Lens of



33

Appreciation

of a system

Understanding Variation

Theory

ofKnowledge

Human

Behaviour

Deming’s Lens of

Profound Knowledge

QI

“ The system of profoundknowledge provides alens. It provides a newmap of theory by whichto understand and

optimise ourorganisations .”(Deming, Out of the Crisis )

It provides an

opportunity fordialogue and learning!



34

Appreciation for a System• Interdependence, dynamism of the parts

• The world is not deterministic

• Direct, indirect and interactive variables

• The system must have an aim

• The whole is greater than sum of the parts

Understanding Variation• Variation is to be expected!

• Common or special causes of variation

• Data for judgement or improvement?

• Ranking, tampering & performance management• Potential sampling errors

Theory of Knowledge• What theories drive the

system?

• Can we predict?

• Learning from theory and

experience

• Operational definitions(what does a concept

mean?)

• PDSAs for learning and

improvement

Human Behavior• Interaction between people

• Intrinsic versus extrinsic

motivation

• Beliefs, values & assumptions

• What is the Will to change?

What insi ts might be obtained by looking

through the Lens of Profound Knowledge

Exercise



• Apply the Lens of Profound Knowledge to an improvementproject.

• This is best accomplished with an improvement team.

• Use the PK Worksheet (next page) to record yourresponses. Remember that there are no right or wrong

responses.

• Engage in a dialogue on PK (not a debate, a discussion or

idle chit-chat but rather a true dialogue about the theoriesand assumptions surrounding the project and the degree to

which it is “messy.”

• Share the results of this exercise with others to obtain their

thoughts and input.

Exercise

Profound Knowledge



36

Profound Knowledge Worksheet

Appreciation for a System

•

•

•

•

• Human Behaviour

•

•

•

Theory of Knowledge

•

•

•

•


•

•

•

•




37

Can you help providers start

to apply Profound

Knowledge to their messy

problems?



38

• Is applicable to all types oforganizations.

• Provides a framework for the

application of improvement

methods guided by theory.

• Emphasizes and encourages the

iterative learning process of

deductive and inductive thinking.

• Allows project plans to adapt as

learning occurs.

1996 API* added three basic questions to supplement the PDSA Cycle.

The PDSA Cycle is used to develop, test, and implement changes.

*API = Associates in Process Improvement



Langley, J. et al. The Improvement Guide. Jossey-Bass Publishers, 2009.

The IHI Approach

When you

combine

the 3

questionswith the…

…the Model

forImprovement.

PDSA cycle,you get…

Foundation for the QI Learning



Foundation for the QI LearningDeming’s System of Profound Knowledge

Seven Propositions:1. Grounded in the Scientific Method

2. Foundation of conceptualistic pragmatism

3. Embraces a weak from of psychologism

4. Considers context of justification and discovery

5. Recognizes value of operationism

6. Variation is defined by chance-cause system

7. Systems theory

Subject Matter Knowledge

Key Improvement Methods:Model for Improvement with PDSA

Shewhart charts

Operational Definitions

Analytic StudiesGraphical Data Analysis

Intrinsic motivation

Multi-disciplinary teams

Improvement

A r e u

s e d w i t h

Characteristics of the Applied Science of Improvement:

1. Bias toward action learning2. Focus on prediction of future outcomes

3. Multiple testing cycles before implementation

4. Visual display to learn from data

5. Learning from special and common causes

6. Simple and complex study designs

7. Ongoing interaction of scientists and practitioners

Provides the Philosophical

and Theoretical Base for

Source: Provost, L., Perla, R., Parry, G.,Seven Propositions of the Science of Improvement: ExploringFoundations. Q Manage Health Care Vol. 22, No. 3, 2013: 70 –186.

Di l



Dialogue

Science of Improvement

What is your current level of knowledge about theScience of Improvement (SOI)?

Could you explain to a provider how the SOI can

help them to achieve better performance?

Are you and your colleagues prepared to engage ina dialogue with providers on how to move from a QAperspective to a QI perspective?

What structures and process can be established tosupport providers in their quality journeys?



Improvement?(improving the effectiveness or

efficiency of a process)

Accountab i l i ty

or Judgement?(making comparisons;

no change focus)

Research?(testing theory and building

new knowledge; efficacy)

The answer to this question will guide your entire

quality measurement journey!

Why are you measuring?

The Three Faces of



The Three Faces of

Performance Measurement

Aspect Improvement

Accountability

(Judgement) ResearchAim Improvement of care

(efficiency & effectiveness)

Comparison, choice,

reassurance, motivation for

change

Build new theories and

knowledge

(efficacy)

Methods:

• Test ObservabilityTest observable

No test, evaluate current

performance Test blinded or controlled

• Bias Accept consistent bias Measure and adjust to

reduce bias

Design to eliminate bias

• Sample Size “Just enough” data, small

sequential samples

Obtain 100% of available,

relevant data

“Just in case” data

• Flexibility of

Hypothesis

Flexible hypotheses, changes

as learning takes place No hypothesis

Fixed hypothesis

(null hypothesis)

• Testing Strategy Sequential tests No tests One large test

• Determining if achange is animprovement

Analytic Statistics

(statistical process control)

Run & Control charts

No change focus

(maybe compute a percent

change or rank order the

results)

Enumerative Statistics

(t-test, F-test,

chi square,

p-values)

• Confidentiality ofthe data

Data used only by those

involved with improvement

Data available for public

consumption and review

Research subjects’ identities

protected

Adapted from: Lief Solberg, Gordon Mosser and Sharon McDonald,Journal on

Quality Improvement vol. 23, no. 3, (March 1997), 135-147.




Source: Provost, Murray & Britto (2010)

Example of Data for Judgement




Slide #45Slide #45

How Is the Error Rate Doing?





Slide #46Slide #46

How is Perfect Care Doing?


So how do you view the Three Faces




So, how do you view the Three Faces

of Performance Measurement?

Or,

As… As a…

I m p r o

v e m e n t

J u d

g m e n t

R e

s e a r c h

Integrating the




The three faces of performancemeasurement should not be seen as

mutually exclusive silos. This is not an

either/or situation.

All three areas must be understood asa system. Individuals need to build

skills in all three areas.

Organizations need translators who

and be able to speak the language ofeach approach.

The problem is that individuals identify

with one of the approaches and

dismiss the value of the other two.

Integrating the

Three Faces of Performance Measurement

Dialogue




Dialogue

Why are you measuring?

• How much of your organization’s energy is aimed at

improvement, accountability and/or research?

• Does one form of performance measurement dominate

your journey?

• Is your organization building silos or a Rubik's cube when it

comes to data collection and measurement?

• Do you think the three approaches can be integrated or arethey in fact separate and distinct silos?

• How many “translators” exist within your organization? Are

people being developed for this role?

Now how would you design a study to



©2015 Institute for Healthcare Improvement/R. C. Lloyd50

Now, how would you design a study to

improve performance?

Li fe is fu l l of

opt ions!

E ti A l ti St di d



Enumerative versus Analytic Studies and

Related Statistical Techniques

“The teaching of pure statistical theory in universities, including

the theory of probability and related subjects is almost

everywhere excellent. Application to enumerative studies is

mostly correct, but application to analytic problems is deceptive

and misleading.

Analysis of variance, t-test, confidence intervals, and other

statistical techniques taught in books, however interesting, are

inappropriate because they provide no basis for prediction and

because they bury the information contained in the order of production. Most if not all computer packages for analysis of

data, as they are called, provide flagrant examples of

inefficiency.” Dr. Deming, Out of the Cr isis, page 132.



Deming classified studies into two types depending on the type of actionthat will be taken:

• Enumerative Studies – ones in which action will be taken on the entire

universe. The aim of an enumerative study is estimation of some aspect of the

universe. Action will be taken on the universe based on this estimate through the

sampling frame. The U.S. Census is a classic example of an enumerative study.

• Analytic Studies – ones in which action will be taken on a cause system to

improve performance of a product, process, or system in the future. The aim of an

analytic study is prediction that one of several alternatives will be superior to the

others in the future.

In an analytic study, the focus is on the cause system. There is no

identifiable universe, as there is in an enumerative study, and, therefore, no

frame.

Source: Quali ty Improvement Through Planned Experimentat ion by R. Moen, T. Nolan and L.

Provost, McGraw-Hill, New York, 1999, 2nd edition.

Enumerative versus Analytic Studies



“On Probability as Basis for Action”

W. E. Deming, The American

Stat ist ic ian , November 1975, vol. 29,

No. 4. Pages 146-152.

53

“It is possible, in an

enumerative problem, to

reduce errors of sampl ingto any specif ied level. In

contrast, in an analyt ic

problem, i t is imposs ib le

to compute the r isk of

mak ing a wrong

decision.”



Enumerative and Analytic Studies

Enumerative: a Pond Analytic: a River

Fixed population-universe, frameRandom sampling

Probability based

Purpose:

- determine how much variation in a sample

- apply learning to the sample

(should not extrapolate)

- reject or do not reject sampled population

Hypothesis, statistical tests (t-test,F-test, chi square, p-values)

No fixed populationPopulation-ongoing “stream” of data

Also uses judgment sampling

Not totally based on probability

Purpose:

-how much variation, what type

-take action on underlying process to

Improve future outcome of process

Run charts or Shewhart control charts

Pull one sample from

this spot, walk away

and make a conclusion

about the total pond!

But, how do you pull

a sample from a

moving process?



• Descriptive Study – summarize all the fish in one barrel by type.

• Enumerative Study – take a sample from one barrel as a point estimate(audit) of the fish and generalize to all barrels on the boat’s deck.

• Analytic Study – understand the process that places fish in one barrel by

studying previous and future barrels. Why are these fish in this barrel?

Different Types of Studies

The approach toresearch and the

statistical methodsused should be based

on the question(s)being asked.



Does th is purp ose sound l ike i t w i l l be

an enumerative or analyt ic study?

56

Case Study: The Chicago Tr ibuneMonday, September 19, 2011

“The purpose of the study, whichrepresents the most

com prehensive exam inat ion o f

rai lroad pedestr ian fatal i t ies in

no rtheastern Il l inois, was to

determ ine the facto rs leading to

the inc idents and recommend

solut ions the researchers said.”



57

Variables in the study

• Train type (Metra, Amtrak or Freight)

• Number of pedestrian deaths by age

• Number of pedestrian deaths by gender

• Pedestrian death rate by Metra route

• Pedestrian deaths (count) and rate by municipality

• Percentage of deaths by season

The Chicago Tr ibuneMonday, September 19, 2011

Th Chi T ib



58

The Chicago TribuneMonday, September 19, 2011



59

“Fatal rail pedestrian

inc idents are occurr ing

at an average of about

one every 10 days in

the Chicago area,” the

study said. “Last

week, there were two,

both on Thursday.”

The Chicago TribuneMonday, September 19, 2011

Now what do you th ink?

Is th is an enumerat ive or analy t ic s tudy?

Enumerative Studies frequently



Enumerative Studies frequently

suffer from 20-20 Hindsight!

“Managing a process on the basis of monthly

(or quarter ly) averages is l ike trying to d rive a

car by looking in the rear view mirror.”

D. Wheeler

UnderstandingVariation, 1993.

Dialogue 61



Dialogue

Enumerative and Analytic Studies

When you consider the use of data in the CommissioningProcess, do you think it is designed around an Enumerative

or an Analytic approach?

If it is more aligned more with an Enumerative approach,how will this lead to improving care processes and

outcomes?

If you think the use of data in the Commissioning Process

is more aligned with an Analytic approach, then what are

you doing to convey this approach to providers?

61

Read more about Enumerative and



In the spring of 2010 the BMJ sponsored the Vin McLoughl in Sympos ium on the

Epistemology of Improvin g Heal th Care . The papers that grew out of this symposium

are freely available online under the BMJ journal’s unlock scheme:

http://qualitysafety.bmj.com/site/about/unlocked.xhtml

Read more about Enumerative and

Analytic Studies

BMJ Qual i ty & Safety

April 2011 Vol. 20, No Suppl. 1

Epis temology (from Greek epistēmē), meaning"knowledge, science", and (logos), meaning "study

of" is the branch of philosophy concerned with the

nature and scope (limitations) of knowledge.

It addresses the questions:

• What is knowledge?

• How is knowledge acquired?

• How do we know what we know?

M t f th 2 d ti





Langley, G. et al, The Improvement Guide, API, 2009

Measurement focuses on the 2nd question

But, do you know the Milestones



, y

in the Quality Measurement Journey (QMJ)?

Milestones in the



65

AIM (How good? By when?)Concept

Measure


Data Collection Plan

Data Collection

Analysis ACTION

Source: R. Lloyd. Quality Health Care: A Guide to Developing and

Using Indicators. Jones and Bartlett Publishers, 2004.

Quality Measurement Journey

Milestones in the



AIM – reduce patient falls by 37% by the end of the year

Concept – reduce patient falls

Measures – Inpatient falls rate (falls per 1000 patient days)

Operational Definitions - # falls/inpatient days

Data Collection Plan – weekly; no sampling; all IP units

Data Collection – unit collects the data

Analysis – control chart (u-chart) ACTION


Milestones in the



67

AIM (How good? By when?)

Concept

Measure



Data Collection

Analysis ACTION






NHS Mental Health Dashboard

But remember to build a



69

Look

your

at

system

as a cascade!

of measures

But remember to build a

Cascading System of Measures

A Cascading Approach to Measurement



Percent compliancewith all “bundles”

Percent

compliance

with

Pathology

investigation

s bundle

Percent

compliance

with Cardiac

investigation

s bundle

Percent

compliance

with

Physical

observation

s bundle

Complication

rates

+ +

Percent service userson antipsychotics with

baseline investigations

M CRO

MESO

MICRO



©Copyright 2013 Institute for Healthcare Improvement/R. Lloyd71

AIM (Why are you measuring?)

Concept

MeasureOperational Definitions


Data CollectionAnalysis ACTION

The Quality Measurement Journey



72

You have performance data!

Now, what do you

do with it?

U d t di i ti t ll



73

“If I had to reduce

my message for

management to justa few words, I’d say

i t al l had to do w ith

reducing variation.” W. Edwards Demin g

Understanding variation conceptually

Th P bl !



74

The Problem!

Aggregated data presented in tabular

formats or with summary statistics,

will not help you measure the impactof process improvement efforts.

Aggregated data can only lead to

judgment, not to improvement.

Average Percent of Patients who Fall



75

Average Percent of Patients who FallStatic View of Before and After the Implementation of a New Protocol

P e r c e n t o f P a t

i e n t s

w h o F a l l

Time 1 Time 2

3.8

5.2

5.0%

4.0%

WOW!A “sign i f icant drop ”

from 5% to 4%

Conclusion -The protocol was a success!

A 20% drop in the average mortality!

Protocol implemented here

Average Percent of Patients who Fall



76

24 Months

1.0

9.0

Now what do you conclude about the

impact of the protocol?

5.0

UCL= 6.0

LCL = 2.0

CL = 4.0

Protocol implemented here

Average Percent of Patients who FallDynamic View of Before and After the Implementation of a New Protocol

P e r c e n t o f P a

t i e n t s

w h o F a l l

If you don’t understand the variation that



77

y

lives in your data, you will be tempted to ...

• Deny the data (It doesn’t fit my view of reality!)

• See trends where there are no trends

• Try to explain natural variation as special events

• Blame and give credit to people for things over

which they have no control

• Distort the process that produced the data

• Kill the messenger!

D C b ll' I i ht Di t ti



Dr. Campbell's Insight on Distortion

P78

“The more any quantitative social

indicator is used for social decision-

making, the more subject it will be to

corruption pressures and the more apt itwill be to distort and corrupt the social

processes it is intended to monitor.”

"Campbell's Law" from Assessing the Impact of Planned Social

Change, 1976

http://www.sciencedirect.com/science/article/pii/014971897990048X

https://www.globalhivmeinfo.org/CapacityBuilding/Occasional

%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdf

Donald T. Campbell,

Ph.D., social

psychologist

(1916-1996)

D D i ’ C l f F

https://www.globalhivmeinfo.org/CapacityBuilding/Occasional%20Papers/08%20Assessing%20the%20Impact%20of%20Planned%20Social%20Change.pdf








Dr. Deming’s Cycle of Fear

Source: William Scherkenbach. The Deming Route to Quality and Productivity. Ceep Press, Washington, DC, 1990, page 71.

K il l the

MessengerIncreased

Fear

Filtered

Informat ion

Micro-

management



“A phenomenon will

be said to be

contro l led when,

through the use ofpast experience, we

can predict , at least

w i th in l im i ts , how the

phenomenon may beexpected to vary in

the future” W. Shewhart. Economic Control of

Quality of Manufactured Product , 1931

Dr. Walter A Shewhart

“What is the variation in one system



over time?” Walter A. Shewhart - early 1920’s, Bell Laboratories

81

time

UCL

Every process displays variation:

• Controlled variationstable, consistent pattern of variation

“chance”, constant causes

• Special cause variation“assignable”

pattern changes over time

LCL

Static View

S t a t i c V i e

w

Dynamic View

Types of Variation



82

Common Cause Variation• Is inherent in the design of the

process

• Is due to regular, natural or ordinarycauses

• Affects all the outcomes of a process

• Results in a “stable” process that ispredictable

• Also known as random orunassignable variation

Special Cause Variation• Is due to irregular or unnatural

causes that are not inherent in the

design of the process

• Affect some, but not necessarilyall aspects of the process

• Results in an “unstable” process

that is not predictable

• Also known as non-random or

assignable variation

Types of Variation

P i t V i ti i t !



Point …Variation exists!

Common Cause does not mean “Good Variation.” It only

means that the process is stable and predictable. For

example, if a patient’s systolic blood pressure averaged

around 165 and was usually between 160 and 170 mmHg,this might be stable and predictable but completely

unacceptable.

Similarly Special Cause variation should not be viewed as

“Bad Variation.” You could have a special cause thatrepresents a very good result (e.g., a low turnaround time),

which you would want to emulate. Special Cause merely

means that the process is unstable and unpredictable.

Appropriate Management Response to



84

pp p g p

Common & Special Causes of Variation

Type of variation

Right Choice

Wrong Choice

Consequences of

making the wrong

choice

Is the process stable?

YES NO

Only Common

If not at targetchange the process

Treat normal variation as a

special cause (tampering)

Increased

variat ion!

Special + Common

Change the process

Wasted

resources!( t ime, effort, mo rale,

money)

Investigate the origin ofthe special cause

Source: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process

Control Applications. ASQ Press, Milwaukee, WI, 2001, page 153.

2 Questions



85

2 Questions …

1. Is the process s table?

If so , it is p red ic table.

2. Is the process capable?

The chart w i l l tel l you i f the process is

stable and predic table.

You have to decide if the outpu t of the process is capable ofmeeting th e target or goal you have set!

(NOTE: we wil l talk abou t sett ing targets and goals sho rt ly)

Attributes of a Leader Who



Understands Variation

Leaders understand the different ways that variation isviewed.

They explain changes in terms of common causes and

special causes.

They use graphical methods to learn from data and

expect others to consider variation in their decisions

and actions.

They understand the concept of stable and unstableprocesses and the potential losses due to tampering.

Capability of a process or system is understood before

changes are attempted.

Dialogue



©Copyright 2013 Institute for Healthcare Improvement/R. Lloyd

• Select several measures you review on a regular

basis.

• Do you and other CCG members as well as

providers evaluate these measures according the

criteria for common and special causes of

variation?

• If not, what criteria do you use to determine ifdata are improving or getting worse?

• Do these methods allow you to understand the

variation inherent in the data?

1 2 / 9 5

2 / 9 6

4 / 9 6

6 / 9 6

8 / 9 6

1 0 / 9 6

1 2 / 9 6

2 / 9 7

4 / 9 7

6 / 9 7

8 / 9 7

1 0 / 9 7

1 2 / 9 7

2 / 9 8

4 / 9 8

6 / 9 8

8 / 9 8

1 0 / 9 8

1 2 / 9 8

2 / 9 9

4 / 9 9

6 / 9 9

m ont h

P e r c e n t

C - s e c t i o n s

0. 0

5. 0

10. 0

15. 0

20. 0

25. 0

30. 0

35. 0

UCL = 2 7 . 7 0 1 8

CL=18. 0246

L CL = 8 . 3 4 7 3

nt of Cesa rean Sections Performed Dec 95 - Jun

W eek

N u m

b e r

o f

M

e d i c a t i o n s

E r r o r s

p e r

1 0 0 0

P a t i e n t

0 . 0

2 . 5

5 . 0

7 . 5

1 0 . 0

1 2 . 5

1 5 . 0

1 7 . 5

2 0 . 0

2 2 . 5

UCL = 1 3 . 3 9 4 6 1

CL =4 . 4 2 0 4 8

L CL = 0 . 0 0 0 0 0

Medication ErrorRate

DialogueCommon and Special Causes of Variation

Conclusions




1. The same data can show different patterns of variation

dependent on how much of it you present and how you

statistically analyse and display the data.

2. Data presented over time (i.e., plotting the data by day,week or month) is the only way you will ever be able to

improve any aspect of quality or safety!

3. Avoid using aggregated data and enumerative statistics if

you are serious about improving quality and safety!

4. A leaders job is to understand patterns of variation and

ask why!


Understand variation statistically




89

STATIC VIEW

Descriptive StatisticsMean, Median & Mode

Minimum/Maximum/RangeStandard Deviation

Bar graphs/Pie charts

DYNAMIC VIEWRun Chart

Control Chart

(plot data over time)

Statistical Process Control (SPC)

R a t e p e r 1 0 0 E D P

a t i e n t s

Unplanned Returnsto Ed w/in72 Hours

M41.78

17

A43.89

26

M39.86

13

J40.03

16

J38.01

24

A43.43

27

S39.21

19

O41.90

14

N41.78

33

D43.00

20

J39.66

17

F40.03

22

M48.21

29

A43.89

17

M39.86

36

J36.21

19

J41.78

22

A43.89

24

S31.45

22

Month

ED/100

Returns

u chart

1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9

0.0

0.2

0.4

0.6

0.8

1.0

1.2

UCL=0.88

Mean=0.54

LCL=0.19

Understand variation statistically

How do we analyze variation for



90

quality improvement?

With Stat ist ical Process Con tro l (SPC) charts!

Run and Con trol Charts are the best

tools to determine:

1. The variation that lives in the process

2. If our improvement strategies have had thedesired effect.

Three Uses of SPC Charts



Process Improvement: Isolated Femur Fractures

0

200

400

600

800

1000

1200

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients

M i n u t e s E D

t o O R

p e r

P a t i e n t

Holding the Gain: Isolated Femur Fractures

0

200

400

600

800

1000

1200


M i n u t e s E D t o O R

p e r

P a t i e n

t

3. Determine if we are holding the gains

Current Process Performance: Isolated Femur Fractures

0

200

400

600

800

1000

1200


M i n u t e s E D

t o O R

p e r

P a t i e n t

Three Uses of SPC Charts

2. Determine if a change is an

improvement

1. Make process performance visible

Plotting dataover time to

understand the

variation!

How do we analyze variation

t ti ti ll f lit i t?




92

statistically for quality improvement?

M e a s u r e

Time

M e a

s u r e

Time

A Run Chart:

• is a time series plot of data

• The centerline is the Median

• 4 Run Chart rules are used to determine

if there are random or non-random

patterns in the data

A Control Chart:

• is a time series plot of data

• The centerline is the Mean

• Added features include Upper and lowercontrol Limits (UCL & LCL)

• 5 Control Chart rules are used to

determine if the data reflect common or

special causes of variation

Run Chart

Control Chart




Let’s start fitting the pieces

together

The Goal: To build information and learning for improvement.

Organisation Name Region April 14

Dementia

Diagnosis Rate

May 14

Dementia

Diagnosis

Rate

June 14

Dementia

Diagnosis

Rate



Rate Rate

NHS Barking & Dagenham CCG NE 55.10 54.58 55.33

NHS Harrow CCG NW 38.14 38.37 38.76

NHS Redbridge CCG NE 49.95 49.71 50.03

NHS Sutton CCG South 45.13 45.18 46.40

NHS Havering CCG NE 45.47 46.17 47.11

NHS Richmond CCG South 52.85 52.31 53.40

NHS Kingston CCG South 42.53 41.98 41.99

NHS Croydon CCG South 46.50 46.50 46.73

NHS Camden CCG NE 64.88 65.27 65.21

NHS Hillingdon CCG NW 42.84 41.38 42.88

NHS Bexley CCG South 50.04 50.18 50.91

NHS Enfield CCG NE 49.49 49.08 50.10

NHS Greenwich CCG South 54.80 54.64 55.33

NHS Bromley CCG South 44.89 44.98 45.21

NHS Lewisham CCG South 53.52 53.62 54.50

NHS Wandsworth CCG South 56.12 56.17 56.86

LONDON AREA TEAM LAT 54.94 54.90 55.49

NHS West London (K&C & QPP) CCG NW 57.35 57.41 56.05

NHS City and Hackney CCG NE 68.78 68.53 68.54

NHS Newham CCG NE 63.87 63.68 63.82

NHS Merton CCG South 49.88 49.46 50.52

NHS Southwark CCG South 58.57 55.74 56.33

NHS Waltham Forest CCG Ne 54.29 54.48 54.69

NHS Barnet CCG NE 57.53 57.65 57.50

NHS Hammersmith and Fulham CCG NW 57.03 57.20 60.32

NHS Hounslow CCG NW 54.26 53.77 53.73

NHS Central London (Westminster) CCG NW 59.15 59.59 61.10

NHS Brent CCG NW 54.37 55.23 55.86

NHS Haringey CCG NE 53.92 53.57 55.72

NHS Tower Hamlets CCG NE 66.62 66.97 66.89

NHS Ealing CCG NW 54.19 54.28 54.94

NHS Lambeth CCG South 55.50 57.50 57.71

NHS Islington CCG NE 69.88 70.41 70.27

Organisation Name Region April 14 Dementia

Diagnosis Rate

May 14

Dementia

Diagnosis

Rate

June 14

Dementia

Diagnosis

Rate

July 14

Dementia

Diagnosis

Rate

August 14 Dementia

Diagnosis Rate

September 14

Dementia

Diagnosis Rate

October 14

Dementia

Diagnosis Rate

November 14

Dementia

Diagnosis Rate

December 14

Dementia

Diagnosis Rate

January 15

Dementia

Diagnosis Rate

February 15

Dementia

Diagnosis Rate

March 15

Dementia

Diagnosis Rate




Dementia Diagnosis Rates for 32 NHS CCGs, April 2014-March 2015

NHS Barking & Dagenham CCG NE 55.10 54.58 55.33 55.57 54.31 56.25 59.47 61.69 62.77 62.84 63.07 63.96

NHS Harrow CCG NW 38.14 38.37 38.76 37.97 37.44 40.09 40.24 42.30 43.14 39.35 43.29 50.30

NHS Redbridge CCG NE 49.95 49.71 50.03 49.21 48.18 48.98 49.30 53.45 55.71 56.05 57.38 59.62

NHS Sutton CCG South 45.13 45.18 46.40 45.51 44.13 47.94 54.31 54.63 56.68 55.82 56.21 55.56

NHS Havering CCG NE 45.47 46.17 47.11 46.42 46.35 47.67 48.20 49.67 49.87 50.15 51.14 51.61

NHS Richmond CCG South 52.85 52.31 53.40 51.40 50.76 53.07 52.06 54.83 55.82 58.04 60.20 63.60

NHS Kingston CCG South 42.53 41.98 41.99 41.82 39.27 41.12 40.62 42.82 48.17 49.30 51.28 51.92

NHS Croydon CCG South 46.50 46.50 46.73 46.66 46.18 46.51 46.28 47.51 48.78 50.33 51.43 51.83

NHS Camden CCG NE 64.88 65.27 65.21 63.84 62.56 65.02 66.57 67.39 67.00 67.45 67.00 68.73

NHS Hillingdon CCG NW 42.84 41.38 42.88 41.62 41.75 42.37 42.99 43.95 47.09 48.72 52.40 54.23

NHS Bexley CCG South 50.04 50.18 50.91 49.86 51.11 50.41 50.38 51.87 52.63 53.65 55.41 57.56

NHS Enfield CCG NE 49.49 49.08 50.10 48.14 49.03 51.91 52.29 52.51 53.78 55.68 56.44 59.73

NHS Greenwich CCG South 54.80 54.64 55.33 55.60 55.77 56.84 56.12 57.78 59.72 59.88 62.95 69.33

NHS Bromley CCG South 44.89 44.98 45.21 43.81 43.46 44.94 48.07 48.22 49.51 49.99 52.30 57.56

NHS Lewisham CCG South 53.52 53.62 54.50 53.77 54.28 52.96 53.33 52.61 52.94 53.17 58.36 61.52

NHS Wandsworth CCG South 56.12 56.17 56.86 56.03 54.87 55.95 55.78 56.48 55.92 56.37 58.62 58.61

LONDON AREA TEAMLAT 54.94 54.90 55.49 54.72 54.51 55.62 56.35 57.79 58.87 60.33 62.60 65.79

NHS West London (K&C & QPP) CCG NW 57.35 57.41 56.05 55.77 53.71 57.91 61.53 63.26 64.69 65.23 68.57 73.06

NHS City and Hackney CCG NE 68.78 68.53 68.54 66.51 66.17 67.83 68.83 67.96 68.22 68.54 69.41 70.22

NHS Newham CCG NE 63.87 63.68 63.82 64.14 62.66 63.85 63.71 63.93 64.77 65.81 65.68 68.35

NHS Merton CCG South 49.88 49.46 50.52 49.75 49.48 51.86 51.30 52.39 53.52 55.80 57.52 66.45

NHS Southwark CCG South 58.57 55.74 56.33 55.66 58.04 57.16 58.52 63.19 63.47 64.39 67.49 68.54

NHS Waltham Forest CCG Ne 54.29 54.48 54.69 53.99 53.25 54.09 53.77 56.48 56.52 62.97 66.36 70.31

NHS Barnet CCG NE 57.53 57.65 57.50 57.47 56.60 57.57 57.78 57.96 58.52 62.64 64.30 67.70

NHS Hammersmith and Fulham CCG NW 57.03 57.20 60.32 60.41 60.17 62.23 61.47 60.11 60.49 62.94 65.63 68.18

NHS Hounslow CCG NW 54.26 53.77 53.73 53.43 52.84 54.26 54.73 54.25 55.18 57.55 61.99 69.68

NHS Central London (Westminster) CCG NW 59.15 59.59 61.10 59.67 59.97 62.60 62.17 63.25 63.38 64.76 69.88 71.68

NHS Brent CCG NW 54.37 55.23 55.86 55.80 55.05 55.89 56.58 58.87 59.58 66.06 68.97 70.70

NHS Haringey CCG NE 53.92 53.57 55.72 54.85 53.21 54.16 53.48 54.30 55.31 56.94 61.17 64.23

NHS Tower Hamlets CCG NE 66.62 66.97 66.89 66.86 67.54 66.52 66.71 66.45 66.14 71.40 71.93 73.09

NHS Ealing CCG NW 54.19 54.28 54.94 54.49 56.45 54.80 55.13 57.21 57.60 57.91 60.14 62.98

NHS Lambeth CCG South 55.50 57.50 57.71 57.71 57.53 58.18 62.70 63.80 64.74 64.99 65.28 64.30

NHS Islington CCG NE 69.88 70.41 70.27 69.39 67.82 69.03 68.85 69.08 71.27 72.91 74.70 77.83

How do we improve performance of the system with this data?




NHS Mental Health Dashboard:

The beginning of a bridge betweenEnumerative and Analytic studies

80.00London Area Team - I Chart

B l ’ l k h d f

Created by Forid Alom, ELFT



UCL

LCL

35.00

40.00

45.00

50.00

55.00

60.00

65.00

70.00

75.00

A p r - 1 4

M

a y - 1 4

J

u n - 1 4

J u l - 1 4

A

u g - 1 4

S

e p - 1 4

O c t - 1 4

N

o v - 1 4

D

e c - 1 4

J

a n - 1 5

F

e b - 1 5

M

a r - 1 5

D e m e n

t i a D i a g n o s i s R a t e

Mean = 57.6

But now, let’s look at the data from

an Analytic Approach:

32 CCGs (London Team)

All London Area Teams Dementia Diagnosis Rate

April 2014-March 2015

A Trend: 6 or more consecutive data

point increasing (or decreasing)

80.00I Chart of selected 18 CCG'sCreated by Forid Alom, ELFT



UCL

LCL

35.00

40.00

45.00

50.00

55.00

60.00

65.00

70.00

75.00

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J

u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

D e

m e n t i a D i a g n o s i s R a t e

A Trend: 6 or more consecutive data

point increasing (or decreasing)

18 London Area Teams Dementia Diagnosis Rate

April 2014-March 2015

Mean = 53.5

Looking at Data from an

Analytic Approach:18 CCGs

70

75

80

s i s R a t e

NHS Barking &Dagenham CCG - I

Chart

NHS Harrow CCG- I Chart

NHS RedbridgeCCG - I Chart

NHS Sutton CCG -I Chart

NHS HaveringCCG - I Chart

NHS RichmondCCG - I Chart



UCL

LCL

35

40

45

50

55

60

65

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

D e m e n t i a D i a g n o

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

35

40

45

50

55

60

65

70

75

80

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

D e m e n t i a D

i a g n o s i s R a t e

NHS Kingston CCG - IChart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS CroydonCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Camden CCG- I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS HillingdonCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Bexley CCG -I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Enfield CCG -I Chart

UCL

LCL

35

40

45

50

55

60

65

70

75

80

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

D e m e n t i a D i a g n o s i s R a t e

NHS Greenwich CCG - IChart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Bromley CCG- I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS LewishamCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS WandsworthCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS West London(K&C & QPP) CCG

- I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS City andHackney CCG - I

Chart

Dashboard of 18 London Area Teams Dementia Diagnosis Rates, April 2014-March 2015


Exercise

Understanding Variation across 18 CCGs

100




Understanding Variation across 18 CCGs

For these 18 selected CCGs:

• What do we learn from these 18 charts?

• Are all 18 CCGs performing the same?

• Do all 18 charts match the overall performance pattern shownon the aggregated chart?

• Do these 18 CCGs exhibit common or special causes of

variation?

• What will it take to get these 18 CCGs performing as asystem?

• Should each CCG’s improvement strategy be the same?

• Are any of the CCGs demonstrating excellent performance?

70

75

80

o s i s R a t e

NHS Barking &Dagenham CCG - I

Chart

NHS Harrow CCG- I Chart

NHS RedbridgeCCG - I Chart

NHS Sutton CCG -I Chart

NHS HaveringCCG - I Chart

NHS RichmondCCG - I Chart



UCL

LCL

35

40

45

50

55

60

65

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

D e m e n t i a D i a g n o

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

UCL

LCL

35

40

45

50

55

60

65

70

75

80

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

D e m e n t i a D

i a g n o s i s R a t e

NHS Kingston CCG - IChart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS CroydonCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Camden CCG- I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS HillingdonCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Bexley CCG -I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Enfield CCG -I Chart

UCL

LCL

35

40

45

50

55

60

65

70

75

80

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

D e m e n t i a D i a g n o s i s R a t e

NHS Greenwich CCG - IChart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS Bromley CCG- I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS LewishamCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS WandsworthCCG - I Chart

UCL

LCL

A p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

NHS West London(K&C & QPP) CCG

- I Chart

UCL

LCL

A p r -

1 4

M a y -

1 4

J u n -

1 4

J u l -

1 4

A u g -

1 4

S e p -

1 4

O c t -

1 4

N o v -

1 4

D e c -

1 4

J a n -

1 5

F e b -

1 5

M a r -

1 5

NHS City andHackney CCG - I

Chart

Dashboard of 18 London Area Teams Dementia Diagnosis Rates, April 2014-March 2015





Finally, we developed a dashboard of the 18

CCGs performance over time on control

charts.

Then, we looked at the aggregate performance

for a segment of the system (18 CCGs)

So, we’ve looked at the aggregate performance

for the entire system (all 32 CCGs in the

London area).





I know…what can

a CCG do toimprove system

performance?

What can a CCG do to supportsystem improvement?

104




system improvement? Use the Commissioning data and the related findings to identify

opportunities for provider improvement. Help providers to take responsibility for their data.

Understand the factors that drive a particular measure.

Look at data as a time series not in the aggregate or with summary

statistics.

Work with providers to set up improvement teams to work on improving

the measures.

Stress that providers need to identify a dedicated group of QI advisors and

coaches who can support the improvement teams in their work.

Build capacity and capability for improvement thinking and practice

throughout the system (from the Board and Non-Execs through senior

management, middle management and front-line staff)

Create a process to review progress of the improvement teams.

Be transparent with data and results.



ReducingHarm

Physicalviolence

Medicationerrors

Falls

Pressureulcers

Restraints

It starts with

having a

strategic focus!

Right care,right place,right time

Improvingpatient and

carerexperience

Reliable deliveryof evidence-based care

Reducing delaysand

inefficiencies inthe system

Improved accessto services at

the rightlocation

A Driver Diagram with Aim, Primary

and Secondary Drivers




It thenrequires

identifying

the factors

that drive theoutcomes!

AIM

Primary

DriversSecondary Drivers




A plan for building capacity and capability for

the science of improvement is also essential

Estimated number needed to train = 5000

Needs = introduction to quality

improvement, identifying problems, change

ideas, testing and measuring change

Pocket QI commenced in October2015. Aim to reach 200 people by

Dec 2016.

All staff receive intro to QI at

induction



Experts by experience

All staff

Staff involved in or

leading QI projects

QI coaches

Board


Needs = deeper understanding of

improvement methodology, measurement

and using data, leading teams in QI


Needs = deeper understanding of

improvement methodology, understanding

variation, coaching teams and individuals

Needs = setting direction and big goals,

executive leadership, oversight of

improvement, being a champion,

understanding variation to lead

Estimated number needed to train = 11Needs = deep statistical process control,

deep improvement methods, effective plans

for implementation & spread

induction

500 people have undertaken the

ISIA so far. Wave 5 = Luton/Beds

(Sept 2016 – Feb 2017)

30 QI coaches graduating in

January 2016. To identify and train

second cohort in mid-late 2016

Most Executives will have

undertaken the ISIA.

Annual Board session with IHI &

regular Board development

discussions on QI

Currently have 3 improvementadvisors, with 1.5 wte deployed to

QI. To increase to 8 IA’s in 2016/17

(6 wte).

Internalexperts (QI

team)

Bespoke QI learning sessions for

service users and carers. Over 50

attended in 2015. Build into recovery

college syllabus, along with

confidence-building, presentation

skills etc.

Needs = introduction to quality

improvement, how to get involved in

improving a service, practical skills in

confidence-building, presentation,

contributing ideas, support structure for

service user involvement

Then it is time to lay out your



Quality Dashboard(organisation-level view)

Then it is time to lay out your


ELFT Quality Dashboards



Safety



Finally, build the ability to track individual teams



ACCESS TO SERVICES

COLLABORATIVEDASHBOARDDecember 2015

Finally, build the ability to track individual teams

December 2015 1- Baseline data

UCL

70Average waiting time from referral to 1st face to face appt (Collaborative, 9/11 teams) - X-bar Chart



A C C E S S T O S E R V I C E S C O L L A B O R A T I V E

60.7

52.2LCL

40

45

50

55

60

65

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J

u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M a y - 1 5

J u n - 1 5

J

u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N o v - 1 5

A v e r a g e W a i t i n g

T i m e / D a y s

1021.8

1211.0

UCL

LCL

800

900

1000

1100

1200

1300

1400

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M

a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N

o v - 1 4

D

e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M

a y - 1 5

J u n - 1 5

J u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N

o v - 1 5

N o . o f R

e f e r r a l s

No. of referrals received (Collaborative, 9/11 teams) - C Chart

32.50%

25.52%

UCL

LCL

18%

23%

28%

33%

38%

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J

u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M a y - 1 5

J u n - 1 5

J

u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N o v - 1 5

D N A /

%

% of 1st face to face appt DNAs (Collaborative, 9/11 teams) - P Chart

Where would the average be for

all this data?

Psychological Therapy Service (City and Hackney, Newham & Tower Hamlets) December 2015

125

Average waiting time from referral to 1st face to face appt (PTS) - X-bar Chart

4- Baseline data



104.0

88.9

UCL

LCL

65

75

85

95

105

115

J a n

- 1 4

F e b

- 1 4

M a r - 1 4

A p r - 1 4

M a y

- 1 4

J u n

- 1 4

J u l - 1 4

A u g

- 1 4

S e p

- 1 4

O c t - 1 4

N o v

- 1 4

D e c

- 1 4

J a n

- 1 5

F e b

- 1 5

M a r - 1 5

A p r - 1 5

M a y

- 1 5

J u n

- 1 5

J u l - 1 5

A u g

- 1 5

S e p

- 1 5

O c t - 1 5

N o v

- 1 5

A v e r a g e W a i t i n g T i m e / D a y s

211.7

UCL

LCL

100

150

200

250

300

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J

u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M a y - 1 5

J u n - 1 5

J

u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N o v - 1 5

N o . o f R e

f e r r a l s

No. of referrals received (PTS) - I Chart

29.75%

UCL

LCL

10%

15%

20%

25%

30%

35%

40%

45%

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M a y - 1 5

J u n - 1 5

J u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N o v - 1 5

D N A / %

% of 1st face to face appt DNAs (PTS) - P Chart

S E R V I C E

L E V E L

QI0043 & QI0175 – Newham Psychological Therapy Service December 2015

140

Average waiting time from referral to 1st face to face appt (NH PTS) - X-bar Chart

5- Baseline data



85.4

56.6

UCL

LCL

20

40

60

80

100

120

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J

u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M a y - 1 5

J u n - 1 5

J

u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N o v - 1 5

A v e r a g e W a i t i n g T

i m e / D a y s

58.4

UCL

LCL

0

20

40

60

80

100

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M

a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N

o v - 1 4

D

e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M

a y - 1 5

J u n - 1 5

J u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N

o v - 1 5

N o . o f R e f e r r a l s

No. of referrals received (NH PTS) - I Chart

32.73%

22.91%

UCL

LCL

0%

10%

20%

30%

40%

50%

60%

J

a n - 1 4

F

e b - 1 4

M

a r - 1 4

A

p r - 1 4

M

a y - 1 4

J

u n - 1 4

J u l - 1 4

A

u g - 1 4

S

e p - 1 4

O

c t - 1 4

N

o v - 1 4

D

e c - 1 4

J

a n - 1 5

F

e b - 1 5

M

a r - 1 5

A

p r - 1 5

M

a y - 1 5

J

u n - 1 5

J u l - 1 5

A

u g - 1 5

S

e p - 1 5

O

c t - 1 5

N

o v - 1 5

D N

A / %

% of 1st face to face appt DNAs (NH PTS) - P Chart

T E A M L E V E L

QI0104 – Newham Memory Service December 2015

45

50Average waiting time from referral to 1st face to face appt (NH Memory Service) - X-bar Chart

8- Baseline data



28.5

UCL

LCL

5

10

15

20

25

30

35

40

45

J a n - 1 4

F e b - 1 4

M

a r - 1 4

A

p r - 1 4

M a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O

c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M

a r - 1 5

A

p r - 1 5

M a y - 1 5

J u n - 1 5

J u l - 1 5

A u g - 1 5

S e p - 1 5

O

c t - 1 5

N o v - 1 5

A v e r a g e W a i t i n g T

i m e / D a y s

124.6

UCL

LCL

30

50

70

90

110

130

150

170

190

210

J a n - 1 4

F e b - 1 4

M a r - 1 4

A p r - 1 4

M

a y - 1 4

J u n - 1 4

J u l - 1 4

A u g - 1 4

S e p - 1 4

O c t - 1 4

N o v - 1 4

D e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

A p r - 1 5

M

a y - 1 5

J u n - 1 5

J u l - 1 5

A u g - 1 5

S e p - 1 5

O c t - 1 5

N o v - 1 5

N o . o f R e f e

r r a l s

No. of referrals received (NH Memory Service) - I Chart

17.20%

UCL

LCL

0%

5%

10%

15%

20%

25%

30%

35%

J a n - 1 4

F e b - 1 4

M a r - 1 4

A p r - 1 4

M

a y - 1 4

J u n - 1 4

J u l - 1 4

A

u g - 1 4

S

e p - 1 4

O c t - 1 4

N

o v - 1 4

D

e c - 1 4

J a n - 1 5

F e b - 1 5

M a r - 1 5

A p r - 1 5

M

a y - 1 5

J u n - 1 5

J u l - 1 5

A

u g - 1 5

S

e p - 1 5

O c t - 1 5

N

o v - 1 5

D N A / %

% of 1st face to face appt DNAs (NH Memory Service) - P Chart

T E A M L E V E L



All 4 acute admissions wards

in Tower Hamlets started

working on violence

reduction at the end of 2014…



Learnin

g Set 1Learnin

LearningSet 4 o r k b e g i n s

PDSA DATA(AFTER)

Learning

Set 6 &

35.0

UCL

LCL0

20

40

60

80

100

0 6 - J a n -

1 4

2 0 - J a n -

1 4

0 3 - F e b -

1 4

1 7 - F e b -

1 4

0 3 - M a r -

1 4

1 7 - M a r -

1 4

3 1 - M a r -

1 4

1 4 - A p r -

1 4

2 8 - A p r -

1 4

1 2 - M a y - …

2 6 - M a y - …

0 9 - J u n -

1 4

2 3 - J u n -

1 4

0 7 - J u l -

1 4

2 1 - J u l -

1 4

0 4 - A u g -

1 4

1 8 - A u g -

1 4

0 1 - S e p -

1 4

1 5 - S e p -

1 4

2 9 - S e p -

1 4

1 3 - O c t -

1 4

2 7 - O c t -

1 4

1 0 - N o v -

1 4

2 4 - N o v -

1 4

0 8 - D e c -

1 4

2 2 - D e c -

1 4

0 5 - J a n -

1 5

1 9 - J a n -

1 5

0 2 - F e b -

1 5

1 6 - F e b -

1 5

0 2 - M a r -

1 5

1 6 - M a r -

1 5

3 0 - M a r -

1 5

1 3 - A p r -

1 5

2 7 - A p r -

1 5

1 1 - M a y - …

2 5 - M a y - …

0 8 - J u n -

1 5

2 2 - J u n -

1 5

0 6 - J u l -

1 5

2 0 - J u l -

1 5

0 3 - A u g -

1 5

1 7 - A u g -

1 5

3 1 - A u g -

1 5

1 4 - S e p -

1 5

2 8 - S e p -

1 5

1 2 - O c t -

1 5

2 6 - O c t -

1 5

0 9 - N o v -

1 5

2 3 - N o v -

1 5

0 7 - D e c -

1 5

2 1 - D e c -

1 5

Incidents resulting in physical violence (PICU's only)per 1000 occupied bed days (OBD) - U Chart

n s w i t h w a r d s

n c e

w o r k

5.8

2.5

UCL

LCL0

2

4

6

8

10

12

14

16

N o . o f I n c i d e n t s p e r 1 0 0

0 O B D

… our baseline data told

us our wards wereexperiencing 5.8 violent

incidents every two

weeks per 1000Occupied Bed Days



We started testing change ideasto improve how we

communicate and worktogether…



…and to better identify and

anticipate when our service users

might feel their needsweren’t being met.

UCL

14

16

0 0 O B D



BASELINE DATA(BEFORE)

LearningSet 3

Introduce safety

culture bundle

Learning Set 1

Learning Set 2

LearningSet 4

Learning Set 5: Safety

Huddle outcomes

Q I W o r k b e g i n s

PDSA DATA(AFTER)

LearningSet 6 &

GeneralAdult

wards gosmoke

freeLearning

Set 7

35.0

UCL

LCL0

20

40

60

80

100

0 6 - J a n - 1 4

2 0 - J a n - 1 4

0 3 - F e b - 1 4

1 7 - F e b - 1 4

0 3 - M a r - 1 4

1 7 - M a r - 1 4

3 1 - M a r - 1 4

1 4 - A p r - 1 4

2 8 - A p r - 1 4

1 2 - M a y - …

2 6 - M a y - …

0 9 - J u n - 1 4

2 3 - J u n - 1 4

0 7 - J u l - 1 4

2 1 - J u l - 1 4

0 4 - A u g - 1 4

1 8 - A u g - 1 4

0 1 - S e p - 1 4

1 5 - S e p - 1 4

2 9 - S e p - 1 4

1 3 - O c t - 1 4

2 7 - O c t - 1 4

1 0 - N o v - 1 4

2 4 - N o v - 1 4

0 8 - D e c - 1 4

2 2 - D e c - 1 4

0 5 - J a n - 1 5

1 9 - J a n - 1 5

0 2 - F e b - 1 5

1 6 - F e b - 1 5

0 2 - M a r - 1 5

1 6 - M a r - 1 5

3 0 - M a r - 1 5

1 3 - A p r - 1 5

2 7 - A p r - 1 5

1 1 - M a y - …

2 5 - M a y - …

0 8 - J u n - 1 5

2 2 - J u n - 1 5

0 6 - J u l - 1 5

2 0 - J u l - 1 5

0 3 - A u g - 1 5

1 7 - A u g - 1 5

3 1 - A u g - 1 5

1 4 - S e p - 1 5

2 8 - S e p - 1 5

1 2 - O c t - 1 5

2 6 - O c t - 1 5

0 9 - N o v - 1 5

2 3 - N o v - 1 5

0 7 - D e c - 1 5

2 1 - D e c - 1 5


C

o n v e r s a t i o n s w i t h w a r d s

r e v i o l e n c e w o r k

5.8

2.5LCL0

2

4

6

8

10

12


UCL

14

16

0 0 O B D



BASELINE DATA(BEFORE)

LearningSet 3

Introduce safetyculture bundle

Learning Set 1

Learning Set 2

LearningSet 4

Learning Set 5: Safety

Huddle outcomes

Q I W o r k b e g i n s

PDSA DATA(AFTER)

LearningSet 6 &

GeneralAdult

wards gosmoke

freeLearning

Set 7

35.0

UCL

LCL0

50

100

0 6 - J a n - 1 4

2 0 - J a n - 1 4

0 3 - F e b - 1 4

1 7 - F e b - 1 4

0 3 - M a r - 1 4

1 7 - M a r - 1 4

3 1 - M a r - 1 4

1 4 - A p r - 1 4

2 8 - A p r - 1 4

1 2 - M a y - …

2 6 - M a y - …

0 9 - J u n - 1 4

2 3 - J u n - 1 4

0 7 - J u l - 1 4

2 1 - J u l - 1 4

0 4 - A u g - 1 4

1 8 - A u g - 1 4

0 1 - S e p - 1 4

1 5 - S e p - 1 4

2 9 - S e p - 1 4

1 3 - O c t - 1 4

2 7 - O c t - 1 4

1 0 - N o v - 1 4

2 4 - N o v - 1 4

0 8 - D e c - 1 4

2 2 - D e c - 1 4

0 5 - J a n - 1 5

1 9 - J a n - 1 5

0 2 - F e b - 1 5

1 6 - F e b - 1 5

0 2 - M a r - 1 5

1 6 - M a r - 1 5

3 0 - M a r - 1 5

1 3 - A p r - 1 5

2 7 - A p r - 1 5

1 1 - M a y - …

2 5 - M a y - …

0 8 - J u n - 1 5

2 2 - J u n - 1 5

0 6 - J u l - 1 5

2 0 - J u l - 1 5

0 3 - A u g - 1 5

1 7 - A u g - 1 5

3 1 - A u g - 1 5

1 4 - S e p - 1 5

2 8 - S e p - 1 5

1 2 - O c t - 1 5

2 6 - O c t - 1 5

0 9 - N o v - 1 5

2 3 - N o v - 1 5

0 7 - D e c - 1 5

2 1 - D e c - 1 5


C

o n v e r s a t i o n s w i t h w a r d s

r e v i o l e n c e w o r k

5.8

2.5LCL0

2

4

6

8

10

12


The number of violent incidents has nowmore than halved to 2.5 incidents per 1000

Occupied Bed Days, every two weeks.



What is your current level of knowledge about

quality measurement?



This self-assessment is designed to help quality facilitators and improvement teammembers gain a better understanding of where they personally stand with respect to

the milestones in the Quality Measurement Journey (QMJ). What would your

reaction be if you had to explain why is it preferable to plot data over time rather than

using aggregated statistics and tests of significance? Can you construct a run chart

or help a team decide which measure is more appropriate for their project?

You may not be asked to do all of the things listed below today or even next week.

But, if you are facilitating a QI team or expect to be able to demonstrate improvement,

sooner or later these questions will be posed. How will you deal with them?

The place to start is to be honest with yourself and see how much you know about

concepts and methods related to the QMJ. Once you have had this period of self-reflection, you will be ready to develop a learning plan for yourself and those on your

improvement team.

Source: R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. Jones & Bartlett Publishers, 2004: 301-304.

quality measurement?

Measurement Self-Assessment

Response Options



Response Options

Use the following Response Scale. Select the one response whichbest captures your opinion.

1. I'd definitely have to call in an outside expert to explain and

apply this topic/method.

2. I'm not sure I could apply this appropriately to a project.3. I am familiar with this topic but would have to study it further

before applying it to a project.

4. I have knowledge about this topic, could apply it to a project but

would not want to be asked to teach it to others.

5. I consider myself an expert in this area, could apply it easily to a

project and could teach this topic/method to others.


Measurement Self-Assessment WorksheetSource: R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators.

Jones & Bartlett Publishers, 2004: 301-304.



Measurement Topic or SkillResponse Scale

1 2 3 4 5

1. Help people in my organization understand where and how measurement fits into our quality journey

2. Facilitate the development of clear Aim Statements

3. Move teams from concepts to specific quantifiable measures

4. Building clear and unambiguous operational definitions for our measures

5. Develop data collection plans (including stratification and sampling strategies)

6. Explain why plotting data over time (dynamic display) is preferable to using aggregated data andsummary statistics (static display)

7.Explain the differences between random and non-random variation

8. Construct run charts (including locating the median)

9. Explain the reasoning behind the run chart rules

10. Interpret run charts by applying the run chart rules

11. Explain the various types of control charts and how they differ from run charts

12. Construct the various types of control charts

13. Explain the control chart rules for special causes and interpret control charts

14. Help teams link measurement to their improvement efforts


Knowing how to properly use

Shewhart Control Charts



Fe b r u a r y Ap r il

1

2

3

4

5

6

7

8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

16 Pat ient s in Febr uar y and 16 Pat ient s in Apr il

M

i n u t e s

2 . 5

5 . 0

7 . 5

1 0 . 0

1 2 . 5

1 5 . 0

1 7 . 5

2 0 . 0

2 2 . 5

2 5 . 0

2 7 . 5

3 0 . 0

A

B

C

C

B

A

UCL=1 5 . 3

CL = 1 0 . 7

L C L = 6 . 1

Xm R Char t

Shewhart Control Charts(Wait Time to See the Doctor)

Intervention

Where

will theprocess

go?

Freeze the Control Limits and Centerline, extend them and

compare the new process performance to these reference

lines to determine if a special cause has been introduced as

a result of the intervention.

BaselinePeriod

Using a Shewhart Control Chart(Wait Time to See the Doctor)




1

2

3

4

5

6

7

8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2


M

i n u t e s

2 . 5

5 . 0

7 . 5

1 0 . 0

1 2 . 5

1 5 . 0

1 7 . 5

2 0 . 0

2 2 . 5

2 5 . 0

2 7 . 5

3 0 . 0

A

B

C

C

B

A

UCL=1 5 . 3

CL = 1 0 . 7

L C L = 6 . 1

Xm R Char t

Freeze the Control Limits and compare

the new process performance to the

baseline using the UCL, LCL and CL from

the baseline period as reference lines

A Special Cause is

detected

A run of 8 or more

data points on one

side of the centerlinereflecting a sift in the

process

Intervention

Baseline

Period

Using a Shewhart Control Chart(Wait Time to See the Doctor)




1

2

3

4

5

6

7

8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2


M

i n u t e s

2 . 5

5 . 0

7 . 5

1 0 . 0

1 2 . 5

1 5 . 0

1 7 . 5

2 0 . 0

2 2 . 5

2 5 . 0

2 7 . 5

3 0 . 0

A

B

C

C

B

A

UCL=1 5 . 3

CL = 1 0 . 7

L C L = 6 . 1

Xm R Char t

Intervention Make new control limits for

the process to show the

improvement

Baseline

Period

( )

But the Charts Don’t Tell You…



130

• The reasons(s) for a Special Cause.

• Whether or not a Common Cause process

should be improved (is the performance ofthe process acceptable?)

• How the process should actually beimproved or redesigned.

Improvement Teams need a Framework for

Performance Improvement



p

• Establish appropriate measures.

• Set an aim and goal for each measure.

• Develop theories and predictions on how they plan onachieving the aim and an appropriate time frame for

testing.

• Test theories, implement change concepts, follow the

measures over time and analyse the results with SPC.

• Revise the strategy as needed.

311

A few thoughts onBenchmarking

132




g

“Benchmarking i s the cont inuo us p rocess of measur ing

products, serv ices, and p ract ices against the toug hest

competi tors or tho se com panies recogn ized as indust ry

leaders .”

“Benchmarking is a structu red process i t is f irs t and foremo st asearch fo r knowledge and learning .”

Camp, R. Benchmarking: The Search for Industry Best Practices that

Lead to Superior Performance ASQ Press, 1989.

A benchmark is a noun.Benchmarking, on the other hand, is a verb that requires

exploration and investigation of why the ‘benchmark’ number

was achieved!

Benchmarking versusComparative Reference Data

133




p

Or,regional or

national

norms

1 B h ki b d d t b t if t t th

More thoughts on Benchmarking




1. Benchmarking uses numbers and data but if you stop at the

numbers you will never achieve the potential that

benchmarking offers.

2. Benchmarking is a way to identify and understand best

practices that enable organizations to realize new levels of

performance (i.e., the targets and goals that can become

benchmarks).3. Confusion over these concepts leads an organization to

accept a number, either from an internal or external source

as “THE Benchmark .” This orientation typically leads to a

fairly singular focus on the numbers (outcomes) without

giving due consideration to the interplay of the structures and

processes that produce the numbers.

(continued)



More thoughts on Benchmarking(continued)




4. While you will hear organizations claim that, “We are

benchmarking ” this statement usually means that they

are hoping that they hit “the benchmark metric” but have

not developed a strategy for achieving this ethereal

number. By what method wi l l you get there?

5. The end result is usually confusion amongst the staff and

frequently unrealistic expectations on the part of

management and the board.

Source: R. Lloyd. Quality Health Care: A Guide to Developing and Us ing Indic ators. Jones and Bartlett Publishers, 2004.

(continued)

Five Phases Of Benchmarking and the

Ten Related Specific Steps




A Few thoughts on SettingTargets and Goals

137




g

On Setting Targets and Goals138




“ Goals are necessary fo r you andfor me, bu t numerical goals set

for o ther peop le, w i thou t a road

map to reach the goal, haveeffects oppos i te to the effects

sough t . By what method do you

plan to achieve the goal?” Deming, E. Out of the Crisis. Massachusetts Institute of Technology,

Cambridge, MA, 1992

On Setting Targets and Goals139




• Targets are short-term markers of achievement that areachieved over the span of several months or several

years.

• Goals, on the other hand, are more long term in nature,

usually in the range of 3-5 years.

• A target or a goal can be based on a benchmark (the

noun) if it is derived from an organisation that is

considered the “best of the best .”

• The benchmarking process (as a verb) is one of the best

ways, therefore, to develop a plan for achieving new

performance levels.

DialogueSetting Targets and Goals

140




Do you distinguish between targets and

goals? Or do you consider them as being

synonymous?

How do you set targets and goals?

Would you say some targets and goals thatare set are arbitrary?

Suggestions onSetting Targets and Goals

141




Establish baseline data on the relevant measure to

determine what the current performance of the processactually is.

Develop a control chart to determine what the statistical

capability of the process is.

Use the control chart as a basis to discuss the chances

(probability) that the current process will be able to achieve

the proposed target or goal.

If the current baseline performance is so far from the target

or goal then a discussion must occur related to Deming’s

basic question: “By what method ?””

The Primary Drivers of

Organisational Improvement



Organisational Improvement

Will

Ideas Execution

QI

Having the Will (desire) to change the current stateto one that is better

Developing Ideas

that will contribute

to makingprocesses and

outcome better

Having the capacity

and capability to

apply QI theories,

tools andtechniques that

enable successful

Execut ion of your

ideas

How prepared is the system?



Key Components* Self-Assessment

• Will (to change)

• Ideas

• Execution

• Low Medium High

• Low Medium High

• Low Medium High

*All three components MUST be viewed together. Focusing onone or even two of the components will guarantee sub-optimised performance. Systems thinking lies at the heart of QI!

o p epa ed s t e syste

A closing thought…




It must be remembered that there is nothingmore difficult to plan, more doubtful ofsuccess, nor more dangerous to manage than

the creation of a new system.For the initiator has the enmity of all whowould profit by the preservation of the old

institution and merely lukewarm defendersin those who would gain by the new one.Machiavelli The Prince 1513

Appendix A



• Evolution of Quality Management over time – Age of Craftsman

– Age of Mass Production

– Age of Quality Management



• Fourth Generation Management (Dr. Brian Joiner)

• Evolution of Quality Management in Healthcare

• What is Lean?

Evolution of Quality Management

MarketResearch

NeedPurposeoftheOrganization

Measurement& F db k

Design &Redesign of Processes &

Plan toImprove



Evolution of Quality

Management Age of the Craftsman

B.C. - 1800's

Age of Mass Production

Early 1800's - Present

Age of

Quality Management

1950's - Present

Theory of

Management

* Person doing the work

manages the entire job,

from planning to job

completion.

* Craftsman is responsible

for communication with

suppliers and customers.

* Rewards are tied to the

customer.

* Scientific study is used for

simplification of methods for

individual tasks.

* Planning is separated fromexecution.

* Focus of management is on

production at low cost.


individual.

* Management views all work

as processes that link to

form a system.

* The focus of managementis on improving the system.

* Improvement requires

partnership between

suppliers and customers.


customer and teamwork.

Impact on Quality

* Quality = High cost.

* Responsibility for quality

belongs to the craftsman.

* Direct customer

feedback provides the

definition of quality.

* Quality = High cost and low

productivity.

* Simplification objective

establishes the Q.C.

Department to measure and

report.

* Focus is on reducing costs.

* Quality is achieved by

inspection and sorting.

* Quality = Low cost and

high productivity.

* Quality is the focus of the

organization.

* Quality is defined by the

need of the customer.

* Q.C. Department assumes

the role of consultant for

improvement activities.

Production of Product or Service

B

C Distribution

Customers

Suppliers

A

D

E

F

G

& Feedback Products

Support Processes

Source: Ron Moen, Associates in

Process in Improvement

Evolution of Quality Management (1950-1974)

1951 – Total Quality Control

published by Armand Feigenbaum



1956 – Western Electric “Statistical

Quality Control Handbook “

1958 – Genichi Taguchi begins

teaching his methods of loss

function and robust design.

1962 – Quality Circles start. Kaoru

Ishikawa asked a number of

Japanese companies to participatein an experiment.

1974 – Kaoru Ishikawa publishes

Guide to Quality Control, 7 simple

tools for improvement.

Taguchi

Source: Ron Moen, Associates in

Process in ImprovementSource: Ron Moen, Associates in

Process in Improvement

1978 – George Box, William G. Hunter and J.

Stuart Hunter publish their landmark book

Evolution of Quality Management (1978-2014)



p

Statistics for Experimenters

1979- Philip Crosby publishes Quality is Free

1980 - Quality revolution begins in US

– NBC airs If Japan Can, Why Can’t We?”

– Deming consults for Ford and GM

1987 - Malcolm Baldrige National Quality

Award is established.

1994 – Deming publishes the New Economics which emphasizes the use of the System of

Profound Knowledge.

Present - Quality programs spread to Service

Industries under a variety of names, tools and

approaches.

– Proliferation of quality programs: TQM, Six

Sigma, Kaizen, SQC, SPC, Taguchi

Methods, Benchmarking, CQI, Lean Six

Sigma, etc.

– Attempts are being made to package the

various contributions from the past into an

overall “one best approach.”

Source: Ron Moen, Associates in Process in Improvement

Dr. Brian Joiner149



First Generation: do it yourself.

Second Generation: Master craftsperson takes on apprentices but

remains the model and arbiter of production (and quality).

Third Generation: manage by results—usually by specifying the

goals required without detailing the methods (by what method?).

Fourth Generation: simultaneous focus on three chunks of

work: quality, the scientific approach and all one team, the JoinerTriangle (see next page for details).

Dr. Brian Joiner, a student of Dr.

Deming’s, described four generationsof management:

The Joiner Triangle150



Quality

Scientific

ApproachAll One

Team

the Joiner Triangle provides a framework for

implementing Quality Improvement. It consists of:

• Quality as seen through the eyes of our customers

• The Scientific Approach as the methodology for

solving problems and making decisions; iterative

learning, using data effectively, to build and maintaineffective methods.

• The All One Team aimed at unifying staff work

efforts, getting all employees involved with quality

efforts, collaboration and respect for people.

Evolution of Quality Management in Healthcare

B C – Hippocrates (3rd century B C ) Medicine was and is taught and learned as a craft



B.C. Hippocrates (3rd century B.C.). Medicine was and is taught and learned as a craft.

1973 – Avedis Donabedian proposed measuring the quality of healthcare by observing :

structure, processes, and outcomes.1970s – Quality Assurance (QA) of hospital care using structural standards

1980s – QA by government and insurers. The regulatory route relied on punishment and blame.

1986 – Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) announced

its Agenda for Change and stated that the “philosophical context” for the Agenda of

change is set by the theories of Continual Quality Improvement (QI).

1986 – National Demonstration Project (NDP) on Quality Improvement in Healthcare. Ademonstration project to explore the application of modern quality improvement methods

to healthcare.

1990 – NDP report: Berwick, D, Godfrey, J and Roessner, J. Curing Health Care. Jossey-Bass,

1990.

1991 – Don Berwick founded the Institute for Healthcare Improvement (IHI) committed to

redesigning health care delivery systems in order to ensure the best health careoutcomes at the lowest costs.

1993 – IHI adopts API Model for Improvement as its foundation for Improvement.

Source: Ron Moen, Associates in Process in Improvement

Beginning of Modern Health Care QI152



“The National Demonstration Project on QualityImprovement in Health Care” (“NDP”)

20 Hospitals and 21 Quality Improvement Experts

8 Months – September 1986 to June 1987

Initial and Summary Conference

“Curing Health Care”

Dr. Don Berwick formed IHI at end of project

153Lessons from Curing Health Care(Berwick et al, 1990)




Lesson 1: Quality Improvement Tools Can Work in Health Care

Lesson 2: Cross-Functional Teams Are Valuable in Improving Health CareProcesses

Lesson 3: Data Useful for Quality Improvement Abound in Health Care

Lesson 4: Quality Improvement Methods are Fun to Use

Lesson 5: Costs of Poor Quality Are High and Savings are Within Reach

Lesson 6: Involving Doctors is Difficult

Lesson 7: Training Needs Arise Early

Lesson 8: Non-clinical Processes Draw Early Attention

Lesson 9: Health Care Organizations May Need a Broader Definition of

Quality

Lesson 10: In Health Care, as in Industry, the Fate of Quality Improvement

Is First of All in the Hands of Leaders

What is “Lean” 154



“Reducing the timeline fromcustomer order to building anddelivering a product byeliminating waste”

- Jeff Liker, The Toyota Way

Lean: A systematic approach to identifying and eliminating waste(non-value-added activities) through continuous improvement byflowing the product at the pull of the customer in pursuit of perfection(Improvement Gui de, p. 463)

Lean incorporates aspects of Quality Planning, Quality Control, andQuality Improvement

“All we’re trying to do isshorten the time line…” - Taiichi Ohno (credited with developing

lean at Toyota)

Why “Lean”? 155



Term given its current meaning at MIT in 1987.

Born of a need to describe a product development,production, supplier management, customersupport, and planning system (exemplified by Toyotapractice) for what it did.

Compared to traditional mass production methods(e.g., GM), this system required less time, humaneffort, capital, and space to produce products withfewer defects in wider variety more quickly.

Because it needed less of every input to createvalue, we called it “lean”.

From: James P. Womack, President, Lean Enterprise Institute

The Lean Ideal (Aim)156



The output is defect free.

The product or service is delivered in response tocustomer need (pull, on demand).

The response is immediate.

Products or services are provided 1x1 in the unitsize of use (tailored to the identified needs of thecustomer).

Work is done without waste.

Work is done safely.

Work is done securely.

Spear, S. and H. K. Bowen (1999). "Decoding the DNA of the Toyota Production

System." Harvard Business Review 77(5): 96-106.

The Core Ideas of Lean Thinking157



All value is the result of a process (which we often

call a “value stream”.) Move a manager’s focal plane to the organization’svalue creating processes, rather than theorganization itself and the utilization of its assets.

For each value stream (process): – Accurately specify the value desired by the customer.

– Identify every step in the value stream and remove thewaste.

– Make value flow from beginning to end.

– Based on the pull of the customer. – In pursuit of perfection.

From: James P. Womack, President, Lean Enterprise Institute

Common Tools and Methods in Lean158



5 S – Workplace Organization

Visual Management

Continuous Flow / Cell / JIT

Production Layout

Small Lot Production

Quick Setup / Changeover TPM (Total Productive Maintenance)

Standardized Work

Level Scheduling

Pull System – KANBAN Supplier Rationalization

Appendix BGeneral References on Quality



159

• The Improvement Guide: A Practical Approach to EnhancingOrganizational Performance. G. Langley, K. Nolan, T. Nolan, C.Norman, L. Provost. Jossey-Bass Publishers., San Francisco, 1996.

• Quality Improvement Through Planned Experimentation. 2nd edition. R.Moen, T. Nolan, L. Provost, McGraw-Hill, NY, 1998.

• The Improvement Handbook . Associates in Process Improvement. Austin, TX, January, 2005.

• A Primer on Leading the Improvement of Systems,” Don M. Berwick,BMJ, 312: pp 619-622, 1996.

• “Accelerating the Pace of Improvement - An Interview with ThomasNolan,” Journal of Quality Improvement, Volume 23, No. 4, The JointCommission, April, 1997.

Appendix CReferences on Measurement



160

• Brook, R. et. al. “Health System Reform and Quality.” Journal of the American Medical Association 276, no. 6 (1996): 476-480.

• Carey, R. and Lloyd, R. Measuring Quality Improvement in healthcare: AGuide to Statistical Process Control Applications. ASQ Press, Milwaukee,WI, 2001.

• Lloyd, R. Quality Health Care: A Guide to Developing and Using Indicators.

Jones and Bartlett Publishers, Sudbury, MA, 2004.

• Provost, L. and Murray, S. The Healthcare Data Guide. Jossey-Bass,2011.

• Nelson, E. et al , “Report Cards or Instrument Panels: Who Needs What?

Journal of Quality Improvement, V olume 21, Number 4, April, 1995.

• Solberg. L. et. al. “The Three Faces of Performance Improvement:Improvement, Accountability and Research.” Journal of QualityImprovement 23, no.3 (1997): 135-147.

Appendix D

Robert Lloyd, PhD Bio



Robert Lloyd, PhD is Vice President for the Institute for Healthcare Improvement (IHI). Dr.

Lloyd provides leadership in the areas of performance improvement strategies, statisticalprocess control methods, development of strategic dashboards and building capacity and

capability for quality improvement. He also serves as lead faculty for various IHI initiatives

and demonstration projects in the north America, the UK, Sweden, Qatar, Denmark, New

Zealand, Australia and Africa.

Before joining the IHI, Dr. Lloyd served as the Corporate Director of Quality Resource

Services for Advocate Health Care (Oak Brook, IL). He also served as Senior Director of

Quality Measurement for Lutheran General Health System (Park Ridge, IL), directed theAmerican Hospital Association's Quality Measurement and Management Project (QMMP)

and served in various leadership roles at the Hospital Association of Pennsylvania. The

Pennsylvania State University awarded all three of Dr. Lloyd’s degrees. His doctorate is in

agricultural economics and rural sociology.

Dr. Lloyd has written many articles and chapters in books. He is also the co-author of the internationally

acclaimed book, Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control

Applications (American Society for Quality Press, 2001, 5th printing) and the author of Quality Health Care: AGuide to Developing and Using Indicators, 2004 by Jones and Bartlett (Sudbury, MA).

Documents

Workshop for Commissioners - 21 January 2016 - Using Data to Support System Improvement