TRUE BLUE Quest For Quality. Data Sanity Matthew S. Wayne MD, CMD Chief Medical Officer

TRUE BLUEQuest For Quality

PATIENT FOCUSED

CARE

CommuniCare

Assessments

Clinical Workflow

QAPI

MDS

Quality Measures

Clinical

Standards

Data Sanity

Matthew S. Wayne MD, CMDChief Medical Officer

Objectives

• Perform an in-depth evaluation of current data analysis processes and how they can be improved to improve the quality of care in your nursing home

• Review the 3 steps in proper data analysis• Utilize control charts to analyze data in your

nursing home• Distinguish between common cause and

special cause variation and discuss specific strategies to address both types of variation

5

ACA Provision

Section 6102(c) of the Affordable Care Act (ACA) directs the Secretary to provide technical assistance and promulgate regulations for each nursing home to implement a QAPI system, and permits the Secretary to sequence these actions so the technical assistance is available prior to the regulations.

QAPIQuality Assurance -

Performance Improvement

QA+PI=QAPIQuality Assurance

• Compliance with standards

• Inspection• Reactive• Remove outliers• Narrow• Involves only a few

Performance Improvement

• Continuously improving processes

• Prevention• Proactive• Processes/Systems• Systemic• Involves entire IDT

6

U.S. Department of Health and Human Services, Health Resources and Services Administration. Quality Improvement adapted from http://www.hrsa.gov/healthit/toolbox/HealthITAdoptiontoolbox/QualityImprovement/whatarediffbtwqinqa.html

7

QA vs QI

Balestracci p285

5 Elements of QAPI

• Design & Scope• Governance & Leadership• Feedback, Data Systems and

Monitoring• Performance Improvement Projects

(PIPs)• Systematic Analysis & Systemic

Action

5 Elements of QAPI• Design and Scope

oComprehensive and ongoing plano Includes all departments and functionsoAddresses safety, quality of care, QOL,

resident choice, transitionsoBased on best available evidenceoQAPI plan

5 Elements of QAPI• Governance and Leadership

oBoards/owners and executive leadership• Buy in and support

o Training and organizational climate• Administration sees value

o Sufficient resourceso Sustainability

5 Elements of QAPI• Feedback, Data monitoring Systems, and

MonitoringoMultiple sources, including resident and

staffoBenchmarking and targetingoAdverse events

5 Elements of QAPI• Performance Improvement Projects

o Prioritized topics• Number of PIPs depend on the facility

programo Team Charteredo PDSA Cycle

5 Elements of QAPI• Systematic Analysis and Systemic action

oRoot cause analysiso Systems thinkingo Systematic changes as needed

National Rollout: Timeline

• By statute, nursing homes will be expected to have QAPI programs in place that meet a defined standard, one year after CMS issues a QAPI rule. CMS expects to issue a draft regulation for comment in 2012. A final rule is likely to be issued by early 2013.

15

Quality Improvement: Case 1

Year 2000 Falls 93

Goal – reduce 10% next year

16

Quality Improvement: Case1

Year 2000 2001Falls 93 80 14.0%Everybody gets pizza!!!!!!!!!!!!

17

Quality Improvement: Case 1

Falls

70

75

80

85

90

95

2000 2001

18

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 1

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

19

Quality Improvement: Case 2

Case 2 "M edication Errors" - Graph 2

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

20

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 3

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

21

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 4

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

22

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 5

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

23

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 6

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

24

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 7

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

25

Quality Improvement: Case 2

Case 2 "Medication Errors" - Graph 8

15

17

19

21

23

25

27

29

31

33

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

26

Quality Improvement: Case 2

What if we were to tell you that this was not medication error data but ………………….

Coin Flip Data

27

Basic Statistical Lesson 2

Key Concept -Variation

• Case 2: Coin Flip : 50 people- 25 times- # Heads

14

16

18

20

22

24

26

28

30

32

34

1 3 5 7 9 11

13

15

17

19

21

23

25

28

Key Concept -Variation

• We learn nothing of importance by comparing two or three results when they all come from a stable processMost data of importance to management are from stable processes

29

Quality Improvement

•Process

• Variation

• Priority

30

Process Oriented Thinking

Systems Thinking• System - Definition

oA group of interdependent processesoA network of functions or activities

within an organization that work together for the aim of the organization

The Big Picture• Group of related interdependent

processes working together to achieve a common goal

• Made up of a culture, structure and boundary

System

• Sequence of tasks aimed at accomplishing a goal

• Produce data which can be analyzedProcess

• Have beliefs, values, interests, needs

• Have roles which are made up of functions and tasks

People

32

Process Oriented Thinking

• Process- Definition

oSequences of tasks aimed at accomplishing a particular outcome

oTransformation of inputs into outputs

33

Basic Statistical Lesson 1

• Given two different numbers,

one will be larger

Or- Two numbers that are not the same :are different

34

Quality Improvement

Falls

70

75

80

85

90

95

2000 2001

35

Basic Statistical Lesson 1

• Is the process that produced the second number the same that produced the first number?

Real Question 1

36

Basic Statistical Lesson 1

• If this number is different from a desired goal, is this variation from the goal due to common cause or special cause process?

• What is common cause? Special cause?

Real Question 2

37

Process

• First: Your current processes are perfectly designed to get the results they are already getting and designed to get,

• with it's corollary:o insanity is doing things the way you have always done them while

expecting different results

38

Process

• Second, the current process are also perfectly designed to take up more than 100% of people's time working in them,

• with it's corollary, o it is amazing how much waste can be

disguised as useful work.

39

Process• Third :

improving quality = improving process

• Problems :Breakdown in current work processes, or,Lack of consistent work process

40

Process

• All work is a process

• All processes exhibit variation and have measurable values associated with them

• The performance of any component process is to be evaluated in terms of its contribution to the aim of the system.

41

85/15 Process Rule • Individuals have direct control over only 15% of

their work problems.

• The other 85% are controlled by the process in their work environment.

• Deming 4% - 96%

42

Quality Improvement• Change in focus from the 15% to

the 85%:oThe processoNot people

43

Worker controllable problems

• People need to have the means:

oFor knowing what they were supposed to do

oFor knowing what they were actually doing

oTo close the loop between what they were doing and what they should be doing

44

It’s processes not people

• While we must still hold individuals responsible for high standards of performance, we now recognize that most errors result from faulty systems,

• not faulty people.

45

Process Oriented Thinking

• Concentrating on the process inherent in any improvement situation leads to:oGreater cooperation due to a

common languageoElimination of blameoSimpler, more effective solutions

46

Quality Improvement

• Process

•Variation

• Priority

47

Basic Statistical Lesson 2

Key Concept -Variation

48

Basic Statistical Lesson 2

Key Concept -Variation

• Walter Shewhart- 1920’so There is always variation in anything

that is being measured

o In statistical thinking terms: there are inputs causing variation that are always present and conspire in random ways to affect a process’s output.

49

Basic Statistical Lesson 2

Key Concept -Variation

• Case 2: Coin Flip : 50 people- 25 times- # Heads

14

16

18

20

22

24

26

28

30

32

34

1 3 5 7 9 11

13

15

17

19

21

23

25

50

Basic Statistical Lesson 2

Key Concept -Variation

• Questions to ask:

oFirst: Is the process stable? In other words, is the process in statistical control? oSecond: What are the causes of variation in the process?

51

Basic Statistical Lesson 2

Key Concept -Variation

• Two types of variation:

oControlled (stable) variation

oUncontrolled (unstable) variation

52

Basic Statistical Lesson 2

Key Concept -Variation

• Controlled (stable) variation

o Predictable within well-defined limits, but impossible to predict where any specific result will lie within those limits

o Controlled variation is due to the way that the processes and systems have been designed and built.

o Common Cause

53

Basic Statistical Lesson 2

Key Concept -Variation

• Uncontrolled (unstable) variation

oProcess affected by special causes

oBehavior changes unpredictably

oNo one can predict process capability

54

Basic Statistical Lesson 2

Key Concept -Variation

• Walter Shewhart- 1920’so Two kinds of mistakes

• Mistake 1. Treating a fault, complaint, mistake, accident as if it came from a special cause when in fact there was nothing special at all, ie it came from the system: from random variation due to common causes – Tampering

• Sounding a false alarm

55

Basic Statistical Lesson 2

Key Concept -Variation

• Walter Shewhart- 1920’soTwo kinds of mistakes

• Mistake 2. Treating a fault, complaint, mistake, accident as if it came from a common cause, when in fact it was due to a special cause

• Missing a signal in the data

56

Basic Statistical Lesson 2

Key Concept -Variation

• The 2-point Curveo Common practice

• Last month to this month• Last year to this year• Last quarter to this quarter

57

Basic Statistical Lesson 2

Key Concept -Variation

• The 2-point Curve

0

10

20

30

40

50

60

70

80

90

100

Jan-10

Feb-10

Mar-10

Apr-10

May-10

Jun-10

Jul-10 Aug-10

Sep-10

Oct-10

Nov-10

Dec-10

Jan-11

58

Basic Statistical Lesson 2

Key Concept -Variation

• The 2-point Curve Variation w/ last years data

0

10

20

30

40

50

60

70

80

90

100

Jan-10

Feb-10

Mar-10

Apr-10

May-10

Jun-10

Jul-10

Aug-10

Sep-10

Oct-10

Nov-10

Dec-10

Jan-11

59

Basic Statistical Lesson 2

Key Concept -Variation

• The 3-point Curve - TrendsoAlso Common practice

3 Point Curves

Upward Trend

Rebound

DownwardTrend

Turnaround

Setback

Downturn

61

Basic Statistical Lesson 2

Key Concept -Variation

• The 3-point Curve – “Trends”

oFalse explanations given to each “trend” resulting in false solutions that increase variation and increase costs.

62

Variation

•Human tendency is to treat ALL variation as special cause

63

Data Analysis- Run Charts

• Graphical representation of data over time

• Ignoring the time element implicit in every data set can lead to incorrect statistical conclusions.

64

Data Analysis- Run Charts

• What information can you get from the run chart?

Stability

Common cause vs. special cause

65

Quality Improvement: Case 1Falls with Median

0

2

4

6

8

10

12

14

66

Data Analysis- Control Charts

• Control chartoTime plot of the data that includes lines added for the average and natural process variation.

67

Data Analysis- Control Charts

• Control chart - limitso These limits represent a common

cause range around the average where individual data points may be expected to fall if the underlying process does not change.

Long Stay Residents Receiving an

Antipsychotic- One Facility

69

Data Analysis- Run Charts- Rules

• Rule #1 – Trendo A sequence of SEVEN or more points

continuously increasing or decreasing (six successive increases or decreases) (21-199 points)

o Less than 21 points- SIX points neededo Greater than 200 -EIGHT points needed

o Omit entirely any points that repeat the preceding value. Such points neither add to the length of the run nor break it.

70

Quality Improvement: Case 4

Trends?

0

2

4

6

8

10

12

1 3 5 7 9

11

13

15

17

19

21

23

25

27

29

NO

71

Quality Improvement: Case 5

Trends?

0

2

4

6

8

10

12

14

1 3 5 7 9 11

13

15

17

19

21

23

25

27

29

NO

72

Data Analysis- Run Charts- Rules

• Rule #2 – “Clump of Eight” – the presence of a shift in the process

o A run of EIGHT consecutive points either all above or all below the median.

o It is broken and begins a new run when a data point literally crosses the median.

o Any data point on the median neither breaks nor adds to the current run

o Then, over the time period covered by the data, the process exhibited at least two different averages.

o The special cause may not have occurred at the beginning of the run

73

Quality Improvement: Case 5

“Clump of Eight?”

0

2

4

6

8

10

12

14

1 3 5 7 9

11

13

15

17

19

21

23

25

27

29

YES

Follow Up

Short Stay Pain- One Facility

76

Data Analysis- Control Charts

• Control chart ruleso1. A special cause is indicated when a single point falls outside a control limit

77

Special Cause• Trend• Clump of 8• Single data point falls outside control limits

78

Data Analysis- Run Charts

SPECIAL CAUSE VARIATION • Indicates different processes at work,

even if unintended or perhaps even desirable and appropriate

o Distinct shift(s) – due to outside interventions that have now become part of the everyday process inputs

o Process has changed

79

Data Analysis- Run Charts

COMMON CAUSE VARIATION • Each source (input) of common cause

contributes a random, small amount of variation• No one can predict in advance which particular

source (input) will affect the process at any given time.

• However, the range of resulting outputs can be predicted

• Data points can not be treated and reacted to individually

80

Quality Improvement: Case 1Falls with Median

0

2

4

6

8

10

12

14

81

Quality Improvement

• Process

• Variation

•Priority

82

The Pareto Principle

• 80% of the observed variation in a process is caused by only 20% of the process inputs.

• 20% of the variation causes 80% of the problemso Juran 1920’s

• The “vital few” vs the “trivial many”

83

The Pareto Principle

• Motivates staff to recognize the importance of identifying and exposing the real, underlying, hidden opportunities

• Special causes are isolated as a result, allowing a more specific action to focus on solving the problem.

• The goal must be to expose, locate and focus, and then further focus on a major opportunity that can have a significant impact.

84

Key Concept -Improvement

• Process Improvement

oPhase 1 – stabilization oPhase 2 – active improvementoPhase 3 - monitoring

85

Key Concept - Improvement

• Process Improvement

oPhase 1 – stabilization oPhase 2 – active improvementoPhase 3 - monitoring

86

Key Concept - Improvement

• Process ImprovementoPhase 1 – stabilization

• Eliminate special causes• Gets the process where it should have been in the first place

• Problem solving, putting out fires• No real improvement at this level• Control, Run charts

87

Key Concept - Improvement

• Process Improvement

oPhase 1 – stabilization oPhase 2 -active improvementoPhase 3 - monitoring

88

Key Concept - Improvement

• Process ImprovementoPhase 2 – active improvement

•Eliminate common causes• Pareto analysis• Fish-bones• Flow charting• Recalculate control limits

89

Key Concept - Improvement

• Process Improvement

oPhase 1 – stabilization oPhase 2 – active improvementoPhase 3 - monitoring

90

Key Concept - Improvement

• Process ImprovementoPhase 3 – monitoring

•Constant vigilance• Implement additional improvements as the need arises (Continuous Improvement)

91

Strategies for Reducing Variation

The differences between common cause and special cause variation require us to use different managerial approaches to deal with each if we are going to be effective

92

Strategies for Reducing Variation

• Most problems arise from common causes.

• However, it is better to work on special causes first.oCloud the pictureoLess false leads

93

Improving an Unstable Process

Special Cause Strategy

1. Get timely data so special causes are signaled quickly• Indicators that give a clear signal when

something affects our results• Act rapidly or the trail will grow cold• Look for ways to monitor process

factors that are highly correlated with process outputs

Short Stay Pain- One Facility

Follow Up

96

Improving a Stable Process

Common Cause Strategy

97

Improving an Stable Process Common Cause Strategy

oStratify

oExperiment

oDisaggregate the process

98

Improving an Stable Process Common Cause Strategy

oStratify• Sort data into groups or categories based on

different factors• Look for patterns in the way that the data

points cluster or do not cluster that may point to the source of the trouble

• Focusing to identify the leverage points where a little effort bring major improvement.

• Must have information on conditions related to the data: type of job, day of week, shift, unit, etc.

Long Stay Residents With a UTI-

43 Facilities

Stratification- Individual Facilities

101

The Pareto Principle

• 80% of the observed variation in a process is caused by only 20% of the process inputs.

• 20% of the variation causes 80% of the problemso Juran 1920’s

• The “vital few” vs the “trivial many”

102

Improving an Stable Process Common Cause Strategy

oExperiment• Make planned changes and learn from

the effects• Trying out ideas• Two keys to effective experimentation

oHaving good ideas to test – need in depth knowledge of how a process does and should work

oHaving good ways to assess and learn- Plan

• PDSA – Plan, Do, Study, Act

103

Improving an Stable Process Common Cause Strategy

oDisaggregate the process• Divide the process into component pieces

and manage the pieces• Every process has multiple steps or

phases that can be monitored and improved individually

• Making the elements of the process visible through measurements and data

• Special causes may be buried in components of a process

104

Process analysis

• The most serious problems in service processes result from variation caused by:

The lack of agreed-upon processes

105

Process analysis• A lack of agreed-upon processes

o Unintended variation in individual work processes

o Management’s perceptions of these processes

o There can be big differences between what is written down- the way the system is intended, or thought to operate, and what actually happens

106

Process analysis• FlowchartsoAn opportunity for those involved

in a process to describe it’s current operation in a concise, visual way

oEstablish agreement on what the current process actually is

o If a process can not be written down, it probably does not exist or it functions more on whim or “gut-feeling”

107

Describing the Process• Include “front-line” personnel

o They can tell you what is stopping them from doing their job.o Also gives you an opportunity to see if they:

• Know what should be done.• Know how to do it.• Understand why it is important.• Think their way is better than the required

way.

108

MDS

completed Does Falls RAP trigger?

Is there any other reason to believe patient is at high falls

Risk?

Routine precautions

Identify modifiable (intrinsic or extrinsic) risk

factors

YesNo

No

Yes

Establish care plan

Write care plan in chart and on aide assignment sheets

Activity Documentation Yes /No Decision point

Admission

Nursing

Assessment

109

Process analysis

• Fishbone Diagrams

• Show the causes of a certain event. A Fishbone or Ishikawa Diagram can be useful to break down (in successive layers of detail) root causes that potentially contribute to a particular effect.

110

Fishbone Diagram

111

Weight Loss

Fishbone Diagram

112

Weight Loss

Type of Patient

Dietary Staffing

CNA assistance with meals

Food Not Appetizing

Fishbone Diagram

113

Weight Loss

Type of Patient

Hospice

Obese patienton dietOrtho

Rehab

Dietary Staffing

Holiday call-offs

Wages not competitive

NewDietician

CNA assistance with meals

Short staffed

Wages not competitive

Holiday call-offs

Inadequatetraining

Lack ofinterest

High toileting needs

Don’t understandimportance

Food Not Appetizing

Monotonous Menu

Wrong Temperature

Poor presentation

Fishbone Diagram

114

Generate SolutionsHow / How Form

Goal:Decrease number of residents losing weight

How?Improve Caloric Supplementation

How?Eliminate restrictive diets

How?Improve food appearance

How? Greater variety of supplements

How? Limit # of therapeutic diets available

How? Optimal timing of supplements

How? Team to review need for restrictions on individual patients

How? Provide garnishes

How? Table settings

Long Stay Residents With a UTI-

43 Facilities

Follow Up

“Data Sanity, A Quantum Leap to Unprecedented

Results”

Davis Balestracci Jr., MS

MGMA Press, 2009