Statistical Process Control (SPC) and Shewhart Charts QA Cohort 6 Residency 3

Statistical Process Control (SPC) and Shewhart Charts

QA Cohort 6 Residency 3.

Shewhart Charts

aka Control Charts

Objectives

By the end of this session, you will be able to:

1.Determine when to use a control chart in your work

2.Explain key features of a control chart

3.Choose the right chart for your type of data

4.Apply tests to analyze a control charts

5.Understand purpose of revising control limits on the chart

Reference for all content:

A Control Chart

A Control Chart with Different Sample Sizes

Methods for Improvement, Accountability and Research

Improvement Accountability/Performance Measurement

Clinical Research

Purpose Improvement of care; application of evidence

Comparison between organizations or to a criteria, reassurance, spur for change

Generate evidence or new knowledge

Data Collection Accept consistent bias in data collection

Statistical adjustments to reduce bias Design of experiment to reduce bias

Sample Size Small sequential samples of data

100% of available data Large samples so there is enough data for possible hypothesis

Analysis Run Charts of Shewhart Charts NA Inferential statistics and hypothesis testing(assumes same : T-tests, F-tests, Regression Analysis, etc). with p-value

Adapted from: Solberg, L.I., Mosser, G., McDonald, S. (1997). The three faces of performance measurement: improvement, accountability, and research. Joint Commission Journal on Quality Improvement, 23(3), 135-147.

Control Charts: For Improvement or for Accountability/Performance Measurement?

Local level improvement: Shewhart Chart – use to learn about factors that are

influencing your system

Can use either a Shewhart Chart or a Run Chart to show evidence of improvement

Accountability Use to show the impact of a large improvement effort, or a

series of improvement efforts, over time

Review of Run Charts

What do run charts tell us?

1. Make performance visible

2. Is there improvement – probability based tests.

3. Is the improvement holding?

4. Gives a sense of impact of various PDSA cycles.

11

1. Make performance visible

12

2. Evidence of improvement: this “trend” is non-random pattern of data based on probability

Non-Random Signals on Run Charts

A Shift: 6 or more

An astronomical data point

Too many or too few runs

A Trend5 or more

The Data Guide, p 3-11

Evidence of a non-random signal if one or more of the circumstances depicted by these four rules are on the run chart. The first three rules are violations of random patterns and are based on a probability of less than 5% chance of occurring just by chance with no change.

14

3. Is the improvement holding?

15

4. Impact of PDSA cycles or specific changes

Why would you need a control chart?

Control Charts

A control chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes

Differences between a Run Chart and a Control Chart

Control Limits: use to identify Special Cause Variation

3 lines on a control chart:

- centre line (mean)

- upper control limit (mean+ 3 sigma)

- lower control limit (mean – 3 sigma)


Calculations for limits:

- depend on the type of data

- not the same as confidence intervals

- recommend using SPC software


Statistical properties:

probability of misinterpretation is low (less than 11% overall; less than 5% for some data)

When to use a control chart instead of a run chart?

1. To determine change quickly (improvement will be picked up more quickly than in a run chart)

2. Different samples sizes in each period

3. To see if results are stable and predictable?

4. Predict performance in coming weeks or months.

5. Reasons for variation

1. Evidence of Special Cause Variation (Improvement)

2. Different Sample Size for Each Time Period

3. Is system stable and predictable?

4. What will result be next month?

5. What are the sources of variation?

Rational ordering – covered in Res 4

Is Special Cause Variation Good or Bad?

For improvement work:

1.Bad: evidence of SCV in our existing system (baseline) means system is not stable.

2.Good: if we start off with a stable system, then we introduce improvement, SCV is evidence of improvement.

Special Cause Variation as Evidence of Improvement – Plain Language Interpretation

We can be confident that the variation we are seeing is not merely

due to chance, nor due to the effect of common causes – it is likely

due to the improvement effort.

Reflection

1. Opportunities for control charts in your improvement project – what will they tell you?

2. Opportunities for control charts in other parts of your work – what will they tell you?

Analyzing a Control Chart

1. Choose type of chart appropriate for the data (e.g. a P chart or a C chart)

2. Create a control chart once you have about 12 data points

3. Should have approximately 20 data points before establishing initial limits (freeze baseline limits)

4. Analyze using tests to detect special cause variation

5. Revise control limits as required







Commonly Used

Type of Data Type of Chart

Example Specs

Percent data P chart Percent of patients who received the flu shot.

Need numerator and denominator for each time period

Count Data (usually counts of something we are trying to reduce)

C chart Number of complaints received

Need to count the complaints each time period

Count Data as a Rate (usually infections or injuries)

U chart Infections per patient days

Need to count the infections and know what your base is (how many patient-days in the time period)

Averages X bar and S Average length of stay Need raw data – length of stay for each patient in that time period. You will get 2 charts displayed – one showing the average plotted on the control chart and one with the variation between points plotted on the chart.

Individual Data or Data Already Calculated

I Chart Length of Stay for Each patient plotted separately

Average length of stayPercent of patients who received flu shot.

Use if you only have one number for each time period. Either because you are only including one patient or if all you have is data that was already calculate.

Type of Data

Count or Classification (Attribute Data)

Count (nonconformities)

1,2,3,4, etc.

Classification (nonconforming)

either/or, pass/fail, yes/no

Equal area of

opportunity

Unequal area of opportunity

Unequal or equal subgroup size

Continuous (Variable Data)

Time, money, scaled measures,

Workload or productivity

Subgroup size of 1 (n=1)

Unequal or equal subgroup size (n>1)

C Chart U Chart P Chart I Chart (also known as X Chart) X Bar and S

Number of nonconformities

Nonconformities per unit

Percent nonconforming

Individual measurement

Average and standard deviation

Shewhart Chart Selection Guide

Each subgroup is composed of a single data value

Each subgroup has more than one data value

-

Health Care Data Guide, p. 151

Adapted from Health Care Data Guide, p. 151 & QI Charts Software

Practice Choosing a Chart

Attribute or Continuous

If Attribute, is it a count or classification (2 buckets)

If count, is there an equal or unequal ‘opportunity’

If continuous– is subgroup size=1 or >1?

Type of Chart

1. You have length of stay for all patients on your unit. You want to display it by month.






Type of Chart

2. Each week you calculate the number of patients who have a follow up booked (of all the patients you saw in the clinic)






Type of Chart

3. Every week, you plot the number of patient complaints received by your office






Type of Chart

4. Every month, you plot the number of times a restraint or seclusion was initiated, taking into account patient-days in that month.






Type of Chart

5. Every day in January, you calculate and plot the number of minutes between the 1st scheduled surgery start time and actual start time for the 1st surgery.






Type of Chart

6. Every day in January, you want to plot the number of minutes between the scheduled surgery start time and actual start time for all surgeries that day.

Homework: fill this table out for each key measure on your charter





Type of Chart







Using SPC Software













Case Study #1a

Case Study #1b

Percent of cases with urinary tract infection

Case Study #1b


Case Study #1c


Case Study #1c


Case Study #1d

Case Study #1d

Case Study #1e


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Au

g 20

11

Se

p 20

11

Oct

201

1

Nov

20

11

Dec

20

11

Jan

201

2

Fe

b 2

012

Ma

r 2

012

Ap

r 2

012

Ma

y 2

012

Jun

201

2

Jul 2

01

2

Au

g 20

12

Se

p 20

12

Au

g 20

12

Se

p 20

12

Oct

201

2

Nov

20

12

Dec

20

12

Jan

201

3

Fe

b 2

013

Fe

b 2

013

Ma

r 2

013

Ap

r 2

013

Case Study #1e


Outer third for Dec 2012 point is anywhere between 31 and 39

Subtract UCL-Centre Line (39-14=25)Divide by 3 (thirds) (25/3 =8)

Subtract result from UCL (39-8 =31)

Outer third







Revising Limits After Evidence of Improvement

Initial Extended Limits Reveal Improvement

40

50

60

70

80

90

100

M-04

M J S N J -05

M M J S N J -06

M M J S N J -07

M M

Ave

rage

Day

s CL = 88.16

UL = 97.86

LL = 78.47LL = 78.47

Revised Limits After Improvement

40

50

60

70

80

90

100

M-04

M J S N J -05

M M J S N J -06

M M J S N J -07

M M

Ave

rage

Day

s

CL = 88.16

UL = 97.86

LL = 78.47

CL = 62.1

UL = 71.8

LL = 88.2

Health Care Data Guide,

p. 124

Investigating Special Cause

• Analyzing control charts is an iterative process

• Based on analysis:

1. If SCV indicates a new system – revise limits

2. If SCV is a problem that you have dealt with – take out point and update limits

As an example, let’s say we discovered that the data in Period 8 was really low because there was a new clerk on that month. We now train all our new and casual staff on Med Rec. We can now actually remove this point.

Investigating Special Cause

Percent of Transfers with Med Rec Complete

More examples – interpreting SPC

64

Is there evidence of SCV? What do you think it means?

65

This system is unstable. Should investigate the reasons for special causes before trying to implement improvement.

13/14 P1 13/14 P2 13/14 P3 13/14 P4 13/14 P5 13/14 P6 13/14 P7 13/14 P8 13/14 P9 13/14P10

13/14P11

13/14P12

13/14P13

13/14P14

13/14P15

13/14P16

13/14P17

13/14P18

13/14P19

13/14P20

13/14P21

13/14P22

13/14P23

13/14P24

Subgroup 31.72% 37.91% 43.69% 51.55% 51.62% 51.76% 54.21% 55.40% 58.93% 55.64% 57.06% 60.05% 62.38%

Center 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74%

UCL 61.51% 60.93% 60.95% 61.58% 61.92% 61.47% 61.07% 60.85% 61.11% 62.51% 60.79% 60.96% 61.10%

LCL 41.98% 42.56% 42.54% 41.90% 41.56% 42.01% 42.42% 42.64% 42.38% 40.97% 42.69% 42.52% 42.39%

UCL

LCL

10%

20%

30%

40%

50%

60%

70%

80%

90%

Is there SCV in this baseline data? What is the interpretation?

Percent of cases seen by appropriate care provider.

13/14 P1 13/14 P2 13/14 P3 13/14 P4 13/14 P5 13/14 P6 13/14 P7 13/14 P8 13/14 P9 13/14P10

13/14P11

13/14P12

13/14P13

13/14P14

13/14P15

13/14P16

13/14P17

13/14P18

13/14P19

13/14P20

13/14P21

13/14P22

13/14P23

13/14P24

Subgroup 31.72% 37.91% 43.69% 51.55% 51.62% 51.76% 54.21% 55.40% 58.93% 55.64% 57.06% 60.05% 62.38%

Center 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74% 51.74%

UCL 61.51% 60.93% 60.95% 61.58% 61.92% 61.47% 61.07% 60.85% 61.11% 62.51% 60.79% 60.96% 61.10%

LCL 41.98% 42.56% 42.54% 41.90% 41.56% 42.01% 42.42% 42.64% 42.38% 40.97% 42.69% 42.52% 42.39%

UCL

LCL

10%

20%

30%

40%

50%

60%

70%

80%

90%

Might be evidence of improvement in P13. Revise limits after 20 data points and then interpret.

Percent of cases seen by appropriate care provider.

This hospital is part of a falls Collaborative. Is there SCV? What is the interpretation? What do the extended control limits tell you?

Stable system. No evidence of improvement (or no evidence it is getting worse). If no changes to the system, falls can be anywhere from 2 to 24 per month.

As part of a project to increase teams with patient reps, we went back and collected baseline data for one year then “extended” and “froze” these limits. New data that is added (post implementation) will be compared to the baseline limits.

The new data is now being compared to the old data. Note how the limits did not change with this new data (because we froze them). Is there evidence of improvement? Starting where?

New system. We will compare new data to these new limits now – which make more sense for the new system we have created.

Your updated job description….. as a QI expert

Knowing what kind of Shewhart Chart to use for different data

Knowing how to create the chart with control limits

Applying tests for analyzing Shewhart Charts (evidence of SCV)

Collaborating with others to understand the source of SCV

Deciding when/if to revise the limits – using software to do revisions

Explaining to others the benefit of SPC and why it is more appropriate than inferential statistics for healthcare improvement

Practice Analysis and Interpretation – Case Studies

Case Study #1

1. What is the appropriate chart – P, C, U, I, or Xbar and S?

2. Is there evidence of special cause variation? Mark it on the chart. Would you consider revising limits – where and why?

3. If you were discussing this with the manager, what would you suggest in terms of next steps?

4. Note data display principles (no gridlines, use of white space). Any issues?

Turnaround time to clean exam room after each patient. Manager is considering an improvement project to reduce turnaround time to 6 minutes or less. She saw an average turnaround time for last month and says the average is 8 minutes, so she doesn’t think they need any major changes. They measure turn around times for 60 consecutive patients and you plot it on a control chart.

Case Study #2


2. Is there evidence of special cause variation? Mark it on the chart.

3. Would you consider revising your limits?

4. What would your interpretation be. Write a one paragraph news article.

This is the number of falls (indoors) reported in PSLS in residential care facilities (province wide). PSLS office is being asked to comment to the Vancouver Sun on the number of falls and whether they have been decreasing over the last 4 years, since there was a campaign to prevent falls.

Case Study #3


2. Is there evidence of special cause variation? Mark it on the chart. (note – the upper limit is “artificial” because it caps at 100. Do not use upper line in your interpretation.

3. Would you consider revising your limits? What would this help you determine?

4. Your staff are frustrated and don’t think it is possible, because when patients have lots of meds it takes too long. Also, they say that when the project first started, the nurses were more keen to do it but now too many other improvement projects have taken precedence.

This is the percent of patients with med reconciliation completed upon discharge (by week). Your unit just started this practice, and you set a goal of 60% by the end of this year.

Case Study #4

1. This is an Xbar and S chart. Can you describe what each chart is displaying?

2. Is there evidence of special cause variation? Mark it on the chart.

3. Do you think the project has worked?

4. Would you consider revising your limits? What would this help you determine?

This is the average wait time for first time patients to see a occupational therapist in your clinic. Your clinic started a system of pooled referrals in order to decrease wait times to under 21 days.

Documents

Statistical Process Control (SPC) and Shewhart Charts QA Cohort 6 Residency 3