Understanding and Presenting Quality Data in Healthcare Center for Clinical Effectiveness Loyola University Health System

Understanding and PresentingQuality Data in Healthcare

Center for Clinical EffectivenessLoyola University Health System

PDSAPDSA

PlanPlan an intervention including a plan for collecting data

DoPerform the intervention

StudyAnalyze the data and

study the results

ActRefine the change based on what was learned

How do you

get the team

to agree on one

analysis?

PlanPlan an intervention including a plan for collecting data

DoPerform the intervention

StudyAnalyze the data and

study the results

ActRefine the change based on what was learned

3

Why graph data?Why graph data?

Year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

2001 90.0 90.0 95.0 96.0 95.0 97.0 96.0 94.0 94.0

2002 95.0 97.0 96.0 94.0 95.0 98.0 99.0 98.0 99.0 94.0 94.0 94.0

2003 94.0 97.0 97.0 93.0 93.0 95.0 95.0 92.0 93.0 96.0 88.0 86.6

2004 94.0 93.0 92.0 93.0 93.2 87.0 91.7 82.9 87.4 84.9 81.7 81.0

2005 91.0 89.9 84.3 82.2 97.0 80.4 83.7 82.9 81.9 82.0 84.0 76.1

2006 81.3 88.2 82.6

4

What’s your analysis?What’s your analysis?

Mortality before and after new protocol

4%5%

0%

1%

2%

3%

4%

5%

6%

2004 2005

Perc

ent m

orta

lity

New Protocol in January

5

More details, so what’s the story?More details, so what’s the story?

Feb-04Apr-04

Jun-04Aug-04

Oct-04Dec-04

Feb-05Apr-05

Jun-05Aug-05

Oct-05Dec-05

Perc

ent m

orta

lity


New Protocol introduced in January

7%

8%

2%

3%

4%

5%

6%

Variation

“Extent to which a thing varies; amount of departure from a position or state; amount or rate of change”

- Webster’s Collegiate Dictionary

7

Two types of variationTwo types of variation

Common Cause Variation…1. Regular, random, or expected variation

2. Not “assignable” to any specific cause

3. A natural part of all processes

4. Performance remains predictable

5. No “real change”

Special Cause Variation…1. A real change (assignable variation)

2. Not an essential part of a process

3. The underlying cause should always be identified

4. Sometimes unanticipated

Feb-04 Apr

-04 Jun-0

4Aug

-04 Oct-04

Dec-04 Feb

-05 Apr-05 Ju

n-05

Aug-05 Oct-

05Dec

-05

Perc

ent m

orta

lity


New Protocol

introduced in

January7%

8%

2%

3%

4%

5%

6%

Feb-04 Apr

-04 Jun-0

4Aug

-04 Oct-04

Dec-04 Feb

-05 Apr-05 Ju

n-05

Aug-05 Oct-

05Dec

-05

Perc

ent m

orta

lity


New Protocol

introduced in

January7%

8%

2%

3%

4%

5%

6%

What kind of variation was this?

Common Cause Variation…1. Regular, random, or expected variation

2. Not “assignable” to any specific cause

3. A natural part of all processes

4. Performance remains predictable

5. No “real change”

Special Cause Variation…1. A real change (assignable variation)

2. Not an essential part of a process

3. The underlying cause should always be identified

4. Sometimes unanticipated

8

How can I differentiate…How can I differentiate…

– Natural variation

from

– Meaningful change

So what?– Natural variation

from

– Meaningful change

9

Results of misinterpreting random variation as Results of misinterpreting random variation as a significant changea significant change

We try to “correct” random variation

We celebrate random variation

How can we avoid these mistakes?

Control Charts

Getting the team on the right track

11

Parts of a control chartParts of a control chartM

edia

n G

luco

se V

alue

(mg/

dL)

Median Glucose

These information are confidential and to be used for quality improvement purposes onlyMonth (number of glucose tests)

03/20

05 (n

=2145

)

04/20

05 (n

=1641

)

05/20

05 (n

=1624

)

06/20

05 (n

=1238

)

07/20

05 (n

=1660

)

08/20

05 (n

=163

4)

09/20

05 (n

=1154

)

10/20

05 (n

=162

0)

11/20

05 (n

=152

5)

12/20

05 (n

=141

5)

01/20

06 (n

=178

0)

02/20

06 (n

=186

5)

03/20

06 (n

=160

3)

04/20

06 (n

=1651

)

05/20

06 (n

=1924

)

06/20

06 (n

=1110

)

115

120

125

130

(Each color change represents one std dev from the mean)

Lower Control Limit = Mean – 3 standard deviations

Upper Control Limit = Mean + 3 standard deviations

Mean = 122.6 mg/dL

This information is confidential and to be used for quality improvement purposes only

MonthJa

n-03

Feb-03

Mar-03

Apr-03

May-03

Jun-03

Jul-0

3

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Goal = 0.3

Title

Rat

e Analysis: ???

This information is confidential and to be used for quality improvement purposes onlyMonth

Jan-03

Feb-03

Mar-03

Apr-03

May-03

Jun-03

Jul-0

3

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

0.0

0.2

0.4

0.6

0.8

1.0UCL = 1.04

Mean = 0.49

Goal = 0.3

Title

Rat

eAnalysis: The rate remains consistently above goal.


Jan-03

Feb-03

Mar-03

Apr-03

May-03

Jun-03

Jul-0

3

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

Jan-04

Feb-04

Mar-04

Apr-04

May-04

Jun-04

Jul-0

4

Aug-04

0.0

0.2

0.4

0.6

0.8

1.0

Goal = 0.3

Title

Rat

eAnalysis: ???


Jan-03

Feb-03

Mar-03

Apr-03

May-03

Jun-03

Jul-0

3

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

Jan-04

Feb-04

Mar-04

Apr-04

May-04

Jun-04

Jul-0

4

Aug-04

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Mean = 0.52

Goal = 0.3

Title

Rat

eAnalysis: The rate remains consistently above goal.


Jan-03

Feb-03

Mar-03Apr-0

3

May-03

Jun-03

Jul-0

3

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

Jan-04

Feb-04

Mar-04Apr-0

4

May-04

Jun-04

Jul-0

4

Aug-04

Sep-04

Oct-04

Nov-04

Dec-04

Jan-05

Feb-05

Mar-05Apr-0

5

May-05

Jun-05

Jul-0

5

0.0

0.2

0.4

0.6

0.8

1.0

Goal = 0.3

Title

Rat

eAnalysis: ???

Rat

e

Title


Jan-03

Feb-03

Mar-03Apr-0

3

May-03

Jun-03

Jul-0

3

Aug-03

Sep-03

Oct-03

Nov-03

Dec-03

Jan-04

Feb-04

Mar-04Apr-0

4

May-04

Jun-04

Jul-

04Aug-04

Sep-04

Oct-04

Nov-04

Dec-04

Jan-05

Feb-05

Mar-05Apr-0

5

May-05

Jun-05

Jul-0

5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Mean = 0.45

Goal = 0.3

Analysis: The rate has decreased past goal (0.26 since December 2004). Intervention #1 was begun in Nov-04, and phase-in of intervention #2 began in Feb-05.

Intervention #2

Intervention #1

18

UnwantedUnwanted special causes special causes

VS

19

What do you do with What do you do with unwanted special cause variation?unwanted special cause variation?

Mea

sure

men

t Sca

le

Special cause variation

Jan 0

3

Feb 03

Mar 03

Apr 03

May 03

Jun 0

3Ju

l 03

Aug 03

Sep 03

Oct 03

Nov 03

Dec 03

Jan 0

4

Feb 04

Mar 04

Apr 04

0

50

100

150

200

250

300 UCL = 285.7

Mean = 140.3

Time (quarter, month, week, etc)

White boxes note: Data point is contributing to a special cause

Blue points note: Data point is common cause variation

Red boxes note: Data point is a special cause

20

1.1. Identify and label the causeIdentify and label the cause2.2. Correct (if a problem)Correct (if a problem)

Mea

sure

men

t Sca

le

SPECIAL CAUSE VARIATION

Jan 0

3

Feb 03

Mar 03

Apr 03

May 03

Jun 0

3Ju

l 03

Aug 03

Sep 03

Oct 03

Nov 03

Dec 03

Jan 0

4

Feb 04

Mar 04

Apr 04

0

50

100

150

200

250

300


Lost RN manager

EKG machines down for 2 weeks

Hired triage NP

Mea

sure

men

t Sca

le


Jan 0

3

Feb 03

Mar 03

Apr 03

May 03

Jun 0

3Ju

l 03

Aug 03

Sep 03

Oct 03

Nov 03

Dec 03

Jan 0

4

Feb 04

Mar 04

Apr 04

0

50

100

150

200

250

300


Mea

sure

men

t Sca

le


Jan 0

3

Feb 03

Mar 03

Apr 03

May 03

Jun 0

3Ju

l 03

Aug 03

Sep 03

Oct 03

Nov 03

Dec 03

Jan 0

4

Feb 04

Mar 04

Apr 04

0

50

100

150

200

250

300


Lost RN manager

EKG machines down for 2 weeks

Hired triage NP

Analysis: This process is unpredictablePe

rcen

t

Title

* Preliminary data for quality improvement purposes onlyMonth

Jan-04

(n=8)

Feb-04

(n=5

)

Mar-04

(n=2)

Apr-04 (

n=7)

May-04

(n=5

)

Jun-04

(n=3

)

Jul-0

4 (n=3

)

Aug-04 (n

=2)

Sep-04

(n=1

)

Oct-04

(n=8

)

Nov-04 (

n=4)

Jan-05

(n=3)

Feb-05

(n=4

)

Mar-05

(n=4

)

Apr-05 (

n=3)

May-05

(n=2

)

Jun-05

(n=4

)

Jul-0

5 (n=5

)

Aug-05 (n

=2)

Sep-05

(n=2

)

Oct-05

(n=2

)

Nov-05 (

n=3)

Dec-05

(n=2

)

*Jan-06

(n=9

)

*Feb-06

(n=5

)

*Mar-

06 (n

=3)

*Apr-0

6 (n=2)

*May

-06 (n

=4)

*Jun-06

(n=1

)

0

50

100

150

200

UCL = 132.02

Mean = 55%

LCL = 0.00

Self Test

Perc

ent

Title

These data are confidential and to be used for quality improvement purposes only.

Month (number of patients)

UCL = 102.95

Mean = 91.5%

LCL = 80.02

07/20

04 (n

=54)

08/20

04 (n

=55)

09/20

04 (n

=49)

10/20

04 (n

=57)

11/20

04 (n

=49)

12/20

04 (n

=45)

01/20

05 (n

=42)

02/20

05 (n

=58)

03/20

05 (n

=58)

04/20

05 (n

=63)

05/20

05 (n

=45)

06/20

05 (n

=56)

07/20

05 (n

=56)

08/20

05 (n

=44)

09/20

05 (n

=42)

10/20

05 (n

=57)

11/20

05 (n

=49)

12/20

05 (n

=57)

01/20

06 (n

=52)

02/20

06 (n

=59)

03/20

06 (n

=55)

04/20

06 (n

=47)

05/20

06 (n

=45)

06/20

06 (n

=36)

80

85

90

95

100

105

What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?

91.5%

80% - 100%



-1.3 (combined pre and post intervention)

Unclear (due to improvement)


Perc

ent

Title

This information is confidential and to be used for quality improvement purposes only.

Quarter

UCL = 2.3

Mean = -1.3

LCL = -4.9

2002

Q1 (

N=130

)

2002

Q2 (

N=133

)

2002

Q3 (

N=117

)

2002

Q4 (

N=111)

2003

Q1 (

N=121

)

2003

Q2 (

N=102

)

2003

Q3 (

N=88)

2003

Q4 (

N=118

)

2004

Q1 (

N=117

)

2004

Q2 (

N=130

)

2004

Q3 (

N=112

)

2004

Q4 (

N=130)

2005

Q1 (

N=152

)

2005

Q2 (

N=137)

2005

Q3 (

N=130

)

2005

Q4 (

N=155

)

2006

Q1 (

N=140)

-5

-4

-3

-2

-1

0

1

2

Intervention

Perc

ent

Title

May-04

Jul-0

4

Sep-04

Nov-04

Jan-05

Mar-05

May-05

Jul-0

5

Sep-05

Nov-05

Jan-06

Mar-06

May-06

0

20

40

60

80

100

120

UCL = 101.21

Mean = 80.8%LCL = 60.43


80.8%

Unsure (unpredictable)


What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)? Unclear (due to

improvement)

143 Can separate before after interventionWhat is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?

Valu

e

Title

These data are confidential and to be used for quality improvement purposes only.Month (number of results)

UCL = 150.07

Mean = 143

LCL = 135.73

04/20

04 n=(1

1031

)

05/20

04 n=(1

2789

)

06/20

04 n=(6

269)

07/20

04 n=(7

022)

08/20

04 n=(1

2089

)

09/20

04 n=(1

2990

)

10/20

04 n=(1

2739

)

11/20

04 n=(1

2563

)

12/20

04 n=(1

3853

)

01/20

05 n=(1

3486

)

02/20

05 n=(1

0513

)

03/20

05 n=(1

3535

)

04/20

05 n=(1

0912

)

05/20

05 n=(1

1773

)

06/20

05 n=(1

0645

)

07/20

05 n=(1

0543

)

08/20

05 n=(1

3798

)

09/20

05 n=(1

2604

)

10/20

05 n=(1

1875

)

11/20

05 n=(1

3137

)

12/20

05 n=(1

5766

)

01/20

06 n=(1

4068

)

02/20

06 n=(1

2611

)

03/20

06 n=(1

4501

)

136

138

140

142

144

146

148

150

152

Intervention

Rat

e pe

r 100

cas

es

Title

This information is confidential and to be used for quality improvement purposes onlyQuarter (Number of LUMC cases)

UCL = 0.3

Mean = 0.1

2004

Q1 (

n=255

7)

2004

Q2 (

n=250

7)

2004

Q3 (

n=249

0)

2004

Q4 (

n=237

2)

2005

Q1 (

n=244

5)

2005

Q2 (

n=246

4)

2005

Q3 (

n=240

4)

2005

Q4 (

n=235

9)

0.05

0.10

0.15

0.20

0.25

0.30


0.1/100 casesNeither (too early)

Unsure (too early)


Blo

odst

ream

Infe

ctio

n R

ate

NICU Central Line Related Bloodstream Infection Rate

305

1

299

2

335

2

263

2

310

0

474

4

427

1

365

2

317

1

270

0

209

2

285

1

480

2

596

1

488

3

371

1

423

1

639

4

644

3

449

3

439

5

471

3

353

3

246

3

434

3

602

2

531

3

531

0

573

2

499

5

420

4

303

0

257

1

539

4

521

1

# Line Days# Infections

Jan-03

Mar-03

May-03

Jul-0

3

Sep-03

Nov-03

Jan-04

Mar-04

May-04

Jul-0

4

Sep-04

Nov-04

Jan-05

Mar-05

May-05

Jul-0

5

Sep-05

Nov-05

0.0

0.5

1.0

1.5

2.0

UCL = 1.56

Mean = 0.51

LCL = 0.00

ICU POINT PREVALENCE IN NOV AND DEC 2004

INSERTION GUIDELINES DISSEMINATED MARCH 11, 2005

E-LEARNING MODULES ACTIVATED MAY 27, 2005

Rat

e pe

r 100

day

s

Title305

1

299

2

335

2

263

2

310

0

474

4

427

1

365

2

317

1

270

0

209

2

285

1

480

2

596

1

488

3

371

1

423

1

639

4

644

3

449

3

439

5

471

3

353

3

246

3

434

3

602

2

531

3

531

0

573

2

499

5

420

4

303

0

257

1

539

4

521

1

# Days# Events

Jan-03

Mar-03

May-03

Jul-0

3

Sep-03

Nov-03

Jan-04

Mar-04

May-04

Jul-0

4

Sep-04

Nov-04

Jan-05

Mar-05

May-05

Jul-0

5

Sep-05

Nov-05

0.0

0.5

1.0

1.5

2.0

UCL = 1.56

Mean = 0.51

LCL = 0.00

Intervention #3Intervention #2

Intervention #1


0.51/100 days

0–1.5 /100 days


Perc

ent

Title

Confidential Material for Quality Improvement Purposes

Month (Total Patient Years of Therapy)

Mean = 0.3%

Comparison

Jan-04

(n=1

13)

Feb-04

(N=1

12)

Mar-04

(N=11

4)

Apr-04 (

N=113

)

May-04

(N=1

19)

Jun-04

(N=12

2)

Jul-0

4 (N=1

23)

Aug-04 (N

=128

)

Sep-04

(N=12

8)

Oct-04

(N=1

25)

Nov-04 (

N=127

)

Dec-04

(N=1

31)

Jan-05

(N=1

32)

Feb-05

(N=1

25)

Mar-05

(N=13

3)

Apr-05 (

N=133

)

May-05

(N=1

38)

Jun-05

(N=1

38)

Jul-0

5 (N=1

33)

Aug-05 (

N=136

)

Sep-05

(N=13

6)

Oct-05

(N=1

35)

Nov-05 (

N=136

)

Dec-05

(N=1

36)

Jan-06

(N=1

35)

Feb-06

(N=1

38)

Mar-06

(N=1

40)

Apr-06 (

N=138

)

May-06

(N=1

41)

0.0

0.5

1.0

1.5

2.0

2.5


0.3%

0-1.7%


30

Benneyan RG, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003; 12: 458-464.

Carey, Raymond G. and Lloyd, Robert C. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. White Plains, NY: Quality Resources Press, 1995.

Wheeler Donald J, Understanding Variation: The Key to Managing Chaos. Knoxville, TN: SPC Press, 1993.

Additional SPC Reading:Additional SPC Reading:

Documents

Understanding and Presenting Quality Data in Healthcare Center for Clinical Effectiveness Loyola University Health System