Tutorial on Risk Adjusted P-chart Farrokh Alemi, Ph.D

Tutorial on Risk Adjusted P-chart

Farrokh Alemi, Ph.D.

Have Changes Led to Improvement?

Common cause variation (changes in outcomes because of chance) is everywhere.

Decision makers often mistakenly attribute positive outcomes to their own skills and

negative outcomes to others, while in reality both could be a chance outcome

Why Chart Data?

To discipline intuitions

To communicate data in vivid graphical ways

Purpose of Risk Adjustment

Purpose of p-chart Purpose of risk-adjusted p-chart

To detect if the process has improved beyond

historical levels. Assumes historically we have been serving the same type of

patients as now

To detect if the process has improved beyond what can be expected from patient conditions

Data Needed

Data collected over time

Risk (expected outcomes) for each patient

Outcomes for each patient

The purpose is to improve not to get so lost in measurement to loose sight of improvement.

What Is Risk?

A patient’s condition or characteristics that affects the expected outcomes for the patient A severity index used to predict patient

outcomesClinicians’ consensus regarding expected

outcomesPatient’s self rating of expected outcomes

Time 1 Time 2 Time 3 Time 4 Time 5 Time 6 Time 7 Time 8

# deaths 2 3 1 3 2 2 4 2

8 9 7 7 7 7 7 8

Case

1 0.18 0.97 0.85 0.27 0.12 0.07 0.96 0.05

2 0.88 0.88 0.61 0.71 0.44 0.05 0.05 0.96

3 0.33 0.04 0.27 0.07 0.18 0.93 0.75 0.96

4 0.29 0.29 0.28 0.74 0.67 0.24 0.04 0.14

5 0.14 0.03 0.8 0.08 0.51 0.14 0.96 0.05

6 0.24 0.19 0.71 0.04 0.62 0.58 0.71 0.58

7 0.15 0.14 0.85 0.76 0.67 0.05 0.15 0.07

8 0.04 0.74 0.16

9 0.07

Mortality Risks for Inidvidual Patients

MI Patients Over 8 Months in One Hospital

Observed mortality during this time period

Nu

mb

er o

f ca

ses

Expected probability of mortality for case 8 in time period 1. Estimated from severity indices or experts’ consensus.

Elements of a Control Chart

X axis shows timeY axis shows probability of adverse eventsObserved rates are plotted against time sequenceUpper control limit is a line drawn so that points above it are rare to be observed by mere chanceLower control limit is a line drawn so that points below it are rare to observe by mere chance Lets take

a look

An Example of P-chart

Example p-chart

0

0.2

0.4

0.6

0.8

0 1 2 3 4 5 6 7

Time

Pro

ba

bil

ity

of

ad

vers

e o

utc

om

eUpper control limit

Lower control limit

Observed rate

Steps in Creating P-chart for Mortality

1. Check assumptions

2. Calculate observed rates and plot them

3. Calculate expected rates and plot them

4. Calculate expected deviation

5. Calculate control limits and plot them

6. Interpret findings

7. Distribute chart and interpretation

Step One: Check Assumptions

We are examining discrete events that either happen or do not happen, e.g. mortality among MI patients, falls among nursing home patients, error in medical record entry, etc.The event is not rare, meaning the probability of it occurring exceeds 5% for each time period.Observed events are independent from each other. The probability of the event occurring does not change over time. This assumption is violated if one patient’s outcomes affects the outcomes for others, e.g. when dealing with infectious diseases.

Step Two: Calculate Observed Rates and Plot

Pi = Mortality rate in time period “i”

Oi = Mortality in period “i”

ni = Number of cases in time period “i”

Pi = Oi / ni

Number Observed Mortalityof cases Mortality rate

8 2 0.25

9 3 0.33

7 1 0.14

7 3 0.43

7 2 0.29

7 2 0.29

7 4 0.57

8 2 0.25

Observed Mortality Rates for All Time Periods

Plot of mortality rates

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25

T2 9 0.33

T3 7 0.14

T4 7 0.43

T5 7 0.29

T6 7 0.29

T7 7 0.57

T8 8 0.25

Plot of the Observed Rates

Time period 7 and 3 seem different but don’t rush to judgment.

Wait, until you see control limits of what could have been expected.

0.00

0.10

0.20

0.30

0.40

0.50

1 2 3 4 5 6 7 8

Time periodM

ort

ali

ty r

ate

Step Three: Calculate Expected Mortality

Eij = Expected mortality of case ‘j’ in time period “i”

Ei = Expected mortality for time period “i”

Ei = (j=1,…,ni Eij ) / ni

Sample calculation:

E1= (.18+.88+.33+.29 +.14+.24+.15+.04)/8

Expected Mortality Rates for All Time Periods

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25 0.28

T2 9 0.33 0.37

T3 7 0.14 0.62

T4 7 0.43 0.38

T5 7 0.29 0.46

T6 7 0.29 0.29

T7 7 0.57 0.52

T8 8 0.25 0.37

Plot of Expected Mortality

Plotting expected mortality helps interpret the observed rates but does not settle the question of whether differences are due to chance.

0.00

0.20

0.40

0.60

0.80

T1 T2 T3 T4 T5 T6 T7 T8P

rob

ab

ility

of

mo

rta

lity

Observed mortality Expected mortality

Step Four: Calculate Expected Deviation

Eij = Expected mortality of case ‘j’ in time period “i”

Di = Standard deviation of expected mortality in time period “i”, called by us as the Expected Deviation

Di = ( j=1,…,ni Eij (1-Eij))0.5 / ni

See sample calculation

Expected Deviation for Time Period 1

Case

Expected mortality E1j E1j *(1-E1j)

1 0.18 0.15

2 0.88 0.11

3 0.33 0.22

4 0.29 0.21

5 0.14 0.12

6 0.24 0.18

7 0.15 0.13

8 0.04 0.04

1.15

1.07

0.13Divided by # of casesSquare root of sum

SumExpected deviation

A

BC

D

Results: Expected Deviations for All Time

Periods

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25 0.28 0.13

T2 9 0.33 0.37 0.11

T3 7 0.14 0.62 0.16

T4 7 0.43 0.38 0.14

T5 7 0.29 0.46 0.17

T6 7 0.29 0.29 0.13

T7 7 0.57 0.52 0.05

T8 8 0.25 0.37 0.11

Step Five: Calculate Control Limits

UCLi = Upper control limit for time period “i”

LCLi = Lower control limit for time period “i”

t = Constant based on t-student distribution

UCLi = Ei + t * Di

LCLi = Ei - t * Di

Where for 95% confidence intervals:Degrees of

freedom

95% student t

value

7 2.37

8 2.31

9 2.26

10 2.23

Calculation of Control Limits for Time Period 1

UCL = .28 + 2.37 * .13 = .59

LCL = .28 –2.37 * .13 = -.03

t-value

Negative limits are set to zero as negative probabilities are not possible

Results: Control Limits for All Time Periods

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25 0.28 0.13 0.60 0.00

T2 9 0.33 0.37 0.11 0.62 0.12

T3 7 0.14 0.62 0.16 1.02 0.23

T4 7 0.43 0.38 0.14 0.72 0.04

T5 7 0.29 0.46 0.17 0.88 0.04

T6 7 0.29 0.29 0.13 0.60 0.00

T7 7 0.57 0.52 0.05 0.64 0.39

T8 8 0.25 0.37 0.11 0.63 0.12

Plot Control Limits

UCL

LCL

Step Six: Interpret Findings

There are no points above UCL. There is one point below LCL.In time period 3, mortality is lower than what can be expected from patient’s conditions.All other time periods are within expectations, even time period 7 with its high mortality rate is within expectation.

Step Seven: Distribute Control Chart

Include in the information:How was severity measured and expected

mortality anticipated?Why are assumptions met?What does the control chart look like?What is the interpretation of the findings?

Index of ContentClick on the Slide You Wish to Review

1. Check assumptions

2. Calculate and plot observed mortality

3. Calculate expected mortality

4. Calculate expected deviation

5. Calculate and plot control limits

6. Interpret findings

7. Distribute control chart