Presenting Data & Standardized Infection Ratios

Presenting Data&

Standardized Infection Ratios

Tim Wiemken PhD MPH CICAssistant Professor of MedicineAssistant Director of Epidemiology and BiostatisticsUniversity of LouisvilleSchool of Medicine, Division of Infectious DiseasesClinical and Translational Research Support [email protected]: 502.852.4627www.ctrsc.net

Pre-Presentation Comment

• Do not collect any data you do not plan to use (unless your boss tells you to).

Pre-Presentation Comment

• What constitutes ‘use’?1. Displaying somewhere other than just a

meeting

2. Preliminary to a follow-up to collect data that will be used

3. Education

4. Targeted prevention

5. Surveillance (only if you educate with it!)

Overview

• What makes a good report?

• How to choose a chart

• Components of good charts

• Tips for reports

• Standardized Infection Ratios

Overview






What Makes a Good Report?

The catch all answer – it depends!

There are some key ideas you should think about when preparing any report


Create reports that can be USED, not just looked at.

…more later


1. How much time do you have?


2. Who is your audience?• How sophisticated are they?• What do they already know?• Remember that people think and speak differently


3. Why are you giving the presentation?• To show data? (I hope not!)• To provide the status of something?• To prove your worth?• To affect change?• To get someone fired?


Each of these helps you decide one key piece of information:

What do you NEED to present?


A few rules:

1. Never present more than you need to present.


A few rules:


2. Never present less than you need to present.


A few rules:


2. Never present less than you need to present.

3. Keep it as short as possible.


Therefore, focus on what is necessary to meet your goals.


Yeah, so smarty pants, what is necessary?


IT DEPENDS!


Although what is needed depends on how much time you have, your audience, and your goal:

1.It is rarely OK to present raw data by itself:


Infection Prevention and Control Report, May 2012:

CLABSI: 5

VAP: 4

CAUTI: 7

MRSA: 10



CLABSI: 5

VAP: 4

CAUTI: 7

MRSA: 10



This is only slightly better.This is only slightly better.


Although what is needed depends on how much time you have, your audience, and your goal:

2.It is rarely OK to present significant amounts of text

– E.g. policies, procedures, etc.


2. It is rarely OK to present significant amounts of text

– No one wants a six page policy handed to them in a meeting to review.

– It will not get reviewed (at least not well).– Send these out early… and many times.


The Holy Order of Data Quality (worst to best).

1.Text2.Raw numbers3.Raw rates4.Tables of chronological numbers or rates5.Charts (hierarchy of charts later…)6.Charts with #47.Charts with #1, #3, and more fun statistics

Overview






Choosing a Chart

• Charts– Numbers alone are bad

Choosing a Chart

• Charts– Visual depiction is easier to quickly

comprehend

Choosing a Chart


comprehend– Choosing the correct chart is extremely

important

Choosing a Chart



important– Some charts show trends or comparisons in

data

Choosing a Chart



important– Some charts show trends or comparisons in

data– The best chart does not have to be explained

Choosing a Chart

Choosing a Chart

Choosing Charts

• Pie– Less appropriate than one would think– Categorical data that comprise of portions that

add to 100% – Never use 3D pie charts – Do not use a pie chart if you have a lot of

categories

Choosing Charts

• Pie– Do not use a pie chart if you have a lot of

categories

Choosing Charts

• Pie

“Use a pie chart when you don’t have anything to say”

- Dr. Julio Ramirez

“Use a pie chart when you don’t have anything to say”

- Dr. Julio Ramirez

Pie Charts

No!

Choosing a Chart

Choosing Charts

• Bar– Snapshots of data [rates in one month]– Comparing data across different categories– Never use 3D charts unless you have a 3rd

dimension!

Bar Charts

No!

Bar Charts

• Is it ever appropriate to use 3D charts?

Yes! (Really?)

Choosing a Chart

Choosing a Chart

• Line – Longitudinal data (data over time)– Contiguous points

Choosing a Chart

• Line – Longitudinal data (data over time)– Contiguous points– Rates, Counts, etc. by Month/Quarter/Year

• Run charts• Statistical Process Control charts

Choosing a Chart

• Run charts– Use when you have few time periods (e.g.

<25 months).

Choosing a Chart

• Run charts– Anatomy

• Center Line / Median –represents the median of all of the data points.

• X-axis –represents the time period of interest (days, weeks, months, quarters, years).

• Y-axis –represents the scale of the plotted data points (e.g. rate or count of infection).

• Data points – the actual data values.

Choosing a Chart

• Run chartsY-axis (Rate)

X-axis (Month)

Center Line (Median)

Data Points (<25)

Choosing a Chart

• Run charts– Use

• Use to identify when the data are different than you expect (for better or worse) through detecting abnormal variation

Choosing a Chart

• Run charts– Rules for abnormal variation

1.Seven or more consecutive points on either side of the Center Line (median).

2.Five or more consecutive points increasing or decreasing.

3.Fourteen or more consecutive points alternating up and down.

Choosing a Chart

• Run charts

Rule 17 points below median

Rule 25 consecutive points increasing

Choosing a Chart

• Statistical Process Control (SPC) charts– Use when you have many time periods (e.g.

≥25 months).– These are much better than run charts.

Choosing a Chart

• SPCcharts– Anatomy

• Center Line / Mean–represents the average of all of the data points.

• X-axis –represents the time period of interest (days, weeks, months, quarters, years).

• Y-axis –represents the scale of the plotted data points (e.g. rate or count of infection).

• Data points – the actual data values.• Standard deviation lines (control limits) – represent 1,

2 or 3 standard deviations on each side of the center line

SPC Chart Components

MeanMeanMeanMean


+1 SD +1 SD +1 SD +1 SD

-1 SD-1 SD-1 SD-1 SD


+2 SD +2 SD +2 SD +2 SD



+3 SD +3 SD +3 SD +3 SD


Upper Control Limit (UCL)

Lower Control Limit (LCL)

Types of SPC Charts

Choosing the correct chart can be difficult - it is also very important.Control limits are calculated differently for each chart....although they are marked in standard deviations, the formulas vary...

Types of SPC Charts

Choosing a Chart

• SPC charts– Use

• Same as Run Charts, except• Identify different types of variation

Variation

Common Cause - ‘ In Control’Common Cause - ‘ In Control’

Special Cause - ‘Out of

Control’


Control’

Rules to Detect Special-Cause Variation

• one point above or below 3SD

• two of three points above/below 2SD

• four of five points above/below 1SD

• eight points in a row on either side of the mean

• trends of 6 points in a row increasing or decreasing

• fifteen points in a row within 1SD

• fourteen points in a row alternating up and down

• eight points in a row outside of 1SD

Variation

Common Cause - ‘ In Control’Common Cause - ‘ In Control’


Control’


Control’

A Special SPC Chart

• Often it is useful to provide reports that identify the ‘time since the last event’.

• This motivates and provides some competition

A Special SPC Chart

• Rare events make terrible charts.

• Rates for rare events do not show you much.

A Special SPC Chart

• You can combine the benefits of the ‘time between events’ and the issues with rare events into an SPC g Chart.

C chart (number of events)

g chart (time between events)

Choosing a Chart

Choosing Charts

• Radar– Rarely appropriate – confusing– May work for showing multiple percentages

in one small chart – Bar chart may still be better.

Radar Charts

Overview






Components of Good Charts

• Good charts tell the complete picture of what they contain


• Good charts tell the complete picture of what they contain

• Ideally, someone who is not familiar with the data being presented should be able to easily and quickly understand what the chart is representing


• Never use 3D charts unless you have a 3rd dimension


• The scale used for charts (that have a scale) should begin at zero and end near the highest value– Changing the axis scale can tell a drastically

different story to those that are just glancing

Same Data, Different Scale

What would you say about these two services?

MRSA colonization rate by service

Same Data, Different ScaleMRSA colonization rate by service

MRSA colonization rate by service

How about now?

Overview






Tips for Reports

• Title must be very specific– Include person (organism/disease

state/compliance measure, etc.), place and time

• Example:– Hospital-associated Methicillin-resistant

Staphylococcus aureus Isolates: Hospital X, Acute Care ICU, January 2007-January 2010

Tips for Reports

• Data definitions should be included as a footnote or on an introductory page.

Tips for Reports

• Data definitions should be included as a footnote or on an introductory page.

• Example: Clostridium difficile– Definition for numerator

• Are you measuring toxin only? Antigen? By PCR? On liquid stool only?

– Definition for denominator• Patient-days? Admissions?

Example introductory slide for MRSA rates

Hospital-associated Methicillin-resistant Staphylococcus aureus

(MRSA)

Case of MRSA (Numerator): A case of MRSA was defined as a new and unique, hospital-associated (isolated >48 hours after admission), microbiological isolate from a patient admitted to hospital x during the month of interest without a prior history of MRSA.

Patient days (Denominator): The denominator for the calculation of the rate of MRSA was defined as the number of patient-days for hospital x during the month of interest, regardless of risk status.

Rate: (Numerator / Denominator) * 1,000

Tips for Reports

• For charts, you should also include a formula for how the rate was calculated.

Tips for Reports

• For charts, you should also include a formula for how the rate was calculated.– # Unique, hospital-associated MRSA

isolates divided by # Patient-days of care X 1,000

Tips for Reports

• For charts, you should also include a formula for how the rate was calculated.– # Unique, hospital-associated MRSA

isolates divided by # Patient-days of care X 1,000

– Be specific. Isolates, Infections, Colonizations, etc.

Tips For Reports

• Include the appropriate chart!

UCL

0.004

CL 0.001

0.0000

0.0005

0.0010

0.0015

0.0020

0.0025

0.0030

0.0035

0.0040

0.0045

Ra

te P

er

Be

d-d

ay

of

Ca

re

Date

Tips for Reports

• Raw data is important to be able to understand what is in the chart and to verify the data.

Tips for Reports


• Some people just like the numbers.

Tips for Reports



• Raw data should include numerator, denominator and standardized rate.

Tips for Reports



• Raw data should include numerator, denominator and standardized rate.

Tips for Reports

• Including aggregated data for quick comparisons is also a good idea.

• Yearly average rates may work.

• When possible, include P-values to detect statistically significant differences (rate comparison example to come..)

Average Rate per 1000 Bed-days of Care 2008


Average Rate per 1000 Bed-days of Care 2010 YTD

1.3 1.7 0.7

Tips for Reports

• Always include a description of the assessment of the chart and your plans.

Tips for Reports

• Always include bubbles or call-out boxes that adequately describe dates and components of any interventions.

Tips for Reports

• You may use a bubble to describe if your surveillance definition changes.

Tips for Reports

• You may use a bubble to describe if your surveillance definition changes.

• A better idea is to just start over.

UCL 0.003

CL 0.001

0.0000

0.0005

0.0010

0.0015

0.0020

0.0025

0.0030

0.0035

0.0040

0.0045

0.0050

Ra

te P

er

Be

d-d

ay

of

Ca

re

Date

June 2006:Rate peaked to 4.30. Suspect increase was due to a change in MRSA precautions implemented affected staff compliance with when to wear PPE.

Hospital-Associated Methicillin-resistant Staphylococcus aureus Isolates: Hospital X, MICU, January 2006 – January 2010

# HA MRSA Isolates divided by # Bed-days of Care

Date

Number of Isolates

Number of Bed-Days of Care

Rate Per 1000 Bed-days of Care

Jan 10

Feb 10

Mar 10

Apr 10

May 10

Jun 10

Jul 10Aug 10

Sep 10

Oct 10

Nov 10

Dec 10

0 2

1319 1117

0.0 1.8



Average Rate per 1000 Bed-days of Care 2010 YTD

0.4 0.6 0.9

Assessment: Process is in statistical control.

Plan: Continue surveillance activities.

Tips for Reports

• Some reports must be text, although every attempt to transform into a graphic should be made.

Tips for Reports


• Keep text to a minimum with summary information presented.

Tips for Reports


• Keep text to a minimum with summary information presented.

• Email out early and often.

Tips for Reports

• Another option is a dashboard.

• This may be particularly useful for providing back to staff to post in their work areas.

• After all – a report that isn’t used for something other than looking pretty is not particularly worth your time!

Tips for Reports

Tips for Reports

• KHA has a wonderful template for you!

Overview






Standardized Infection Ratio

• SIR

• A way to standardize/risk adjust rates for better comparability across institutions


• SIR

• Why are rates not comparable?– Different types of institutions have different

patient populations– These patients have different risks. – Therefore, comparing a University Hospital

MICU to a 4 bed Cardiac Unit in Wyoming is not useful.


• SIR

• SIR takes into account the types of patients seen in various different types of units or wards in a healthcare facility.


• SIR

• Compares the number of infections you have to the number you expect to see, based on some benchmark rate.


• SIR– SIR>1 : Worse than expected numbers– SIR <1 : Better than expected numbers– SIR = 1 : As expected

Standardized Infection Ratio• Advantages

– Adjusts based on benchmarked rates

– Adjusts for various locations [ICU vs Ward] or types of surgeries, etc.

– Excellent for comparing low numbers [removes high variation]

– Greater precision for adjustment

• Disadvantages

– Statistical bias = may not make appropriate comparisons

– Utility beyond SSI is relatively unstudied

– Wide understanding is just not there.

ICHE April 2006, 27(4): 427-429. ICHE January 2005, 26(1):8-9.


SIR will NEVER replace rates, particularly for interhospital comparisons (e.g. trends in your facility).

Their main use is for interstate comparisons and comparisons between facilities in a healthcare network.

SIR for CLABSI

1. Get observed numerator as usual

SIR for CLABSI


2. Get number of patient-days (line-days, vent-days, etc) as usual

SIR for CLABSI



3. Get NHSN benchmark rate from the relevant publication

SIR for CLABSI



3. Get NHSN benchmark rate from the relevant publication

4. Multiply benchmark rate by the number of patient days (line days, ventilator-days, etc)= expected number of infections

SIR for CLABSI

Divide your observed number by the expected number = SIR!

SIR for CLABSI

Example1.Get observed number of CLABSIs as usual: 5

SIR for CLABSI

Example1.Get observed number of CLABSIs as usual: 5

2.Get number of line-days as usual: 1,234

SIR for CLABSI

Example3.Get NHSN benchmark rate from the relevant publication

Can compare to different percentile

rates if that is your goal!

SIR for CLABSI

Example1.Get observed number of CLABSIs as usual: 52.Get number of line-days as usual: 1,2343.Get NHSN benchmark rate from the relevant publication: 1.9 4.Multiply benchmark rate by the number of line-days and divide by 1,000 = expected number of infections:

(1.9 *1,234) / 1,000 = 2.35

SIR for CLABSI

ExampleDivide your observed number by the

expected number = SIR!:

Observed (5)/ Expected (2.35) = 2.13

SIR for CLABSI

Example

SIR = 2.13

You are doing worse than expected given the benchmark.

SIR for CLABSI

• This method calculated an SIR for one particular unit.

• You can calculate an overall SIR that adjusts for multiple types of units. This makes it more comparable to other institutions.

Lets see how this works…

SIR for CLABSI

Example Using Free Software

http://innovationsforglobalhealth.com/ipstat/

Standardized Infection Ratio and SPC

• SIRs can be plotted on an XmR chart and may be more sensitive to detect special causes (very much controversy here!)

Final Thoughts

• Charts are good – use them whenever possible.

Final Thoughts

• Charts are good – use them whenever possible.

• Knowing your audience is important – don’t assume they know what you know or are interested in what you are interested in.

Final Thoughts

• Charts are good – use them whenever possible

• Knowing your audience is important – don’t assume they know what you know or are interested in what you are interested in.

• SIRs may be useful in some situations

Final Thoughts

• Make sure everyone knows how to read your charts

POST YOUR REPORTS WHEREVER YOU CAN

More time?

• Example of how to make a control chart using Excel.

• Calculating P-values to compare rates

• You can download this file at:http://innovationsforglobalhealth.com/files/Stat_Tests_v13.xls

Questions?

Thanks!

Documents

Presenting Data & Standardized Infection Ratios