Www.ideasontario.ca Data is your Friend Collecting, Charting, Analyzing, and Interpreting Data to...

www.ideasontario.ca

Data is your Friend

Collecting, Charting, Analyzing, and Interpreting Data to Support Quality Improvement

Michael Campitelli and Ruth CroxfordQI Epidemiologists, Institute for Clinical Evaluative Sciences (ICES)

Doug MitchellDirector Decision Support, Guelph General Hospital

Susan TaylorDirector, QI Program Delivery, Health Quality Ontario

www.ideasontario.ca 2

• Relationships with commercial interests:– Grants/Research Support: None

– Speakers Bureau/Honoraria: None

– Consulting Fees: None

– Other: None

Faculty:Ruth Croxford

– Other: None

Faculty:Michael Campitelli

– Other: None

Faculty:Susan Taylor

• Relationships with commercial interests:

– Grants/Research Support: None– Speakers Bureau/Honoraria: None– Consulting Fees: None– Other: None

Faculty:Doug Mitchell

Outline

• Review (45-50 minutes)• Bar charts and Pareto charts• Scatter plots• Run charts and SPC charts• Statistical testing between groups

• Break (10 minutes)

• Case Studies specific to the health care sector you identify with the most (45-50 minutes)• Primary care• Long-Term care• Acute care

GUELPH GENERAL HOSPITAL – HIP AND KNEE REPLACEMENT IMPROVEMENT INITIATIVE

Abbreviated, anonymized version of the data

HC Data Guide, p 65 (fig 2.28)

Tools to learn from variation in data

Bar Charts

A B C0

% wait time 1 within target% wait time 2 within target

Bar Charts (fictional data)

Statistics are like bikinis.

What they reveal is suggestive, but what they conceal is vital.

Aaron Levenstein, Professor of Business, Baruch College

Pareto Charts

Pareto Charts(fictional data)

Reason (fictious data) FrequencyReferral missing X-rays 86Referral missing other health information 75Patient weight loss required 35Other medical 22Patient refused offered consult date 10Patient requested specific surgeon 43Surgeon schedule 25

Scatter Plots

Histograms

Bikini #1

Histograms

Surgeon % within target

Mean Median

A 33% 176 163B 24% 154 155C 66% 68 71

HC Data Guide, p 65 (fig 2.28)

Tools to learn from variation in data

Run Charts

Four (probability-based) rules to identify non-random signals of change in a run chart (Health Care Data Guide, pgs 76 – 85)• A trend

– Five or more consecutive points all going up or all going down.

• A shift– Six or more consecutive points either all above or all below

the median

• Too many or too few runs (crossings of the median)– Depends on the number of points on the graph - requires a

• An astronomical data point

http://www.qihub.scot.nhs.uk/knowledge-centre/quality-improvement-tools/run-chart.aspx

Run Charts

The Run Chart as a Bikini

Corresponding Shewhart Chart

Statistical Process Control (SPC) Charts

Shewhart Chart Selection Guide

HC Data Guide p. 151(fig 5.1)

Types of Measures – Continuous Variables

Patient ID

Wait 2 (days) - decision to procedure

1 3122 3493 4724 3155 2926 2867 2558 2659 297

10 27211 1112 28613 12214 16215 24716 281

Month Case 1 Case 2 Case 3 Case 4 Case 5 Case 6May-11 312 349Jun-11 472 315 292Jul-11 286 255 265 297 272 11

Aug-11 286 122 162 247 281 288Sep-11 211 289 391 226 272Oct-11 121 299 243 160 240 278

Nov-11 122 129 164 110 110 138Dec-11 112 101 104 139 251 87Jan-12 70 73 79 154 178 173Feb-12 153 190 344 359 21 286Mar-12 85 272 202 206 210 182Apr-12 184 287 208 260 31 122

MonthAverage Wait 2

May-11 330.5Jun-11 359.7Jul-11 148.2

Aug-11 153.4Sep-11 277.8Oct-11 147.9

Nov-11 200.8Dec-11 155.8Jan-12 200.3Feb-12 186.3Mar-12 152.8Apr-12 152.7

May-12 105.7Jun-12 115.3Jul-12 105.3

Types of Measures – Count Data

• Requires two columns of data for each time period: the count and the number of “opportunities”

• The event being counted can occur more than once per “opportunity”

WeekNumber of flash

sterilizationsNumber of surgeries

1 42 842 47 1463 51 914 45 1065 36 886 34 1267 37 818 49 869 39 8310 46 7711 28 7812 46 10813 34 7214 44 13115 41 83

Rate of flash sterilizations (flash sterilizations per

100 surgeries)

Types of Measures – Classification Data

• Requires two columns of data for each time period: the total number of people or events that were observed, and the number of “non-conforming” events.

Number within target wait time

Total number

Apr-11 0 1May-11 0 2Jun-11 0 3Jul-11 0 6Aug-11 1 12Sep-11 0 5Oct-11 1 16Nov-11 9 29Dec-11 6 21Jan-12 7 25Feb-12 15 39Mar-12 9 31Apr-12 7 22May-12 15 28Jun-12 9 31Jul-12 5 24

Percent (percent of patients seen within the target

Learning from a Shewhart Chart

• Rules for detecting special cause variation.• Annotation• Setting and re-setting the baseline

SPC Chart Rules

Baseline Data

First PDSA

New Baseline

PDSA 2

Final Graph

Rare Events (fictitious data)

date of complications

time from previous event

1-Jul-11 2930-Jul-11 24

23-Aug-11 1911-Sep-11 23

4-Oct-11 281-Nov-11 43

14-Dec-11 2912-Jan-12 3314-Feb-12 3823-Mar-12 424-May-12 4013-Jun-12 546-Aug-12 40

15-Sep-12 504-Nov-12 43

17-Dec-12 5611-Feb-13 50

2-Apr-13 4921-May-13 8312-Aug-13 888-Nov-13 6916-Jan-14 62

19-Mar-14 763-Jun-14 5124-Jul-14 82

14-Oct-14 70

Rare Events (fictitious data)

Data for Judgement vs. Data for Improvement

• Measurements towards a target may hide or discourage authentic and sustainable improvement

• Targets for accountability may focus on what is easily measured rather than what has value (process rather than outcome)

(fictitious data)

Statistical Testing

Statistical testing

• Statistical testing is a common form of analysis used in clinical research and epidemiological studies

• Tests the hypothesis that the average/proportion/rate of some outcome in one group of patients is equal to the average/proportion/rate in another group of patients

• Statistical tests produce a P-value, which represents the likelihood that the observed difference in the outcome between the two groups is due to chance

• Studies often set the significance level at 0.05, meaning if there is less than 5% chance the observed results are due to chance, we deem the results `statistically significant`

Statistical testing versus QI analysis

• While used heavily in clinical research and epidemiology, statistical testing is not the analytic method of choice (e.g., the `Gold Standard`) for quality improvement

• QI involves conducting sequential tests of change over time to some existing process; therefore, it is logical that tracking outcome and process measures over time in an SPC chart would be the preferred method of analysis

• Performing statistical tests, rather than tracking measures over time, may cause us to claim improvement when none has occurred, or miss improvement when some has occurred.

A medical unit has 40 COPD discharges per month.

On July 1, they implement a self-management training program prior to discharge for all patients.

There is a statistically significant decrease (p=0.026) in COPD readmission rates after the implementation.

Pre (Jan - Jun) Post (Jul - Dec)0.0%

12.5%Readmissions

P = 0.026

Here are the readmission rates plotted by month.

There is an apparent decrease happening throughout the year, perhaps due to other quality improvement initiative.

Difficult to tie decrease to the July 1 initiative

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

Readmissions1

Is statistical testing forbidden in QI…

• All QI projects should strive to track measures over time and use annotated run and SPC charts for analysis of their data

• Having said that, sometimes it is not feasible to collect data in any other fashion (e.g., satisfaction surveys which are burdensome and time-consuming to complete), and you are stuck with having to do a pre-post comparison

• The following website has multiple online calculators to help you perform basic statistical tests between 2 groups for averages (means), proportions, and rates:• http://www.socscistatistics.com/tests/

Thank You

• michael.campitelli@ices.on.ca• ruth.croxford@ices.on.ca

Delivered in partnership and collaboration with:

Funding provided by the Government of Ontario

Www.ideasontario.ca Data is your Friend Collecting, Charting, Analyzing, and Interpreting Data to...

Documents

Co-op Charting – Akron Power Squadron 2008 USPS – NOS Coop Charting Program - Chart 14844 Position Data Worksheet Method – DGPS(WAAS) Datum – WGS 84 Manufacturer

Oracle JDeveloper 11g Release 2 Tutorials - Charting With ADF Data Visualization Components

JAPANESE CANDLESTICK CHARTING TECHNIQUES · Library of Congress Cataloging-in-Publication Data Nison, Steve. Japanese candlestick charting techniques : a contemporary guide to the

Managing Third Party Risk In a world fraught w/Risk Trust In the Cloud How are you Protecting Customer Data? February 26, 2014 Case Study Vincent Campitelli

Qualifying Interferometric Swath Bathymetry Data for Charting€¦ · Qualifying Interferometric Swath Bathymetry Data for Charting. a Cohort plc company ... a Cohort plc company

DECEDENTS’ Need a lawyer? ESTATE NOTICES Find one at lrs ... · IMPERIA BETTY CAMPITELLI, a/d, BETTY CAMPITELLI, late of Hempfield Township , Westmoreland County, PA Exrx.: Gina

CHARTING THE COURSE: USING DATA TO SUPPORT FOSTER …

Re-Charting the Waters of Library-Writing Partnerships · Collaborative Assessment, and Learning Transfer ... quantitative data from pre- and post-learning ... Re-Charting the Waters

Charting UML Model Data in Sparx Systems Enterprise Architect · 2020. 9. 7. · Charting UML Model Data in Sparx Systems Enterprise Architect - Standard Charts7 September, 2020 ·The

Charting the Expanding Horizons of Big Data · Report Summary: Charting the Expanding Horizons of Big Data 5 Strategy and Driver Findings • Big Data! Big Time! – Nearly 9 of 10

Dojox charting Karthick S. Dojo Charting Presenting statistical data in a readable, eye-catching manner is important, but it can also be difficult. The

Charting the Expanding Horizons of Big Data9sight.com/pdfs/BigData_2016_Survey.pdf · 2016-11-22 · Charting the Expanding Horizons of Big Data 7 Hybrid Data Ecosystem Findings:

JAPANESE CANDLESTICK CHARTING TECHNIQUESforex-ofsite.ru/data/documents/S.Nison-Yaponskie-svechi... · 2020. 9. 18. · CANDLESTICK CHARTING TECHNIQUES A Contemporary Guide to the

Anthony Campitelli Joe Lang Flexo Mike Dudenhaver

FIRST CHARTING WEB FIRST CHARTING DESK FIRST CHARTING MOBILE APPLICATION PLATFORMS:

Excel Tutorial 4: Analyzing and Charting Financial Data

Charting a Security Landscape in the Clouds: Data ... · Charting a Security Landscape in the Clouds: Data Protection and Collaboration in Cloud Storage G. Itkis B.H. Kaiser J.E

LAB 2: Charting Worksheet Data - Mr. Brock's Computer … · · 2009-10-27LAB 2: Charting Worksheet Data ... includes different body and heading fonts. ... section are designed

Charting & Data Visualization in Ext JS 4

Dr. Guillermo Campitelli Prof. Craig Speelman School … · Dr. Guillermo Campitelli . Prof. Craig Speelman . School of Psychology and Social Science . Edith Cowan University