THE LIVERPOOL SYSTEM - CLASSIFICATION, LEARNING & PREVENTION Incident Reporting and Learning: Anthony Arnold Director Cancer Services, Illawarra Shoalhaven

THE LIVERPOOL SYSTEM - CLASSIFICATION, LEARNING & PREVENTION

Incident Reporting and Learning:

Anthony ArnoldDirector Cancer Services, Illawarra Shoalhaven Local Health [email protected]

--- ROSIS Melbourne Australia 2012 ---

Context

2


The Liverpool System

3

Ref: IJROBP 2010 Volume 78, No 5, Pages 1548-1554


A Problem Worth Solving……

Complexity of radiation oncology At the time no system of analysis was

in place Lack of clinical governance surrounding

reporting There was limited openness about

reporting events The culture was predominantly blame

based Standard reporting systems are

ineffective for radiation oncology 4


ClassificationPrescription

Simulation

Computing / Dosimetry

Pre-Tmt

TreatmentImaging

Bolus

Shielding

Documentation

5

6

Incorrect Tmt Site Prescribed Incorrect Energy Prescribed Incorrect Dose or Dose per Fraction

Bolus not Prescribed when Req’d Shielding not Prescribed when Req’d Prescription not signed by Rad Onc

Other: Prescription Related Incorrect Site Scanned / Simulated Insufficient Scan Area Applied

Incorrect CT Procedure Applied Laser Shift / Related Error Sim Film Marked Incorrectly

Simulation Tattooing Related Error Inappropriate Pt Positioning Used Volume / Voluming Related Error

Landmarking Related Error / Omission Contrast Related Error Other: Simulation Related

Incorrect CT-Density Conversion Used Incorrect Weight/Calc/Dose Point Used Incorrect Normalisation Applied

Other: Computer Planning Error Incorrect MU Calculation Incorrect Detail in R+V System

Tray / Wedge Code Missing / Error Attenuation Factor Missing / Error QA Check/s Not Completed / Error

Treatment Sheet Annotation Error TLD Related Error / Omission Pacemaker Related Error / Omission

Other: Pre-Treatment Related Geographic Miss: Incorrect Site Geographic Miss: FSD/SSD Error

Geographic Miss: Incorrect Tattoo Geographic Miss: Landmarking Error Geographic Miss: Incorrect Fld Used

Geographic Miss: Field Size Error Patient Not Treated When Required Field Not Treated When Required

Incorrect Wedge / Wedge Orientn Incorrect Energy Delivered Changes / amendments not in R+V

Immobilisation Device Error History / Chart Check Missed / Error Other: Treatment Related

Bolus not Applied When Required Bolus Applied to Incorrect Site Incorrect Bolus Material Used

Incorrect Bolus Thickness Used Other: Bolus Related Shielding Not Applied When Req’d

Incorrect Shielding / Cut-out Used Shielding Applied to Incorrect Area Shielding Mounting / Tray Error

MLC Pattern / Related Error MLC File Missing / Not Attached to Fld MLC Checks Missed / Not Done

Other: Shielding Related Image Not taken When Specified Film / EPI Labelled Incorrectly

Image Not Reviewed When Required Iso Adjustments Applied Incorrectly Iso Shifts Not Applied When Req’d

Other: On-line / Off-line Related Documentation Error: Sim Documentation Error: Tmt Sheet

Documentation Error: R+V System Documentation Error: Imaging Documentation Error: Other

Prescription

Simulation

Computing

Pre-Treatment

Treatment

Bolus

Shielding / MLC

Imaging

Documentation


Classification Advantage

7


Error / Event Definitions

Event: “event or circumstance which could have

resulted, or did result in harm to a patient”

Actual Error: “Error resulting in radiation exposure other

than that intended or prescribed – correctable or otherwise”

Near Miss: “Error or non-conformance detected

before reaching the patient”

8


High Level Structure…….PDSA

PLAN

•Classification designed, database constructed, education

•System implementation, clinical leadership and support

DO

•Staff encouraged to report all events irrespective of magnitude

•Supporting governance, openness, process based

STUDY

•Summary reports analysed monthly across various forums

•Trend patterns analysed to highlight areas / systems in need

ACT

•The data itself was used to focus QA and improvement activity

•Focussed education, workflow redesign, protocol changes

9


Reporting and Managing an Event

Detect

•Staff detecting initiates report (narratives, tells story)

•Manage patient and situation, immediate actions

Review

•Team review, contributing factors, further actions

•Agree on report as a team

Share

•Reverse back through other staff and depts involved

•Learning, prevention, further analysis, additional factors

Manage

•Review and classifiy, explore issues, system breaks

•Consider recommendations, initiate change / improvements

10


Department AnalysisLiverpool

Macarthur CTC (2004-2007)

Illawarra CCC(2006-2009)

4-5 linear acceleratorsSuperficial / orthovoltage

BrachytherapyWidespread conformal

3DCRTIMRT on horizon

Large metropolitan centre

688 reports / 3925 courses

2 linear acceleratorsSuperficial / orthovoltage

No brachytherapyWidespread conformal

3DCRTIMRT widespread clinical use

Small semi-regional centre

670 reports / 3645 courses11


Results - Initial Pilot

688 reports were logged during the study period 155 Actual errors (23%) 533 Near Miss (77%)

Attendances

Actual Error Near Miss

Analysis of 1st 3 years of operation (May 2004-May 2007)

12


Results - Subsequent Pilot

670 reports were logged during the study period 67 Actual errors (10%) 603 Near Miss (90%)

Attendances

Actual Error

Near Miss

Analysis of 1st 4 years of operation (2006-2009)

13


Time Trends Statistics

Actual Error

Near Miss

Total Errors

No. of Attendance

s

% Actual Error

p-value for

Actual Error Rate

Difference

% Near Miss

p-value for Near

Miss Rate

Difference

% Total Even

ts

p-value for Total

Errors Rate

Difference

Year 1 63 184 247 21788 0.3% 0.8% 1.1%

Year 2 58 199 257 38134 0.2% p<0.01 0.5% p<0.000

1 0.7% p<0.0001

Year 3 34 150 184 55006 0.1% p<0.001 0.3% p<0.000

1 0.3% p<0.0001

Ref: Simple Interactive Statistical Analysis online statistical calculator. Available at: http://www.quantitativeskills.com/sisa/statistics/t-test.htm. Accessed 29 January 2008 14


Time Trends Statistics

Actual Error

Near Miss

Total Errors

No. of Attendance

s

% Actual Error

p-value for

Actual Error Rate

Difference

% Near Miss

p-value for Near

Miss Rate

Difference

% Total Even

ts

p-value for Total

Errors Rate

Difference

Year 1 16 145 161 6221 0.26% - 2.33

% - 2.59% -

Year 2 12 173 185 15687 0.08% 0.0016* 1.10

%p<0.000

1*1.18

% p<0.0001*

Year 3 27 128 155 17028 0.16% 0.1695* 0.75

%p<0.000

1*0.91

% p<0.0001*

Year 4 12 157 169 15582 0.08% 0.0017* 1.01

%p<0.000

1*1.08

% p<0.0001*Ref: Simple Interactive Statistical Analysis online statistical calculator. Available at: http://www.quantitativeskills.com/sisa/statistics/t-test.htm. Accessed 29 January 2008 15

Time Trends 1st Pilot: Attendances

04/05 05/06 06/07

No. of Attendances 21788 38134 55006

Actual Error 63 58 34

Near Miss 184 199 150

5000

15000

25000

35000

45000

55000

25

75

125

175

225

275

325

375

425

475

Att

en

dan

ces

Inci

den

ts

16

Time Trends – 2nd Pilot: Courses

2006 2007 2008 2009

Total Courses 894 901 946 904

Actual Errors 47 12 27 12

Near Misses 427 173 128 157

50150250350450550650750850950

50

150

250

350

450

550

Pati

en

t C

ou

rses o

f Tre

atm

en

t

Rep

ort

ed

Even

t C

ou

nt

17


Study Time period

Actual Error Rate per

treatment episode

Total Error Rate per

treatment episode†

Reporting scope

CommentsSimulation Prescription Planning

Treatment delivery

Our study

2004-05 0.3% 1.1%

2005-06 0.2% 0.7%

2006-07 0.1% 0.3%

Macklis et al.[17] 1995

0.2% per treatment

fieldNR x

Block errors most

common

Fraass et al.[24] 96-97 0.4% NR x x x

Treatment set-up and treatment accessory

errors most common

Huang et al.[22] 97-02 0.3% NR x x x

Tight parameters

for error. Treatment

field errors of >0.5cm the

most common.

Calandrino et al.[19] 91-96

0.45% per treatment

course

3.5% per treatment

coursex x x

MU calculations

only

Barthelem-Brichant et

al.[27]

NR 3.5% NR x x x Patton et al.

[6] 99-00 0.2% NR x x x

Swann-D’Emilia[25] 89-90

0.17% per treatment

fieldNR x x x

Most errors were due to

errors in block

placement18


Outcomes – Key Measures

Reduction in Errors

Reduction in Error

Rate

Improved Patient Safety

19


Patient Safety Risk Improvement REDUCTION IN REPORTED EVENTS as a function of

attendances Actual Error rate reduced from 0.26% to 0.08%

(p=0.0017) Near Miss rate reduced from 2.33% to 1.01% (p<0.0001)

IMPROVED RELATIVE PATIENT SAFETY RISK per treatment course

Actual error rate reduced from 1 in 19 courses to 1 in 75 courses; in other words from 5% down to 1.3% risk of detectable error (p=0.0003)

Near miss rate reduced from 1 in 2 courses to 1 in 6 courses; in other words from 50% down to 17% (p<0.0001)

20

THANK YOU

Documents

THE LIVERPOOL SYSTEM - CLASSIFICATION, LEARNING & PREVENTION Incident Reporting and Learning: Anthony Arnold Director Cancer Services, Illawarra Shoalhaven