EEI Leading Indicator Study: Preliminary Resultsesafetyline.com/eei/conference s/EEIspring2013... · 2013-05-02 · • National Safety Council (NSC) • Nuclear Industrial Health

EDISON INTERNATIONAL®

SM

EEI Leading Indicator Study:

Preliminary Results

Larry Pena & Blakely Smith

Southern California Edison

Edison Electric Institute Occupational Safety & Health Committee Conference

Indianapolis, Indiana April 23, 2013


SM

Overview

• Industry safety metrics

• SCE Leading Indicator study

• EEI Leading Indicator Study

• Preliminary results

• Q&A


SM

Types of Indicators

•Quantifiable

•What was done

•Investigation

•Qualitative

•How well was it done

• Lagging Indicators

• Measures achievement

• OSHA

• DART

• Leading Indicators

• Detects Trends

• Measures effort

Predictive Experience

Objective Subjective


SM

Background

• Measurement of safety – Historically has been focused on lagging metrics (e.g., OSHA,

DART)

– Proactive metrics have been elusive

• Shifting industry focus – Proactive measures of the steps taken to reduce injuries

– Inclusion of leading indicators as a component of performance

indication

• Building the case for change – Determine the relationship to traditional indicators ( e.g. OSHA,

DART, LT, Severity)

– Determine best practices through EEI Leading Indicator Study


SM

Adapting Heinrich: Leading Indicators as

valuable clues

Lost Workdays

Recordable Injuries

Fatalities

Lagging Indicators

Leading Indicators


SM

SCE 2012 Preliminary Leading

Indicator Study


SM

7

Southern California Edison Safety Culture

Journey

Safety

Culture

Survey

Focus

Groups

and

Interviews

EIX Safety

Culture

Report &

‘Feedback

Days’

Assess

Safety

Culture

Safety

Culture

Initiatives

Injury-Free

Goal

2007 71%

Response

160 Groups

or

1,179

Employees

October

2007 –

June

2008

2011

80%

Response;

161 Groups or

1,208

Employees

2012 -

2013

Safety

Culture

Initiatives

Awareness

2008-2010

2005-2006

deaths,

serious

injuries,

PBR, etc.

Assess

Safety

Culture

2014

Safety Culture Assessment

Measure and Compare Improve Repeat


SM

External Benchmarking Methodology

• Electronic survey

– Forums

• EEI companies

• National Safety Council (NSC)

• Nuclear Industrial Health and Safety Association (NISHA)

• Phylmar Regulatory Roundtable

• Companies with safety culture efforts, e.g. Coors-Miller,

John Deer, General Mills, Shaw

• 10 questions (5 demographic, 4 leading indicator,

1 lessons learned)

– 57 respondents

• In-depth benchmarking with high-performing companies

(OSHA < 1.5; Approximately 10 companies)

8


SM

Leading Indicator Frequency of Use by

Company (in rank order)

Leading Indicator

% of

companies

use

1) Safety observations 80

2) Close calls 73

3) Corrective action plans completed on time 55

4) Safety awards and recognition 55

5) Safety training completed 53

6) Safety team effectiveness and attendance 46

7) Hazard resolutions 44

8) Number of tailboards 24

9) Preventative health account 13 9


SM

Leading Indicators that were Reported as

Most Effective (in rank order)

Leading Indicator

% of

companies

who chose as

most effective

1) Safety observations 100

2) Close calls 50

3) Corrective action plans completed on time 25

4) Hazard resolutions 25

5) Safety training completed 12.5

6) Safety team effectiveness and attendance 12.5

7) Safety rewards and recognition 12.5

8) Number of tailboards 12.5

9) Preventative health account 0 10


SM

In-Depth Benchmarking Best Practices

• Behavioral Safety Observations

– Built into performance management system

– Tracked in database

– Utilize observation card

– Metric used is frequency (count)

• Close Calls

– Anonymous – No name no blame

– Used as a tool to share lessons learned

– Metric used is CAP

• CAPS

– Various CAPs (Audit Findings, Observations, Ergo

Assessments, OSHA Citations) are entered into databases

maintained by corporate

11


SM

12

Leading Indicators by Safety

Performance for EEI participants

Companies

Safety

Obs Training

Hazard

Resolution

On

Time

CAPs

Awards Safety

Team

# of

Tailbo

ards

Near

Miss OSHA DART

Company A** X X X X X .86 .32

Company B** X X X X .67 .34

Company C** X X X X X 1.34 .38

Company D ** X X X X X X X X .92 .42

Company E** X X X X X X X X .92 .42

Company F *** X X X X X X .97 .48

Company G *** X X X 1.27 .59

Company H*** X X X X X X X 1.19 .61

**EEI 1st quartile ***EEI 2nd Quartile


SM

Does tracking matter?

Table 1: Number of Programs

Average DART

Low (0-2) 2

Moderate (3-5) 0.9222

High (6-9) 0.9582 13

• A significant difference in DART was found depending

on the total number of leading indicators a company is

tracking.

• Tracking 3-5 leading indicators is associated with lower

DART Rates

• f (1, 31)=13.87, p<.05 (ANOVA)


SM

Which Leading Indicators make a

difference?

• Independent comparisons

– Each leading indicator

– Grouping variable -- Tracked vs. Not tracked

• Average differences

– OSHA

– DART


SM

Testing the differences: OSHA

Indicator Average OSHA

Tracking Not Tracking

*CAPS 1.57 2.25

*Hazard

Resolutions

1.74 2.04

*Training 1.60 2.11

*Near Misses 1.69 2.28

*Recognition 2.07 1.73

Safety Teams 1.94 1.84

Safety

Observations

1.76 2.36

Tailboards 1.74 1.93

*= Significant

Difference, p< .05

No Significant

Difference, p> .05


SM

Testing the differences: DART

Significant

Difference, p< 05

Not Significant

Difference, p> 05

Indicator Average DART

Tracking Not Tracking

*CAPS .80 1.55

*Hazard

Resolutions

.96 1.30

*Training .81 1.26

*Near Misses .92 1.43

Safety Teams 1.08 1.15

Safety

Observations

.93 1.74

Tailboards .97 1.16

Recognition 1.10 1.14


SM

1st and 2nd Quartile EEI Companies

(Based on 2011 DART Rate)

Safety Observations

– 100 % of 1st and 2nd quartile companies are tracking

Close Calls

– 100 % of 1st and 2nd quartile companies are tracking

CAPs Completed on Time

– 100% of 1st quartile companies are tracking

– 71% of 2nd quartile companies are tracking

Training



17


SM

Recognition



Safety Teams



Frequency of Tailboards



1st and 2nd Quartile EEI Companies

(Based on 2011 DART Rate)


SM

EEI 2013 Leading Indicator Study


SM

EEI Leading Indicator Study Objectives

• Benchmark the industry

• Determine best practices

• Determine the relationship leading indicators

have with safety performance


SM

EEI Leading Indicator Project Team

• Chuck Kelly, EEI

• Paul Mackintire, Northeast Utilities

• Bruce Bolger, Northeast Utilities

• Ray Kelly, DTE Energy Co.

• Gary Hatcher, AEP

• Keith Williams, Intec

• Larry Pena, SCE

• Blakely Smith, SCE


SM

Methodology

• Survey launch: April 9, 2013 from EEI

– Platform: Survey Monkey

– Network: EEI membership

• 25 questions

– 12 demographic items including safety performance

– 5 items on 16 frequently reported leading indicators

– 7 items on general implementation of leading

indicators


SM

2013 Timeline

Major Milestones Jan Feb Mar April May Jun July Aug Sep Oct Nov Dec 14

Planning and Survey

Development

Survey Launch and

Data Collection

Survey Promotion

Survey Analysis

Report out of results

to membership


SM

Planned Analysis

• Descriptive data for each question

– Averages

– Rankings

– Frequencies

• Additional analyses

– Analysis of Variance (ANOVA)

– Comparisons of leading indicators to safety

performance data

– Content analysis of comments and text

**Please note: Data will be reported in aggregate form.


SM

Preliminary Results


SM

Preliminary Results as of 4/22/13:

Demographics

• Number of responses : 11

• Industry types:

– T&D: 100%

– Wind: 44%

– Gas: 66%

– Hydro: 44%

– Coal: 44%

– Nuclear: 22%


SM


Most frequently tracked leading

indicators

• Safety Observations

• Near Misses

• Safety Awards and Recognition

• Safety Training

• Work Readiness

• Safety Culture Perception


SM

• Safety Observations

• Leadership Engagement

• Near Misses


Leading indicators reported as most

effective


SM

• Upper management only: 44%

• All management: 55%

• All employees: 33%


Who receives communication about

leading indicators?


SM

• Corporate Level: 55%

• Individual Organizational Unit Level: 66%

• Local Site Level: 66%


At what level are you reporting?


SM

Accessing the Survey

http://www.surveymonkey.com/s/EEILeading

Indicators

http://www.surveymonkey.com/s/EEILeadingIndicators

http://www.surveymonkey.com/s/EEILeadingIndicators


SM

Questions?

Documents

EEI Leading Indicator Study: Preliminary Resultsesafetyline.com/eei/conference s/EEIspring2013... · 2013-05-02 · • National Safety Council (NSC) • Nuclear Industrial Health