2012 CANSO Asia Pacific Safety Seminar APAC Conference... · 2012 CANSO Asia Pacific Safety Seminar Welcome Maggie Geraghty Safety Program Manager CANSO

2012 CANSO Asia Pacific Safety Seminar

Welcome

Maggie Geraghty Safety Program Manager

CANSO

2011 APAC Safety Seminar Survey Results

Survey Results from the 2011 APAC Safety Seminar in Bangkok were

used to:

Build the 2012 Agenda

Support Global Efforts

2011 feedback included requests for more information and discussion

on:

Runway safety; excursions, incursions; safety level during approach

phase;

Just Culture; discussion from ANSPs that have experience implementing

a just culture; a CEO’s perspective of safety culture

Safety management; safety performance measurement; hazard tracking

SMS Implementation

Claire Marrison Manager Safety System Risk &

Analysis Airservices Australia

Overview of session

• Introduce the type of support which is available to assist those implement SMS

• Current Program

– Evolution Guide

– Annex 19

CANSO Standard

• Does not seek to over ride existing regulations

– Consistent and Complimentary

• Not based on single statements

– Continual Improvement Path

– Guidance on what to work on first

• Provides an industry standard

CANSO Standard of Excellence in SMS

Positioning of CANSO Standard

International Civil Aviation Safety Organisation

Procedures for Air Navigation

Services, Document 4444, ATM 501

Chapter 2: ATS Safety Management

Safety Management Manual

Document 9859

Domestic regulator

requirements

ANSP SMS

Requirement

Guidance &

Information at

aviation

industry level

Requirement

CANSO

CANSO Standard of Excellence

in SMSImplementation Guide

Guidance on ANSP

best practice

Guidance on how other

ANSPs have successfully

implemented their SMS

Phased Approach to Implementation

Phase Group Element

1 Safety Policy Safety Policy

Organizational and Individual Safety Responsibilities

2

Safety Assurance Safety Reporting, Investigation and Improvement

Safety Achievement Competency

Safety Culture Development of a positive and proactive safety culture

Safety Achievement Safety Standards and Procedures

Safety Promotion Adoption and Sharing of Best Practises within the ANSP

3

Safety Achievement Risk Management

Safety Assurance Safety Performance Monitoring

Safety Achievement Safety Interfaces

4

Safety Assurance Operational Safety Surveys and SMS Audits

Safety Promotion Adoption and Sharing of Best practises with external stakeholders

Safety Culture Safety Culture

Safety Policy Timely Compliance with International Obligations

Maturity Levels

Example: Safety Performance Monitoring

Objective

Initiating Planning / Initial Implementation

Implementing Managing & Measuring Continuous Improvement

9.1 An established and active monitoring system that uses suitable safety indicators and associated targets (e.g., lagging and leading indicators)

There are no indicators, thresholds, or formal monitoring system in place to measure safety achievements and trends.

There is a plan to implement a monitoring system. A limited set of indicators has been implemented.

The safety monitoring system has been implemented and documented. Indicators and targets have been set.

The monitoring system is used for safety improvement. All indicators are tracked against thresholds/targets on a regular basis. Trends are analysed for safety improvement purposes.

There are comprehensive metrics in place to measure and monitor indicators and thresholds throughout the system.

9.2 Systematic measurement of safety indicators (both leading and lagging)

Some ad-hoc safety indicators are used.

A plan is in place to introduce a formal and comprehensive set of safety indicators.

A formalised system limited to indicators defined by regulatory requirements is in place.

Additional indicators are also defined and monitored to meet both organizational and local safety objectives.

Safety indicators covering all aspects of the system/operations are mature and used to measure safety improvement.

9.3 Methods to measure safety performance, which is compared within and between ANSPs

Ad-hoc safety performance data related to individual incidents is available, but there is no systematic approach for measuring safety performance.

The implementation of some qualitative and quantitative techniques in certain parts of the organization has started. However, there is insufficient data to analyse.

Qualitative techniques are in place, and the implementation of quantitative techniques has started.

Safety performance is measured using statistical and other quantitative techniques. Internal comparative analysis is done, and external comparative analysis has begun.

The reporting, safety survey, and auditing programmes are integral parts of the management and operational processes. Results are used to optimise performance and to drive further safety improvements across all organization. Internal and external comparative analysis is well-established.

International Benchmarking

• Annual benchmarking exercise

• External agency

• Questionnaire and follow-up interview

• Allows ANSPs: – to track improvements over time

– Benchmark performance against other ANSP

• Allows CANSO to identify issues – Programs to assist

Benchmarking Results 2011

I; 2

J;

1

V;

1

N;

1

C4;

1

L;

1

D2;

1

E;

10

A6;

9

D1;

6

D3;

4

C9;

3

M;

5

C3;

1

T;

4

K;

3

S;

3

D;

1

Q;

1

U;

11

B3;

12

B7;

11

A3;

6

C7;

5

B8;

5

1

2

3

4

65 75 85

Overall Maturity Score (detail)

Le

ve

l

Implementation Guide

• Support CANSO Standard

• Addresses the ‘how’ to implement each element – Enablers

– Process

– Outcomes

– Examples

• Based on experiences of a number of major ANSPs – What they do

– What would they change if they started again

Evolution Guides

• Concerns within the ANSP community – Process and approach not producing ‘right’ results

– Systems need to evolve to keep pace with operational developments

• Safety Standing Committee want focus on: – Risk Management

– Safety by Design

– Fatigue Risk Management

– Safety Performance Metrics

– Safety Assurance Techniques

• Guides are being developed jointly by a number of ANSP – Techniques will be evaluated

ICAO: Annex 19 Safety Management

• Consolidate all Safety Management requirements in annexes

• No requirements could be eliminated

• Air Navigation Commission reviewed Annex in May

• Council endorsement to be advised

• Annex will have a supporting manual

ICAO Annex 19: SMS Framework

SAFETY POLICY AND OBJECTIVES

Management commitment and responsibility

Safety accountabilities

Appointment of key safety personnel

Coordination of Emergency Response Plan

Safety Management System Documentation

SAFETY RISK MANAGEMENT

Hazard identification

Safety risk assessment and mitigation

SAFETY ASSURANCE

Safety Performance Monitoring and Measurement

The Management of Change

Continual improvement of the SMS

SAFETY PROMOTION

Safety Communication

Training and Education

Revisions to CANSO Standard

SAFETY CULTURE

Development of a positive and proactive safety culture

SAFETY POLICY AND OBJECTIVES

Safety Policy

Organisational and individual safety responsibilities

Compliance with international obligations

Coordination of Emergency Response Plan Safety Management System Documentation

SAFETY RISK MANAGEMENT

Risk Management Process

SAFETY ASSURANCE

Safety Performance Monitoring and Measuring

The Management of Change

Safety Reporting Investigation and Improvement

Operational Safety Surveys and SMS Audits

Continual improvement of the SMS

SAFETY PROMOTION

Safety Communication

Training and Education

SAFETY ACHIEVEMENT

Safety Interfaces

Safety by Design

Fatigue-related Risk Management

SMS Outputs by November 2012

• Revised CANSO Standard of Excellence

• Two draft evolution guides – Risk Management

– Safety by Design

CANSO SMS Publications

Just Culture

Stephen Angus

•What is it?

•What isn’t

it?

•Important?

Just Culture – What?

•Just Culture

- Important

ingredients



• Behaviours

• Processes

• Procedures

• Reporting

• Feedback

• Learning

• Values

• Policy

• Principles

• Practices

• Attitudes

• Intent

Just Culture – a definition

Just Culture: NATS Definition

“A culture where staff are not punished for actions, omissions or

decisions taken by them that are commensurate with their

experience and training, but where gross negligence, wilful

violations and destructive acts are not tolerated”.

Slide 27

• 5 reasons why

1.? 2.? 3.? 4.? 5.?

Key element – The value of reporting

Just Culture

JUST CULTURE

↑Leads to↑

OPEN REPORTING

↑ Leads to ↑

LEARNING

↑ Changes ↑

PERCEPTION OF RISK

↑ Changes ↑

ATTITUDE TO SAFETY

↑ Changes ↑

SAFETY BEHAVIOUR

Slide 29

Just Culture – The barriers/ The issues

• Understanding

• Politics

• Laws

• Leadership

• Commitment

• Expectations

• Acceptance

• Perceptions

• 3 ways to improve Just Culture

1.? 2.? 3.?

Just Culture – How?

Just Culture – Next Steps?

• Action I will take!

1

Just Culture

‘A journey begins with

a single step’

ATM Investigations

Dr. David Harrison Safety Director

NATS

The NATS Investigation process

Electronic secure reporting from any network PC into the NATS Safety

database (STAR).

This includes direct submission to the Regulator. This

supports Just Culture.

7500 Operational Safety Reports raised each year.

2000 Engineering Safety Reports

All safety reports are investigated to extent necessary.

All investigations must result in an investigation report, which is

distributed to the Regulator, Unit Management and the Staff reporting

the event.

The investigation report may be a full analysis requiring up to 20 pages

with radar stills, jpegs and photographs

or it may be one line of text.

The NATS Investigation Process

Initial Investigation of the event is by the ATC/Engineering

Supervisor.

This includes securing media, data gathering and discussion of event

with all staff involved.

Local Investigators carry out investigation. Monitored and assisted

centrally.

Validation of conclusions and risk assessment centrally.

Tracking of recommendations and actions – local ownership, central

checking.

Investigation – External Liaison

Flight Safety Information shared with Customers, Ministry of Defence

and Other ANSPs via confidentiality agreements.

(subject to Just Culture and Data Protection requirements)

UK Airprox Board

Air Accident Investigation Branch

CHIRP

Safety Partnership Agreement (SPA)

Why do we carry out Incident Investigations?

To establish WHAT happened

To establish HOW and WHY it happened

To enable CHANGE

To consider and act upon all events and issues regardless of

immediate severity to prevent future incidents

The Art of Investigation – ATM Contributory Factors

Every safety report raised will have at least one ATM Contributory

factor assigned to it.

Some reports may have up to 20 ATM factors listed.

This enables trend analysis and Safety Improvement.

ATM Contributory Factors

In addition to the contributory Factors, NATS includes Mitigation.

A method of formally recording “What went well” during the event.

It helps to tell the whole story and provides a mechanism to

acknowledge the good actions of individuals as well as the

errors/slips and misjudgements.

Good Recommendations

Clear

Targeted to an individual / post-holder

Time-bound

Appropriate to the severity and repeatability of the event

Focussed on demonstrable change

Must have Closure Criteria

Training and Competency for Investigators

All investigation staff must be trained, appropriate to the task. There is

a requirement for investigators to pass an in-house 4 day

investigation course.

Competency is on continuous assessment basis.

Investigation reports are monitored centrally and feedback or coaching

is given to investigators.

A minimum standard must be obtained for at least 5 reports a year.

Q & A ?

Data Analysis

Dr. David Harrison Safety Director

NATS

Contents

Safety data sources

NATS safety reporting

NATS Measuring safety

Using safety data to understand and manage risk

Infringements

Level busts

Runway Incursions

Conclusions

Safety Data Sources

Event Data

SSEs MORs Obs

System Data

SMF CAIT RT

TCAS STCA

Survey Data

Scratch pad trials Questionnaires

Observation & Interaction

Data

Day to Day Safety Culture

Safety Data Pyramid

MORs

Open Reporting

Other data sources

More data

Greater ability to

manage potential

safety risks

proactively

Airprox and Safety Significant Events

STAR* incident data

*NATS’ Safety Tracking and Reporting database

Safety Pyramid – leading and lagging indicators

There are two main types of Safety Reports in NATS, MORS and Observations.

Mandatory Occurrence Reporting Scheme (MORS) CAP 382 is a legal requirement. An MOR can be defined as “any incident which endangers or which, if not corrected, would endanger an aircraft, its occupants or any other person”

Observations or ‘open reports’ can be any safety event or issue that is outside the scope of CAP382, but that needs to be brought to the attention of Unit Management. Reporting of safety observations is encouraged as part of a strong safety culture.

NATS Safety Reporting

Type of Events by Year

Type of event 2006 2007 2008 2009 2010 2011

Observations (open reports) 1,761 2,648 2,697 2,432 2,230 2,049

MORs 4,429 4,554 4,418 4,542 4,479 5,119

Loss of Separation 352 318 297 290 263 265

Total 6,190 7,202 7,115 6,974 6,709 7,168

NATS

NATS has two main measuring safety Airprox – an independent measure by the UK Airprox

Board Safety Significant Events (SSEs) – NATS’ own measure

NATS Measuring Safety

NATS Airprox where NATS was providing a service

0

10

20

30

40

50

60

70

80

90

100

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

Nu

mb

er

of

Air

pro

x

NATS Attributable, Risk Bearing NATS Attributable All

Level of Control <=50% >50% and <=66% >66% and <100% >=100%

Providence SSE1a SSE1b SSE1c SSE1d

Pilot SSE2a SSE2b SSE2c SSE2d

ATC but not effectively SSE3a SSE3b SSE3c SSE3d

ATC SSE4a SSE4b SSE4c SSE4d

Standard Separation

NATS SSE – Barrier Model

111

49

168

97

159

101 99

146

50

126

51

114

0

20

40

60

80

100

120

140

160

180

Nu

mb

er

of

Eve

nts

Total SSE1-4a/b 97 168 101 159 99 146 50 126 51 114 49 111

SSE4a/b 37 121 44 108 43 111 25 91 24 94 18 93

SSE3a/b 39 25 40 30 40 22 20 24 18 12 21 10

SSE2a/b 21 15 16 16 15 9 5 6 7 7 9 7

SSE1a/b 0 7 1 5 1 4 0 5 2 1 1 1

NATS/Dual Non-NATS NATS/Dual Non-NATS NATS/Dual Non-NATS NATS/Dual Non-NATS NATS/Dual Non-NATS NATS/Dual Non-NATS

2006 2007 2008 2009 2010 2011

NATS SSE1-4 a/b events by attribution

NATS SSE1-4 a/b by Event Type

2006 SSE1-4a/b Events

76, 29%

35, 13%

140, 53%

14, 5%

Infringements Level Busts Runway Incursions Other

2011 SSE1-4a/b Events

55, 34%

11, 7%

79, 50%

15, 9%


2006 2011

Total number of SSE1-4a/b events = 265

% of SSE1-4a/b events assigned to Infringements, Level Busts or Runway Incursions = 47%

Total number of SSE1-4a/b events = 160

% of SSE1-4a/b events assigned to Infringements, Level Busts or Runway Incursions = 50%

Weighting SSE Events

Standard Separation

Level Of Control

<=1/2 >1/2 and <=2/3rds

>2/3rds and <100%

>=100%

Providence Weighted SSE Index

SSE1a 33.9

SSE1b 33.9

SSE1c SSE1d

Pilot Weighted SSE Index

SSE2a 5.3

SSE2b 5.3

SSE2c SSE2d

ATC but not effectively

Weighted SSE Index

SSE3a 1.7

SSE3b 1.7

SSE3c SSE3d

ATC Weighted SSE Index

SSE4a 1

SSE4b 1

SSE4c SSE4d

NATS SSE1-4 a/b weightings applied

2006 2011

Total number of weighted SSE index points = 730

% of weighted SSE points assigned to Infringements, Level Busts or Runway Incursions = 56%

Total number of weighted SSE points = 329

% of SSE points assigned to Infringements, Level Busts or Runway Incursions = 51%

2006 Weighted SSE Index

242, 33%

119, 16%

317, 44%

52, 7%


2011 Weighted SSE Index

71, 22%

35, 11%163, 49%

60, 18%


Causal Factors associated with SSE1-4 a/b events 2006

0 10 20 30 40 50 60 70 80

Unauthorised entry into regulated airspace

Civil uncontrolled aircraft

Incorrect decision / plan

Correct pilot readback followed by incorrect action

Not see

Misjudge / mis-project

Failure to follow ATC instruction

Omission of action

Distraction - job related

Mis-perceive visual information

Controller/Pilot under training

Mis-Hear

No detection of information

Insufficient decision / plan

Entered runway without clearance

Inadequate Mentoring

Forgot previous action

Military Controlled Aircraft

Forgot to check or monitor

Forget to perform action

Causal Factors associated with SSE1-4 a/b events 2011

0 10 20 30 40 50 60 70 80

Unauthorised entry into regulated airspace

Civil uncontrolled aircraft

Not see

Incorrect decision / plan

Distraction - job related

Correct pilot readback followed by incorrect action

Misjudge / mis-project

Mis-perceive visual information

Inaccurate / no recall from working memory

Insufficient decision / plan

Convey / record no information

Failure to follow ATC procedure

Responded to TCAS/GPWS

Conflict outside of controlled airspace

Mis-perceive auditory information

Entered runway without clearance

Forgot to check or monitor

Coordination Issues

Isolated individual non-conformance

2011

Top Causal Factors for SSE1-4a/b Infringements Total

Unauthorised entry into regulated airspace 55

Civil uncontrolled aircraft 34

Controller/Pilot under training 3

Not see 3

Convey / record incorrect information 2

Incorrect pre-flight briefing 2

Mis-perceive visual information 2

Unauthorised entry into restricted airspace 2

Airspace design issues 1

Altimeter setting error 1

Correct pilot readback followed by incorrect action 1

CRM issues 1

Distraction - job related 1

Failure to follow ATC procedure 1

Forgot to check or monitor 1

Inaccurate recall from long term memory 1

Incorrect decision / plan 1

Insufficient decision / plan 1

Isolated individual non-conformance 1

Links between SSE Events and Causal Factors

2011

Top Causal Factors for SSE1-4a/b Level Busts Total

Responded to TCAS/GPWS 3



Excessive workload 2

Altimeter setting error 1

Bandboxing or splitting of sectors 1

Convey / record no information 1

CRM issues 1

Forgot planned action 1

Incorrect pilot readback by correct aircraft 1

Military ATS 1

Military Controlled Aircraft 1

Mis-see 1

Poor FMS Handling 1

Poor manual handling 1

Rate of turn/climb/descent 1

Surveillance problems 1

Unusual or unfamiliar task 1

Windshear / downdraft 1

Infringements Level Busts

Links between SSE Events and Causal Factors

Runway Incursions

2011

Top Causal Factors by Year Total

Entered runway without clearance 8

Not see 7

Convey / record no information 4


Airport Authority Airside Operations 3


Incorrect decision / plan 3

Procedures - Airport 3

Airfield ground lighting 2

Experience issues 2

Airfield layout 1

Airport Emergency Services 1

ASMGCS corruption 1

Convey / record incomplete information 1

Convey / record incorrect information 1

Failure to follow ATC procedure 1

Forgot to check or monitor 1

Glare 1

Ground services operator 1

Reducing Operational Risk – Airspace Infringements

2006 % Weighted SSE Index and Number of Points Assigned

to

Airspace Infringements

488, 67%

242, 33%

Infringement Other


to

Airspace Infringements

71, 22%

258, 78%

Infringements Other

Total weighted SSE Index points = 730

Weighted SSE index points assigned to airspace infringements = 242


Weighted SSE index points assigned to airspace infringements = 71

2006 2011

Environment Introduction of LARS – Lower Airspace Radar Advisory Service Establishment of the Stansted Transponder Mandatory Zone Tools Deployment of Controlled Airspace Infringement Tool (CAIT) Initial deployment of Monitoring Codes in selected areas of airspace Deployment of the AWARE Airspace Warning Device Launch of Sky Demon, pre flight planning tool Capability Publication of London CTR Heliroute Video Guides Deployment of Infringement Investigation Questionnaire to infringing pilots. Development and initial delivery of an Airspace Awareness module as part of Flight Instructor renewal seminars. Publication of Flying Around the LTMA Video Guides

Reducing Operational Safety Risk - Infringements

Reducing Operational Risk – Level Busts


Weighted SSE index points assigned to Level Busts = 119


Weighted SSE index points assigned to Level Busts = 35


to

Level Busts

35, 11%

294, 89%

Level Busts Other

2006 2011


to

Level Busts

610, 84%

119, 16%

Level Busts Other

Environment Improved SID charts clearly depicting steps in SIDS to reduce inadvertently missing intermediate levels Tools Introduction of Barometric Pressure Setting Advisory Tool at Swanwick LTC Introduction of Electronic Flight Data with Cleared Flight Level /Selected Flight Level at Prestwick Centre. Introduction of MODE “S” at both centres Capability Airline performance league tables

Reducing Operational Safety Risk – Level Busts

Reducing Operational Risk – Runway Incursions


Weighted SSE index points assigned to Runway Incursions = 52


Weighted SSE index points assigned to Runway Incursions = 60

2006 2011


to

Runway Incursions

52, 7%

678, 93%

Runway Incursions Other


to

Runway Incursions

268, 82%

60, 18%

Runway Incursions Other

Environment 24 Hour Runway Stop Bars. This is a proven and highly-effective mitigation against runway incursions. From having only one airport operating in this way in 2006, today H24 stop bars are in use at half of NATS airports with others planning implementation. Tools NATS has worked with equipment suppliers, regulators, other ANSPs and airlines to begin to define a framework for the use of technology in runway safety. Capability Lights On Policy – through the NATS SPA and IATA a policy was agreed for aircrews to operate aircraft lights to improve the visibility of the aircraft when operating on or near runways.

Reducing Operational Safety Risk – Runway Incursions

Conclusions

Safety data is available from a range of sources. Over the last 6 years using the UK independent measure and NATS own measure of Safety performance has improved. Safety improvements have been made by utilising the available data, focusing on the key risk areas and employing targeted initiatives.

Any Questions?

Back-up

Role of Directorate of Safety

NATS Directorate of Safety is responsible for providing independent

assurance to the Chief Executive and Board on the safety of all NATS’

operations.

We also provide constructive challenge to the pace and rigour of risk

mitigation and safety improvement activities, including expertise and

support to all areas of NATS on Safety and Human Factors.

Organisation of Investigation in NATS

Safety Director NATS

↓

Head of Safety Performance & Improvement, NATS

↓

Head of Safety Investigation, NATS

↓ ↓

NATS Airspace NATS Airports

↓

↓ Manager Investigations

↓

2 ACCs 16 Airport Units

10 Full Time Investigators 1 Full Time Investigator

50 P/t Investigators

Roles and Responsibilities

Head of Safety Investigation

Sets policy, strategy and standards

Training, Coaching and Competency of Investigators

Oversight of current investigations

Ensures investigation tools and procedures are fit for purpose and

compliant with regulatory requirements

ATM Contributory Factors – Mitigation

Mitigating factors include:

Good Planning and Tactical Decisions

Good Teamwork

Good Defensive Controlling

Good Resolution Action

Exceptional Personal Contribution

The Art of Investigation when the investigator understands

What does he/she want the future to be?

What change would do that?

Is it do-able? (not the investigator’s problem)

Write recommendations to achieve it.

Good Recommendations

Clear

Targeted to an individual / post-holder

Time-bound

Appropriate to the severity and repeatability of the event

Focussed on demonstrable change

Must have Closure Criteria

Recommendations and Actions that follow are tracked to closure

Validation of the Investigation

Need to demonstrate that recommendations and action has been

completed. Not just “Action complete” statement.

Documents, plans, meeting notes must be linked to the investigation

The NATS Safety Database STAR

To be replaced in 2013 by new safety database, specification in

writing, to be platformed on Microsoft Sharepoint.

This new safety database will be available either as a stand-alone

configuration or as a NATS managed-service.

Challenges for Investigations

Just Culture. Balance of lesson learning responsibility and

confidentiality

How do we know that the recommendations produced the right

outcomes?

Cross-industry trust and partnerships

Voluntary reporting culture – Lots of Information, but how much

knowledge? Matching resources and the expectations of both

reporters and company.

Balance of carrying out in-depth investigations into things that have

gone seriously wrong – but are gone; against putting resource into

precursor information.

Future Risk Models – Global ANSPs & Pan industry.

Future Data Sharing – Global ANSPs & Pan industry.

The Art of Investigation

What Happened? – this is usually easy

Why did it happen? – nobody sets out to have an accident or make a

mistake. With hindsight YOU can see it coming. So why didn’t they?

What made sense to the people at the time?

If you don’t know this – you cannot fix the problem!

The Art of Investigation

Find out WHY people did what they did.

What did they think about doing and not do?

Why did systems interact in that way?

Keep asking WHY?

ATM Contributory Factors

The NATS set of Causal Factors has been adapted slightly to become

the Eurocontrol ATM Contributory Factors.

The EU is proposing to mandate this list for use by all ANSPs from

2015.

It is also the set of Causal Factors being adopted by the FAA.

CANSO, under discussion.

Safety Assessment

NATS has 4 Safety Assessment schemes:

En-Route

Airport

Oceanic Environments

Engineering

All to be replaced by the European Risk Analysis Tool (RAT)

mandated to be operational across Europe from 2015