Embed Size (px)
DESCRIPTION
Overview A Performance-based Approach to Safety Monitoring Safety Performance Performance Measurement – the essential tool Measuring Safety Maturity Performance-based Approach to Managing Risk Risk management – part of SMS Safety oversight aspects – the role of ESARR 1 Risk classification methodology – defining tolerable safety Conclusions
Citation preview
Peter StastnyHead of Safety Regulation [email protected]
ICAO Montreal, Thursday 29 March 2007
Ensuring the Safety of Future Developments
Overview A Performance-based Approach to Safety
Monitoring Safety Performance
Performance Measurement – the essential tool
Measuring Safety Maturity
Performance-based Approach to Managing Risk
Risk management – part of SMS
Safety oversight aspects – the role of ESARR 1
Risk classification methodology – defining tolerable safety
Conclusions
Risk Assessment New Systems
Recruitment/Selection
Training
CompetencyChecks
Refresher/Advanced Training
Procedures Operational Processes
Interface ATS CNS, AIM, Airports
CNS /AIMMaintenance Procedures
Emergency Procedures
Risk Assessment ATM Procedures
Risk Assessment Airspace Changes
Risk Assessment Software Changes
Incident Reporting
Incident Investigation
Lessons Learnt
Safety Surveys
and Follow-up
A systematic approach to the A systematic approach to the management of safetymanagement of safety
Proactive Reactive
“Historic” Safety Performance Measurement
Performance – part of SMS
Safety Maturity Measurement
Performance – part of Safety Oversight
ESARR 1….
Defines minimum arrangements/ processes for ATM safety oversight:
with certification
or without certification
A unique basis for harmonising and reinforcing the role and operation of national regulatory bodies
Requires monitoring of safety performance as part of safety oversight
II N TN T HH E FE F UU
TT UU RR E . . .E . . .
ESARR 1ESARR 1
SAFETY OVERSIGHT IN ATM
EUROCONTROL SAFETY REGULATORY REQUIREMENT(ESARR)
EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION
EDITION : 1.0EDITION DATE : 27-02-2002STATUS : RELEASED ISSUECLASS : GENERAL PUBLIC
KPIs
PerformanceIndicators
Information
ManagementMeasurement Metrics
Laws Incidents
Accidents Culture
Audit Compliance
SMS ProceduresResourcesAIB Recommendations
Public/Industry
States/Industry
Interested Parties
OrganisationalLevel(Service Providers)
Key Principles
• Information to public/stakeholders• Call to action by stakeholders
• Facilitates identification of scope of action required
• Facilitates management of improvement of service
The whole process needs to be a continuous improvement activity
Safety Performance Measurement
Occurrence-based performance measurement
0
5
10
15
20
25
30
35
40
45
2001 2002 2003 2004 2005
Runway Incursions(occurrence per million aircraft movements and severity)
A
B
C
E
D
Not classif ied
0
0.5
1
1.5
2
2.5
2001 2002 2003 2004 2005
Near Controlled Flight Into Terrain(occurrence per million flight hours and severity)
A
B
C
E
D
Not classif ied
EUROCONTROL has developed safety data reporting to identify key risk areas at European level…
GrossGrossnegligencenegligence OmissionsOmissions SlipsSlips
LapsesLapses MistakesMistakes ViolationsViolationsCriminal Criminal OffencesOffences
unintentionaldeliberate deliberateManagement Statement in
Safety Policy
Procedures
Proactive
Management
Procedures
Proactive
Management
LAWSLAWS
Establishing a Just Culture
ATM Safety System Maturity in ECAC States
Independent maturity assessment system Applied across ECAC Region Now being expanded to
neighbouring States
ANSP Global Maturity
0
10
20
30
40
50
60
70
80
90
100
0 0.2 0.4 0.6 0.8 1
Normalised State Count
Mat
urity
Sco
re
ASP 2002ASP 2004ASP 2006
Regulator Global Maturity
0
10
20
30
40
50
60
70
80
90
100
0 0.2 0.4 0.6 0.8 1
Normalised State Count
Mat
urity
Sco
re
Reg 2002Reg 2004Reg 2006
Performance-based Approach to Managing Risk
Risk Assessment New Systems
Recruitment/Selection
Training
CompetencyChecks
Refresher/Advanced Training
Procedures Operational Processes
Interface ATS CNS, AIM, Airports
CNS /AIMMaintenance Procedures
Emergency Procedures
Risk Assessment ATM Procedures
Risk Assessment Airspace Changes
Risk Assessment Software Changes
Incident Reporting
Incident Investigation
Lessons Learnt
Safety Surveys
and Follow-up
A systematic approach to the A systematic approach to the management of safetymanagement of safety
Proactive Reactive
Risk Assessment and Mitigation
Risk Assessment and Mitigation - 1
Empirical methods of risk assessment no longer sufficient
Systems more complex – failure modes more difficult to identify
Mitigation methods are more complex too – and more costly
Performance-based approach to mitigation is needed – what are the design targets to be met?
Risk Assessment and Mitigation - 2
Transparency is also required by those who will: Own and operate the system Ultimately rely on the safety of the system Bear liability if the system fails
A formal, structured and visible approach is the only answer
It is required by ESARR 4 and the EC’s Common Requirements for ANS provision
A risk classification scheme is a necessary start point for the decision-making that must follow
This is the approach being implemented inThis is the approach being implemented inAir Traffic Management in EuropeAir Traffic Management in Europe
Risk Management
Risk Management is primarily… …a task for the service provider
The provider / operator manages the system and its hazards
Risk management processes are conducted as part of a Safety Management System
Legal requirement for service providers to conduct risk assessment and mitigation in relation to the
implementation of changes to the ATM system
DETERMINATION& SPECIFICATION
DESIGN ANDDEVELOPMENT
INSTALLATIONAND TRANSITION
OPERATIONOPERATION
PROJECT
‘‘SAFETY CASE’SAFETY CASE’
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIESRISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCEACCEPTANCEREVIEW OF THE RISK ASSESSMENTREVIEW OF THE RISK ASSESSMENTAND MITIGATION DOCUMENTATIONAND MITIGATION DOCUMENTATION
This is the sort of process required in ESARR 4 ...This is the sort of process required in ESARR 4 ...Risk Management
This has to be done by the provider...This has to be done by the provider...
DETERMINATION& SPECIFICATION
DESIGN ANDDEVELOPMENT
INSTALLATIONAND TRANSITION
OPERATIONOPERATION
PROJECT
‘‘SAFETY CASE’SAFETY CASE’
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIESRISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCEACCEPTANCEREVIEW OF THE RISK ASSESSMENTREVIEW OF THE RISK ASSESSMENTAND MITIGATION DOCUMENTATIONAND MITIGATION DOCUMENTATION
DETERMINATION & SPECIFICATION
DESIGN ANDDEVELOPMENT
INSTALLATIONAND TRANSITION
OPERATIONOPERATION
PROJECT
‘‘SAFETY CASE’SAFETY CASE’
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIESRISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCEREVIEW OF THE RISK ASSESSMENTAND MITIGATION DOCUMENTATION
Risk Management
But what about this ?But what about this ?
DETERMINATION & SPECIFICATION
DESIGN ANDDEVELOPMENT
INSTALLATIONAND TRANSITION
OPERATIONOPERATION
PROJECTPROJECT
‘‘SAFETY CASE’SAFETY CASE’
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIESRISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCEREVIEW OF THE RISK ASSESSMENTAND MITIGATION DOCUMENTATION
DETERMINATION& SPECIFICATION
DESIGN ANDDEVELOPMENT
INSTALLATIONAND TRANSITION
OPERATION
PROJECT
‘SAFETY CASE’
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
Risk Assessmentand MitigationDeliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCEACCEPTANCEREVIEW OF THE RISK ASSESSMENTREVIEW OF THE RISK ASSESSMENTAND MITIGATION DOCUMENTATIONAND MITIGATION DOCUMENTATION
WHO ACCEPTS THE WHO ACCEPTS THE INTRODUCTION OF NEW INTRODUCTION OF NEW
SYSTEMS AND CHANGES ?SYSTEMS AND CHANGES ?
Risk Management
Acceptance of new systems and changesAcceptance of new systems and changes
In some cases, the provider In some cases, the provider decides about the change…decides about the change…
… … uusing risk assessment and sing risk assessment and mitigation process to support its mitigation process to support its
internal decision-making.internal decision-making.
This is possible if:This is possible if: The provider’s process isThe provider’s process is demonstrated to be effective,demonstrated to be effective,
Enough safety oversight isEnough safety oversight is focused on these processesfocused on these processes (e.g. by means of audits)(e.g. by means of audits)
Regulators may identify new Regulators may identify new systems and changes…systems and changes…
… … to be directly accepted by the to be directly accepted by the regulatory authority through a regulatory authority through a
formal acceptance (or approval)formal acceptance (or approval)
The Regulator makes the final The Regulator makes the final decision on the acceptability of decision on the acceptability of the system to go into operationthe system to go into operation
The review of the ‘safety case’ The review of the ‘safety case’ provides the Regulator with provides the Regulator with evidence to support his decisionevidence to support his decision
EUROCONTROLEUROCONTROL
The provider…The provider… The regulator…The regulator…
WHO ACCEPTS THE WHO ACCEPTS THE INTRODUCTION OF NEW INTRODUCTION OF NEW
SYSTEMS AND CHANGES ?SYSTEMS AND CHANGES ?The ESARR 1 Approach
The ESARR 1 ESARR 1 process for the safety oversight of changesprocess for the safety oversight of changes to the ATM system:
Is implemented by the Regulator by considering results from the risk assessment and mitigation process conducted by the provider Defines a minimum category of changes, whose safety case must be reviewed by the Regulator…
…Based on the severity of the hazards identified by the provider in relation to the change
Provides the regulator with discretion to review other changes
EUROCONTROLEUROCONTROL
WHO ACCEPTS THE WHO ACCEPTS THE INTRODUCTION OF NEW INTRODUCTION OF NEW
SYSTEMS AND CHANGES ?SYSTEMS AND CHANGES ?
Accepted throughATM provider’s procedures(which are subject toregulator’s auditing)
Implementationby the provider of the change (as accepted
by the regulator)
REGULATORREGULATORCONDUCTSCONDUCTS
SAFETYSAFETYREGULATORYREGULATORY
AUDITSAUDITS
REGULATOR APPLIES
DIFFERENT APPROACH DEPENDING
ON THE CHANGE
MajorMajor
MinorMinor
Provider conducts risk assessment
and mitigation and produces a
‘safety case’
Planned Change
(new system or change to
existing system)
Yellow = provider Red = regulator
REGULATOR REVIEWS REGULATOR REVIEWS SAFETY CASESAFETY CASE
Acceptanceby the
regulator
Additional Safety
Conditions imposed
MAJOR = 1. Those changes whose assessment of the potential effects of hazards on
the safety of aircraft, conducted by the provider in accordance with ESARR 4, identifies hazards with potential to lead to an accident or serious incident
2. Other changes that the Regulator considers appropriate to review
The Role Of OversightEUROCONTROLEUROCONTROL
European ATM service providers are required to implement risk assessment and mitigation as part of their SMS:
Risk assessment and mitigation processes are subject to regulatory auditing as any other safety-related process
In addition, the Regulator will specifically review the results of these processes in relation to, at least, the most critical safety-related changes
The implementation of these changes will be subject to regulatory acceptance based on the results.
Summarising the Approach to Risk Management
Risk Classification Scheme
We now have a severity classification scheme for the identification of the effects of ATM/CNS related hazards on the safety of aircraft. (EC law)
We also have a risk classification scheme with a maximum tolerable probability for ATM directly contributing to accidents in the
ECAC region (severity class 1) ….but
maximum tolerable probability for the severity classes 2 to 5 have still to be developed.
States, EC and EUROCONTROL acting together to complete and update those probabilities,
Development of regulatory material for the establishment of a quantified risk classification scheme at regulatory level .
Tolerable?
Continue the design
54321
Severity of the effect
CatastrophicMajor Average Minor No effect
likelihood
I
II
IIIIV
V
VI
effects
Hazard identification.
Likelihood
Severity
Mitigation
Risk
noyes
Safetytarget
Identifying Tolerability of Change
Safety objectives
Performance-based ATM framework… We are on the way…good progress being made. Experience so far…
A performance-driven approach requires: -
Data (occurrences, maturity etc.)
“Just Culture” – overcoming inhibitors to progress
A measurement system, harmonised globally
Analysis capability
Key Performance Indicators (ultimately)
Conclusions - 1
We’ve had a risk-based approach to the management of safety for decades, but….
The risks are more difficult to identify now
Move from “historic” to “predictive” risk assessment
A formal, visible assurance methodology
We need systems to measure the risks before and after changes to the system (was mitigation successful?)
A fully functioning SMS will provide the tools to do the job
Conclusions - 2
Global needs in safety: -
A common approach to safety – management and regulation
Common minimum levels of safety
Availability of information on which to base a performance-driven approach
Common safety “language” – terms, taxonomy and appreciation of risk
The correct balance between State functions and those of other stakeholders
Conclusions - 3
Peter StastnyHead of Safety Regulation [email protected]
ICAO Montreal, Thursday 29 March 2007
Ensuring the Safety of Future Developments
