© D

édal

e &

EU

RO

CO

NT

RO

L

Systemic Occurrence Analysis Methodology

Tony LICUEUROCONTROL

Glasgow – Aug 2005

SOAM

A Reason-Based Organisational Methodology

A tool for the analysis of safety occurrences

(accidents & Incidents)

EUROCONTROL SOAM

SOAM Antecedents

The Reason Model ~ circa 1990 Developed from Professor James Reason’s

work on human error and “organisational accidents”

Tripod Delta ~ circa 1994 Developed for Shell Petroleum, based on

Reason Model ICAM ~ circa 2000

Developed for BHP Billiton, based on Reason Model and Tripod Delta

EUROCONTROL SOAM

“Recent” fatal accidents in Europe with ATM

contribution

Überlingen, 1 July 2002

Linate, 8 October 2001

EUROCONTROL SOAM

Call for Action

Aviation Safety is still at a good level Traffic however is increasing Two major accidents involving Air

Traffic Management in 2001 and 2002 …..

In well developed countries in the heart of Europe …

From which many lessons can be learnt Wake up call for actionWake up call for action

EUROCONTROL SOAM

AGAS/ SSAP Priority Areas

1. Safety related human resources in ATM

2. Incident reporting and data sharing

3. ACAS/TCAS

4. Ground-based safety nets

5. Runway safety

6. Enforcement of ESARRs and implementation

monitoring

7. Awareness of safety matters

8. Safety and human factors research & development

An Action Group of European aviation safety experts An Action Group of European aviation safety experts identified the following areas as needing immediate identified the following areas as needing immediate focus: focus: 1. Safety related human resources in ATM

2. Incident reporting and data sharing

3. ACAS/TCAS

4. Ground-based safety nets

5. Runway safety

6. Enforcement of ESARRs and implementation monitoring

7. Awareness of safety matters

8. Safety and human factors research & development

EUROCONTROL SOAM

Why anotherinvestigation tool?

Support ESARR 2 implementation and Strategic Safety Action Plan

Higher quality reports and AST returns ~ a need to:

clearly identify causes and report them concisely

go beyond the human errors, to find systemic causes

use a simple, consistent approach for events of all severity levels

ensure recommendations are relevant and effective

EUROCONTROL SOAM

Current investigation methods

What can we improveabout the way we conduct

safety occurrence investigations?

EUROCONTROL SOAM

How SOAM can help

A methodology that includes structured processes to:

identify and classify a range of contributing factors

sort out irrelevant, non-contributing facts

move from a focus on human error/s to identify systemic causes ~ support for ‘Just Culture’

analyse simple events through to high severity incidents and accidents

clearly link recommendations to the facts of the analysis

EUROCONTROL SOAM

Elements of Organisational Occurrences

Organisational Factors: Latent system failures that produce or allow

conditions under which accidents are possible Contextual Conditions:

Situational factors involving characteristics of the task, the environment or human limitations

Human Involvement: Errors and/or violations which have an immediate

adverse affect (“active failures”) Inadequate or absent barriers/defences:

Failure to identify and protect the system against human errors or violations, local conditions

EUROCONTROL SOAM

The Reason Model Organisational Error Chain

Organisational and System Factors

“Unsafe

Acts”

Latent Conditions (adapted from Reason, 1990)

ActiveFailures

Contextual Conditions Human

Involvement

Limited window/sof opportunity

Absent or Failed Barriers

ACCIDENT

People, Task, Environment

EUROCONTROL SOAM

Accident “Causes”

A man has a bad argument with his wife. He storms out of the house to the nearest bar and

drinks four whiskies. He then decides to go for a drive. It is night-time, there is a skim of snow on the

ground, and the tyres on our victim’s car are smooth.

In rounding a poorly banked curve at excessive speed, the right front tyre blows out, the car leaves the road and is demolished.

What is the cause of the accident?

(Johnston, 1996)

EUROCONTROL SOAM

SOAM Worked Example

Runway Overrun, Bangkok

September 1999

EUROCONTROL SOAM

Accident Summary

On 23 September 1999, at about 2247 local time,a Boeing 747-438 aircraft overran runway 21 Left (21L) while landing at Bangkok International Airport, Thailand.

EUROCONTROL SOAM

Accident Summary

The aircraft sustained substantial damage during the overrun. None of the three flight crew, 16 cabin crew or

391 passengers reported any serious injuries.

The overrun occurred after the aircraftlanded long and aquaplaned on a runway which was

affected by water following very heavy rain.

EUROCONTROL SOAM

SOAM analysis key steps

Review the Facts

Identify the Organisational Factors

Identify the Contextual Conditions

Identify the Human Involvement

Identify the Absent or Failed Barriers

Validate the OFs against the Occurrence

“CHECKQUESTIONS”

HELP TOSORT ANDCLASSIFY

FACTS

EUROCONTROL SOAM

The SHEL Model(after Edwards, 1972)

LIVEWARE

Operators

L

HARDWARE

Equipment, vehicles, tools,controls, switches, levers,

workplace design, seating etc

H

L LLIVEWARE/LIVEWARE

Interface between people.Operators, controllers,

managers, etc

E

ENVIRONMENT

Site, terrain,weather, roads,

traffic,remoteness

etc

SSOFTWARE

Procedures,checklists,manuals,training

materials,charts etc

First Officer

Other pilots

Captain

PEOPLE

Crew employed flaps 25/ idle reverse landing

configuration

Very heavy rainfall, runway surface

affected by water

Captain awake 21 hours at time of

accident

Importance of reverse thrust as stopping force on

water-affected runways not known

Qantas B747s generally operated in

good weather & toaerodromes with long,good quality runways

FO awake for 19 hoursat the time of the

accidentConfusion after

thrust levers retarded, in high

workload situation

Most pilots not fullyaware about 'aquaplaning'

Crew did not use an adequate risk mgt

strategy for approachand landing

No formal risk assessment conducted when changed landing procedure researched

“Landing on SlipperyRunways” (Boeing

doc) not distributed in Qantas since 1977

Captain & FO quite low levels of flying prior 30

days

No policies or procedures for maintenance of

recency for management pilots

Normal practice to use flaps 25/idle

reverse

Documents unclear (eg., key terms not

well defined)

FO did not fly theaircraft accurately

during final approach

No formal review of new procedures after

'trial' periodAbsence of reverse

thrust during landing roll not

noticed, not used

Captain cancelled go-around decision by

retarding thrust levers

SOFTWAREHARDWARE ENVIRONMENT ORGANISATION

Raw Data Collection AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.

AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.

Captain did not order ago-around earlier

Recent crew experience using full reverse thrust lacking

No appropriatelydocumented info,

procedures regarding operations on water-

affected runways

No policies, procedures on duty

or work limits for pilots with flying & non-flying duties

Reduced visibility & distraction: rain andwindscreen wipers

High workload situation, distraction or

inexperience

Contaminated runwayissues not covered in recent years during crew endorsement,

promotional or recurrent training

Cost-benefit analysisof new landing

procedure was biased

Gather data relevant to the occurrence

Gather data relevant to the occurrence

Partial loss of external visual reference due to

heavy rain

Revised approach/ landing procedure introduced in 1996:

flaps 25, idle reverse thrust

Boeing advised that if idle reverse technique is

adopted, it should be the exception

rather than the rule

Most pilots disagreed they had adequate

training on landing on contaminated

runways

Introduction of newlanding procedure

poor

Bangkok runway was resurfaced in 1991

First Officer

Other pilots

Captain

PEOPLE

Crew employed flaps 25/ idle reverse landing

configuration

Very heavy rainfall, runway surface

affected by water

Captain awake 21 hours at time of

accident

Importance of reverse thrust as stopping force on

water-affected runways not known

Qantas B747s generally operated in

good weather & toaerodromes with long,good quality runways

FO awake for 19 hoursat the time of the

accidentConfusion after

thrust levers retarded, in high

workload situation

Most pilots not fullyaware about 'aquaplaning'

Crew did not use an adequate risk mgt

strategy for approachand landing

No formal risk assessment conducted when changed landing procedure researched

“Landing on SlipperyRunways” (Boeing

doc) not distributed in Qantas since 1977

Captain & FO quite low levels of flying prior 30

days

No policies or procedures for maintenance of

recency for management pilots

Normal practice to use flaps 25/idle

reverse

Documents unclear (eg., key terms not

well defined)

FO did not fly theaircraft accurately

during final approach

No formal review of new procedures after

'trial' periodAbsence of reverse

thrust during landing roll not

noticed, not used

Captain cancelled go-around decision by

retarding thrust levers

SOFTWAREHARDWARE ENVIRONMENT

Raw Data Refinement

Captain did not order ago-around earlier

Recent crew experience using full reverse thrust lacking

No appropriatelydocumented info,

procedures regarding operations on water-

affected runways

No policies, procedures on duty

or work limits for pilots with flying & non-flying duties

Reduced visibility & distraction: rain andwindscreen wipers

High workload situation, distraction or

inexperience

Contaminated runwayissues not covered in recent years during crew endorsement,

promotional or recurrent training

Cost-benefit analysisof new landing

procedure was biased

Partial loss of external visual reference due to

heavy rain

Revised approach/ landing procedure introduced in 1996:

flaps 25, idle reverse thrust

Boeing advised that if idle reverse technique is

adopted, it should be the exception

rather than the rule

Most pilots disagreed they had adequate

training on landing on contaminated

runways

Introduction of newlanding procedure

poor

Bangkok runway was resurfaced in 1991

Sort out the non-contributing facts of the investigation

Sort out the non-contributing facts of the investigation

Boeing advised that if idle reverse technique is

adopted, it should be the exception

rather than the rule

Bangkok runway was resurfaced in 1991

ORGANISATION

AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.

AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.

First Officer

Other pilots

Captain

PEOPLE

Crew employed flaps 25/ idle reverse landing

configuration

Very heavy rainfall, runway surface

affected by water

Captain awake 21 hours at time of

accident

Importance of reverse thrust as stopping force on

water-affected runways not known

Qantas B747s generally operated in

good weather & toaerodromes with long,good quality runways

FO awake for 19 hoursat the time of the

accidentConfusion after

thrust levers retarded, in high

workload situation

Most pilots not fullyaware about 'aquaplaning'

Crew did not use an adequate risk mgt

strategy for approachand landing

No formal risk assessment conducted when changed landing procedure researched

“Landing on SlipperyRunways” (Boeing

doc) not distributed in Qantas since 1977

Captain & FO quite low levels of flying prior 30

days

No policies or procedures for maintenance of

recency for management pilots

Normal practice to use flaps 25/idle

reverse

Documents unclear (eg., key terms not

well defined)

FO did not fly theaircraft accurately

during final approach

No formal review of new procedures after

'trial' periodAbsence of reverse

thrust during landing roll not

noticed, not used

Captain cancelled go-around decision by

retarding thrust levers

SOFTWAREHARDWARE ENVIRONMENT

Raw Data Refinement

Captain did not order ago-around earlier

Recent crew experience using full reverse thrust lacking

No appropriatelydocumented info,

procedures regarding operations on water-

affected runways

No policies, procedures on duty

or work limits for pilots with flying & non-flying duties

Reduced visibility & distraction: rain andwindscreen wipers

High workload situation, distraction or

inexperience

Contaminated runwayissues not covered in recent years during crew endorsement,

promotional or recurrent training

Cost-benefit analysisof new landing

procedure was biased

Partial loss of external visual reference due to

heavy rain

Revised approach/ landing procedure introduced in 1996:

flaps 25, idle reverse thrust

Most pilots disagreed they had adequate

training on landing on contaminated

runways

Introduction of newlanding procedure

poor

Use the remaining factors to build the Analysis chart

Use the remaining factors to build the Analysis chart

ORGANISATION

AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.

AC1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway.

First Officer

Other pilots

Captain

PEOPLE

Crew employed flaps25/ idle reverse landing

configuration

Captain awake 21hours at time of

accident

Importance ofreverse thrust asstopping force on

water-affectedrunways not known

Qantas B747sgenerally operated in

good weather & toaerodromes with long,good quality runways

FO awake for 19 hoursat the time of the

accidentConfusion after

thrust leversretarded, in high

workload situation

Most pilots not fullyaware about'aquaplaning'

Crew did not use anadequate risk mgt

strategy for approachand landing

No formal riskassessment conductedwhen changed landingprocedure researched

“Landing on SlipperyRunways” (Boeing

doc) not distributed inQantas since 1977

Captain & FO quite lowlevels of flying prior 30

days

No policies orprocedures formaintenance of

recency formanagement pilots

Normal practice touse flaps 25/idle

reverse

Documents unclear(eg., key terms not

well defined)

FO did not fly theaircraft accurately

during final approach

No formal review ofnew procedures after

'trial' period

Absence of reversethrust during

landing roll notnoticed, not used

Captain cancelled go-around decision by

retarding thrust levers

SOFTWAREHARDWARE ENVIRONMENT

Raw Data QF1 overruns runway at Bangkok after landing long,recent heavy rainfall, and water on runway.

Captain did not order ago-around earlier

Recent crewexperience using fullreverse thrust lacking

No appropriatelydocumented info,

procedures regardingoperations on water-

affected runways

No policies,procedures on duty

or work limits forpilots with flying &non-flying duties

Reduced visibility &distraction: rain andwindscreen wipers

High workloadsituation, distraction

or inexperience

Contaminated runwayissues not covered inrecent years duringcrew endorsement,

promotional orrecurrent training

Cost-benefit analysisof new landing

procedure was biased

Partial loss of externalvisual reference due to

heavy rain

Revised approach/landing procedureintroduced in 1996:

flaps 25, idle reversethrust

Most pilots disagreedthey had adequatetraining on landingon contaminated

runways

Introduction of newlanding procedure

poor

ORGANISATION

Very heavy rainfall,runway surface

affected by water

ACCIDENTABSENT OR

FAILED BARRIERSHUMAN

INVOLVEMENTCONTEXTUALCONDITIONS

ORGANISATIONAL FACTORS

Very heavy rainfall, runway surface

affected by water

Building the Analysis Chart

Very heavy rainfall, runway surface

affected by water

Very heavy rainfall, runway surface

affected by water

?

Very heavy rainfall, runway surface

affected by water

?

EUROCONTROL SOAM

Absent or Failed Barriers

Describe the “last minute” measures which failed or were missing, and therefore did not prevent the accident

Check Question:

“Does the item describe a work procedure, aspect of human awareness, physical obstacle, warning or control system, or protection measure designed to prevent an occurrence or lessen its consequences?”

EUROCONTROL SOAM

Human Involvement

Describe the errors or violations (actions or omissions) by operators at the scene which “triggered” the accident

Check Question:

“Does the item describe an action or non-action (error or violation) that immediately contributed to the occurrence?”

First Officer

Other pilots

PEOPLE

Captain awake 21hours at time of

accident

Importance ofreverse thrust asstopping force on

water-affectedrunways not known

Qantas B747sgenerally operated in

good weather & toaerodromes with long,good quality runways

FO awake for 19 hoursat the time of the

accidentConfusion after

thrust leversretarded, in high

workload situation

Most pilots not fullyaware about'aquaplaning'

Crew did not use anadequate risk mgt

strategy for approachand landing

No formal riskassessment conductedwhen changed landingprocedure researched

“Landing on SlipperyRunways” (Boeing

doc) not distributed inQantas since 1977

Captain & FO quite lowlevels of flying prior 30

days

No policies orprocedures formaintenance of

recency formanagement pilots

Normal practice touse flaps 25/idle

reverse

Documents unclear(eg., key terms not

well defined)

FO did not fly theaircraft accurately

during final approach

No formal review ofnew procedures after

'trial' period

Absence of reversethrust during

landing roll notnoticed, not used

Captain cancelled go-around decision by

retarding thrust levers

SOFTWAREHARDWARE ENVIRONMENT ORGANISATION

Raw Data QF1 overruns runway at Bangkok after landing long,recent heavy rainfall, and water on runway.

Captain did not order ago-around earlier

Recent crewexperience using fullreverse thrust lacking

No appropriatelydocumented info,

procedures regardingoperations on water-

affected runways

No policies,procedures on duty

or work limits forpilots with flying &non-flying duties

Reduced visibility &distraction: rain andwindscreen wipers

High workloadsituation, distraction

or inexperience

Contaminated runwayissues not covered inrecent years duringcrew endorsement,

promotional orrecurrent training

Cost-benefit analysisof new landing

procedure was biased

Partial loss of externalvisual reference due to

heavy rain

Revised approach/landing procedureintroduced in 1996:

flaps 25, idle reversethrust

Most pilots disagreedthey had adequatetraining on landingon contaminated

runways

Introduction of newlanding procedure

poor

Very heavy rainfall,runway surface

affected by water

ACCIDENTABSENT OR

FAILED BARRIERSHUMAN

INVOLVEMENTCONTEXTUALCONDITIONS

ORGANISATIONAL FACTORS

Building the Analysis Chart

Crew employed flaps 25/ idle reverse landing

configuration

Crew employed flaps25/ idle reverse landing

configuration

Crew employed flaps 25/ idle reverse landing

configuration

Very heavy rainfall, runway surface

affected by water

Crew employed flaps 25/ idle reverse landing

configuration

?

EUROCONTROL SOAM

Contextual Conditions

Describe the context of the event ~ the conditions existing immediately prior to, or at the time of the accident

Check Question:

“Does the item describe an aspect of the workplace, local organisational climate, or a person’s attitudes, personality, performance limitations, physiological or emotional state that helps explain their actions?”

EUROCONTROL SOAM

Organisational Factors

Describe the organisational and system factors (failures) which created, or allowed, the prevailing contextual conditions

Check Question:

“Does the item describe an aspect of an organisation’s culture, systems, processes or decision-making that existed before the occurrence and which resulted in the contextual conditions or allowed those conditions to continue?”

•Aircraft overranrunway afterlanding long

•No serious injuries(391 pax, 19 crew)

•Potential for moreserious outcome

•Aircraft repair cost:$100,000,000 (?)

•Damage tocompanyreputation

Very heavy rainfall, runwaysurface affected by water

Crew not aware of criticalimportance of reverse thrust

as stopping force onwater-affected runways

FO awake for 19 hoursat the time of the accident

CM No formal review of newprocedures after 'trial' period

Most pilots not fullyaware about 'aquaplaning'

PP No appropriatelydocumented info,

procedures re operations onwater-affected runways

WM No policies, procedures onduty or work limits for pilots

with flying & non-flying duties

PP Regulationscovering emergency

procedures &EP training

were deficient

AC CASAsurveillance ofairline flight

operations deficient

Recent crew experience usingfull reverse thrust lacking

RM No formal risk assessmentconducted when changed

landing procedure researched

SOAM ChartAircraft Accident Boeing 747-438

Bangkok, ThailandSeptember 1999

ACCIDENTABSENT OR

FAILEDBARRIERS

HUMANINVOLVEMENT

CONTEXTUALCONDITIONS

ORGANISATIONAL FACTORS

OTHER SYSTEM FACTORS

First Officerdid not fly the

aircraft accuratelyduring the

final approach

Captain cancelledgo-around decision

by retardingthe thrust levers

New 1996 approach/ landingprocedure inappropriate

CO Documents unclear (eg., key terms not well defined)

CO “Landing on SlipperyRunways” (Boeing doc) not

distributed in Qantas since 1977

Absence ofreverse thrustduring landingroll not noticed,reverse thrust

not used

Flight crew did notuse an adequate

risk managementstrategy forapproach

and landing

Reduced visibility & distraction: rain andwindscreen wipers

Qantas B747s generallyoperated in good weather

& to aerodromes with long,good quality runways

Captain awake 21 hoursat time of accident

High workload situation

TR Contaminated runwayissues not covered during crewendorsement, promotional or

recurrent training in recent years

Captain & FO quite lowlevels of flying prior 30 days

WM No policies or proceduresfor maintenance of recency

for management pilots

Normal practice to useflaps 25/idle reverse

CM Introduction of newlanding procedure poor

CG Cost-benefit analysisof new landing procedure

was biased

PP Regulationscovering

contaminated runwayoperations deficient

Crew employedflaps 25/idle

reverse landingconfiguration

Captain did notorder a go-

around earlier

Landingprocedure

inappropriate

Crew ResourceManagement

deficient

OC Mgt decisions informal,“intuitive”, “personality-driven”

EUROCONTROL SOAM

Provide recommendationsthat will preventrecurrence of this scenario

Recommendations shouldbe directed to theresponsible position,and must addressall identified:

1 Absent or Failed Barriers

2 Organisational Factors

Recommendations

•Aircraft overranrunway afterlanding long

•No serious injuries(391 pax, 19 crew)

•Potential for moreserious outcome

•Aircraft repair cost:$100,000,000 (?)

•Damage tocompanyreputation

Very heavy rainfall, runwaysurface affected by water

Crew not aware of criticalimportance of reverse thrust

as stopping force onwater-affected runways

FO awake for 19 hoursat the time of the accident

CM No formal review of newprocedures after 'trial' period

Most pilots not fullyaware about 'aquaplaning'

PP No appropriatelydocumented info,

procedures re operations onwater-affected runways

WM No policies, procedures onduty or work limits for pilots

with flying & non-flying duties

PP Regulationscovering emergency

procedures &EP training

were deficient

AC CASAsurveillance ofairline flight

operations deficient

Recent crew experience usingfull reverse thrust lacking

RM No formal risk assessmentconducted when changed

landing procedure researched

SOAM ChartAircraft Accident Boeing 747-438

Bangkok, ThailandSeptember 1999

ACCIDENTABSENT OR

FAILEDBARRIERS

HUMANINVOLVEMENT

CONTEXTUALCONDITIONS

ORGANISATIONAL FACTORS

OTHER SYSTEM FACTORS

First Officerdid not fly the

aircraft accuratelyduring the

final approach

Captain cancelledgo-around decision

by retardingthe thrust levers

New 1996 approach/ landingprocedure inappropriate

CO Documents unclear (eg., key terms not well defined)

CO “Landing on SlipperyRunways” (Boeing doc) not

distributed in Qantas since 1977

Absence ofreverse thrustduring landingroll not noticed,reverse thrust

not used

Flight crew did notuse an adequate

risk managementstrategy forapproach

and landing

Reduced visibility & distraction: rain andwindscreen wipers

Qantas B747s generallyoperated in good weather

& to aerodromes with long,good quality runways

Captain awake 21 hoursat time of accident

High workload situation

TR Contaminated runwayissues not covered during crewendorsement, promotional or

recurrent training in recent years

Captain & FO quite lowlevels of flying prior 30 days

WM No policies or proceduresfor maintenance of recency

for management pilots

Normal practice to useflaps 25/idle reverse

CM Introduction of newlanding procedure poor

CG Cost-benefit analysisof new landing procedure

was biased

PP Regulationscovering

contaminated runwayoperations deficient

Crew employedflaps 25/idle

reverse landingconfiguration

Captain did notorder a go-

around earlier

Landingprocedure

inappropriate

Crew ResourceManagement

deficient

OC Mgt decisions informal,“intuitive”, “personality-driven”

CM No formal review of newprocedures after 'trial' period

PP No appropriatelydocumented info,

procedures re operations onwater-affected runways

WM No policies, procedures onduty or work limits for pilots

with flying & non-flying duties

PP Regulationscovering emergency

procedures &EP training

were deficient

AC CASAsurveillance ofairline flight

operations deficient

RM No formal risk assessmentconducted when changed

landing procedure researched

CO Documents unclear (eg., key terms not well defined)

CO “Landing on SlipperyRunways” (Boeing doc) not

distributed in Qantas since 1977

TR Contaminated runwayissues not covered during crewendorsement, promotional or

recurrent training in recent years

WM No policies or proceduresfor maintenance of recency

for management pilots

CM Introduction of newlanding procedure poor

CG Cost-benefit analysisof new landing procedure

was biased

PP Regulationscovering

contaminated runwayoperations deficient

OC Mgt decisions informal,“intuitive”, “personality-driven”

Absence ofreverse thrustduring landingroll not noticed,reverse thrust

not used

Landingprocedure

inappropriate

Crew ResourceManagement

deficient

EUROCONTROL SOAM

Questions?

EUROCONTROL SOAM