1

COGNITIVE RESILIENCE APPLIED

TO STAMP 2013 STAMP Conference

Boston, Mass.

March 26-27, 2013

CONTENTS

• Introduction

• Relevance for STAMP

• Experimental studies

• Conclusion

2

INTRODUCTION TO COGNITIVE RESISTANCE

COGNITIVE RESISTANCE

Definition

The capacity to endure a discrepancy between

reality and activated mental schemata, despite

salient cues that are essentially perceived.

4

(De Boer, Badke-Schaub & Santema, (2013, submitted): The Episodic Nature of Cognitive Resilience, Human Factors

PERCEIVE & BELIEVE

• How many of each animal did Mozes take along in the Arc?

Not Noah!

Kahneman, D. (2011). Thinking, fast and slow (1st ed.).

COGNITIVE RESISTANCE

• We ignore stimuli that do not seem relevant to tasks at hand

• Saves resources

• Improves routine operations

• Describes a (real) physiological and neurological process

• Implicit perception

• A lag in stimulus matching

• Sudden explicit perception / reflection, sometimes accompanied by

surprise

• Can terminate in two ways:

• The discrepancy is brought into consciousness (i.e. reflection)

• Reality realigns itself with the activated mental schema (i.e. the cues go

away)

6

(De Boer, Badke-Schaub & Santema, (2013, submitted): The Episodic Nature of Cognitive Resilience, Human Factors

COGNITIVE RESISTANCE IS WELL

KNOWN Similar Phenomena

• Looking-but-not-seeing /

inattentional blindness

• Cognitive Fixation

• Automation surprise

• Automation bias

Australian Transport Safety Board. (2009).

Dutch Safety Board. (2010). Crashed during approach, Boeing 737-800, near Amsterdam Schiphol airport, 25 February 2009.

Dutch Safety Board. (2011). Take-off from Taxiway Amsterdam Airport Schiphol (p. 94). The Hague, the Netherlands.

Leveson, N. (2011). Engineering a safer world: Systems thinking applied to safety.

Manzey, D., & Onnasch, L. (2012). Human Performance Consequences of Automated Decision Aids :, 6(1), 57–87.

Sarter, N., & Woods, D. (1995). How in the world did we ever get into that mode? Mode error and awareness in supe control. Human Factors:

Woods, D. D., Dekker, S., Johannesen, L. J., Cook, R. I., & Sarter, N. (2010). Behind human error (2nd ed.)

Sarter, N. B., Woods, D. D., & Billings, C. E. (1997). Automation surprises. Handbook of hf and ergonomics (Vol. 2, pp. 1926–1943).

Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst : sustained inattentional blindness for dynamic events. Perception, 28, 1059–1074.

7

Examples

• A320 mode input failure,

January 20th, 1992

• Turkish Airlines crash at

Amsterdam, February 25th,

2009

• Emirates 407, 20 March 2009,

Melbourne Australia : A340

• Taxiway take-off, February 10th

2010

WHY A NEW CONSTRUCT?

• An epistemologically sound definition (hopefully..)

• Describes normal rather than erroneous human behavior

• Focuses on the episodic nature of the phenomenon

8

RELEVANCE TO STAMP

10

Leveson, N. (2011). Engineering a safer world:

Systems thinking applied to safety.

RECENT CRITICISM OF HUMAN FACTOR

IN STAMP

“The human factors element of STAMP is

somewhat limited and under-specified. Human

error is conceptualised as essentially a failure of

the operator’s mental model of the system [...].

The model of human behaviour implicit in

STAMP is somewhat deterministic. “

11

Harris, D., & Li, W.-C. (2011). An extension of the Human Factors Analysis and Classification System for use in open systems. Theoretical

Issues in Ergonomics Science, 12(2), 108–128. doi:10.1080/14639220903536559

STOCHASTIC MODEL OF COGNITIVE

RESISTANCE

De Boer, R.J. (2012). Seneca’s Error: An

Affective Model of Cognitive Resistance Environment

Cognitive

Resistance

Implicit

Perception

Perception,

Reflection

EXPERIMENTAL STUDY

EXPERIMENT

• Number Reduction Task

• 2 rules

• Time and error limit

• 81 participants

• Professional engineers

• Engineering students

• Manipulation: Impossible to

meet the task objective while

applying the learned rules

16

De Boer, R.J. (2012). Seneca’s Error: An Affective Model of Cognitive Resistance

TYPICAL RUN

17

De Boer, R.J. (2012). Seneca’s Error: An Affective Model of Cognitive Resistance

0

2

4

6

8

10

12

14

16

18

20

1 11 21 31 41 51 61 71 81 91

Dura

tion p

er tr

ial [

sec]

Trial [sequence number]

Progress of WARP trials - example

Learned model

Pattern model

EXPERIMENTAL RESULTS (N=81)

18

0%

25%

50%

75%

100%

0 20 40 60 80 100 120 140

Cu

mu

lati

ve f

req

ue

ncy

of

refl

ect

ion

Test duration (trials)

(De Boer, Badke-Schaub & Santema, (2013, submitted): The Episodic Nature of Cognitive Resilience, Human Factors

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0 10 20 30 40 50 60 70 80 90 100

Pro

bab

ility

of

refl

ect

ion

Test Duration (trials)

UNIMODAL LOG-LOGISTIC DISTRIBUTION OF

COGNITIVE RESISTANCE

19

(De Boer, Badke-Schaub & Santema, (2013, submitted): The Episodic Nature of Cognitive Resilience, Human Factors

EXPERIMENTAL RESULTS

20

0%

25%

50%

75%

100%

0 20 40 60 80 100 120 140

Cu

mu

lati

ve f

req

ue

ncy

of

refl

ect

ion

Test duration (trials)

(De Boer, Badke-Schaub & Santema, (2013, submitted): The Episodic Nature of Cognitive Resilience, Human Factors

Satisficing behavior

Learning behavior

Constant stimuli

Environment

Cognitive

Resistance

Matching

Emotion Implicit

Perception

Perception,

Reflection

(De Boer, Badke-Schaub & Santema, (2013, submitted): The Episodic Nature of Cognitive Resilience, Human Factors

REINFORCEMENT LEARNING FRAME-

WORK FOR COGNITIVE RESISTANCE

EFFECT OF EMOTION ON TERMINATION

RATE OF COGNITIVE RESISTANCE

23

0%

20%

40%

60%

80%

100%

rapid moderate none

Re

acti

on

to

cu

es

Speed of termination

No emotion

Surprise

Remorse

Anger

Distress

Joy

REPEAT IN MORE PRACTICAL SETTING

Thrust lever malfunction: cues

• N1 axis speed indications

• Exhaust gas temperature

• N2 rotation speed

• Fuel flow

• Rudder deflection

• 6 cues on ECAM page

• Fuel consumed (2x)

• Oil pressure

• Vibrations,

• Oil quantity,

• Remaining fuel.

• But not on ECAM

24

0.00%

0.02%

0.04%

0.06%

0.08%

0.10%

0.12%

0.14%

0.16%

- 200 400 600 800

Pro

bab

ility

of

refl

ect

ion

Test Duration (seconds)

LOG-LOG DISTRIBUTION OF COGNITIVE

RESISTANCE – SIM RESULTS (N=27)

26

Adapted from: Heems, W.J.H. ; A. Speet, R.S. Stam (2012): Automation Surprise.

CONCLUSION

CONCLUSION – VALUE OF COGNITIVE

RESISTANCE FOR STAMP

• Cognitive Resistance describes human perception

performance

• Stochastic with known probability distribution (unimodal log-

logistic)

• Learning or satisficing process

• Emotions as reward / penalty mechanism

• Contribution to STAMP

• Normal rather than erroneous performance

• Stochastic rather than deterministic

28

29

THANK YOU FOR YOUR ATTENTION

Robert J. de Boer, rj.de.boer@hva.nl

Website: http://www.hva.nl/kenniscentrum-dt/onderzoeksthema/aviation/

CONTROL FLAWS

• Inadequate control actions • Unidentified hazards

• Inappropriate etc. control actions

• Design of control process do not enforce constraints

• Inconsistent etc. process models

• Inadequate coordination

• Inadequate execution of control action • Communication flaw

• Inadequate actuator (operator) operation

• Time lag

• Inadequate or missing feedback • Not provided in system design

• Communication flaw

• Time lag

• Inadequate sensor operation

30 Leveson, N. (2004). A New Accident Model for Engineering Safer Systems

MEASURING COGNITIVE RESISTANCE

31

Situational Awareness (SAGAT) • Aircraft state • Flight path • Traffic • Terrain • Weather • …

Automation Awareness (AAGAT) • Equipment state • Normal / alternate law • Operational mode • …