New Guidance for CAST: Case Study of a US Freight Rail...

New Guidance for CAST: Case Study of a US Freight Rail Stop Signal Overrun & Collision

MeganFrance,JordanMulter,&HadarSafar,U.S.DOTVolpeCenter;EmilieRoth,RothCogniCveEngineering

MITSTAMPWorkshopCambridge,MAMarch27,2018

Objec&ves&BackgroundNewCASTGuidanceFreightRailCaseStudyDiscussion

3

Objectives

q  ApplyCASTtoafreightrailcasestudyinordertounderstandsystemiccausesofstopsignaloverruns

q  SuggestaddiConalguidanceforCASTtohelpanalystsunderstandwhichtypesofinformaContoinclude

4

Background: Stop Signal Overruns (SSOs)

q  Consequencescanincludederailmentorcollision;mayresultininjuries,fataliCes,andmajorpropertydamage

q  Railroadstendtoblametheoperators(engineer,conductor),butmanyfactorscontributetotheseincidents!

5

Background: Stop Signal Overruns (SSOs)

q  2priorstudiesatpassengerrailroadsexaminingSSOs

q  Focusgroups,interviews,andobservaCons

q  IdenCfiedcontribuCngfactorsatalllevelsofthesystemhierarchy!

(Safar,Multer&Roth,2017)

ObjecCves&BackgroundNewCASTGuidanceFreightRailCaseStudyDiscussion

7

Additional Guidance: Mental Model Flaws

Recentwork(France,2017)providedaddiConalguidanceformentalmodelflawsinSTPA;canbeappliedtoCAST

*ThispresentaConusesmentalmodelinplaceofprocessmodelwhenreferringtohumancontrollers.

1.SafetyRoles&Responsibili&es

2.UnsafeControlAc&ons(UCAs)

3.MentalModelFlaws*

4.ContextualFactors

8

Foundation of Mental Model Guidance

STPA–EngineeringforHumansExtension

MentalModels

ProcessState

ProcessBehavior

Environment&Controllers

MentalModelUpdates

ControlAc&onSelec&on

SensoryInputsControl

AcCons

HumanController

(Thomas&France,2016;France,2017)

9

Guidance for Mental Models

MentalModelofProcessStateq  Beliefsaboutmodesandmodechangesq  Beliefsaboutthecurrentprocessstage

(forprocesseswithmulCplestages)q  Beliefsaboutsystemvariables(e.g.true/

false,on/off)

(France,2017)

MentalModels

ProcessState

ProcessBehavior

Environment&Controllers

10

Guidance for Mental Models

MentalModelofProcessBehaviorq  Beliefsaboutwhatthesystemcandoq  Beliefsabouthowthesystemwillbehave

inaparCcularmodeorprocessstageq  Beliefsaboutif-thenrelaConshipsbetween

operatorinputandsystemoutput

MentalModels

ProcessState

ProcessBehavior

Environment&Controllers

(France,2017)

11

Guidance for Mental Models

MentalModelofEnvironment&Controllersq  ChangestoenvironmentalcondiConsq  Familiarorunfamiliarenvironmentsq  Othercontrollers’statesandbehaviorsq  SocialandorganizaConalfactors

MentalModels

ProcessState

ProcessBehavior

Environment&Controllers

(France,2017)

12

Additional Guidance: Contextual Factors

Contextualfactorsisabroadcategory,leavinganalystswithpotenCalquesCons:q  WhichtypesofinformaConbelongin

“contextualfactors”?q  Howshouldfindingslistedunder

“contextualfactors”bestructured?

1.SafetyRoles&Responsibili&es

2.UnsafeControlAc&ons(UCAs)

3.MentalModelFlaws

4.ContextualFactors

13

Foundations of Contextual Guidance

(Leveson,2004;adaptedfromRasmussen,1997andRasmussen&Svedung,2000)

(Safar,Roth,&Multer,2015)

*Note:“PASS”standsfor“passingastopsignal”andissynonymouswith“StopSignalOverrun”orSSO.

*

14

Guidance for Contextual Factors

Fivecategoriesofcontextualfactorstoconsider:q  Physicalsystemandtechnologyfactorsq  Individualandteamfactorsq  OrganizaConalfactorsq  Regulatoryfactorsq  External/Environmentalfactors;“Other”

NOTE:ThisguidanceDOESNOTreplacetheneedtocreateasafetycontrolstructureandexaminethecontrolandfeedbackloopsspecifictoYOURsystemandcontrolledprocess!ThisisNOTachecklist;it’sarecommendaNontoconsiderfactorsatalllevelsofyoursystem!

15

Guidance for Contextual Factors

PhysicalSystemandTechnologyFactorsq  OperaCngenvironmentdesignq  Interfacedesign(e.g.displaysandalerts)q  Maintenance/operaConalstatusofphysicalsystemsq  Availabilityornon-availabilityofjobaidsq  Otherphysicalfactors(e.g.weather)

16

Guidance for Contextual Factors

IndividualandTeamFactorsq  CommunicaConandteamwork;coordinaConq  DistracCons/compeCngdemandsforaoenConq  Experiencelevel;qualificaConandtrainingq  FaCgue;workscheduleq  Medicalfitnessfordutyq  ExpectaCons;similarsituaConsencountered

17

Guidance for Contextual Factors

Organiza&onalProcessesq  SupervisoryprioriCes/safetycultureq  Resourceconstraints&producConpressuresq  Policiesandprocedures(workschedules,training,

discipline,etc.)q  Degreeoffeedbackfromemployees

18

Guidance for Contextual Factors

RegulatoryAc&vi&esq  Degreeofsupportto/controloverorganizaConsq  Feedback(data)collectedfromorganizaConsq  RegulaConsregardingemployeesq  RegulaConsregardingphysicalsystemsand

technologies

19

Guidance for Contextual Factors

ExternalFactorsq  High-levelsocietal,governmental,etc.influencesq  Economiccontext,fundingsourcesq  DemandsforservicedrivingproducConpressuresq  PoliCcalclimate’seffectsonfunding,regulaCon,etc.

REMINDER:Theselistsarenotcomprehensive,theyaresimplyasetofexamples!

20

Complete CAST Controller Guidance

3.ProcessModel(MentalModel)FlawsExplainwhyUnsafeControlAcNonsappearedappropriatetothecontroller.

A.  MentalmodelofprocessstateB.  MentalmodelofprocessbehaviorC.  Mentalmodelofenvironment

4.ContextualFactorsSystemicfactorsthatinfluencecontrollers’mentalmodelsanddecisions.

A.  PhysicalsystemandtechnologyfactorsB.  IndividualandteamfactorsC.  OrganizaConalprocessesD.  RegulatoryacCviCesE.  Externalfactors

1.SafetyRoles&Responsibili&es

2.SafetyConstraintsViolatedorUnsafeControlAc&ons(UCAs)

21

ObjecCves&BackgroundNewCASTGuidanceFreightRailCaseStudyDiscussion

22

Head-On Collision of Two Union Pacific Railroad Freight Trains

q  3fataliCes§  Eastboundtrain’s

engineer&conductor§  Westboundengineer

q  1survivor§  Westboundconductor

q  5locomoCvesand32carsderailed

q  $14.8millioninesCmateddamage

Goodwell,OklahomaJune24,2012,10:02a.m

Photosource:theGuymonDailyHeraldcourtesyofMaritEdwards

(NTSB,2013)

23

Accidents and Hazards

ACCIDENTq  Atraincollideswithanothertrainonthesamestretchof

track

HAZARDSq  Atrainentersanareaoftrackitwasnotclearedtoenterq  Atrainentersanareaoftrackalreadyoccupiedbyanother

train

24

System Safety Constraints

q  Dispatchersmustusestopandapproachsignalstoalerttraincrewstoareastheydonothaveclearancetoenter

q  Traincrewsmustslowdownatapproachsignalsandstopatstopsignals

q  TraincrewsmusthaveadequatevisualacuityandcolorvisiontorecognizesignalindicaCons

q  Railroadsmustensurethatcrewsareadequatelyqualifiedandmedicallyfitforduty

25

GoodwellSiding

Theeastboundtrainwassupposedtowaitatthestopsignalforthewestboundtraintoenterthesiding.

ZLAAH22EastboundTrain

AAMMLX22WestboundTrain

Event Overview: Planned Route

26

GoodwellSiding

Theeastboundtraincouldthenproceededforwardsafely,followedbythewestboundtrain.

Event Overview: Planned Route

AAMMLX22WestboundTrain

ZLAAH22EastboundTrain

27

GoodwellSiding

Instead,theeastboundtrainpassedthestopsignalandcollidedwiththewestboundtrainbeforeitreachedthesiding.

AAMMLX22WestboundTrain

Event Overview: Actual Route

ZLAAH22EastboundTrain

Run-throughswitch

28

Event Timeline: Eastbound ZLAAH 22

StopSolidred

StopunClgiven

clearancetoproceed

Approx.2miles

AdvancedApproachFlashingyellow

Reducespeedto40mph

9:56am

Approx.2miles

ApproachSolidyellowoversolidred

Reducespeedto30mph;

preparetostop

9:58am 10:00am

Collision

<2miles

AAMMLX22WestboundTrain

10:01am

29

Event Timeline: Eastbound ZLAAH 22

“…the engineer appeared to make throttle and dynamic brake adjustments that maintained train speed close to the 70 mph limit, as would be expected for a train operating on a clear signal”

(NTSB,2013)

30

Physical System Analysis

SafetyControlsandEquipmentq  Stopsignalsindicatewheretrainsmustnotgoq  “Approach”signalswarnofupcomingstopor

speedrestricConsq  Sidingsallowtrainstopasssafelyonsingletrack

SafetyConstraintsViolatedq  TwotrainsmovingoppositedirecConswere

allowedonasingletrack

(NTSB,2013)

31

Physical System Analysis

FailuresandUnsafeInterac&onsq  SignalsfuncConedasintended,butthe

eastboundtraindidnotobeythemq  NoPosiCveTrainControl(PTC)present

ContextualFactorsandAddi&onalQues&onsq  Individual:IfsignalswereproperlyfuncConing,

whydidtheengineerpassastopsignal?q  OrganizaConal:Whywasn’tposiCvetrain

controlinstalled?

(NTSB,2013)

32

Controller Analysis: Engineer

Speedcontrols(throole&brake)

CurrentspeedSignalindicaCon

Engineer

Physical System (Trains and Signals)

33

Controller Analysis: Engineer

RolesandResponsibili&esq  Controltrainspeedinaccordancewithpostedsignals;

calloutsignalindicaContoconductorq  BemedicallyfittooperatealocomoCveUnsafeControlAc&onsq  Didnotslowdownasrequiredbyapproachsignalsor

stopatthestopsignalq  Operatedwhileknowinghehadinconsistentcolorvision

anddiminishedvisualacuity

34

Controller Analysis: Engineer

MentalModelFlaws

ModelofProcessState:q  Incorrectlybelievedsignalswere“clear”(green)

ModelofProcessBehavior:q  N/A–understoodmeaningofsignals&requiredacNons

ModelofEnvironment&Controllers:q  Mayhaveincorrectlybelievedconductorwouldinterveneq  Likelyunawareofthepresenceofanothertrain

35

Controller Analysis: Engineer

IndividualContextualFactors:q  PerceptuallimitaCons:engineerhadseverevisualimpairments

frommulCplecondiCons;hadfailedacolorvisiontestin2009q  CoordinaCon:shouldhavebeencoordinaCngwithconductorto

calloutsignalsandensureproperspeed;whatwasconductordoing?

q  ExpectaCons:signalsmaybetypicallyclear,high-visibilityq  FaCgue:faCguepossiblebasedonirregularworkschedule

Organiza&onalContextualFactors:q  WhatorganizaConalfactorsallowed(orincenCvized)operaCng

despiteseverevisualseverelimitaCons?

36

ConductorEngineer

Physical System (Trains and Signals)

Controller Analysis: Conductor

Emergencybrake

SignalindicaConsCurrentspeed

Calloutsignals

37

Controller Analysis: Conductor

RolesandResponsibili&esq  CalloutsignalindicaCons;ensureengineerobeyssignals;useemergencybrakeifnecessary

UnsafeControlAc&onsq  Apparentlydidnotcallouttheapproachandstopsignals;didnotwarnengineer

q Didnotpulltheemergencybrake

38

Controller Analysis: Conductor

ProcessModelFlawsModelofProcessState:q  WaslikelyunawarethatsignalswereinarestricCvestate

ModelofProcessBehavior:q  N/A–wouldhaveunderstoodmeaningofsignals

ModelofEnvironment&Controllers:q  Waslikelyunawarethattheengineerwasnotrespondingto

signals;likelybelievedengineercouldreadthesignals

39

Controller Analysis: Conductor IndividualContextualFactors:q  DistracCon/inaoenCon:conductormayhavebeenasleeporabsentq  ExpectaCons:signalsmaybetypicallyclear,high-visibilitycondiCons,

mayhavebelievedengineerwascapableofrecognizingsignalsq  FaCgue:faCguepossiblebasedonirregularworkscheduleq  CoordinaCon:engineermayhaveallowedconductortorest

Organiza&onalContextualFactors:q  WhatorganizaConalfactorscontributedtolackofrequired

coordinaConbetweentheengineerandconductor?

40

ConductorEngineer

Physical System (Trains and Signals)

Dispatcher

Train Crew

Controller Analysis: Dispatcher

Clearances&instrucCons

LocaCon/issues

Setsignals&switches

Signal&switchstatusTrainposiCons?Trainspeeds?

41

Controller Analysis: Dispatcher

RolesandResponsibili&esq  SetsignalsandswitchesforsaferouCngoftrainsq  CommunicaterouteinformaContotraincrewsUnsafeControlAc&onsq  Apparentlydidnotwarneastboundtraincrewabouttheupcomingstopsignal&othertrain

42

Controller Analysis: Dispatcher

ProcessModelFlawsModelofProcessState:q  Didnotrealizeeastboundtrainhadpassedthesignals

ModelofProcessBehavior:q  Mayhavebelievedthatifcrewspassedasignal,hewouldbe

alertedandhaveCmetointervene

ModelofEnvironment&Controllers:q  Assumedcrewwouldobeysignalsevenifnotgivenwarning

43

Controller Analysis: Dispatcher PhysicalSystemContextualFactors:q  DispatcherwasnotnoCfiedthatthetrainwasoverthespeedlimitq  DispatcherwasonlynoCfiedoncetheswitchattheendofthe

sidingwaspushedoutofalignmentbytheeastboundtrain

IndividualContextualFactors:q  ExpectaCons:didnotexpectcrewstopassastopsignal(veryrare)q  CoordinaCon:dispatcherwasresponsiblefor10-12crewsq  Timing:accidentoccurredapprox.5minsarerfirstmissedsignal,

andlessthan2minutesarerthedispatcherreceivedanalert

44

Controller Analysis: UP Management Railroad Management

ConductorEngineer

Physical System (Trains and Signals)

Outside Medical Providers Dispatcher

Train Crew

DocumentaConofVisionTestResults

Diagnoses&TreatmentsofEyeCondiCons

VisualAcuity

Training,Policies,

etc.

Safetyissues

Training,Policies,

etc.

Safetyissues

45

Controller Analysis: UP Management

RolesandResponsibili&esq  Ensurethatemployeesaremedicallyfitfordutyand

adequatelytrainedonjobtasks,includingcoordinaConq  EnsurethatphysicalsystemcomplieswithregulaConsUnsafeControlAc&onsq  DidnothavePTCinstalledontherouteq  Didnotprovidetrainingoncrewresourcemanagementq  DidnotrequiredocumentaConofengineer’svisual

acuityresults;didnotrequirefollow-uptesCngq  Usedacolorvisiontestofunknownvalidity/reliability

46

Controller Analysis: UP Management

MentalModelFlaws

q  Mayhaveincorrectlybelievedengineer’svisionwasokayq  Mayhavebelievedthatengineer’svisionwouldnotbea

problemwiththeconductorassisCngbycallingoutsignalsq  Mayhavebelievedresourceswerenotavailableto

implementsafetymeasures,orthatsafetymeasureswerenotasurgentasotherprioriCes(CRM,PTC)

47

Controller Analysis: UP Management Organiza&onalContextualFactors:q  Resourceconstraints:PTC/CRMrequirelargeamountofresourcesq  Staffingconstraints:mayhaveneededengineersbadly,decidedto

keeptheengineerdespitevisuallimitaConsq  Policies:didnotuniformlyapplypolicyforverifyingvisualacuitywas

toobtainwrioendocumentaConq  SchedulingpracCces:irregularschedulesarecommoninrail

industry;maycontributetofaCguedespitehoursofservicelimitaCons

RegulatoryContextualFactors:q  WasUPinviolaConofregulaCons,orwereregulaConsinadequate?

48

Controller Analysis: FRA

Railroad Management

ConductorEngineer

Physical System (Trains and Signals)

Outside Medical Providers Dispatcher

Train Crew

Federal Railroad Administration

RegulaCons Safetydata

Railroad Management

ConductorEngineer

Physical System (Trains and Signals)

Outside Medical Providers Dispatcher

Train Crew

49

Controller Analysis: FRA

RolesandResponsibili&esq  RegulaterailroadstoensuresafetystandardsUnsafeControlAc&onsq  AllowedretakingvisiontestswithoutvalidaCngrailroad’stesCngmethods

q DidnotmandatePTCinstallaConsoonerq DidnotmandateCRMtrainings

50

Controller Analysis: FRA ProcessModelFlawsq  BelievedCmelineforPTCimplementaConwasappropriateq  BelievedrailroadshadadequateresourcesforCRMq  BelievedrailroadswoulduseavalidvisiontesCngmethod

RegulatoryContextualFactors:q  FRAalreadyrequiredimplementaConofPTCby2015q  FRAconductedresearchintoCRM,providedrailroadswithfundingforpilot

programs

ExternalFactorsq  DesiretomaintainposiCverelaConshipswithrailroads&workwithintheir

resourceconstraints(e.g.PTCregulaConCming)

51

ObjecCves&BackgroundNewCASTGuidanceFreightRailCaseStudyDiscussion

52

Enhanced CAST Controller Guidance

q  ProvidesaddiConalguidanceonwhattypesofcontenttoinclude

q  UsefulfornewpracCConersofCAST

q  RevealedinteresCngresultsinourfreightcasestudy

3.ProcessModel(MentalModel)Flaws

A.  MentalmodelofprocessstateB.  MentalmodelofprocessbehaviorC.  Mentalmodelofenvironment

4.ContextualFactors

A.  PhysicalsystemandtechnologyfactorsB.  IndividualandteamfactorsC.  OrganizaConalfactorsD.  RegulatoryfactorsE.  External/environmentalfactors;“other”

1.SafetyRoles&Responsibili&es

2.UnsafeControlAc&ons(UCAs)

53

References

q  France,M.,“EngineeringforHumans:ANewExtensiontoSTPA,”Master’sThesis,MassachuseosInsCtuteofTechnology,2017.

q  France,M.,“EngineeringforHumans:Human-AutomaConInteracConinSTPA,”presentedatthe6thAnnualMITSTAMPWorkshop,2017.

q  Leveson,N.G.“ANewAccidentModelforEngineeringSaferSystems.”SafetyScience,vol.42,no.4,pp.237-270,2004.

q  Leveson,N.G.,“EngineeringaSaferWorld:SystemsThinkingAppliedtoSafety.”TheMITPress,2012.q  NaConalTransportaConSafetyBoard,“HeadonCollisionofTwoUnionPacificRailroadFreightTrainsNear

Goodwell,Oklahoma,June24,2012.”NTSB/RAR-13/02.Washington,DC:NTSB,2013.q  Rasmussen,J.,“RiskManagementinaDynamicSociety:AModellingProblem,”SafetyScience,Vol.27,No.

2/3,pp.183-213,1997.q  Rasmussen,J.&Svedung,I.“ProacCveRiskManagementinaDynamicSociety.”SwedishRescueServices

Agency,2000.q  Safar.H.,Multer,J,.&Roth,E.“AnInvesCgaConofPassingStopSignalsataPassengerRailroad”

Washington,DC:FederalRailroadAdministraCon,2015.q  Safar.H.,Multer,J,.&Roth,E.“Whydopassengertrainspassstopsignals?Asystemsview”Washington,

DC:FederalRailroadAdministraCon,2017.q  Thomas,J.andM.France.“EngineeringforHumans:STPAAnalysisofanAutomatedParkingSystem,”

presentedatthe5thannualMITSTAMPWorkshop,2016.

54

ThankyouforyouraoenCon!Contact:MeganFrance,USDOTVolpeCentermegan.france@dot.gov

55

BackupSlides:PTCImplementaCon

56

Positive Train Control Timeline

Source:hops://www.fra.dot.gov/ptc

57

PTC Implementation in Freight RRs

Source:hops://www.fra.dot.gov/ptc

58

PTC Implementation at UP

AsofDecember31,2017

Source:hops://www.fra.dot.gov/ptc

59

Ongoing/Future Work on SSOs

q  Currentstudyfocusesonfreightenvironment§  CASTanalysesareapreliminarystep§  Follow-upwithinterviews,focusgroups,etc.andbriefingtorailroad

management

q  Otherongoingwork§  ImprovingSSOdatacollecConusingacommonformtemplate§  CommunicaCngfindingstorailroadsina“GoodPracCceGuide”