New Guidance for CAST: Case Study of a US Freight Rail ...psas.scripts.mit.edu/home/wp-content/uploads/2018/04/Tues-Franc… · New Guidance for CAST: Case Study of a US Freight Rail

New Guidance for CAST: Case Study of a US Freight Rail Stop Signal Overrun & Collision

MeganFrance,JordanMulter,&HadarSafar,U.S.DOTVolpeCenter;EmilieRoth,RothCogniCveEngineering

MITSTAMPWorkshopCambridge,MAMarch27,2018

Objec&ves&BackgroundNewCASTGuidanceFreightRailCaseStudyDiscussion

3

Objectives

q  ApplyCASTtoafreightrailcasestudyinordertounderstandsystemiccausesofstopsignaloverruns

q  SuggestaddiConalguidanceforCASTtohelpanalystsunderstandwhichtypesofinformaContoinclude

4

Background: Stop Signal Overruns (SSOs)

q  Consequencescanincludederailmentorcollision;mayresultininjuries,fataliCes,andmajorpropertydamage

q  Railroadstendtoblametheoperators(engineer,conductor),butmanyfactorscontributetotheseincidents!

5

Background: Stop Signal Overruns (SSOs)

q  2priorstudiesatpassengerrailroadsexaminingSSOs

q  Focusgroups,interviews,andobservaCons

q  IdenCfiedcontribuCngfactorsatalllevelsofthesystemhierarchy!

(Safar,Multer&Roth,2017)

ObjecCves&BackgroundNewCASTGuidanceFreightRailCaseStudyDiscussion

7

Additional Guidance: Mental Model Flaws

Recentwork(France,2017)providedaddiConalguidanceformentalmodelflawsinSTPA;canbeappliedtoCAST

*ThispresentaConusesmentalmodelinplaceofprocessmodelwhenreferringtohumancontrollers.

1.SafetyRoles&Responsibili&es

2.UnsafeControlAc&ons(UCAs)

3.MentalModelFlaws*

4.ContextualFactors

8

Foundation of Mental Model Guidance

STPA–EngineeringforHumansExtension

MentalModels

ProcessState

ProcessBehavior

Environment&Controllers

MentalModelUpdates

ControlAc&onSelec&on

SensoryInputsControl

AcCons

HumanController

(Thomas&France,2016;France,2017)

9

Guidance for Mental Models

MentalModelofProcessStateq  Beliefsaboutmodesandmodechangesq  Beliefsaboutthecurrentprocessstage

(forprocesseswithmulCplestages)q  Beliefsaboutsystemvariables(e.g.true/

false,on/off)

(France,2017)

MentalModels

ProcessState

ProcessBehavior


10


MentalModelofProcessBehaviorq  Beliefsaboutwhatthesystemcandoq  Beliefsabouthowthesystemwillbehave

inaparCcularmodeorprocessstageq  Beliefsaboutif-thenrelaConshipsbetween

operatorinputandsystemoutput

MentalModels

ProcessState

ProcessBehavior


(France,2017)

11


MentalModelofEnvironment&Controllersq  ChangestoenvironmentalcondiConsq  Familiarorunfamiliarenvironmentsq  Othercontrollers’statesandbehaviorsq  SocialandorganizaConalfactors

MentalModels

ProcessState

ProcessBehavior


(France,2017)

12

Additional Guidance: Contextual Factors

Contextualfactorsisabroadcategory,leavinganalystswithpotenCalquesCons:q  WhichtypesofinformaConbelongin

“contextualfactors”?q  Howshouldfindingslistedunder

“contextualfactors”bestructured?



3.MentalModelFlaws

4.ContextualFactors

13

Foundations of Contextual Guidance

(Leveson,2004;adaptedfromRasmussen,1997andRasmussen&Svedung,2000)

(Safar,Roth,&Multer,2015)

*Note:“PASS”standsfor“passingastopsignal”andissynonymouswith“StopSignalOverrun”orSSO.

*

14

Guidance for Contextual Factors

Fivecategoriesofcontextualfactorstoconsider:q  Physicalsystemandtechnologyfactorsq  Individualandteamfactorsq  OrganizaConalfactorsq  Regulatoryfactorsq  External/Environmentalfactors;“Other”

NOTE:ThisguidanceDOESNOTreplacetheneedtocreateasafetycontrolstructureandexaminethecontrolandfeedbackloopsspecifictoYOURsystemandcontrolledprocess!ThisisNOTachecklist;it’sarecommendaNontoconsiderfactorsatalllevelsofyoursystem!

15


PhysicalSystemandTechnologyFactorsq  OperaCngenvironmentdesignq  Interfacedesign(e.g.displaysandalerts)q  Maintenance/operaConalstatusofphysicalsystemsq  Availabilityornon-availabilityofjobaidsq  Otherphysicalfactors(e.g.weather)

16


IndividualandTeamFactorsq  CommunicaConandteamwork;coordinaConq  DistracCons/compeCngdemandsforaoenConq  Experiencelevel;qualificaConandtrainingq  FaCgue;workscheduleq  Medicalfitnessfordutyq  ExpectaCons;similarsituaConsencountered

17


Organiza&onalProcessesq  SupervisoryprioriCes/safetycultureq  Resourceconstraints&producConpressuresq  Policiesandprocedures(workschedules,training,

discipline,etc.)q  Degreeoffeedbackfromemployees

18


RegulatoryAc&vi&esq  Degreeofsupportto/controloverorganizaConsq  Feedback(data)collectedfromorganizaConsq  RegulaConsregardingemployeesq  RegulaConsregardingphysicalsystemsand

technologies

19


ExternalFactorsq  High-levelsocietal,governmental,etc.influencesq  Economiccontext,fundingsourcesq  DemandsforservicedrivingproducConpressuresq  PoliCcalclimate’seffectsonfunding,regulaCon,etc.

REMINDER:Theselistsarenotcomprehensive,theyaresimplyasetofexamples!

20

Complete CAST Controller Guidance

3.ProcessModel(MentalModel)FlawsExplainwhyUnsafeControlAcNonsappearedappropriatetothecontroller.

A.  MentalmodelofprocessstateB.  MentalmodelofprocessbehaviorC.  Mentalmodelofenvironment

4.ContextualFactorsSystemicfactorsthatinfluencecontrollers’mentalmodelsanddecisions.

A.  PhysicalsystemandtechnologyfactorsB.  IndividualandteamfactorsC.  OrganizaConalprocessesD.  RegulatoryacCviCesE.  Externalfactors


2.SafetyConstraintsViolatedorUnsafeControlAc&ons(UCAs)

21


22

Head-On Collision of Two Union Pacific Railroad Freight Trains

q  3fataliCes§  Eastboundtrain’s

engineer&conductor§  Westboundengineer

q  1survivor§  Westboundconductor

q  5locomoCvesand32carsderailed

q  $14.8millioninesCmateddamage

Goodwell,OklahomaJune24,2012,10:02a.m

Photosource:theGuymonDailyHeraldcourtesyofMaritEdwards

(NTSB,2013)

23

Accidents and Hazards

ACCIDENTq  Atraincollideswithanothertrainonthesamestretchof

track

HAZARDSq  Atrainentersanareaoftrackitwasnotclearedtoenterq  Atrainentersanareaoftrackalreadyoccupiedbyanother

train

24

System Safety Constraints

q  Dispatchersmustusestopandapproachsignalstoalerttraincrewstoareastheydonothaveclearancetoenter

q  Traincrewsmustslowdownatapproachsignalsandstopatstopsignals

q  TraincrewsmusthaveadequatevisualacuityandcolorvisiontorecognizesignalindicaCons

q  Railroadsmustensurethatcrewsareadequatelyqualifiedandmedicallyfitforduty

25

GoodwellSiding

Theeastboundtrainwassupposedtowaitatthestopsignalforthewestboundtraintoenterthesiding.

ZLAAH22EastboundTrain

AAMMLX22WestboundTrain

Event Overview: Planned Route

26

GoodwellSiding

Theeastboundtraincouldthenproceededforwardsafely,followedbythewestboundtrain.

Event Overview: Planned Route



27

GoodwellSiding

Instead,theeastboundtrainpassedthestopsignalandcollidedwiththewestboundtrainbeforeitreachedthesiding.


Event Overview: Actual Route


Run-throughswitch

28

Event Timeline: Eastbound ZLAAH 22

StopSolidred

StopunClgiven

clearancetoproceed

Approx.2miles

AdvancedApproachFlashingyellow

Reducespeedto40mph

9:56am

Approx.2miles

ApproachSolidyellowoversolidred

Reducespeedto30mph;

preparetostop

9:58am 10:00am

Collision

<2miles


10:01am

29

Event Timeline: Eastbound ZLAAH 22

“…the engineer appeared to make throttle and dynamic brake adjustments that maintained train speed close to the 70 mph limit, as would be expected for a train operating on a clear signal”

(NTSB,2013)

30

Physical System Analysis

SafetyControlsandEquipmentq  Stopsignalsindicatewheretrainsmustnotgoq  “Approach”signalswarnofupcomingstopor

speedrestricConsq  Sidingsallowtrainstopasssafelyonsingletrack

SafetyConstraintsViolatedq  TwotrainsmovingoppositedirecConswere

allowedonasingletrack

(NTSB,2013)

31

Physical System Analysis

FailuresandUnsafeInterac&onsq  SignalsfuncConedasintended,butthe

eastboundtraindidnotobeythemq  NoPosiCveTrainControl(PTC)present

ContextualFactorsandAddi&onalQues&onsq  Individual:IfsignalswereproperlyfuncConing,

whydidtheengineerpassastopsignal?q  OrganizaConal:Whywasn’tposiCvetrain

controlinstalled?

(NTSB,2013)

32

Controller Analysis: Engineer

Speedcontrols(throole&brake)

CurrentspeedSignalindicaCon

Engineer

Physical System (Trains and Signals)

33


RolesandResponsibili&esq  Controltrainspeedinaccordancewithpostedsignals;

calloutsignalindicaContoconductorq  BemedicallyfittooperatealocomoCveUnsafeControlAc&onsq  Didnotslowdownasrequiredbyapproachsignalsor

stopatthestopsignalq  Operatedwhileknowinghehadinconsistentcolorvision

anddiminishedvisualacuity

34


MentalModelFlaws

ModelofProcessState:q  Incorrectlybelievedsignalswere“clear”(green)

ModelofProcessBehavior:q  N/A–understoodmeaningofsignals&requiredacNons

ModelofEnvironment&Controllers:q  Mayhaveincorrectlybelievedconductorwouldinterveneq  Likelyunawareofthepresenceofanothertrain

35


IndividualContextualFactors:q  PerceptuallimitaCons:engineerhadseverevisualimpairments

frommulCplecondiCons;hadfailedacolorvisiontestin2009q  CoordinaCon:shouldhavebeencoordinaCngwithconductorto

calloutsignalsandensureproperspeed;whatwasconductordoing?

q  ExpectaCons:signalsmaybetypicallyclear,high-visibilityq  FaCgue:faCguepossiblebasedonirregularworkschedule

Organiza&onalContextualFactors:q  WhatorganizaConalfactorsallowed(orincenCvized)operaCng

despiteseverevisualseverelimitaCons?

36

ConductorEngineer


Controller Analysis: Conductor

Emergencybrake

SignalindicaConsCurrentspeed

Calloutsignals

37


RolesandResponsibili&esq  CalloutsignalindicaCons;ensureengineerobeyssignals;useemergencybrakeifnecessary

UnsafeControlAc&onsq  Apparentlydidnotcallouttheapproachandstopsignals;didnotwarnengineer

q Didnotpulltheemergencybrake

38


ProcessModelFlawsModelofProcessState:q  WaslikelyunawarethatsignalswereinarestricCvestate

ModelofProcessBehavior:q  N/A–wouldhaveunderstoodmeaningofsignals

ModelofEnvironment&Controllers:q  Waslikelyunawarethattheengineerwasnotrespondingto

signals;likelybelievedengineercouldreadthesignals

39

Controller Analysis: Conductor IndividualContextualFactors:q  DistracCon/inaoenCon:conductormayhavebeenasleeporabsentq  ExpectaCons:signalsmaybetypicallyclear,high-visibilitycondiCons,

mayhavebelievedengineerwascapableofrecognizingsignalsq  FaCgue:faCguepossiblebasedonirregularworkscheduleq  CoordinaCon:engineermayhaveallowedconductortorest

Organiza&onalContextualFactors:q  WhatorganizaConalfactorscontributedtolackofrequired

coordinaConbetweentheengineerandconductor?

40

ConductorEngineer


Dispatcher

Train Crew

Controller Analysis: Dispatcher

Clearances&instrucCons

LocaCon/issues

Setsignals&switches

Signal&switchstatusTrainposiCons?Trainspeeds?

41


RolesandResponsibili&esq  SetsignalsandswitchesforsaferouCngoftrainsq  CommunicaterouteinformaContotraincrewsUnsafeControlAc&onsq  Apparentlydidnotwarneastboundtraincrewabouttheupcomingstopsignal&othertrain

42


ProcessModelFlawsModelofProcessState:q  Didnotrealizeeastboundtrainhadpassedthesignals

ModelofProcessBehavior:q  Mayhavebelievedthatifcrewspassedasignal,hewouldbe

alertedandhaveCmetointervene

ModelofEnvironment&Controllers:q  Assumedcrewwouldobeysignalsevenifnotgivenwarning

43

Controller Analysis: Dispatcher PhysicalSystemContextualFactors:q  DispatcherwasnotnoCfiedthatthetrainwasoverthespeedlimitq  DispatcherwasonlynoCfiedoncetheswitchattheendofthe

sidingwaspushedoutofalignmentbytheeastboundtrain

IndividualContextualFactors:q  ExpectaCons:didnotexpectcrewstopassastopsignal(veryrare)q  CoordinaCon:dispatcherwasresponsiblefor10-12crewsq  Timing:accidentoccurredapprox.5minsarerfirstmissedsignal,

andlessthan2minutesarerthedispatcherreceivedanalert

44

Controller Analysis: UP Management Railroad Management

ConductorEngineer


Outside Medical Providers Dispatcher

Train Crew

DocumentaConofVisionTestResults

Diagnoses&TreatmentsofEyeCondiCons

VisualAcuity

Training,Policies,

etc.

Safetyissues

Training,Policies,

etc.

Safetyissues

45

Controller Analysis: UP Management

RolesandResponsibili&esq  Ensurethatemployeesaremedicallyfitfordutyand

adequatelytrainedonjobtasks,includingcoordinaConq  EnsurethatphysicalsystemcomplieswithregulaConsUnsafeControlAc&onsq  DidnothavePTCinstalledontherouteq  Didnotprovidetrainingoncrewresourcemanagementq  DidnotrequiredocumentaConofengineer’svisual

acuityresults;didnotrequirefollow-uptesCngq  Usedacolorvisiontestofunknownvalidity/reliability

46

Controller Analysis: UP Management

MentalModelFlaws

q  Mayhaveincorrectlybelievedengineer’svisionwasokayq  Mayhavebelievedthatengineer’svisionwouldnotbea

problemwiththeconductorassisCngbycallingoutsignalsq  Mayhavebelievedresourceswerenotavailableto

implementsafetymeasures,orthatsafetymeasureswerenotasurgentasotherprioriCes(CRM,PTC)

47

Controller Analysis: UP Management Organiza&onalContextualFactors:q  Resourceconstraints:PTC/CRMrequirelargeamountofresourcesq  Staffingconstraints:mayhaveneededengineersbadly,decidedto

keeptheengineerdespitevisuallimitaConsq  Policies:didnotuniformlyapplypolicyforverifyingvisualacuitywas

toobtainwrioendocumentaConq  SchedulingpracCces:irregularschedulesarecommoninrail

industry;maycontributetofaCguedespitehoursofservicelimitaCons

RegulatoryContextualFactors:q  WasUPinviolaConofregulaCons,orwereregulaConsinadequate?

48

Controller Analysis: FRA

Railroad Management

ConductorEngineer



Train Crew

Federal Railroad Administration

RegulaCons Safetydata

Railroad Management

ConductorEngineer



Train Crew

49

Controller Analysis: FRA

RolesandResponsibili&esq  RegulaterailroadstoensuresafetystandardsUnsafeControlAc&onsq  AllowedretakingvisiontestswithoutvalidaCngrailroad’stesCngmethods

q DidnotmandatePTCinstallaConsoonerq DidnotmandateCRMtrainings

50

Controller Analysis: FRA ProcessModelFlawsq  BelievedCmelineforPTCimplementaConwasappropriateq  BelievedrailroadshadadequateresourcesforCRMq  BelievedrailroadswoulduseavalidvisiontesCngmethod

RegulatoryContextualFactors:q  FRAalreadyrequiredimplementaConofPTCby2015q  FRAconductedresearchintoCRM,providedrailroadswithfundingforpilot

programs

ExternalFactorsq  DesiretomaintainposiCverelaConshipswithrailroads&workwithintheir

resourceconstraints(e.g.PTCregulaConCming)

51


52

Enhanced CAST Controller Guidance

q  ProvidesaddiConalguidanceonwhattypesofcontenttoinclude

q  UsefulfornewpracCConersofCAST

q  RevealedinteresCngresultsinourfreightcasestudy

3.ProcessModel(MentalModel)Flaws

A.  MentalmodelofprocessstateB.  MentalmodelofprocessbehaviorC.  Mentalmodelofenvironment

4.ContextualFactors

A.  PhysicalsystemandtechnologyfactorsB.  IndividualandteamfactorsC.  OrganizaConalfactorsD.  RegulatoryfactorsE.  External/environmentalfactors;“other”



53

References

q  France,M.,“EngineeringforHumans:ANewExtensiontoSTPA,”Master’sThesis,MassachuseosInsCtuteofTechnology,2017.

q  France,M.,“EngineeringforHumans:Human-AutomaConInteracConinSTPA,”presentedatthe6thAnnualMITSTAMPWorkshop,2017.

q  Leveson,N.G.“ANewAccidentModelforEngineeringSaferSystems.”SafetyScience,vol.42,no.4,pp.237-270,2004.

q  Leveson,N.G.,“EngineeringaSaferWorld:SystemsThinkingAppliedtoSafety.”TheMITPress,2012.q  NaConalTransportaConSafetyBoard,“HeadonCollisionofTwoUnionPacificRailroadFreightTrainsNear

Goodwell,Oklahoma,June24,2012.”NTSB/RAR-13/02.Washington,DC:NTSB,2013.q  Rasmussen,J.,“RiskManagementinaDynamicSociety:AModellingProblem,”SafetyScience,Vol.27,No.

2/3,pp.183-213,1997.q  Rasmussen,J.&Svedung,I.“ProacCveRiskManagementinaDynamicSociety.”SwedishRescueServices

Agency,2000.q  Safar.H.,Multer,J,.&Roth,E.“AnInvesCgaConofPassingStopSignalsataPassengerRailroad”

Washington,DC:FederalRailroadAdministraCon,2015.q  Safar.H.,Multer,J,.&Roth,E.“Whydopassengertrainspassstopsignals?Asystemsview”Washington,

DC:FederalRailroadAdministraCon,2017.q  Thomas,J.andM.France.“EngineeringforHumans:STPAAnalysisofanAutomatedParkingSystem,”

presentedatthe5thannualMITSTAMPWorkshop,2016.

54

ThankyouforyouraoenCon!Contact:MeganFrance,[email protected]

55

BackupSlides:PTCImplementaCon

56

Positive Train Control Timeline

Source:hops://www.fra.dot.gov/ptc

57

PTC Implementation in Freight RRs


58

PTC Implementation at UP

AsofDecember31,2017


59

Ongoing/Future Work on SSOs

q  Currentstudyfocusesonfreightenvironment§  CASTanalysesareapreliminarystep§  Follow-upwithinterviews,focusgroups,etc.andbriefingtorailroad

management

q  Otherongoingwork§  ImprovingSSOdatacollecConusingacommonformtemplate§  CommunicaCngfindingstorailroadsina“GoodPracCceGuide”

Documents

New Guidance for CAST: Case Study of a US Freight Rail ...psas.scripts.mit.edu/home/wp-content/uploads/2018/04/Tues-Franc… · New Guidance for CAST: Case Study of a US Freight Rail