COMPARATIVE REPORT, Volume 1 - ihst.org Comparative Accident Report Vol 1.pdf · COMPARATIVE REPORT, Volume 1 ... Joint Aircraft System/Component (JASC) Codes ... Mark Colborn Dallas

COMPARATIVEREPORT,Volume1

U.S.JHIMDATDatatoU.S.JHSATData

U.S.JointHelicopterImplementationMeasurementDataAnalysisTeam

To

TheUnitedStatesHelicopterSafetyTeam

March2014

TheU.S.JHSATanalyzed523U.S.registeredhelicopteraccidentsthatoccurredinCY2000,CY2001andCY2006(JHSATdata).TheU.S.JHIMDATanalyzed415U.S.registeredhelicopteraccidentsthatoccurredinCY2009‐2011(JHIMDATdata).ThisreportcontainsthecomparativeJHIMDATdatatoJHSATdataanalysis.Thepurposewastoidentifydifferencesofstatisticalsignificance.Bynotingareasofimprovement,regression,andstagnation,thisanalysisallowedtheopportunityforastrategicapproachtofuturehelicopteraccidentratereductionefforts.

1

TableofContents

U.S.JHIMDATMembers.........................................................................................................................................................3

Description.................................................................................................................................................................................4

Industry.......................................................................................................................................................................................6

Activity.........................................................................................................................................................................................7

IndustryandActivityPairs..................................................................................................................................................8

OccurrenceCategory............................................................................................................................................................10

Sub‐OccurrenceCategory...................................................................................................................................................12

JointAircraftSystem/Component(JASC)Codes......................................................................................................14

InitiatorofSystemComponentFailure(SCF)............................................................................................................15

GeneralAccidentCharacteristics....................................................................................................................................17

ProportionsofFatalandNon‐FatalAccidents..........................................................................................................18

AccidentsbyPilot’sFlightHours....................................................................................................................................19

SummaryandConclusions................................................................................................................................................22

References................................................................................................................................................................................24

2

ListofTables

Table1.IndustryComparison...........................................................................................................................................6Table2.ActivityComparison.............................................................................................................................................7Table3.IndustryandActivityPairsComparison.....................................................................................................9Table4.OccurrenceCategoryComparison...............................................................................................................11Table5.Sub‐OccurrenceCategoryComparison......................................................................................................13Table6.JASCCodes..............................................................................................................................................................14Table7.InitiatorofSystemComponentFailures(SCFs).....................................................................................16

ListofFigures

Figure1.DistributionofAccidents‐Pilots'Make/ModelFlightHours.........................................................20

3

U.S.JHIMDATMembersThefollowingJHIMDATMemberorganizedthedata,conductedthecomparativeanalysis,andwrotethecomparativereportbetweentheJHIMDATandJHSATdata.

Name Company/OrganizationLeeRoskop FAARotorcraftStandardsStaff

ThefollowingJHIMDATMemberscontributedtooneorallofthefollowingactivities:analysisoftheJHIMDATdata,reviewofthecomparativereport,anddevelopmentofsummaryandconclusions.

Name Company/Organization PositionSteveGleason SikorskyAircraftCorporation JHIMDATCo‐Chair(Industry)ScottTyrrell FAARotorcraftStandardsStaff JHIMDATCo‐Chair(Government)EricBarnett FAARotorcraftStandardsStaff MemberScottBurgess Embry‐RiddleAeronauticalUniversity MemberTomCaramancio BoeingMilitaryAircraft MemberMarkColborn DallasPoliceDepartment MemberMunroDearing NASA MemberRoyFox BellHelicopterTextron‐Retired MemberGaryHowe BellHelicopterTextron MemberLeeRoskop FAARotorcraftStandardsStaff MemberBillWallace BillWallace&Associates Member

4

DescriptionIn2006,theU.S.JointHelicopterSafetyAnalysisTeam(JHSAT)begananalysisworkonU.S.accidentsthatoccurredincalendaryear2000.Afterpublishingareportwiththeresultsoftheirwork,theJHSATproceededtocompletecomparableanalysisontheU.S.accidentsthatoccurredin2001andfollowedcompletionofthatprojectwithanalysisof2006.In2011,theJHSATconsolidatedtheresultsofallthreerespectiveyearsintoacompendiumreport(hereafterknowninthisreportas“JHSATdata”).TheJHSATdataestablishedabaselineforfutureU.S.helicopteraccidentanalysis.

Asafollowuptothecompendiumreport,theU.S.JointHelicopterImplementationMeasurementandDataAnalysisTeam(JHIMDAT)attemptedtomeasureprogressbycomparingtheJHSATdatatoanalysisperformedonthethreecalendaryears2009‐2011(hereafterknowninthisreportas“JHIMDATdata”).AnalysisoftheJHIMDATdataremainedconsistentwiththeJHSAT’smethodology.However,theextensivedepthofanalysisperformedontheJHSATdatarequiredmonthlymeetingsofoneweekdurationthatwerenolongerfeasibleforanalysisoftheJHIMDATdata.TheJHIMDATtransitionedtoquarterlymeetingsconsistingofamuchsmallerteamcompositionthantheoriginalJHSAT.Toaccountfortherealitiesofasmallerteamthatmetlessfrequently,theJHIMDATreducedthescopeoftheanalysiseffort.Theteamaccomplisheda“highlevel”analysisofthethreeyearsfrom2009‐2011throughquarterlymeetingsconductedovera9monthperiod.

TheJHIMDAT’spreferencewouldhavebeentouseratebasedcomparisonsintheiranalysis.Theratewouldhavebeencalculatedusingaccidentcountsasthenumeratorandflighthoursasthedenominator.However,whileitwaspossibletomakesomereasonableestimatesforthetotalU.S.rotorcraftflighthours,itwasnotpossibletoaccuratelyrefinethatflighthourestimateacrossthenumerousdifferentanalysisareasusedbytheJHIMDAT.Withthislimitation,theJHIMDATmadecomparisonsintheirresearchbasedonaccidentcountsthatwereconvertedtopercentagestoexpressfrequency.

TheJHIMDAT’sintentwastofindwheremeaningfuldifferencesexistedbetweentheJHIMDATdataandtheJHSATdata.Areaswhereimprovementoccurredwerecandidatesforexploringsuccessfulimplementationmeasuresforuseinotherareaswhereprogresswaslesssubstantial.Areaswhereregressionoccurredorstagnationwasevidentwerecandidatesforadaptingdifferentinterventionstrategiestoencourageimprovement.Ineithercase,adjustmentscouldbemadetofurthertheprogresstowardthe80%accidentratedecreasesoughtbytheIHST.InitialcomparisonoftheJHIMDATdataandJHSATdatashoweddifferencesdidexist.However,theJHIMDATsoughttofurtheridentifyareaswherestatisticallysignificantdifferencesexisted.Theteamoptedforachisquaredanalysistosupportthispurpose,basedontheJHSAT’sandJHIMDAT’smethodoforganizingtheaccidentanalysisdataaccordingtoanominalscale.“Nominalscale”describesagroupingofdataintovariouscategoriesbasedonsomecommoncharacteristic.Apartiallistofexampleswouldincludethefollowing:1)“Industry”,suchasPersonal/Private,Instructional/Training,orAerialApplication;2)“Activity”descriptions,suchasPositioning/RTB,Instructional/Training(Dual),orMaintenance/Testflight;and3)“OccurrenceCategories”,suchasLossofControl,Autorotation,orSystemComponentFailure.

5

Thechisquaredanalysisconsistedofeithera“goodnessoffit”test(mostcases)ora“testofhomogeneity”(afewcases).ThegoodnessoffittestallowedtheteamtodirectlycompareonegroupofJHIMDATdatatothecorrespondinggroupofJHSATdatatoassessstatisticaldifferences.Thetestofhomogeneitywasaslightlymorecomplexapproachthattheteamusedforafewlimitedcaseswherethegoalwastoexaminetwocategories(e.g.,VMCorIMC)againsttwoothercategories(e.g.,fatalaccidentornon‐fatalaccident).ThetestofhomogeneitydidnotallowthesamedirectcomparisonofJHIMDATdatatoJHSATdataaswaspossiblewiththegoodnessoffittest.However,aviablecomparisonwasstillpossibleinthesecasesaccordingtothefollowingmethod.TheteamfirstappliedthetestofhomogeneitytoidentifywhetherastatisticaldifferenceexistedwithintheJHSATdata,thenappliedthesametestofhomogeneitytothecorrespondingcategorieswithintheJHIMDATdata,andconcludedbyassessingiftheresultsbetweenthetwodatasetsweresimilarordifferent.Alsoaspartofthechisquaredanalysis(forbothgoodnessoffittestandtestofhomogeneity),theteamoftenusedthestandardizedresidualtodeterminewhatspecificnominaldatawasamajorcontributortodrivingthestatisticaldifference.Forexample,ifastatisticaldifferencewasfoundintheareaofindustrysectors,thestandardizedresidualwouldidentifythespecificindustrysectorsmostresponsibleforthestatisticaldifference.

Foreachanalysisscenariointhisreport,theJHIMDATusedthechisquaredcriticalvaluesassociatedwithp=.01.The“pvalue”isassociatedwiththeprobabilityofrejectinganullhypothesisthatisactuallytrue.Thepvaluesassociatedwithmeasurementsofstatisticalsignificancearetypically.05orless,sotheJHIMDAT’suseofp=.01wasconsistentwithusingarigorousstandardtoassessstatisticalsignificance.InthecaseoftheJHIMDATandJHSATdatacomparison,thenullhypothesisineachcasewasthattherewasnotadifferencebetweenthenominaldatafromtheJHIMDATascomparedtothenominaldatafromtheJHSAT.Inordertorejectthenullhypothesisforanyoftheareasstudiedinthisreport,thecalculatedchisquaredvaluehadtoexceedthechisquaredcriticalvalueforp=.01.AnytimetheteamfoundastatisticaldifferenceintheJHIMDATandJHSATdata,useofp=.01meantthattherewasa1%(orless)probabilitythatourconclusionwasincorrectandastatisticaldifferencebetweenthedatadidnotexist.

AgoaloftheJHIMDATwastoproduceareportthatwassuccinctandeasilyreadsothatresultsfromthereportcouldbequicklytranslatedbythereadersintoimplementation.Wesoughttoavoidacumbersomeandoverlydetaileddocumentthatwouldneverberead.Consistentwiththisapproach,theteamseparatedthecomparativeanalysisintotwovolumes.TheintentwastoensureVolume1emphasizedthemostimportantaspectsofthecomparativeanalysis.TheteamintentionallytruncatedmostofthetablesinVolume1suchthatonlythemostfrequentlycitedareasintheanalysiswerelisted.Thedatasuggestedtheseareascouldhavethemostsignificantimpactonfutureaccidentreductionefforts.Volume2ofthecomparativeanalysishasthecompletedatatablesforthereaderwhowouldliketoexploretheanalysisatanadditionallevelofdetail.Volume2alsoincludessomedatatablesthatwerelimitedtoatextualsummaryinVolume1.

6

IndustryIndustrydescribedthetypeofworkoroperationwherethehelicopteristypicallyused.TheteamcategorizedeachaccidentfromboththeJHSATdataandJHIMDATdatabyIndustryintooneoffifteenpossibleoptions.Table1liststhefivemostfrequentlycitedIndustrycategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheIndustrycategoriesarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.

ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtoIndustry.

1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyIndustryagainsttheaggregategroupofJHSATaccidentscategorizedbyIndustry.Theconclusionwasthetwodatasetswerestatisticallydifferent.MajorcontributorstothestatisticaldifferenceweretheincreaseintheproportionofAerialApplicationaccidentsandthedecreasesintheproportionsofFirefighting,Logging,andElectronicNewsGatheringaccidents.

2) Theteamconductedaseparate,individualanalysisofeachspecificIndustrysegmentcomparingthetwodatasets.AsshowninTable1,AerialApplicationwastheonlyoneofthefivemostfrequentlyoccurringIndustrycategorieswhereastatisticallysignificantdifferenceexistedbetweentheJHSATdataandtheJHIMDATdata.Inthatparticularcase,theteamfoundastatisticallysignificantincreaseinaccidents.

Table1.IndustryComparison

20.7% (86) 18.5% (97)20.5% (85) 17.6% (92)15.7% (65) 10.3% (54)8.0% (33) 7.6% (40)7.5% (31) 7.5% (39)

DenotesstatisticallysignificantincreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata

DenotesstatisticallysignificantdecreaseinproportionofaccidentsfromJHSATdatatoJHIMDATdata

DenotesnostatisticallysignificantchangeinproportionofaccidentsfromJHSATdatatoJHIMDATdata

AerialApplication

CommercialEmergencyMedicalServices

JHIMDAT(CY09‐11):415Accidents JHSAT(CY00‐01,06):523AccidentsIndustry

Personal/PrivateInstructional/Training

7

ActivityActivityisdifferentfromIndustryinthatitdescribedthespecificfunctionthehelicopterwasengagedinatthetimetheaccidentactuallyoccurred.TheteamcategorizedeachaccidentfromboththeJHSATdataandJHIMDATdatabyActivityintooneoftwenty‐sevenpossibleoptions.Table2liststhefivemostfrequentlycitedActivitycategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheActivitycategoriesarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.

ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtoActivity.

1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyActivityagainsttheaggregategroupofJHSATaccidentscategorizedbyActivity.Theconclusionwasthetwodatasetswerestatisticallydifferent.AmongsomeofthemajorcontributorstothestatisticaldifferencewastheincreaseintheproportionofaccidentsinthefollowingActivities:Instructional/Training(Dual),Personal/Private,andInstructional/Training(Solo).

2) Theteamconductedaseparate,individualanalysisofeachspecificActivitycomparing

theJHIMDATdatatotheJHSATdata.TheresultsforthefivemostfrequentlyoccurringActivitiesareinTable2.BothInstructional/Training(Dual)andPersonal/Privateshowedstatisticallysignificantincreases.

Table2.ActivityComparison

19.3% (80) 14.0% (73)17.8% (74) 12.4% (65)13.7% (57) 13.2% (69)8.9% (37) 8.0% (42)5.3% (22) 3.3% (17)

JHIMDAT(CY09‐11):415Accidents

JHSAT(CY00‐01,06):523AccidentsActivity

Instructional/Training(Dual)Personal/Private


Positioning/RTBAerialApplication‐Spraying/DisbursingInstructional/Training(Solo)



8

IndustryandActivityPairsThecombinationoftheIndustrycategoryforeachaccidentandtheActivitycategoryforeachaccidentcanbepairedtogethertorefinetheunderstandingoftheaccidentsthathappened.Thispairingresultedinninety‐sevenIndustryandActivitycombinationsthatwereusedintheJHIMDATandJHSATanalysis.Table3liststhethirteenmostfrequentlyoccurringIndustryandActivitypairsfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheIndustryandActivitypairsarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.

TheteamaccomplishedtwoseparatechisquaredanalysesasrelatedtoIndustryandActivitypairs.

1) WeanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyIndustryandActivitypairsagainsttheaggregategroupofJHSATaccidentscategorizedbyIndustryandActivitypairs.Theconclusionwasthetwodatasetswerestatisticallydifferent.AmongsomeofthemajorcontributorstothestatisticaldifferencewastheincreaseintheproportionofaccidentsinthefollowingIndustryandActivitypairs:Personal/Private–Personal/Private,Instructional/Training‐Instructional/Training(Solo),andAerialApplication–Positioning/RTB.

2) Theteamattemptedaseparate,individualanalysistocomparethedatasetsforeachofthetopthirteenIndustryandActivitypairs.However,anaccurateindividualanalysiswasonlypossibleforsevenofthethirteenpairs.Fortheremainingsixpairs,somecalculationsassociatedwiththechisquaredanalysisinthesecasesresultedinnumbersbelowtheminimumacceptablevaluesnecessarytoensureanaccuratestatisticalcalculation.ThesixIndustryandActivitypairsexcludedfromtheanalysisareannotatedaccordinglyinTable3.TheonlystatisticaldifferenceamongthesevenpairsthatwereindividuallyanalyzedwasastatisticallysignificantincreaseinPersonal/Private–Personal/Private.

ThetenthmostfrequentlyoccurringIndustryandActivitypairfromtheJHIMDATdatashowninTable3wasAerialApplication/AerialApplication–Other.Sincethe“Other”designationisnondescriptwiththepotentialforvariousinterpretations,theteamdeterminedthefollowingclarifyingdefinitionwasappropriate.AerialApplication–OtherdescribedanAerialApplicationaccidentwhereitwaseitherunclearwhatactivitythepilotwasengagedinatthetimeofthecrashortheactivitythepilotwasengagedinwassoinfrequentitdidnotwarrantamorespecificActivitycategory.

9

Table3.IndustryandActivityPairsComparison

Activity

Personal/Private 17.8% (74) 12.0% (63)Instructional/Training(Dual) 15.2% (63) 13.6% (71)AerialApplication‐Spraying/Disbursing 8.9% (37) 8.0% (42)Instructional/Training(Solo) 4.3% (18) 2.7% (14)Positioning/RTB 4.1% (17) 5.5% (29)Positioning/RTB 3.4% (14) 1.1% (6) XPassenger/Cargo(Commercial) 2.7% (11) 3.3% (17)Instructional/Training(Dual) 2.7% (11) 0.4% (2) XExternalLoad‐Line 2.4% (10) 2.5% (13)AerialApplication‐Other 2.4% (10) 1.0% (5) XPassenger/Cargo(Commercial) 2.2% (9) 2.1% (11) XEmergencyMedicalServices 1.9% (8) 1.1% (6) XAerialObservation‐Fish/GameSpotting 1.9% (8) 1.0% (5) X

X

ExternalLoadAerialApplication

Instructional/Training




Excludedfromindividualchisquaredanalysis

JHIMDAT(CY09‐11):415Accidents

JHSAT(CY00‐01,06):523Accidents

Personal/PrivateInstructional/Training

AerialApplication

OffshoreEmergencyMedicalServices

AerialObservation

Industry

EmergencyMedicalServicesAerialApplicationCommercialLawEnforcement

10

OccurrenceCategoryOccurrenceCategoriesprovidedaconcisedescriptionof“whathappened”inanaccident.Table4liststheninemostfrequentlycitedOccurrenceCategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromJHSATdata.ThetableliststheOccurrenceCategoriesindescendingorderoffrequencybasedontheJHIMDATcolumn.

AchisquaredanalysisthatcomparedtheaggregategroupofJHIMDATtoJHSATaccidentsbyOccurrenceCategorieswasnotpossibleusingthesameformatastheIndustryandActivitydata.ThisisbecauseeachaccidentwasnotlimitedtoassignmentinasingleOccurrenceCategory.BoththeJHSATandJHIMDATmethodologyallowedeachindividualaccidenttobecodedwithuptofourdifferentOccurrenceCategories.ThisaspectofthemethodologyexplainswhythepercentagesofOccurrenceCategoriesfromeachdatasetshouldnotbesummedwiththeexpectationoftheresulttotaling100%.MultipleOccurrenceCategoriesperaccidentensuredthesummedtotalwillfarexceed100%.

However,theteamwasabletoperformaseparate,individualchisquaredanalysisforeachOccurrenceCategory.Foreachcase,thiswaspossiblebygroupingallaccidentswhereanOccurrenceCategoryhadbeenusedatleastonceintoonesetofdata(e.g.,allLossofControlaccidents),whilegroupingallaccidentswherethatsameOccurrenceCategorywasneverusedintoanothersetofdata(e.g.,allaccidentswhereLossofControlwasnotcited).TheresultsfortheninemostfrequentlyoccurringOccurrenceCategoriesareinTable4.LossofControlandControlledFlightIntoTerrain(CFIT)showedstatisticallysignificantincreaseswhileSystemsComponentFailure(SCF)showedastatisticallysignificantdecrease.

TheteamdecidedfurtherexplanationwasneededforoneoftheOccurrenceCategoriesinTable4.AlthoughAbnormalRunwayContact(ARC)islistedasthethirdmostfrequentlyassignedOccurrenceCategoryintheJHIMDATdata,itwasexcludedfromtheindividualchisquaredanalysis.TheteamassignedARCto24.6%oftheaccidentsintheJHIMDATdataascomparedto7.6%oftheaccidentsintheJHSATdata.ItwouldappeartheOccurrenceCategoryisaclearcandidateforassessingwhetherasignificantstatisticaldifferenceexisted.However,theJHIMDATattributedthisrelativelylargedifferenceinproportionstoadifferentunderstandingandassignmentoftheARCcategoryintheJHIMDATanalysisasopposedtohowitwaspreviouslyusedintheJHSATanalysis.TheJHIMDATdidnotthinktherewasanactualincreaseinthecasesofARC;rather,therewasadifferentapplicationoftheOccurrenceCategorybytheJHIMDAT.ThisledtoinconsistencyinhowitwasassignedintheJHIMDATdatawhencomparedtotheearlierJHSATwork.

11

Table4.OccurrenceCategoryComparison

47.5% (197) 41.5% (217)32.8% (136) 31.7% (166)24.6% (102) 7.6% (40)21.4% (89) 27.5% (144)19.8% (82) 16.4% (86)8.2% (34) 7.6% (40)8.0% (33) 10.7% (56)7.0% (29) 6.1% (32)6.7% (28) 3.1% (16)

X


LOC‐LossofControl

ARC‐AbnormalRunwayContact

VIS‐Visibility

AUTO‐Autorotation

SCF‐SystemComponentFailureSTRIKE

OccurrenceCategory JHIMDAT(CY09‐11):415Accidents

FUEL

AbnormalRunwayContactexcludedfromindividualchisquaredanalysis

FIRECFIT‐ControlledFlightintoTerrain




12

Sub‐OccurrenceCategoryTheSub‐OccurrenceCategoryservedthepurposeofprovidingamoredetaileddescriptionwithineachOccurrenceCategory.Table5liststheelevenmostfrequentlyassignedOccurrenceCategory/Sub‐OccurrenceCategorypairsfromtheJHIMDATdataalongsidetherespectivepercentagesfromJHSATdata.ThetableliststheOccurrenceCategory/Sub‐OccurrenceCategorypairsindescendingorderoffrequencybasedontheJHIMDATcolumn.

ForthesamereasonsnotedintheOccurrenceCategoryportionofthereport,achisquaredanalysisthatcomparedtheaggregategroupofJHIMDATtoJHSATaccidentsbyOccurrenceCategory/Sub‐OccurrenceCategoriespairswasnotpossible.

However,theteamdidperformaseparate,individualchisquaredanalysisforeachOccurrenceCategory/Sub‐OccurrenceCategorypair.Foreachcase,thiswaspossiblebygroupingallaccidentswhereapairhadbeenusedatleastonceintoonesetofdata,whilegroupingallaccidentswherethatsamepairwasneverusedintoanothersetofdata.TheresultsfortheelevenmostfrequentlycitedOccurrenceCategory/Sub‐OccurrenceCategorypairsareinTable5.Fourofthetopelevenmostfrequentlycitedpairsshowedastatisticallysignificantincreaseintheproportionofaccidents.SystemComponentFailure–HelicopterwastheoneIndustry/Activitypairthatshowedimprovementasevidentbythestatisticallysignificantdecreaseinaccidents.

AsalsonotedintheOccurrenceCategorysectionofthereport,theteamdidnotanalyzetheSub‐OccurrenceCategoryofAbnormalRunwayContact(ARC)forastatisticaldifference.Theexplanationisthesameaspreviouslynoted.TheteamdidnotthinktherewasanactualincreaseinthecasesofARC;rather,therewasadifferentapplicationoftheOccurrenceCategorybytheJHIMDATandthisledtoinconsistencyinhowitwasassignedbytheJHIMDATwhencomparedtotheearlierJHSATwork.

13

Table5.Sub‐OccurrenceCategoryComparison

24.6% (102) 7.6% (40) X22.9% (95) 21.8% (114)21.7% (90) 15.1% (79)11.8% (49) 10.9% (57)8.7% (36) 5.5% (29)8.7% (36) 4.0% (21)8.4% (35) 5.9% (31)8.0% (33) 10.1% (53)7.5% (31) 14.3% (75)6.7% (28) 3.1% (16)6.0% (25) 5.2% (27)

X

LossofControl‐PerformanceManagement

OccurrenceCategory/Sub‐OccurrenceCategory JHIMDAT(CY09‐11):415Accidents


AbnormalRwyContact‐AbnormalRwyContactAutorotation‐Emergency

Autorotation‐PracticeStrike‐LowAltitudeMissionStrike‐TakeofforLandingLossofControl‐DynamicRollover

Fire‐PostImpact



AbnormalRunwayContactexcludedfromindividualchisquaredanalysis

SystemComponentFailure‐EngineSystemComponentFailure‐HelicopterCFIT‐CFIT


14

JointAircraftSystem/Component(JASC)CodesTheoriginalJHSATdatadelineatedSystemComponentFailure(SCF)accidentsintooneoffourcategories:1)Engine,2)Helicopter,3)MissionEquipment,or4)Unconfirmed/Perceived.TheJHIMDATsoughttoexpandonthespecificityassociatedwithSCFaccidentanalysisthroughuseofJASCcodes.TheteamincludedaJASCCodeanalysisforthe89SCFaccidentsfromtheJHIMDATdata(comprising21.4%ofthe415totalaccidents)andthenreturnedtotheoriginalJHSATdataanddidlikewiseforthe144SCFaccidents(comprising27.5%ofthe523totalaccidents).Theteamusedthirty‐ninedifferentfourdigitJASCCodesintheJHIMDATdataandfifty‐sixdifferentfourdigitJASCCodesintheJHSATdata.

BecauseofthemanydifferentJASCCodesusedbetweenthetwodatasets,theteamhadtoconsolidatetheminordertoaccomplishanaccuratestatisticalcomparisonbetweentheJHIMDATandtheJHSATdata.WesortedthevarietyofJASCCodesintothreegeneralcategories.Table6liststheconsolidatedJASCCodecategoriesfromtheJHIMDATdataalongsidetherespectivepercentagesfromtheJHSATdata.TheJASCCodesarelistedindescendingorderoffrequencybasedontheJHIMDATcolumn.

ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtoJASCCode.

1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyJASCCodeagainsttheaggregategroupofJHSATaccidentscategorizedbyJASCCode.Theconclusionwasthetwodatasetswerenotstatisticallydifferent.

2) Theteamconductedaseparate,individualanalysisforeachoftheJASCCodecategoriesinTable6.EachoftheconsolidatedcategorieswascomparedtoallaccidentswherethatconsolidatedcategoryofJASCCodeswasnotused.AsshowninTable6,therewasnotastatisticallysignificantdifferencebetweentheJHSATdataandtheJHIMDATdataforanyofthethreecategories.

Table6.JASCCodes

55.1% (49) 45.8% (66)29.2% (26) 38.2% (55)15.7% (14) 16.0% (23)


PowerplantSystems(7000,8000series)

RotorSystems(6000series)

AirframeSystems(2000,3000,4000,5000series)

JHIMDAT(CY09‐11):89SCFAccidents

JHSAT(CY00‐01,06):144SCFAccidents

JASCDescription



15

InitiatorofSystemComponentFailure(SCF)TheJHIMDATobservedthateachSystemComponentFailure(SCF)hadaninitiatingevent.TheteamdesignatedthiseventastheInitiatorofSCFandfoundthatfourmajorcategoriesexisted:MaintenanceError,ManufacturingError,PilotError,orUnknown.Table7hasthefourcategoriesorganizedindescendingorderoffrequencybasedontheJHIMDATcolumn.IdenticaltotheJASCCodeanalysis,notethatthepercentagesshowninthetablearebasedonlyonthesubsetofSCFaccidents.Thisconsistedof89accidentsfromtheJHIMDATdata(21.4%ofthe415totalaccidents)and144accidentsfromtheJHSATdata(27.5%ofthe523totalaccidents).

ThereweretwoseparatechisquaredanalysesaccomplishedasrelatedtotheInitiatorofSCF.

1) TheteamanalyzedtheaggregategroupofJHIMDATaccidentscategorizedbytheInitiatorofSCFagainsttheaggregategroupofJHSATaccidentscategorizedbytheInitiatorofSCF.Theconclusionwasthetwodatasetswerestatisticallydifferent.MajorcontributorstothestatisticaldifferenceweretheincreaseintheproportionofaccidentswheretheInitiatorofSCFwasUnknownandthedecreaseintheproportionofaccidentswheretheInitiatorofSCFwasManufacturingError.

2) Theteamattemptedaseparate,individualanalysisforeachspecificInitiatorofSCFcomparingtheJHIMDATdatatotheJHSATdata.TheresultsareinTable7.Ofnote,however,isthattheteamhadtoexcludethePilotErrorcategoryfromindividualanalysis.ThiswasnecessarybecausesomecalculationsassociatedwiththechisquaredanalysiswiththePilotErrorcategoryresultedinnumbersbelowtheminimumacceptablevaluesneededtoensureanaccuratestatisticalcalculation.

ThestatisticallysignificantincreaseintheUnknowncategoryoftheJHIMDATdatainhibitedtheabilitytodrawmoremeaningfulconclusionsfromtheanalysis.TheUnknowncategoryeffectivelymasksthetrueproportionsofaccidentsinthemoredescriptivecategoriesassociatedwithInitiatorofSCF.Forexample,notethattheInitiatorofSCFsattributabletoManufacturingErrorshowedastatisticallysignificantdecreasebetweendatasets.However,itispossiblethedecreasewasartificial.WedidnotknowhowmanyoftheaccidentswithintheUnknowncategorymayhaveactuallybeenattributabletoManufacturingErrorifmoreinvestigativeinformationhadbeenavailable.So,ManufacturingErrormayhaveshownatruestatisticallysignificantdecrease,orthedecreasemayhavebeenartificialifmoreManufacturingErrorcaseswerehiddenintheUnknowncategory.ThissameconditioncouldhaveaffectedtheresultsoftheMaintenanceErrorandPilotErrorcategories.

16

Table7.InitiatorofSystemComponentFailures(SCFs)

44.9% (40) 18.1% (26)40.4% (36) 50.7% (73)7.9% (7) 21.5% (31)6.7% (6) 9.8% (14) X

X Excludedfromindividualchisquaredanalysis

PilotError




ManufacturingError

IniatorofSCF JHIMDAT(CY09‐11):89SCFAccidents

JHSAT(CY00‐01,06):144SCFAccidents

UnknownMaintenanceError

17

GeneralAccidentCharacteristicsTheJHIMDATalsocomparedthefollowingtraitsbetweenthetwodatasets:TypeEngineInstalled,AccidentsbyMonth,WeatherCondition,andLightCondition.

TypeEngineInstalledTheteamgroupedaccidentsfromboththeJHIMDATdataandtheJHSATdataintooneofthreecategoriesofinstalledengineorengines:turbinetwin,turbinesingle,orreciprocating.WeanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbyTypeEngineInstalledagainsttheaggregategroupofJHSATaccidentscategorizedbyTypeEngineInstalled.Theconclusionwasthetwodatasetswerestatisticallydifferent.Majorcontributorstothestatisticaldifferenceweretheincreaseintheproportionofaccidentswhererotorcrafthadareciprocatingengineinstalledandthedecreaseintheproportionofaccidentswhererotorcrafthadtwinturbineenginesinstalled.AccidentsbyMonthTheteamgroupedaccidentsfrombothdatasetsbasedonthemonthofoccurrence.WeanalyzedtheaggregategroupofJHIMDATaccidentsascategorizedbymonthofoccurrencecomparedtotheaggregategroupofJHSATaccidentscategorizedbymonthofoccurrence.Therewasnotastatisticaldifferencebetweenthetwodatasets.Theteamalsoattemptedaseparate,individualanalysisofaccidentsbymonthforeachofthefivemostfrequentlycitedIndustrycategories:1)Personal/Private,2)Instructional/Training,3)AerialApplication,4)EMS,and5)Commercial.However,anaccurateindividualanalysiswasonlypossibleforInstructional/Training.FortheremainingfourIndustrycategories,somecalculationsassociatedwiththechisquaredanalysisforeachcaseresultedinnumbersbelowtheminimumacceptablevaluesnecessarytoensureanaccuratestatisticalcalculation.Instructional/TrainingshowedastatisticallysignificantdifferencefromtheJHSATdatatotheJHIMDATdata.ThemajorcontributorstothestatisticaldifferenceinInstructional/TrainingaccidentsbymonthwereanincreaseintheproportionofaccidentsinOctoberandadecreaseintheproportionofaccidentsinDecember.WeatherConditionTheteamgroupedaccidentsfromboththeJHIMDATandtheJHSATaccordingtowhethertheaccidentoccurredinVisualMeteorologicalConditions(VMC)orInstrumentMeteorologicalConditions(IMC).TherewasnotastatisticallysignificantdifferenceintheproportionofVMCandIMCaccidentsbetweenthetwodatasets.LightConditionTheteamgroupedaccidentsfrombothdatasetsaccordingtowhethertheaccidentoccurredduringthedayorduringthenight.Wecategorizedthefollowinglightconditionsasnightforpurposesoftheanalysis:dawn,dusk,twilight,nightbright,andnightdark.TherewasnotastatisticallysignificantdifferenceintheproportionofdayandnightaccidentsfromtheJHSATtotheJHIMDATdata.

18

ProportionsofFatalandNon‐FatalAccidentsTheteamselectedseveraldifferentdatacategoriestoanalyzewhetherastatisticaldifferenceexistedintheproportionsoffatalversusnon‐fatalaccidentsintheJHIMDATdataascomparedtotheJHSATdata.Foreachdatacategoryselected,theteamperformedaseparatechisquaredanalysis.Thecategoriesselectedwere:

AccidentsbyOccurrenceCategory VMCOnlyAccidents IMCOnlyAccidents AccidentsbyPilot’sMake/ModelFlightHours

Regardlessofthedatacategoryusedforanalysis,theconclusionwasthesame.Theproportionsoffatalversusnon‐fatalaccidentsintheJHIMDATdatawerenotstatisticallydifferentwhencomparedtotheJHSATdata.

19

AccidentsbyPilot’sFlightHoursTheteamanalyzedaccidentsfromtheJHIMDATdataandtheJHSATdatausingtwodifferenttypesofpilot’sflighthours:rotorcraftflighthoursandmake/modelflighthours.Amajorlimitationwasassociatedwithdrawingconclusionsrelatedtotheproportionofaccidentsascategorizedbypilot’sflighthoursinaspecificflighthourincrement.Thelimitationwasthattheactualflighthourswereunknownforthenon‐accidentU.S.helicopterpilots.Historicalstudiesthatpre‐dateIHSTsuggestedthepercentageofhelicopteraccidentsassociatedwithaspecificincrementofpilot’sflighthourshaveahighpositivecorrelationtothepercentageofthetotalpopulationofhelicopterpilots(accidentpilotsandnon‐accidentpilots)withineachcorrespondingpilot’sflighthourincrement.Ifthesepasthistoricalstudiescontinuetobereliable,thefollowingexampleillustratestheimplications.Itwouldbepossiblethatifthepercentageofaccidentsistwiceashighforpilotswith0‐500hoursrotorcrafthoursascomparedtopilotswith501‐1,000rotorcrafthours,thereasonmaybeattributabletothetotalpopulationofpilotswith0‐500rotorcrafthoursbeingtwiceaslargeasthepopulationofpilotswith501‐1,000rotorcrafthours.Thisconclusionisquitedifferentthanconcludinglowerflighthoursalone(lessflyingexperience)isthecauseofwhypilotswith0‐500rotorcrafthourshavetwiceasmanyaccidentsasthoseinthe501‐1,000hourincrement.Inthecontextofacomparativeanalysissuchasthisreport,thesamelimitationapplies.Theunderlyingreasonaspecificpilot’sflighthourincrementmayshowastatisticaldifferenceinproportionofaccidentsbetweentheJHIMDATdataandtheJHSATdatamaybebecausetheflighthourincrementsforthetotalpopulationofpilotsmayhavechanged.Insummary,anyconclusionsrelatedtoanincreaseordecreaseinthepercentageofrotorcraftaccidentsassociatedwithaspecificpilotflighthourincrementmustincludedeliberationoftheincreasesordecreasesassociatedwiththetotalpopulationofpilotsinthatsameflighthourincrement.RotorcraftFlightHoursTheteamanalyzedtheaggregategroupofaccidentsfromtheJHIMDATdatabyrotorcraftflighthourincrementsagainsttheaggregategroupofJHSATdatabyrotorcraftflighthourincrements.Weusedflighthourincrementsof500flighthoursfrom0upto7,500flighthours(e.g.0‐500hours,501‐1,000hours,1,001‐1,500hours,etc.).Therewereaccidentswherethepilothadgreaterthan7,500rotorcraftflighthours.However,thenumberofpilotspereach500hourincrementabove7,500rotorcraftflighthourswaslow.Thesecasescouldnotbeincludedintheanalysiswithoutsacrificingtheaccuracyofthestatisticaltest.Accordingtothechisquaredanalysis,therewasnotastatisticallysignificantdifferencebetweenthetwodatasetswhencomparingtheaccidentsbyrotorcraftflighthourincrements.AccidentswithPilot’sRotorcraftFlightHours<or=600HoursCaseswherethepilotintheaccidenthadlessthanorequalto600rotorcraftflighthourswereorganizedbyincrementsof50flighthoursfrom0upto600flighthours(e.g.0‐50hours,51‐100hours,101‐150hours,etc).Theteam’sinitialapproachwastoincludeallaccidentswherethepilot’srotorcraftflighthourswerebelow1,000hours.However,thenumberofpilotspereach50hourincrementbetween600and1,000rotorcrafthourswaslow.Thesecasescouldnotbe

20

includedintheanalysiswithoutsacrificingtheaccuracyofthestatisticaltest.Accordingtothechisquaredanalysis,therewasnotastatisticallysignificantdifferencebetweenthetwodatasetswhencomparingaccidentsbyrotorcraftflighthoursforpilotswith600hoursorless.

AccidentsbyPilot’sMake/ModelFlightHoursTheteamanalyzedtheproportionofaccidentscategorizedbypilot’smake/modelflighthoursforboththeJHIMDATandJHSATanalyses.Theanalysisconsistedoforganizingthemake/modelflighthoursbyincrementsof500flighthoursfrom0upto4,500flighthours.Therewereaccidentswherethepilotshadgreaterthan4,500make/modelflighthours.However,thenumberofpilotspereach500hourincrementabove4,500make/modelflighthourswaslow.Thesecasescouldnotbeincludedintheanalysiswithoutsacrificingtheaccuracyofthestatisticaltest.Accordingtothechisquaredanalysis,therewasnotastatisticallysignificantdifferencebetweenthetwodatasetswhencomparingtheaccidentsbymake/modelflighthourincrements.Figure1illustratesthedistributionofpilots’make/modelflighthours.Thefigurealsoincludestheaccidentswherethepilot’smake/modelflighthoursexceeded4,500flighthoursthatwereexcludedfromthechisquaredanalysis.Forpurposesofconsolidatingthedatainthefigure,allcaseswherethepilot’smake/modelflighthoursexceeded10,000flighthourswereincludedinonecategory,showninthetableas“10,001+”.Figure1.DistributionofAccidents‐Pilots'Make/ModelFlightHours

*45JHIMDATaccidentsand51JHSATaccidentsdidnothavepilotmake/modeltime

21

AccidentswithPilot’sMake/ModelFlightHours<or=1,000HoursTheteamanalyzedtheproportionofaccidentsfromboththeJHIMDATandJHSATanalysesforcaseswherethepilotintheaccidenthadlessthanorequalto1,000make/modelflighthours.Theanalysisconsistedoforganizingthemake/modelflighthoursbyincrementsof50flighthoursfrom0upto1,000flighthours.Accordingtothechisquaredanalysis,theJHIMDATdatawasstatisticallydifferentfromtheJHSATdata.Theflighthourincrementswherethenumberofaccidentschangedenoughtobeamajorcontributortothestatisticaldifferencewere:

FlightHourIncrement Change451‐500hours Increasedfrom2.0%to4.3%ofaccidents651‐700hours Increasedfrom1.6%to3.6%ofaccidents801‐850hours Increasedfrom0.7%to2.4%ofaccidents

22

SummaryandConclusionsInthecomparativeanalysisofU.S.helicopteraccidentsfromtheJHIMDATdata(2009‐2011)againsttheJHSATdata(2000,2001,and2006),theteamobservedthateitherstagnancyorregressionwasthemostevidentcharacteristic.Themostfrequentlyciteditemswereconsistentbetweenthedatasetsinmanyofthedifferentanalysisareas.IntheIndustryanalysis,thetopfiveindustriesfromTable1werethesameandappearedinthesamerankorderfromtheJHSATdatatotheJHIMDATdata:1)Personal/Private,2)Instructional/Training,3)AerialApplication,4)EmergencyMedicalServices,and5)Commercial.Thesefiveindustriesaccountedforabout70%ofallU.S.accidentsintheJHIMDATdata,soimplementationmeasuresmusttargetmoreimprovementintheseindustriestostimulatemoresignificantaccidentreductions.Regrettably,thestatisticalanalysisshowednosignificantdifferencebetweentheproportionsofaccidentsinfourofthetopfiveindustries,anindicationofstagnancy.TheonlyIndustrywheretheteamnotedastatisticallysignificantdifferencewasinAerialApplication,whereregressionoccurredasevidentbythesignificantlyhigherproportionofaccidents.TheIndustry/ActivitypairsfromTable3reinforcedtheobservationsabouthowalargevolumeoftheaccidentsaredisproportionatelyrepresentedbyasmallnumberofindustries.ThethreemostfrequentlycitedIndustry/ActivitypairsfromtheJHIMDATanalysisaccountedforover40%oftheaccidentsfrom2009‐2011.AccidentsfromthePersonal/PrivateIndustryengagedinaPersonal/PrivateActivityaccountedforthehighestpercentageofaccidentsobservedintheJHIMDATdataandshowedastatisticallysignificantincreaseintheproportionofaccidentsfromtheJHSATdata.ThesecondandthirdmostfrequentIndustry/ActivitypairingsintheJHIMDATdataweretiedtotheInstructional/TrainingandAerialApplicationIndustrysectors.BothofthesepairingsshowedstagnancyinthattherewasnostatisticallysignificantchangeintheproportionofaccidentsobservedintheJHIMDATdatawhencomparedtotheJHSATdata.AseparatebutstillperplexingproblemintheIndustryandActivitypairsishowfrequentlythePositioning/RTBActivityresultsinaccidentsforboththeEmergencyMedicalServiceandAerialApplicationindustries.Positioning/RTBinvolvesmovingthehelicopterfromonelocationtoanotherandisnottypicallycharacterizedbythesamelevelofcomplexityaseitheroftheseIndustrysectorswouldexperienceinexecutingtherestoftheirtypicalflightprofile.Yet,Positioning/RTBwaspreciselytheActivitythataccountedforthehighestnumberofaccidentsineachofthesetwoIndustrysectors.IntheOccurrenceCategoryanalysisfromTable4,therankorderforfiveoutofthetopsixOccurrenceCategoriesintheJHIMDATdatamatchedtheJHSATdataandprovidedfurtherevidencethattheareasinneedofthestrongestimplementationmeasureshadnotchanged.LossofControlcontinuedtohavethehighestfrequencyofoccurrenceandalsoshowedastatisticallysignificantregression.OneofthepositiveobservationsintheOccurrenceCategoryanalysiswasthattheSystemComponentFailurecategoryhadastatisticallysignificantimprovement.TheSub‐OccurrenceCategoryanalysisfromTable5revealedthiswasattributabletoastatistically

23

significantimprovementinthehelicopterrelatedSystemComponentFailures(versusthosethatwereenginerelated).Unfortunately,therestofthemostfrequentlycitedSub‐OccurrenceCategoriesshowedeitherstagnationorregression.The21%reductioninthenumberofaccidentsthatoccurredinthethreeyearscoveredbytheJHIMDATdata(415accidents)ascomparedtothethreeyearsintheJHSATdata(523accidents)isprogressandshouldnotbedisregardednorminimized.However,thecomparativeanalysisbetweenthetwodatasetsclearlyshowedthattheareasresponsibleforthemajorityofU.S.helicopteraccidentsrequiremoreattentioninordertoachieveamoresignificantreductionintheoverallnumberofaccidents.Akeyelementmissingfromthecomparativeanalysisbetweenthedatasetswasclearanswerstocritical“Why”questions.Forexample,“Whyisstagnancyandregressionsoprevalentintheanalysis?”or“Whyhaveimplementationmeasuresbeenlesseffectiveintheseareas?”TheoriginofthesequestionsisthatwewereuncertainastowhatextentanyofthetoolsandresourcesdevelopedbytheIHSThadeverreachedactualimplementationwithhelicopteroperators.Theambiguityleftopenseveralpossibilities.Perhaps,somedegreeofimplementationoccurredyetdidnotproduceanyimprovement.Unfortunately,thereisalsothepossibilitythatimplementationmayneverhaveoccurredatall.Therewasnoclearevidenceforeitherofthesetwoscenariosoranyothers,sothisreportchosenottospeculatefurther.However,theprevalenceofstagnancyandregressioninthecomparativeanalysismadeitclearthattheimplementationmeasuresforthehighestaccidentproducingIndustrysegmentsshouldbereevaluated.TheUSHSTmusteitherdevelopmorerigorousimplementationmeasuresorformulateamoreeffectiveplanforensuringthatoperatorsareactivelyusingtheimplementationresourcesalreadyavailable.

24

ReferencesU.S.JointHelicopterSafetyAnalysisTeam.(2011,August).TheCompendiumReport:TheU.S.

JHSATBaselineofHelicopterAccidentAnalysis,VolumeI.Retrievedfromhttp://www.ihst.org/portals/54/US_JSHAT_Compendium_Report1.pdf

U.S.JointHelicopterSafetyAnalysisTeam.(2011,July).TheCompendiumReport:TheU.S.JHSATBaselineofHelicopterAccidentAnalysis,VolumeII.Retrievedfromhttp://www.ihst.org/portals/54/US_JSHAT_Compendium_Report2.pdf

Documents

COMPARATIVE REPORT, Volume 1 - ihst.org Comparative Accident Report Vol 1.pdf · COMPARATIVE REPORT, Volume 1 ... Joint Aircraft System/Component (JASC) Codes ... Mark Colborn Dallas