Upload
clara-york
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
1JHSAT - June 2006
JHSAT Status Briefing to IHSTJune 29, 2006
Mark Liptak FAA ANE-110Jack Drake HAI
2JHSAT - June 2006
JHSAT Status Briefing
Objective:
Brief the IHST on the status of the JHSAT
Obtain IHST concurrence on key program elements, avoid surprises or questions downstream:
dataset selectionadequacy of team membershipscoring criteriageneral program timingoutreach approachuse of existing safety reports
3JHSAT - June 2006
JHSAT Status BriefingSlide Topics:
SynopsisCharterWeb Site Team CompositionDataset SelectionProcess Flow - Timing/ScheduleJHSAT Scoring CriteriaEstablishing Standard Problem StatementsScoring MethodInternational PartnersLeveraging Existing Safety ReportsIssues for IHST Awareness/Concurrence
4JHSAT - June 2006
Synopsis
The JHSAT team is fairly well solidified, 14 members, making progress on adapting the CAST/JSAT process for analysis of helicopter events. We’re off to a good start.
Team members are enthusiastic about the basic CAST process, believing it will produce high value results. Have engaged in many detailed discussions on dataset selection, event scoring, data quality, mission diversity, etc.
Recent meetings spent evolving and detailing the JHSAT process and analyzing helicopter accident event data. We have made significant progress, but its been more time consuming than expected.Looking for ways to boost efficiency.
Industry is stretched thin, IHST should consider reiterating it’sappreciation for the commitment needed to make this happen.
The following slides contain information on key JHSAT topics.
5JHSAT - June 2006
Charter
Final charter developed and agreed to by team, posted on JHSAT web site:
Goal: Provide a prioritized assessment of the most safety critical hazards to commercial, private and military rotorcraft in worldwide operations as derived from selected rotorcraft data sources.
Provide intervention strategies to the IHST and Joint Helicopter Safety Implementation Team (JHSIT) that maximize the likelihood of reducing worldwide helicopter accident rates by 80 percent by 2016.
Provide a JHSAT report documenting the team’s findings to the IHST by 1Q 2007.
6JHSAT - June 2006
Web Site
The JHSAT has established a web site at http://www.ihst.org/jhsat/jhsat_top.htmId=jhsat, password=helicoptersafe
This is an access controlled site, please limit to the IHST only.
The JHSAT is using it to post NTSB data, team analysis results, meeting minutes, agendas, action lists, etc.
7JHSAT - June 2006
JHSAT Team Composition
Mark Liptak (FAA ANE) JHSAT co-chair Jack Drake (HAI) JHSAT co-chair Barry Rohm (Rolls Royce) Ray Wall (Bristow)Roy Fox (Bell) Tony Alfalla (Sikorsky) Ed Stockhausen (Airmethods) Joe Syslo (Eurocopter)Laura Iseler (IHST) Sandy Hart (NASA)Clark Davenport (FAA ASW) Matt Rigsby (FAA ASW)Ann Azevedo (FAA Risk consultant) Steve Gleason (Schweizer)
JHSAT team membership is light on small helicopter experts. We have discussed participation with Robinson, they do not appear too enthusiastic. Schweizer has committed only as a part time participant. Industry is stretched thin and long term commitment is difficult. We could use additional support from the pilot/operational side and perhaps a training expert.
8JHSAT - June 2006
JHSAT Dataset Selection
The initial dataset selected for JHSAT analysis is year 2000 NTSBaccidents. This will serve as the basis for the 1Q 07 report out.
The JHSAT will conduct a detailed analysis of these accidents. There are approximately 190 accidents to be analyzed.
This NTSB dataset is nearly fully populated with final narrative causal/factual information. Also, less likely to encounter litigation constraints with this dataset.
Consideration of accident causal factor trends for the last 24 years gives the team high confidence that detailed analysisof year 2000 events will yield high value recommendations that will mitigate long standing trends (see next slide)
Future JHSAT activity will target year 2001, 2002, etc.
9JHSAT - June 2006
24 Years - Little Change
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Aircraft Powerplant Auxilary Equipment Human Performance Environment / Facilities Unknown
24 year look back – Relative stability in helicopter accident causal factors
Final causaldata not yet fullydeveloped
US dataset represents 50% of worldwide fleet
10JHSAT - June 2006
JHSAT Basic Process Flowchart
CharterDevelopment
EstablishTeam
Select Data Set
ReviewNTSB
Docket Data
DevelopEvent
Sequence
IdentifyProblems
(what/why)
Assign StdProblem
Statements
ScoreProblem
Validity &Importance
Identify InterventionStrategies
ScoreInterventionAbility/Usage
Prioritize byOverall
Effectiveness
TechnicalReview
&Expert
Validation
Conflicts? ReportResults
No
Yes
IHST
JHSIT
11JHSAT - June 2006
JHSAT Process
JHSAT is structured to be similar to the CAST/JSAT processwhich has been highly successful in the part 121 community
Basic tenets:
Engaging stakeholders/experts from the helicopter community
Findings based on real world helicopter accident data
Recommendations ranked by a structured scoring method
JHSAT is sticking to these basic tenets but making significant modifications elsewhere in the process. Our last two meetings have centered around completing foundational process work and processing the event data.
12JHSAT - June 2006
JHSAT Scoring CriteriaValidity (V) - How valid do we think the Problem Statement is in this accident based on the available accident information?
Range:Guess = 0 Absolutely Sure = 4
Importance (I)- How important is this particular Problem Statement in the causal chain in this accident?
Range:No importance = 0Cause of accident = 4
Ability (A) - How effective is this particular intervention in mitigating the problem statement of this accident?
Range:No effect on Problem Statement = 0 Would Prevent Problem Statement = 4
Usage (U) - How sure are we that this intervention will be utilized as expected in the real world.
Range:No Utilization = 0100% Utilization = 4
Applicability (App) – likelihood that the problem statement will be present in future aircraft/operations
Range:No Utilization = 0100% Utilization = 7
Importance (I) - How important is this particular Problem Statement in the causal chain in this accident?
Range:No importance = 0Cause of accident = 7
Ability (A) - How effective is this particular intervention in mitigating the cause or contributing factors of this accident?
Range:No effect on Problem Statement = 0 Would Prevent Problem Statement = 7
Confidence (C) - How strongly do we believe the intervention will perform as expected in the real world?
Range:No Confidence = 0 Complete Confidence = 7
JSAT Scoring Criteria
13JHSAT - June 2006
JHSAT Scoring Criteria
The scoring criteria used by the JHSAT are similar to those used in CAST/JSAT with the following exceptions:
JSAT’s Applicability was a forward look at fleet characteristics and assessing intervention effectiveness. JHSAT has eliminated this criterion due to the belief that future fleet mix and mission will not change enough to make this a meaningful issue to consider.
JHSAT has added a Validity criterion. This is a scoring factor based on the spectrum of documentation of helicopter accidents. Some are very well documented, others very poorly.
The Confidence and Usage criteria are similar, however some of the detailed definition language is still being worked.
14JHSAT - June 2006
Determination of Standard Problem Statements (SPS)
A key element to the JSAT process is establishing standard problem statements (SPS) when identifying specific causal factors in accidents. Correctly executed, this allows for identifying high value common threads between accidents, and thus identifies high value mitigation areas.
JHSAT is developing a standard problem statement list from three sources:
To jumpstart our process we drew from the SPS developed by the HAAT team(1). We also pulled SPS statements determined to be applicable from the CAST SPS list (~500 topics) These two sources gave us a starting point. As we analyze events we are continuing to populate our SPS list with unique helicopter problem areas as we encounter them.
(1) Helicopter Accident Analysis Team, NASA, circa 1997, early version of the Boeing APS/CAST JSAT process.
15JHSAT - June 2006
Scoring Method
The JSAT process developed and used an arithmetic relationship to rank recommendations by an Overall Effectiveness (OE) rating. Using the scoring criteria from the previous slide, the relationship is as follows:
OE = ((I x A) x 2/(I +A)) x C/6 x App/6
This relationship places weighting factors on the Importance and Applicability scoring criteria.
The JHSAT has not yet made a determination on whether a weighted arithmetic relationship is appropriate. We have intentionally constructed our scoring criteria such that we will have flexibility later in the process to use a variety of arithmetic relationships to help identify the most effective recommendations.
16JHSAT - June 2006
USDataset Selection
JSAT ProcessUnderstanding
Refining
JHSAT Analysis
IdentifyFCAA &MilitaryPartners
Indoctrinate in JHSATProcess
ConductJHSATbased
analyses
Expert Review/
Validation
MitigationRecs - Global
MitigationRecs
FCAA & Mil
Coordinating JHSAT US and International Efforts
First SetUS Mitigation
Recs
Mar/Apr 06May/June 06
May-Dec 06Jan 07
1Q 07
4Q 07
TBD 07TBD 07
TBD 06/07Sept 06
TechReview
Jan 07
17JHSAT - June 2006
US CivilDataset
Selection (1/2 of
helicoptersWorldwide)
JSAT ProcessUnderstanding
Refining
JHSAT Analysis
IdentifyFCAA &MilitaryPartners
Train FCAA &
Military Partners in JHSATProcess
FCAA & Military Partners conductJHSAT
analyses on their
accident data
Expert Review/
Validation
MitigationRecs - Global
MitigationRecs
FCAA & Mil
Coordinating JHSAT US and International Efforts
First SetUS Mitigation
Recs
Mar/Apr 06May/June 06
May-Dec 06Jan 07
1Q 07
4Q 07
TBD 07
TBD 07TBD 06/07
Sept 06
TechReview
Jan 07
IHST
18JHSAT - June 2006
Coordinating JHSAT US, International and Military Sources
Matt Rigsby from FAA-ASW will be acting as the contact point for identifying overseas and military partners.
To date we have interest from:
Canada – may use Canada as a test case, good dataset, strong interest, close proximity
EASA, UK, Australia, Chile, US Navy
Looking into:Brazil, Japan
Any other entities that possess large helicopter accident datasets that could be processed by the JHSAT method will be considered.
19JHSAT - June 2006
Leveraging Existing Safety Reports
JHSAT is reviewing the recommendations of the following reports:
NASA - U.S. Civil Rotorcraft Accidents, 1963 Through 1997NASA - Analysis of US Civil Rotorcraft Accidents from 1990 to 1996 and Implications for a Safety Program NASA - ASRS Rotorcraft Incident Study - Draft Data Summary Aviation Safety Reporting SystemNASA - Helicopter Accident Analysis TeamAMPA - A Safety Review and Risk Assessment in Air Medical TransportCRS - Report for Congress - Military Aviation SafetyOGP - Safety Performance of Helicopter Operations in the Oil and Gas Industry - 2000 DataTSB Canada - Lessons Learned from TSB Investigation of Helicopter Accidents (1994 - 2003)Bell Textron - History of Helicopter SafetyOther NTSB and international reports may be considered.
Approach: Identify recommendations that call for systemic change via oversight, policy, SOP development, investment areas, etc. Examples are changes to safety culture across the industry, adopting SMS approaches, foundational work for universal adoption of a new technology. Recommendations calling for specific design changes, specific training topics, specific changes to specific platforms or operational missions, etc, will not be considered at this point. We will allow the JHSAT process and NTSB data to guide us on the latter type of recommendation.
20JHSAT - June 2006
Issues for IHST Awareness/Concurrence
Dataset selection will be NTSB year 2000 accidents.
Team membership is light on small helicopter pilots/ops, may need help from IHST to push for renewed commitment from some industry members.
Scoring criteria adapted from that used in the CAST/JSAT.
General JHSAT program timing (slide 15)
Early work underway to identify international partners, will teach them the JHSAT process and have them analyze their own datasets.
Reviewing existing safety reports for systemic change recommendations, if any identified they will be passed to the IHST/JHSIT this summer/fall.