• TAWS Implementation

• Runway Safety Initiative

• CAST Status

• TCAS Action Plan

• JIMDAT Status

• Data Mining

Kyle L OlsenKyle L OlsenKyle L. Olsen Aviation Safety Advisor, FAA Kyle L. Olsen Aviation Safety Advisor, FAA Seoul , 27-28 April 2009

COMMERCIAL AVIATION COMMERCIAL AVIATION COMMERCIAL AVIATION COMMERCIAL AVIATION SAFETY TEAMSAFETY TEAM (CAST)(CAST)SAFETY TEAMSAFETY TEAM (CAST)(CAST)

2020 Safety Plan StatusF b 2009

2

February 5, 2009

CAST Safety Plan - February 5, 2009In QuestionQ

1%Late6%

On Tgt

70 Safety Enhancements48 Complete22 U dOn Tgt

24%

17

22 Underway4

Complete69%

CompleteOn Tgt

48

LateIn Question

CHANGES from last meeting SE-47 CLOSED.[CRM for Tower Controllers]

32020 Plan Risk Reduction Estimate 74 %

2020 CAST SAFETY PLAN – WORKING SEs(Total Plan – 70 SE; 48 Complete; 22 Underway)

101R1AIA

39ANM

53ARA

120ATA

121ATA

125R1ATA

127R2AIR

131R1ATA

133R1AIA

134R2AIA

136ATA

159R1ATO

165R3AFS

169R1AFS

170R2AIA

172R1AFS

175R1ATA

179R1AFS-1

180ATS-1

181R1AAS

182R1ATO-1

183R1AFS-1

184MVA

185RNAV

ON TRACKLATE

SE #LOOSEC

ActionNeededCOMPLETED

4

IN QUESTIONLATE OP

Commercial Aviation Safety Team (CAST)

Commercial Aviation Safety Team (CAST)(CAST)

Joint Implementation Data Analysis Team (JIMDAT(CAST)

Joint Implementation Data Analysis Team (JIMDAT

JIMDAT UpdateJIMDAT UpdateppCommercial Aviation Safety Team

Herndon VACommercial Aviation Safety Team

Herndon VAHerndon, VAFebruary 5, 2009

Herndon, VAFebruary 5, 2009

Jay Pardee, FAAPaul Russell, BoeingJay Pardee, FAAPaul Russell, Boeing

Miscellaneous Miscellaneous ––Bi d ik D bBi d ik D bBirdstrike DatabaseBirdstrike Database

Request for manufacturer support through Request for manufacturer support through AIA subcommittee to update engine AIA subcommittee to update engine bi d t k d t b ( tl th hbi d t k d t b ( tl th hbirdstrke database (currently goes through birdstrke database (currently goes through 2000) to validate size and effect 2000) to validate size and effect assumptions and calculationsassumptions and calculationsassumptions and calculationsassumptions and calculations

Propose as a CAST activity Propose as a CAST activity –– bring results bring results back to CAST determine if any CASTback to CAST determine if any CASTback to CAST, determine if any CAST back to CAST, determine if any CAST action Req’d.action Req’d.

ATA Safety Council CAST ClassATA Safety Council CAST ClassATA Safety Council CAST ClassATA Safety Council CAST ClassHeld January 8, 2009; 14 airline Directors of Held January 8, 2009; 14 airline Directors of

SafetySafetySafetySafety 33--1/2 hr training session covering CAST and 1/2 hr training session covering CAST and

ASIAS history and current productsASIAS history and current productsASIAS history and current productsASIAS history and current productsDOS’s will brief their carriers on material covered; DOS’s will brief their carriers on material covered;

have requested condensed briefing for a futurehave requested condensed briefing for a futurehave requested condensed briefing for a future have requested condensed briefing for a future session involving senior managementsession involving senior management

Feedback that shared data efforts and quality of Feedback that shared data efforts and quality of q yq yASIAS analytical products were impressiveASIAS analytical products were impressive

Directed Study: Traffic Alert and Collision Avoidance System (TCAS) Resolution

Advisory (RA) Analysis

Presented to CAST

Mike Borer, jetBlue

Presented to CAST

5 February 2009

jVince Zaccardi, jetBluePat Massimini, MITRE

84 February 2009

5 February 2009

Study Objectives• Leverage quantitative data sets to assess TCAS RA landscape

throughout the NAS

• Utilize NAS-wide results and expert input to guide focusedUtilize NAS wide results and expert input to guide focused investigations at key airports

– Categorize RAs by degree of risk– Group by operational situation: closely spaced parallel approach IFR / VFR– Group by operational situation: closely spaced parallel approach, IFR / VFR

separation, ... – Identify common threads / contributing factors

• Determine compliance rates and factors that contribute to observed• Determine compliance rates, and factors that contribute to observed rates

• Measure TCAS RA event rates experienced by international carriers while operating at US airportswhile operating at US airports

– Determine whether those rates exceed the rates experienced by domestic carriers inside US airspace

– Determine whether those rates exceed the rates that would otherwise

94 February 2009

Determine whether those rates exceed the rates that would otherwise occur outside US airspace

Data Sources and Analysis

NOP (National Offload Program radar surveillance data)Run simulations that will encompass all carriers and traffic to better estimate RA rates and degree of risk. Perform domestic / foreign comparisons. (Provides intruder information.)

TOPA (TCAS OperationalFOQAAssess NAS wide TCAS RA landscape. Investigate compliance with advisories

TOPA (TCAS Operational Performance Assessment)Debug, tune, and assess accuracy of simulation software. Mode-S downlink provides information similarcompliance with advisories.

(Lacks intruder information.)downlink provides information similar to FOQA, and is enhanced with surveillance radar data.

Text Reports (ASAP, ASRS, OE) Help get the story behind the numbers at selected airports.

International Data (AEA, AAPA text reports and event counts) Facilitate comparisons between domestic and foreign RA rates

104 February 2009

foreign RA rates.

Summary and Next Steps

• Continue NAS-wide lookContin e anal sis of target areas• Continue analysis of target areas

• Drill down at selected airports• Debug and tune TCAS simulation softwareDebug and tune TCAS simulation software

using TOPA data• Run TCAS simulation software with NOP

data for all selected airports– Use intruder information to assess risk– Perform domestic / international comparisonsPerform domestic / international comparisons

• Complete ASAP report review

114 February 2009

Commercial Aviation Safety Team (CAST)

Terrain(CAST)

Terrain Awareness and Warning SystemWarning System

(TAWS) Study

February 5th, 2009

Executive Steering Group:Jay Pardee Paul Russell

Working Group Chairs:Steve EricksonVivek Sood

Analytical Task

Conduct an in-depth analysis of selected airports, (including Conduct an in depth analysis of selected airports, (including those identified in VSIS EGPWS study - Reno, San Diego, San Francisco, Oakland, Las Vegas, Albuquerque, Burbank, and Boise) to develop mitigation strategies for—Boise) to develop mitigation strategies for Flight crewmember desensitization to TAWS warnings caused by the

occurrence of unwanted warnings when the aircraft is not in imminent danger.g

TAWS warnings triggered by the interaction of airspace design (for example, minimum vectoring altitudes) and aircraft flight path trajectories.

Potential incompatibilities between aircraft and air traffic control warning systems.

The role of Traffic Collision Avoidance System (TCAS) alerts issued f ll i i TAWS ifollowing crew reaction to a TAWS warning.

Next Stepsp

Request CAST approval for JIMDAT to develop:q pp p SE-184 Evaluate MVA airspace designs in relation to terrain and traffic

flows in terrain airportsflows in terrain airports SE-185 RNAV or other procedures to reduce unnecessary terrain alerts

and to provide better separation from terrain and to provide better separation from terrain Develop Supplemental Implementation Plan for SE-120 Substantiate rationale for installing enhanced TAWS software

(218 or equivalent) and GPS (also suitable for DOS)(218 or equivalent) and GPS (also suitable for DOS) Develop business model for GPS

Provides rationale for DOS use as well

Overview ofAccident Risk Metrics Applied to the

ASIAS FOQA Archive

Presented to CAST

Doug Schaefer, Austin DigitalDoug Schaefer, Austin DigitalPat Massimini, MITRE

5 F b 2009

154 February 2009

5 February 2009

Metric Development

St 1St 1 St 2St 2Stage 1: Analytical Data Quality Review

Stage 1: Analytical Data Quality Review

Stage 2: Operational Data Quality Review

Stage 2: Operational Data Quality Review2.97 million

flights from 5

Iterate on data quality profilesIterate on data quality profilesFilter results to

eliminate known problem events

Filter results to eliminate known problem events

airlines

Iterate on event profiles

Iterate on event profiles

problem events or parameters

problem events or parameters

Today’s results Will complete over next 2 months

164 February 2009

Current State of the ArtCurrent State-of-the-Artin Data Mining Methodsg

Presentation to CAST

Amy PritchettNASA Aviation Safety Program DirectorNASA Aviation Safety Program Director

With Assistance from NASA Data-mining Team

Feb 5, 2009

Domains with Vast D t Mi i R hData Mining Research Science

Earth Science / Climate Earth Science / Climate Space

Medicine Business Business

Credit Card Fraud Market Trends Intrusion Detection

Similarities abound across these domains!

Directed Marketing Engineering

AviationS Space

All Others! Web Analysis

Google/Yahoo! Searches

National Aeronautics and Space Administration 18

Google/Yahoo! Searches Blogosphere

Summary: Data Mining Methods

Extensions to MonitoringC B M lti di i l

Monitoring Can Be Multi-dimensional Can Be Based on Pre-defined Definitions of

‘Nominal’ – or Can Automatically Identify the ‘Off-nominal’

Trending Anomaly Detection

P d l C li Spans a Range of Phenomenon (FDR, Cockpit Switches, Sky’s the Limit…)

Procedural Compliance

Extensions to Text MiningC B B d P d fi d C t i

Text Mining

P di ti Prediction

Can Be Based on Pre-defined Categories –or Automatically Identify ‘Features’

Can Apply Natural Language Techniques

Extensions to Information Protection Information Protection

Prediction Can Identify Degradation Before Failure

Prediction

National Aeronautics and Space Administration 19

Must be Done Very Carefully Extended Methods May Facilitate Analysis

Runway Safetyy yInitiative

Jim BurinJim BurinDirector of Technical Programs

Regional PerspectiveRegional PerspectiveAccident Rates Vary by Region of the WorldAccident Rates Vary by Region of the WorldWestern-built transport hull loss accidents, by airline domicile, 1998 through 2007

Europe0 7

C.I.S.1

United Statesand Canada

0.4

0.7China

0.3

Middle Eastdd e ast2.3

Africa10.0

Asia2.1

(ExcludingChina)

Latin Americaand Caribbean

2.1

Oceania0.0

World1.1

Accidents permillion departures

1 Insufficient fleet experience to generate reliable rate.

Regional PerspectiveRegional PerspectiveAccident Rates Vary by Region of the WorldAccident Rates Vary by Region of the WorldWestern-built transport hull loss accidents, by airline domicile, 1998 through 2007

Europe0 7

C.I.S.1ESSI

COSCAP CIS

United Statesand Canada

0.4

0.7China

0.3

Middle EastCAST

70 SEs

ESSI35 SEs JAA11 SEs EASA62% reduction

COSCAP NA, SA, SEA40 SEs in work60% reduction

dd e ast2.3

Africa10.0

Asia2.1

(ExcludingChina)

48 complete22 in work74% reduction COSCAP GulfCOSCAP

BAG

Latin Americaand Caribbean

2.1

Oceania0.0PAAST

CFIT and ALARASET

World1.1

Accidents permillion departures

1 Insufficient fleet experience to generate reliable rate.

Thank youThank you