CARE ASAS Validation Framework. Partners CARE ASAS Board - Francis Casaux EURCONTROL - Mick van...

CARE ASAS Validation Framework

Partners

CARE ASAS Board - Francis Casaux EURCONTROL - Mick van Gool, Ulrich

Borkenhagen

Consortium Partners Aena, Isdefe, NATS, NLR, QinetiQ

Currently 6 months into 12 month programme

Validation of ASAS

Validation Does proposed operational concept provide anticipated

performance?

Operational concept

Performance Framework

High Level Validation Process

ASAS Framework Requirements

Operational Concept - Airborne Separation Assurance Systems

Performance Framework Reference Scenario High level objectives Metrics

Links to Previous work Support to Users

Operational Concept

ASAS application categories, from PO-ASAS

Airborne Traffic Situation Awareness• Current ATC rules apply

Airborne Spacing • Aircrew given more tasks

Airborne Separation• Temporary delegation of separation to aircrew

Airborne Self-separation• Full delegation of separation

Performance Framework

ATM 2000+ Strategy Objectives Safety Capacity etc

Framework Scenario Operationally realistic Scenario data repository

Metrics System Performance Metrics Human Performance Metrics

Validation Exercises

Operations

Operationaltrial

Shadow-modetrial

Field test

Large-scalereal-time

simulation

Small-scalereal-time

simulation

Fast-timesimulation

Analyticmodelling

Links to Previous Work

CARE Integra MAEVA EMERALD RTD Plan Other EUROCONTROL work

Support to User

Guidelines - assistance to validation teams and managers

Case Studies (Worked examples) Scenario database Documentation Dissemination Forum

Work Packages

S ce n ario D a ta R e po s ito ry

F ra m ew o rk S ce n a rioW P 1

Isd e fe , A e na & N LR

S ys te m P e rfo rm a n ce M e tricsW P 2

Q in e tiQ & A e na

H u m an P e rfo rm an ce M e tricsW P 3

N L R & N A TS

M e tric s

V a lid a tio n F ram e w o rk

A p p lica tio n G u ide lin esIsde fe

T w o De ta ile d C ase S tu d iesN A T S

E M E R A L D R T D P lanQ in e tiQ

F ra m e wo rk A pp lica tionW P 4

N A T S

O ve ra ll M a n ag e m e n t a n d C o ord ina tionW P 0

N A T S

Scenario Framework

• STANDARDIZE the validation scenarios for ASAS Applications

Scenario Framework based on Template

• Facilitate the TRACEABILITY of the scenarios

• SUPPORT DESIGNERS in the creation of validation scenarios for ASAS applications

• ENSURE THOROUGH AND CONSISTENT validation of ASAS applications

• Why is a scenario template needed ?

• MAIN GOAL: Supporting the designer in the creation of a validation scenario

Objectives of Template

• Structure based on PO-ASAS categories and applications with EMERALD

• One template for each PO-ASAS category: Airborne Traffic Situational Awareness, Airborne Spacing, Airborne Separation, Airborne Self-Separation

• Items for scenario definition clustered in seven groups: Objectives, Airspace, Traffic, ATS Involved, Rules, Tasks, Actors (Performer & Supervisor), Technology

• USER SELECTS the ITEMS that will form the scenario but SOME of them are MANDATORY

• Based on the chosen items, designer develops the scenario

• OBJECTIVES: to be grouped by types (Safety, Capacity…)

• AIRSPACE: Restrictions, Types, Areas, Elements, Geographical Scope

• TRAFFIC: Volume, Complexity, Flight Schedule, Timeframe, Aircraft Type, Equipment Type, Aircraft Performance

• ATS INVOLVED: ATC, FIS, TIS-B, APP, Alert Service, GNSS, ATFM, Performance Management, New Services

• RULES: Flight Rules, Longitudinal Separation, Lateral Separation, Phraseology, RVSM, Aircraft Sequencing, Conflict Resolution Strategy, Co-ordination and Transfer Procedures, Trained Flight Crews

• TASKS AND ACTORS - ‘Perform Separation Assurance’: Performer & Supervisor

• TECHNOLOGY:– GROUND: Communications, Surveillance, ATM– AIRBORNE: Redundancy, Communications, Surveillance, Navigation,

Display

Scenario Definition Groups

Example: Technology GroupGROUND

Communications:VoiceData linkInter-sector information transferSurveillance:Radar coverageNo radar coverageTIS-BADS-BCWP HMI with aircraft equipment, ID and delegation statusController assistance toolSequencing toolSeparation monitoring toolATMPlanningAutomation

AIRBORNERedundancyCommunications:VoiceData linkSurveillance:Mode-STCAS II/ETCASAudio/visual alertsTraffic information processingSurveillance data processing Spacing function processingNavigation:GNSS with augmentationFMSAutomatic management of the separation (connection with FGCS)Display:CDTI/EFIS/MCDU HUD

Enhanced Visual Adquisition, EMERALD

Longitudinal Station Keeping, EMERALD and NATS

Closely Space Parallel Approaches, EMERALD

Station Keeping Approach, EMERTA

Autonomous Aircraft, EMERALD

Baseline En-route Free Flight, NLR

TMA ASAS Autonomous Aircraft, NLR

FREER-EACAC, EEC

Multi agent optimal ASAS operations (COAST), Glasgow University

Optimal FFA/MAS Transition Methodology, Glasgow University

ASAS Crossing Procendure, CENA

Sector capacity assessment for ASAS, CENA

Applied on Historic Data

Scenario Database

Gathered data from sources in CARE ASAS and the VF consortium

Uses EUROCONTROL Validation Data Repository

Agreed data structure for scenarios based on template

Obtained agreement for sharing of data

Access via EUROCONTROL

System Performance Metrics

System Performance Metrics

Aim was not to develop new metrics

Accessible advice to validators

High level objectives Hierarchy mapping objectives to metrics

• Many to many

Taxonomy Perspectives

Objectives Objectives covered, based on ATM 2000+ Strategy

are: Safety Capacity Efficiency Environment Security & Defence

Security & Defence extended to include terrorist threat

Uniformity and Quality not considered further

Hierarchy

OBJECTIVES

PERFORMANCE AREAS

( ASPECTS)

METRICS

Hierarchy

PERFORMANCE AREAS broken down into ASPECTS where appropriate

Example: ACCESS (PERFORMANCE AREA)

• Airports• Sectors (ASPECTS)• Routes

Assists use with scenarios that look at specific airspace

Taxonomy

METRICS INDICATORS CARE-ASAS PRC Validation C/AFT Framework TORCH

Database tool For easy manipulation of data and linkages

Database tool Enabling instant switch from top-down...

Database tool … to bottom-up view

Perspectives

Different views (perspectives) can be applied to the information stored:

Airline perspective as in C/AFT

ATM perspective as in PRC

Scenario

Perspectives

Further perspectives under consideration are:

Study type (FTS, RTS, Analytic or Survey)

ASAS categories (Situational awareness…Self Separation)

Links Human Performance metrics

Links to Previous and Parallel Work

Potential Links

CARE Integra

MAEVA

EMERALD

Ongoing EUROCONTROL work

CARE Integra

EUROCONTROL funded study

Metrics Capacity

Safety

Environment

Economics

QinetiQ involvement in Integra and VF

Use of Integra metrics

MAEVA

A Master ATM European Validation Plan

Supporting EC in management of ATM projects Validation Guideline Handbook - exercise level Validation Master Plan - project validation plans Monitoring of projects

Final Consolidation of Results from Projects

NATS and Isdefe involvement in MAEVA & VF

Alignment of VF with MAEVA VGH

MAEVA Five-Step Validation Methodology

Step 1: Aims

Step 5: Conclusions

Step 2: Preparation

Step 4: Analysis

Step 3: Execution

Others

EMERALD RTD Plan

Plan for RTD for future technology for ATM systems

NATS and QinetiQ involvement in EMERALD & VF

Production of updated EMERALD RTD Plan

On-going EUROCONTROL work Validation Guideline VALSUP

Alignment of VF with VALSUP

Support to the User

Documentation

Final report incorporating work package reports Guidance Material for application of framework Case studies of two applications

In-descent spacing Fully autonomous aircraft

Detailed RTD Plan developed from EMERALD Plan

Data and Tools

Scenario data repository Produced in WP1

Metrics database Produced in WP2 Not a deliverable under contract

Dissemination Forum

Dissemination Forum 10th October at Bretigny

Summary

Validation Framework still under development Exploiting existing work for ASAS validation

exercises Scenario template developed Scenario data collated for VDR Mapping of system performance metrics & human

performance metrics to objectives - in progress Support user - about to start