Resilience2050 - s3.eu-central-1.amazonaws.com · 1.1.3 D4.2 Relation with Resilience2050 future tasks This deliverable will be a key input for Resilience 2050 research activities

Resilience2050.eu

NEW DESIGN PRINCIPLES FOSTERING SAFETY, AGILITY AND RESILIENCE FOR ATM

Grant agreement no: 314087

Theme AAT.2012.6.2-4. Building agility and resilience of the ATM system beyond SESAR

Funding Scheme: Collaborative Project (small or medium-scale focused research project)

D4.3 NEW BALANCED MODELLED IN VERIFIED SIMULATION ENVIRONMENT

Date: December 2015

Status: Final version (version 3.0)

Dissemination Level: Public (after completion)

Prepared: Innaxis, NLR, DHMI, ITU

Reviewed: Resilience2050 consortium

Approved: The Innaxis Foundation and Research Institute

Record of versions:

Revision Date Reason for Revision Modified sections

V0.01 September 2015 Creation of the document All

V0.02 November 2015 Version for review by partners All

V0.03 December 2015 Version reviewed, ready for final approval All

V0.04 December 2015 Final version All

Abstract

This document is deliverable D4.3, "New balanced concept modelled in verified simulation environment", which as described in the DoW is the third and last deliverable of WP4. The deliverable D4.3 describes the simulation environment within Resilience2050 as the new balanced concept that has been modelled in the context of a future resilient ATM system.

Table of Contents 1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.1 Resilience2050 project objectives. ............................................................................................... 41.1.2 Scope ............................................................................................................................................ 41.1.3 D4.2 Relation with Resilience2050 future tasks .......................................................................... 5

2 State of the art of modelling and simulation in ATM .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1 Agent-based modelling and simulation .............................................................................................. 6

2.2 Queuing networks and discrete event/time modelling ...................................................................... 8

2.3 Stochastic integer programming & optimization ............................................................................... 8

2.4 Data science modelling ....................................................................................................................... 9

2.5 Other models ..................................................................................................................................... 10

3 Context and objectives of the simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 The Resil ience-efficiency 2050 model (REM2050) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5 Verif ication of the simulation environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.1 The context of verification and validation ......................................................................................... 16

5.2 Verification of the model implementation ........................................................................................ 16

5.3 Integrity of model input data ............................................................................................................. 16

6 Visualisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.1 The need for better visualisation techniques ................................................................................... 18

6.2 Visualisation techniques in Resilience2050 ...................................................................................... 186.2.1 A circular resilience schematic map ......................................................................................... 206.2.2 2 A geographical interactive map .............................................................................................. 206.2.3 3 Propagation trees .................................................................................................................... 21

7 References .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Appendix A. Safety analysis queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Error! Bookmark not defined.

Resilience2050 D4.3 · 4

1 Introduction

Resilience2050.eu is a collaborative project funded through the FP7 AAT Call 5, topic AAT.2012.6.2-4: Identifying new design principles fostering safety, agility and resilience for ATM. The project aims to:

• Develop adequate mathematical modelling and analysis approaches to support systematic analysis of resilience in ATM scenarios; taking into account the different ATM disturbances that can take place in the European airspace.

• Develop metrics to systematically define resilience, addressing the concepts of “Responding”, “Monito-ring”, “Learning” and “Anticipating”. This work will result in a Resilience Analysis Framework, enabling the definition of new ATM design principles that fosters safety, agility and especially resilience.

• Provide an extensive overview of human contributions to resilience in current ATM.

The project is carried out by an international consortium composed of: The Innaxis Research Institute as the Project Coordinator, Spain; Deutsches Zentrum für Luft- und Raumfahrt e.V (DLR), Germany; Universidad Politécnica de Madrid (UPM), Spain, Nationaal Lucht- en Ruimtevaartlaboratorium (NLR), Netherlands; Istanbul Teknik Üniversitesi (ITU), Turkey; Devlet Hava Meydanlari Isletmesi Genel Müdürlügü (DHMI), Turkey; and King’s College London (KCL), UK.

The project was launched on the 1st of June, 2012 and will last 43 months. (The project was originally 36 months with an agreed extension of 7 months)

1.1.1 Resilience2050 project objectives.

The key objective of the Resilience2050.eu project is to analytically define the concept of "resilience" within the context of Air Traffic Management (ATM), and make an effort to improve the resilience of the European ATM system in the future.

• The first WP provided the theoretical framework, that lead to a definition of Resilience within the ATM domain. The first WP also explored other novel ideas such as factoring in the human role in ATM resilience. In addition, this WP explored the proper terminology (Resilience, Robustness, Disturbances, Perturbations) from other socio-technical domains.

• WP2 tackled the data sets and data mining analyses which enabled, along with the developments of WP1, a deep study of the "Resilience level" in the current European ATM system for each particular disturbance at a micro-scale. The WP also included insights into delay propagation patterns in the European ATM system, through a macro-analysis approach.

• WP3 then represented the resilience concept through a multilayer approach. WP3 provided a way to measure resilience, including a full list of Resilience Metrics of the current ATM system. WP3 concludes with Resilience design principles presented in D3.3, together with an operational assessment.

• Finally, WP4 and WP5 designs, develops and executes a simulation environment. The guidelines of a future resilient ATM operational concept is proposed using operational insights from the resilience design principles extracted in WP3, including the most effective resilience mechanism. A balance between resilience, efficiency and safety is proposed. In order to validate the result and compare it with a current situation, a future traffic scenario is run, including inputs such as expected future traffic levels and/or stress testing techniques, among other inputs.

1.1.2 Scope

There are several research activities reported in this deliverable, as specified in the description of work. Each activity flows directly from tasks T4.5 and T4.6 as specified in the DOW. Concretely, the following results from the research activities are reported in this deliverable:

• Modelling of the specifications of the balanced concept presented in D4.2 • Development of a simulation environment capable of modelling the future resilient ATM concept in a

computer simulation environment. • Verification of the modelling methodologies


Therefore, this deliverable documents the modeling and simulation tools and methodologies, including the data/processing flows used between the various tools. Through a top-down approach the document structure presents the report of the work performed under task D3.3, regarding the resilience design principles. Concretely:

• The cover page includes the deliverable title, cover page, record of revisions, abstract and table of contents.

• Section 0 contains an overall introduction of deliverable D4.3, including general information about its context within the Resilience2050 project and the current structure of the deliverable with brief explanations of each section. It also includes the explanation of the relationships between D4.3 and the rest of the project deliverables in terms of inputs/outputs.

• Section 1 describes the state of the art in modelling and simulation in ATM. This provides contextual framework of the different techniques, despite some not ultimately used within the project.

• Section 2 explains the context and objectives of the aforementioned simulations. The model is executed within the project context and describes the different modelling possibilities.

• Section 3 documents the model and simulator ("REM2050"), including details regarding the toolset, processes and data flow between the different modules.

• Section 4 verifies and explains the simulation environment. • Section 5 describes the visualization tools. • Section 6 has a list of compiled references.

1.1.3 D4.2 Relation with Resilience2050 future tasks

This deliverable will be a key input for Resilience 2050 research activities and deliverables, specifically:

• WP5 - Simulation results and general project outcomes (D5.2 and D5.2)

The following graphic also provides a visualisation of the connection between all the remaining tasks of the Resilience2050 project:


2 State of the art of modelling and simulation in ATM

Air transport and air traffic management are among the most complex, largely distributed socio-technical systems, involving large numbers of human operators and advanced technology, including large numbers of intrinsic and extrinsic varying conditions. To properly understand and effectively manage this complex system, a variety of models have been developed in ATM research and development. These models describe the ATM socio-technical system at different levels of granularity (e.g. technical system components, human-machine interaction, single flights, interacting flights, networks), at different dynamical scales (seconds to decades), and for various purposes (e.g. safety, delays, environment, costs).

This section presents a full overview of recent modelling approaches used in ATM, although within the context of the Resilience2050 project, only a few approaches have been used. For the sake of completion, the current deliverable contains a full review of the current state-of-art completed. The following types of models are discussed in the next subsections: agent-based modelling and simulation, queuing networks and discrete event/time modelling, stochastic integer programming & optimization, data science modelling, and other models (economic, bargain etc). The full list of references are compiled at the conclusion of the deliverable.

2.1 Agent-based modelling and simulation Agent-based modelling and simulation (ABMS) is an approach for modelling complex systems by describing the behaviour and interactions of a collection of autonomous decision-making entities, called agents (Bonabeau, 2002; Macal and North, 2010; Van Dam et al., 2013). The overall system behaviour emerges as a result of the individual agent processes and their interactions. ABMS provides a highly modular and transparent way of structuring a model, thus supporting systematic analysis, both conceptually and computationally. ABMS has been used in a wide range of application fields, including molecular physics, cell biology, ecology, epidemiology, social sciences, economy, market analysis, archaeology, and anthropology (Macal and North, 2010). Steps for ABMS of sociotechnical systems are presented in (Nikolic et al., 2013; Van Dam, 2009; Van Dam et al., 2013).

Davidsson et al. (2005) provide an analysis of research from the years 1992-2005 on agent-based approaches to transportation and traffic management, including air, road, rail, sea and intermodal modalities. It concludes that many of the logistic problems studied can be well addressed by agent-based systems, but that the maturity of the research was low and few systems had been deployed by then. A more recent review by Chen and Cheng (2010) on agent technology in traffic and transportation systems also shows that multi-agent methods and techniques have been applied to many aspects of traffic and transportation systems. This includes multi-agent technology in systems for dynamic routing, congestion management, and intelligent traffic control. Predominantly, agent-based research in ATM is focused on ABMS of traffic scenarios. Within ATM research there have been a range of ABMS applications, many of which are described in continuation.

In (Lee et al., 2007) agent-based modelling and simulation was demonstrated for analysis of aircraft arrivals into Los Angeles International airport (LAX) using various spacing techniques. A main output of the simulations are numbers of separation violations given the spacing techniques. The agent models of human operators (pilots and air traffic controllers) herein are based on the Air-MIDAS model. The study was done using the agent-based Reconfigurable Flight Simulator (RFS) modelling and simulation framework. This simulation platform was constructed in C++ and is described in (Shah, 2006).

An agent-based model for analysing control policies and the dynamic service-time performance of a capacity-constrained air traffic management facility is reported in (Conway, 2006). The model is implemented in the generic ABMS environment Repast. The model emulates commercial airline demand at a busy airport with a simplified hub-and-spoke route structure. The model includes primitive


representations of system capacity, demand, airline schedules and strategy, and aircraft capability. The agent-based simulations reveal the interdependences between these system attributes. Agent-based modelling using Repast was also done for location planning of intermodal freight hubs by Van Dam et al. (2007).

Agent-based modelling and simulation of air traffic for analysis of ATM network delays was achieved by Grether et al. (2013)/ Grether (2014), for which they used MATSim; a generic framework to implement large-scale agent-based transport simulations.

The Jet:Wise agent-based model (Niedringhaus, 2004) models competitive decision making of airlines with respect to aspects such as fleet mix, itineraries, schedules and fares in the context of delays and missed connections.

The development of the Airspace Concept Evaluation System (ACES) is presented in (Meyn et al., 2006). The company Intelligent Automation, Inc. (IAI) is contributing to this development using its CybelePro agent infrastructure to simulate and assess the performance of the National Airspace System (NAS). ACES was also used in a simulation study for evaluation of separation standards of UAS and manned aircraft in US airspace (Johnson et al., 2015).

A multi-agent simulator of unmanned aerial vehicles called Agentfly is presented in (Šišlák et al., 2008). The UAVs follow a trajectory plan and during flight the software agents in the UAVs detect conflicts and engage in peer-to-peer negotiation for adaptation of the route to avoid collision.

In the context of TOPAZ air traffic safety risk assessment (Blom et al., 2006), ABMS is used to assess probabilities of rare safety occurrences, such as aircraft collisions. Herein, ABMS is referred to as agent-based dynamic risk modelling (DRM) (Everdij et al., 2014). Applications include assessment of runway incursion risk (Stroeve et al., 2013), and safety of airborne self-separation (Blom and Bakker, 2012). These studies explicitly include a range of hazards in the ABMS, i.e. conditions, events or circumstances which could contribute to the occurrence of an accident, such as system failures, misunderstandings, or bad weather.

ATOMS (Air Traffic Operations & Management Simulator) is an air traffic and airspace modelling and simulation system for the analysis of Free Flight concepts (Alam et al., 2008). It is an intent-based simulator which discretises the airspace in equal sized hyper rectangular cells to maintain intent reference points. It can simulate end-to-end airspace operations and air navigation procedures for conventional air traffic as well as for Free Flight. ATOMS uses multi-agent based modelling paradigm for modular design with easy integration of various air traffic sub systems. It uses BADA aircraft performance models. Results from this includes flight profiles, flight duration and fuel consumption.

Alam et al. (2015) present a study on the use of an evolutionary optimization approach in combination with the ATOMS air traffic simulator and the ICAO collision risk model in support of minimizing risks in strategic lateral offset procedures.

Chao et al. (2009) present agent-based modelling and simulation of air traffic flow control. Results of their simulations are provided for relationships between traffic load, traffic throughput and delay in ATC Chinese sectors. These indicate a rise in delay and a decline in throughput for large traffic loads. Traffic flow control strategies can effectively avoid such congestion.

In conclusion, agent-based modelling and simulation has been used quite extensively in ATM research. For some applications generic agent-based modelling environments (e.g. Repast) were used, but most applications used dedicated software for the agent-based simulations. The ABMS applications cover a range of key performance areas, including safety (separation analysis, collision avoidance, collision risk assessment), capacity, delay and cost-efficiency. The level of granularity in the ABMS applications depends on the key performance areas studied. Typically, lower levels of granularity are used in safety-focused studies (including particular human operators and technical systems), and higher levels of granularity for studies of other key performance areas.


2.2 Queuing networks and discrete event/time modelling Queuing network models are based on combinations of queues and servers, with particular assumptions on the stochastic characteristics of arrival rates and server service rates. In simpler cases, arrivals are represented by Poisson processes and service times are distributed exponentially. Queuing systems originated in telecommunications, but now has a broad application domain, including manufacturing, software testing, and different kinds of network performance evaluation (Balsamo et al., 2003).

Application of a queuing network model for studying ATM technologies in the US airspace system is presented in (Long et al., 1999). The network model LMINET describes 64 airports, wherein each airport is modelled as a sequence of an arrival server, a taxi-in server, a turn-around delay, a taxi-out server, and a departure server. These service processes are modelled as either Poisson or Erlang processes. In combination the LMINET airport model can describe the evolution of delays during a day. In the model airport capacity is evaluated by using runway capacities as a Pareto frontier for the arrival-rate / departure-rate relation. The effects of a large set of ATM technologies are studied by reasoning on their effects on parameters in the model. An improved version (LMINET2) which includes models for flight delay propagation and cancellation for 310 US airports is presented in (Long and Hasan, 2009).

An aggregate flow model for ATM is presented in (Sridhar et al., 2006). It describes the traffic flows in US airspace by a 23-center network model for the 22 network centres and international airspace. It uses a discrete-time time-variant linear dynamic model to describe traffic flows. It is argued by Sridhar et al. (2006) that time-variance is an important characteristic of the model, since departure counts vary considerably during the day and time-invariant Poisson distributions do not account for this. The model was fit using data containing four consecutive days of US traffic and assumed a Gaussian distribution of the departure uncertainty. It was shown that traffic on another day was predicted within the uncertainty bounds of the model.

Clarke et al. (2007) developed the MIT Extensible Air Network Simulator (MEANS) to analyse overall combined effects of air traffic control, flow management, airline operations control, and airline scheduling on the performance of the US air transport system in terms of congestion, throughput and delays. It uses an event-based simulation framework with modules for aircraft movements during the phases en-route, tower (arrival / departure queues), taxi, gate (turn around), as well as modules influencing the flight performance, namely air traffic control, airline, and weather.

Tandale et al. (2008) provide a review of queuing network models for air traffic and present an approach for incorporating trajectory uncertainty into queuing networks.

Pyrgiotis et al. (2013) describe the Approximate Network Delays (AND) model for evaluation of delays due to local congestion at individual airports in combination with the effects of propagation of these delays in an air transport network. It includes a queuing engine that uses non-stationary Poisson arrival processes and time-dependent Erlang service times to represent arrivals and departures at airports. In addition, it uses a delay propagation algorithm to update flight schedules and demand rates at the airports in the network in response to local delays as computed by the queuing engine. The results of the model are illustrated for a network of 34 US airports. The model results are sensitive for the setting of slacks in ground turnaround times, they do not account for en-route delays or other types of delays (e.g. due to mechanical problems), nor for airline reactions to congestion and delay.

In conclusion, the aforementioned ATM applications of queuing networks and discrete time/event models focus on analysis and prediction of the relation between capacities and delays in air traffic networks. They mostly use single flights passing through various flight phases as the lowest aggregation level, and have statistics of the traffic flows as main output.

2.3 Stochastic integer programming & optimization Integer programming considers mathematical optimization problems wherein variables are restricted to integers and stochastic integer programming is suited for optimization problems involving uncertainty. These techniques are used extensively in production planning, operations research, financial planning, scheduling and telecommunication networks (Birge and Louveaux, 2011).


Liu et al. (2008) developed airport capacity models to support air traffic flow management. The approach uses capacity scenario models to describe airport capacity profiles (time series of capacity values). It develops sets of capacity scenarios with associated probabilities, which are called scenario trees. Using these scenario trees optimal ground holding strategies are developed. They demonstrated the feasibility of applying the approach and identifying capacity scenarios using historical data from US airports.

Stochastic integer programming models are used in support of strategic flow control decisions when weather reduces the capacity of an airport or its arrival fixes / approach paths. In (Mukherjee and Hansen, 2009) an advanced approach is presented which includes both ground delays and dynamic rerouting options in the optimization. This outperforms approaches that consider less flexible set of control variables, such as static rerouting and a pure ground delay holding strategy.

Bosson et al. (2015) extends stochastic integer programming modelling for flow control decisions by including taxiway and runway operations.

Clarke et al. (2009) developed an approach for traffic flow management, which converts weather forecasts into stochastic airspace capacity information and then uses it to dynamically route aircraft depending on the capacity predictions.

Sridhar et al. (2015) used a modelling approach for fuel consumption during oceanic flights in combination with strategic de-confliction for achieving fuel-optimal flights.

Taylor et al. (2015) use decision tree modelling and optimization by genetic algorithms in support of traffic flow management (TFM). The objective is to identify the best current TFM strategy while accounting for potential future actions that may be needed. The decision trees represent forecast ensembles over a decision horizon, wherein uncertainty is typically due to weather and decision nodes represent TFM decisions at selected times during a planning horizon. The genetic algorithm minimizes a cost function that is defined for the effects of the variety of possible decisions.

In conclusion, the above optimization applications in ATM mostly focus on air traffic flow management in which the main optimization technique used is stochastic integer programming.

2.4 Data science modelling Data science considers research on knowledge extraction from large volumes of data using a broad range of descriptive techniques including machine learning, advanced statistics, data mining, and probability models (Dhar, 2013).

Cook et al. (2015a) provide an overview of data analysis and modelling approaches from complexity science that can be used in support of air traffic management. Zanin and Lillo (2013) present a literature review for the application of complex network theory to air transport. Complex network metrics of air transport networks are presented, such as: node degree, node betweenness, shortest paths, and clustering coefficients. These metrics are discussed in relation to point-to-point and hub-and-spoke topologies. Weighted networks include information on the use of network links, e.g. the numbers of flights or passengers. Evolution of the networks over short periods (weeks, seasons) or longer periods (years) provides insight into the use and development of the air transport networks. Additional studies of the dynamics of the air transport networks consider passenger connectivity's, the emergence of traffic jams, and the propagation of diseases.

Delay propagation at US airports was studied by a multi-factor approach by Xu et al. (2008). They used piece-wise linear regression models to represent generation and absorption of delays at 34 main US airports.

Mondoloni et al. (2015) use Monte Carlo simulations driven by statistics from operational data to quantify the effects of 4D trajectory feedback in collaborative flight planning, with applications to route optimization and ground/delay optimization.

A descriptive model of the air traffic network in China using statistical network topology techniques was presented in (Wang et al., 2015).


The FAA Aviation Environment Design Tool models fuel burn, emissions and noise of aircraft. This is applied in (Belle et al., 2015) for the assessment of NextGen operational improvements.

Avery and Balakrishnan (2015) present discrete-choice modelling of airport runway configuration, using multinomial logit and nested logit models and a range of attributes that can have an impact on the runway choice (e.g., current configuration, wind, demand, visibility, noise abatement). The choice models were fit for two US airports and the results show that limited improvement over change-nothing scenarios can be attained for 3 hours predictions.

Belkoura and Zanin (2015) present a statistical analysis wherein they compare planned and achieved flight trajectories to analyse occurrences of increase and decrease of delays in European airspace. They consider the ATM system being resilient if occurrences leading to delay increase can be compensated by occurrences leading to delay decrease. In this way they conclude that the en-route traffic is resilient with respect to delay formation and absorption.

Cook et al. (2015b) developed a passenger-focused delay simulation model for European airspace, which evaluates the effects of various strategies for prioritisation of flights. It uses network complexity techniques to obtain an understanding of the most critical network connections for delay propagation.

Van Baren et al. (2015) present descriptive statistics on aircraft separation distances, speeds and separation buffers during final approach for a number of European airports.

In conclusion, there are many data descriptive models being used in ATM research, ranging from straightforward statistical analyses and decision modelling to advanced data mining and network topology modelling. Many of the applications in ATM focus on delay analysis on a network scale, but applications also include operational decision making, environmental impact analysis and separation analysis.

2.5 Other models In addition to the model types described above, there are other types of mathematical models that are used in ATM research and development. Some recent examples are presented below.

Muñoz et al. (2014); Upchurch et al. (2015) present rule-based models for defining clear boundaries of unmanned aircraft systems (UAS), i.e. boundaries for UAS to avoid collisions with other airborne traffic. Monte Carlo simulations were done to characterise the effects of well-clear boundary models on time to well-clear violations and interoperability with existing airborne collision avoidance systems.

Groskreutz and Dominguez (2015) use a model for separation assurance budget, consisting of various components of a separation minimum, including a collision zone, a surveillance uncertainty zone, an intervention buffer (representing a wide spectrum of detection, reaction and communication of controllers and pilots, communication systems, environment, aircraft performance, and ATC rules and situation complexity), and a wake turbulence zone. This rationalises contributions between separation distance needed to address errors and update rates of various surveillance systems.

De Visscher et al. (2015) propose a model for characterization of a wake vortex encounter using the rolling moment coefficient as metric. They show the validity of their approach by experimental flight data.

De Smedt et al. (2015) present a model of the longitudinal variance during controlled time of arrival operations. It is a deterministic model that can be used to compute windows of the estimated time of arrival. Adler et al. (2015) developed an economic gaming model for charges by air navigation service providers and airline reactions to such charges to evaluate various scenarios for changes in the European ATM organisation. Proost et al. (2015) present a union bargaining model for European air navigation service providers to gain understanding of wage formation and other socio-economical effects.


3 Context and objectives of the simulations

The resilience and efficiency concepts have already been defined in previous deliverables. This current deliverable specifies the modelling environment capable of executing the simulations needed to make an assessment of the Resilience2050 balanced concept. In order to assess the efficiency and resilience under the Resilience2050 operational concept, future traffic scenarios are modelled. The goal is to move from the resilience performance assessment of the current traffic and operational scenario ("A", in the figure below) to asses how a resilient ATM paradigm will perform with future conditions of traffic ("D", in the figure below).

The focus is on A and D. The remaining scenarios "B" and "C" are also interesting, although partially falling out of the scope of the project.

• Scenario "C" would be useful to evaluate the savings (e.g. in minutes of delay) that could be achieved today if a different paradigm was in place. These simulations are normally performed to build a business case of a paradigm change and to evaluate the savings that could be achieved. This also helps to justify investment in new technologies or paradigm change. Scenario C will not be simulated in the project and liest outside of project's scope.

• Scenario "B" consists of simulating the current ATM paradigm with future traffic and disturbances scenarios. This simulation is useful in evaluating the future situation if the paradigm is not changed/ in order to compare its results with scenario D. This simulation is sometimes called the "do nothing" scenario. These simulations are normally performed to build a business case of paradigm change and evaluate the situation when the current modus operandi is unchanged, evaluating the degradation in performance as future traffic grows in number and complexity. Scenario B has been modelled in order to provide an assessment in D5.3 by comparing it with Scenario D.

The process of evaluating a future ATM paradigm with future ATM traffic (moving from A to D in the picture above) requires several intermediate steps. Those steps require a combination of modelling and simulation, data analysis, operational insights, ATM domain analysis, future traffic forecasts, data visualization, modelling verification and recommendations for a future, more resilient ATM system. It also requires the development and use of a computer simulation infrastructure capable of assessing both the efficiency and the resilience of the scenarios.

The following figure details the 11 steps necessary to extract conclusions. Step 7 is documented in this deliverable, as well as the approach for steps 8 and 9. The remaining steps will be presented in D5.2 and D5.3. For the sake of completion, a brief description of every step is already documented in this section:


Step Deliv. Description of tasks Comments

1 D3.3 Definition and computation of efficiency

In order to compute the metrics that provide the efficiency of the system, a data analysis and simulation infrastructure has been built. This infrastructure is capable of measuring efficiency through the methodology described in deliverable D3.3.

2 D3.3 (D3.2)

Definition and computation of resilience

In order to compute the metrics that provide the resilience of the system, a data analysis and simulation infrastructure has been built. This infrastructure is capable of measuring resil ience through the methodology described in deliverable D3.3. Further details about the process and mathematical steps are in D3.2.

3 D3.3 Design principles of efficiency

The methodology provides results that lead to the identification of critical elements and proposal of design principles for efficiency.

4 D3.3 Design principles of resilience

The methodology provides results that lead to the identification of critical elements and proposal of design principles for resilience.

5 D3.3 Operational assessment of design principles

Both sets of design principles are verified operationally. It is ensured that resilience design principles make sense from an operational point of view. This was already described in D3.3.

6 D3.3, D4.2, D4.3

The balanced concept In D3.3 other variables not yet covered such as safety and the human role, were added to the research scope. So far, the project had only included efficiency and resilience concepts-variables. The combination of efficiency and design principles specify a "balanced concept" that describes a set of recommendations for efficiency (breaking points due to capacity). It also describes certain levels of improved R values for city pairs as a function of the disturbances.


Step Deliv. Description of tasks Comments

7 D4.3 Modelling the efficiency in the balanced RES2050 concept

The design principles and the balanced concept are the specifications of the future, more resilient ATM system proposed by the project team. This future ATM operational paradigm is specified in terms of the efficiency and resilience levels that different parts of the system should have. These standards ensure resilience is taken into account as a key performance characteristic of the system. These operational paradigm specifications are modeled into a simulation environment to test them against future levels of traffic. In order to do this, some changes to the data analysis and simulation infrastructure need to be implemented.The new model implementing the design principles, the "Resilience-efficiency 2050 model" or REM2050 is fully documented in this deliverable.

7+ D4.3 Visualization + verification of model

Verification guidelines of the simulation environment. Visualization of modelling results.

8 D4.3 - D5.2

How to use the future traffic scenarios (From Deliverable 5.1) in the RES2050 model

The future traffic scenarios are used in the REM2050 simulations to understand how the system will behave in terms of efficiency and resilience under different future scenarios. They were documented in deliverable 5.1, but are also explained in the current document within Section 4.3.4.1, they provide low-level details on how the traffic has been artificially created within the model.

9 D4.3 -D5.2

How to model the future disturbances (From Deliverable 5.1) in the RES2050 model

The inclusion of the model of future disturbances follows a similar approach to future traffic scenarios. The basic data sources covered in D5.1, provide further explanation and "hands-on" information on how to include them (intensity, duration, starting-ending times) in the model. It is explained in D4.3 and D5.2.

10 D5.2 Stress testing the resilience of the balanced RES2050 concept

The simulation environment will be used to stress test the balanced concept and obtain results on how resilient and efficient the system could become in the new scenario. Deliverable D5.2 will document the input and output/results of these simulations.

11 D5.2, D5.3

Conclusions on how the balanced concept behaves in terms of efficiency and resilience

Overall WP4 and WP5 conclusions, results and assessment of the implementation of the design principles in the future traffic scenarios will be documented in deliverable D5.3.

Steps documented in current deliverable


4 The Resilience-efficiency 2050 model (REM2050)

In order to model, simulate and ultimately execute the different processes described in the previous section, a number of computer models, data flows, data analysis and data visualisation tools have been built within the project. Hence, the overall model "REM2050" consists of different modelling pieces, elements and data sources. The following flowchart helps to report how the different pieces of the model work together.

The flowchart shows three high-level processes and their corresponding set of elements:

• On the left hand side there are current traffic and current disturbances. Current (and real) traffic used has been extracted from EUROCONTROL ALLFT+ data set (grey container, labeled 1). Disturbances were extracted from several other datasets and documented in D2.2 (second grey container, labeled 4). The information from current traffic and information from datasets that report current disturbances provided insights on both the resilience (green box, labeled 7) and the efficiency of the system (orange box, labeled 8). The output from these two analyses are used to build the resilience Principles (yellow square, labeled 11).


• The right side of the diagram represents the data flows to build the simulations for the 2050 scenarios and it is fairly complex. Future traffic is not known in advance therefore, the team needed to use the STATFOR datasets to forecast future traffic and socio-technical analysis to create future traffic scenarios. The future traffic scenarios are documented in D4.1 (white box, labeled 5) and have different potential levels of demand that serve as input to the stress tests. A model of the disturbances to be experienced in a 2050 scenario, require a modelling exercise - this analysis (white box, labeled 9) serves as input to the stress tests as well. Therefore, the future traffic scenarios and the disturbance analysis were used to build the stress tests (red box, labeled 10).

• The design principles (yellow box, labeled 11) and the stress tests (red box) have been used to build the third and last part: the REM2050 model. The exercise allows us to model the future scenarios by considering the resilience design principles rules (green box, labeled 12). The results of this simulation are compared to the simulation of future scenarios without using any design principles (green box, labeled 13). This comparison is the RES2050 performance assessment (yellow box, labeled 14).

The following table lists the 14-steps of the modelling exercise and reports the deliverable in which they have been documented:

Step REM2050 element Deliv. Description

1 Current traffic D2.1, D2.2

This is the current traffic database used. Eurocontrol provided the European traffic for the 4-year period between 2010 and 2014. For the first part of the project (WP2) a subset of 9 months of data from 2011 was used.

2 STATFOR data D4.1 This is the main data source used to build the forecast for future scenarios.

3 Socio-technical scenarios forecast D4.1 Different European socio-financial-political scenarios for 2050 were taken into account and reported in D4.1

Additional data sources D2.1 Weather, staffing and other disturbance datasets.

5 Future traffic D5.2 Using the STATFOR data and analysis for future scenarios built in D4.1, future actual traffic datasets are built for the 2050 simulations. This is reported in the deliverable D5.2

6 Current disturbances D2.1, D2.2

Extracted from the datasets, filtering affected flights

7 Resilience model and metrics D3.2, D3.3

R metrics methodology and calculation

8 Efficiency model and metrics D3.3 Efficiency metrics derived from the efficiency model

9 Disturbances analysis D2.2, D3.3

Assessment of disturbances at micro&macro level in D2.2. Implication of design principles in WP5 deliverables

10 Stress tests D5.1 The stress tests have been designed using the information from the analysis of current disturbances and the future traffic configuration. They are reported in detail in D5.1.

11 RES2050 Design Principles D3.3 Conclusions on design principles for a resilient ATM by design. Reported in detail in D3.3 and put in place in WP5

12 Simulation of the future scenario not (or only partially) taking into account the RES2050 design principles

D5.2, D5.3

Simulation results are reported in D5.3 and D5.2

13 Simulation of the future scenario taking into account the RES2050 design principles

D5.2, D5.3

Simulation results and assessment are reported in D5.3

14 RES2050 Performance assessment D5.3 Assessment of the balanced concept is reported in D5.3


5 Verification of the simulation environment

This Section describes how verification of the simulation environment has been achieved. This verification approach is inspired by the approach taken by the ComplexityCosts project that Innaxis participates in and is led by the University of Westminster.

5.1 The context of verification and validation Validation procedures ensure we are building the right model for the requirements laid out by Resilience2050. Verification ensures that the system is built correctly, without errors or software bugs (EUROCONTROL, 2010). Different models are developed according to different maturity levels defined in the table below: basic principles of phenomena, application formulated, development of proof of a concept, etc

Level of model Key characteristics

Level 1 Demonstrates a basic process Level 2 Characterises a particular phenomenon so that conclusions can be applied more widely Level 3 Reproduces a particular phenomenon as exactly as possible, usually with the intention of

prediction or retrodiction of another state, based on given changes

Different stages of concept development require different verification and validation techniques. According to this type of framework, Resilience2050 is considered to be between Level 1 and Level 2. The project was setup as a long term research effort and aims to reach TRL2. TRL2 is defined by NASA (2010) as a "technology concept and/or application formulated" (NASA developed the original TRL definitions) and echoed by SESAR (2015) in its Exploratory Research formulation, whereby:

• a theory and scientific principle is applied to a very specific application area in order to analyse and define the concept of costs trade-offs

• the characteristics of investment mechanisms and disturbance types are described, evaluating their potential cost benefits

• analytical tools are developed for simulation and analysis of the mechanisms under disturbance • uncertainty is taken into account and integrated into the model

Both formal validation and verification start at TRL5. However, some elements of validation and verification are recommended at any TRL. This section elaborates on different actions that have been carried out during the project to ensure verification is executed and major pitfalls are avoided. An early stage of the validation, performed at TRL2, should be understood as a credibility assessment. Once the simulation trials are done and results are analysed, it will be documented in deliverable D5.3,

5.2 Verification of the model implementation Verification of the model provides proof that the model has been implemented correctly and according to the specifications previously established by the project. It is not part of the verification to evaluate the conformity of the theoretical models to be implemented that are taken for granted. Rather, it is important to evaluate how closely the theoretical models have been implemented. Absolutely zero bug state in software programming is just a delusion: errors will always appear and unexpected behaviour is always possible. However, verification is responsible for ensuring that the number of errors is dropped to a minimum and that unexpected behaviours have the least possible impact on overall performance.

There are two main verification approaches used in the Resilience2050 development: dynamic and static testing. Dynamic testing implies running the model and running either a unit test (isolated components or class method) or integration test (groups of classes and interacting methods).


Dynamic tests are usually divided into three categories:

• Functional tests use simplified inputs with known outputs to evaluate critical functionalities of implementation.

• Structural tests use simplified inputs to test the code structure. That is to say, inputs may not correspond to a realistic scenario but rather a worst-case scenario. For instance, a case of extreme values testing.

• Sequential or random tests are exhaustive test tools in which inputs are thrown into the tested components and outputs are analysed checking for inconsistencies. For instance, using a simplified input set with minimal variation to perform a sensitivity analysis (i.e. expected solutions should be continuous and smooth).

On the other hand, static testing does not involve running the model or any part thereof. Rather, it involves source code analysis (coding practices, patterns use ratio, code design, compliance with standardised best practices, etc.) as well as right types for input and output data and parameter matching between methods. Exhaustive static tests have been performed against the model and in some cases, have led to improvements and debugging. However the static testing phase is an iterative process that continues during the whole project execution.

5.3 Integrity of model input data Integrity of the model data concerns both validation and verification. From a validation point of view, the right datasets have to be taken into account to ensure that there are not missing pieces in the modelling tasks. The data provided by Eurocontrol constitutes the most comprehensive dataset on air traffic available. However several of the data found in the understanding phase were incorrectly formatted, inaccurate, missing data or had other inconsistencies. This led to a subset of the original dataset that the team considered appropriate to compute metrics on. The integrity of the data and the subsequent cleaning process to ensure its integrity was executed in WP2 and reported in deliverable D2.2.

6 Visualisation 6.1 The need for better visualisation techniques The evolution of tens of thousands of flight operations every day in Europe requires complex coordination of airport resources, air navigation service providers, airlines, and authorities. All those interactions generate very large amounts of data that when correctly analysed, can provide extremely useful insights on air traffic management performance.

Analysing complex datasets is a challenging task. Without pre-defined models and metrics, the analysis of different ATM phenomena relies on expert analysis of data through visual representations. However, the complexity of the data and the interdependencies of the different datasets often require the comparison of datasets or analysis of the comparative evolution of different data. Interactive Visual Analytics is a powerful tool that allows the ATM experts to gather different insights on performance, enabling human visual perception to be a powerful tool to understand data patterns.

The last decades have given researchers very powerful computer tools capable of processing and representing large quantities of data. Current technology allows for standard ways to representing data through a number of standard graphic layouts and descriptive statistics that use standard computer tools (mostly general software, like Excel, but also more specific programming languages, like Matlab or R). Although the simplicity of the use of the tools allow experts with no specific training in visual analytics to produce simple graphics, more sophisticated, interactive, visually rich, dashboards require significant investment. Visualization and interactive exploration of complex and vast data constitute a crucial component of an analytics infrastructure.

Two different (and complementary) usage scenarios should be considered for Interactive Visual Analytics techniques:

• Analysis and pre-analysis of a concrete dataset, referred to as Visual Computing. • Dynamic dashboards, confluence between the two previous and driven by concrete Case Studies,

where visualisation meets decision making.

Having proper visualisation techniques is not just a communication task. Lack of proper visualisation tools forces the simplification of the indicators and the relationships presented so that humans can process the data efficiently. Pie charts, bar charts, scatter plots, histograms or bar charts are part of a limited visualisation framework. Therefore, the indicators used need to be simple: such as averages, percentages or measures of dispersion. The use of powerful visualisation tools allows for an increase in the complexity of the indicators without increasing difficulty when using of those indicators.

6.2 Visualisation techniques in Resilience2050 As part of an effort to build a modelling and simulation infrastructure, the project team has worked to research how visual computing and dynamic dashboards can help to efficiently communicate complex indicators. While building specific support technology capable of providing a visual analytic framework for a variety of metrics is out of scope, the project team researched the following challenges:

• Responsiveness or Interactivity in the knowledge discovery process, interconnecting state of the art frameworks for visualisation, database management and machine learning, being able to provide indicators for a variety of situations interactively and displayed interactively. Large datasets imply difficulties for visualisation and analytics systems to provide interactivity and data analysis simultaneously. Hence, new computational techniques and paradigm shifts should look into integrating user modelling and data modelling. Steering enables the operator to steer the system toward the desired subspace of the original dataset, avoiding unnecessary computations on the rest of the datasets and engaging the user in the process of complexity reduction. A possible solution is interactive query-based visualisation or human-assisted dimension reduction. This usually starts with a projection algorithm adjustable via parametric values. Automatic selection of the right visualisation for a particular data query, the right "way" to view it.


• Visualising complex datasets with tens of features. Dimension reduction models narrow complex data to two dimensions. For instance, relationships between high-dimensional data objects can be shown in two dimensions by leveraging dimension reduction models. These models include: principal component analysis, multi-dimensional scaling, force-directed layouts.

• Coupling of visual analysis techniques with data science computation. A possible solution is Semantic Interaction which allows for direct interaction with the information and thus, defines the models and gains insight. ATM data scientists require tools that integrate both disciplines without the need for knowledge of IT architectures, but uses the latest technology in visualisation, like D3. A visualisation can leverage on certain machine learning techniques, such as causal analysis or comparative dependency network learning. Both provide a more meaningful performance dashboard.

In this context, the project has chosen several "classical" visual means, like scatter plots, histograms and heatmaps. The following are some examples:

On top of that, effort has been specifically allocated to develop additional new vehicles for displaying the resilience metrics:

6.2.1 A circular resilience schematic map

The circular resilience schematic map presents the values of resilience per city-pair for all the city-pairs analysed and for all included disturbances. The map has the advantage of presenting the values for a entire European network and for all disturbances. The map has been built with interactive functions, so the entire metric spaces can be explored when used on a computer by moving the mouse over the city pairs, locking destinations or origins, etc. The interactive version of the map is displayed on the resilience2050.eu website. The technology used is D3.js (Javascript for data-driven documents).

6.2.2 A geographical interactive map

The same map can be displayed in a geographical lay-out and is also available on the resilience2050.eu website.


6.2.3 Propagation trees

Propagation trees presented in deliverable D33 present the values of the resilience metric for one single disturbance and the entire network. While these maps do not allow the dynamic representation of different disturbances and are more limited in terms of the interactivity, they have the advantage of being generated automatically by the data analysis computing infrastructure used by the project in Matlab. This automation allows for faster generation of different maps showing different indicators, like nominal values or values for the critical propagation.

7 References Adler, N., Hanany, E., & Proost, S. (2015). Managing Change in European Air Traffic Control Provision. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Alam, S., Abbass, H., & Barlow, M. (2008). Atoms: Air traffic operations and management simulator. Intelligent Transportation Systems, IEEE Transactions on, 9(2), 209-225.

Alam, S., Hossain, M.M., Al-Alawi, F., & Al-Thawadi, F. (2015). Shift for safety: A differential evolution approach to optimize lateral airway offset for collision risk mitigation. 11th USA/Europe ATM R&D Seminar Lisbon, Portugal.

Avery, J., & Balakrishnan, H. (2015). Predicting airport runway configuration: A discrete-choice modeling approach. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Balsamo, S., De Nitto Personè, V., & Inverardi, P. (2003). A review on queueing network models with finite capacity queues for software architectures performance prediction. Performance Evaluation, 51(2–4), 269-288. doi: http://dx.doi.org/10.1016/S0166-5316(02)00099-8

Belkoura, S., & Zanin, M. (2015). A micro view to en-route delays. 11th USA/Europe ATM R&D Seminar, Portugal, Lisbon.

Belle, A., McConnachie, D., & Bonnefoy, P. (2015). A methodology for environmental and energy assessment of operational improvements. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Birge, J.R., & Louveaux, F. (2011). Introduction to stochastic programming: Springer Science & Business Media.

Blom, H.A.P., Stroeve, S.H., & De Jong, H.H. (2006). Safety risk assessment by Monte Carlo simulation of complex safety critical operations. In F. Redmill & T. Anderson (Eds.), Developments in Risk-based Approaches to Safety: Proceedings of the Fourteenth Safety-critical Systems Symposium, Bristol, U.K., 7-9 February 2006: Springer.

Blom, H.A.P., & Bakker, G.J. (2012). Can airborne self separation safely accommodate very high en-route traffic demand? Proceedings AIAA ATIO Conference, Indianapolis, Indiana, USA.

Bonabeau, E. (2002). Agent-based modeling: methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences of the USA, 99(3), 7280-7287.

Bosson, C., Xue, M., & Zelinski, S. (2015). Optimizing integrated arrival, departure and surface operations under uncertainty. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal, p.

Chao, W., Jing, G., & Xiaohao, X. (2009). Analysis of Air Traffic Flow Control through Agent-Based Modeling and Simulation. Computer Modeling and Simulation, 2009. ICCMS '09. International Conference on, 20-22 Feb. 2009, p. 286-290.

Chen, B., & Cheng, H.H. (2010). A review of the applications of agent technology in traffic and transportation systems. IEEE Transactions on Intelligent Transportation Systems, 11(2), 485-497.

Clarke, J.-P., Melconian, T., Bly, E., & Rabbani, F. (2007). MEANS—MIT Extensible Air Network Simulation. Simulation, 83(5), 385-399.

Clarke, J.-P.B., Solak, S., Chang, Y.-H., Ren, L., & Vela, A.E. (2009). Air traffic flow management in the presence of uncertainty. Proceedings of the 8th USA/Europe Air Traffic Seminar (ATM'09), p.

Conway, S.R. (2006). An agent-based model for analyzing control policies and the dynamic service-time performance of a capacity-constrained air traffic management facility. ICAS 2006 - 25th Congress of the International Council of the Aeronautical Sciences; Hamburg; 3-8 Sep. 2006; Germany. http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20060048296_2006250468.pdf

Cook, A., Blom, H.A.P., Lillo, F., Mantegna, R.N., Miccichè, S., Rivas, D., … Zanin, M. (2015a). Applying complexity science to air traffic management. Journal of Air Transport Management, 42, 149-158. doi: http://dx.doi.org/10.1016/j.jairtraman.2014.09.011

Cook, A., Tanner, G., Cristóbal, S., & Zanin, M. (2015b). Delay propagation - new metrics, new insights. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.


Davidsson, P., Henesey, L., Ramstedt, L., Törnquist, J., & Wernstedt, F. (2005). An analysis of agent-based approaches to transport logistics. Transportation Research part C: Emerging technologies, 13(4), 255-271.

De Smedt, D., Bronsvoort, J., & McDonald, G. (2015). Model for Longitudinal Uncertainty during Controlled Time of Arrival Operations. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

De Visscher, I., Winckelmans, G., & Treve, V. (2015). A simple wake vortex encounter severity metric. 11th USA/Europa ATM R&D Seminar, Lisbon, Portugal.

Dhar, V. (2013). Data science and prediction. Commun. ACM, 56(12), 64-73. doi: 10.1145/2500499

EUROCONTROL, 2010. E-OCVM Version 3.0, European Operational Concept Validation Methodology, Volume I.

Everdij, M.H.C., Blom, H.A.P., Stroeve, S.H., & Kirwan, B. (2014). Agent-based dynamic risk modelling for ATM: A white paper: Eurocontrol.

Grether, D., Fürbas, S., & Nagel, K. (2013). Agent-based Modelling and Simulation of Air Transport Technology. Procedia Computer Science, 19(0), 821-828. doi: http://dx.doi.org/10.1016/j.procs.2013.06.109

Grether, D. (2014). Extension of a multi-agent transport simulation for traffic signal control and air transport systems PhD, Technical University of Berlin, Berlin, Germany.

Groskreutz, A.R., & Dominguez, P.M. (2015). Required surveillance performance for reduced minimal-pair arrival separations. 11th USA/Europa ATM R&D Seminar, Lisbon, Portugal.

Johnson, M., Mueller, E.R., & Santiago, C. (2015). Alerting criteria for encounters between UAS and manned aircraft in class E airspace. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Lee, S.M., Pritchett, A.R., & Corker, K.M. (2007). Evaluating transformations of the air transportation system through agent-based modeling and simulation. USA/Europe ATM R&D Seminar, Barcelona,Spain.

Liu, P.-c.B., Hansen, M., & Mukherjee, A. (2008). Scenario-based air traffic flow management: From theory to practice. Transportation Research Part B: Methodological, 42(7–8), 685-702. doi: http://dx.doi.org/10.1016/j.trb.2008.01.002

Long, D., Lee, D., Johnson, J., Gaier, E., & Kostiuk, P. (1999). Modeling air traffic management technologies with a queuing network model of the national airspace system: National Aeronautics and Space Administration, Langley Research Center.

Long, D., & Hasan, S. (2009). Improved prediction of flight delays using the LMINET2 system-wide simulation model. 9th AIAA Aviation Technology, Integration, and Operations Conference (ATIO), Hilton Head, SC, p.

Macal, C.M., & North, M.J. (2010). Tutorial on agent-based modelling and simulation. Journal of Simulation, 4, 151-162.

Meyn, L., Windhorst, R., Roth, K., Van Drei, D., Kubat, G., Manikonda, V., ... Couluris, G. (2006). Build 4 of the Airspace Concept Evaluation System. AIAA Modeling and Simulation Technologies Conference and Exhibit Keystone, Colorado, USA.

Mondoloni, S., Liu, S., & Kirk, D. (2015). Performance Improvements Through Trajectory Feedback in the Future Collaborative Flight Planning Environment. 11th USA/Europe ATM R&D Seminar Lisbon, Portugal.

Mukherjee, A., & Hansen, M. (2009). A dynamic rerouting model for air traffic flow management. Transportation Research Part B: Methodological, 43(1), 159-171. doi: http://dx.doi.org/10.1016/j.trb.2008.05.011

Muñoz, C.A., Narkawicz, A., Chamberlain, J., Consiglio, M., & Upchurch, J. (2014). A family of well-clear boundary models for the integration of UAS in the NAS. AIAA AVIATION 2014 -14th AIAA Aviation Technology, Integration, and Operations Conference, p.NASA, 2010. Technology Readiness Level Definitions - NASA. www.nasa.gov/pdf/458490main_TRL_Definitions.pdf (accesses August 2015)

Niedringhaus, W.P. (2004). The Jet:Wise Model of National Air Space System Evolution. Simulation, 80(1), 45-58. doi: 10.1177/0037549704042029

Nikolic, I., Van Dam, K.H., & Kasmire, J. (2013). Practice. In K. H. Van Dam, I. Nikolic & Z. Lukszo (Eds.), Agent-based modelling of socio-technical systems (pp. 73-137). Dordrecht, The Netherlands: Springer.


Proost, S., Glazer, A., & Blondiau, T. (2015). Air traffic control regulation in a union bargaining model setting. 11th USA/Europa ATM R&D Seminar, Lisbon, Portugal.

Pyrgiotis, N., Malone, K.M., & Odoni, A. (2013). Modelling delay propagation within an airport network. Transportation Research part C: emerging technologies, 27, 60-75. doi: http://dx.doi.org/10.1016/j.trc.2011.05.017

SESAR, 2015. SESAR 2020 Exploratory Research: First Call for Research Projects (V 1.1), March 2015.

Shah, A.P. (2006). Analysis of transformations to socio-technical systems using agent-based modeling and simulation. PhD, Georgia Institute of Technology.

Šišlák, D., Pěchouček, M., Volf, P., Pavlíček, D., Samek, J., Mařík, V., & Losiewicz, P. (2008). AGENTFLY: Towards multi-agent technology in free flight air traffic control Defence Industry Applications of Autonomous Agents and Multi-Agent Systems (pp. 73-96): Springer.

Sridhar, B., Soni, T., Sheth, K., & Chatterji, G. (2006). Aggregate flow model for air-traffic management. Journal of Guidance, Control, and Dynamics, 29(4), 992-997.

Sridhar, B., Chen, N.Y., Rodionova, O., Delahaye, D., Ng, H.K., & Linke, F. (2015). Strategic planning of efficient oceanic flights. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Stroeve, S.H., Blom, H.A.P., & Bakker, G.J. (2013). Contrasting safety assessments of a runway incursion scenario: Event sequence analysis versus multi-agent dynamic risk modelling. Reliability Engineering & System Safety, 109, 133-149. doi: 10.1016/j.ress.2012.07.002

Tandale, M.D., Sengupta, P., Menon, P., Cheng, V.H., Rosenberger, J., Subbarao, K., & Thipphavong, J. (2008). Queuing network models of the national airspace system. 8th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference, p. 14-19.

Taylor, C., Masek, T., Wanke, C., & Roy, S. (2015). Designing traffic flow management strategies under uncertainty. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal, p.

Upchurch, J.M., Munoz, C.A., Narkawicz, A.J., Consiglio, M.C., & Chamberlain, J.P. (2015). Characterizing the Effects of a Vertical Time Threshold for a Class of Well-Clear Definitions. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Van Baren, G.B., Chalon-Morgan, C., & Treve, V. (2015). The current practice of separation delivery at major European airports. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Van Dam, K.H., Lukszo, Z., Ferreira, L., & Sirikijpanichkul, A. (2007). Planning the Location of Intermodal Freight Hubs: an Agent Based Approach. Networking, Sensing and Control, 2007 IEEE International Conference on, 15-17 April 2007, p. 187-192.

Van Dam, K.H. (2009). Capturing socio-technical systems with agent-based modelling. Delft University of Technology, Delft, the Netherlands.

Van Dam, K.H., Nikolic, I., & Lukszo, Z. (2013). Agent-based modelling of socio-technical systems. Dordrecht, The Netherlands: Springer.

Wang, H., Wen, R., & Zhao, Y. (2015). Topological characteristics of air traffic situation. 11th USA/Europe ATM R&D Seminar, Lisbon, Portugal.

Xu, N., Sherry, L., & Laskey, K. (2008). Multifactor model for predicting delays at US airports. Transportation Research Record: Journal of the Transportation Research Board(2052), 62-71.

Zanin, M., & Lillo, F. (2013). Modelling the air transport with complex networks: A short review. The European Physical Journal Special Topics, 215(1), 5-21. doi: 10.1140/epjst/e2013-01711-9

Documents

Resilience2050 - s3.eu-central-1.amazonaws.com · 1.1.3 D4.2 Relation with Resilience2050 future tasks This deliverable will be a key input for Resilience 2050 research activities