Embed Size (px)
Citation preview
Your best ally A-CDM Systems Testing
in a collaborative way
Manuel Fernández Astaburuaga, ASDA July 2013
1 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
1
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
A-CDM concept
Insufficient or unreliable information
No convergent interests between stakeholders
No single partner has the complete picture
Accurate information is provided too late for a partner to be ready
High operating costs
Lack of image and customer’s satisfaction due to delays, poor information, etc…
The problems A-CDM has to face
Improve Airport Operations Management in Air Side
2 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
2
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
A-CDM concept
By reducing delays, improving predictability and optimizing the use of all infrastructure and
resources, technical and human…
Achieve better resource allocation increasing capacity and safety
Improve productivity
Increase punctuality
Reduce environmental impact
Reduce operating costs
A-CDM objectives:
Better information sharing eliminates a very large proportion of these problems.
Each Airport CDM Partner must share with others the defined information
in an accurate and timely way, in order to optimize their decision, knowing their
preferences, limitations and constraints.
3 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
3
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Information sharing
As it is said in the implementation manual:
“Information Sharing is the core Airport CDM Element and the foundation for the other
Airport CDM Elements. It needs to be implemented before any other Concept Element”
All participants shall adapt their personnel, systems, procedures and tools to this philosophy
Largest amount of information is shared within systems
This adaptation in the SW needs TESTING
Information sharing within systems and participants is the glue that ties everything together.
Testing guarantees that this glue is strong enough for it’s purpose.
Integrated testing and information sharing between partners, from the very
beginning of the life cycle will allow VERIFYING and VALIDATING all systems and
procedures in the A-CDM in a holistic way.
This way of work will consolidate the A-CDM concept in a TECHNOLOGICAL
PLATFORM giving parameters and indicators which will be the key to success.
4 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
4
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
A-CDM STAKEHOLDERS
Testing
LAB
Ground Handler
Air Navigation
Service Provider
Airport Operator
Airlines
CFMU
Each of these actors may have different interests, objectives and policies
They use several different tools, system technologies and providers
But they have to work closer together when it comes to decision making
CHALLENGE: to drive for increased integration of computer systems across
multiple organizations in order to provide immediacy, accuracy and completeness
to the distribution of information to help their organizations to succeed in the
overall integration within the A-CDM
5 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
5
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
How to ensure Stakeholders collaboration?
BOTTOM-UP, technically
TOP-DOWN strategically
BOTTOM-UP
Stakeholders Systems integration will be facilitated and assesed in the Test laboratory.
To enable further operational use, the adaptation of each system, or any new one, have to be
tested, which means ensuring that those systems can exchange information with each other
and can use that information properly, without endangering their core functions or the others.
TOP-DOWN
Test Project Plan is needed to get all systems on time in the right place, the test lab facility,
with the changes implemented. In this Test Project Plan, together with dates, commitments
and other features, all stakeholders should be compelled to provide the best quality
information available.
Service Level Agreements (SLA)
Non-disclosure Agreements (NDA)
Collaboration between stakeholders must be acheived from two ways:
6 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
6
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
How to ensure Stakeholders collaboration?
Service Level Agreements (SLA)
SLA signed taking into account the nature of each participant (developers, testers,
requirements...), including penalties when incompliance.
Scheduling
Quality
Availability of the information
Non-disclosure Agreements (NDA)
stakeholders will have to share information concerning the core of their business, even with
competitors
the appropriate access levels for sensitive information will have to be defined in advance and
implemented in a CMS
7 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
7
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
0
10
20
30
40
50
60
70
Relative Cost to correct a defect
Development
Testing Acceptance
Testing
Operation Requirements Design Code
The importance of testing throughout the whole process
Continuos information sharing and collaboration
is needed from the first stages.
Considering only
engineering costs
Testing means verification and validation in all
stages.
Evidences of stage completion is guarantee of
quality
8 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
8
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Requirements Design Other Code
Defects Causes Estimated effort to correct the defects
56%
7% 10%
27%
Requirements
Errors (82%)
Other Errors (4%)
Design Errors
(13%)
Coding Errors (1%)
The importance of good requirements
Special effort on: Requierements definition, review, validation and updating. All stakeholders have to be informed of any error, change or update. Use of Life cycle tools, and CMS ( expert panels, wiki, forum, blogs…)
Continuos and direct participation of Airport OPS personnel is mandatory.
9 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
9
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
RISK MANAGEMENT (project and product risks)
Risk Management is needed
INFORMATION SHARING
RISKS
UPDATE
New risks
Severity update
Mitigation
measures MONITORING
TESTING
SW Development
THREADS as
Environment limitations
Time constraints
Requirements Specification
10 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
10
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
QUALITY ASSURANCE
TEST DESIGN
AND
IMPLEMENTATION
TEST
ENVIRONMENT
DEFINITION,
SET-UP
AND DATA
SELECTION
TEST
EXECUTION
TEST
REPORTING
TEST PROCESS
TEST MANAGEMENT PROCESS
TEST PLANNING TEST MONITORING
AND CONTROL TEST COMPLETION
Test Process
11 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
11
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
TEST DESIGN
To ensure the coverage of all the functional and technical requirements by test case
definition and specification, in order to verify them.
The validation of the operational concept, by continuous reviewing of the requirements.
The certification of the systems and concepts by testing in an Operational Environment as
close as possible to the final one with previously selected data.
To test a shadow-mode operation, whereby the appropriateness of the parameters can be
checked against real values.
Test design goals:
Test design includes:
Test Environment definition and set-up.
Test Data selection (real or simulated).
HOW?
WHEN?
WHICH? WHAT?
12 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
12
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Tools integration to facilitate Traceability
R1
.
.
.
Ri*
.
.
.
Rn
Rn+1
.
.
.
Rm
CR1
.
.
.
.
.
.
.
.
CRy
.
.
.
.
.
CRw
Test cases
R1
.
.
.
.
.
.
.
.
.
Rm
Requirements
Test Plan
.
.
.
ID1
.
.
.
IDy
Requirements
Test Management Tool
Issues, Defects
Ex1 ... EXn …EXn+1 … EXm
Test run Executions
SW development
Configuration
items
TC1
.
.
.
.
.
.
TCx
Test Data
Test environment
definition
13 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
13
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
AODB context diagram in AENA case study
Network
FIDS
A-SMGCS
AIRPORT SLOTS
ALLOCATION
SYSTEM
Aircraft movements
eCDM
CUTE
AIR TRAFFIC CONTROL
AUTOMATED SYSTEM
AMAN DMAN
SLOTS
Flight
updates
FPL updates
CFMU
DPI FUM AIRLINE AND GROUND
HANDLING RESOURCES
AND OPERATIONS
MANAGEMENT SYSTEMS
TSAT
TTOT
TLDT
MVT
Flight
updates
TOBT
Flight
updates
AODB
AIRPORT REPORTING
OFFICE AUTOMATED
SYSTEM Network
HOW?
WHEN?
WHICH? WHAT?
14 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
14
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
LAB Environment. Simulation Tools
FIDS
Aircraft
movements
eCDM
AIR TRAFFIC CONTROL
AUTOMATED SYSTEM
AMAN DMAN
SLOTS
Flight updates
FPL
updates
CFMU DPI
FUM
TSAT
TTOT TLDT
Flight
updates
TOBT
Flight updates
AODB
SIMULATION TOOL
LAB
CUTE
SIMULATION TOOL
A-SMGCS
AIRPORT SLOTS
ALLOCATION
SYSTEM
AIRPORT REPORTING
OFFICE AUTOMATED
SYSTEM
MVT
Network
AIRLINE AND GROUND
HANDLING RESOURCES
AND OPERATIONS
MANAGEMENT
SYSTEMS
15 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
15
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
SYSTEM SIMULATION
TECHNOLOGY INFORMATION TECHNOLOGY
Airlines Network SW Tools to simulate
messages MVT Middleware
CUTE Real System
operating ASBT
SOCKET TCP
Middleware
Airport Reporting
Office Automated
System
SW Tools to simulate
messages
FPL regulation
messages updates
(FLS, SAM, SLC,
SRM), FUM, DPI
Middleware
Air Traffic Control
Automated System
SW Tools to simulate
messages
Flight updates,
DMAN: TSAT, AMAN:
TLDT
Middleware
A-SMGCS SW Tools to simulate
messages OBK, IBK, DEP, LND Middleware
Data simulation
Simulation capacity is required in cases that controlled data is needed or when the system is
not available
Simulation is necessary when performing integration test:
From 1-1 to N-N
16 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
16
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Test execution
The purpose of the Test Execution Phase is to execute the Test Plan designed before in the
controlled and monitored test environment and with the selected data.
Actual results are recorded and compared with the expected results.
Final results will be made available to the Airport CDM Partners to enable them to make
improvements as necessary.
Throughout the execution of the tests, Monitoring and Control determine whether testing progresses in accordance with the Test Plan, identifying risks to initiate control or mitigation actions if necessary, and to identify necessary updates to the Test Plan or
even to the Requirements Specification.
17 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
17
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Validation
Once verification of the requirements has been completed, data have to be analyzed and
compared with real operational data in order to validate the A-CDM concept.
Information given by each system will be compared:
ESTIMATED VS ACTUAL
Information given by each system will be compared against others, including statistics of real
data
PRIORITIES, PARAMETERS, TABLES AND ALGORITHMS
Post-event analyses will show when a process turned out unsuccessful, will indicate the
reason and will allow finding and proposing solutions
18 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
18
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Validation. A-CDM Milestones
Subtítulo de la Diapositiva
ATC Flight plan Activation
EOBT-2hr
Take off oustation
Local Radar update
Final Approach
Landing
In block Ground handling
start
Update TOBT
TSAT Issue
Start Boarding
Aircraft ready
Start up request
Start up approved
Off block
Take off
INBOUND TURNAROUND OUTBOUND
19 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
19
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
MS7
AIBT ELDT AIBT vs EIBT = ELDT+EXIT
ACARS ACARS ∆%
A-SMGCS A-SMGCS ∆%
Validation
ARRIVAL TIME ACCURACY
MS16
AOBT ATOT ATOT vs TTOT
=AOBT+EXOT
ACARS ACARS ∆%
A-SMGCS A-SMGCS ∆%
MVT MVT ∆%
DMAN ∆%
DEPARTURE TIME ACCURACY
20 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
20
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
MS15
AOBT AIBT AOBT- AIBT vs SOBT-SIBT
ACARS ACARS ∆%
A-SMGCS A-SMGCS ∆%
Validation
TOBT ACCURACY
MTTT ACCURACY
MS12
TOBT ARDT TOBT vs ARDT
ACARS ACARS ∆%
MS13
TOBT ASRT TOBT vs ASRT
ACARS ACARS ∆%
21 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
21
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Test completion and report
Once the test cases are executed within the Operational Trial Scenarios, test results are
recorded, evaluated and communicated to all relevant stakeholders. This phase is crucial
and it has several outcomes and goals:
All test requirements are satisfied and verified; if not, via detailed incident reporting defects
should be corrected as soon as possible to face another test run
CDM concept is validated
Staffing and training needs are identified and communicated to all stakeholders
Lessons learned are recorded and shared.
Conclusions and recommendations are a must to get feedback to all participants in every
phase and to fulfill continuous improvement of the process and to all the inputs
(requirements, data, test definition, test environment, procedures …)
22 Isdefe
08.07.2013
Título
de la
Pre
senta
ció
n
22
A-C
DM
Syste
ms T
estin
g in
a C
olla
bora
tive w
ay
Information sharing
Needs Airport OPS
Procedures
Requirements Technical
Design
Users Analists Technical Team
Code
From users needs to code and return
Su mejor aliado Your best ally
Isdefe
C/ Beatriz de Bobadilla, 3
28040 Madrid
Tel.: +34 91 411 50 11
Email: [email protected]
www.isdefe.es
