Saab Propriety

TEST I VIDAREUTVECKLINGEN AV GRIPENS AVIONIK- OCH MARKSTÖDSYSTEM

Håkan Anderwall, Saab AB Jacob Gustafsson, Combitech 2011-11-17

Saab Propriety

AGENDA

!   The Gripen weapon system – an overview !   Part 1: The Gripen Aircraft Computer System

•  Scope – what is done, what needs to be done, and why? •  Aircraft Computer System, IMA •  Architecture and System functions •  Verification and Validation methods and goals •  Test environments and Test architecture •  Automated Verification and Validation flow

!   Part 2: The Mission Support System (MSS) •  Description of the MSS system •  The test process, steps & content •  The build process •  Test methods and experience

Saab Propriety

THE GRIPEN WEAPON SYSTEM

Gripen A/C Data Transfer Unit Mission Support System

Saab Propriety

A NEW AIRCRAFT COMPUTER SYSTEM IN GRIPEN !   Why?

•  Obsolescence issues for several equipments. •  Old development environment/tools, language. •  Software is not cost effective to maintain and add

functionality in.

!   A new IMA architecture has been engineered •  More effective, allocate functions more freely among

common computational resources. •  Focus on model based development, from models on

entire aircraft and its hardware and software components and their interfaces, to design of applications and library functions. (Using Rhapsody, BridgePoint, Matlab/Simulink)

!   Re-implemented system functions requires re-verification/validation!

Saab Propriety

INTEGRATED MODULAR AVIONICS (IMA)

!   IMA is a shared set of flexible, reusable, and interoperable hardware and software resources that, when integrated, form a platform that provides services, designed and verified to a defined set of safety and performance requirements, to host applications performing aircraft functions.

!   Apart from Gripen, there are IMA systems also in Lockheed Martin F-22 and F-35, Dassault Aviation Rafale, and Airbus A350 and A380.

!   Guidance and certification aspects for the development of an IMA system is described in RTCA DO-297.

Saab Propriety

Flight Critical Functions

IOP APP

Flight Critical Functions

IOP APP

Flight Critical Functions

IOP APP

Flight Critical Functions

IOP APP

GRIPEN - ARCHITECTURE

Flight Critical Functions

IOP APP

Tactical Functions

IOP APP

Displays

IOP APP

Head-up Display

IOP APP

Stores Functions

IOP APP

General Functions

IOP APP

Video Processing IOP APP

Data Transfer

IOP APP

Nav

igat

ion,

Flig

ht c

ontro

l, C

omm

unic

atio

n sy

stem

s et

c.

!   Many new DCM’s (Digital Computing Modules), each hosting two or more processors, using one of several new RTOSes supporting partitioning.

!   Re-implemented functions in APP (APlication Processor)

!   Kept interfaces to unchanged systems

!   A large amount of reverification/validation

Nav

igat

ion,

Flig

ht c

ontro

l, C

omm

unic

atio

n sy

stem

s et

c.

Saab Propriety

SYSTEM FUNCTIONS, SOFTWARE APPLICATIONS

!   A set of DCM software applications and libraries performs a system function using the IMA platform.

!   Needs to verify all CSCIs (software components), system requirements and validate system functions.

Symbolfile BIN

Symbolfile BIN

AGS (A661 Server)

Software Application Display_Server

Display

APP Partition

DCM

APP Partition

Software Application A Library Service

(e.g. A661 Adapter)

Sensor

Actuator

Saab Propriety

THE OVERALL GOAL FOR VERIFICATION AND VALIDATION

Software – 95% host/target breadboard test All tests fully automated Software – 5 % Equipment test Hardware – 100% Equipment vendor

< 5% Aircraft Ground and Flight test ~15-20% System simulation, some automated ~10-30% Analysis ~50-70% Reuse SW and HW level verification

~100% ref to SSS, SRS, and EqS verification results with analysis and justification

Stakeholder Requirements / contract

~100% ref to SSS, SRS, and EqS verification results with analysis and justification

Acceptance Test

Product requirements and functions

Product VoV

Subsystem requirements and functions

System integration tests

Software requirements (several levels)

Application/ Library/ Unit tests

Code & Models Static code & model analysis

Automated rules checking ~70% Manual rules checking ~30%

VoV C

ost

Saab Propriety

VERIFICATION METHODS - SOFTWARE

!   SW/SW Integration Tests •  Black-box verification of CSCI (Application, Library)

!   HW/SW Integration Tests •  Integration with hardware interfaces (mostly the platform CSCI:s).

!   Unit Tests •  Verification of Computer Software Units.

!   Analysis/Inspection •  Visual inspection of code or design. •  Static analysis of models and code where possible.

Saab Propriety

VERIFICATION & VALIDATION METHODS - SYSTEM !   Analysis

•  Analysis of design, specifications etc. of a systems ability to perform its intended functions.

!   Inspection •  Inspection of a system items fulfillment of required characteristics (e.g.

emission, conformity to standards and regulations)

!   Test & Demonstration •  MySim •  SW verification (reused) •  Simulation (models) •  Flight Test / Ground Test •  Avionics Rig / Total System Rig

!   Service Experience •  Previous usage of subsystem/system

Saab Propriety

MySim !   Desktop simulator for system development, and software

development. !   Validation of concepts. !   Background mode:

Automated execution of system integration test cases.

Saab Propriety

VERIFICATION AND VALIDATION – AUTOMATION GOAL !   Compile and link all tests for different test environments:

•  MySim Automated system integration tests

•  Solaris/Linux Host software integration and unit test, code coverage measured •  Target breadboard Target software integration and unit test, object code analysis •  Testrig with DCM Hardware/software integration test (incl. WCET)

!   Execute all possible tests automatically and evaluate: •  MySim, Host, Target breadboard •  Daily VoV result presentation – how many test failed etc.?

!   Static code analysis against C/C++/Ada95 ”Coding Standards” !   Model consistency check of models against each tool ”Modeling

Standard” !   Automated model code generation !   DOORS checks – Untested / changed requirements, traceability etc.?

Saab Propriety

Test environments (Applications and libraries)

Test in Models

Test on Host Test on Target

Processor (breadboard)

Collect evidence of the CSCIs requirements fulfillment

No big surprises!

Daily test of generated code

Code coverage

Quick feedback

Automated tests enables quick regression testing!

Test in System Simulator (MySim)

Test in DCM (Rig)

CSCI Level

System Level

Application/ Application integration

Application/ Platform integration

Manual Automated

Partly automated

Saab Propriety

TEST ARCHITECTURE: BREADBOARD TEST CLUSTER !   A set of breadboards for all DCM CPU architectures !   Managed by a continuous integration server deploying test binaries

per dedicated CPU architecture Flight

Critical Functions

APP

Tactical Functions

APP

Tactical Functions

APP

Tactical Functions

APP

Flight Critical

Functions APP

Tactical Functions

APP

Tactical Functions

APP Displays

APP

Head-up Display APP

Tactical Functions

APP

Tactical Functions

APP

Video Processing

APP

Breadboard test server

Use

r net

wor

k

Continuous integration server

<<Send test binaries>>

<<Send test results>>

Flight Critical

Functions APP

<<Deploy and Execute>>

<<Get test result>>

Saab Propriety

AUTOMATED VERIFICATION AND VALIDATION FLOW

Check-out, compile, link, execute

CI server: sw test

Host test

report

*.atr *.ctr *.log

Post process report files

transform

xml test

summary

transform & present

Test Page Summary CI X Pass

transform

*.csv import (manual) DOORS

SVC module

Check-out, cross compile, link

report *.atr *.ctr *.log

Target test

target breadboard Execute

CI server: system integration test

Check-out, compile, link, execute

MySim test

report

*.log

DOORS SSTD

module

Accumulated

code coverage

Saab Propriety

Content

!   MSS and its vital role in the tactical loop !   The different test steps and phases !   The build & installation process !   Experience from different test methods

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

MISSION SUPPORT SYSTEM (MSS) Scope of Functionality

!   Planning missions objectives and routes for Gripen missions

!   Loading planned data to the aircraft !   Loading flown data from the aircraft !   Analysis and replay of the flown missions !   Functionality for mission simulations

!   The functionality is vital to achieve mission success

Saab Propriety

Data loading overview

Digital Maps

Library data

Communication Data

MSS Mission Support System

DTU-P

MMC

Order, Intelligence, Weather & Reports

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

Data Support

Map info

Libraries

Rehearsal Scenario

Simulation

3D View

Preparation Data Libraries

Navigation data Other data Settings

Evaluation

Mission Replay including • Flight path

• Events

• Cockpit

• Audio/Video

Mission Analysis

Planning

Navigation

Sensors

Communication

Configuration Intelligence Updates

DTU-P MMC

Mission Report

Intelligence Data

Recorded Data

Flight Path

Events

Cockpit panels

Audio & Video

Image Analysis

TACTICAL INFORMATION HANDLING

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

Test phases (performed weekly)

Automatic build of the code

Load on host for development

Development tests (unit & functionality)

Early feedback

Load on target for further testing System test

CSCI Level

System Level

Sanity & integration testing

Early feedback Validation test

Performed in steps throughout the development Requirement tests

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

Type of test Responsible Definition

Code analysis Development team Code analysis to find initialized variables, false pointers etc.

Unit test Development team Development test in parallel with the implementation

Functional test Development team Functional tests of the implemented code

Integration test Development team Tests in the teams to early assure correct integration between components

Sanity test Common test team Short “up & running” test of the complete product

Integration test of complete product

Common test team Integration test performed weekly & every sprint (3 weeks) to assure that the integration of the complete product is correct

System test Development team Contractual requirement tests with validation perspective

Dry Run & Cust Demos Common test team Customer demos to present the system

Validation test Common test team Final test with validation perspective

TEST ACTIVITIES

Saab Propriety

Testing characteristics

!   The amount of automatic unit testing is increasing. Time consuming for old code as it requires much effort

!   Still a large amount of manual testing !   Integration testing on target is tested from a user point of

view. Uses several simulation tools to verify interfaces !   GUI testing of the integrated product has not been

successful. Tricky to develop and requires a lot of under haul during development

!   System & final testing is performed with a validation perspective in order to assure customer satisfaction according to contractual requirements

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

Build overview MISSION SUPPORT SYSTEM (MSS)

.h .cpp svn

code repository

gcc

installations

buildscripts jenkins

unit tests code analysis

.iso .rpm .exe

test lab

test results installation files

Saab Propriety

Build process & installation

The build process is automatic and serves many advantages: !   Enables quick response & support to developers. Host system

environment is similar to the target environment !   Easy to switch between customer configurations on host machines !   The code is built automatically once checked in !   Build results are distributed to the development teams !   Several customer configurations are built over night !   Unit tests are run automatically as regression tests with distributed

results !   Installation requires little manual effort and will be performed

automatically

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

Test methods & experience !   Team organized versus functional organisation !   Used to have separate teams for integration & system testing !   Intended to shorten development time and increase quality !   Team organisation:

•  Resulting in many faults discovered late in the development process •  “Somebody else” tested the development teams implemented functionality •  Development teams relied on the test teams to find faults •  Unless good communication, the test teams were not fully updated on “what’s new”

!   Functional organisation: •  The teams are responsible for a wider functional scope •  The testers are located in the development teams •  The testers know exactly which functionality that is under development •  Integration tests are performed earlier in the development phase •  Test cases are developed with a widened content

MISSION SUPPORT SYSTEM (MSS)

Saab Propriety

Contact

Jacob Gustafsson E-mail: jacob.gustafsson@saabgroup.com Phone: +46709-891744

MSS

Contact

Håkan Anderwall E-mail: hakan.anderwall@saabgroup.com Phone: +46734184234

GRIPEN