DO-178B to DO-178C

HighRely: Rely On Us 6h

2009-4-VH

Copyright 2011 – Vance Hilderman Slide 1

By Vance Hilderman, Principal Founder, HighRely Inc.

[email protected]


2009-4-VH

Copyright 2004 -2011 – Vance HildermanSlide 2

• “DO-178 is the worst standard in the world; except for all the others …” (Winston Churchill Paraphrased)

• “The School Of “Avionics Wishful Thinking” has many students, but no graduates …” (Vance Hilderman)

Almost Famous Quotes


2009-4-VH

Copyright 2004 - 2011 – Vance HildermanSlide 3

Overview

• Aviation Safety Framework

• Aircraft, Systems, Hardware and Software

• DO-178B and DO-254: History

• Scope and Application

• DO-178B Overview

• DO-178C Overview

• DO-178C Implications on

Verification & Certification


2009-4-VH

Copyright 2005-2010 Slide 4

About AtegoHighRely

• North America’s Largest Avionics Services Company

– 30% Avionics Software Engineering

– 20% Avionics Systems Engineering

– 20% Avionics Software/Hardware Testing

– 10% DO-178/254 Training & Project Management

– 10% Strategy, Gap Analysis, JumpCert-178

– 10% DER’s/Certification

• Partnerships with the top tool vendor in all five major product categories

• Largest repository of DO-178B/DO-254 Whitepapers

One Stop Supplier for all your avionics development needs.


2009-4-VH

Copyright 2004 -2011 – Vance Hilderman Slide 5

Safety, System, Software & Hardware

Software

DO-178B

Hardware

DO-254

System Development

ARP 4754

Safety

Assessment

ARP 4761

• Criticality Level

• Architectural

Inputs

SW Rqmts HW Rqmts

Tests Tests


2009-4-VH


• Certification standards for airborne equipment– DO-178 => Software

– DO-254 => Hardware

• Regulated by the FAA

• Required if target aircraft flies in commercial U.S. airspace

• Covers full engineering lifecycle:– Planning (CM, QA, Development, Testing)

– Development (Requirements/Design/Implementation)

– Testing/Verification

– Certification

What are DO-178 and DO-254?


2009-4-VH


“Guidance” (“Considerations”)

• “Considerations”, not

requirements

• Developed via committee: all

things to all people

• Applies to huge and tiny

systems alike

• Applies to very critical Level A

systems, and also Level D

• Complaints by industry that

compliance with 178/254 is

vague and too expensive; true?


2009-4-VH


History of DO-178 and DO-254

• RTCA DO-178B: “Software Considerations in Airborne Systems and Equipment Certification”

• Developed 1980 – 1992 via 100+ Industry and Government personnel

• Many compromises to satisfy different goals

• Not a recipe book or “How To” guide

• “Discussion” flow for guidance; able to accommodate many different development approaches

• Lawyers versus Software Engineers; who wins?

• In practice: The Golden Rule …


2009-4-VH

Copyright 2004 - 2011 – Vance Hilderman Slide 9

DO-178: Evolution History

Version Year Basis Themes

DO-178 1980 -

1982

498 &

2167A

Artifacts, documents, traceability,

testing

DO-178A 1985 DO-178 Processes, testing to improve

quality, components, four

criticality levels, reviews, waterfall

methodology

DO-178B 1992 DO-178A Integration, transition criteria,

diverse development methods,

data (not documents), verification

to assess quality, tools

DO-178C 2008?

(underway)

DO-178B Reducing subjectivity; Address

modeling, OOT, improved tools,

formal methods


2009-4-VH

Slide 10

Avionics Safety History: 1946 - 2008


2009-4-VH


DO-178B Document Layout

1. Planning

2. Development

3. Correctness

1. Overview

2. System Aspects

3. Lifecycle

4. Planning Process

5. Development Process

6. Verification

7. Configuration Mgmt

8. Quality Assurance

9. Certification Liaison

10. Overview of Aircraft And Engine

Certification

11. Data & Considerations

A. Objectives by Cert Level


2009-4-VH


Three Key Processes(same for DO-178 and DO-254)

• Planning Process – Occurs first

• Development Process – Follows Planning

• Correctness Process – Continuous Throughout

Project

1. Planning

Process

2. Development

Process

3. Correctness Process


2009-4-VH


Optimal DO-178 & 254

Engineering RouteSafety

Assessment

& RqmtsSystems

Rqmts

Develop Plans,

Stnds, Chklsts

Develop

Traceability

Implement

CM

High-Level

Rqmts

Start QA

Low-Level

Rqmts

Design

Code &

Logic

Verification & Validation

Time (Planning Phase)

Time (Development & Correctness Phases)

Integration

Conformity

Review

SOI

#1

SOI

#2

SOI

#3

SOI

#4

Cert


2009-4-VH


DO-178 and DO-254 Key Attributes(similar for DO-178B and DO-254)

1. Detailed planning

2. Five Criticality Levels (A, B, C, D, E)

3. Consistency & Determinism

4. Traceability: top-to-bottom, and back

5. Independence (especially Levels A/B)

6. Path testing

7. Proven Tools (“Qualification”)

8. Up to 20 artifact types and 66 objectives

9. “Guilty Until Proven Innocent”

http://soulworker.punt.nl/upload/key.gif







2009-4-VH


DO-178B Objectives by Level

• Level A: 66 Objectives (25 with independence)

• Level B: 65 Objectives (14 with independence)

• Level C: 57 Objectives (no mandatory independence, but independent reviews recommended)

• Level D: 28 Objectives (no mandatory independence)

• Level E: No Objectives


2009-4-VH


DO-178B Five Key Plans

PSAC: Plan for Software Aspects of Certification

SQAP: Software Quality Assurance Plan

SCMP: Software Configuration Management Plan

SWDP: Software Development Plan

SWVP: Software Verification Plan*** Plus 3 Standards: Requirements, Design and Coding

1.

PSAC

2.

SQAP

3.

SCMP

4.

SWDP

5.

SWVP








2009-4-VH


Scope of DO-178B & DO-254?

PLD

ASIC

FPGA

CPU

RTOS

BSP

Math

APP SW

Drivers

DO-178B

DO-254

Typical Avionics LRU


2009-4-VH


Criticality Levels

Criticality Level Pyramid

A

B

C

D

E

• Level A: Catastrophic

• Level B: Hazardous/Severe

• Level C: Major

• Level D: Minor

• Level E: No Effect


2009-4-VH


Criticality Levels

• “Software whose anomalous behavior, as shown by the system safety assessment process, would cause or contribute to a failure of system functions…

A. …resulting in a catastrophic failure condition for the aircraft.” Level A = <1E-09

B. …resulting in a hazardous/severe-major failure condition for the aircraft.” Level B <1E-07

C. …resulting in a major failure condition for the aircraft.” Level C <1E-05

D. …resulting in a minor failure condition for the aircraft.” Level D > 1E-05

E. …with no effect on aircraft operational capability or pilot workload.” Level E = No further application of 178/254 required.

• 4

Level A

<1E-09

Level B

<1E-07

Level C

<1E-05

Level D

>1E-09

Level E

NA

Level E

NA

Level D

>1E-05

Level C

<1E-05

Level B

<1E-07

Level A

<1E-09


2009-4-VH


Typical System Criticality Level

DAL examples of Systems

DO178B Level Sample of historical Systems

A Flight controls, Engine Controllers,

Primary Displays

B FMS, Many Radios and communication

systems. Many navigation Systems

C Back up Displays, back up

communication systems (SATCOM),

D Maintenance systems, Monitoring

Systems (Engine Vibration Monitors,

etc.)

E In Flight entertainment Systems (May be

Level D), Video Systems. Coffee makers

and galley services.


2009-4-VH


Why Different Criticality Levels?

• Why Does 178/254 Have Different

Criticality Levels?

– Who were major 178/254 contributors?

– What were their major concerns?

• Schedule

• Cost

• Safety, but with reasonableness

Level A

<1E-09

Level B

<1E-07

Level C

<1E-05

Level D

>1E-05

Level E

NA


2009-4-VH


DO-178B Criticality Level Comparison

(NOT for DO-254; See DO-254 Whitepaper!)

DO178B Aspect Level A Level B Level C Level D

Independence Level High Medium Low Very Low

Necessity of Low-Level

RequirementsYes Yes Yes No

Statement Structural Coverage Yes Yes Yes No

Decision/Condition Structural

CoverageYes Yes No No

MCDC Structural Coverage Yes No No No

Configuration Management Tight Tight Medium Low

Source to Binary Correlation Yes No No No

Requirements Correlate to

Target processorYes Yes No No

Architecture & Algorithms

VerificationYes Yes Yes No

Code Reviews Yes Yes Yes No

SQA Transition Criteria Yes Yes Yes No


2009-4-VH


Special Terminology

• “Certified”: the entire “system” is Certified for flight, while

components (LRUs) may have different certification Levels

• “Certifiable”: a component (LRU) within a system achieving its

highest certification status prior to certifying it with a “certified”

system

• “Compliant”: certification via an entity other than the

FAA (e.g. Military or non-commercial avionics)

• “Qualified”: formal approval of a tool which (since it does not “fly”)

does not require “certification”


2009-4-VH

Copyright 2004 -2011 – Vance

Hilderman

Slide 24

178/254 For Military?

• Since 2001, worldwide militaries adopting 178/254.

Why?– Improved subcontractor consistency

– Improved re-usability

– Improved schedule

– Improved cost

– Improved reliability

– Safety requirements

• “Military” 178 Examples:

– JSF, C-17, C-130, A400M

– Global Hawk

– T50A


2009-4-VH

Copyright 2004 -2011 – Vance

Hilderman

Slide 25

DO-178’s Verification Equation

V = R + T

• Verification = Reviews + Tests

• What is Reviewed?

– Virtually Everything (for Levels A, B, & C)

• What is Tested?

– All Requirements & All Code (for Levels A, B, & C)


2009-4-VH


Avionics Testing

Four Categories of Tests:

1. Functional Tests

– All Requirements

2. Normal Range Tests

– “Sunny Day” conditions

3. Robustness Tests

– “Rainy Day” conditions

4. Structural Coverage Tests

– Cover all code

Test

Functional

Tests

Normal Range

Tests

Robustness

Tests

Structural

Coverage

Analysis


2009-4-VH


Software Testing

• Four Categories of Tests:

– One Black Box

– Three White Box

– Mind the Overlap

– Note the Relative Sizes: to Scale

SW

Test

Functional

Tests

Normal Range

Tests

Robustness

Tests

Structural

Coverage

Analysis


2009-4-VH


Black Box Vs. White BoxSW

Test

Functional

Tests

Normal Range

Tests

Robustness

Tests

Structural

Coverage Tests

Black Box Vs. White Box

What is the difference?


2009-4-VH


Weaknesses of DO-178B Verification

• Overly dependent upon subjective:

– Requirement granularity

– Design/Coding standard review criteria

• Complex asynchronous interactions not necessarily

verified at functional/system level

• Reduced benefit of model based development (MBD)

and advance development tools

Which leads us to DO-178C …


2009-4-VH

Copyright 2011 – Vance HildermanSlide 30

DO-178C Preview

• Almost 20 years since DO-178B released

• Software landscape has changed ...

• Advancements in:

– Tools & Automation

– Modeling & Object Oriented Technology

– Formal Methodologies

• Commercial world has embraced the

above; Avionics has slowly followed …


2009-4-VH


DO-178C Preview

• Since 2005, committees have met to

discuss, and update, DO-178B

• Like 178B, includes Industry & Agencies

• Unlike 178B, more Tool Vendors

– Obvious focus on “acceptability” of certain

types of tools, particularly “theirs”

• Predominantly America & Europe, nearly

equal; quarterly meetings.


2009-4-VH


DO-178C Preview

• Seven “Sub-Groups” (SG’s):

1. SG1: Document Integration

2. SG2: Issues & Rationale

3. SG3: Tool Qualification

4. SG4: Model Based Design (MBD) & Verification

5. SG5: Object Oriented (OO) Technology

6. SG6: Formal Methods (FM)

7. SG7: Safety Related Considerations (and

ground-based systems)


2009-4-VH

Copyright 2011 – Vance HildermanSlide 33

DO-178C Preview

• Unlike the DO-178A to DO-178B update, the “core”

update to 178C is modest

• Instead, changes are handled via four “Supplements”,

which “clarify”:

• A. Tools Supplement

• B. MBD Supplement

• C. OO Supplement

• D. FM Supplement

• Reduced subjectivity for Testing


2009-4-VH


Tool Qualification

DO-178B:

• Two Criteria:

1. Development

2. Verification

DO-178C:

• Three Criteria:1. Development

2. Verification & Augments other

development or verification

activities

3. Verification only

• Five Tool Qual Levels:

1. For Level A

2. For Level B

3. For Level C

4. Tool Operational Rqmts (TOR),

Arch, Additional Verification

5. TOR Verification


2009-4-VH


MBD & OO (Continued)

DO-178B:• No Explicit Provisions

• Assumes “structured design”

• OO acceptance, but user-defined (subjective)

• Maximize Determinism & Visibility

• Weak on OO and MBD traceability

• Weak on structural coverage application to OO & Models

DO-178C:

• Allow controlled modeling & OO

• Bound MBD & OO acceptability

• Emphasize traceability

• Address memory management &

exception handling

• Verify “type consistency” (verify

substitutes,

• Each subclass passes all tests

applicable to parent

• Verify all callable methods for each

invocation

• Emphasize detailed MBD & OO design

standards

• Allow defined generics

• Acceptable Virtualization (“code” versus

“data”)


2009-4-VH


Formal Methods (for Verification)

DO-178B:• No Explicit

Provisions

• (But commonly

applied, subjectively,

in Europe via ED-

12B)

DO-178C:• Recognize acceptance of formal

methods for:

– Requirements correctness,

consistency, and reviews

– Source code reviews, particularly

autocode generation from models

(low level requirements)

– Test cases covering low level

requirements

– Replacement of some forms of

testing via formal method-based

reviews

– “Potential” to reduce testing via

code analysis


2009-4-VH


• More information? Just Email:

[email protected]: ask for:

- “Military Certification” Whitepaper

- “DO-178 Costs Versus Benefits” Whitepaper

• Private DO-178B Training

(over 7,500 trained by Vance;

2 or 3-day customized sessions

at your site.)

mailto:[email protected]

Documents

DO-178B to DO-178C