Roderick Chapman and Peter Amey Praxis Critical Systems

Copyright © 2001 Praxis Critical Systems Limited

Roderick Chapman and Peter Amey

Praxis Critical Systems

Practical Experiences of Safety-Critical Ada Technologies


Programme

• Introduction• What is High Integrity Software?• Reliable Programming in Standard Languages

– Coffee

• Standards Overview• DO178B and the Lockheed C130J

– Lunch

• Def Stan 00-55 and SHOLIS• ITSEC, Common Criteria and Mondex

– Tea

• Compiler and Run-time Issues• Conclusions


Programme


– Coffee


– Lunch


– Tea



Safety-critical Software

• Safety is a system property– A system is safe if it will:

• not endanger life

• not cause a major environmental problem

• etc.

• Where the system relies on the operation of software to achieve safe behaviour the software is safety-critical

• Not all software in safety-related systems is safety-critical


Why Have Safety-critical Software?

• Advantages of software– Complexity

• Disadvantages of software– Complexity!


Programme


– Coffee


– Lunch


– Tea



High-integrity Software

• Code where reliability is more important than– Efficiency

– Cost

– Time to market

– Functionality

• Clearly safety-critical software should be high-integrity (but reverse is not always true)


Examples of High-integrity (but not S-C)

• Security systems• Systems where direct financial loss could be

large• Avoidance of “down-time”• Avoiding product recall or loss of customer

confidence• Loss of costly one-off mission capability (e.g.

Mars lander)



Unique Characteristic of High-integrity Software

• The need to be able to show, before there is any service experience, that a system will meet requirements– Qualitatively different problem from “normal” software

– Standards required may be very high

• 109 hours > 114,000 years

• Meeting this challenge is not the same as certification

• Not enough just to be “more careful”



Some Definitions

• Verification is the process of determining that a system (or component) meets its specification

• Validation is the process of determining that a system is appropriate for its purpose

• Certification is persuading an external regulatory body that a set of specific requirements have been met or processes followed


A Single-sentence Standard

The fitness for purpose of a software program shall be established by logical reasoning


V&V Techniques

• Inspections and reviews– e.g.

• Requirements reviews

• Code inspections

• Test– e.g.

• Requirements-based

• White box

• Analysis– e.g.

• Flow analysis

• Model checking

• Proof


Inspection and Reviews

• Advantages– Very flexible, we can review an entity for internal consistency,

compare different artefacts for equivalence etc.

– No special tools needed

– Can be done early

• Disadvantages– Informal process; may spot errors but cannot guarantee absence

– What is feasible limited by:

• Semantic precision of artefact being inspected

• Semantic gap between artefacts being compared


Advantages of Dynamic Testing

• Spans the entire development process– Potentially able to identify:

• Requirements errors

• Specification errors

• Coding errors

• Compiler bugs

• Hardware problems


Disadvantages of Dynamic Testing

• Theoretical limitations– Exhaustive testing almost invariably impossible

– High levels of confidence require infeasible amounts of testing

• Practical disadvantages– Can only take place towards the end of development

– Can be hard to diagnose unexpected behaviour

– Frequently a bottleneck (e.g. Shared use of test rig)

– Very expensive

– Significant source of project risk


A Note on Theoretical Limitations of Testing

• Ultra-high reliability region (<10-7 failures per hour)• Bayesian mathematics clearly limits what we can

claim from testing alone• Reliability growth models cannot provide necessary

assurance• See:

– The infeasibility of quantifying the reliability of life-critical real-time software. Butler & Finelli. NASA Langley Research Center

– Validation of Ultrahigh Dependability for Software-based Systems. Littlewood & Strigini. CACM Nov 1993


Advantages and Disadvantages of Analysis

• Advantages– Can be used early in the development process

– Can establish properties that cannot be demonstrated in any other way. e.g.

• Proof of absence of run-time errors

• Freedom from timing deadlocks

• Disadvantages– Can only compare artefacts (e.g. Code against specification)

– What can be achieved is limited by precision of descriptions and notations used


A Balanced Approach

• Showing “correctness” is harder than building correct systems

• Use a variety of complementary verification techniques

• Bug detection and correction is expensive so:– Focus on bug prevention

– Use techniques that find bugs early

– Regard final testing as demonstration of correct behaviour rather than method of finding bugs


Resources

• Littlewood, Bev; and Strigini, Lorenzo: Validation of Ultrahigh Dependability for Software-based Systems. CACM 36(11): 69-80 (1993)

• Butler, Ricky W.; And Finelli, George B.: The Infeasibility of Quantifying the Reliability of Life-critical Real-time Software. IEEE Transactions on Software Engineering, vol. 19, no. 1, Jan. 1993, pp 3-12

• Littlewood, B: Limits to Evaluation of Software Dependability. In Software Reliability and Metrics (Procedings of Seventh Annual CSR Conference, Garmisch-Partenkirchen). N. Fenton and B. Littlewood. Eds. Elsevier, London, pp. 81-110


Programme


– Coffee


– Lunch


– Tea



Outline

• Language issues• Special purpose languages• Dialects• Subsets

– MISRA-C

– Ada HRG report

– SPARK

– Other Ada subsets

• Conclusions


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


Languages Affect the Way We Think

• Less likely to think of abstractions in machine code or assembler

• Think more in machine terms in C• Unlikely to think in object-terms in FORTRAN or

in functional terms in Smalltalk

• For larger systems, language support for abstraction and encapsulation is vital


When to Get Formal?

• Object (machine) code is formal– The execution machine provides operational semantics for it

• Typically we cannot reason about machine code we can only observe its behaviour

• Reliance on dynamic testing loads the lifecycle towards the back (expensive) end

• Early reasoning saves money and reduces risk but is hampered by lack of formality of programming languages


Causes of Uncertainty

• Deficiencies in language definitions– Semantics of C integer division

– Pascal named vs. Structural type equivalence

• Implementation freedoms– Execution order

– Parameter passing mechanisms


Consequences of Uncertainty

• Leads to either:

– Ambiguity; programs whose behaviour cannot be predicted from the source code; or

– Insecurity; violations of language rules that cannot be detected


Example of an Ambiguity

y = f(x) + g(x);

Suppose function f modifies x as a side-effect of its operation. In this case the meaning of the expression depends on whether f or g is evaluated first.

A rule stating “functions are not permitted to have side effects” simply turns the ambiguity into an insecurity


A Simple C Ambiguity

i = v[i++];

Page 46 of the C++ Annotated Reference Manual states that this leads to the value of i being undefined.

Incidentally, this is a violation of MISRA-C Rule 46


A Tiny Ada Ambiguity

procedure Init2(X, Y : out integer)

is

begin

X := 1;

Y := 2;

end Init2;

What is the meaning of:

Init2(A, A); Incidentally, this call is not legal SPARK


Resolution

• Ambiguities are resolved .... by compiler authors

• Insecurities are left for the user to discover

• Possible solutions– Invent new languages without these problems

– Work with dialects associated with compilers

– Use logically coherent language subsets to overcome ambiguities and insecurities


Outline


– MISRA-c

– Ada HRG report

– SPARK


• Conclusions


Special-purpose Languages

• Inventing new languages is fun! e.g.

– Gypsy,

– Newspeak

• But very small user base leads to:

– Poor or non-existent tools

– Staff shortages

– Lack of training support


The VIPER Analogy

• VIPER was a “formally verified” microprocessor

– Commercial failure

– Users preferred “defective” processors with wide support

• Abandoning the wider user-base is a high-risk

strategy


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


Using Language Dialects

• Find out what your compiler does

• Document it

• Include it in:

– Coding standards

– Review checklists


Problems With Dialects

• What is the compiler’s behaviour?– Is it consistent?

– Does it change with new releases?

– Is it the same on host and target?

• Knowing when it matters—to know the dialect meaning of Init(A, A); we must both:– recognise that implementation dependent behaviour is

present; and

– know what the compiler’s behaviour is in this case


Dialect Observations

• Having a working knowledge of the compiler is always useful, but

• You cannot expect to avoid all anomalous behaviour by this method

• The approach may be valuable in special cases. e.g. small, specialized processors where one company provides processor, compiler and analysis tools


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


Language Properties Required

• Simple

• Application oriented

• Predictable

• Formally verifiable

• Sound supporting

tools

Produced by independent specialists on behalf of UK MoD.


• No currently standardised language could be recommended without reservation for the most critical applications without subsetting”

Requirements for Programming Languages, Computer Standards and Interfaces 1992 BAWichmann, National Physical Laboratory


Safe Subsets

• Potentially give us the best of both worlds:– Logical soundness and predictability

– Access to standard compilers, tools, training, staff

But• Level of integrity achievable depends on

foundation language chosen• Subsetting alone may not be enough for the

highest integrity levels


Constructing a Safe Subset

• Selection of base language

• Removal of the most troublesome language features

• Limitations on the way remaining features may be used

• Introduction of annotations to provide extra information


The Subset Spectrum

• We can construct subsets that vary on 4 axes:– Precision (security and lack of ambiguity)

– Expressive power

– Depth of analysis possible

– Efficiency of analysis process

• Trade-offs quite complex; we are trying to avoid surprise: unexpected behaviour which we don’t find until test– Removing problematic features may reduce this risk

– Increased precision may require reduction in expressive power but improves depth of analysis

– We may be able to combine expressiveness with depth of analysis but at the cost of efficiency of analysis


The Subset Spectrum (Contd.)

• Fundamental trade-off is between discipline we accept to reduce bug insertion and the effort we are prepared to make in bug detection

• For example:– Unrestricted C provides little or protection from bug

insertion

– Ada requires extra discipline (e.g. strong typing) which reduces bug insertion rate

• A qualitative shift in what is possible only occurs when precision becomes exact


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


MISRA-c

• C subset defined by UK motor industry research association (MIRA) and associated software reliability association (MISRA) (www.misra.org.uk)

• Comprises 127 “rules” presented in the form of a coding standard

• The language designers regard it as suitable for “SIL 3” systems; they recommend Ada or Modula-2 if available


Constructing a Safe Subset - MISRA-C

• Selection of base language–


–


–


–


Constructing a Safe Subset - MISRA-C

• Selection of base language– ISO/IEC 9899:1990 + technical corrigendum 1995


– e.g. pointer arithmetic


– e.g all switch statements should have final default


– not provided


Language Issues Revisited

• ...• Possible solutions

– Invent new languages without these problems

– Work with dialects associated with compilers

– Use logically coherent language subsets to overcome ambiguities and insecurities


Language Issues Revisited

• ...• Possible solutions

– Invent new languages without these problems

– Work with dialects associated with compilers– Use logically coherent languages subsets to overcome ambiguities

and insecurities.

Rule 41: The implementation of integer division in the chosen compiler should be determined, documented and taken into account.


MISRA-C Overview

• MISRA-C concentrates on the “remove most troublesome language features” approach

• There is no attempt to make the subset logically coherent and free from ambiguity and insecurity

• Machine checkability well short of 100%– Some rules cannot be machine-checked at all

• Rule 4. Provision should be made for appropriate run-time checking

– Some rules are unclear

• Nevertheless, the rules should prevent many common C errors


Some Examples

• Taken from “A Software Fault Prevention Approach in Coding and Root Cause Analysis” by Weider D. Yu

• Obtained from analysis of several million lines of code in the Lucent 5ESS switching system

• Nearly 50% of the errors found were this kind of low-level coding error


This is a violation of MISRA-C Rule 47

“No dependence should be placed on C’s operator precedence rules in expressions”

Operator Precedence

if (blkptr->rpthead.fltdesc &

HMFLTCLAS == HWMATEFLT)

Should be corrected to:

if ((blkptr->rpthead.fltdesc &

HMFLTCLAS) == HWMATEFLT)



“Only expressions concerned with loop control should appear within a for statement”


For Loop Control

for (idx = 0; idx < 40;

dispstring[idx] = COTsuccess[idx++]);

for (idx = 0; idx < 40; idx++) {

dispstring[idx] = COTsuccess[idx]

};



“The statements forming the body of an if, else if, else, while, do…while or for statement shall always be enclosed in braces”


if (condition == TRUE) {

flag = DBYES;

};

Flow Control Statements

if (condition == TRUE)

flag = DBYES;


if (condition == TRUE) {

flag = DBYES;

ASfunction();

};

Flow Control Statements

if (condition == TRUE)

flag = DBYES;

ASfunction();


Logic Faults on Lucent 5ESS Project

• Use of uninitialized variables • Misuse of break and continue

statements• Operator precedence• Loop boundaries• Indexing outside arrays• Truncation of values• Misuse of pointers• Incorrect AND and OR tests• Assignment/equality

• Bit fields not unsigned or enum• Incorrect logical AND and mask

operators• Preprocessor conditional errors• Comment delimiters• Unsigned variables and

comparisons• Misuse of type casting


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


The Ada HRG

• ISO committee under WG9 to investigate high-integrity Ada issues:– Interpretation and development of Annex H of the LRM

– Focus for developing Ada in the high-integrity field

• Members from:– Tool vendors

– Users

– Government (e.g. UK MoD)

– Academia


HRG Report

ISO/IEC JTC1/SC22/WG9

Programming Languages -

Guide for the Use of the Ada

Programming Language in High

Integrity Systems

• Does not define a subset but identifies the basis on which a subset might be selected


General Approach

• Identifies verification techniques• Compares each technique with Ada language

features– Included - the feature and technique are fully compatible

– Excluded - use of the feature prevents cost-effective use of the technique

– Allowed

• Use of the feature makes the technique difficult

• The difficulty can be circumvented

• Tables have explanatory notes for excluded and allowed items


Techniques (Methods) and Groups

Type of Method Group Name Method NameControl Flow

Flow Analysis (FA) Data FlowInformation Flow

Symbolic Analysis (SA) Symbolic ExecutionAnalysis Formal Code Verification

Range Checking (RC) Range CheckingStack Usage (SU) Stack UsageTiming Analysis (TA) TimingOther Memory Usage (OMU) Other Memory UsageObject Code Analysis (OCA) Object Code AnalysisRequirements-based Testing (RT) Equivalence Class

Boundary ValueTesting Modified Condition/Decision Coverage

Structure-based Testing (ST) Branch CoverageStatement Coverage


Example Table: Subprograms

Feature FA SA RC SU TA OMU OCA RT STProcedures10 Inc Inc Inc Inc Inc Inc Inc Inc IncFunctions10 Inc Inc1 Inc Inc Inc Inc Inc Inc IncDefaultExpression

Inc Alld2 Inc Inc Inc Inc Alld2 Inc Inc

IndefiniteFormalParameters

Inc Inc Inc Alld3 Alld3 Alld3 Inc Inc Inc

IndefiniteReturn types

Inc Inc Inc Exc4 Exc4 Exc4 Inc Inc Inc

InlineExpansion

Inc Inc Inc Inc Inc Inc Alld5 Inc Alld5

Return inProcedures

Alld6 Alld6 Inc Inc Inc Inc Alld6 Inc Inc

Aliasing Exc7 Exc7 Inc Inc Inc Inc Inc Inc Alld7

AccessParameters

Exc8 Exc8 Inc Exc9 Inc Exc9 Inc Inc Inc


Points of Interest

• Exceptions– HRG clearly had to reconcile two views of exceptions:

• They are an essential aspect of the language for high-integrity applications

• They must be avoided because of the difficulties they cause for flow and symbolic analysis

– Propagation is clearly a key issue

• Tasking– HRG report defines the “Ravenscar Profile”

– (More on this later)


Exclusions Summary

Feature FA SA RC SU TA OMU OCA RT STDiscriminated records Exc Exc Alld ExcClass-wide operations Exc Exc Alld Alld ExcAliased objects : complex Exc Exc Exc ExcRenaming: complex/dynamic Exc Exc Exc ExcGoto Exc Exc Exc ExcComplex return types Exc Exc ExcAccess parameters Exc Exc Exc ExcUnchecked access Exc Exc Exc Exc Alld Exc Alld Exc ExcStreams Exc Exc Exc Exc Exc Exc Exc Exc ExcFull access types Exc Exc Exc Exc ExcControlled types Exc Exc Alld ExcIndefinite objects Alld Alld Alld Exc Exc Exc Exc ExcNon-Ravenscar tasking Exc Exc Exc Exc Exc Exc ExcRemote-call interface Exc Exc Exc Exc Exc Exc Exc Exc ExcPartition communications Exc Exc Exc Exc Exc Exc Exc Exc Exc


General Language Issues

• There are four justifiable reasons for restricting the use of language features:– To avoid features that are unpredictable

– To avoid features that cannot be modelled

– To avoid features that cannot be tested

– To avoid features that make verification ineffective

• Any set of restrictions on Ada creates another “Ada-like” language - that language should be coherent and consistent


Using the Guide

• “Determine the verification techniques required from ... standards or guidelines”

• “Identify and understand the objectives to be satisfied by each of the ... techniques”

• “Using the tables in section 5, determine the actual rating of the language features”

• “Confirm that the choice of subset ... can satisfy the requirements. This step should take into account available tools”


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


What Is SPARK?

• A sub-language of Ada with particular properties that make it ideally suited to the most critical of applications:– Completely unambiguous

– Free from implementation dependencies

– All rule violations are detectable

– Formally defined

– Tool supported

• SPARK subsets of both Ada 83 and Ada 95 are defined

• Language definition is open and widely available


SPARK Design Considerations

• Logical soundness• Simplicity of formal language definition• Expressive power• Security• Verifiability• Bounded space and time requirements• Correspondence with ... Ada?• Verifiability of compiled code• Minimal run-time system requirements


Constructing a Safe Subset - SPARK

• Selection of base language–


–


–


–


Constructing a Safe Subset - SPARK

• Selection of base language– ANSI/MIL-STD-1815A-1983 & ISO-8652:1995


– e.g. tasking


– e.g. limitations on placement of exit and return


– core (e.g. Global) and proof (e.g. Post) annotations


Core Annotations

• Global definitions declaring use of global variables by subprograms

• Dependency relations of procedures, specifying information flow between their imports and exports

• Inherit clauses to restrict penetration of packages• Own variable clauses to control access to package

variables, and to define refinement• Initialisation specifications to indicate initialisations by

packages of their “own” variables

These annotations are related to executable code by static-semantic rules, which are checked mechanically


Static Analysis - the SPARK Examiner

• Language subset compliance

• System-wide data flow analysis

• Information flow analysis

• Demonstration, prior to execution, that a program is

“exception free”

• Formal verification including proof of safety

properties

} “free”


SPARK

• Amplifies the strengths of Ada

• Gives program source text a precise meaning

• Guarantees freedom from certain classes of error

• Simplifies early detection of other errors

• Captures important design information in the code

• Is compatible with HRG guidance and compiler-

defined high-integrity subsets


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


Other Ada Subsets

• Tend to fall into two groups:– Informal coding standards involving fairly arbitrary

exclusion language features

• Systeam “Safe Ada”

– Subsets generated by compiler back-end and run-time library considerations

• GNORT

• C-Smart

• Raven

• Can be evaluated against HRG criteria• Tool support for first group is important issue

more on these later


Outline


– MISRA-C

– Ada HRG report

– SPARK


• Conclusions


Conclusions

• Language subsets are useful ways of helping reduce risk and cost

• Ada subsets are the best defined and best supported

• There is a variety of subsets each with a distinct mix of: precision, expressive power, depth of analysis possible, efficiency of analysis

• A qualitative shift in what is possible requires a subset 100% precise semantics

• Tool support is vital


“… one could communicate with these machines in any language provided it was an exact language …”

“… the system should resemble normal mathematical procedure closely, but at the same time should be as unambiguous as possible.”

Alan Turing, 1948


Tool Issues

• Without tools all rules are insecurities• Tool capability depends on the precision of the

language chosen– Simple lexical and syntactic checking

– Heuristic approaches

– Precision (only possible with “exact” subsets)

• Inability to check rules leads to insecurity; but, excessive false alarm rates can overwhelm with detail


Resources

• Cook, David. Evolution of Programming Languages and Why a Language Is Not Enough to Solve Our Problems. Crosstalk Dec 99. pp 7-12

• Carré, Bernard: Reliable Programming in Standard Languages. In High-integrity Software. RSRE Malvern, Chris Sennett (ed). ISBN 0-273-03158-9, 1989

• Finnie, Gavin et al: SPARK - The SPADE Ada Kernel. Edition 3.3, 1997, Praxis Critical Systems

• Finnie, Gavin et al: SPARK 95 - The SPADE Ada 95 kernel. 1999, Praxis Critical Systems• Barnes, John: High Integrity Ada - the SPARK Approach. Addison Wesley Longman, ISBN

0-201-17517-7• I.F. Currie: Newspeak - a Reliable Programming Language. In high-Integrity Software.

RSRE Malvern, Chris Sennett (ed). ISBN 0-273-03158-9, 1989• Motor Industry Research Association: Guidance for the Use of the C Language in Vehicle

Based Software. April 1998. www.misra.org.uk• Yu, Weider D: A Software Fault Prevention Approach in Coding and Root Cause Analysis.

Bell Labs Technical Journal April-Jun 1998• ISO/IEC JTC1/SC22/WG9. Programming Languages - Guide for the Use of the Ada

Programming Language in High Integrity Systems. www.dkuug.dk/jtc1/sc22/wg9/n359.pdf


Programme


– Coffee


– Lunch


– Tea


Documents

Roderick Chapman and Peter Amey Praxis Critical Systems