An Automated Approach to Object Code Verification€¦ · DO-178B. In DO-178C this objective was added (A-7 9) ... •Much more reliable than the representative code constructs approach

An Automated Approach to Object Code Verification

Shan Bhattacharya

LDRA Overview

Provider of Software Quality, Compliance Management & Testing Solutions

Established 1975

ISO 9001 certified company

Certified for use in safety related software development according to IEC 61508, EN 50128, ISO 26262, IEC 62304 & IEC 60880

Active participants in standards e.g. DO-178C, MISRA C/C++, CERT

2

Experts in Safety and Security Critical Software

Aerospace Defence Medical

Industrial& Energy

RailTransportation Automotive

3

LDRA Standards Experience & Pedigree

Professor Mike Hennell Bill St Clair Shan

BhattacharyaMember of SC-205 /

WG-71 (DO-178C) formal methods subgroup

Member of MISRA C committee and MISRA

C++ committeeMember of the working

group drafting a proposed secureC annex for the C

language definition (SC 22 / WG14)

Member of SC-205 / WG-71 (DO-178C) Object

Oriented Technology subgroup

Member of FACE Consortium Technical

Working Group Conformance Verification

Matrix Subcommittee Member of FACE

Consortium Integration Workshop

Standing Committee Cybersecurity Assurance

Testing Task Force4

LDRA Standards Experience & Pedigree

Dr Clive Pygott Liz Whiting Chris Tapp

Member of ISO software vulnerabilities working group (SC 22 / WG 23)

Member of MISRA C++ committee

Member of the working group drafting a proposed secureC annex for the C

language definition (SC 22 / WG14)

Member of MISRA C committee language

definition

Chairman of MISRA C++ committee

Member of MISRA C committee language

definition

5

DO-178C Challenges

6

Tool Qualification

Objectives and Projects

Documents

Verification Evidence and

Audit Trail

Integration with Targets

Structural Coverage

Data Flow & Control Flow

Meeting Coding

Standards

Requirements Traceability

Through Code Testing

DO-178C Challenges

Certificate Of QualitySoftware has been

tested and conforms to DO-178C

Reduce Cost of Compliance

ManageDistributed Team

Reduce Time To Compliance And Market

6

Standards Objectives

Select and configure pre-defined standards objectives Identify compliance gaps prior to program

reviews Objectives compliance reports link to other

reports

7




reports

7




reports

7

DO-178C requires bi-directional traceability from each level of requirements to it’s upstream and downstream requirements Low-level requirements must be traced to source code. Requirements must be traced bi-directionally to tests that verify them

Traceability Across Requirements, Code, and Tests in DO-178C

Security Safety

8

A spec tree (short for specification tree) shows the relationships between different groups of requirements and tests that may exist in different repositories or documents

Requirements Specification Tree

In this example the system-level and high-level requirements are both in IBM Rational DOORS.

The low-level requirements were written in Word

Functional and low-level tests cases are written in Excel

The source code should be managed in a configuration management tool

Traceability must be reconciled across repository boundaries9

Pre-empt security and reliability issues early in the lifecycle Reduce defects, verification cost Consistent coding style and form across teams Portability and reusability across environments

Security vs Reliability MISRA AC AGC, MISRA C++:2008, MISRA

C:2012 CERT, CWE focus on security MISRA C:2012 AMD1/ADD2 safety and security Tailorable for new development, legacy code, and

runtime error checking DO-178C Objective “Source code conforms to

standards” A-5 4

Coding Standards Adherence

Hand Code

Auto-generated

Code

Legacy Code

10

Automated Testing and Code Coverage

Functional and unit level tests should be automated Run with or without instrumentation Code coverage aggregation Artefacts must show compliance to Table A-7 Verification of

Verification Process Results and its objectives11

Automated Testing and Code Coverage

Functional and unit level tests should be automated Run with or without instrumentation Code coverage aggregation Artefacts must show compliance to Table A-7 Verification of

Verification Process Results and its objectives11

Requirements traceability to code ensures that the requirement is implemented and that the code serves a valid purpose Requirements traceability to test ensures that the requirement is

verified and that the test is useful These techniques can be combined with structural coverage data

to show that the code implemented for a requirement is exercised by tests for that requirement

The Role of Requirements, Code and Test in Verification

12

Automated Low-level Testing

Achieve structural coverage objectives, verify low level requirements and greatly reduce verification costs

13

Object Code Verification

Source code and assembly control flow is rarely the same Optimisation for speed and size dramatically effect generated object code Compilers aren’t certified to the standard and are not deterministic 14

In 6.4.4.2, b, there is a Level A Objective that was “hidden” objective in DO-178B. In DO-178C this objective was added (A-7 9)

Object Code Verification Introduction

15

In 6.4.4.2, b, there is a Level A Objective that was “hidden” objective in DO-178B. In DO-178C this objective was added (A-7 9)

Object Code Verification Introduction

For level A, it is necessary to show 100% coverage of not just the high-level language, but also verify any additional, non-traceable Executable Object Code (EOC)

15

In DO-178C, Executable Object Code verification is now added to Table A-7, the Objectives table for testing “Verification of additional code, that cannot be traced to Source

Code, is achieved.” The Activity associated with this new Objective is in the same

paragraph, 6.4.4.2 b, as in DO-178B

In DO-178C EOC Verification Is Official!

16

Ensuring additional code added/modified by the compiler is verified Compiler, silicon specific technology Automation and qualification reduces schedule and risk uncertainty

Object Code Verification (A7-9)

17

Consider the following simple C code:


18

In order to get 100% Statement and 100% Branch/Decision coverage of the “C” code, we need to create four test cases

Test Cases Required for 100% Coverage

19

C Code Coverage

20100% Statement and Branch/Decision Coverage Achieved

C Code Coverage by Test Case

Visualising structural coverage accrued per test case Requirements driven test cases accruing incremental code coverage Aids in structural coverage analysis and associated review activities

21

The object code generated by a compiler will depend on the optimisation setting, the compiler vendor, the target and a host of other issues

The following is an example of the resulting assembler code generated by a widely used commercially available compiler with optimisationdisabled

Corresponding Assembler Code

22

As we can see, the structure of the generated assembler code is quite different to that of the C code

Assembly Code Structure

23

At the same time, when we inspect the code coverage on the assembler code, we observe that we have not achieved 100% coverage


C_CashRegister_asmwrkfls\specialoffer.s

24

Assembly Code Coverage

• There is one decision that is not covered

25

With the addition of one extra test, we can get 100% coverage of the assembler code:

100% Assembly Level Coverage

26

Tool Qualification Support Packs

Programming Rules Checking

Structural Coverage Analysis

Dynamic Data Flow Coverage

Assembler Coverage Analysis

250+ Certifications For 80+ Companies With 75 Level A Certifications

Unit Test / Low Level Test

RTCA/DO-178B/C

Structural CoverageAnalysis (SCA)

Tool QualificationSupport Pack

Overview

27

Object Code Verification Considerations

Low incremental cost of identifying executed object code

• Flexibility to change compiler settings for performance and size as gaps in coverage can be quickly assessed

• Straight forward regression for black box (Behavior), white box (C coverage), and red box (assembly coverage) modes

• Supported across a wide range of architectures and compilers primarily due to the flexibility in the parsing technology

Widely accepted by DERs and other certification authorities

• Qualification packages for high order language and assembly code• Much more reliable than the representative code constructs approach • Use of automated unit test tools is clearly preferable over the alternative of

manual inspection and extensive tool chain documentation

With DO-178C, Object Code verification is now mandatory for level A without exception

28

Q A&

Any Questions

29

.com

Need more information?

[email protected]

Contact Us

30