Software Safety Case Management - OMG … · Software Safety Case Management ... Goal Structure Build-Up 2Goal Structure Build-Up 6Goal Structure Build-Up 1Group ... zNoun-Phrase

Page 1

© Copyright Tim Kelly, 2007 Not to be reproduced without permission of author

Software Safety CaseManagement

Dr Tim KellyDepartment of Computer Science

University of York

E-mail: [email protected]

2Software Safety Case Management Tutorial – Tim Kelly© Copyright Tim Kelly, 2007 Not to be reproduced without permission of author

Tutorial Overview

Part One: 1300-1515Introduction to Safety CasesThe Importance of Safety ArgumentsThe Goal Structuring Notation (GSN)

Part Two: 1530-1745Software Safety CasesProblems with Current ApproachesThe Structure of a Typical Software Safety ArgumentEstablishing Hazard-Directed Software Safety ArgumentsLinking Process and Product Arguments

Page 2


Part 1: Overview

Safety Case concept and purposeRequirements from standardsSafety Case contentsSafety arguments

presenting clear argumentsGoal Structuring Notation (GSN)

Creating Arguments in GSNWhere, When and How to Create Assurance Arguments


MotivationMany (UK) standards establish the need for production of a safety case, e.g.

“Safety Cases are required for all new ships and equipment as ameans of formally documenting the adequate control of Risk and

demonstrating that levels of risk achieved are As Low AsReasonably Practicable (ALARP).” (JSP430)

A person in control of any railway infrastructure shall not use or permit it to be used for the operation of trains unless

(a) he has prepared a safety case …(b) the Executive has accepted that safety case …”

(HSE Railway Safety Case Regulations)

“The Software Design Authority shall provide a Software Safety Case …”(U.K. Defence Standard 00-55)

Page 3


The Purpose of a Safety Case

Principal Objective:Safety case presents the argument that a system will be acceptably safe in a given context

‘System’ could be ...physical (e.g. aero-engines, reactor protection systems)procedural (e.g. railway operations, off-shore)

Safety Cases can be prepared for ..commissioningmaintenancedecommissioning ...


Some Definitions

"A safety case is a comprehensive and structured set of safety documentation which is aimed to ensure that the safety of a specific vessel or equipment can be demonstrated by reference to:

safety arrangements and organisationsafety analysescompliance with the standards and best practiceacceptance testsauditsinspectionsfeedbackprovision made for safe use including emergency arrangements”

"The software safety case shall present a well-organised and reasoned justification based on objective evidence, that the software does or will satisfy the safety aspects of the Statement of Technical Requirements and the Software Requirements specification."

(JSP 430)

(DS 00-55)

Page 4


Argument & Evidence

A safety case requires two elements:Supporting EvidenceResults of observing, analysing, testing, simulating and

estimating the properties of a system that provide the fundamental information from which safety can be inferred

High Level ArgumentExplanation of how the available evidence can be

reasonably interpreted as indicating acceptable safety – usually by demonstrating compliance with requirements, sufficient mitigation / avoidance of hazards etc

Argument without Evidence is unfoundedEvidence without Argument is unexplained


Argument & Evidence 2

Safety Requirements & Objectives

Safety Evidence

Safety Argument

Page 5


Safety Cases vs. Safety Case Reports

Safety Case is the totality ofthe safety justification + all thesupporting material: testingreports, validation reports,relevant design information etc

Safety Case Report is the document that summarises all the key components of the Safety Case and referencesall supporting documentation in a clear and concise format


Safety Case Reports

Exact contents depends on regulatory environmentThe following are key elements of most standards:

ScopeSystem DescriptionSystem HazardsSafety RequirementsRisk AssessmentHazard Control / Risk Reduction MeasuresSafety Analysis / TestSafety Management SystemDevelopment Process JustificationConclusions

Page 6


The Safety Case is not just a collection of disparate pieces of informationThe Safety Argument should form the ‘spine’ of the Safety Case showing how these elements are related and combined to provide assurance of safety

within the limits defined [Scope], the system [System Description] is SAFE because all identified hazards [System Hazards] and requirements [Safety Requirements] have been addressed. Hazards have been sufficiently controlled and mitigated [Hazard Control / Risk Reduction Measures] according to the safety risk posed [Risk Assessment]. Evidence [Safety Analysis / Test] is provided that demonstrates the effectiveness and sufficiency of these measures. Appropriate roles, responsibilities and methods were defined throughout the development of this system [Development Process Justification] [Safety Management System]and defined future operation

Safety Arguments


Safety Arguments – Text Example

The Defence in Depth principle (P65) has been addressed in this system through the provision of the following:• Multiple physical barriers between hazard sourceand the environment (see Section X)

• A protection system to prevent breach of thesebarriers and to mitigate the effects of a barrierbeing breached (see Section Y)

...

Safety Arguments should clearly describe how a safety objective / requirement / claim has been achieved in the system as proposed

how it has been interpretedultimately, what evidence supports the requirements

Page 7


Safety Arguments – Text Problems

For hazards associated with warnings,the assumptions of [7] Section 3.4 associated with the requirement to present a warning when no equipment failure has occurred are carried forward. In particular, with respect to hazard 17 in section 5.7 [4] that for test operation, operating limits will need to be introduced to protect against the hazard, whilst further data is gathered to determine the extent of the problem.

not everyone can write clear Englishcan take many readings to decipher meaningmultiple cross-references in text can be awkwardis there a clear shared understanding of the argument?


Presenting Clear Arguments

It is possible in text – at least sometimesuse simple language and short sentencesuse bullet points for key statementsbreak down the argument one step at a time

and refer to following sub-sectionsstructure document sub-sections around separate concepts

e.g. Section 6.2 – Control of Hazard – ‘Inadvertent Chaff Release’

But it is easier with pictures!use a graphical notation to summarise argument

Goal Structuring Notation (GSN)Claims Argument Evidence (CAE)

Page 8


The Goal Structuring Notation

Purpose of a Goal Structure

To show how goals are broken down into sub-goals,

and eventually supported by evidence (solutions)

whilst making clear the strategies adopted,

the rationale for the approach (assumptions, justifications)

and the context in which goals are stated

A/J


Control Systemis Safe

All identifiedhazards eliminated

/ sufficientlymitigated

Softwaredeveloped to I.L.

appropriate tohazards involved

I.L. Process Guidelinesdefined by Ref X.

Hazards Identifiedfrom FHA (Ref Y)

Tolerability targets(Ref Z)

Fault TreeAnalysis

FormalVerification

ProcessEvidenceof I.L. 4

Probability of H2occurring

< 1 x 10-6 perannum

H1 has beeneliminated


< 1 x 10-3 perannum

Primary ProtectionSystem developed

to I.L. 4

SecondaryProtection Systemdeveloped to I.L. 2

ProcessEvidence of

I.L. 2

J

1x10-6 p.a.limit for

CatastrophicHazards

A Simple Goal Structure

Page 9


Control Systemis Safe

All identifiedhazards eliminated

/ sufficientlymitigated

Softwaredeveloped to I.L.

appropriate tohazards involved

I.L. Process Guidelinesdefined by Ref X.

Hazards Identifiedfrom FHA (Ref Y)

Tolerability targets(Ref Z)

Fault TreeAnalysis

FormalVerification

ProcessEvidenceof I.L. 4


< 1 x 10-6 perannum

H1 has beeneliminated


< 1 x 10-3 perannum

Primary ProtectionSystem developed

to I.L. 4

SecondaryProtection Systemdeveloped to I.L. 2

ProcessEvidence of

I.L. 2

J

1x10-6 p.a.limit for

CatastrophicHazards

A Simple Goal Structure

Safety Requirements & Objectives

Safety Evidence

Safety Argument

Group Exercise

Create a goal structure to support the claim:

TopThis is a good pint ofbeer

Goal Structure Build-Up 6Goal Structure Build-Up 2Goal Structure Build-Up 1Group Exercise


Page 10

Identifygoals to besupported

Definebasis on whichgoals stated

Identifystrategy to

supportgoals

Define basison whichstrategystated

IdentifyBasic

SolutionSTOP

Step 1

Step 2

Step 3

Step 4

Step 6

Step 5 - Elaboratestrategy

Process Overview



Step 1 - Identify Goals: Phrasing

Goals should be phrased as propositionsStatements that can be said to be TRUE / FALSE (e.g. “The sky is blue” or “York is a beautiful city”)NB: not limited to statements that can be objectively provenStatement should be expressed as a single statement (1 sentence) of in the form:

<NOUN-PHRASE><VERB-PHRASE>Noun-Phrase identifies the subject of the goalVerb-Phrase defines a predicate over the subject

2 4

65

31

Page 11



The following are examples of correctly stated goals:

2 4

65

31

Subject<Noun-Phrase>

Predicate<Verb-Phrase>

Component XAll identified hazards for System YNon-destructive examination ofweld-site ZDesign A

has no ‘critical’ failure modeshave been sufficiently mitigatedhas been performed

employs triple modularredundancy


The following are examples of incorrectly stated goals:

Test: can we say goal is TRUE / FALSE?

Reason:

• “Hazard Log for System Y” NP – describes an entity -not a statement

• “Fault Tree for Hazard H1” As above• “Perform Fault Tree Analysis of

Hazard H1”VP - an action - not astatement

• “How many failure modes doescomponent X have?”

Question - not astatement


2 4

65

31

Page 12

Press is acceptably safe tooperate within CCCWhatford Plant

G1

Goal Structure Build-Up 1

As with conventional safety case report – we wish to clearly set out the objective and scope of the safety argument being presented

Step 1 – Example



System implementation (carried outby The Contractors) was done inaccordance with Safety Principles Athru' E

G2

C1

System ImplementationActivities

(Ref Y)

C2

The Contractors for thisproject are BTR

Construction

C3

Safety Principles (Ref Z)

Step 2 - Define basis for goals: ContextExamine goal statement for terms / concepts not clearly defined within scope (context inherited as with models)

It is justified to leave a term ‘free’ if it will be defined through the course of the argument 2 4

65

31

Page 13



G1

C1Press Design

C2Press Operation

C3CCC Whatford Plant

Terms – Press, Operate and CCC Whatford Plantdrawn out explicitly as contextual informationAcceptably Safe left for expansion through the supporting argument

Goal Structure Build-Up 1Step 2 – Example



Step 3 - Identify strategyNext step is to work out how to substantiate the stated goal

“What reasons are there for saying the goal is TRUE?”“What statements would convince the reader that the goal is TRUE?”

Aiming for statements that are easier to support than the larger goal

Breaking into a number of smaller goals - i.e. Divide-and-ConquerRelating goal more closely to specific application in question (e.g. for a generic requirement)

2 4

65

31

Page 14

Step 3 - Identify strategy

System is SAFE

G5

Sub-system A is SAFE

G7

Sub-system B is SAFE

G8

Sub-system C is SAFE

G9

Divide-and-Conquer

Requirement 6.3 (Defence inDepth) has been met

G10

3 independent protection systems inplace to shutdown system in case ofdetected abnormal operation

G11

Interpretation / Particularisation

Examples:

2 4

65

31



Step 3 - Identify strategy: PhrasingThe role of a strategy node is to clearly explain the relationship between a goal and a set of sub-goalsAn analogy:

Strategy statement should succinctly describe the argument approach adopted, ideally in the form:

“Argument by … <approach>”

Example statements:“Argument by appeal to test results”“Argument by consideration of historical data”“Quantitative argument using simulated run data”

3xy3 + 2x2y2 + 5xy = 17y (Divide both sides by y)3xy2 + 2x2y + 5x = 17

Strategy

2 4

65

31

Page 15

Step 3 - Identify strategy

System is SAFE

G5

Sub-system A is SAFE

G7

Sub-system B is SAFE

G8

Sub-system C is SAFE

G9

ExampleS1

Argument bybreaking-down over allidentified sub-systems

? Is it obvious?

These terms have been introduced

Any assumptions?

2 4

65

31



C2Press Operation

C1Press Design


G1

S1

Argument by addressing allidentified operatinghazards

S2

Argument of compliancewith all applicable safetystandards & regulations


Two separate strategies – for reader’s benefit

Goal Structure Build-Up 2Goal Structure Build-Up 1Goal Structure Build-Up 3


Page 16


Step 4 - Define basis for strategy

In the same way as is necessary for goals, must examine what contextual information (including models) is requiredSame process - examine strategy for terms / concepts introduced but not ‘bound’

e.g. for sub-system breakdown strategy the term ‘All Identified sub-systems” is used

Ask what information is required in order to expand / fulfill strategy outlined

2 4

65

31

S2



G1

C1Press Design

C2Press Operation


S1


C4All identified

operating hazards

C5

All applicable safety safetystandards & regulations


Needs Justification?Any Assumptions?

Goal Structure Build-Up 2Goal Structure Build-Up 1Step 4 - Example

2 4

65

31


Page 17


Step 5 - Elaborate strategy

Having identified an approach, it is necessary to lay out the goals that fulfill that approach, e.g.

e.g., for strategy ranging over all sub-systems - expand for goals over each individual sub-systeme.g. for quantitative results strategy - provide quantitative goal statements

In elaborating the strategy, again defining goals (back to Step 1)If strategy, and basis of strategy, are clear - this step can be straightforward

E.g. see next slide

2 4

65

31

Goal Structure Build-Up 5Goal Structure Build-Up 1Step 5 - Example

2 4

65

31

C5

All applicable safetysafety standards &

regulations

C4All identified

operating hazards

S1


S2


Hazard of 'Operator HandsTrapped by Press Plunger' sufficiently mitigated

G2

Hazard of 'Operator HandsCaught in Press DriveMachinery' sufficiently mitigated

G3

Hazard of 'Operator UpperBody trapped by PressPlunger' sufficiently

G4

Press compliant with U.K.HSE Provision and Use ofWork Equipment Regulations

G5

PES element of press designcompliant with IEC61508

G7

Press compliant with U.K.enactment of EU MachineryDirective

G6


Page 18


Testing of Module XYZ123showed no anomalies

G7

Sn1Software Test

Results for ModuleXYZ123

Step 6 - Identify Solutions

Eventually, faced with a goal that doesn’t need further expansion, refinement, explanation …In such cases, simply have to reference out to information that supports claim by means of solutionAs references, solutions should be NOUN-PHRASEs

2 4

65

31

Hazard of 'Operator Hands Caughtin Press Drive Machinery'sufficiently mitigated

G3

Motor / Clutch / Drive Beltssurrounded with safety cage

G8

Press operation will (safely)halt if safety cage tamperedwith

G9

Sn10Press Design(Safety Cage)

More explanation required here


Reference to source of information that would substantiate claim

Goal Structure Build-Up 2Goal Structure Build-Up 1Step 6 – Example


Page 19


How to Create Goal Structures

Two main approaches:As an individualAs a group

Where aim is to reach common understanding and agreement of structure of the safety argument

Attendance: Safety Argument Owner (Principal Stakeholder), Experts, GSN Facilitator, Secretary (Optional)Requires Key Documents to be circulated beforehand


Where to Brainstorm the Argument?

Q: Where is it worth brainstorming an argument?Answers:

Wherever there is most uncertainty about the argument (key claims, evidence)Wherever the argument is currently confused or is over-complexWherever there is disagreement about the argumentWherever the consequences of having a wrongargument are high (in terms of rework, delays etc.)

Page 20


When to Visualise the Argument?

Q: At what stage in a project is it worth visualising the argument?Answers:

Early on (high level) to get a clear picture (and gain agreement) of argument structure

Useful as a scoping exercise and effort allocationAs project is progressing, in order to monitor status towards completion of an acceptable argumentAt end of project in order to present the final argument and evidence that exists


How to Present Goal Structures

Customers are keen to see goal structures within safety documentsPossible approaches to inclusion of GSN:

In full as Appendix / Annex to documentIntegrated within body of document

Goal structure (1 level), Text, Goal structure, Text …See ‘Nuclear Trip System Safety Case Example’

As ‘Executive Summary’ at beginning of documentMaximum 2 pages of structure, 2-3 levels of decomposition

As separate, stand-alone, Index Documente.g. to explain argument distributed across many safety case documents

Page 21


Potential GSN BenefitsImproving comprehension of existing argumentsImproving discussion and reducing time-to-agreement on argument approaches being adopted(Having identified argument structure up front) focusing activities towards the specific end-objectivesRecognition and exploitation of successful (convincing) arguments becomes possibleSupports ‘light-weight’ evolution of an argumentSupports monitoring of project progress towards a successful safety case


Conclusions

Within conventional safety case reports the ‘chain of argument’ can often get lost

The argument is more important than the document!GSN has been found to be a useful basis for mapping out and evolving the structure of the Safety Arguments

Provides a Road-map for a document / set of documentsProvides a basis for discussion amongst engineers and between developers and assessorsCreating an outline arguments towards beginning of project can be seen as making progress towards a final solution

Page 22

© Copyright Tim Kelly, 2007 Not to be reproduced without permission of author

Software Safety CaseManagement – Part Two

Dr Tim KellyDepartment of Computer Science

University of York

E-mail: [email protected]

Acknowledgements: Rob Weaver, Ibrahim Habli


Part 2 Overview

Current approaches (and the problems) for Software Safety Case development An Evidence-based Framework for software safety arguments

Software Level Safety RequirementsDifferent types of EvidenceDifferent types of Hazardous Failure Modes To provide an overview of a ‘generic’ computer system safety argument structure

We will look at:Process ArgumentsProduct ArgumentsHazard-Based vs. Requirements-Based ArgumentsFunctional vs. Non-Functional Issues

Page 23


Problems with Current PracticeDiscontinuity in the safety case

System Argument has a Product FocusSoftware Argument has a Process Focus

Does Good Process = Good Product ? Questionable Assumption

Application to Legacy and Commercial Off The Shelf (COTS) softwareModern Practices – e.g. Code GeneratorsStandards prescribe Software Development Process

Discourages intelligent thought about what evidence is useful orrelevant

Responsibility for safety lies with those that set the standard rather than those that build the software


G1:[Computer system]acceptably safe tooperate in [context]

S1:Argument over

product and process

C2: Operatingcontext

C1: Computersystem definition

G3:[Computer system]does not contributeto [system] hazards

G4:[Computer system]

hardware and softwaredeveloped to [standard(s)]

S2:Argument over

functional and non-functionalproperties

G5:Functional behaviour of

[computer system] does notcontribute to [system] hazards

G6:Non-functional properties and

behaviours of [computersystem] do not contribute to

[system] hazards

C3: Applicablestandards

S3:Argument over safety

lifecycle phases / activities

G7:System

RequirementsHazard Analysis

performed

G8:Software


performed

G9:Top Level

Design HazardAnalysis

performed

G10:Detailed Design

Hazard andSafety Analysis

performed

G11:Code level

Safety Analysisperformed

G12:Hazard-directed

testingperformed

G13:Safety maintained

throughmaintenance and

change procedures

C5: Computer SystemContributions to Sys Level

Hazards


satisfies explicitsafety requirements

C4: [Computersystem] requirements

G14Potential [computer system]

software and hardwarecontributions to system

hazards have been identified

G16[Computer system] software

and hardware contributions tosystem hazards have been

eliminated or mitigated

G15[Computer system]functions related to

system hazards havebeen identified

G18[Computer system]

requirements defineds.t. contribution


S3:Argument over each

contributionSn1

System LevelHazard Analysis

Sn2Functional Failure

Analysis


architecture defined s.t.contribution eliminated

or mitigated

G20 (Analysis)[Computer system]

code implementationdefined s.t. contributioneliminated or mitigated

G21[Computer system] testing

defined such thatelimination or mitigation ofcontribution demonstrated

G22[Computer system] testingdemonstrates elimination

or mitigation ofcontribution

Sn3Requirements Level

Hazard Analysis

Sn4Design Level

Hazard Analysis

Sn5Code Level Hazard

Analysis

Sn6Test Coverage

Analysis

Sn7Test Evidence

G4a:[Computer system]

hardware and softwaredeveloped to safety

[standard(s)]

G4b:[Computer system]

hardware and softwaredeveloped in accordance

with appropriatedevelopment [standard(s)] Covers aspects such as Config. Mgt.,

Gen Quality Mgt., conformance tocoding standards, Development

Environment

Covers aspects such as Resource Usage / Starvation, Bus Clash, IllegalInstructions, Exception Handling, Control Flow Anomalies, Data Flow

Anomalies, Moding Problems, Variable Initialisation, etc.

Non-functional, functional split also possiblehere (e.g. timing requirements - leading to test /

analysis evidence could be addressed here)

Issues of Required SIL can govern requiredlevel of conformance to standards in this part of

the argument

Process Evidence required here - e.g. audits - possibly withrequirement for independent auditing and/or assessment

Overall S/W SC Structure

Page 24


Top Level ArgumentImportant of Context and Clear System DefinitionProcess and Product (Historically more Process Emphasis)Both +/ve and -/veaspects to ProductArgument

G1:[Computer system]acceptably safe tooperate in [context]

S1:Argument over

product and process

C2: Operatingcontext

C1: Computersystem definition







Software ‘Fault Free’ Pattern

As seen in U.K. DS 00-55

G2: <property x>enforced by software

G1: Software elementof system is 'fault-free'

C1: Fault = deviation from intendedbehaviour that could lead to a

system level hazard

C2: Free = Software itself does notinitiate any events that could lead

to a system level hazard

S1: Argument bysatisfaction of allsoftware safety

properties/requirements

S2: Argument byshowing software

cannot causeany of the identifiedhazardous software

conditions

C4: Identified HazardousSoftware Conditions(m = # of conditions)

C3: Identified SoftwareRequirements / Properties

(n = # of requirements / properties)

n

G3: <condition y> canonly occur by physical

component failure

m

Page 25


Conformance to Standards ArgumentSILS can govern required level of conformanceRole for both conventional dev’t and ‘safety’ standards



C3: Applicablestandards





[standard(s)]





Environment

Issues of Required SIL can govern requiredlevel of conformance to standards in this part of

the argument


SIL Tailoring of Process Arguments

SIL (Process) Justifications:Annex D - Tailoring Guide Across Differing Safety Integrity Levels

Clause S1 S2 S3 S4 Comments36 CodingProcess36.1 CodingStandards

J1 J1 M M

36.2 M M M M36.3 J2 J1 M M36.4 J2 J1 M M36.5 StaticAnalysis andFormalVerification36.5.1 J1 J2 M M… … … … … …

• M = Must be Applied• J1 = Justification of not

following clause• Inapplicability• Cost-benefit (ALARP)

• J2 = Less detailed / rigourous justification

Page 26


DO-178B Example

Table A-3 Verification of Output of Software Requirements Process

Objective Applicabilityby SW Level Output

ControlCategory by

SW LevelDescription Ref. A B C D Description Ref. A B C D

1 Software high-level requirementscomply with systemrequirements

6.3.1a Software Verification Results 11.14

2 High-level requirements areaccurate and consistent

6.3.1b Software Verification Results 11.14

3 High-level requirements arecompatible with target computer

6.3.1c Software Verification Results 11.14

4 High-level requirements areverifiable

6.3.1d Software Verification Results 11.14

5 High-level requirements conformto standards

6.3.1e Software Verification Results 11.14

6 High-level requirements aretraceable to systemrequirements

6.3.1f Software Verification Results 11.14

7 Algorithms are accurate 6.3.1g Software Verification Results 11.14

LEGEND: The objective should be satisfied with independenceThe objective should be satisfied

Blank Satisfaction of objective is at applicant’s discretionData satisfies the objectives of Control Category 2


Safety Standards Argument

Structured here by lifecycle phaseArgument that safety considered appropriately at various points in the traditional development lifecycleProcess Evidence, possibly with requirement for independence, needed here



G7:System


performed

G8:Software


performed

G9:Top Level

Design HazardAnalysis

performed

G10:Detailed Design

Hazard andSafety Analysis

performed

G11:Code level

Safety Analysisperformed

G12:Hazard-directed

testingperformed

G13:Safety maintained

throughmaintenance and

change procedures



[standard(s)]

Process Evidence required here - e.g. audits - possibly withrequirement for independent auditing and/or assessment

Page 27


‘Traditional’ Development Standards Arg.

Necessary but not sufficient elementsStill concerns general ‘integrity’ of finished product







Environment


Product Argument

Functional and Non-functional split possible both under hazard based argument and requirements conformance argumentRequirements based argument can deal with cascade from system levelCan be overlap between requirements and hazard based arguments

S1:Argument over

product and process


S2:Argument over




C4: [Computersystem] requirements

Non-functional, functional split also possiblehere (e.g. timing requirements - leading to test /

analysis evidence could be addressed here)

Page 28


Functional Behaviour ArgumentS2:

Argument overfunctional and non-functional

properties

G5:Functional behaviour of

[computer system] does notcontribute to [system] hazards

G14Potential [computer system]

software and hardwarecontributions to system

hazards have been identified




G15[Computer system]functions related to

system hazards havebeen identified

Sn1System Level

Hazard Analysis

Sn2Functional Failure

Analysis

Must demonstrate exhaustiveidentification of contributionsTypically requires bothtop-down and bottom-upapproachesUseful to pin-pointcontributionsto dev’t itemsin softwarestructure


Functional Contribution Argument 1

C5: Computer SystemContributions to Sys Level

Hazards





requirements defineds.t. contribution


S3:Argument over each

contribution


architecture defined s.t.contribution eliminated

or mitigated

G20 (Analysis)[Computer system]

code implementationdefined s.t. contributioneliminated or mitigated

G21[Computer system] testing

defined such thatelimination or mitigation ofcontribution demonstrated

G22[Computer system] testingdemonstrates elimination

or mitigation ofcontribution

Sn3Requirements Level

Hazard Analysis

Sn4Design Level

Hazard Analysis

Sn5Code Level Hazard

Analysis

Sn6Test Coverage

Analysis

Sn7Test Evidence

Page 29


Functional Contribution Argument 2

Need to demonstrate systematic consideration of each contributionIdeally, wish to see consideration through lifecycleOften best solutions to potential contribution are early lifecycleArgument can be based on both analysis (e.g. of specification) and test (of implementation)For Testing – must demonstrate coverage of contributions

May be possible to integrate with conventional acceptance testing


S2:Argument over


G6:Non-functional properties and

behaviours of [computersystem] do not contribute to

[system] hazards

Covers aspects such as Resource Usage / Starvation, Bus Clash, IllegalInstructions, Exception Handling, Control Flow Anomalies, Data Flow

Anomalies, Moding Problems, Variable Initialisation, etc.

Non-Functional Contribution ArgumentRemember, non-functional requirements dealt with elsewhereDealing here with non-intentional implementation ‘hazards’

Sometimes described at ‘Computing’ Hazards

Problem of CompletenessStandard Lists exist (e.g. 882C)

Page 30


Evidence-Based FrameworkSystem Level RequirementsRequirement for System Level Evidence

Validation, Satisfaction, TraceabilitySoftware Level Safety Requirements

Focus on Software Contributions to System Level HazardsRequirement for System level Evidence

Validation, Satisfaction, (Traceability)Classification of Hazardous Failure Modes

Omission, Commission, Early, Late, ValueGeneric ArgumentsTargeted Selection of Evidence


Example Case Study – Air to Air Missile

Seeker

InertialMeasurementUnit(IMU)

Electronics& PowerUnit(EPU)

Motor

FuzeWarhead

(or TelemetryUnit)

Umbilical Actuator

Data link

Page 31


System Level Safety Requirements

System Level Hazard AnalysisHandling HazardsHazards to Launcher

Premature Detonation (or Break-up, on telemetry rounds)Premature LaunchDisintegration near or in front of launcher due to high-g or high-roll manoeuvresHitting the launcher due to incorrect trajectoryPremature fin movement (i.e. prior to launch, perhaps damaging missile or launcher)Hang-fire (excessive delay between ignition and thrust)Hang-up (missile remains on launcher but thrusts)

Hazards to Friendly-ForcesHazards of Mission Failure


System Level Evidence Categorisation

ValidationDemonstration that the set of System Safety Requirements is complete

SatisfactionDemonstration that all System Safety Requirements have been met

TraceabilityDemonstration that all System Safety Requirements have been tracked throughout System Development and Safety Analysis

Next stage is to consider Hardware, Software and Other contributions to System-level Hazards

Page 32


Pattern – System-level SRSystemSafe

{System} is acceptably safeto operate from a hazardcontrol perspective

SysDefn

SystemDefinition

DefnAccSafe

Definition ofacceptably safe

ReqValid

System SafetyRequirements are valid

HazAccept

All identified system levelhazards occur at acceptablylow rates

SysHaz

Identified SystemLevel Hazards

Traceability

Traceability of safetyrequirements and safetyevidence has been shown

ArgSWHWOtherArgument across software,hardware and other parts of{System} that may causehazards

J

DependExplicit

System can be decomposedas all dependencies betweendifferent parts of the systemare explicit







Page 33


Pattern – Software Level SRArgSWHWOther

Argument across software,hardware and other parts of{System} that may causehazards

J

DependExplicit

System can be decomposedas all dependencies betweendifferent parts of the systemare explicit

HWContribAccept

Hardware contributions toSystem Level Hazards areacceptable

SWContribAccept

Software contributions toSystem Level Hazards areacceptable

OtherContribAccept

Other contributions toSystem Level Hazards areacceptable

SWContrib

Identified SoftwareContributions to SystemLevel Hazards = SoftwareHazardous Failure Modes

SWDefn

SoftwareDefinition

HWDefn

HardwareDefinition

HWContribIdentified HardwareContributions toSystem LevelHazards

OtherDefn

Other ComponentsDefinition

OtherContrib

Identified Contributionsof Other Components toSystem Level Hazards


Software Level Safety Requirements

Derived from System Level Safety RequirementsFrom System Level Perspective can be seen as a “Basic Event”From Software Level Perspective can be seen as the “Top Level”

Hazard BasedPotential Failures within the Software that can lead to System Level Hazards

Example – Software Safety RequirementAcceptability of Software Failure Mode - Software Fails to block premature launch

Page 34


Software Level Evidence Categorisation

ValidationDemonstration that the set of Software Safety Requirements is complete

SatisfactionDemonstration that all Software Safety Requirements have been met

TraceabilityDemonstration that all Software Safety Requirements have been tracked throughout System Development and Safety Analysis


Pattern – Software Level SRSWContribAccept

Software contributions toSystem Level Hazards areacceptable

ArgOverSWContribArgument over allidentified softwarecontributions to systemlevel hazards

SWContribIdent

All software contributions tosystem level hazards havebeen identified

SWContrib

Identified SoftwareContributions to SystemLevel Hazards = HazardousSoftware Failure Modes

SWSRTraceability

Traceability of softwaresafety requirements andsafety evidence has beenshown

HSFMAccept

All causes of HazardousSoftware Failure Mode{HSFM} are acceptable

nn = # software hazardous

failure modes from SWContrib

SWDefn

SoftwareDefinition

Page 35


Validation and Traceability

ValidationCompleteness

Functional Failure Analysis/HAZOP

TraceabilityDO-178B

Traceability between the system requirements and software requirements.Traceability between the low-level requirements and high-level requirementsTraceability between the Source Code and the low-level requirements

Traceability Matrix






Omission, Commission, Early, Late, ValueGeneric ArgumentsEvidence Characteristics

Relevance, Coverage, Independence

Page 36


Hazardous Failure Mode Classification

Omission, Commission, Early, Late, ValueService Provision, Service Timing, Value

Common arguments for different classesCertain arguments (and evidence types) can be used for different failure types

It is possible to produce generic safety case arguments that can be reused.

ExampleSoftware Failure Mode - Software Fails to block premature launch


Example - Software Architecture 1

ModeControl

IMU

clk1

Aircraft

clk2 Read AircraftMessages

Write AircraftMessages

clk3

StatusReporting

TransferAlignment& Checks

AircraftINSData

AircraftMessages

In

AircraftMessages

Out

ManageBIT

AircraftINS

StatusReports

InitialPosition

TargetPosition

FiringInterlock

ActuatorsSeparationAutopilot

InertialNavigation

MissileState

ActuatorDemands

MissileState

InterlockHandler

clk4

Mode ModeEvents

StopFiring

BodyMotion

MissileStatus

Summary

MissileIdent

StationData

Umbilical

clk2

Station

Page 37


Example - Software Architecture 2

Software Fails to block premature launch caused by:Software interlock handler fails to write to interlock pool – Omission Failure Type (Service Provision Failure Type)

ModeControl

FiringInterlock


InertialNavigation

MissileState

ActuatorDemands

InterlockHandler

clk4

ModeMode

Events

StopFiring

BodyMotion

ModeControl

FiringInterlock


InertialNavigation

MissileState

ActuatorDemands

InterlockHandler

clk4

ModeControl

FiringInterlock


InertialNavigation

MissileState

ActuatorDemands

InterlockHandler

clk4

ModeMode

Events

StopFiring

BodyMotion

ModeMode

Events

StopFiring

BodyMotion


Pattern –Failure Mode Classification

HSFMOmissionAccept

All causes of HazardousSoftware Failure Mode {HSFM}of type Omission are acceptable

HSFMEarlyAccept

All causes of HazardousSoftware Failure Mode{HSFM} of type Early areacceptable

HSFMValueAccept

All causes of HazardousSoftware Failure Mode {HSFM}of type Value are acceptable

DefOmFM

Definition ofOmission FailureMode

DefEarlyFM

Definition of EarlyFailure Mode

DefValueFMDefinition ofValue FailureMode

HSFMAccept

All causes of HazardousSoftware Failure Mode{HSFM} are acceptable

1-of-5

HSFMCommissionAccept

All causes of HazardousSoftware Failure Mode {HSFM}of type Commission areacceptable

DefComFM

Definition ofCommission FailureMode

HSFMLateAccept

All causes of HazardousSoftware Failure Mode{HSFM} of type Late areacceptable

DefLateFM

Definition of LateFailure Mode

HSFMHazardousSoftware FailureMode

SysHaz

System LevelHazard

SWDefn

SoftwareDefinition

Page 38








Arguing about Requirements

Satisfaction of a Typed Hazardous Failure Requirement

Demonstration that the requirement has been met shown through combination of

AbsenceHandling(Probability – we can’t show this for software)

ExampleShow Absence of Omission Hazardous Failure Mode - Software output module fails to provide braking value to actuator

Page 39


Pattern – Argument ApproachHSFM{type}Accept

All causes of HazardousSoftware Failure Mode {HSFM}of type {type} are acceptable

AbsHSFM{type}

Hazardous Software FailureMode {HSFM} of type {type}absent in contributary softwarefunctionality (CSF)

HandlHSFM{type}

Occurence of HazardousSoftware Failure Mode {HSFM} oftype {type} in contributarysoftware functionality (CSF)acceptably detected and handled

ArgAbsHandlArgument over {absenceand/or handling} ofHazardous Software FailureMode

n-of-2


ContribSWFunc

Identified SoftwareFunctionality whichcontributes to softwarehazardous failure mode{SHFM}

SWDefn

SoftwareDefinition


Argument for Absence Omission FailureSoftware Failure Mode (of type Omission) is absent if:

Primary ArgumentAll feasible paths through software functionality contain a unique output statement

Secondary ArgumentFailure of other software functionality which could lead to a failure of primary software functionality does not occurAll necessary resources exist to support correct operation of primary software functionality

Control ArgumentPrimary software functionality is scheduled and allowed to run (at least) once

Page 40


Software Architecture

ModeControl

FiringInterlock


InertialNavigation

MissileState

ActuatorDemands

MissileState

InterlockHandler

clk4

ModeMode

Events

StopFiring

BodyMotion

ModeControl

FiringInterlock


InertialNavigation

MissileState

ActuatorDemands

MissileState

InterlockHandler

clk4

ModeControl

FiringInterlock


InertialNavigation

MissileState

ActuatorDemands

MissileState

InterlockHandler

clk4

ModeMode

Events

StopFiring

BodyMotion

ModeMode

Events

StopFiring

BodyMotion

Primary ArgumentSecondary Argument


Product Orientated Decomposition

AbsHSFMOmission

Hazardous Software FailureMode {HSFM} of type Omissionabsent in contributary softwarefunctionality (CSF)

AbOmPrimary

All feasible control pathsthrough CSF include aunique output statement

AbOmSecondary

Failures of other componentswhich could lead to CSFOmission Hazardous FailureMode are acceptable

AbOmControl

CSF is scheduled andallowed to run once

ContribSWFunc

Identified SoftwareFunctionality whichcontributes to softwarehazardous failure mode{SHFM}

SafReqCSF

Safety Requirementsof contributorysoftware functionality

A

ContextCSF

Within the context ofcontributory softwarefunctionality (CSF)

CauseOmHaz

Known causes ofOmission HazardousFailure Mode

J

AllCauses

Identified failuremechanisms describe allknown causes of OmissionHazardous Failure Mode

DefnOPS

Definition ofoutput statement


ArgFailureMech

Argument overfailure mechanisms

Page 41


Secondary Causes Decomposition

ArgOtherSWFuncCause

Argument over other SoftwareFunctionality, Hardware Componentsor Other Components identified ascause of hazardous software failuremode {HSFM} within CSF

Ab{type}OtherFn

Failures of other functionalitywhich could lead to CSF {type}Hazardous Failure Mode areacceptable

AbResources

Necessary Resources existto support correct operationof CSF

Ab{type}Secondary

Failures of other componentswhich could lead to CSF {type}Hazardous Failure Mode areacceptable


ContribSWFunc

Identified SoftwareFunctionality whichcontributes to hazardoussoftware failure mode{HSFM}

SafReqCSF

Safety Requirementsof contributorysoftware functionality

A

ContextCSF

Within the context ofcontributory softwarefunctionality (CSF)

SWDefn

SoftwareDefinition

Resources

IdentifiedResources


Secondary Causes Decomposition 2

HSFM2Accept

All causes of HazardousSoftware Failure Mode{HSFM2} are acceptable

nHWF

Hardware Failures cannotlead to Hazardous SoftwareFailure Mode {HSFM}

OtherF

Other Failures cannot leadto Hazardous SoftwareFailure Mode {HSFM}

HWFAbsent_Hardware SafetyArgument

Hazardous Hardware Failure Modeabsent in contributory hardware

Hardware Safety Argument

HWFProb_Hardware Safety Argument

Probabililty of Hazardous HardwareFailure Mode in contributory hardwareacceptably low

Hardware Safety Argument

HWFHandlOccurence of HazardousHardware Faillure Mode{HHFM} of type {type}acceptably detected andhandled

HHFMAccept

All causes of HazardousHardware Failure Mode{HHFM} are acceptable

m

a-of-3

HOFMAccept

All causes of HazardousOther Failure Mode {HOFM}are acceptable

p

OFHandl

Occurence of Hazardous OtherFaillure Mode {HOFM} of type{type} acceptably detected andhandled

b-of-3

OFAbsent_Other Safety Argument

Hazardous Other Failure Modeabsent in other contributorycomponent

Other Safety Argument

OFProb_Other Safety Argument

Probabililty of Hazardous Other FailureMode in other contributory componentacceptably low

Other Safety Argument

HSFM2HazardousSoftware FailureMode

HHFMHazardousHardware FailureMode

HOFMHazardous OtherComponent FailureMode

HWDefn

HardwareDefinition

OtherDefn

Other ComponentsDefinition

Page 42


Targeted Selection of EvidenceIEC 61508

Selection of techniques is difficult except with respect to the Safety Case Objectives

A problem with the current standardsApproach drives the argument down to low-level objectives first


Process and Product Argument 1

Product Argument

Top Level

Process Argument

Top Level

Page 43


Process and Product Argument 2

Top

System is acceptably safe to operate

SysDefn

System Definition

OpCtxt

System operating context

HazMitMitigation argument for all identified hazards

HazLog

Hazard Log

HazXMit

Hazard X is sufficiently mitigated

HazYMit

Hazard Y is sufficiently mitigated

HazZMit

Hazard Z is sufficiently mitigated

AllHazIdent_Process Argument

All credible hazards have been exhaustively identified

Process Argument

Product Argument


Process and Product Argument 3Process Argument

AllHazIdent

All credible hazards have been exhaustively identified

SystHA

Hazard Analysis conducted using an established, systematic techniques

SQEPHA

Suitably Qualified and Experienced Personnel undertook the Hazard Analysis

DataInHA

Hazard Analysis was based on accurate and representative system data

Page 44


Elements of Process Argument

Traceability of artefactsE.g. traceability of requirements to design, implementation and verification

Competency of personnelE.g. experts, practitioners or supervised

Suitability and reliability of MethodsE.g. testing or analysis, formal or informal

Qualification of toolsE.g. Development or verification tools, qualified or unqualified

Suitability and clarity of notationsE.g. formal, informal or structured

IndependenceE.g. independence at the personnel and organisation level

Product Argument Example

Black box testing and state machine analysis provide explicit and independent evidence (solutions) directly related to system artefact rather than appealing to quality of development process


Page 45


Product Argument Example: Process Uncertainty

Black box testing processIs testing team independent from design team?Is process of generating, executing and analysing test cases carried out systematically and thoroughly?Is traceability between safety requirements and test cases well established and documented?

State machine analysis processHow accurate is correspondence between the mathematical model and software behaviour at run-time?Is analysis carried out by mathematically qualified engineers?


Integrating Process Arguments into Product Arguments

Process uncertainty should be addressed by linking process arguments to items of evidence used in product argument

Safety Requirements

Process Criteria

Product Argument

ProcessArgument

Evidence

Useful in absence of prescribed processEvidence-based ≠ Product-Based

Page 46


Product Argument Example Revisited


Product & Process Argument in Modular GSN

G1

C/S Logic is fault free

S2

Argument by omission of all identified software hazards

C1

Identified software hazards

G2

Press controls being 'jammed on' will cause press to halt

G3

Release of controls prior to press passing physical PoNR will cause press operation to abort

G4

C/S fails safe (halts) on, and annunciates (by sounding klaxon), all single component failures

Sn1

Black Box Test Results

G5

'Failure1' transition of PLC state machine includes BUTTON_IN remaining true

G7

'Abort' transition of PLC state machine includes BUTTON_IN going FALSE

Sn2

C/S State Machine

G8

Unintended opening of press (after PoNR) can only occur as a result of component failure

G9

Unintended closing of press can only occur as a result of component failure

Sn3

Fault tree analysis cutsets for event 'Hand trapped in

press due to command error'

Sn4

Hazard directed test

results

TestingProcessArg

BXTestingTrustworthy

Black box testing is trustworthy

SMachineProcessArg

SMachineTrustworthy

State machine analysis is trustworthy

FTAProcessArg

FTATrustworthy

Fault tree analysis is trustworthy HzTestingProcessArg

HzTestingTrustworthy

Hazard directed testing is trustworthy

Away Goal

S1

Argument bysatisfaction of all C/Ssafety requirements

HzIdentProcessArg

HzIdentTrustworthy

Process of identifying software hazards is trustworthy

Page 47

BXTestingTrustworthy

Black box testing is trustworthy

S1ArgBXTestProcess

Argument by considering black box testing process

BXTestTeam

Testing team is competent

BXTestCV

Experience, authority &

training

BXTestInd

Independence from design

team

BXTestRqSys

Testing method addresses safety requirements systematically

S1ArgBXTestCaseGen

Argument by considering test case generation & execution

BXTestRqGen

Test case generation was thorough

BXTestCaseCov

Safety requirements

coverage assessment

BXTestCaseExec

Test cases were executed on stable platform & object code version

BXTestPlatFrm

Target platform

BXTestCodeV

Final version of object code

BXTestAnalysis

Black box test result was analysed

BXTestRecord

Report of Black box test result analysis

BXTestTraceability

Testing process is traceable

CTrustworthiness

Trustworthiness definition

CProcess

Process definition

ConfigProcessArg

ConfigMang

Configuration management process is Trustworthy

Process Argument: Black Box Testing is

Trustworthy


Summary

Generic Computer System Safety Argument shownBoth process and product elements required

However, historically perhaps over-emphasis on processEvidence-based S/W Safety Argument Framework

Traceable to system-level hazardsBased upon taxonomy of software failure mode typesValidation, Satisfaction and TraceabilityPatterns of required argumentsLeading to targeted evidence selection

Linking Process and Product ArgumentsImportance of a Structured Approach to Assurance Case Construction and Presentation!