(Moonconf 2016) Fetching Moths from the Works: Correctness Methods in Software

FETCHING MOTHS FROM THE WORKSCORRECTNESS METHODS IN SOFTWARE

- or -

An Investigation into the Nature of Software with

a Particular Concern toward its Effective

Construction

- or -

Why do Computers Fail and What can be Done About It?

Why do Computers Stop and What Can be Done About It?

Jim Gray - 1986

@bltroutwine Moonconf, 2016

“The resulting systems have hardware MTBF

measured in decades or centuries. ”


“Unfortunately, it says nothing about tolerating

the major sources of failure. . .”


“Software”


design failures, open doors --


The Case of the Three Engineers vs. BART

Gordon Friedlander - 1974


agile iteration takes to the sky --


SIGSOFT Vol. 6 No. 2: Frontmatter

*gSoft Editor - 1981


a bug affects the staging prototype --


The BUG Heard 'Round the World Discussion of The Software Problem Which Delayed the First Shuttle Orbital Flight

John Garman - 1981


“Maintaining software systems in the field, absorbing large changes or additions in the middle of development

cycles. . . @bltroutwine Moonconf, 2016

. . . reconfiguring software systems to ‘fit’ never-quite-identical vehicles or missions are our real problems today.”


That was the late 1970s, have we made

progress?


Yes!


Sorta!


‘Correct’ is not a state, it’s a goal.


What’s needed is an understanding of how we fail to achieve it.


Analyzing Software Requirements Errors in Safety-Critical, Embedded Systems

Robin R. Lutz - 1993@bltroutwine Moonconf, 2016

“Few internal faults were uncovered

during integration and system testing.”


“Functional faults are the most common kind

of software error.”


What kind of software faults are there?


Program Faults • Internal mistakes • Interface violations • Functional violations


Program Faults • Internal • Interface • FunctionalBugs


Human Error • Intra-team comms. • Extra-team comms. • Misunderstanding spec. • Mishandling spec.


Human Error • Intra-team comms. • Extra-team comms. • Misunderstanding spec. • Mishandling spec.

Comm. Problems


Process Error • Inadequate testing • Inadequate specs. • Unknown requirements • Incorrect requirements


Process Error • Inadequate testing • Inadequate specs. • Unknown requirements • Incorrect requirements

Org. Goofs


‘Correct’ breaks down into two sub-goals.


- Validation -


- Verification -


What steps can we take today?


Step 0.


Convince your organization to

invest.


Eliminating Embedded Software Defects Prior to Integration Test

Ted Bennett, Paul Wennberg - 2005


“The more faults that pass undetected into integration test

and beyond, the more the project will cost and the longer

it will take to complete.”@bltroutwine Moonconf, 2016

Step 1.


Aim to make systems both safe

and reliable.


Engineering a Safer World Systems Thinking Applied to Safety

Nancy Leveson - 2011


Step 2.


Be clear on what your system must and mustn’t do.


The Role of Software in Spacecraft Accidents



“. . .software specifications often describe nominal behavior well but are very incomplete with respect to required software behavior under off-nominal

conditions . . .@bltroutwine Moonconf, 2016

“Most safety-related requirements. . .are best described using. . .design

constraints.”@bltroutwine Moonconf, 2016

Step 3.


"We don't want nobody that nobody

sent."


The Role of Software in Spacecraft Accidents



“It is widely believed that because software has executed safely in other applications, it will be safe

in the new one. . .


(M)ost accidents involve software that is doing exactly what it was designed to do (but) it reliably performs the wrong function.”


Step 4.


Audit and review all code. Aid with automated tests.


The OpenBSD Culture

David Gwynne - 2006


Going Fast Slowly

Poul-Henning Kamp, 2016


How SQLite is Tested

Dwayne Hipp - 2009


Step 5.


Use randomized testing and track coverage.


An Evaluation of Randomized Testing

Joe Duran, *meon Ntafos - 1984


“Our experiments have shown that random testing

can discover some relatively subtle errors without a great

deal of effort.”@bltroutwine Moonconf, 2016

QuickCheck A Lightweight Tool for Random Testing of Haskell Programs

Coen Claessen, John Hughes - 2000


Step 6.


Be willing to change your approach.


An Experimental Evaluation of the Assumption of Independence in Multiversion Programming

Nancy Leveson, John Knight - 1986@bltroutwine Moonconf, 2016

Step 7.


Use tools amenable to formal methods.


Rigorous Software Development An Introduction to Program Verification

Jose Almedia et al., 2011


Building High Integrity Applications with SPARK

John McCormick, Peter Chapin - 2015


Step 9.


Use formal methods.


Formal Specification and Documentation with Z A Case Study Approach

Jonathan Bowen, 2003


Moving Fast with Software Verification

Cristiano Calcagno et al., 2015


Step 9.


Build simple.


Out of the Tar Pit

Ben Moseley, Peter Marks - 2006


Normal Accidents Living with High-Risk Technologies

Charles Perrow - 1986


Step 10.


Build for failure.


Crash-Only Software

George Candea, Armando Fox - 2003


Making Reliable Distributed Systems in the Presence of Software Errors

Joe Armstrong - 2003


“We assume that such programs do contain errors, and investigate methods for

building reliable systems despite such errors.”


What must we invent?


Formal specification tools a project

manager can love.


Effective system modeling tools.


Methods for the effective analysis of running systems.


A techno-political culture of excellence.


What can we study?


Lots!


The End!