124
Cleanroom Software Engineering: An Overview Presented by: William Bail The MITRE Corporation and The University of Maryland SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000 SIGAda 2000 Tutorial SF7 12 November 2000 1

Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

  • Upload
    vuthien

  • View
    216

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

Cleanroom Software Engineering: An Overview

Presented by:

William Bail

The MITRE Corporation andThe University of Maryland

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

SIGAda 2000Tutorial SF712 November 2000

1

Page 2: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 222

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 3: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 333

Introduction½ Developed by Dr. Harlan Mills while at IBM

½ Cleanroom Software Development is a set of practices that, when followed, is said to improve the development experience and the resulting product

� Higher quality

� More efficient

� Repeatable results

½ Cleanroom has evolved from its original concept, as have other techniques

� Your preconceptions about Cleanroom may be inaccurate

� Currently viewed as a family of techniques, united by a common concept

Page 4: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 444

Introduction (cont’d)½ The name was borrowed from the Cleanroom facility required for

semiconductor manufacturing

� Foundries have to be very very clean to avoid faults on the microcircuits» Keep the defects from occurring during manufacturing (e.g., dust)

� The central idea behind Software Cleanroom is to create defect-free software

» Avoid the defects rather than finding them after development viatesting

%XW�GRQ·W�ZRUU\�� \RX�GRQ·W�KDYH�WR�FOHDQ�\RXU�GHVN

Page 5: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 555

Overview of Tutorial½ MITRE, as a part of its continuing evaluation of software

techniques, has identified Cleanroom as a key process suitable for adoption for the development of critical systems

½ This tutorial is based on Cleanroom references that are readily available

½ Purpose of this tutorial - to provide an introduction to gain familiarity and appreciation

½ Does not provide enough detail to begin practice

� Will provide an overview of the basic concepts and practices

� Will tantalize and tease, but for deeper information, you will need to consult the references

½ The material is scheduled for four 1 ½ hour sessions

Page 6: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 666

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 7: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 777

Basic Principles½ Primary focus is on defect avoidance rather than defect removal

� Make it right the first time

� Recognize that defects are not inevitable

� Avoid the costs of rework which are more expensive than originalconstruction

½ Emphasis on software development as an exercise in mathematics

� Exploit mathematics to facilitate the definition and verification of correct programs

» Some view Cleanroom as being semi-formal� Avoid trial-and-error as a strategy

� Employ statistics to certify quality

½ Four key features -

� Incremental development process

� Formal specification and hierarchical design techniques

� Formal reviews/ inspections to verify correctness

� Statistical quality control

Page 8: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 888

Key Features - Incremental Development½ Break problem down into multiple, smaller problems

� Not just 2 or 3 - but 5, 10, and more

� Each more manageable than the whole system

� As a series of incremental builds

½ With each increment a full product is developed, but with partial functionality

� “100% of 20%”, rather than “20% of 100%”

½ Each increment is cumulative, building to a full product

½ Many small steps result in a more manageable development by reducing complexity

Page 9: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 999

Key Features - Formal Specification / Hierarchical Design½ Based on three types of “Box Structures”, each of which provide

different views of the same component

� Black Box - external view of the component - precisely defines behavior

� State Box - mixed external/internal view -defines behavior linked to a limited set of design features (state variables)

» Intermediate step between behavior and design» Derived from Black Boxes by identifying states and state variables

� Clear Boxes - internal view - defines the full design, with possible use of lower-level Black Boxes (hence hierarchical)

» Based on Structured Programming principles to facilitate proofs of correctness

» Derived from State Boxes by identifying algorithms needed to implement behavior

½ Each Box is derived formally from the previous Box

� and is proven to be semantically equivalent (validation)

Page 10: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 101010

Key Features - Formal Verification½ Formal reviews / inspections to verify correctness

� Similar to Fagan inspections» Also developed out of IBM

� Proofs based on design structure rather than embedded logic» Therefore simpler to perform than “traditional” proofs of correctness

> which are hard to perform particularly for large programs / components

» Underpinnings are formal, practice is “semi-formal”> Often called “Light-weight” formal methods

� Performed after each development step

� Made possible by restricting design / code to Structured Programming constructs, each with a defined semantics

» <if-then-else>, <do-while>, <sequence>� Correctness for total product inferred from correctness of each

component within the product

Page 11: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 111111

Key Features - Statistical Quality Control½ Statistical quality control

� Achieved through reliability testing on increments

� Supported by creation of Usage Models in the form of Markov Chains» Similar to Use Cases but more comprehensive» Requires understanding how the product will be used

� Test data created by random sampling from the Usage Models» To recreate the operational environment

� Results provide prediction of expected MTTF and of stopping point for testing

» Only approximate of course

Page 12: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 121212

Increment 3Specification(Black Box)

State Box

Clear Box

Verification

Usage Model

Test Planning

Statistical Testing

Certification

Increment 2Specification(Black Box)

State Box

Clear Box

Verification

Usage Model

Test Planning

Statistical Testing

Certification

Overall Development Steps

Increment 1Specification(Black Box)

State Box

Clear Box

Verification

Usage Model

Test Planning

Statistical Testing

Certification

Page 13: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 131313

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 14: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 141414

History½ Originated with Harlan Mills while he was at IBM

� Along with codevelopers Michael Dyer and Richard Linger

� Resulted from an evolution of ideas culminating in their article in IEEE Software in September 1987, “Cleanroom Software Engineering”

½ Strongly based on previous work

� Dijkstra - Structured Programming» Dijkstra, E.W.,Notes on Structured Programming, Academic Press,

1972» Dijkstra, E.W.,a discipline of programming, Prentice-Hall, 1976

� Parkas - Modular design» Parkas, D.L., “On Criteria To Be Used in Decomposing Systems

into Modules”, CACM, vol. 14, no. 1, April 1972, pp. 221-227� Wirth - Stepwise refinement

» Wirth, N., “Program Development by Stepwise Refinement”, CACM, vol. 14, no. 4, 1971, pp. 221.227

Page 15: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 151515

Significant Milestones½ 1979 - Structured Programming: Theory and Practice, by Linger,

Mills, and Witt - Documents formal approach to design and verification

½ 1986 - Jan - “Certifying the Reliability of Software”, Currit, Dyer, and Mills, in IEEE Transactions on Software Engineering - Documents statistical testing approach

½ 1987 - Sep - “Cleanroom Software Engineering”, by Mills, Dyer, and Linger, in IEEE Software - Documents overall Cleanroom process

½ 1988 - DoD ARPA STARS program - Cleanroom selected as a key technology

½ 1991 - Nov - “Cleanroom Process Model”, by NASA Software Engineering Laboratory (SEL) - Reports on NASA’s approach to Cleanroom

½ 1992 - SEI Cleanroom Process Guide - Formalizes Cleanroom process

½ 1995 - SEI correlates Cleanroom to CMM - Maps Cleanroom process to Capability Maturity Model processes and standards

Page 16: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 161616

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 17: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 171717

Experience Using Cleanroom½ Cleanroom is not just a laboratory toy

½ It has been applied to real projects, with great success

½ We will look at 6 case studies

� IBM Cobol Structuring Facility (Cobol/SF)

� NASA Satellite-control project

� NASA Satellite Control

� IBM AOEXPERT/MVS

� IBM 3090E Tape Drive

� Ericsson Telecon S32 OS

Page 18: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 181818

Case Study 1 - IBM Cobol Structuring Facility (COBOL/SF)½ IBM’s 1st commercial Cleanroom product

� Product converts “unstructured” Cobol code into structured code

½ Complexity of product ≈ COBOL compiler

½ Developed in five increments (with no unit testing)

½ Complete product consisted of

� New code - 52 KSLOC PL/I

� Existing code - 28 KSLOC PL/I

½ 179 defects observed

� From first SW test through the end of statistical testing

½ 3.4 defects/KSLOC overall (for new code)

½ Productivity = 740 SLOC/sm

½ Postdelivery experience - 10 defects discovered by users after release for first 3 years (0.2 defects/KSLOC)

Page 19: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 191919

Case Study 2 - NASA Satellite-Control Project½ CFADS - Coarse/Fine Attitude Determination System

� Part of NASA Attitude Ground Support System (AGSS)

½ First Cleanroom project performed by Software Engineering Laboratory (SEL)

� Part of Goddard Space Flight Center

½ 31 KSLOC Fortran in 6 increments

� No unit testing

½ 3.3 errors/KSLOC during statistical testing

½ 780 SLOC/sm productivity

Page 20: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 202020

Case Study 2 - NASA Satellite-Control Project (cont’d)½ Where the defects were found

½ Note that 90% of the defects were discovered before the SW was executed

� I.e.,before testing

½ What is typical? Approximately ...

Design Code Compile TestReview Inspection

32% 53% 5% 10%

Design & Code Unit Test Integration System Test Op DemoInspections

55% 10% 20% 10% 5%

Dyer

85% vrs 55% 15% vrs 45%

Data suggests that Cleanroom finds defects earlier

In general, the earlier you find defects, the less the cost impact to fix

Page 21: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 212121

Case Study 4 - NASA Satellite Control½ 2 systems

� Attitude determination subsystem (20 KSLOC)

� Flight dynamics system (150 KSLOC)

½ The 2nd and 3rd Cleanroom projects at the SEL

½ 4.2 errors/KSLOC

Page 22: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 222222

Case Study 3 - IBM AOEXPERT/MVS½ A decision support system that predicts and prevents problems in

an MVS environment

½ Written in C, PL/I, Rexx, and TIRS - 107 KSLOC

½ 2.6 errors/KSLOC

½ 486 SLOC/sm

½ Experienced no operational errors after beta test

Page 23: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 232323

Case Study 5 - IBM 3090E Tape Drive½ Device controller to support tape cartridge operations

½ Consisted 86 KSLOC C code

½ Software development revealed hardware errors due to requirements definition process

½ 1.2 errors/KSLOC

Page 24: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 242424

Case Study 6 - Ericsson Telecon S32 OS½ 70 staff over 18 months

½ 30 KSLOC OS for new switching computers

½ Error rate of 1.0 errors/KSLOC

Page 25: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 -C

leanroom Softw

are Engineering Tutorial -12 Novem

ber 2000

Novem

ber 1, 2000252525

Range of Published Defect Densities

20.00

4.00

58.00

0.221.25

4.44

1.960.30

20.00

0.00

5.00

10.00

15.00

20.00

25.00

30.00

Akiy ama 1972

Rubey 1975

Belford 1979

Bazzana 1995 - Proj A

Bazzana 1995 - Proj B

Business Week 1991 -

USA

Business Week 1991 -

Japan

Pfleeger, Fenton, & Page

1994 - total system

Pfleeger, Fenton, & Page

1994 - subsystem

Data Source

Defect Density (Defects/KSLOC)

4.444.00

Average defect density in USA (M

. Dyer, 1992)

For comparison…

.but take with a grain of salt...

Page 26: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 262626

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 27: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 272727

Foundations for Cleanroom½ Harlan Mills saw SW development as an exercise in mathematics

� Using sound techniques, he believed that » Programmers could write correct programs» Verify that they are correct using proofs of correctness» Demonstrate their quality by showing how they work under realistic

scenarios� Basing proofs on Structured Programming constructs

» <sequence>, <if-then-else>, and <while-do>» Each of which has formally defined semantics

½ Mills compares Cleanroom to long division where the answer is computed one step at a time and the correctness of each step verified before moving to next step

½ Primary focus on defect avoidance rather than defect removal

� Make it right the first time

� Avoid rework costs which are more expensive than original construction

Page 28: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 282828

Key Features½ Incremental development process

½ Formal specification and hierarchical design techniques based on Box Structures

½ Formal reviews/ inspections to verify correctness

½ Statistical quality control

Page 29: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 292929

General Cleanroom Life Cycle

DefineRequirements

Define Incremental

Builds

Define Black Box Define

State Box

Define Clear Box Verify

Clear Box Certify

Increment

Performed once

Performed once per increment

Verify State Box

Page 30: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 303030

Key Features - Incremental Development ProcessÍ Incremental development process

½ Formal specification and hierarchical design techniques based on Box Structures

½ Formal reviews/ inspections to verify correctness

½ Statistical quality control

Page 31: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 313131

Incremental Development½ Construct product in small steps (increments)

½ Developing with increments breaks the problem up into smaller, more manageable chunks

½ With each increment a full product is developed, but with partial functionality

� “100% of 20%”» A fully executable and usable product is built» with 20% of the functionality that is planned for the whole product

rather than� “20% of 100%”

» A partial product that cannot be executed or tested as a whole

Page 32: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 323232

Cumulative Increments½ Each increment is cumulative, building to a full product

� Each increment contains the full functionality of all previous increments, plus adds new capabilities

� Avoid having to redo portions of earlier builds (too expensive)

½ Cleanroom supports two approaches (*):

� Incremental development - where the requirements are fixed at the beginning, and successive builds implement more and more requirements

� Evolutionary development - where the requirements are revisited after each build

Increment 1 Increment 2 Increment 3 Increment 4

10 +

*MIL-STD-498

Page 33: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 333333

Benefits of Incremental Development½ Clear evidence of progress

� Each increment represents a “complete” product that can be executed and used

� Early demonstration builds confidence and verifies that progress is being made

½ Support for early user feedback

� Users can take the increments out for a spin and kick the tires (so to speak)

� Opportunity for early feedback to redirect requirements

½ Recursive testing enhances product quality

� Each increment is tested as a full product

� Early increments are tested repeatedly (with each successive increment), enhancing quality

� Repeated testing provides early and extensive feedback on product and process quality

Page 34: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 343434

Benefits of Incremental Development (cont’d)½ Greater control for developers

� Multiple increments defined with small increases of functionality with each increment

� Allows developers to more carefully focus on current build and to manage change

� Can decide when in product development the “hard” problems are to be encountered

½ More effective system integration

� Since increments can be integrated early and are integrated repeatedly, system integration has more opportunity to be successful

Page 35: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 353535

Increment Evolution

New

Reused

Stub

Changed

Increment 1 Increment 2 Increment 3

New and changed modules represent added functionality

Stubs represent subordinate Black Boxes and are defined with full semantics but no implementation (Clear Boxes)

This semantics is necessary to support formal verification at each step

Page 36: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 363636

Key Features - Formal Specification and Hierarchical Design½ Incremental development process

Í Formal specification and hierarchical design techniques based on Box Structures

½ Formal reviews/ inspections to verify correctness

½ Statistical quality control

Page 37: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 373737

Formal Specification and Hierarchical Design½ Based on Box Structures

� An approach to system specification and design using stepwise refinement and verification

� Small design steps with verification of each step

½ Defined by Mills in

� Mills, H.D., “Stepwise Refinement and Verification in Box-Structured Systems,” IEEE Computer, June, 1988, pp. 23-36.

½ Guided by 4 principles

� Referential Transparency - All requirements for components should be fully defined (avoiding side effects)

» Allowing change without affecting dependent components� Transaction Closure - The operations on a component must cover the

state data, and the state data must be adequate to cover all operations

� State Migration - Data should be encapsulated in the smallest possible component to ensure locality

� Common Services - Components with multiple uses across the system should be made available as common services (“reuse”)

Page 38: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 383838

Box Structures½ Box Structures provide 3 views of each software component

� Black Box view - a precise definition of the component’s behavior in terms of stimuli and responses

� State Box view - definition of component’s behavior using state machines, and includes state variables to be used in the design

� Clear Box view - defines the algorithms and processing flow, in PDL or code» Based on Structured Programming concepts (limited control

constructs) to facilitate proofs of correctness» Each Clear Box may contain lower level Black Boxes

> Each with a Black, State, and Clear Box views» Results in a decomposition hierarchy

½ Each component is defined by all 3 box types

� Which are semantically equivalent

� Developed in the order - BB → SB → CB» Increasing design content

Page 39: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 393939

Box Structure Hierarchy½ Stepwise refinement produces a hierarchy of black boxes

� Hierarchy defines where each Box is used (a usage hierarchy)» as opposed to where defined

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

BB

SB

CB

Page 40: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 404040

Box Structure Development½ Box structure design consists of 3 steps:

� Define behavior by enumerating stimuli and responses - Black Box� Refine the Black Box by defining state variables and transitions - State

Box» Must be shown to have the same semantics as the Black Box

� Refine the State Box by defining computation necessary to implement the behaviors - Clear Box

» Must be shown to have the same semantics as the State Box

½ Each step requires making design decisions - not a mechanical process

� Each Black Box can be equivalent to many possible State Boxes - one must be selected

� Each State Box can be equivalent to many possible Clear Boxes - one must be selected

Page 41: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 414141

Box Structure Development (cont’d)½ After each step, the product is verified to be correct

� The derived State Box is shown to be equivalent to the Black Box

� The derived Clear Box is shown to be equivalent to the State Box

½ There are many techniques that can be used to define behavioral requirements

� Z

� Various tabular notations (D.L. Parkas. “Tabular Representation of Relations.” CRL Report No. 260, McMaster University, Hamilton, Ontario, October 1992)

� We will describe Sequence-based Specification

Page 42: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 424242

Black Box½ A description of the externally-visible behavior of a specific SW

component

� Whether the component is at the top-level (system) or at the low-level (module)

� Its functional requirements

½ A map of inputs to outputs

� Stimuli are mapped to responses

� Viewing software components as mathematical functions

½ Inputs to be represented as stimulus histories

� Since the outputs (responses) may vary depending on the sequence of stimuli that have been experienced (persistent memory)

½ The Black Box view presents the behavior view of the component

S R

Page 43: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 434343

Underlying Formalisms

BB : S* → RS* R

A regular expression thatdenotes all possible strings constructed out of elements in the set S

Example: If S = { a, b, c } then

S* = { a, b, c, aa, ab, ac, ba, bb, bc, ca, cb, cc, aaa, aab, aac, aaaa, aaab, aaac, … }

Strings of length 1Strings of length 2Strings of length 3Strings of length 4etc…...

A responseDefines the function BBthat maps from strings inS* to a single element of R

A Black Box is a function that maps stimuli histories to responses

Page 44: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 444444

A Protest from the Audience

:KR�DUH�\RX�NLGGLQJ"��1RW�LQ�0<�OLIHWLPH��

Actually, it is not that bad…stay tuned...

2XU�JRDO�LV�WR�HQXPHUDWH�DOO�SRVVLEOH�LQSXWV�DQG�VHTXHQFHV�RI�LQSXWV��DQG��IRU�HDFK��GHILQH�WKH�H[SHFWHG�RXWSXW

Page 45: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 454545

Example

Let S = { a, b, c } and let R = { Yes, No }

Define a function f that returns a Yes whenever its input contains 2 a’s in a row and returns a No otherwise.

That is:

Yes if i contains 2 a’s in a rowf(i) =

No otherwise

Page 46: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 464646

Example (cont’d)

Input Outputa Nob Noc Noaa Yesab Noac Noba Nobb Nobc Noca Nocb Nocc No

Input Outputaaa Yesaab Yesaac Yesaba Noabb Noabc Noaca Noacb Noacc Nobaa Yesbab Nobac No

Input Output Input Outputbba Nobbb Nobbc Nobca Nobcb Nobcc Nocaa Yescab Nocac Nocba Nocbb Nocbc No

Input Output Input Outputcca Noccb Noccc No

Input OutputInput Output

aaaa Yes

infinite number ofpossible strings

Strings oflength 2

Strings oflength 1

Strings oflength 3

Strings oflength 4

Page 47: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 474747

Some Considerations½ Stimuli sequences can be thought of as being processed

� One-by-one, or

� All-at-once

½ All-at-once

� Stimuli are individual input strings, perhaps selected from an infinite set

� The function processes one input and provides an output

� No stimulus history is needed (or is relevant)

½ One-by-one

� Each stimulus is input, processed, and a response is emitted

� Stimulus histories may be necessary to define the function

½ Our discussion focuses on one-by-one stimuli

� More interesting

� Allows us to focus on the state-based computation

� Fully equivalent to the all-at-once view

Page 48: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 484848

Some Considerations (cont’d)½ Other techniques that we could have used to define sequences

include:

� Regular expressions - to describe stimuli histories

� Grammars - to define stimuli histories

� Others? ... join in with ideas ...

Page 49: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 494949

What about...?½ What happens if no output is necessary for a specific input?

� Define a special output � (the null response)which denotes no output

½ What about an empty input sequence?

� Define a special symbol to denote this situation : λ� Note that BB(λ) = �

½ What about an input sequence that is not legal?

� Unfortunately these happen all the time with real systems

� Since our function definitions have to be complete (represent total functions), we need to handle these appropriately

» Exception handling

½ What about an input sequence that is impossible?

� That is, physically impossible due to external conditions

� Map these to the special symbol §

� That is, BB( xxx ) = § where xxx is impossible

� Not an output - just an indication of not_possible

Page 50: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 505050

Example Revisited

σ BB(σ)λ �

a Nob Noc Noaa Yesab Noac Noba Nobb Nobc Noca Nocb Nocc No

Define BB : S*Å Rwhere

S = { a, b, c, …, z }R = { Yes, No }Let σ denote a string from S* - that is, σ ∈ S*

σ BB(σ)aaa Yesaab Yesaac Yesaba Noabb Noabc Noaca Noacb Noacc Nobaa Yesbab Nobac Nobba No

σ BB(σ)bbb Nobbc Nobca Nobcb Nobcc Nocaa Yescab Nocac Nocba Nocbb Nocbc Nocca Noccb Noccc No

Note that certain sequences are equivalent.

Once an stimulus history has experienced an aa pattern, then the response will always be “Yes” for all subsequent stimuli.

aaa, aab, aac, baa, caa

are all equivalent to aa

etc...

Page 51: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 515151

Refinements½ Identifying equivalent sequences

� Two stimuli sequences are equivalent iff they produce the same responses for all future stimuli

� If a sequence is equivalent to another one, it is reduced» In the example, stimulus aac is reduced to sequence aa» ccc can be reduced to cc» aba can be reduced to a

� If a sequence is not equivalent to any previous one, it is irreducible

In the sense of a sequence ordered by length

Page 52: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 525252

New Enumeration Tableσ BB(σ) Equivλ �

a Nob Noc No baa Yesab No bac No bba No abb No bbc No bca No acb No bcc No baaa Yes aaaab Yes aaaac Yes aaaba No aabb No babc No baca No aacb No b

σ BB(σ) Equivacc No bbaa Yes aabab No bbac No bbba No abbb No bbbc No bbca No abcb No bbcc No bcaa Yes aacab No bcac No bcba No acbb No bcbc No bcca No accb No bccc No b

Note that some of the sequences are irreducible, other can be reduced.

Any sequence that is legal and irreducible is called canonical

Note that once we have enumerated all sequences of length = 3, we are done, since they all have equivalent sequences that are shorter.

Page 53: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 535353

Other Refinements½ Identifying equivalent sequences (as we did in the example)

½ Abstractions - Functions that shorten the stimulus histories� Avoid having to list out many repetitive stimuli

» e.g., 10-Clock-pulses - denotes a stimulus which is the 10th clock pulse

» e.g., phone-number - denotes a fully-formed phone number> Abstract the primitive (atomic) inputs into compound data items

½ Specification functions� Avoid having to provide detail on deterministic results by providing a

function name for a set of responses» e.g., sqrt, max(ai), etc...

But it gets better...

Page 54: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 545454

State Box½ Derived from the Black Box

� Regards the stimulus history as a state

� Based on the canonical sequences

½ For each stimulus, use the black box function to compute the response from the stimulus history

½ State box consists of a description of -

� The externally-visible behavior of a specific SW component

� Its state variables

½ A map of inputs to states and outputs

� Stimuli are mapped to state transitions and to responses

� Expressing the same semantics as the Black Box but with some design information

½ There are actually two stages to defining state boxes

� First, identify the states based on the equivalent enumerations (still requirements)

� Second, define state variables to capture the states (now some design)

Page 55: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 555555

State Box Definition

S R

StateT

SB: S* × T* Å R × T

where:SB State box functionS StimuliR ResponseT State space

Specifies state data and transitions needed to achieve the behavior defined for the black box

Page 56: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 565656

Stage 1 - Identify State Machine½ Identify the stimulus histories that form equivalences

� Canonical sequences

� Denote them as states

½ Sequences will have repeating patterns

� I.e., two different input sequences will have the same response for the same input and for all future inputs

� The different sequences end up in the same program state

½ For each stimulus, define responses and state transition to define the same function as the enumerations

½ Program behavior is then defined by current state and current input rather than stimuli history and current input

Note that this form of the State Box is still purely at the requirements level - expressing externally visible behavior

Page 57: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 575757

Exampleσ BB(σ) Equivλ �

a Nob Noc No baa Yesab No bac No bba No abb No bbc No bca No acb No bcc No baaa Yes aaaab Yes aaaac Yes aaaba No aabb No babc No baca No aacb No b

σ BB(σ) Equivacc No bbaa Yes aabab No bbac No bbba No abbb No bbbc No bbca No abcb No bbcc No bcaa Yes aacab No bcac No bcba No acbb No bcbc No bcca No accb No bccc No b

Remember this table?

The canonical sequences now become our states

Page 58: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 585858

Example (cont’d)

Current Stimulus Response NextState State

@ a No <a>@ b No <b>@ c No <b>

<a> a Yes <aa><a> b No <b><a> c No <b><b> a No <a><b> b No <b><b> c No <b>

<aa> a Yes <aa><aa> b Yes <aa><aa> c Yes <aa>

@ = Initial state

One_aNo_aNo_a

Two_aNo_aNo_a

One_aNo_aNo_a

Two_aTwo_aTwo_a

A convenient renaming of the states

A finite state machine

Page 59: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 595959

A State Diagram

@

No_a

Two_a

One_aa / No

b ∨ c / No

a / Yes

a / Nob ∨ c / No a ∨ b ∨ c / Yes

This is a complete definition of the functional behavior of this module

b ∨ c / No

Note the convention we use here:On each arc is an indication of the stimulus that causes the state transition, and the output that is produced

Page 60: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 606060

Stage 2 - Identify State Variables½ At this stage, developers define variables that will capture the state

of the machine

½ The State Box at this point becomes part of design

� Internal implementation detail

½ For the example we could define three state variables

� Each with True / False values

� or we could define one state variable, with 3 values:

No_aOne_aTwo_a

Num_a

= 0 if last symbol not an “a”= 1 if last symbol is an “a” and symbol before last is not an “a”= 2 if last 2 symbols are “a”s

Page 61: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 616161

Formalisms½ Derivation of state box is supported finite state automata theory

½ Kleene’s Theorem

� Any regular set can be recognized by some finite-state machine

SB : S × T → T × R

where:SB State box functionS StimuliR ResponsesT State space

Here the stimuli history is captured in the current state

A set is regular if it can be defined by a regular expression,e.g. S*

Page 62: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 626262

Discussion1RW�VR�IDVW«

1RW�DOO�LQSXW�VHWV�DUH�DV�OLPLWHG�DV�UHJXODU�H[SUHVVLRQV6RPH�DUH�FRQWH[W�IUHH��VRPH�DUH�FRQWH[W�VHQVLWLYH��DQG�VRPH�DUH�SKUDVH�VWUXFWXUH�JUDPPDUV$QG�WKHVH�FDQQRW�EH�UHFRJQL]HG�E\�)6$V����

½ So true, but consider …

%\�WKH�ZD\��DUH�ZH�ORVLQJ�DQ\RQH�KHUH"��7KLV�PD\�JHW�D�ELW�WHFKQLFDO��EXW�LV�LQFOXGHG�WR�VDWLVI\�WKH�FXULRVLW\�RI�VRPH�RI�WKH�DXGLHQFH���:H�QHHG�WR�VKRZ�MXVW�D�WLQ\�ELW�RI�WKH�XQGHUO\LQJ�WKHRU\���7UXVW�XV���WKLV�UHDOO\�ZRUNV���

Page 63: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 636363

Considerations½ We need to define all possible inputs, not just legal inputs

� If C is the set of all single inputs, then C* is the set of all possible input strings» Including legal and illegal» Even if the legal strings need to be defined by a context-free

grammar� When defining input sequences, we partition the input space into

classes of input strings» Legal, illegal, improbable, likely,... (looking ahead to usage models)

½ Input strings can be defined in several ways, including:

� Regular expressions - as we have seen

� Grammars - such as for programming languages» These can be finite state, context free, context sensitive, or

unlimited» Note that creating a recursive descent parser is similar to the

approach used by Cleanroom for developing programs> The design is based on the input specification

Page 64: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 646464

Hard Problems½ Some problems we cannot solve

� e.g., the halting problem

½ Some may be hard to solve

� NP-Complete

� e.g. Hamilton’s problem - Given a graph, is there a path through the graph which visits each certex precisely once? Is there a path which ends up where it started?

½ What about input sequences that do not have patterns?

� e.g., identifying irrational numbers

½ These are all hard to solve using conventional techniques as well

½ Many problems (as defined by their input strings) can be solved...WKDQN�JRRGQHVV

½ For interesting problems that are solvable, Cleanroom provides apowerful approach

Page 65: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 656565

Clear Box½ Now that we have defined our State Box, what next?

½ Refine the State Box by defining the procedures that implement the functions

� May need to define lower-level Black Boxes to provide functionality

½ Within the Clear Boxes, we need to limit the control structures to the 7 structured programming constructs

� Function - f

� Sequence - f; g

� If…then - if p then f

� If…then…else - if p then f else g;

� Whiledo - while p do f ;

� Do until - do f until p;

� Do…while…do - do f while p do g ;

� Do wa diddy

Page 66: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 666666

Why?½ Why?

½ Because these forms have well defined semantics that support formal verification

½ These forms are prime programs

� A prime program is a proper program that has no proper subprogram of more than one node

� A proper program» has a single entrance and a single exit» all nodes have a path through that node from the entry to the exit

½ The 7 listed are unique -

� There are 15 possible prime programs containing 1 to 4 nodes

� These 7 are the only ones that contain function nodes» i.e., can process data

Page 67: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 676767

So What?½ So each one has a single entry and a single exit

� So no side effects and unintended consequences may occur

� They have the form of a mathematical function

� They supports referential transparency

½ When we develop programs that contain only these 7 prime programs, we limit ourselves concerning side effects

½ We also make things more scalable

� Neat, well-defined building blocks

Page 68: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 686868

The Control Structures

g h sequence

p

g

if...then...

p

g

h

if...then...else

Page 69: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 696969

The Control Structures (cont’d)

p

g

while...do

pg do...until

pg

h

do...while...do

Page 70: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 707070

Intended Functions½ For each control structure, intended functions are attached

½ These express (as a comment) what the control structure is intended to do

½ For example:

[ f ]gh

[ f ]if

pthen

gend if

intended function for the sequence

should describe the function:

h ( g ( i ) )

intended function for the if...then

should describe the function:

(p → f = g | ¬p → nil )

Page 71: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 717171

Code Generation½ Based on the state variables, control structures can be directly

generated

� Almost a mechanical process

½ The form is

Procedure CB (inputs, outputs)Declare state variablesDeclare local variablesDetermine stateTest for state

Processing to be done in that stateTest for state

Processing to be done in that stateTest for state

Processing to be done in that stateTest for state

Processing to be done in that state

Page 72: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 727272

State Box versus Clear Box

If Curr_Symb = “a” THENNum_a := Num_a + 1;If Num_a > 2 THEN

Num_a := 2;END IF;

ELSENum_a := 0;

END IF;

IF Num_a = 0 THENPut (“No”);

ELSIF Num_a = 1 THENPut (“No”);

ELSIF Num_a >= 2 THENPut (“Yes”);

END IF;

Current Stimulus Response Next Num_aState State

@ a No <a> 1@ b No <b> 0@ c No <b> 0

<a> a Yes <aa> 2<a> b No <b> 0<a> c No <b> 0<b> a No <a> 1<b> b No <b> 0<b> c No <b> 0<aa> a Yes <aa> 2<aa> b Yes <aa> 2<aa> c Yes <aa> 2

Determinestate

Processstate

Page 73: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 737373

Summary of Box Structure Expansion Process½ Eleven steps -

� Black box definition1 Define all possible stimuli and stimuli histories2 For each, define the required response of the black box

� State box definition3 Correlate for each response, the stimulus histories that cause that

response4 Identify state data to define each state5 Identify the internal black boxes needed to create the state box6 Verufy that the state box is correct relative to the black box

definition� Clear box design

7 For each state datum, identify where it is referenced8 Define data abstractions for each state datum9 Define processing to replace internal black boxes10 Verify the clear boxes

� Iteration11 Repeat until complete

Page 74: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 747474

Key Features - Formal reviews/ inspections to verify correctness

½ Incremental development process

½ Formal specification and hierarchical design techniques based on Box Structures

Í Formal reviews/ inspections to verify correctness

½ Statistical quality control

Page 75: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 757575

Verification of Correctness½ Formal reviews/ inspections to verify correctness

� Proofs based on design structure rather than embedded logic

� Made possible by restricting Clear Boxes to Structured Programming constructs, each with a defined semantics

� Correctness for product inferred from correctness of each component

½ Performed at each step of the decomposition

� After defining the State Box for a Black Box, they must be shown to be equivalent

» I.e., compute the same function» Technique - demonstrate that the state machine captures all stimuli

sequences with the proper responses� After defining the Clear Box for a State Box, they must be shown to be

equivalent

Page 76: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 767676

Basic Premise½ Verify that a program is equivalent to its intended function

� Decomposed to proving correctness of parts and then by inference the whole

� Based on Prime Programs

½ Each control structure is abstracted to its function

� and then compared to the intended function

½ A specific set of questions are posed at each level, based on the specific control structure

� Defined by Linger, Mills, and Witt in 1979

½ The correctness demonstration proceeds from the prime programs upwards to the component level

½ Since the code format strictly adheres to the state machine structure, the abstraction can be easily compared to the original state machine

½ The process is effective even when performed semi-formally

� Provides a focus to the formal inspection process

Page 77: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 777777

Verification Process Flow

f = Specification

P = ProgramProgramfunction = [P]

abstraction

compare development

1

2

3

Page 78: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 787878

Sample Correctness Questions

Sequence [ f ]g;h;

Composition questionDoes h ( g ( i ) ) = f ?

If...then [ f ]if

pthen

gend if

Composition questionIf p is true, does p (i) = fIf p is false, does g(i) = f?

Page 79: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 797979

Key Features - Statistical Quality Control½ Incremental development process

½ Formal specification and hierarchical design techniques based on Box Structures

½ Formal reviews/ inspections to verify correctness

Í Statistical quality control

Page 80: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 808080

Certification½ Statistical quality control

� Achieved through reliability testing on increments

� Supported by creation of Usage Models in the form of Markov Chains

� Test data created by random sampling from the Usage Models

� Results provide prediction of expected MTTF and of stopping point

½ Based on statistical testing

½ Each increment is certified by challenging it with test data selected from its operational profile

Page 81: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 818181

Usage Models½ A model of how the system is to be used

� Operational profiles

½ Based on the State Boxes

� Augmented by probabilities for each input / state transition

� Markov chain

½ Method -

� Assign to each input a probability of occurrence corresponding to expected usage

� Create separate models for classes of users / modes of usage / environment

» Process called Stratification� Some failures are more critical than others

» Stratification helps to focus on the more critical ones

Page 82: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 828282

Example

Current Stimulus Response Next Naïve User ExpertState State Probability Probability

@ a No <a> 0.2 0.5@ b No <b> 0.3 0.3@ c No <b> 0.5 0.2

<a> a Yes <aa> 0.1 0.5<a> b No <b> 0.3 0.3<a> c No <b> 0.6 0.2<b> a No <a> 0.2 0.5<b> b No <b> 0.3 0.3<b> c No <b> 0.5 0.2

<aa> a Yes <aa> 0.2 0.1<aa> b Yes <aa> 0.3 0.4<aa> c Yes <aa> 0.5 0.5

Different user strata

Page 83: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 838383

How Do You Know? ½ There are several techniques for selecting probabilities of stimuli

½ History

� Based on experience with a previous system

½ Projected usage

� Based on models of how the system will be used after deployment

� e.g., Object Oriented Use cases and scenarios

½ Measurement

� Based on measures of usage obtained from early increments

½ Approximation

� Equally allocate probability to all alternatives

½ Continual refinement through each increment provides a converging set of estimates

Page 84: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 848484

Statistical Testing Rationale½ Reliability of SW “in the field” depends on two factors:

� Location of defects in the code

� Usage profile for the SW» Which dictates what parts of the code will be executed

½ Defects are not, by themselves, harmful

� They cause problems only when they are executed» And sometimes not even then!

½ Goal of statistical testing is to test the software under the same conditions that it will experience when “in service”

� Its behavior will then be similar to what users should experience after release

½ Test cases are based on a random sample taken from the usage model

� If usage model is accurate, random sample will closely approximate deployment experience

Page 85: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 858585

Data Collection½ When performing statistical testing using Markov chains, certain

histories need to be kept

� Number of states visited during test

� Number of arcs visited during test

½ Goal is to at least visit every state and every arc

½ Also keep track of pass/fail histories over increments

� Support the computation of stopping criteria

½ Based on testing histories, reliability estimates can be computed

� MTTF

� Approximate but provide assessment of software behavior during deployment

Page 86: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 868686

Defects and Reliability½ Consider the following program

PROCEDURE Sample (I : Integer) ISj : Integer;

BEGINGet (I);IF (I < 0) THEN

xx;

ELSIF (I < 1 THENyy;

ELSIF (I < 5) THENzz;

ELSEaa;

END IF;END;

½ Suppose that this program has defects in all 3 THEN clauses

� xx, yy, and zz

� Defect density = 3 defects/14 SLOC or 214 defects / kSLOC

½ Suppose that when used, I is always > 5

� No failures will be experienced

� Reliability = 1.00

Page 87: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 878787

Defects and Reliability (cont’d)½ Consider the following program

PROCEDURE Sample (I : Integer) ISj : Integer;

BEGINGet (I);IF (I < 0) THEN

xx;

ELSIF (I < 1 THENyy;

ELSIF (I < 5) THENzz;

ELSEaa;

END IF;END;

½ Suppose that this program has a defect in aa

� Defect density = 1 defects/1,013 SLOC or 0.99 defects / kSLOC

½ Suppose that when used, I < 0 once per 1000 invocations (or onceper hour) and I > 5 the rest of the time

� The component will fail almost every time, and succeed once per hour

� Reliability = 1 failure every 3.6 sec

1000 SLOC

Page 88: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 888888

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 89: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 898989

Cleanroom Process Model½ In 1996, the SEI released a Cleanroom Software Engineering

Reference Model (CRM)

� Richard C. Linger, Carmen J. Trammell, “Cleanroom Software Engineering Reference Model Version 1.0”, Technical Report CMU/SEI-96-TR-022, ESC-TR-96-022 November 1996

½ The CRM is intended to serve as a guide for projects which intend to use Cleanroom by defining the life cycle model

½ The CRM does not provide instruction in how to apply Cleanroom

� Technical details, specific techniques,…

½ This section of the tutorial summarizes CRM

� Using material extracted from the Technical Report

Page 90: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 909090

Cleanroom Life Cycle Phases½ Requirements analysis - formal definition of product behavior

½ Increment definition - sequence of requirements to be built into successive builds (increments)

½ Black box definition - formal definition of product for current increment

½ State box definition - identification of product states based on black box

½ Clear box development - development of code

½ Clear box verification - formal proof of correctness of the clear box

½ Certification - statistical testing of the increment

Page 91: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 919191

The 14 Cleanroom ProcessesCleanroom Management ProcessesProject Planning Process

Cleanroom Engineering GuideSoftware Development Plan

Project Management ProcessProject Record

Performance Improvement ProcessPerformance Improvement Plan

Engineering Change ProcessEngineering Change Log

Cleanroom Specification ProcessesRequirements Analysis Process

Software RequirementsFunction Specification Process

Function SpecificationUsage Specification Process

Usage SpecificationArchitecture Specification Process

Software ArchitectureIncrement Planning Process

Increment Construction Plan

Cleanroom Development ProcessesSoftware Reengineering Process

Reengineering PlanReengineered Software

Increment Design ProcessIncrement Design

Correctness Verification ProcessIncrement Verification Report

Cleanroom Certification ProcessesUsage Modeling and Test Planning Process

Usage ModelsIncrement Test PlanStatistical Test Cases

Statistical Testing and Certification ProcessExecutable SystemStatistical Testing ReportIncrement Certification Report

The processThe products of the process

Page 92: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 929292

Cleanroom Process Flow

RequirementsAnalysis

UsageSpecification

FunctionSpecification

IncrementPlanning

BlackBox

StateBox

ClearBox

Statistical Testingand Certification

Usage Modelingand Test Planning

For lastincrement

For intermediateincrements

For evolutionary life cycles

For incremental life cycles

Increment Design

Correctness Verification

Software Reengineering

Cleanroom Management Processes are concurrent across entire life cycleArchitecture Specification Process is concurrent across entire life cycle

Page 93: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 939393

Cleanroom Life Cycle vrs “Waterfall”

Requirementsanalysis

Architecturaldesign

DetailedDesign

Code/Unit Test

Integration& Test

FormalQualification

Test

RequirementsAnalysis

FunctionSpecification Correctness Validation

CertificationBlack Box State Box Clear Box

Cleanroom Requirements Analysis is similar to Waterfall Requirements Analysis

Cleanroom Function Specification carries RA further by defining system level Black, State, and Clear Boxes which capture the detailed behavior plus the architectural design

Cleanroom BB-SB-CB-Validation corresponds to Waterfall Detailed Design, Code/Unit Test, and some of Integration and Test.

Cleanroom Certification contains elements of Waterfall Integration and Test and FQT

Page 94: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 949494

Cleanroom Management Processes½ Project Planning

� Development of detailed plans for how the project will be conducted

½ Project Management

� Manage all activities involved with the Cleanroom Development

½ Performance Improvement Process

� On a continual basis» Monitor the development activities as well as key performance

attributes» Identify shortfalls and failures» Determine causes» Implement process enhancement

½ Engineering Change

� Configuration management functions

� Control and manage changes and updates to the work products

Page 95: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 959595

Architecture Specification½ This process is performed on a continuing basis

½ Define the strategy and guidelines under which the architecture for the product will be developed

½ Monitor architectural development and feedback improvements in acontrolled manner

½ Identify and monitor use of legacy, COTS, and NDI software resources

Page 96: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 969696

Architectural Concept

CSCI F

Cap

abilit

ies

CSCI C

Inputs Outputs

CSCI A

CSCI D

CSCI E

Cap

abilit

ies

Hardware

System Functional RequirementsCSCI A Requirements

CSCI B

CSCI B Requirements CSCI C Requirements CSCI F Requirements

CSCI E Requirements

CSCI C

CSCI C Requirements

HardwareHW Requirements

Page 97: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 979797

Requirements Analysis½ This activity is similar to the “traditional” requirements analysis

phase

½ Goal is to capture

� Capabilities to be provided� Functions to be implemented� Necessary performance (temporal and capacity)� Usage profile (use cases and scenarios)� The environment in which the system will be used

½ Requirements are documented in a Software Requirements document

½ Interaction with the customer is crucial to ensure acceptance

½ Quality criteria:

� Complete� Consistent� Feasible� Testable

Page 98: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 989898

Function Specification½ For most techniques, usually a part of requirements analysis

½ The goals for Cleanroom are more ambitious

� Requires more detail about the capabilities/functions

Specify, the complete functional behavior of the software in allpossible circumstances of use

Mathematically precise, complete, and consistent

½ Obtain agreement with the customer on the specified function

½ Not really, and not consistently

½ There is a wide range of practice in developing software requirements documents

� Current approaches tend to fall between capability descriptions and functional behaviors

%XW�ZDLW�� ZH�DOUHDG\�GR�WKLV�DV�D�SDUW�RI�UHTXLUHPHQWV�DQDO\VLV����5LJKW"

Page 99: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 999999

Levels of Requirements½ Requirements can be specified at various levels of detail

� Very general - e.g., “The product shall allow users to perform word processing”

to� Very detailed - “Pressing Ctrl and I at the same time results in the

selected text being converted to an italics font within 0.5 sec”

½ Regardless of the level and amount of detail, as long as the descriptions address external behavior, they are still requirements and not design

½ When developing a system, requirements typically start at the general level

� List of capabilities - what the users want to be able to do

½ As development proceeds, the general requirements are refined until they become specific behaviors

or should be

Page 100: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 100100100

Degrees of Specificity

Degrees of Specificity

General statementof intent

Specific functionalbehavior

Wordprocessing

Handle italics Use Ctl-I foritalicization

Page 101: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 101101101

Capabilities versus Functions

Capabilities

A

B

C

Alternative Requirements Sets

Each alternative requirements set represents a different approach to providing the capabilities.

The requirements analysis process involves defining some alternative sets of requirements, and choosing one.

The customer needs to verify that the selected alternative satisfies his needs.

Once an alternative is selected, the customer agrees that implementing that alternative will satisfy his needs.

When implemented, acceptance testing needs to focus on the implemented requirements. If successfully implemented, the capabilities are de facto provided.

Page 102: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 102102102

“What” versus “How”

½ “What” and “how” are misleading clichés

� Often used to differentiate requirements from implementation

� What “implies” requirements

� How “implies” design

½ Often confusing, certainly inaccurate

½ Problems:

� Fail to recognize hierarchical nature of requirements/design» Designs involve functional requirements of system

components� Fail to recognize colloquial English

» “How does the user interface provide help information?” -requirements

» “What tasking structure does this component use?” - design

<HV��EXW�WKH�FDSDELOLWLHV�WHOO�PH�ZKDWWKH�FXVWRPHU�ZDQWV��DQG�WKH�GHWDLOV�WHOO�PH�KRZ��ZKLFK�LV�UHDOO\�GHVLJQ�

Page 103: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 103103103

Externally Visible Behavior½ Cleanroom takes a more precise view of requirements

½ Requirements cover all externally-visible behavior

� Include functions, screen formats, GUIs, menus, interfaces, etc.

½ Cleanroom defines the detailed behavioral requirements in a functional specification using a formal, mathematical approach

� Considers every possible input, every possible user scenario

Page 104: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 104104104

Usage Specification½ The approach taken by Cleanroom for testing is based on

challenging the increments with tests based on how the software will be used in deployment

� Statistical testing

½ The Usage Specification activity develops the models of expectedbehavior

Page 105: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 105105105

Increment Planning½ Define the phased implementation of requirements (functions) into

successive incremental builds

½ Coordinate with the customer to ensure a consistent and useful progression

½ Ensure that components preserve referential transparency

½ Design increments so that increasing functions are coordinated to permit a smooth architecture enhancement

½ Guarantee that increments are

� Small enough to facilitate a quality and efficient development

� Large enough to represent effective plateaus

Page 106: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 106106106

Software Reengineering½ This process is used when when reused software is to be

incorporated into the product

� COTS

� NDI

� Legacy

½ The purpose is to ensure that the semantics are appropriate for the design

½ Restructure to ensure referential transparency if necessary

½ Create BB level enumerations (or the equivalent) to support verification

½ Incorporate into increments to ensure smooth enhancement

Page 107: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 107107107

Correctness Verification½ Perform the verifications required

� Verify that all State Boxes are equivalent to their predecessor Black Boxes

� Verify that all Clear Boxes are equivalent to their predecessor State Boxes

½ These are generally performed as formal inspections via peer reviews

½ All defects are to be documented and traced to ensure correction

Page 108: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 108108108

Usage Modeling and Test Planning½ Prepare Usage Models for each component

½ Verify that the Usage Models are equivalent to the State Boxes relative to semantics

½ Ensure that the probabilities are reasonable and based on appropriate criteria

½ Construct usage strata to cover all expected modes of use of theproduct

½ Develop a statistical test plan

Page 109: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 109109109

Statistical Testing and Certification½ Create test cases by taking random samples over the Usage

Models

½ Apply the test cases to the increments and verify correct (or incorrect) results

½ Compute the expected reliability

½ Determine how much more testing is required to achieve the desired level of reliability and quality

Page 110: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 110110110

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 111: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 111111111

Cleanroom and the CMM½ The Capability Maturity Model was developed by the Software

Engineering Institute to aid in assessing the level of maturity of an organization’s software development processes

½ It consists of a set of 18 key process areas (KPAs) and 5 levels of maturity

½ Organizations are assessed against these KPAs and are awarded a maturity level based on the assessment

½ In 1996, the SEI mapped the 18 KPAs to the 14 Cleanroom processes

:K\"If a project adopts Cleanroom, the mapping assists in determining the CMM level of maturity that can be expected

It also shows the extent to which Cleanroom covers development activities (and where you need to augment)

Material in this section extracted from: Linger, Richard C., Mark C. Paulk, and Carmen J. Trammell, “Cleanroom Software Engineering Implementation of the Capability Maturity Model (CMMsm ) for Software”, December 1996, Technical Report CMU/SEI-96-TR-023, ESC-TR-96-023

Page 112: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 112112112

CMM - Five Levels of Maturity½ For reference, the five levels are:

1 Initial The software process is characterized as ad hoc, and occasionally even chaotic.Few processes are defined, and success depends on individual effort and heroics.

2 Repeatable Basic project management processes are established to track cost, schedule, andfunctionality. The necessary process discipline is in place to repeat earliersuccesses on projects with similar applications.

3 Defined The software processes for both management and engineering activities aredocumented, standardized, and integrated into a standard software process forthe organization. All projects use an approved, tailored version of theorganization’s standard software process for developing and maintainingsoftware.

4 Managed Detailed measures of the software process and product quality are collected.Both the software process and products are quantitatively understood andcontrolled.

5 Optimizing Continuous process improvement is enabled by quantitative feedback from theprocess and from piloting innovative ideas and technologies.

Page 113: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 113113113

The KPAs½ For reference, the KPAs are correlated to the 5 levels as follows:

Level 2RequirementsManagement (RM)

Software ProjectPlanning (PP)

Software ProjectTracking andOversight (PT)

Software SubcontractManagement (SM)

Software QualityAssurance (QA)

SoftwareConfigurationManagement (CM)

Level 3Organization ProcessFocus (PF)

Organization ProcessDefinition (PD)

Training Program (TP)

Integrated SoftwareManagement (IM)

Software ProductEngineering (PE)

Intergroup Coordination(IC)

Peer Reviews (PR)

Level 4Quantitative ProcessManagement (QP)

Management (QM)

Level 5Defect Prevention (DP)

Management (TM)

Process ChangeManagement (PC)

Page 114: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 114114114

Levels of Correlation½ Correlation is scored in 5 levels

� From highly correlated to low correlation to not included

Correspondence Category Keyword

The CMM element is consistent with Cleanroom processes, High (H)and implementation by Cleanroom processes is high.

The CMM element is consistent with Cleanroom processes, Partial (P)and implementation by Cleanroom processes is partial.

The CMM element is consistent with Cleanroom processes, Low (L)and implementation by Cleanroom processes is low.

The CMM element is consistent with Cleanroom processes, but is not Consistent (C)implemented in the Cleanroom processes, or is implemented in an indirect way.

The CMM element is not included in Cleanroom processes, and an alternative Alternative (A)implementation is defined by Cleanroom processes.

Page 115: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 115115115

Cleanroom and the CMM - The Results

Level 2 KPAs: RepeatableKey Process Area Cleanroom ImplementationRequirements Management HighSoftware Project Planning HighSoftware Project Tracking & Oversight HighSoftware Subcontract Management ConsistentSoftware Quality Assurance PartialSoftware Configuration Management Partial

Level 3 KPAs: DefinedKey Process Area Cleanroom ImplementationOrganization Process Focus ConsistentOrganization Process Definition PartialTraining Program PartialIntegrated Software Management HighSoftware Product Engineering HighIntergroup Coordination HighPeer Reviews High

Page 116: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 116116116

Cleanroom and the CMM - The Results (cont’d)

Level 4 KPAs: ManagedKey Process Area Cleanroom ImplementationQuantitative Process Management HighSoftware Quality Management High

Level 5 KPAs: OptimizingKey Process Area Cleanroom ImplementationDefect Prevention HighTechnology Change Management PartialProcess Change Management Partial

Page 117: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 117117117

Cleanroom and the CMM - Conclusions½ The Cleanroom processes cover most of the important aspects of

software development

� as shown by the number of KPAs covered by Cleanroom» 10 of 18 have High correlation» 6 of 18 have Partial correlation» 2 have Consistent correlation (not implemented)

� This suggests that Cleanroom is not a scattered and isolated technique» It focuses on the key processes

½ If a project adopts all Cleanroom processes “faithfully”, then the project has fulfilled most of the KPAs needed for CMM 4

� And has partially satisfied CMM 5 needs

Page 118: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 118118118

Agenda½ Introduction and Overview

½ Basic Principles

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 119: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 119119119

Cleanroom and Object-Oriented Development½ After looking at Cleanroom, an initial observation might be:

1LFH��EXW�FOHDUO\�D�VWUXFWXUHG�DQDO\VLV��IXQFWLRQDO�GHFRPSRVLWLRQ�VW\OH�RI�GHVLJQ��XQIRUWXQDWHO\�RXW�RI�GDWH�FRPSDUHG�WR�D�PRGHUQ�KLJKO\�HIIHFWLYH�REMHFW�RULHQWHG�GHYHORSPHQW�PHWKRG�

½ Philosophy aside, there is more correlation to OO that would appear in the surface

½ The basic concept of a Black Box is similar to that of a class or object

� State data and behavior encapsulated and managed

� Internal data hidden from the outside

� Functions achieved only through defined interfaces (methods)

� Abstract Data Type theory underlies both Cleanroom and OO

Page 120: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 120120120

OO / Cleanroom Similarities½ OO inheritance similar to increment growth

� The concept of incremental development closely parallels OO inheritance

� Refining component behavior can be accomplished via subclassing» Using inheritance to avoid recoding or redesigning

� Necessary to ensure semantic consistency however» So subclassing should avoid redefining existing methods

½ Use of state machines supported in UML

� A consistent view across methods

½ OO use cases and scenarios provide direct support to the creation of Usage Models

½ An extensive study was performed blending OO methods with Cleanroom� http://source.asset.com/stars/loral/cleanroom/oo/guidhome.htm

Page 121: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 121121121

Agenda½ Basic Principles

½ Introduction and Overview

½ History of Cleanroom

½ Experience using Cleanroom (case studies)

½ Foundations of Cleanroom

½ Cleanroom Process Model

½ Cleanroom and the CMM

½ Cleanroom and Object-Oriented Development

½ Conclusion

Page 122: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 122122122

Conclusion½ Cleanroom is an effective and powerful technique that supports the

development of quality and reliable software

½ Any questions……

Page 123: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 123123123

ReferencesCurrit, P.A., M. Dyer, and H.D. Mills, "Certifying the Reliability of Software," IEEE Transactions on Software Engineering, vol. SE-12, no. 1, (January, 1986), pp. 3-11.

Deck, M.D., “Cleanroom and Object-Oriented Software Engineering: A Unique Synergy,” Software Technology Conference, Salt Lake City, UT, April, 1996

Linger, R.C, Carmen J. Trammell, Cleanroom Software Engineering, Reference Model, Version 1.0, November 1996, Technical Report, CMU/SEI-96-TR-022, ESC-TR-96-022

Linger, R.C, Mark C. Paulk, Carmen J. Trammell, Cleanroom Software Engineering Implementation of the Capability Maturity Model (CMM sm) for Software, Technical Report CMU/SEI-96-TR-023, ESC-TR-96-023 December 1996

Linger, R.C. and A.R. Hevner, "The Incremental Development Process in Cleanroom Software Engineering," Proc. Workshop on Inf. Tech. & Systems (WITS-93), A.R. Hevner, ed., Coll. Bus. & Mgmt., U. of Maryland, College Park, December, 1993, pp. 162-171

Linger, R.C. and H.D. Mills, "A Case Study in Cleanroom Software Engineering: The IBM COBOL Structuring Facility," Proc. 12th International Computer Science and Applications Conference, October, 1988.

Linger, R.C., "Cleanroom Process Model," IEEE Software, March, 1994, pp. 50-58.

Linger, R.C., "Cleanroom Process Model," IEEE Software, March, 1994, pp. 50-58.

Linger, Richard C., Mark C. Paulk, and Carmen J. Trammell, “Cleanroom Software Engineering Implementation of the Capability Maturity Model (CMM sm ) for Software”, December 1996, Technical Report CMU/SEI-96-TR-023, ESC-TR-96-023

Mills, H. D., M. Dyer, and R. Linger, “Cleanroom Software Engineering”, IEEE Software, September 1987, pp. 19-25

Mills, H.D., “Stepwise Refinement and Verification in Box-Structured Systems,” IEEE Computer, June, 1988, pp. 23-36.

Mills, H.D., “Stepwise Refinement and Verification in Box-structured Systems.” IEEE Computer, 21(6):23-36, (June 1988).

NASA, “Cleanroom Process Model”, Software Engineering Laboratory Series, SEL-01-004, Nov 1991

Poore, J.H., H.D. Mills, and D. Mutchler, "Planning and Certifying Software System Reliability,” IEEE Software, January, 1993, pp. 88-9

Prowell, S., C. Trammell, R. Linger, and J. Poore, Cleanroom Software Engineering, The SEI Series in Software Engineering, 1999

Selby, R.W., V.R. Basili, and F.T. Baker, "Cleanroom Software Development: An Empirical Evaluation," IEEE Transactions on Software Engineering, vol. SE-13, no. 9, (September, 1987), pp. 1027-1037

Whittaker, J.A., and M.G. Thomason, "A Markov Chain Model for Statistical Software Testing,” IEEE Transactions on Software Engineering, October, 1994, pp. 812-824.

Whittaker, J.A.;A Markov chain model for statistical software testing Thomason, M.G. Software Engineering, IEEE Transactions on , Volume: 20 Issue: 10 , Oct. 1994 Page(s): 812 -824

Page 124: Cleanroom Software Engineering: An Overvie · Cleanroom Software Engineering: An Overview Presented by: ... ‰ Cleanroom Software Development is a set of practices that, ... behavior

SIGAda 2000 - Cleanroom Software Engineering Tutorial - 12 November 2000

November 1, 2000 124124124

WEB Pages on Cleanroom½ http://source.asset.com/stars/loral/cleanroom/tutorial/cleanroom.html

� Cleanroom tutorial developed for The DoD Software Technology for Adaptable, Reliable Systems (STARS) program

½ http://www.rspa.com/spi/cleanroom.html

� Roger Pressman links on Cleanroom

½ http://www.sei.cmu.edu/publications/documents/96.reports/96.tr.022.html

� Cleanroom SW Eng reference model

½ http://www.cleansoft.com/

� Cleanroom Software Engineering Inc. home page

½ http://www.dacs.dtic.mil/databases/url/key.hts?keycode=64

� DACS Cleanroom page

½ http://source.asset.com/stars/loral/cleanroom/oo/guidhome.htm

� Guide to integrating Cleanroom and OO

½ http://www.sei.cmu.edu/str/descriptions/cleanroom.html

� Cleanroom description