Cleanroom Software Engineering

Casey EhlersApril 28th, 2011

Outline of Presentation

1. Background and History of Cleanroom2. Who Uses Cleanroom Software

Development?3. Basics of Cleanroom Software Process4. Analysis of the Cleanroom Process

What is a Cleanroom?

CleanroomNearly pollutant free environment used for

scientific research and manufacturing. Some statistics on air pollutants:

Pollutants ( .5 micrometers or larger) in environment:

35,000,000 particles per cubic meter (urban)Pollutants (.5 micrometers or larger) in

Cleanroom: 0 particles per cubic meter

Pollutants (.3 micrometers or smaller) in Cleanroom:

No more than 12 particles per cubic meter

What is the goal of Cleanrooms?

Prevention!Errors and defects are not allowed into the

system during it’s initial design and construction

Dr. Harland MillsIBM – mid ’80’sTook the goals of a hardware Cleanroom and

applied them to the software design process.Mills wanted to prevent the entry of errors

into software instead of just detecting them after they are designed.

Published a paper in 87 on his new methodology and called it “Cleanroom Software Engineering”

Who Uses Cleanroom S.E.?Mission and Life-Critical Systems

NASA and the Military are a few of the organizations who have reported using the Cleanroom process.

Companies who are willing to trade-off efficiency for certifiable reliability. I will re-examine this comment later.

Limited use otherwise.Too rigorous in a lot of cases.

Cleanroom In A Nutshell

Cleanroom is an adaptation of the Incremental model.

Increments are defined using formal methods which can be checked for correctness.

Correctness Verification: Does the system do what it is designed to do?

How the increment is designed and verified is what makes Cleanroom different from normal

Incremental Software Development

Validateincrement

Develop systemincrement

Design systemarchitecture

Integrateincrement

Validatesystem

Define outline requirements

Assign requirements to increments

System incomplete

Finalsystem

Cleanroom Software Development

Defining IncrementsFormal methods are used to rigorously define

what the increment are suppose to do.From there, it is easy to verify that the

specifications meet the designed result.Additional tests can be ran on the increments

to produce “statistical quality control.”The results can be shown to the customer for

approval or reworking.

Box StructurePrograms regarded as rules or mathematical

functionsMaps specify how transformations take place

from inputs to outputsAllows for easy verification.Scalable

You can have a box structure for a whole software system, a specific class, or even a specific function or statement

3 Generally used box structures for Cleanroom

Box Structure - Black

Input, and history needed to for transition to occur.

Transition function:((current stimulus, current history)

response )Example of Black Box:

Calculator Function Identical input could yield different responses based

on current history of the system.

Box Structure - State

History and current state of system needed to be transitioned into a response and a new state.

Transition Function:((current stimulus, current state)

(response, new state))

Box Structure - Clear

Derived from State BoxSame transition function, but each transition

is specified by a procedure.Transition Function:

((current stimulus, current state) (response, new state)) by

procedure

Box Structure Applied to OO

Assume any Object-Oriented systemDecomposition strategy of Box Structures

when designing increments:Black Box- specifies behavior of an objectState Box – specifies data encapsulationClear Box – procedures and methods

Design Example

Page 66 of DyerA program that determines the type of

triangle based on inputs3 inputs4 outcomes: equilateral, scalene, isosceles, not

a triangle Box terminology not used, but design is close

to a clear-box

Correctness Verification

Basic Steps to Software Verification:1. Specify the function (or increment)2. Select the design construct (pseudo-code)3. Use a correctness proof to show equivalence4. Decide next step based on results:

Rework/Reiterate the increment Proceed to next function or increment

Correctness ProofWhat questions do we ask to proof that a

function is correct?Step through the increment and prove

correctness of each construct (if, for, while, etc)

Page 88, Dyer

How can we verify the correctness of the triangle solver?Page 93-95, Dyer

Testing an Increment

Results of correctness proofs offer statistical certainty that the system is doing what it was designed to do.

Results can be show percent correct and percent covered.

These results are really easy to report to a customer for feedback

Page 137, Dyer

Additional Usage TestingPath-Based Testing

Structured Path Testing limits unnecessary based by used a specified set of inputs.

Boundary and loop Testing

Linear Code Sequence/Jump TestingSimilar to using a debugger.

Statements are broken down into sequences which then can be stepped through and tested for defects.

Evaluation of Cleanroom

Rigorous!Look at the example of the triangle solver.Imagine using this process one your senior

project…

EffectiveStatistically proven to increase software

reliability

“Pro’s” of Cleanroom

ReliabilityCurrent reports claim that Cleanroom

increases quality of software around 10-11 times.

Greater Quality Higher profit marginsHappier usersOne study reports:

90 percent of defects were found BEFORE testing phase of incremental development (1,000 – 50,000 L.o.C. systems)

“Pro’s” continued…Efficiency???How can a process that is so intense and

rigorous yield better efficiency?Less ambiguity on designLess time spent testing and re-engineering

We know that testing is one of the most time consuming and expensive parts of software development.

With fewer defects entering the testing phase, less time is spent in this phase.

NASA reported a 4.6 productivity increase

“Con’s” of CleanroomUnnecessary a lot of the times.

Traditional methods of software develop are a lot of the times sufficient or more logical.

TrainingEveryone of the development team needs to be

formally trained in the Cleanroom method. What if the design is wrong?

Cleanroom seems to imply that the formal and usage specifications are designed correctly…

Summary

Cleanroom: Rigorous and thorough modification of the incremental software process.

Offers certifiable reliable software with minimum defects

Requires training and unnecessary in the majority of design scenarios

Resources  [1] Michael Dyer. (1992) The Cleanroom Approach To Quality Software

Development New York, NY: John Wiley & sons  [2] Richard C. Linger and Carmen J. Trimmell (1996) Cleanroom Software

Engineering Reference ModelVersion 1.0 Pittsburgh, PN <http://www.sei.cmu.edu/reports/96tr022.pdf>

  [3] Roger S. Pressmen (1992) Software Engineering: A Practitioner’s Approach

(3rd Edition) New York, NY: McGraw-Hill Inc  [4] Shawn P. Garbett (2003) Cleanroom Software Engineering Retrieved from

http://drdobbs.com/architecture-and-design/184405405  [5] Chaelynne M. Wolak (2001) Taking the Art out of Software Development: An

In Depth Review of Cleanroom Software Engineering Retrieved From http://www.scisstudyguides.addr.com/papers/cwdiss725paper1.htm