Formal Methods in Conventional Software Development Environments

Company Confidential

1 © 2007 Nokia BCS FACS 2007/IJO

Experiences of Formal Methods in Conventional Software and Systems Design

Ian Oliver

BCS FACS Workshop

17 Dec 2007


Nokia Internal Use Only


Relationship between Formal Methods and Industry
















Contents

• Typical Situations

• What we want...

• Previous Experiences

• Case Study

• A Unique Scenario

• Results

• Observations

• Conclusions




Typical Situations

• Design by Comittee

||

• Rapid generation of code (we like code, we have isolated teams dedicated to design)

• any code, lots of it too!

• more code = better

• SLOC is the best metric we have

• Bonuses awarded based on number of bugs found (extra if you correct them too)

||

• Lots of Design (we like design, we have isolated teams dedicated to design)

• more design = better

• more design = more complexity

• more complexity = better ... apply this principle to UML...

||

• Process (we like process, you get the idea...)

• lots of process, meetings

• CMM level expressed as a complex number: -2+2i (cf: Finkelstein et al)




Modelling Tools and Formality

• Word, Powerpoint, • Visio for the lucky ones

• Doors, Rational Rose etc

• UML

• here starts (all?) most of the problems...

• diagramitis

• the more complex the diagram the better....yes?

• behaviour

• idea for new Grand Challenge ... yet another UML statemachine semantics ... the correct one this time please

• current statemachine implementations too close to code ( *==C )

• SDL/C, Java

• Formal Languages, Verification etc

• testing suffices for everything ... really!

• what?

• Hardware people are more familiar with this: eg: PSL, no debug possibilities




Typical Situations

• are Fred Brooke’s worst nightmare

• maybe in 15 years...




What we want...

• Pragmatism

• ”Pragmatic Refinement”

• Dependability over Correctness/Completeness

• thanks QinetiQ

• modified to.... Assurance over Dependebility over Correctness/Completeness

• thanks PGL (discussion over lunch)




Previous Experiences

• GSM Datastructure (B/UMLB)

• DSP Specification (B/Raven)

• Hardware SOA Architecture (B/UML) -> Rodin CS3 / Nokia NoTA Architecture

• Minor uses of Z (Z/EVES), OCL (USE)

• other people: Petri-nets for Symbian O/S behaviour

• Problem:

• often this work is handed over to other ”non-formal” groups to continue/develop

• no complete view of the whole process (cf: NoTA)

• no comparison/measurement work possible

• eg: Agile vs Formal

• lack of metrics

• We do not have any manadated use of formal methods




Case Study

• Semantic web computation environment




A Unique Scenario

• One set of requirements

• Two teams

• one formal

• one non-formal

• Same goal

• produce a working system

• demonstrate

• Good manager

now describe




Results

• Working system produced

• Bugs/Errors

• no changes to the architecture

• bugs found were/are very localised

• often just typos, case errors, = for ==

• change requirements

• often discharged as convenience functions

• Method

• secondary but a natural flow between Requirements->UML->Alloy->Code

• not perfect (too big semantic gap between Alloy->Code)

• Inter/Intra-Team Communication




Results

• Regarding the semantic gap...

Requirements

ER/OO

Alloy

Code

Here be dragons...




Results


Requirements

ER/OO

Alloy

Code

details...some details need more analysis (FM forces you to do this...properly)




Results


Requirements

ER/OO

Alloy

Code

Promela/Spin

B/EventB

Communication Protocol specification

Small Scope, specific control structures

No need to specify everything!

Use the right language for the job




Observations • The lack of knowledge about abstraction

• The overriding need for implementation (of something) versus the need for background research and thought before implementation

• never underestimate ow much time is saved by two weeks in the lab instead of two hours in the library

• The concentration on small implementation details even at requirements level • aka: coding before thinking

• The concentration of syntactical issues and confusion of terms between divergent domains. • (eg: smartspace vs SmartSpace(TM), session vs connection vs ISO 7-layer denitions

• The usefulness of animation/simulation in understanding the design • manager friendly

• Understanding the completeness of the models. • specify what you need to specify ... general ideas, parts you don’t fully understand

• Tendency to discover the atomic behaviours

• The lack of usefulness of the UML beyond static structure

• Discovery of the properties of the system during specification and not before




Conclusions

• More automation required

• tool support is critical

• ”management/Powerpoint friendly” results

• show the animations not the proofs

• hands-on

• Testing support

• No need to prove/demonstrate/verify everything

• pragmatism

• refinement

• No strict methods/processes

• its hard enough getting this stuff accepted without the method/process bits...

• Advantages

• delivered a working system 3 months earlier

• more time to concentrate on developing application for the system

• better demonstrations

• easier to promote and gain acceptance

• finds bugs faster

• COMMUNICATION

• Smaller code size

• smaller memory footprint

• more efficient code

• better power consumption

• Scares managers

• code not produced early

• however, see: early software release

Art & Photos

Formal Methods in Conventional Software Development Environments