Company Confidential
1 © 2007 Nokia BCS FACS 2007/IJO
Experiences of Formal Methods in Conventional Software and Systems Design
Ian Oliver
BCS FACS Workshop
17 Dec 2007
Company Confidential
Nokia Internal Use Only
2 © 2007 Nokia BCS FACS 2007/IJO
Relationship between Formal Methods and Industry
Company Confidential
Nokia Internal Use Only
3 © 2007 Nokia BCS FACS 2007/IJO
Relationship between Formal Methods and Industry
Company Confidential
Nokia Internal Use Only
4 © 2007 Nokia BCS FACS 2007/IJO
Relationship between Formal Methods and Industry
Company Confidential
Nokia Internal Use Only
5 © 2007 Nokia BCS FACS 2007/IJO
Relationship between Formal Methods and Industry
Company Confidential
Nokia Internal Use Only
6 © 2007 Nokia BCS FACS 2007/IJO
Contents
• Typical Situations
• What we want...
• Previous Experiences
• Case Study
• A Unique Scenario
• Results
• Observations
• Conclusions
Company Confidential
Nokia Internal Use Only
7 © 2007 Nokia BCS FACS 2007/IJO
Typical Situations
• Design by Comittee
||
• Rapid generation of code (we like code, we have isolated teams dedicated to design)
• any code, lots of it too!
• more code = better
• SLOC is the best metric we have
• Bonuses awarded based on number of bugs found (extra if you correct them too)
||
• Lots of Design (we like design, we have isolated teams dedicated to design)
• more design = better
• more design = more complexity
• more complexity = better ... apply this principle to UML...
||
• Process (we like process, you get the idea...)
• lots of process, meetings
• CMM level expressed as a complex number: -2+2i (cf: Finkelstein et al)
Company Confidential
Nokia Internal Use Only
8 © 2007 Nokia BCS FACS 2007/IJO
Modelling Tools and Formality
• Word, Powerpoint, • Visio for the lucky ones
• Doors, Rational Rose etc
• UML
• here starts (all?) most of the problems...
• diagramitis
• the more complex the diagram the better....yes?
• behaviour
• idea for new Grand Challenge ... yet another UML statemachine semantics ... the correct one this time please
• current statemachine implementations too close to code ( *==C )
• SDL/C, Java
• Formal Languages, Verification etc
• testing suffices for everything ... really!
• what?
• Hardware people are more familiar with this: eg: PSL, no debug possibilities
Company Confidential
Nokia Internal Use Only
9 © 2007 Nokia BCS FACS 2007/IJO
Typical Situations
• are Fred Brooke’s worst nightmare
• maybe in 15 years...
Company Confidential
Nokia Internal Use Only
10 © 2007 Nokia BCS FACS 2007/IJO
What we want...
• Pragmatism
• ”Pragmatic Refinement”
• Dependability over Correctness/Completeness
• thanks QinetiQ
• modified to.... Assurance over Dependebility over Correctness/Completeness
• thanks PGL (discussion over lunch)
Company Confidential
Nokia Internal Use Only
11 © 2007 Nokia BCS FACS 2007/IJO
Previous Experiences
• GSM Datastructure (B/UMLB)
• DSP Specification (B/Raven)
• Hardware SOA Architecture (B/UML) -> Rodin CS3 / Nokia NoTA Architecture
• Minor uses of Z (Z/EVES), OCL (USE)
• other people: Petri-nets for Symbian O/S behaviour
• Problem:
• often this work is handed over to other ”non-formal” groups to continue/develop
• no complete view of the whole process (cf: NoTA)
• no comparison/measurement work possible
• eg: Agile vs Formal
• lack of metrics
• We do not have any manadated use of formal methods
Company Confidential
Nokia Internal Use Only
12 © 2007 Nokia BCS FACS 2007/IJO
Case Study
• Semantic web computation environment
Company Confidential
Nokia Internal Use Only
13 © 2007 Nokia BCS FACS 2007/IJO
A Unique Scenario
• One set of requirements
• Two teams
• one formal
• one non-formal
• Same goal
• produce a working system
• demonstrate
• Good manager
now describe
Company Confidential
Nokia Internal Use Only
14 © 2007 Nokia BCS FACS 2007/IJO
Results
• Working system produced
• Bugs/Errors
• no changes to the architecture
• bugs found were/are very localised
• often just typos, case errors, = for ==
• change requirements
• often discharged as convenience functions
• Method
• secondary but a natural flow between Requirements->UML->Alloy->Code
• not perfect (too big semantic gap between Alloy->Code)
• Inter/Intra-Team Communication
Company Confidential
Nokia Internal Use Only
15 © 2007 Nokia BCS FACS 2007/IJO
Results
• Regarding the semantic gap...
Requirements
ER/OO
Alloy
Code
Here be dragons...
Company Confidential
Nokia Internal Use Only
16 © 2007 Nokia BCS FACS 2007/IJO
Results
• Regarding the semantic gap...
Requirements
ER/OO
Alloy
Code
details...some details need more analysis (FM forces you to do this...properly)
Company Confidential
Nokia Internal Use Only
17 © 2007 Nokia BCS FACS 2007/IJO
Results
• Regarding the semantic gap...
Requirements
ER/OO
Alloy
Code
Promela/Spin
B/EventB
Communication Protocol specification
Small Scope, specific control structures
No need to specify everything!
Use the right language for the job
Company Confidential
Nokia Internal Use Only
18 © 2007 Nokia BCS FACS 2007/IJO
Observations • The lack of knowledge about abstraction
• The overriding need for implementation (of something) versus the need for background research and thought before implementation
• never underestimate ow much time is saved by two weeks in the lab instead of two hours in the library
• The concentration on small implementation details even at requirements level • aka: coding before thinking
• The concentration of syntactical issues and confusion of terms between divergent domains. • (eg: smartspace vs SmartSpace(TM), session vs connection vs ISO 7-layer denitions
• The usefulness of animation/simulation in understanding the design • manager friendly
• Understanding the completeness of the models. • specify what you need to specify ... general ideas, parts you don’t fully understand
• Tendency to discover the atomic behaviours
• The lack of usefulness of the UML beyond static structure
• Discovery of the properties of the system during specification and not before
Company Confidential
Nokia Internal Use Only
19 © 2007 Nokia BCS FACS 2007/IJO
Conclusions
• More automation required
• tool support is critical
• ”management/Powerpoint friendly” results
• show the animations not the proofs
• hands-on
• Testing support
• No need to prove/demonstrate/verify everything
• pragmatism
• refinement
• No strict methods/processes
• its hard enough getting this stuff accepted without the method/process bits...
• Advantages
• delivered a working system 3 months earlier
• more time to concentrate on developing application for the system
• better demonstrations
• easier to promote and gain acceptance
• finds bugs faster
• COMMUNICATION
• Smaller code size
• smaller memory footprint
• more efficient code
• better power consumption
• Scares managers
• code not produced early
• however, see: early software release