Embed Size (px)
Citation preview
Software Quality and Team Software Quality and Team Development Practices based on Development Practices based on CERN ExperienceCERN Experience
Rostislav Titov,GS-AIS-EB Section Leader,
CERN
CERNAIS
The reality todayThe reality todayFailure 31.1% of IT projects will be canceled before they ever get
completed 52.7% of projects will cost 189% of their original estimate More than 50% of software projects fail today. 2009: highest failure rate in over a decade!
Success Only 16.2% software projects that are completed on time
and on budget In the large companies only 9% of their projects come in on
time and on budget.
Catastrophes– Ariane 5 (7bn dev, 500 million)
numerical conversion error
CERNAIS
Typical FailuresTypical Failures
User requirements not met
Software unreliable
It works great for me, now deploy it for 10,000 users…
Too late (You took so long that our requirements have changed…)
You did what I asked. but I didn’t say what I meant…
CERNAIS
ReasonsReasons Unrealistic Schedules
– Race through the requirements, produce a superficial design and rush into coding.
Changing Requirements During Development“Writing software from requirements is like walking
on water – its easier when frozen”
Inappropriate Staffing Poor-Quality Work
There's a saying about software quality: “If it doesn't have to work, we can build it really fast.”
Believing in magic– Pitfalls of commercial products
Cheap Good
Fast
Choosetwo
CERNAIS
Project Success factorsProject Success factorsProject Success Factors and % of Responses
1 User Involvement 15.90%
2 Executive Management Support 13.90%3 Clear Statement of Requirements 13.00%4 Proper Planning 9.60%
5 Realistic Expectations 8.20%6 Smaller Project Milestones 0 - 7.7%7 Competent Staff 7.20%8 Ownership 5.30%
9 Clear Vision & Objectives 2.90%10 Hardworking, Focused Staff 2.40%
Other 13.90%Project Challenged Factors and % of Responses
1 Lack of User Inputs 12.80%2 Incomplete Requirements & Specifications 12.30%
3 Changing Requirements & Specifications 11.80%4 Lack of Executive Support 7.50%
5 Technology Incompetence 7.00%6 Lack of Resources 6.40%7 Unrealistic Expectations 5.90%8 Unclear Objectives 5.30%
Other 23.00%
CERNAIS
AnalysisAnalysis State Goal
“send a man to the moon before end of the decade & return him safely to Earth”, JF Kennedy
Specify the problem not the solution Classification
M MustoS ShouldC Could0W Would
Concept of Operations document “This is my understanding of what you want”
Beware of requirements ‘gold plating’
Verifiable : use-cases
Projects completed by the largest American companies have only approximately 42% of the originally proposed features and functions.
CERNAIS
Good Design PrinciplesGood Design Principles
Consider alternative approachesNot tunnel vision
Traceable to the requirements Correct & complete
Not reinvent the wheelUse a pattern
Adaptability Accommodate Change
Maximize Cohesion
Minimize Coupling
Understandabilty A design must be understandable if it is to support modification.
“A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away” - Antoine de Saint Exupéry
Mars Polar Lander ($165 million) – design error
CERNAIS
Good & Bad designGood & Bad design
Good Design
Change in one part of the system doesn't always require a change in another part of the system.
Every piece of logic has one and one home.
The logic is near the data it operates on.
System can be extended with changes in only one place.
Simplicity
Bad Design
One conceptual change requires changes to many parts of the system.
Logic has to be duplicated.
Cost of a bad design becomes overwhelming.
Can't remember where all the implicitly linked changes have to take place.
Can't add a new function without breaking an existing function.
Complexity
CERNAIS
CodingCoding
“It doesn’t matter if I write poor code… the compiler will tell me if there is problem”
“It doesn’t matter if I make a mistake… it will come out in testing”
• Mars Climate Orbiter ($125 million) metric conversion error
CERNAIS
BugsBugs
Experienced software engineers inject one defect in about every 10 lines of code.
The programmers aren't incompetent or lazy - they're just human.
All humans make mistakes, but in software, these mistakes result in defects.
This means that a modest-size program of 100,000 lines of code typically would start with about 10,000 defects.
Examples : INTEL Pentium : no more than 80-90 BugsCell Phone (200 000 loc) up to 600 errors. Windows-95: 10 Mill. loc: up to 200 000 errors.
CERNAIS
The Cost of ChangeThe Cost of Change
Cost
CERNAIS
Time
CERNAIS
Code Reviews - guidelinesCode Reviews - guidelines
Form– Product is guilty until proven innocent – Producer is innocent because he/she is not on
trial – More likely to find bugs if you assume they are
there – Evaluate product not producer – Emphasize "review" aspect; do not "fix it
here". – Raise problems. Do not discuss solutions
CERNAIS
Code Reviews - guidelinesCode Reviews - guidelines
Management Involvement– NONE!– Not a manager's status meeting – Management is not represented during
inspections – Inspections must not be used as a tool to
evaluate workers– … Not a committee, not a working group, not a
status report & not an appraisal instrument …
CERNAIS
BenefitsBenefits
Primary objective – remove defects as early as possible in the development
process
Other benefits : – Early Testing – Project Tracking – Educational – share best practices– Training of new/junior programmers– Improved Communication – Improved Individual Quality– Cross-training– Process-improvement– Shared Responsibility – no individual blame
CERNAIS
The “Yes, buts...”The “Yes, buts...”
I don’t have time for this...
Good programmer’s code doesn’t need reviewing
Its only a ‘minor’ piece of code
Code Changes, then what?– 2nd pair eyes rule– Pair programming
CERNAIS
Coding Standards – why?Coding Standards – why?Why ? 80% of the lifetime cost of a piece of software goes to
maintenance. Hardly any software is maintained for its whole life by the
original author. Code conventions improve the readability of the software,
allowing engineers to understand new code more quickly and thoroughly.
Cannot review unless you have standards... endless debate – was driving too fast? Cannot answer
without defined speed limits Recommend best practices, avoid bad practices Maintainable & reliable software is key
Produces Common Code Ownership
CERNAIS
Now imagine you haveNow imagine you have
A multi-domain, multi-lingual horizontal software application
supporting over 16’000 users in 42 countries composed of :
1.5 million Lines of Code
6,000 Java classes
10,000 HTML templates
many other files
Welcome to EDH!
CERNAIS
EDH: Good Old Days (1991-98)EDH: Good Old Days (1991-98)
“Imagination rules the world”
Mac or PC or Unix?C or C++ or ?
University atmosphere
Freedom & Individualism
Choice, choice, choice...
CERNAIS
ResultsResults
Healthy outside, but unhealthy inside
Evolution from freedom to Chaos!Development Platform : Mac, PC & UnixCode : C,C++,Python, Prolog, ProC, PL/SQL, Perl...Comments : Spanish, Italian, French & English...Bugs : “Y2K don’t care”
Obvious code never reviewed : Why would you show your code to anybody? Never did at university... Results count!
Consequence : Maintenance became the primary resource-consuming activity
CERNAIS
From University to IndustryFrom University to Industry
Individual Development Practices Team Development Practices
Freedom of Choice for Development Environment
Uniform development environment
Common set of tools
Choice of language & technology Single technology choice
Free selection of tools
Individual Code Responsibility (& blame)
Common Code Ownership (& learning)
... driven by the members of the team
(not management imposed)
Quality of the Product Quality of the Process
CERNAIS
Coding: ToolsCoding: Tools
Atlassian JIRA
– Issue tracking and project tracking
– EDH: every change must have a JIRA
• Process should be as lightweight as possible
Atlassian GreenHopper
– Agile project management (Scrum)
– EDH: 2-week sprints
Atlassian Confluence
– Documentation (WIKI style)
CERNAIS
Coding: Tools (2)Coding: Tools (2)
Atlassian Crucible
– Online code reviews
– EDH: Every production line of code must be reviewed
Atlassian FishEye
– Browse version control repository (CVS, SVN)
– Real-time notifications of code changes
– Web-based reporting, visualisation and code sharing
Atlassian Bamboo
– Continuous integration
Atlassian Clover
– Java code coverage metrics
CERNAIS
Coding: Tools (3)Coding: Tools (3)
CERNAIS
TestingTestingBugs Standard Software: 25 bugs per 1000 lines of program.
Good Software: 2 errors per 1000 lines. Space Shuttle Software: < 1 errors per 10000 lines.
Example Handy (Cellular Phone): 200 000 lines of program: up to 600 errors.
Windows-95: 10 Mill. lines: up to 200 000 errors.
Sept 24 2004 – Jpeg buffer overrun bug in MS windows
“an attacker who successfully exploited this vulnerability could take complete control of an affected system, including installing programs; viewing, changing, or deleting data; or creating new accounts with full
privileges”
Defects Testing ≠ Debugging
You may have zero bugs, but s/w may not meet requirements, scale, respond-in time…
CERNAIS
TestingTesting Software testing is an art : requires a tester's
creativity, experience and intuition, together with proper techniques.
Most of the testing methods and practices are not very different from 20 years ago.
Testing is more than just debugging.
Testing is expensive. Automation helps.
Complete testing is infeasible. Tradeoff
Testing, while essential, may not be the most effective method to improve software quality.
CERNAIS
What do you test?What do you test?
Correctness testing
Security testing
Reliability testing
Stress testing
Scalability testing
Performance testing
Usability testing
CERNAIS
And after that?And after that?
Software Maintenance 80% of lifetime of software The Legacy Crisis The relative cost for maintaining
software and managing its evolution now represents more than 90% of its total cost.
Example costs– Annual software maintenance cost in USA has
been estimated to be more than $70 billion– Y2K : $8.38 billion US dollar, $90 million for
Nokia 50% of their time is spent in the process of
understanding the code!!!
CERNAIS
Types of MaintenanceTypes of Maintenance
Corrective Maintenance (21%)– A process that includes diagnosis and correction of
errors. Adaptive Maintenance (25%)
– Activity that modifies software to properly interface with a changing environment (hardware and software).
Perfective Maintenance (50%)– Activity for adding new capabilities, modifying existing
functions and making general enhancements. – This accounts for the majority of all effort expended on
maintenance. Preventive Maintenance (4%)
– Activity which changes software to improve future maintainability or reliability or to provide a better basis for future enhancements.
– Still relatively rare.
CERNAIS
Your challenge !Your challenge !
Come in the 9% of projects on time & in budget
Engineer your software (design, review & maintenance in mind)
Control the spiraling IT costs & improve the reputation of the industry
