INF5181 / Lecture 08 / © Bente Anda 2011, Slide 1
INF5181: Process Improvement and Agile Methods in Systems Development
Lecture 08, 3.11.2011:
Software process improvement in practice:
Estimating software development effort
Bente Anda
Senior advisor, Norwegian Tax AdministrationAdjunct associate professor, IFI
email: [email protected] 2011
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 2
Software process improvement (SPI)
• A process is a series of steps, that is, what we do, to produce a result
• A software process is the set of activities, methods, and practices used in the development and maintenance of software
– This lecture focuses on one sub-process, the estimation process
• Software process improvement is change of some kind to the way software development and maintenance is conducted
• Organizations do software process improvement to reduce defects, improve productivity, lower costs…
• But, changing work habits is difficult, and SPI frequently fails…
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 3
• Appr. 320 people working on software development as well as a large number of consultants
• Appr. 12 ongoing software development projects• Several hundred change requests to existing systems each
year• Methods:
Software development at Skatteetaten
Smidig / Scrum
UP/UML
SMART
Project management Software development
PMBOK
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 4
Effort estimation
• Definition of estimate: Most likely effort to accomplish a software development task
• Estimates of software development effort are necessary to manage scope, effort and resources Scope
Time Resources
• A challenge of defining an estimation process and methodology to improve effort estimation is that estimates often contain an element of magic
X?
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 5
Effort estimation dilemma
• An effort estimate is often produced early when there is a large uncertainty with respect to the software product and software development project
• An estimate may have a large impact on our priorities (and consequently on what we deliver) and on how well we succeed in delivering it
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 6
The role of effort estimates
Process for estimationProcess for estimation
Estimates Estimates
Management (prioritization and descision) Management (prioritization and descision)
Project
Predictability resources
Predictability resources
Predictability delivery
Predictability delivery
Predictability scope
Predictability scope
scope
time resources
Portefolio
strategy/goal
time resources
Control strategy/goal
Control strategy/goal
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 7
Motivation for software process improvement within effort estimation
• Not all software development is estimated
• The estimates are not always used in prioritization, resource allocation and planning
• Estimates often have a very large risk buffer
• Little documentation of prerequisites and assumptions for estimates
:
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 8
Motivation cont.
• Too high estimates mean – unnecessary costs since estimates of time and resources
tend to be fulfilled, and– productivity is too low
• Too low estimates mean – other projects/change requests that would have given better
cost-benefit are not prioritized, and – additional costs when consultants must be hired to “save”
projects• Given the size of the organization, consequences are high
Stages of SPI – the IDEAL model
IDEAL SEI`s approach to software process improvement
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 10
Keys to successful SPI 1. Define and communicate the problem to be solved through the SPI
initiative throughout the organization
2. Align the SPI initiative with business strategy and goals
3. Be sensitive to the organizational context
4. Gain commitment from key stakeholders
5. Dedicate resources to manage the implementation and coordinate the process improvement activities
6. Invest in the necessary tools/infrastructure
7. Document key metrics before and after the process change
8. Publish evidence of SPI success
9. Learn from the feedback results
10.Be aware that SPI is change management
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 11
Challenges of SPI
Technological challenges:• Many methods, techniques and tools for effort estimation to
choose from
Organizational challenges:• Many stakeholders• Complex related processes that frequently change• How to measure improvement
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 12
Challenges of SPI - Technological
• Very diverse tasks need to be estimated
1. Types of task: Small change requests, user stories… large projects
2. Requirements may be known or uncertain
3. Development platform may be familiar or unknown
4. Resources may be known or uncertain
…
• Technology is complex and a moving target, and there are many estimation techniques:
1. Estimation by analogy2. Parametric estimation3. Use case based estimation4. Wideband Delphi5. Successive estimating6. Bottom-up estimation based
on a work breakdown structure
7. Planning poker…
Which technique should be used when?
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 13
Challenges of SPI – Organizational
• Skills – ”I don`t know these techniques, so how can we use
them?”• The nature of effort estimation
– “An effort estimate is nothing more than a guess anyway”, and
– “A number can be guessed very easily without any technique, so why should I bother to use a technique”
• Measurement –– “We will not reveal bad practices”
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 14
Stages of this SPI project
Training
Baseline
Development of methodologyBest practice
Goals
Sponsorship
Motivation
Iteration 1
Pilot 1
Pilot 2
Pilot 3
Iteration 2 Iteration 3 Iteration 4
Measurement
Infrastructure
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 15
Stages of SPI project - Initiating
Stimulus for improvement: Gartner report
Set goals for the improvement
Establish sponsorship
Establish improvement infrastructure
Goals
Sponsorship
Motivation
Infrastructure
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 16
Qu
anti
fiab
le
Financial Non-financial
No
n-
qu
anti
fiab
leSet goals
• Better investments caused by better prioritization
• Better use of resources
• Better possibilities for assessing offers from external suppliers
• Less need to hire additional resources to ”save” projects
• Less effort required to produce estimates
• Estimates of higher quality
• Less need for re-planning caused by budget overruns
• More satisfied employees
• Better reputation in the Department of Finance
• Improved predictability with respect to project completions
• Improved understanding of uncertainty in estimates and of risk
• Improved project management through – Improved understanding of cost drivers– Routines for collecting effort data
throughout the projects
• Documented practice that enables continuous improvement
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 17
Establish sponsorship • The SPI project was part of large project to renew IT function
– Goals
• Reference group with:– Member of IT staff– Project director– Manager of architecture team– Senior manager
Improved strategic management and more efficient deliveries
of IT services
More efficient maintenance and evolution, right quality
on service deliveries
Increased capacity for development
Simplified and renewed portfolio of systems
Free resources Realize
Reduced costs Reduced development time
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 18
Establish improvement infrastructure
• Responsibles for the SPI initiative:
– one senior project manager
– one expert on SPI and effort estimation
• Wiki
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 19
Stages of SPI project - Diagnosing
Establish a baseline of current practice for effort estimation
Develop recommendations based on best practice
Baseline
Best practice
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 20
Techniques used in SPI
• Interviews– Interviews are used to access to tacit knowledge– We used interviews as a primary means to gather a baseline and information
about best practice of effort estimation in Skatteetaten and in comparable organizations
• Workshops– Workshops are conducted to do some work and create a product– We used a workshop to
• describe best practice from the interviews and the collected material• make the first version of the estimation methodology
• Literature reviews– Literature reviews are conducted to establish evidence-based best practice– We used a literature review to identify relevant best practice
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 21
Interviews
• Interviewees: Project and group managers with extensive experience from estimating software development effort (internal and from a number of external organizations)
• Duration: 1.5 hours• Semi-structured interviews
• Interview guide:1. What are the estimation units?
2. What is input to estimating?
3. Who are responsible for estimating and who participates?
4. Which estimation techniques and tools do you use?
5. Do you have historical effort data?
6. How are earlier experiences with effort estimation used?
7. When do you estimate and re-estimate?
8. How do you document the estimates?
9. Positive and negative experiences with existing estimation practice?
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 22
Workshops
• Participants: – Facilitator, documenter (SPI team), IT staff, project manager, 2
managers, 2 developers (all very experienced)– Participants should be chosen according to their influence in
the organization, their skills and interest in the topic, and to what extent they will be impacted by the outcome of the workshop
– Duration: 5 hours• Preparation:
– Informed about focus of workshop and expected results
– Distributed all material from interviews – Room: Walls for notes
Best practice for effort estimation at Skatteetaten-excerpt
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 24
Literature review
• Identify, evaluate and put together
relevant studies to one whole
• Protocol for literature review
1. Select relevant databases for search
(www.scholar.google.com)
2. Search for studies
(Best practice software effort estimation)
3. Determine their quality
4. Put together the relevant information from the studies of sufficiently high quality
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 25 25
General best practice for effort estimation1. Understand the functional size of the software task to be estimated
(the most important cost driver)
2. Understand the uncertainty in product and project and include this in the estimate
3. Use group estimation
4. Combine estimates
1. produced by different people
2. and/or by using different technique
and look at where they are different and where they converge
5. Check that the estimate takes all requirements (also the non functional and implicit requirements) in to account and includes all necessary activities
6. Separate estimate from expectations regarding resources
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 26
Stages of SPI project - Acting
Development of the methodology in four iterations
Testing of the methodology in pilot projects
Establish measurement scheme
Development
Testing
Measurement
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 27
Ideas Prioritization Project Realizing benefts
Early estimatesEarly estimates Planning estimatesPlanning estimates Re-estimationRe-estimation
Forslags-fase
Analyse-fase
Evaluerings-fase
Forstudie-fase
B1 - KvalifiseringB2 - PrioriteringB3 - Vedtak
B3
U1
B2B1
U2
Development
Different estimation processes at different stages
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 28
Early estimatesEarly estimates Planning estimatesPlanning estimates
Task estimatesTask estimates
Re-estimationRe-estimation
Development cont.
Small tasks with low uncertaintyEstimation techniques:• Planning poker• Checklist based
Early project suggestionsEstimation techniques:• Wideband delphi• Successive estimating• Use case based estimation
Requirements specification availableDevelopment platform and resources knownEstimation technique: • Bottom-up based on work breakdown structure
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 29
ExperienceDB
Estimation template
Historical effort data
Analyze project
Adapt work breakdownstructure
Choose estimation process and techniques
Log estimation data
Choose project methodology and
software development methodology
WBS
Development
Configuration of estimation processes
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 30
Design of Wiki (v0.1)
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 31
Piloting
1. Workshops where Planning Poker was used to identify important cost drivers for early estimation
2. Adapted templates for early estimation
3. Tested techniques and templates for effort estimation and re-estimation, particular focus on Wideband delphi
4. Tested routines for collecting historical effort data, including use of estimation units according to RUP
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 32
Examples of tasks to be estimated with PP to identify cost drivers
• Task 1: Establish connection between single owner company and owner in a specific time period
• Task 2: Make new star for MVA to access historical transactions
• Task 3: Make it possible to access 3 tables from a known source in the control mart.
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 33
Measurement
•
Goal Metric
1. Estimates of higher quality Actual effort – estimate before and after estimation method is introduced
2. Less effort required to produce estimates Effort spent on estimation
3. Improved understanding of uncertainty and risk Impression
4. Better use of resources Less need to hire additional resources to ”save”
projectsNOK on unplanned external resources
5. Better assessments of external offers Impression
6. Improved predictability with respect to project completion
Less need for re-planning caused by budget overruns
Actual delivery date – planned delivery# Cases of replanning
7. Better reputation in the Department of Finance Impression
INF5181 / Lecture 08 / © Bente Anda 2011, Slide 34
Summary• Effort estimation is a starting point of software development and effort estimates are vital in prioritizing and planning• There are many techniques for effort estimation• SPI is often difficult as there are both technological and organizational challenges
– Technical understanding, – alignment with business goals, – understanding of the relevant organizational processes, and– commitment and involvement from stakeholders is important
• Models and frameworks for SPI may serve as tools for planning your own SPI initiatives• To learn more about effort estimation: