www.esi-intl.com
The Dollar$ are in the Detail$:
Measuring the Cost of Requirements
Glenn R. Brûlé
Executive Director of Client Solutions, ESI International
[email protected]://www.linkedin.com/in/toglennsnetwork
building talent, driving results 2
Agenda
Count on it…Breaking the Bank SAFE
Breaking the Bank
building talent, driving results
Consistently Challenged
0%
20%
40%
60%
80%
100%
1994 1996 1998 20002002
20042006
2009
Failed
Challenged
Successful
Chaos Reports ~ 2009 – 75% of our Projects are NOT successful!
building talent, driving results
Writing checks we can’t cash?
Why do we have to measure our costs?– Maintain/Grow market share
Are the products meeting customer expectations?
– Quality?– Costs of product to consumers vs.
the cost to build the products– Can we increase the profitability?
– Are we building the right products?– Are we efficient at building the right
products?Right peopleRight processesRight tools?
building talent, driving results
A Balanced Portfolio
Organizational Goals and Objectives
Goods & Services
Business Policies & Rules
Process
Tasks & Activities
6
Cus
tom
er V
alue
Ana
lysi
s
Cor
e C
ompe
tenc
ies
Port
folio
Man
agem
ent
building talent, driving results
So What’s the Point?
7
Organizational Goals and Objectives
Goods & Services
Business Policies & Rules
Process
Tasks & Activities
Increased quantifiable probability of providing the desired deliverables and expectations to meet overall organizational goals and objectives through a clear understanding of our core competencies
Improved goods and services by virtue of standardized methodology/approach and more complete overall work, less iterations by understanding the output from our customer value analysis activities
Increased flexible structure checks and balances that ensure we are compliant with governance, internally or externally
building talent, driving results
So What’s the Point?
8
Better requirements and more closely aligned and prioritized to work being performed
Stronger, well-aligned requirements to alleviate rework down the line
Having the right resources with the right competencies, completing the right tasks that they are capable of is critical to ensure traceability of deliverables to overall goals and objectives.
Finally, any one of these areas are areas for quantifiable consideration
Organizational Goals and Objectives
Goods & Services
Business Policies & Rules
Process
Tasks & Activities
building talent, driving results
Nickels and Dimes…
Organizational goals and objectives– Increase market share by 15% from 7%– Increase revenue by 10%– Decrease the time it takes to respond to a customer
inquiry from 10 minutes to 2 minutesGoods and services
– Increase customer sat by 15 %– Decrease defects in products by 10% from 15%
Business Policies and Rules– Sampling of compliance with rules– Are they prohibitive or restrictive, or to ad-hoc –
could we improve or optimize performance here.
Processes– Increase efficiencies within our own process to
decrease cycle time or number of iterations or the costs of each iteration within an SDLC
Tasks– Are we focused on too many tasks at the
same time– Are the tasks prioritized– Are the tasks related to a real business value– Are the right people performing the right
tasks?
9
building talent, driving results
Requirements Management and Development Assurance Model
10© ESI International, Inc. 2009
building talent, driving results
Top Down Metrics?
11
EnterpriseAnalysis
Business Analysis
Governance
Enterprise ArchitectureSOA
PortfolioManagement
ProgramManagement
ProcessManagement
Project Management
Change Management
ITIL, RUP, AGILE, LEAN, Six Sigma, CMMI
GO
BUSINESS IT
© ESI International, Inc. 2009
SAFE
building talent, driving results
Organizational Starting Point CMMI Model
1. Initial 2. Repeatable 3. Defined 4. Managed 5. Optimizing
Requirements Management and Development practices performed inconsistently
Standard practices and templates
Process is formalized (predictable)
Measurements collected and assessed
Continue process improvement
13
building talent, driving results
CMMI Level 1
CMMI Levels 2-3
CMMI Levels 4-5
Measures for Maturity
14
building talent, driving results
Phase I:Rollout Stage
Phase IIMaturing Capability
Phase IIIMature Continuous Improvement
Measures for Maturity
15
building talent, driving results
An inventory for “measurable”
CMMI Level Framework Environment Resources Business Alignment
Level II Standards and Methodologies
Organization and Structure
Competency and Career Development Enterprise Analysis
Level III Metrics Governance Training and Development Portfolio Management
Level IV Tools Assessment Coaching and Mentoring Program Management
Level V Knowledge ManagementCustomer Relationships
16
building talent, driving results
Rules of the Road
Remember…this is a journey…..
1. Measure to your level of maturity.2. It is best when a baseline can be obtained.
• If not, then just start and use initial findings that as your baseline.• Look at 5-10 projects of a representative sample• Change requests, help desk tickets, iterations, approval processes,
3. Know what your measures mean; i.e. what value the measurement provides to the organization.
4. Consistency is important. - stick with what you can put into place• Even if you learn that the measure needs to be altered; it might be prudent to continue to measure
consistently for a period of time vs. changing it without getting good baseline information.5. Benchmark externally whenever possible.
• What are the industry standards to compete on these levels – this is a true – thinking outside of the box type of concept!
17
Count on it…
building talent, driving results
A Penny for your thoughts…
What are you tracking and measuring? How often?Who sets the standards? How realistic are they?How clear are they?How quickly do you need to report?How often?What variance is acceptable? (+/-)At what level of variance must you take action?What rewards and “penalties” will enforced?
© ESI International, 2006.19
building talent, driving results
Determine an approach that is SMART!
Specific
Measurable
Agreed Upon
Realistic
Timely
© ESI International, 2006.20
building talent, driving results
Measures for Each Level
Phase I: Rollout Stage Level of Organizational Change, i.e. Number of People Trained, Number of Organizations using RMD processes
Number of Requirements Failing an ‘Acid Test’ or other test to determine a ‘good’ requirement.
Phase II / Maturing Capability:
Number of Requirement Defects (Poor or Missing Requirements)Number of Requirements Changes resulting in Project ‘Rework’Customer Satisfaction of Deliverable
Mature / Continuous Process Improvement:
% Requirements/Defects compared to other Projects% Requirements/Change Controls compared to other ProjectsOverall Cycle Time (value of good requirements to satisfactory and speedy
completion of deliverable)Defects in post-production environmentPrediction of future activities accurately Six Sigma – optimization tool
21
Phase I:Rollout Stage
Phase IIMaturing Capability
Phase IIIMature Continuous Improvement
building talent, driving results
Have a “controlled” approach
All methodologies/approaches can leverage metrics
Have a Process of comparing actual performance with planned performance, analyzing variances, evaluating alternatives, and taking corrective action as needed
Concerned with three elements of all aspect of the overall development of deliverables. From GO to tasks and activities.
– Performance– Cost– Time
© ESI International, 2006.22
Ward, Project Management Terms, p. 48Meredith & Mantel, Project Management, p. 508
building talent, driving results
Some Finer Points of Measurement
Consider amount of time/effort spent in each phase of your SDLC
Consider the number of iterations of each phase or overall iterations
Consider the number of change requests over the course of your SDLC
Consider the amount of “rework in each phase
23
Business Architecture
Elicitation
Analysis
Design
Development
Verification & Validation
building talent, driving results
Cost of Different Types Requirements
Total Cost of Requirements + Total Cost of Effort = Total Cost of Solution
24
building talent, driving results
Summary:
Consider the depth and breadth of “measurables” from the “top down”Match your approach to measurements according to the level of overall requirements maturityBe SMART about your approachThis is a journey…metrics provide the roadmap to know when you’ve achieved your goal!
25
www.esi-intl.com
Questions?
Glenn R. Brûlé
Executive Director of Client Solutions, ESI International
[email protected]://www.linkedin.com/in/toglennsnetwork