Integrating Quality into Portfolio Management

PMI Silicon Valley ChapterJan 17, 2011Brent Barton

1

© 2009-2011,

Brent Barton - Sterling Barton, LLC

Partner: Sterling Barton, LLC and President: AgileEVM, Inc.

Former CTO, Development Manager, PMO Manager, Agile Coach, Mentor, Certified Scrum Trainer, ScrumMaster, Product Owner

Active practitioner delivering value using Agile and helping others do it; from small Product companies to very large IT organizations

Agile Articles“Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process”, Cutter Journal 2010

“AgileEVM – Earned Value Management in Scrum Projects”, IEEE 2006

“Implementing a Professional Services Organization Using Type C Scrum”, IEEE

“Establishing and Maintaining Top to Bottom Transparency Using the Meta-Scrum”, AgileJournal

“All-Out Organizational Scrum as an Innovation Value Chain”, IEEE

Email: brent@sterlingbarton.combrent@agileevm.com

Web: www.agileevm.comwww.sterlingbarton.comBlog: gettingagile.com

Follow me on Twitter: brentbarton

2

© 2009-2011,

Two Important Themes for Tonight

Asserting Quality

Leveraging Adaptive Planning

➡Opportunity from Agile

➡Consequence of Agile

3

© 2009-2011,

Complexity Requires Adaptive Planning

It is not possible to completely specify an interactive system.

Wegner’s Lemma, 1995

Uncertainty is inherent and inevitable in software development processes and products.

Ziv’s Uncertainty Principle, 1996

For a new software system the requirements will not be completely known until after the users have used it.

Humphrey’s Requirements Uncertainty Principle, c.1998

4

© 2009-2011,

Meet Earl - Strategic Planner

Earl just finished the annual portfolio budgeting process for the new fiscal year!

5

© 2009-2011,

Meet Geoff -Project Manager

Geoff was a Software Developer

and is now in charge of Saturn

Saturn is a key part of a company-wide strategy

6

© 2009-2011,

Portfolio is Done for the year! (This month is the beginning of our new Fiscal Year)

7

© 2009-2011,

Three Months Later...

8

© 2009-2011,

3 months later, things aren’t looking as good

9

© 2009-2011,

So...what is happening? Who is affected?

10

© 2009-2011,

Another Month Later...

11

© 2009-2011,

Defect Containment is helping...not solving

12

© 2009-2011,

Continuous Integration is “My Friend”

13

© 2009-2011,

Meet Huang -Test Engineer

Huang is a Software Development Engineer in Test - SDET

Huang wants to proud of the quality of every release

How can I help?

14

© 2009-2011, 15

© 2009-2010,

Meet Sonia - Program Manager

Sonia is a Program Manager

Uses Scrum well...good servant leader

How can I help ensure better

Quality?

© 2009-2010,

Definition of Done

Defines the work products that will be delivered with each item as it is ready for acceptance

Typical entries in Definition of Done

Code includes unit tests, reviewed, checked in

Tests described and executed

Build, release notes

Compliance documentation updated to include current functionality

What else?17

© 2009-2010,

Definition of Done as a Compliance Checklist

Acceptance defined criteria for each user story

Unit tests written and passed

Code compiles with no errors and no warnings

New code doesn’t break existing code

Test case review (Dev to review test case written)

Architectural impact assessed and artifacts updated if necessary

Comments in code

Error codes added

Code reviewed by peer

Code checked in with reference to US#/Task#

Tested on FE

Integration test written & passes

Test code reviewed

Environment requirements documented

Interface document updated/added and checked in to SVN

Acceptance criteria verified complete

All P1-P3 bugs for the story are closed

Test approves user story

Story demonstrated to product owner and accepted on Target Platform

© 2009-2010,

How does a “Release Definition of Done” help?

Every release should have clear quality criteria

With a “Release Definition of Done” you can understand targets better

Measure the gap between the teams’ Definition of Done and a Release Definition of Done.

This gap is a source of quality issues and represents significant risk to schedule

© 2009-2011,

[Pause for enlightening discussion]

20

© 2009-2011,

Two Important Themes for Tonight

Asserting Quality

Leveraging Adaptive Planning

21

© 2009-2011,

Meet Earl - Strategic Planner

How do I balance Value and Quality?

22

© 2009-2011,

Balancing Signal Indicators - (some weaker than others)

Value

Quality Constraints(Schedule, Cost, Scope)

Source: Jim Highsmith 23

© 2009-2011,

Project Portfolio Defined

source: PMI The Standard for Portfolio Management — Second Edition

24

A portfolio is a collection of projects or programs and other work that are

grouped together to facilitate effective management of that work to meet

strategic business objectives.

© 2009-2011,

We want to measureoutcomes, not outputs

YES: Business Value

not so much: Completed Projects

Project Portfolio Management

Time

BusinessValue

25

© 2009-2011,

Effective Project Portfolio Management Requires:

Prioritization to maximize Business Value

Effective delivery to minimize costs

Re-allocation of resources when costs are too high or the benefit is too low

source: Cutter Journal 26

Agile partially supports Portfolio Management• Prioritize to maximize

Business Value

• Effectively deliver to minimize costs

• Re-allocate resources when costs are too high or the benefit is too low

Agile

Agile

Earned Value Management (EVM)

Proj

ecte

d Sl

ippa

ge

Management Reserve

Earned Value

(PV)

Total Allocated Budget

TimeNow

CompletionDate

$PMB

EAC

Time

Over Budget

Planned Value

(AC)Actual Cost

Project Management Baseline

Estimate at Complete

(EV)Earned Value

EVM Performance Indicators

CPI < 1 CPI = 1 CPI > 1

Over Budget On Budget Under Budget

SPI < 1 SPI = 1 SPI > 1

Behind Schedule On Schedule Ahead of Schedule

Cost Performance Index (CPI=EV/AC)

Schedule Performance Index (SPI=EV/PV)

Strengths of EVM

• Integrates cost and schedule management

• Forecasts in financial units based on units used for actual cost

• Decades of use

• Part of PMBOK (ANSI/PMI 99-001-2008)

• Part of EVMS (ANSI/EIA-748-B-2007)

Weaknesses of Traditional EVM

• Expects everythingfully defined up front

• No assertion of quality

• Claiming value is earned on intermediate work products

Ugh!

Agile + EVM• We want to measure

outcomes, not outputs

• Prioritize to maximize Business Value

• Effectively deliver to minimize costs

• Re-allocate resources when costs are too high or the benefit is too low

Agile

Agile

EVM

Agile EVM+

[Pause for Reflection]

AgileEVM

AgileEVM Background

• Mathematically proven that Release Dates based on average velocity (story points)≡ estimate at complete (dollars)

• Key Assumption: The ratio of (story points completed)/(total story points in a release) is a good measure of Actual Percent Complete

Scenario 1: Commit

37

38

Scenario 2: Transform

39

40

41

42

Quality Criteria asserted by team

43

Scenario 3: Kill

44

45

Saturn RC-1 Forecasts

Mercury Forecasts

What are some options?

46

© 2009-2011,

Questions?

47

© 2009-2011,

Brent Barton - Sterling Barton, LLC

Partner: Sterling Barton, LLC and President: AgileEVM, Inc.

Former CTO, Development Manager, PMO Manager, Agile Coach, Mentor, Certified Scrum Trainer, ScrumMaster, Product Owner

Active practitioner delivering value using Agile and helping others do it; from small Product companies to very large IT organizations

Agile Articles“Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process”, Cutter Journal 2010

“AgileEVM – Earned Value Management in Scrum Projects”, IEEE 2006

“Implementing a Professional Services Organization Using Type C Scrum”, IEEE

“Establishing and Maintaining Top to Bottom Transparency Using the Meta-Scrum”, AgileJournal

“All-Out Organizational Scrum as an Innovation Value Chain”, IEEE

Email: brent@sterlingbarton.combrent@agileevm.com

Web: www.agileevm.comwww.sterlingbarton.comBlog: gettingagile.com

Follow me on Twitter: brentbarton

48