Balancing Agility and Discipline (Part 2)
CS 566 – Software Management and Economics
Lecture 10 (Boehm and Turner Tutorial 2004;
Chapters 4 – 6, Boehm and Turner Book 2004)
Ali Afzal Malik
LUMS 2
Two Case Studies
• Show how agile and plan-driven methods can be extended
• Provide examples of success and lessons learned
• ThoughtWorks Lease Management– Agile extended with plan-driven
• CCPDS-R– Plan-driven overlaid with agile concepts
LUMS 3
Thoughtworks Lease Management
• XP replaced ineffective traditional development
• Problems when project moved beyond XP assumptions– The effort to develop or modify a story really does not
increase with time and story number– Trusting people to get everything done on time is compatible
with fixed schedules and diseconomies of scale– Simple design and YAGNI scale up easily to large projects
• Disciplined practices enabled XP to scale up– High-level architectural plans to provide essential big-picture
information– More careful definition of milestone completion criteria to
avoid “finishing” but not finishing– Use of design patterns and architectural solutions rather than
simple design to handle foreseeable change
LUMS 4
CCPDS-R
• USAF Command Center Processing and Display System Replacement for early missile warning
• Government contract environment required plan-driven approach, project needed agility
• Practices implemented to provide agility mapped well to the Agile Manifesto – Individuals and Interactions over Processes and Tools
• Milestone content was redefined • Architecture was organized around the performers’ skill levels
– Working Software over Comprehensive Documentation• Later PDR demonstrated working software for high-risk areas
– Customer Collaboration Over Contract Negotiation • Used COCOMO for cost and schedule negotiations
– Responding to Change Over Following a Plan • Automated plans• Architecture for re-use saved effort
LUMS 5
CCPDS-R (2) Cost of Change
RATIONALS o f t w a r e C o r p o r a t I o n
Project Development Schedule15 20 25 30 35 40
40
30
20
10
Design Changes
Implementation Changes
MaintenanceChanges and ECP’s
HoursChange
RATIONALS o f t w a r e C o r p o r a t I o n
Project Development Schedule15 20 25 30 35 40
40
30
20
10
Design Changes
Implementation Changes
MaintenanceChanges and ECP’s
HoursChangeHours
Change
Cos
t of C
hang
e
Time
Beck
LUMS 6
Five Critical Decision Factors
• Size, Criticality, Dynamism, Personnel, Culture
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Agile
Plan-driven
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Agile
Plan-driven
LUMS 9
Outline
• Tutorial learning objectives– Survey of assumptions about participants
• General software trends and implications– Sources of perplexity about agile, plan-driven methods
• Overview of agile and plan-driven methods– XP, TSP “days in the life”– Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline– Small, medium, large examples
• Observations and Way Forward
• Conclusions; review of learning objectives
LUMS 10
Using Risk to Balance
Discipline and Agility - Overview
Step 5. Execute and Monitor
Step 4. Tailor Life Cycle
Step 3. Architecture Analysis
Step 1. Risk Analysis
Step 2. Risk Comparison
Rate the project’s environmental, agility-
oriented and plan-driven risks.
Uncertain about
ratings?
Buy information via prototyping, data
collection and analysis
Compare the agile and Plan-
driven risks
Go Risk-based Agile
Agility risks dominate
Plan-driven risks dominate
Architect application to encapsulate agile parts
Go Risk-based Agile in agile
parts; Go Risk-based Plan-
driven elsewhere
Yes
No
Go Risk-based Plan-driven
Tailor life cycle process around risk patterns
and anchor point commitment milestones
Monitor progress and risks/opportunities,
readjust balance and process as appropriate
Neither dominate
Deliver incremental capabilities according to
strategyNote: Feedback loops present, but omitted for
simplicity
Step 5. Execute and Monitor
Step 4. Tailor Life Cycle
Step 3. Architecture Analysis
Step 1. Risk Analysis
Step 2. Risk Comparison
Rate the project’s environmental, agility-
oriented and plan-driven risks.
Uncertain about
ratings?
Buy information via prototyping, data
collection and analysis
Compare the agile and Plan-
driven risks
Go Risk-based Agile
Agility risks dominate
Plan-driven risks dominate
Architect application to encapsulate agile parts
Go Risk-based Agile in agile
parts; Go Risk-based Plan-
driven elsewhere
Yes
No
Go Risk-based Plan-driven
Tailor life cycle process around risk patterns
and anchor point commitment milestones
Monitor progress and risks/opportunities,
readjust balance and process as appropriate
Neither dominate
Deliver incremental capabilities according to
strategyNote: Feedback loops present, but omitted for
simplicity
LUMS 11
Risks Used in Risk Comparison
• Environmental risks– Technology uncertainties– Many stakeholders– Complex system-of-systems– Legacy compatibility
• Agility risks– Scalability– Use of simple design– Personnel turnover– Too-frequent releases– Not enough agile-capable people
• Plan-driven risks– Rapid change– Need for rapid results– Emergent requirements– Not enough discipline-capable people
LUMS 12
Three Examples
• Agent-based systems
• Small – Event Managers application
• Medium – SupplyChain.com application
• Large – National Information System for Crisis Management (NISCM) application
• Each example results in a development strategy based on risk analyses
LUMS 13
Event Managers Project
• Small startup company
• Diverse set of smaller agent-based planning applications
• This project: support for managing the infrastructure and operations of conferences and conventions
• Widening variety of options and interdependencies, makes an agent-based approach highly attractive
LUMS 14
Event Managers Profile
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Risk Items Risk Ratings
Event Managers
Environmental Risks
E1. Technology uncertainties �
E2. Many stakeholders �
E3. Complex system of systems �
Risks of using Agile Methods
A1. Scalability �
A2. Use of simple design �
A3. Personnel turnover ��
A4. Too-frequent releases �
A5. Not enough agile-capable people �
Risks of using disciplined methods
D1. Rapid change ����
D2. Need for rapid results ����
D3. Emergent requirements ����
D4. Not enough discipline-capable people
�
Risk rating scale: � - minimal risk � - moderate risk
�� - Serious but manageable risk ��� - Very serious but manageable risk ���� - Show-stopper risk
LUMS 15
Event Managers Strategy
1. StartupTeambuilding and Shared
Vision
2. Design, Development and Deployment
• Establish CRACK joint management team
• Develop shared vision, results chain, life cycle
strategy, infrastructure
choices
• Establish commitment to proceed, incremental
capabilities, backlog
• Develop next-increment
features
• Test, exercise, deploy new increment• Re-prioritize next-increment features
• Analyze lessons learned; prepare strategy for next increment
Customer Management; Representative Users
Joint Developer-Customer Agile Team
LCA
LUMS 16
SupplyChain.com Profile
• Turnkey agent-based supply chain management systems
• Distributed, multi-organization teams of about 50 people
• Parts of applications are relatively stable, while others are highly volatile
• Architectures are driven by a few key COTS packages that are also continually evolving
LUMS 17
SupplyChain.com Profile
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Risk Items Risk Ratings
SupplyChain.com
Environmental Risks
E1. Technology uncertainties ��
E2. Many stakeholders �
E3. Complex system of systems �
Risks of using Agile Methods
A1. Scalability ��
A2. Use of simple design ��
A3. Personnel turnover �
A4. Too-frequent releases �
A5. Not enough agile-capable people �
Risks of using disciplined methods
D1. Rapid change ��
D2. Need for rapid results ��
D3. Emergent requirements ��
D4. Not enough discipline-capable people
�
Risk rating scale: � - minimal risk � - moderate risk
�� - Serious but manageable risk ��� - Very serious but manageable risk ���� - Show-stopper risk
LUMS 18
SupplyChain.com Strategy
Startup 1. Teambuilding and Shared
Vision
3. Design, Development and Deployment2. Systems Definition and Architecting
Stakeholders
Furnish CRACK representatives
and alternates
Staff and organize to cover
all success-critical risk areas
• Develop benefits-realization results
chains
• Identify missing
stakeholders• Enhance mutual
understanding
• Explore goals, options, technology,
prototypes, risks
Review; Commit
to
proceed
• Elaborate supply
chain operational concept, prototypes,
transition strategy• Evaluate and determine COTS,
reuse, and architecture choices
• Prioritize and
sequence desired capabilities,
outstanding risks
• Establish project organization, overall
process, and feature
teams
Review; Commit
to
proceed
• Ensure representative exercise of incremental capabilities
• Monitor progress and new operational
development
• Monitor project progress,
risk resolution, and new technology
developments
• Identify and work critical project-level actions
• Develop and integrate
new feature increments
• Communicate
proposed redirections
• Prepare for and execute
acceptance
tests, installation,
operational
evolution
• Analyze
feedback on current
version• Reprioritize
features
• Prepare strategy for
next
increment• Consolidate
process
lessons
learned
• Use risk to determine content of artifacts
Project Manager andRisk Management Team
Agile Feature Team LCA LCO
LUMS 19
NISCM Profile
• Broad coalition of government agencies and critical private-sector organizations
• Support cross-agency and public-private sector coordination of crisis management activities
• Adaptive mobile network, virtual collaboration, information assurance, and information integration technology
• Private-sector system-of-systems integration contractor
LUMS 20
Risk Items Risk Ratings
NISCM
Environmental Risks
E1. Technology uncertainties ���
E2. Many stakeholders ���
E3. Complex system of systems ���
Risks of using Agile Methods
A1. Scalability �� — ����
A2. Use of simple design �� — ����
A3. Personnel turnover ��
A4. Too-frequent releases ��
A5. Not enough agile-capable people �� — ���
Risks of using disciplined methods
D1. Rapid change ��
D2. Need for rapid results ��
D3. Emergent requirements ��
D4. Not enough discipline-capable people
��
Risk rating scale: � - minimal risk � - moderate risk
�� - Serious but manageable risk ��� - Very serious but manageable risk ���� - Show-stopper risk
NISCM Profile
Personnel
Dynamism (% Requirements-change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
5030
105
1
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
LUMS 21
NISCM Strategy
Startup Teambuilding and Shared
Vision
Design, Development and Deployment
Systems Definition and Architecting
Stable, Higher-criticality Module Teams
Stakeholders
Furnish CRACK
representatives
and alternates
• Staff and organize
to cover all
success-critical risk areas
• Prepare for and
select competitors
for concept
definition
• Develop results chains
• Identify missing
stakeholders
• Enhance understanding
• Explore goals, options, technology, prototypes,
risks
• Negotiate mutually satisfactory objectives
and priorities
• Formulate top-level
operational concept, requirements,
architecture, life cycle
plan, feasibility rational
• Select winning system
integration contract
Hold Life
Cycle
Objective
Review. If feasibility
rationale is
valid,
commit to
proceed
• Prepare for/select
competitors for
component subcontract definition
• Elaborate operational
concept, prototypes,
transition strategy
• Evaluate/determine
COTS, reuse, and
architecture choices
• Develop/ validate critical-infrastructure software
• Prioritize/sequence
desired capabilities,
outstanding risks
• Formulate definitive
operational concept,
requirements, architecture, life cycle
plan, feasibility rationale
• Use to solicit/select
component
subcontractors
Hold Life
Cycle
Architecture
Review. If feasibility
rationale is
valid,
commit to
proceed
• Ensure representative exercise of incremental
capabilities
• Monitor progress and new
operational development
• Monitor project progress,
risk resolution, and new
technology developments
• Identify/work critical project-level actions
• Continuously integrate/ test
growing software
infrastructure and
components
Dynamic, Lower-criticality Module Teams
• Develop compatible component-level prototypes, requirements,
architectures, plans,
critical software, feasibility
rationale
• Classify modules by likely
stability and criticality
Organize
into disciplined and agile
module
teams
Plan, specify, develop stable,
higher-criticality
module
increments
Plan, specify, develop dynamic,
lower-criticality module
increments
• Communicate
proposed
redirections
• Prepare for and execute
acceptance
tests,
installation,
operational
evolution
• Analyze
feedback on
current
version
• Reprioritize
features
• Prepare strategy for
next
increment
• Use risk to determine content of artifacts
NSO and I3-led ProjectRisk Management Team
LCALCO
LUMS 22
Risk Profiles Across Examples
Risk Items Risk Ratings
Event Managers SupplyChain.com NISCM
Environmental Risks
E1. Technology uncertainties � �� ���
E2. Many stakeholders � � ���
E3. Complex system of systems � � ���
Risks of using Agile Methods
A1. Scalability � �� �� — ����
A2. Use of simple design � �� �� — ����
A3. Personnel turnover �� � ��
A4. Too-frequent releases � � ��
A5. Not enough agile-capable people � � �� — ���
Risks of using disciplined methods
D1. Rapid change ���� �� ��
D2. Need for rapid results ���� �� ��
D3. Emergent requirements ���� �� ��
D4. Not enough discipline-capable people � � ��
Risk rating scale: � - minimal risk � - moderate risk
�� - Serious but manageable risk ��� - Very serious but manageable risk ���� - Show-stopper risk
LUMS 23
Outline
• Tutorial learning objectives– Survey of assumptions about participants
• General software trends and implications– Sources of perplexity about agile, plan-driven methods
• Overview of agile and plan-driven methods– XP, TSP “days in the life”– Comparisons of differences, strengths, weaknesses
• Risk-based balance of agility and discipline– Small, medium, large examples
• Observations and Way Forward
• Conclusions; review of learning objectives
LUMS 24
Summary of Observations
• No silver bullet
• Clear agile and plan-driven home grounds
• Increased demands for agility, velocity, quality
• Balanced methods are emerging
• Build methods up vs. tailor down
• Methods are important; people more so – Values– Communications– Expectations
management
LUMS 25
No Silver Bullet
• Brooks’ werewolf concerns– Complexity, conformity,
changeability, invisibility
• Agile methods – Handle changeability
and invisibility– Miss on complexity and
conformity
• Plan-driven methods– Handle conformity and
invisibility– Miss on changeability
and complexity
LUMS 26
Future Applications Need Both
• Historically– Many small, non-critical,
well-skilled, agile culture, rapidly evolving projects
– Many large, critical, mixed-skill, ordered culture, stable projects
• In the future– Large projects are no
longer stable– Maintenance of extensive
process and product plans will become too expensive
– Complexity and conformity werewolves are waiting for agile projects
– Attributes of both approaches will be needed
LUMS 27
Balanced Methods are Emerging
• Agile methods– Crystal Orange– DSDM– FDD– Lean Development
• Plan-Driven methods– Rational Unified Process– CMMI
• Hybrid– Boehm-Turner Risk-
based– Code Science/Agile Plus
LUMS 28
Build up – Don’t Tailor Down
• Plan-driven methods traditionally– Define heavyweight process guidelines– Advocate tailoring– Treat mass of processes as security blanket
• Agilists traditionally– Begin with the minimum – Add as needed (and justified by cost-benefit)– Start small; extend only where necessary
LUMS 29
Maybe its not the methods!
• Agile movement has echoed a long line of warning calls
• Success of agile may be due as much to people factors as to technology
• Valuing people over processes is the most important factor in the manifesto I know I saw something
about that in the process somewhere…
LUMS 30
People
• Of the people, by the people, for the people
• Separation of concerns is increasingly harmful
• A few quotes…
– The notion of “user” cannot be precisely defined, and therefore it has no place in computer science or software engineering.Dijkstra
– Analysis and allocation of the system requirements is not the responsibility of the software engineering group, but it is a prerequisite for their work. The Capability Maturity Model for Software.
– Software engineering is not project management. Tucker
LUMS 31
Values
• Reconciling values is a critical people-oriented task
• Stakeholders value different things
• Software engineering is usually value-neutral– Every requirement, object, test case, defect equally important
• Process improvement and plan-driven methods are inwardly-focused – Aimed at productivity improvement– Not higher value to customer
• Agilists attention to prioritization and negotiation are promising
LUMS 32
Communications
• Face it, most engineers can’t talk, much less write
• Need to bridge the customer-developer communications gap
• IKIWISI* reigns
• Rapid change increases need for solid communication
• Few sources of guidance– Cockburn’s Agile Software
Development is a good starting place
*I’ll know It When I See It
LUMS 33
Expectations Management
• Differences between successful and troubled projects is often expectations management
• SW developers have problems with EM – Strong desire to please– Little confidence in prediction– Over confidence in abilities
• Most significant common factor in highly effective plan-driven or agile teams
• EM means not setting up unrealistic expectations– Process mastery– Preparation– Courage
Developers seem to like Sisyphean tasks…
LUMS 34
Do Both Assure Success?
• TSP and agile methods are indeed silver bullets and always succeed spectacularly well.
• People tend to report success more than they do failures.
• The pioneer projects are being done by high-capability early adopters of new methods.
• The pioneer projects are experiencing some Hawthorne effects of performing extra well while being in the spotlight.
• The projects are being compared to particularly poorly performing past projects.
• High improvement numbers make options worth exploring
LUMS 35
Organizational Way Forward
• Convene key stakeholders to determine organizational goals and strategies
• Rebalance agility and discipline in this organizational context
LUMS 36
Determining Organizational
Goals and Strategies
• Current software organization’s status and needs
• Key stakeholders’ needs and trends
• How to cope with future trends
• The increased pace of change and need for agility• The increased concern with software dependability and
need for discipline
• Your ability to satisfy your stakeholders’ evolving value propositions and to keep up with your toughest competitors
• The increasing gap between supply and demand for Cockburn Levels 2 and 3 people
• Your ability to cope with existing and emerging challenges– COTS integration, evolving Internet/Web capabilities,
distributed and mobile operations, agent coordination, multimode virtual collaboration, and outsourcing jobs
LUMS 37
Collins’ Good to Great- HarperBusiness, 2001
Bureaucratic Organization
Start-up Organization
Hierarchical Organization
Great Organization
Low
Low
High
High
Ethic of Entrepreneurship
Culture of Discipline
LUMS 38
Rebalancing Agility and Discipline
In agile or plan-driven
home ground
Highly mixed
agile/plan driven
profile
Graph “typical project” on 5D chart
-Now; 5 years ahead
Develop best home
ground continuous process
improvement strategy
Treat anomalies
as risks to be
managed
Develop incremental
mixed strategy. Try
on pilot project
Integrate, execute process, people, technology plans.
Monitor and control using Balanced Scorecard.
In home ground
with some
anomalies
LUMS 39
Conclusions
• Plan-driven and agile methods both aim to– Satisfy customers– Meet cost and schedule parameters
• Home grounds exist, but the need/opportunity for integration is expanding
• We recommend a self-assessment of your organization and business – Locate your place in the home ground space– Identify areas of change and how they might move your location
– Look for ways to integrate methods to better respond to change
• People concerns dominate method concerns