Upload
meaghan-hamilton
View
61
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Agility and Quality. Barry Boehm, USC. Outline. Increasing importance of both agility and quality Challenges of achieving both agility and quality Approaches for achieving both agility and quality Case studies and critical success factors Conclusions. - PowerPoint PPT Presentation
Citation preview
04/20/23 (c) USC-CSSE 1
Agility and Quality
Barry Boehm, USC
04/20/23 (c) USC-CSSE 2
Outline
• Increasing importance of both agility and quality
• Challenges of achieving both agility and quality
• Approaches for achieving both agility and quality
• Case studies and critical success factors
• Conclusions
04/20/23 (c) USC-CSSE 3
Need for Agility: Increasing Pace of Change
•Technology change
•Related infrastructure and services
•Marketplace dynamics
•Competition dynamics
•Organizational change
•Software is critical
•User agility aids are even more critical
04/20/23 (c) USC-CSSE 4
Agile Methods Examples
• Scrum– 30-day sprints; daily standup Scrums; prioritized backlog
• eXtreme Programming (XP)– Programming: simple design; refactoring; test first;
continuous integration– Communications: on-site customer; metaphor; stories; pair
programming; collective ownership; code standards– Management: small releases; planning game; 40-hour week
• Others– Adaptive Software Development; Crystal; Dynamic Systems
Development; Feature-Driven Development
04/20/23 (c) USC-CSSE 5
The Agile Manifesto
• Individuals and interactions over processes and tools• Working software over comprehensive documentation• Customer collaboration over contract negotiation• Responding to change over following a plan
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
That is, while there is value in the items on the right, we value the items on the left more.
04/20/23 (c) USC-CSSE 6
The Need for Software Quality
• “The world runs on software” – Stroustrup• “With C, you can easily shoot yourself in the foot.
With C++, you can easily blow off your leg” – Stroustrup
• Critical global infrastructure: finance, energy, transportation, communications, trade
• Dependability: everything you depend on– Accuracy, adaptability, affordability, availability, …– Complex attribute conflicts and tradeoffs
04/20/23 (c) USC-CSSE 7
Traditional Quality Approach
• Complete, consistent, testable requirements • Traceable to design, code, test cases• Heavyweight documentation
• COCOMO documentation rates, Very High Reliability projects– Average 120 pp/KSLOC; median 83; range 32-241
• Rewriting needed for 1000 KSLOC project– 160 people; 120,000 pages of documentation– 1% change/month: 1200 pages (7.5 pages/person)– 10% change/month: 12,000 pages (75 pages/person)
04/20/23 (c) USC-CSSE 8
Sarbanes-Oxley• A new US Law
– Congress’ response to Enron, WorldCom, et al– Internal Controls: evaluate and disclose effectiveness– Disclose fraud– Affects public companies and “significant” vendors
• Development process must include internal controls for– Fraud– Asset Management and Safeguarding– Financial Reporting
• Why is this important to executive management?– Executives can go to jail.– IT management can be held grossly negligent and sued by a
company or shareholders.
• In effect since 2004
04/20/23 (c) USC-CSSE 9
What an Auditor Looks for…
Processes and tools over individuals and interactionsComprehensive documentation over working software
Contract negotiation over customer collaborationFollowing a plan over responding to change
An Auditor Manifesto?
04/20/23 (c) USC-CSSE 10
Agile Methods and Quality
• Responding to change over following a plan– Major source of software-induced rocket failures
• Small releases: It’ll be fixed by next month– OK for discomfort; not for safety
• Test-driven development helps, but often leads to patching– Example: Ada compiler validation suite
04/20/23 (c) USC-CSSE 11
Agile and Plan-Driven Home Grounds: Five Critical Decision Factors
• Size, Criticality, Dynamism, Personnel, Culture
Personnel
Dynamism (% Requirements- change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
3010
3.01.0
0.3
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Agile
Plan- driven
Personnel
Dynamism (% Requirements- change/month)
Culture (% thriving on chaos vs. order)
Size (# of personnel)
Criticality (Loss due to impact of defects)
90
70
50
30
10
3
10
30
100
300
35
30
25
20
15
Essential Funds Discretionary
Funds Comfort
Single Life
Many Lives
(% Level 1B) (% Level 2&3)
0
10
20
30
40
Agile
Plan- driven
04/20/23 (c) USC-CSSE 12
Outline
• Increasing importance of both agility and quality
• Challenges of achieving both agility and quality
• Approaches for achieving both agility and quality
• Case studies and critical success factors
• Conclusions
04/20/23 (c) USC-CSSE 13
Using Risk to Balance Discipline and Agility - Overview
Step 5. Execute and Monitor
Step 4. Tailor Life Cycle
Step 3. Architecture Analysis
Step 1. Risk Analysis
Step 2. Risk Comparison
Rate the project’s environmental, agility-
oriented and plan-driven risks.
Uncertain about
ratings?
Buy information via prototyping, data
collection and analysis
Compare the agile and Plan-
driven risks
Go Risk-based Agile
Agility risks dominate
Plan-driven risks dominate
Architect application to encapsulate agile parts
Go Risk-based Agile in agile
parts; Go Risk-based Plan-
driven elsewhere
Yes
No
Go Risk-based Plan-driven
Tailor life cycle process around risk patterns
and anchor point commitment milestones
Monitor progress and risks/opportunities,
readjust balance and process as appropriate
Neither dominate
Deliver incremental capabilities according to
strategyNote: Feedback loops present, but omitted for
simplicity
Step 5. Execute and Monitor
Step 4. Tailor Life Cycle
Step 3. Architecture Analysis
Step 1. Risk Analysis
Step 2. Risk Comparison
Rate the project’s environmental, agility-
oriented and plan-driven risks.
Uncertain about
ratings?
Buy information via prototyping, data
collection and analysis
Compare the agile and Plan-
driven risks
Go Risk-based Agile
Agility risks dominate
Plan-driven risks dominate
Architect application to encapsulate agile parts
Go Risk-based Agile in agile
parts; Go Risk-based Plan-
driven elsewhere
Yes
No
Go Risk-based Plan-driven
Tailor life cycle process around risk patterns
and anchor point commitment milestones
Monitor progress and risks/opportunities,
readjust balance and process as appropriate
Neither dominate
Deliver incremental capabilities according to
strategyNote: Feedback loops present, but omitted for
simplicity
04/20/23 (c) USC-CSSE 14
Hybrid Agile/Plan-Driven Strategy– CRACK: collaborative, representative, authorized, committed, knowledgeable
LCALCOStartup Teambuilding DevelopmentSystems Architecting
Sta
ke
ho
lde
rsP
roje
ct
Le
ad
ers
hip
, R
isk
M
an
ag
em
en
t T
ea
ms
Ag
ile
, P
lan
D
riv
en
D
ev
elo
pe
rs
Furnish CRACK representatives and alternates
•Staff and organize to cover major risk areas
•Develop shared vision
•Negotiate top-level system objectives, architecture, plans, feasibility rationales.
•Prepare for/select developers
•Formulate/negotiate definitive requirements, architecture, plans, feasibility rationales.
•Ensure representative exercise of incremental capabilities
•Monitor, adapt to new developments
•Monitor and manage project progress, risk resolution, and new technology developments
•Continuously integrate/test growing software infrastructure and components
•Develop compatible architectures, plans, feasibility rationales •Develop system
components
Encapsulate agile portions
•
04/20/23 (c) USC-CSSE 15
Incremental Commitment Model:Single Increment View
Increment N Baseline
Rapid Change
High Assurance
Short, Stabilized Development of Increment N
Increment N Transition/O&M
Short Development Increments
Stable Development Increments
Foreseeable Change (Plan)
Increment N Baseline
Rapid Change
High Assurance
Short, Stabilized Development of Increment N
Increment N Transition/O&M
Short Development Increments
Stable Development Increments
Foreseeable Change (Plan)
04/20/23 (c) USC-CSSE 16
Incremental Commitment Model:Single Increment View
Increment N Baseline
Future Increment Baselines Rapid Change
High Assurance
Agile Rebaselining for Future Increments
Short, Stabilized Development of Increment N
V&V of Increment N
Increment N Transition/O&M
Current V&V
Short Development Increments
Future V&V
Stable Development Increments
Continuous V&V
Concerns Artifacts
Deferrals Foreseeable Change (Plan)
Resources Resources
Increment N Baseline
Future Increment Baselines Rapid Change
High Assurance
Agile Rebaselining for
Short, Stabilized Development of Increment N
V&V of Increment N
Increment N Transition/O&M
Current V&V
Short Development Increments
Future V&V
Stable Development Increments
Continuous V&V
Concerns Artifacts
Deferrals Foreseeable Change (Plan)
Resources Resources
Unforseeable Change (Adapt)
04/20/23 (c) USC-CSSE 17
Different Risk Patterns Yield Different Processes
04/20/23 (c) USC-CSSE 18
Pass/Fail Feasibility Rationales
• Evidence provided by developer and validated by independent experts that:If the system is built to the specified architecture, it will– Satisfy the requirements: capability, interfaces, level of service, and
evolution– Support the operational concept– Be buildable within the budgets and schedules in the plan– Generate a viable return on investment– Generate satisfactory outcomes for all of the success-critical
stakeholders
• All major risks resolved or covered by risk management plans
• Serves as basis for stakeholders’ commitment to proceed
04/20/23 (c) USC-CSSE 19
Case Studies andCritical Success Factors
• Diversified, USA (AgileTek)
• Aerospace, USA (Lockheed Martin)
• Medical, USA
• Enterprise Resource Planning, Germany
• Enterprise Infrastructure, Germany
04/20/23 (c) USC-CSSE 20
Agile Tek and Agile+
• Agile+ is XP + … + Business Process Analyses (BPAs) + Story “Actors” + Delphi-STE Estimation + Risk-Based Situation Audits (RBSAs) + Componentized Architecture + Wall Gantts and Instrument Panels + Automated Contract and Regression Testing + Automatic Document Generation Strict Pair Programming 40-Hour Work Week Restriction + Flexibility to meet special needs
04/20/23 (c) USC-CSSE 21
Agile Tek: Solutions to quality issues
• Scaling– Componentized Architecture/Interface Definitions– Automated Build and Test Processes – (Virtual) Team Rooms
• Unpredictability at Macro Scale– Delphi Estimation– STE usage for larger projects
• Vulnerability to changes at system level– Componentized Architecture
• Vague about system testing– Automated Contract and Regression Testing
• Inflexible to special needs– Treat the Special Need as a User Story and prioritize it accordingly
• Some ADM Practices Are Impractical– Use practices that make sense and work in real-world situations– Abandon or modify those that don’t
• Do not Manage Risks Explicitly– Use Risk Based Situation Audits– Establish a risk management philosophy
04/20/23 (c) USC-CSSE 22
Lockheed Martin Experiences
• 5 programs that used Agile to some degree:– Maritime Systems and Sensors– Aeronautics– Space Systems– Up to 4-team Scrum of Scrums
• Experience summary– Agile processes/practices used– What went well– What didn’t go quite so well
04/20/23 (c) USC-CSSE 23
Lockheed Martin: Agile Practices Used
• Project Planning– Sprint Planning– Backlog Lists– Risk Management– Manage scope, not cost/schedule/quality
• Design– Use agile modeling techniques– Keep it simple– Document just enough to keep you going
• Implementation and Test– Pair Programming – Refactoring – Test driven design – Continuous integration– Development on the target system
04/20/23 (c) USC-CSSE 24
Lockheed Martin: What Went Well
• Team empowerment/group ownership
• Plan for and embrace change
• Short cycle times allowed for prompt and frequent feedback
• Continuous integration
• Customer involvement
• Pair Programming
04/20/23 (c) USC-CSSE 25
Lockheed Martin: Needed Improvement
• Increased levels of stakeholder involvement
• Manage expectations
• Agile development processes require agile organizational processes
04/20/23 (c) USC-CSSE 26
Average Change Processing Time: 2 Systems of Systems
• Average number of days to process changes
0
20
40
60
80
100
120
140
160
WithinGroups
AcrossGroups
ContractMods
04/20/23 (c) USC-CSSE 27
Medical Case Study -- USA
• 1400 software people; 7M SLOC; 7 sites– 4 in Europe, 2 in India
• 500 medical applications; 500 financial; others• Survivability-critical software problems
– Reliability, productivity, performance, interoperability– Sarbanes-Oxley requirements– Management receptive to radical change
• Some limited experimental use of agile methods– Led by top software technologist/manager
• Committed to total change around Scrum and XP
04/20/23 (c) USC-CSSE 28
Medical-USA Adoption Profile
0102030405060708090
100
2004Jul
2005Jul
2006Jul
2007Mar
ScrumTeams
• July 2004 - July 2005– Recruit top people from all
sites into core team(s)
– Get external expert help
– Develop architecture
– Early Scrum successes with infrastructure
– Revise policies and practices
– Train, reculture everyone
– Manage expectations
• July 2005 – July 2006– Begin full-scale development
– Core teams as mentors
04/20/23 (c) USC-CSSE 29
Key Practices – USA Medical• Include customers and marketers
– New roles; do’s/don’ts/opportunities; CRACK personnel; full collaboration and teamwork; expectations management
• Scrum; most XP practices; added company practices– 6-12 person teams with team rooms, dedicated servers– Hourly smoke test; nightly build and regression test– Just-in-time analysis; story-point estimates; fail fast; detailed short-term plans;
company architecture compliance– Embrace change in applications and practices– Global teams: wikis, daily virtual meetings, act as if next-door
• Release management– 2-12 week architecting Sprint Zero; 3-10 1-month Sprints; Release Sprint; 1-6 month
beta test– Next Sprint Zero concurrent with Release Sprint
• Initiative manager and team– Define practices; evolve infrastructure; provide training; guide implementation;
evaluate compliance/usage; continuous improvement
04/20/23 (c) USC-CSSE 30
ERP Case Study -- Germany
• 35,000 employees; 32,000 customers worldwide– Major development centers in India, Israel
• Need to improve software development productivity, adaptability of Product Innovation Life Cycle– Invent, Define, Develop, Deploy, Optimize
• Proposed agile development cycle– Scrum; tailored corporate practices; strong, >70%-time
Product Owner and Scrum Master– 10-person teams best; up to 3-team Scrum of Scrums– Release cycle similar to Medical-USA
• Strong, highly experienced agile initiative leader– With top management support
04/20/23 (c) USC-CSSE 31
ERP-Germany Adoption Profile
050
100150200250300350400450500
2005Nov
2006Apr
2006Nov
Scrum-trainedScrumprojects
• Two large projects– 8 teams, 2 countries
– 5 teams, 5 Product Owners, 3 countries
• Mentoring new teams– Pre-training
– Experienced Scrum Master
– Mentor first sprint
– Coach second sprint
• Benefits: visibility, communication, productivity
04/20/23 (c) USC-CSSE 32
Challenges and Lessons Learned
• Challenges– Keeping roles straight– Setup for large projects– Rebaselining, reprioritizing backlog– Cross-cultural training, jelling– Team learning culture
• Lessons learned– Need strong Product Owner and Scrum Master
• Training for Product Owner, other stakeholders• Especially for scaling up to multiple teams
– Reinforce, evolve common corporate Scrum process– Can’t neglect CM, version control, change management
• Of applications and process
04/20/23 (c) USC-CSSE 33
Corporate Infrastructure -- Germany
• Fortune World 100 company• Global development
– Especially US, India, China
• Need to rebaseline corporate infrastructure– Business processes, services, IT infrastructure– Multi-platform portability, always on– With worldwide participation and buy-in– Strategy: RUP/Spiral architecting; Scrum-based development
• Began with multi-site core team of top personnel– Led by strong, results-oriented project/technology leader– With top management support and backing
• Grown to 4 sites, 250 people, 24 teams in 2.5 years– Largest project: 10 teams of 10-person Scrums
04/20/23 (c) USC-CSSE 34
Corporate Infrastructure: Principles
• Service-oriented architecture– Business processing: IBM, SAP, Microsoft– Generic applications: phone, Web, user interface– Infrastructure: servers, gateways, networks
• Learn form others• Select, protect best team• Inclusive stakeholder communication and
involvement• Plan for early wins• Corporate product and process framework with
explicit tailoring areas, continuous improvement
04/20/23 (c) USC-CSSE 35
Development Practices
• RUP/Spiral startup– 2 Inception, 3 Elaboration, 4 Construction cycles– Proposal bay with wall stickies for risk prioritization– 30 top people from all 4 sites
• NetMeeting for remote office involvement
– SharePoint vs. heavy documentation– Dedicated specialists (architecture, performance, test)
• Initial Operational Capability development– Timeboxed sprints with prioritized requirements
• 30% of initial requirements dropped to admit new features
• Use backlog as management instrument
• Post-IOC release sprint for documentation, installation, traning
– Followed by field test, concurrent detailed expansion planning, return of offsite core team members to lead distributed-development scaleup
04/20/23 (c) USC-CSSE 36
Critical Success Factors
• Management commitment, with incremental feasibility checkpoints– Clear message about objectives, scope, and strategy– Involve top people from stakeholder organizations– Build in growth to expansion sites– Lead through early successes
• Thoroughly prepare the ground– Infrastructure, policies, practices, roles, training– Customer buy-in and expectations management– Get help from experts
• Make clear what’s essential, optional– Most frequently, Scrum plus organizational essentials– Precede Development Sprints by Architecting Sprint
• Follow by Release Sprint, beta testing
– Where needed, work compliant mandate interpretations
• Monitor, reflect, learn, evolve
04/20/23 (c) USC-CSSE 37
Conclusions
• Success-critical to achieve both agility and quality• Hybrid agile/plan-driven methods emerging
– Incremental commitment framework– Concurrent engineering with synchronization milestones– Scrum plus organizational essentials
• Success stories emerging– Management commitment to objectives and strategy
• With incremental feasibility checkpoints
– Strong core team of technical and management leaders– Thorough preparation of organizations, people, infrastructure
• Involvement, architecture, policies, practices, plans, training
– Continuous change monitoring and adaptation
04/20/23 (c) USC-CSSE 38
Backup Charts
04/20/23 (c) USC-CSSE 39
Incremental Commitment In Systems and Life:
• Common System/Software stakeholder commitment points– Defined in concert with Government, industry organizations – Initially coordinated with Rational’s Unified Software Development
Process• Exploration Commitment Review (ECR)
– Stakeholders’ commitment to support initial system scoping– Like dating
• Validation Commitment Review (VCR)– Stakeholders’ commitment to support system concept definition and
investment analysis– Like going steady
• Architecting Commitment Review (ACR)– Stakeholders’ commitment to support system architecting – Like getting engaged
• Development Commitment Review (DCR)– Stakeholders’ commitment to support system development– Like getting married
• Incremental Operational Capabilities (OCs)– Stakeholders’ commitment to support operations– Like having children
Anchor Point Milestones
04/20/23 (c) USC-CSSE 40
The Incremental Commitment Life Cycle Process: Overview
04/20/23 (c) USC-CSSE 41
ICM HSI Levels of Activity for Complex Systems
04/20/23 (c) USC-CSSE 42
Process Models
Principles
Stakeholder Satisficing
Incremental Growth
Concurrency Iteration Risk Management
Sequential Waterfall, V
Assumed via initial requirements; no specifics
Sequential No NoOnce at the beginning
Iterative, Risk-Driven Waterfall, V
Assumed via initial requirements; no specifics
Risk-driven; missing specifics
Risky parts Yes Yes
Risk-Driven Evolutionary Development
Revisited for each iteration
Risk-driven; missing specifics
Risky parts Yes Yes
Concurrent Engineering
Implicit; no specifics
Yes; missing specifics
Yes Yes Implicit; no specifics
Agile Fix shortfalls in next phase
Iterations Yes Yes Some
Spiral Process 2001
Driven by stakeholder commitment milestones
Risk-driven; missing specifics
Yes Risk-driven Yes
Incremental Commitment
Stakeholder-driven; stronger human factors support
Risk-driven; more specifics
Yes Yes Yes
Process Model Comparison with respect to Process Principles