Upload
madilyn-devereaux
View
217
Download
4
Tags:
Embed Size (px)
Citation preview
Visual Studio ALM: Lessons Learned Through Dogfooding Brian HarryTechnical FellowMicrosoft Corporation
SESSION CODE: DPR206
AgendaLessons Learned Through Dogfooding
OverviewTFS at MicrosoftGetting started on a releaseDevelopment Tracking and ReportingValidation
Visual Studio 2010
Visual Studio ALM Progression
V1 - TransparencyV2 - Bringing together development & testV3 - ?
TFS at MicrosoftMay 2010 Growth TotalsUnique Active Users
+2,128 22,430
Total Active Users
+1,072 25,208
Projects +128 5,028Work Items -30,206 6,679,715Source Code Files
+1,773,773
91,337,562Metric TFS 2008 RTM
TFS 2010 RTM Ratio
TFS Instances 24 40 1.7XUnique Active Users
7,453 22,430 3X
Projects 1,209 5,028 4.2XWork Items 1,096,318 6,679,715 6.1XSource Code Files 24,952,519 91,337,562 3.7X
TFS at Microsoft
TFS in Developer Division
3,839 Recent users18.7TB Storage826,234,724 Files1,594,094 Checkins839,595 Work Items
At least 10 different locations, including 4 “major” ones
DevDiv TFS Topology
AgendaLessons Learned Through Dogfooding
Getting started on a releasePlanning
The Challenge
What We’ve Learned
Planning And Tracking
Is the organization aligned? How are we doing against biz objectives? Are we making progress on the right things?
Align the organization around key objectives Leverage TFS to reflect the planning process Use TFS Office integration for daily tracking Use TFS for end to end traceability and reporting
Our Process – VS 2010
Features
Feature Groups
Value proposition
Main objectives Pillars
Value Props
Feature Groups
Deliverables Deliverables
Feature Groups
Deliverables
Value Props
Feature Groups
Deliverables
Value Propositions
Deliverables
Features
Deliverables
Tracking Tasks In Microsoft Excel
Tracking Tasks In Microsoft Project
Iteration Reviews – Business
AgendaLessons Learned Through Dogfooding
Getting started on a releaseDevelopment
Organizing the WorkBranching
The Challenge
The Solution
Organizing the Work
Incredibly difficult to drive a release with 2,000 people. High degree of churn. Complex interdependencies. Inexorable schedule
“Feature Crews” Mini-development organization Semi-autonomous Accountable for delivering quality
Checkpoints Keep Focus
Checkpoint 0 – ScopeCheckpoint 1 – PlanCheckpoint 2 – DemoQuality gates & Merge
Establishing Architecture
SCRUM Is Very Popular
Quality as You Go
Development and test are simultaneousDevelopers & Testers share responsibilityDevelopers & Testers share tools
Test planningUnit testingFunctional testingManual testingPerformance testingStress testing
Test Planning
Unit Testing
Manual Testing
Performance & Stress Testing
Dashboards Provide Visibility
Builds
Every checkin built & verifiedBuildCode AnalysisCheckin tests
Nightly builds provide QA updates
Quality Gates Prevent Chaos
Carry no debt in feature developmentFeature Must Pass “Quality Gates” on Finish before
Feature completeTest CompleteAll Bugs FixedSecurity PlanStatic Code AnalysisCode CoverageNo Performance RegressionsLocalization TestingAPI Reviews
Feature may merge to active branch only when Quality Gates are met
2005 Debt vs. 2008 Debt
VS 2005 Beta 1Product Bugs only
VS 2008 Beta 1ALL bug debt
The Challenge
The Solution
Branch Strategy
Massive division – 2,000+ people Code churning at every level
Isolation and Integration Isolate major feature areas in branches Automate branch quality checks Rhythm of regular merges Handles integration complexity
Source/Branch Structure
1 branch per productGated checkin procedureVBLs build in parallelNightly test suites verify branch healthRegular schedule for merges Stringent criteria for merges to main
$/DEV10/Main
$/DEV10/PU/<BranchName>
$/DEV10/feature/<BranchName>
$/DEV10/PU/<BranchName>
$/DEV10/feature/<BranchName>
AgendaLessons Learned Through Dogfooding
Getting started on a releaseDevelopment Tracking and Reporting
Trustworthy TransparencyTop Tracking Metrics
The Challenge
The Solution
Trustworthy Transparency
Each team needs to optimize process differently for their needs Leadership teams need central rollup to trackprogress.
Fantastic hub-spoke federated model…All teams use TFS for central trackingEach team uses best process for its needsSingle reporting system
Iteration Reviews – Business
Iteration Reviews – Business
Iteration Reviews – Product Unit
Iteration Reviews – Product Unit
AgendaLessons Learned Through Dogfooding
Getting started on a releaseDevelopment Tracking and Reporting
Trustworthy TransparencyTop Tracking Metrics
Validation
The Challenge
The Solution
Validation
How do you make sure a public release is high quality?
Integration testing Overall Gut Feel “OGF” Stress testing Slowdown the churn
Shutdown Process
Full test passZero Bug Bounce (ZBB)Zero Resolved Bounce (ZRB)Tell ModeVerification test passAsk ModeEscrow
Tracking Bugs – Aggregate
Tracking Bugs – Rates
Tracking Bugs – By Feature Area
Tracking Bugs – By Person
Exit Criteria Summary
Summary
We have rolled our learnings back into our toolsWe have encountered problemsWe have learnedWe have rolled our learnings back into the tool
We Believe In These ToolsScales to Massive Commercial DevelopmentWidely tested
I hope you can learn and benefit from our experience
Track Resources
Resource 1
Resource 2
Resource 3
Resource 4
Resources
www.microsoft.com/teched
Sessions On-Demand & Community Microsoft Certification & Training Resources
Resources for IT Professionals Resources for Developers
www.microsoft.com/learning
http://microsoft.com/technet http://microsoft.com/msdn
Learning
Complete an evaluation on CommNet and enter to win!
Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st
http://northamerica.msteched.com/registration
You can also register at the
North America 2011 kiosk located at registrationJoin us in Atlanta next year
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
JUNE 7-10, 2010 | NEW ORLEANS, LA