View
4
Download
0
Category
Preview:
Citation preview
Michael KoesterSr. Product Marketing Manager, Visual Studio Online, ALM & DevOps
Part of the Visual Studio product team and a developer technologies specialist
Based in Frankfurt, Germany (it’s the airport …)
Worked for Microsoft for 12 years in a number of different roles:
Program Management, Product Management, Sales, Marketing, Consulting …
Why does this matter?Agile and DevOps practices are changing the way teams develop software
An organization’s ability to deliver quickly and reliably is a key competitive differentiator
Being responsive means understanding what’s going on in a deployed application
The engineering system is central to developer productivity and job satisfaction
The end-to-end tools, processes & culture enabling a software team to
turn a customer scenario into deployed product code and improve it
through a continuous learning system
ToolsProcessCulture
Practicing Build, Measure, Learn
Adopting the best of OSS
Focus on quality
Change one thing – change everything!
SchedulingWe did ask for feedback after each milestone – planning, Beta, RTM
We did find bugs with this process and fix them - no problems there
But we couldn’t react to anything customers using the product were telling us
For the most part, we would tell everybody “sorry”… and push things to the next release
Planning
Beta? RTM?
Schedule
Code Test & Stabilize Code Test & Stabilize
2,5 years
Our planning process has changed - from long upfront planning
cycles to shorter build, measure, learn cycles …
… with continuous alignment to our long term vision
W O R K S F O R
E V E R Y O N E
A G I L E A T S C A L E
M O D E R N C O D I N G
A C T I O N A B L E
I N S I G H T S
M A N A G I N G
T E C H N I C A L D E B T
F I R S T P A R T Y I S
T H I R D P A R T Y
C O N T I N U O U S
D E L I V E R Y
Enterprise Agile
Visual Studio 2015
Our engineering system is hosted on Visual Studio Online:
The same Cloud service we offer to our customers
All new code is deployed to SU0 (our own branch)
first - before our customers get it
VSO SU1
Chicago
VSO SU0
San Antonio
VSO SU5
Amsterdam
Shared Platform Services
San Antonio
All code is deployed, but feature flags control exposure
Flags provide runtime control down to individual user
Users can be added or removed with no redeployment
Enables “dark” launch
Mechanism for progressive experimentation & refinement
Too many actions and distractions can
prevent customers from taking the next
natural step of creating a new project
Previous project creation experience
Focused project creation
experience - integrated
into the IDE flow
IDE
Web
% of accounts creating projects the day of account creationJanuary 22 to February 11 (all sources)
A second experiment,
running in parallel,
invalidated results
First project creation
jumps from 15% to 50%
First project creation
from IDE jumps from 3%
to 20%
% of accounts creating projects the day of account creation January 22 to February 5 (IDE only)
Customer IntelligenceBusiness IntelligenceOperational Intelligence
ActivityLogging
TracesKPI
MetricsJob
HistoryCustomer
IntelligenceSynthetic
Perf Counters
DependentServices
Customer IntelligenceBusiness IntelligenceOperational Intelligence
ActivityLogging
TracesKPI
MetricsJob
HistoryCustomer
IntelligenceSynthetic
Perf Counters
DependentServices
Customer IntelligenceBusiness IntelligenceOperational Intelligence
ActivityLogging
TracesKPI
MetricsJob
HistoryCustomer
IntelligenceSynthetic
Perf Counters
DependentServices
Customer IntelligenceBusiness IntelligenceOperational Intelligence
ActivityLogging
TracesKPI
MetricsJob
HistoryCustomer
IntelligenceSynthetic
Perf Counters
DependentServices
Customer IntelligenceBusiness IntelligenceOperational Intelligence
ActivityLogging
TracesKPI
MetricsJob
HistoryCustomer
IntelligenceSynthetic
Perf Counters
DependentServices
Customer IntelligenceBusiness IntelligenceOperational Intelligence
ActivityLogging
TracesKPI
MetricsJob
HistoryCustomer
IntelligenceSynthetic
Perf Counters
DependentServices
Front-doors, conversion funnel, engagement, top customers
Input metrics vs. output metrics
Watch out for vanity metrics!
Application Insights
Visual Studio 2015
We are bringing the benefits of open source and services culture
to our internal development workflow …
… while maintaining the rigors of enterprise assets
A transition of this magnitude will take time, but
we’re already well on our way!
… source across the company is available to anyone
… any developer can offer improvements to anything in the company
… IP the company has built up over the years is made of re-usable components
… anybody can find and potentially re-use components from anywhere else
… developers are rewarded for creating popular components
… minimal lag from a developer change to when the rest of the company sees it
… developers can move to another team and already know how to work
… products built this way meet the rigors of quality and support our customers expect
Update 1
Update 2
Update N
One code base with multiple delivery streams
Single master branch, multiple release branches
Shared abstraction layer (VSSF)
Be able to understand what it does
Can easily pick it up and use
Fork/Follow/Favorite
Download
Answer questions, write blog posts
Fix/add documentation
Log bugs
Fix typos Actually contribute code patches that fix bugs
Contribute entirely new features
Maintain platforms
Translate
Stick around - influence the direction of the project
Become a core committer (get write access)
Accept/validate code contributions
Nurture new people
Own
Contribute code
Contribute time
Use
Our move to GitMigrated a team of 250+ engineers from TFVC to Git
Left history behind – no complaints so far!
You can never do enough Git training, hands-on workshops is the way to go
Git scales reasonably well with file size
Using a one-branch model with good results (made feasible by significant investments in speeding up our build time)
Pull requests have formalized our code review process
GitStarts with the characteristics and workflows for sharing
Brings the culture of sharing, reinforces componentization mindset
A modern source control system, a popular tool in the industry
Investing to build an enterprise class & scale cloud Git service
Quality in the Cloud cadence optimizes
code velocity and release velocity
Change in the engineering roles was needed to enable this shift
This doesn’t mean our quality bar to customer is lowered - quite the opposite!
Ship frequently with minimum frictionDream of a system that can do hands-free push of a hotfix to production in 15 minutes
Fast and predictable Inner LoopInner Loop = Build >Deploy >Test cycle which runs reliably hundreds of times a day
Quality approach designed for Cloud cadence
Continue to meet a rigorous on-premises quality bar
PrinciplesTests should be written at the lowest level possible
Write once, run anywhere - includes production system
Product is designed for testability
Test code is product code, only reliable tests survive
Testing infrastructure is a shared service
New Engineer role combines responsibilities from dev and testEveryone has a new role, everyone is learning new skills
Every engineer and team has E2E accountabilityNo more handoffs for testing, stronger emphasis on peer reviews
Big cultural shift across the company
IntelliTest
Visual Studio 2015
Code Test & Stabilize Code Test & Stabilize
Beta RTM
Planning
Code
Complete
Cu
stom
er
Jou
rney
Discover Value
Acquire Product
Learn
Use Features
Pay
Optimize Value
Program Manager… understands customer needs
… designs great features
Customer
Cu
stom
er
Jou
rney
Discover Value
Acquire Product
Learn
Use Features
Pay
Optimize Value
Program Manager… understands customer needs
… designs great features
… attracts customers into the funnel
… describes the product value
… documents product usage
… forms business models
… analyzes and obsesses over customer data
Customer
Live Site Health/DebtTime to Detect, Time To Mitigate
Incident prevention items
Aging live site problems
Customer support metrics (SLA, MPI, top drivers)
Engineering Health/DebtBug cap per engineer
Aging bugs in important categories
Pass rate & coverage
Velocity Time to build
Time to self test
Time to deploy
Time to learn (Telemetry pipe)
We are modernizing the process for documentation, engineering
and community to easily create developer content
Documentation writing is coming to the Cloud Cadence
Building a system and culture that treats “Doc as Code”
M2 Feature DevelopmentPlanning M1 Feature Development
Result:
Specifications1 Result:
Feature Complete2
DraftsFirst draft of
documentation
ReviewDraft doc sent
out for review
ReviewChanges are
incorporatedPreviewWriter publishes to
sandbox for preview
StagingWriter promotes
content to staging
Go-LiveWriter promotes
content to live
Schedule
Documentation process
Pub
lished
It takes a full sprint or release
until docs come together and
work is only visible to a small
set of users and stakeholders
Refining
Docs continue to get
refined and updated
Each update is just a
small increment
Review
Automated review mail
is sent to the team and
stakeholders
Docs are automatically
published to an internal
staging website
Week
Planning1
Drafts
Specs created, initial
docs authored by PMs
API docs authored by
PM/Engineers before
any code is written
Week
Coding2/3
Changes
Anyone wanting to add
a doc change simply
does a pull request
Merging
Content owner/PM
gets a notification
and merges
PublishedContent updated to live branch, automatically
pushed to live public website
Sprint
3 Weeks
Microsoft is creating ‘One Engineering System (1ES)’ for the whole
company designed for the Cloud Cadence
Visual Studio Online is the foundation for this system
Our team is building many of its components and leading the change
Summary
Internal shared source experience
An agile work management system
Source code management strategy using Git
New quality approach
Distributed cloud based build system
A componentization system
Live site management tools
Telemetry
We are building all this …
… in
Recommended