Embed Size (px)
Citation preview
Agile Architecture Made Real
Re-architecting the organization for speed - Building a fast and flexible architecture across 50 scrum teams
Introductions and why we are here
Alexis Hui
Co-lead the Deloitte LEAN practice – Agile Advisory and Transformation
10+ years of agile experience - doing, advising and coaching
Helping organizations make agile work @ scale (100’s-1000’s)
My goal is to share with you how agile can work at speed and scale based on a case study to make it
real
Who needs architecture?
Scrum
XP
Kanban
Agile Modeling Method
100’s 1000’s
# people
Ok… something is missing
My toolkit worked great to help organizations become more agile… until we started hitting larger organizations
Architecture is at the heart of making agile work @ scale
Move Fast and Break Things
Self-organizing teams
EXPERIMENT.
FAIL.LEARN.REPEAT.
Speed and scale – Oxymoron?
Large platform 3M+ accounts $2B+ funds 5.5M+ transactions / mth 200+ product features with
multiple products
Distributed organization 1000+ people 60-70% outsourced 7+ locations, 5 major
delivery centers 5 time zones
Lots of Partners Co-branding FSI’s Enterprise back-ends Enterprise servicing
New Speed Demands Handle constantly changing
programs and business development contracts
Heavily regulated industry with frequent changes in regulations
Highly competitive space with “fast” competitors
Enormous pressure to drive big revenue growth and acquire customers
Desire to deploy and release every two weeks
New business unit spun up 4 years ago to drive new revenue streams for a much larger traditional parent FSI organization…
Rethinking the architecture – not just software
What is the real product? Reframing the world, the
product is the platform
How do we scale our architecture to the next level? Organize teams around the
product architecture and let teams do the work for you
How do we release faster than ever? Re-architect everything we
have to release every two weeks
…organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations – M. Conway
Agile Architecture in Action
Refactored their platforms with componentization in mind• Platform refactored into a discrete set of products and platform services
• Agile is enabling teams to focus on component stewardships and technical debt reduction to drive quality improvements
Re-aligned the organizational structure and delivery teams• Product and technology is reorganized into
~50 cross-functional scrum teams
• Each scrum team has ownership of a platform service / application and is responsible for a set of platform KPI’s
Enabled an end-to-end agile process• Prioritization, Intake, Program, Product,
Software and Release Processes are all agile
• Enable releases to become a non-event and occur every two weeks
• Rapid experimentation of testable product innovation and features
Adopted new agile management and automation tools• Agile ALM tooling for managing
programs and product backlogs
• Continuous integration for automating continuous builds, testing and deployment
1
2
3 4
Agile Organization
Case Study: 1000+ product and engineering division within a 10,000+ FSI IT organization
Principles of Componentization:
Define bounded contexts to enable: Single-purpose assembly components Social architecture build around explicit
contracts Independently releasable components
Expose an open source model to provide: Developers to freely modify code and deploy Stewardship and quality to protect developers
from “breaking it” Encourage cross-training and knowledge
sharing
Establish a common product language in the architecture so: The system becomes self-describing Business SMEs and Technologists can both
speak the same language Architecture always stays aligned with the
product
Componentization
Re-organizing the architecture based on a set of independent cells that can bond together to deliver greater value
Responsive Web
Customer and Account Management Platform
Partner Integration Services
Developers can freely contribute to other code bases but must abide by the cells “rules”
Stable teams are organized within the cell based on single purpose components
Cells can independently release components in their bounded context
H
BeLi
MgNa
K Ca
Sc
The cells help establish a common language in business terms
Y
Lu*
Componentization – Case Study
Customer Channels
Platforms Integrations
Platform Platform Platform Platform
Mobile Web IVR Open API Servicing
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
The How – Key Highlights
Defined clear responsibilities and boundaries for each product that could be managed, evolved and supported by one or
more scrum teams
“Componentized” the architecture and code, each product had a set of components which can be independently tested
and released
Refactored each component to avoid direct database calls and instead go through platform services
Applied a package manager system to help manage and automate packaging and versioning of each component
Component
Component
Component
Component
Component
Component
Component
Component
Services
Component
Component
Abstracted product architecture from the case study
Develop a coherent view of your architecture and start thinking how it can be componentized into products
Refactor the design and code as new features come in to increase decoupling and expose discrete APIs for greater reuse
Leverage domain driven design to start expressing the design and code in a manner that traces easily to business and product language
What you can do tomorrow
Where to learn more: Lean Architecture: for Agile Software Development – James O. Coplien Working Effectively with Legacy Code – Michael Feathers Domain Driven Design – Eric Evans
Building Software Product Teams for the Architecture
Establishing effective software product management requires software product teams that:
Sets the product vision for their components by: Owning and managing a product backlog Defining and driving product KPI’s to help
prioritize value and enable validated experimentation
Owns, manages and deploys its code separately to: Deploy on its own schedule (no more “whole
stack deployments”) Build and execute its own test harnesses and
test cases Drives down its own technical debt and is
responsible for maintaining reuse, cohesion and coupling with other components
Using cadence and flow to coordinate with other software product teams to: Establish a big picture product architecture
view Pairs with horizontal specialist teams Plans and synchronizes with other teams
regularly
Establishing a network of software product teams that each own product outcomes, technical quality and coordinate on a cadence
Horizontal teams pair with software product teams to shape end-to-end concerns (architecture, UX, features)
Each team is empowered to set their own product vision and manage their priorities via a product backlog
The product quality (customer features, operational quality) are managed, tested and deployed by the team
Based on flow, horizontal teams and software product teams collaborate to shape new work
Regular cadences help teams coordinate dependencies
Building software product ownership – Case Study
Feature/Experience
Designers
The How – Key Highlights
Established product owners for each scrum team based on the product architecture and formed stable teams by
component
Formed “one product engineering team” consisting of program owners, feature/experience designers, product owners,
and architects to work together to envision and deliver, resulted in ~50 scrum teams
Empowered each team to work with other teams to decouple their code based on the product architecture and to drive
their own technical debt (60/40 backlog ratio)
Architecture
Program Owners
Product Owners and Scrum Teams
Collect and funnel work into a single product backlog where there is a well defined product boundary
Establish a “technical product owner” as a proxy and increase collaboration with business partners to increase communication and awareness of the role and value
Work closer with stakeholders to express requirements in smaller units of value (e.g. user stories) that can be developed and tested more independently earlier in the process
What you can do tomorrow
Where to learn more: Agile Product Management with Scrum: Creating Products that Customers Love –
Roman Pichler Principles of Product Development Flow: Second Generation Lean Product
Development – Donald G. Reinersten User Stories Applied: For Agile Software Development – Mike Cohn
Taking Continuous Integration to the Enterprise Level
To succeed in delivering speed @ scale requires a set of disciplined technical practices:
Establishing an enterprise wide “definition of done” (DoD): Integration testing and other forms of testing
needs to be pushed upfront into the DoD Nothing goes to the mainline or release
without meeting DoD
Shift from managing dependencies to deferring them: Traditional code management practices needs a
rethink, feature branches vs main line and feature toggles/switches to minimize conflicts across developers while balancing early integration
Stubbing/mocking becomes more important so teams can develop more independently of each other
Automation needs to be norm: Automate the end-to-end development, build,
testing, testing architecture and engineering plays a critical role in enabling components and products to be independently testable
Test automation becomes a first class citizen where test frameworks and code becomes just as important as feature code
Enterprise level continuous integration provides the tracks for the agile release train to run and get to continuous delivery
Every two weeks, the train picks up product increments that are “code clean” and tagged for pushing to production
In each sprint, teams work to get stories “code clean” and tag it for release, it’s up to each team to make sure quality passes in the CI process
Continuous integration process runs for every commit and kicks-off a series of tests to verify “code clean” is met
Features that are have dependencies with other teams that are not ready for release yet, are toggled “off” or dialed up or down for experimentation
S1 S2
R1 R2
Taking Continuous Integration to the enterprise level – Case Study
The How – Key Highlights
Leveraging componentization and extending coding practices to always use interface based development allowed for
easier mocking/stubbing to support testing in isolation
Mainline code management was used with feature toggles/switches and API versioning that enabled each component to
merge frequently and push to production without being impacted by dependencies
An enterprise wide baseline “definition of done” (code clean) was defined and followed that forced full testing to be
completed by the end of each sprint/iteration before work could be accepted
A dedicated build environment was setup based on a testing architecture that supported full build automation, continuous
integration, notifications and reports on meeting the “definition of done” based on coverage and quality metrics
Setup or leverage an existing continuous integration server and work with your team to implement
Work with your team to define a baseline “definition of done” that is used for the continuous integration process to assess go/no-go
Refactor code to be more interface-based and work with others to improve and speed up the test automation coverage for the end-to-end range of testing including unit, functional and integration testing
What you can do tomorrow
Where to learn more: Continuous Integration: Improving Software Quality and Reducing Risk – Paul M.
Duvall, Steve Matyas, Andrew Glover Continuous Delivery: Reliable Software Releases through Build, Test and Deployment
Automation – Jez Humble, David Farley How Google Tests Software – James A. Whittaker, Jason Arbon, Jeff Carollo
Bringing it all back together and how to get started
Agile @ Speed and Scale is being done by organizations out there by taking an agile architecture approach. Disney can benefit from advancing the state of agile maturity today in the organization and taking a step wise incremental approach to agile adoption.
1) Adopt agility and speed as a culture by looking at how the three practices can be implemented Componentization Establishing software product teams for the architecture Taking continuous integration to the enterprise level
2) Refine and increase the scale of these practices across the organization to achieve operational stability based on fast and frequent releases
3) Leverage the high speed agile engine and adopt a rapid experimentation approach to think and act more like a lean startup
Thank You!
Q&A
Alexis Hui
Twitter: @alhui
Email: [email protected]
Blog: www.agileconsulting.blogspot.com
