The Future of Software Engineering (Berlin) - GOTO Blog · The Evolution of IT 1996 2016 Running on any available set of physical resources (public/private/ virtualized) Assembled

Mary [email protected] © Tom Poppendieck

Copyright©2016 Poppendieck.LLC

2Photograph © Tom Poppendieck

Business Software:

Why One Server?

Monolithic & Slow to changeERP Database IntegrationOn a Single Server

CAP Theorem:Partitioned databasesmust choose between:

AvailabilityConsistency

Photograph © Tom Poppendieck


3

[c. 2001]


4

Get a bigger computer.


5

Get more computers. Google Hardware c. 1999

Search the entire InternetInstantly

Response:Horizontal ScalingBreak files into small piecesMake three copies of each pieceSpread the pieces across many servers

Manage the pieces with software

Google’s Problem:


6

Amazon ~ 2001Handle a gazillion transactionsAll at once

What Amazon Did:Break transactions into servicesReplicate bottleneck services

Each service owned by a semi‐autonomous “two pizza” team

Option 2:Scale Out

Option 1:Scale Up


7



Pinkham moves to South AfricaAsked to pursue project thereAssembles and leads a teamDevelops EC2 in 2 years

EC2 Launches in 200610 Years Later:

$10 billion/yr.

Autonomous Services Autonomous Service Teams Independent Deployment

Much Stress in Operations

Chris Pinkham (VP Infrastructure) Proposes self‐service deploymentfor development teamsMaybe sell the capability?

Time Passes….

Photograph © Tom PoppendieckCopyright©2016 Poppendieck.LLC 9

Economics Technology

Copyright©2016 Poppendieck.LLC 10

Applications designed for traditional data architectures do not work well in the cloud without a lot of recoding.*

The Cloud is cheaper, more stable, more secure, and more expandable than most on‐premises data centers.


*Arthur Cole ‐ IBM


ContainersAs Process

ServerlessArchitectures

Software DefinedNetworks



The Evolution of IT1996 2016

Running on any available set of

physical resources(public/private/virtualized)

Assembled by developers using best available

services

Thin app on mobile, tabletThick, client‐server app

on thick client

Well‐defined stack:‐ O/S‐ Runtime‐ Middleware

MonolithicPhysical

Infrastructure

www.docker.io13

Scale Out

Technology Platform

Scale Up

EnterpriseArchitecture

Federated Architecture


API’s as Architecture API’s as ProductOwned by a Responsible TeamFocus on API ConsumersEvolving Capabilities


Enable Service ArchitecturesLower Integration FrictionLocal Persistence

API’s Replace

Databases

API’s Enable the Internet of Things

Designing for scale: Resilient Architectures:Big Data systems are inherently distributed; their architectures must explicitly handle:

partial failuresconcurrencyconsistencyreplication communications latencies

Replicate data to ensure availability in the case of failure. Design components to be

statelessreplicatedtolerant of failures of dependent services

Distribution, Data, Deployment: Software Architecture Convergence in Big Data SystemsIan Gorton, John Klein, Software Engineering Institute Carnegie Mellon University, May 2014

“Big data applications are pushing the limits of software engineering …. It is essential that the body of software architecture knowledge evolves to capture this advanced design knowledge for big data systems.”

Photograph © Tom PoppendieckCopyright©2016 Poppendieck.LLC 16

Three Kinds of Systems


17

Nassim Nicholas Taleb: Antifragile – Things that gain from Disorder

Acm Queue, September 2012:“Resilience Engineering: Learning to Embrace Failure”

Picture Credit: André Faria Gomes

Monolithic ArchitectureDatabase as IntegratorStateful ProtocolsConsecutive ExecutionSynchronous CommunicationProcedural ProgrammingDefect Free / Fragile

Federated ArchitectureAPI’s as ConnectorsStateless ProtocolsConcurrent ExecutionAsynchronous CommunicationEvent Driven ProgrammingFault Tolerant / Antifragile


The Internet of Things The Enterprise Legacy

Photograph © Tom Poppendieck Copyright©2016 Poppendieck.LLC 19

For Complex SystemsThis does not work


For Complex SystemsThis works


Acceptance test driven development processCross‐functional teams include Product, QA and OpsAutomated build, testing, db migration, and deploymentIncremental development on the trunk with continuous integrationSoftware always production ready, or everything stops until it isDeploy constantly, release by switch

Slide content credit: Jez Humble

Faster – Safer – Better


Copyright©2016 Poppendieck.LLC 23Photograph © Tom Poppendieck


Online Experimentation at Microsoftby Kohavi , Crook, & Longbotham

Presented at KDD 2009(Knowledge Discovery & Data Mining)

http://exp‐platform.com/expMicrosoft.aspx


Photograph © Dustin Poppendieck

Standish Group Study Reported at XP2002 by Jim Johnson, Chairman


Delivery Problem Solving


Start with SignalsFocus on ProblemsPlan with Hypotheses

Do MultipleExperiments

Use Data to Decide

Not Requirements Not FeaturesNot Estimates

Not a Backlogof Stories

Don’t Guess ata Solution

Signals /Patterns

Problem Statement

HypothesisExperiment

Analysis & Conclusion


Sketch: Jake KnappCopyright©2016 Poppendieck.LLC 28

Google I/O 2014 Design SprintA process for prototyping and testing any product in 5 days.

Sketch: Jake Knapp Copyright©2016 Poppendieck.LLC 29

30



1. Explore multiple ideas, including outliers.

2.Pursue a variety of ideas with champions and volunteers.

3.Gradually narrow the ideas to those that will work.

4.Maintain multiple options as long as possible.



Mary [email protected]

Mary [email protected]

Documents

The Future of Software Engineering (Berlin) - GOTO Blog · The Evolution of IT 1996 2016 Running on any available set of physical resources (public/private/ virtualized) Assembled