Go Faster - slides.yowconference.com · MicroServices Rails / Java Zone SO A Workshop Forward Netflix Services Impact. 0MQ 0MQ 0MQ 0MQ 0MQ 0MQ ... •RabbitMQ written in Erlang •Case

Go Faster:Tech, Process, & Organization

Fred George

[email protected]

@fgeorge52

Copyright © 2015 by Fred George. All rights reserved. 1

Fred George


Consulting Roles

• Change Agent

• Disruptor

• “Hand grenade I am throwing

into development”

• CTO describing Fred to his Vice

President

Industry “Hot” Topics


MicroServices

Cloud

Docker

Dev/Ops Full-stack

developerAgile

Lean startupCassandra

Event busMVP (minimum

viable product

Programmer

anarchy

No-SQL

Why: New Problem Domains: The Cynefin Framework


Simple

ComplicatedComplex

ChaoticCause a Effect

Cause aaa Effect

Effect? Cause?

Cause ? Effect

Disorder

Why: Competition is coming


• Enablement through technologies

• Cloud computing

• Ubiquitous, high bandwidth

• Languages (and supporting frameworks)

• Recognition of business opportunities

• Silicon Valley innovators as role models

• Accelerating business needs

• Few inhibitors for global competitors

• Reduction of entry barriers for niche competitors

• Micro-marketing to specific customer niches

How Fast Can You Go?


Iteration Length in Agile Age


1 mo.

20152000 2005 2010

3 wks.

2 wks.

1 wk.

1 day

XP

XP

XP

XP

SCRUM* *2-3 months

Project Delivery Cycles


5 yrs.

1980 1990 2000

1 mon.

1 week

1 day

2010

1 year

Pro

ject D

ura

tio

n (

log) Waterfall

Waterfall

Agile

Agile

A2

A2

Waterfall

w OO

How Fast Can You Go?


Deployment to Production Every 3.5 minutes

Inhibitors


•Cloud exploitation

•Specialized databases

•New programming languages

•Open source frameworks (users and creators)

•Continuous releases

“Valley Tech”


Hardware Lead Times


6 mo.

1990 1995 2005

30 min.

1 week

1 day

2015

1 mo.

2000 2010

5 min.

5 sec.Ord

er

lead

tim

es (

log)

Data Center

Virtual

Machines

Docker

Commercial

CloudCapacity planning?

Dedicated ops team?


10

1

1000

10

0

100,0001 10 100 1000 10,000

Size (loc)

Num

be

r of S

erv

ices MicroServices

Rails / Java Zone

SO

A

Workshop

Forward

Netflix

Services Impact

0MQ

0MQ 0MQ

0MQ 0MQ 0MQ


Kafka

Service

Emerging: Event Bus

New Architecture Patterns


Bus

ServiceService

Service

Need

Need

Need

Sol’n’sSol’n

•Variants easy

•Graceful degradation

Choose

Sol’n

Incremental Applications


Event Bus

Rental

offers

Car

rent

Legacy

server

Brand

offersLocation

offers

Segmentation

Membership

CarRent.com Login

Sally

A B

Databases:Holy Grail or Ball-and-Chain?

• Fortune 100 view:

• Entity-oriented; consistency essential

• As few as possible

• MicroService view:

• Shift to Event Bus, plus

• DB per MicroService (if persistence needed)

• Poly-glot (various NoSQL, SQL) + event buses

• Few (10%) writable; even fewer transactional



Rental

offers

Car

rent

Legacy

server

Brand

offersLocation

offers

Segmentation

Membership

CarRent.com Login

Sally

A B Rental

offers

Event

publishingSolution

collecting

300ms

responder

Redis


Rental

offers

Car

rent

Legacy

server

Brand

offersLocation

offers

Segmentation

Membership

CarRent.com Login

Sally

A B

Membership

enrichmentETL Key-value

Open Source: Netflix


Open Source: Docker


Disruptive Technology: Functional Languages


•Middleware exploiting:

• Kafka written in Scala

• RabbitMQ written in Erlang

•Case study: MailOnline page rendering

• Legacy system: 130K loc Java/JSP’s

• Replacement: 4,000 loc Clojure

Mitigate Process Inhibitors


Understand Your Problem


Complex

Simple

Complicated

ChaoticCause a Effect

Cause aaa Effect

Effect? Cause?

Cause ? Effect

Disorder


Requirements

Trying ideas

“Experimentation

drives Innovation”

New Vision of “What” to Build


Task

Story

Feature

Project

Initiative

Co-Credit: Greg Reiser, ThoughtWorks

Re-think Interactions

• Bad metrics:

• Lines of code

• Development stages and milestones met

•Business success metrics:

• Sales, clicks, registrations, customer retention, etc.

Measure What Matters


Mitigate Organization Inhibitors


Over-Specialization

Copyright © 2015 by Fred George. All rights reserved.

•Theory:

• Specialist are more productive

•Practice:

• Overhead of communication is under-estimated

• Unbalanced workload creates delays

30

Specialization Institutionalized with Titles


Case Study:

•50 IT professionals

•25+ titles

•0 people understanding projects

Solution: Fix the Titles


Step 1: Define competence in key technologies

Strategic Technologies• Ruby

• Java

• iOS

• Android

• DB (SQL and non-SQL)

• Testing

• OO Design

• Node.js

• HTML/CSS/JavaScript

• …

New Titles for New Projects


Not competent in a

key technology

Competent in a key

technology

Expert in a key

technology

Competent in 5-7

key technologies

Expert in 3-5 key

technologies

“Fix the Furniture”*

Copyright © 2015 by Fred George. All rights reserved. 34*Kent Beck, XP Explained

Non-dedicated Leadership


Bring Work to the Team…


Not musical

chairs for each

small project

Reinforcing Movements


MicroServices

Cloud

Docker

Dev/Ops Full-stack

developerAgile

Lean startupCassandra

Event busMVP (minimum

viable product

Programmer

anarchy

No-SQL

Project Delivery Cycles


NRK

NRK

5 yrs.

1980 1990 2000

1 mon.

1 week

1 day

2010

1 year

Pro

ject D

ura

tio

n (

log) Waterfall

Waterfall

Agile

AgileWaterfall

w OO

Acknowledgements


•Neil Hutchinson – Founder of Forward

• Adopting a “Fail Fast” culture

•Carl Gaywood

• Former programmer who wanted to Go Faster

•Paul Ingles and Mike Jones

• Implementers of “Experimentation drives innovation”

More Information…

•Google:

• MicroService Architecture for videos

• MicroXchg 2015 conference

• Programmer Anarchy for managerless teams

• Docker for latest container deployment


Go Faster:Tech, Process, & Organization

Fred George

[email protected]

@fgeorge52


Documents

Go Faster - slides.yowconference.com · MicroServices Rails / Java Zone SO A Workshop Forward Netflix Services Impact. 0MQ 0MQ 0MQ 0MQ 0MQ 0MQ ... •RabbitMQ written in Erlang •Case