70
REAL-WORLD MICROSERVICES Lessons From The Frontline

Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

Embed Size (px)

DESCRIPTION

Is Microservices gaining momentum? Looking at the interest in microservices in Australia at the moment, it is evident that this is a Service Oriented Architecture making waves. I shared our insights and experiences to over 500 interested attendees at completely sold out YOW! nights events across Sydney, Brisbane and Melbourne. These talks revealed the core lessons ThoughWorkers have learnt building a variety of systems with Microservices architecture globally. They aimed to help viewers identify Microservices and their counterparts; guide them on where to use Microservices; and deliver a series of practices for technologists to build, test, deploy and operate a Microservices architecture. The world of software architecture is excited and energised with the promises of a new Service Oriented Architecture, Microservices; rapid deployment, scalability, autonomy, and faster cycles of experimentation and innovation, and we are too!

Citation preview

Page 1: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

REAL-WORLD MICROSERVICES Lessons From The Frontline

Page 2: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

AGENDA

2

Benefits & Complexities

Definition

Stories & Practices

Page 3: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT IS A MICROSERVICES ARCHITECTURE? Post-continuous delivery SOA

Page 4: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT IS A MICROSERVICES ARCHITECTURE?

Post-continuous delivery SOA Fine-grained domain capability

Page 5: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT IS A MICROSERVICES ARCHITECTURE?

Post-continuous delivery SOA Fine-grained domain capability  Independently releasable

Page 6: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT IS A MICROSERVICES ARCHITECTURE?

Language agnostic integration

Post-continuous delivery SOA Fine-grained domain capability  Independently releasable

Page 7: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT MICROSERVICES ARCHITECTURE IS NOT

Page 8: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT MICROSERVICES ARCHITECTURE IS NOT

Monolithic

Page 9: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT MICROSERVICES ARCHITECTURE IS NOT

Monolithic

Layered and cross- functional

Page 10: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT MICROSERVICES ARCHITECTURE IS NOT    

 Monolithic    

 Layered and cross- functional  

Intelligently integrated

Intelligent Integration

Page 11: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

WHAT MICROSERVICES ARCHITECTURE IS NOT

Monolithic

Intelligently integrated

Centrally integrated

Layered and cross- functional

Page 12: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

 INNOVATION COMPOSABILITY

VALUES

 AUTONOMY    with

RESPOSIBILITY

SPEED OF CHANGE

SCALE  LOWER

CONGITIVE LOAD TECH DIVERSIFICATION

Page 13: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

6

     VALUES &  PRINCIPLES              Autonomy  Speed of Change              Scale      Composability          Tech Diversity

COMPLEXITIES            Communication    Execution    Resilience    Maintenance    Operational

MICRO MACRO

Page 14: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RYAN’S STORY  DOMAIN COMPLEXITY    Supply Chain SAN FRANCISCO

7

Page 15: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SPLITTING FROM MONOLITH – BROWN FIELD

Page 16: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

DOMAIN BOUNDED CONTEXT

SPLITTING FROM MONOLITH – BROWN FIELD

Page 17: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

DOMAIN BOUNDED CONTEXT    RATE OF  CHANGE

SPLITTING FROM MONOLITH – BROWN FIELD

Page 18: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

DOMAIN BOUNDED CONTEXT    RATE OF  CHANGE

TEAM STRUCTURE Conway’s Law

SPLITTING FROM MONOLITH – BROWN FIELD

Page 19: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

DOMAIN BOUNDED CONTEXT    RATE OF  CHANGE

TEAM STRUCTURE Conway’s Law

SPLITTING FROM MONOLITH – BROWN FIELD  

 WHAT HURTS MOST

Page 20: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

CAUTION: PARTITIONING BY EXISTING COUPLING

D3.js Structure 101

FxCop NDepend

Page 21: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

10

GREEN FIELD DOMAIN MODELLING USE THE JOURNEYS

Page 22: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

GREEN FIELD DOMAIN MODELLING INSPECT THE BOUNDED CONTEXTS

Neo4J

Page 23: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

12

One service at a time          

Single responsibility  Loose coupling  High cohesion  SOLID principles

EVOLUTIONARY BACK TO BASIC PRINCIPLES

Page 24: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

EVAN’S STORY  SERVICE OWNERSHIP  Online Real Estate MELBOURNE

13

Page 25: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

LONG-LIVED TEAMS

15

Page 26: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

15

LONG-LIVED TEAMS    

FLUID TEAM MEMBERSHIP

Page 27: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

15

LONG-LIVED TEAMS    

FLUID TEAM MEMBERSHIP  

COLLABORATIVE CODING

Page 28: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

KENT’S STORY

EXECUTION COMPLEXITY PIPELINES Online Retail MELBOURNE

16

Page 29: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

CONSUMER-DRIVEN CONTRACTS  

 Consumer Provider

Purchase  Confirm.    Email

Purchase  Service

Page 30: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

CONSUMER-DRIVEN CONTRACTS  

 Consumer Provider

Purchase  Confirm.    Email

Purchase  Service

Purchase    Service  API Test

Purchase  Service    Mock

Page 31: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

CONSUMER-DRIVEN CONTRACTS  

 Consumer Provider

Purchase  Confirm.    Email

Purchase  Service

Purchase    Service  API Test

Purchase  Service    Mock

Page 32: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

Production DECENTRALISED PIPELINE

 Build Test Integration Test Acceptance Test

Page 33: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

DECENTRALISED PIPELINE  Build Test Consumer Test Acceptance Test Production

Page 34: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

EVAN’S STORY  OPERATIONAL COMPLEXITY - MONITORING  Online Real Estate MELBOURNE

19

Page 35: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

20

Heartbeat for load balancer

Page 36: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

21

Nagios

checks

Page 37: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

22

Full diagnosis  for human

Page 38: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

23

AGGREGATED MONITORING

 Riemann    Event processing

Page 39: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

24

SYNTHETIC TRANSACTIONS

Page 40: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

KENT’S STORY

OPERATIONAL COMPLEXITY TRADEOFFS

Online Retail MELBOURNE

25

Page 41: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

MONITORING AGGREGATED LOGS

26

ID: 123

ID: 123.467

ID: 123.892

Page 42: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS • Up

Page 43: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS •  •

Up Down

Page 44: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS •  •

Up Down

STATUS • All up

Page 45: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS •  •

Up Down

STATUS • All up • All down

Page 46: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS •  •

Up Down

STATUS

• All up • All down • Mostly up

Page 47: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS •  •

Up Down

STATUS

• All up • All down • Mostly up  Up apart from 1 •

Page 48: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

RESILIENCE

STATUS •  •

Up Down

STATUS

• All up • All down • Mostly up  Up apart from 1 •

All of the important bits are up •

Page 49: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

FAULT TOLERANCE PATTERNS

28

CIRCUIT BREAKER

Netflix/ Hystrix

[failure threshold  reached]

CLOSED              [call succeeds]

   Reset

 Trip breaker                [call fails]

   Trip Breaker      HALF

OPEN

OPEN            

   on timeout  Attempt Reset

Page 50: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

JAMES’ STORY MAINTAINABILITY  Telecommunication SYDNEY

29

Page 51: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICES SELF-DISCOVERABILITY

Page 52: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICES SELF-DISCOVERABILITY

Page 53: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICES SELF-DISCOVERABILITY

Page 54: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICES SELF-DISCOVERABILITY

Page 55: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICESSELF-DISCOVERABILITY

Page 56: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICES SELF-DISCOVERABILITY

Page 57: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SCOTT’S STORY  A NEW DOMAIN VIEW  Global Assignee Tax Services MELBOURNE

31

Page 58: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

History is the version of past events that  people have decided to agree upon.

           

   

   - Napoleon Bonaparte

32

Page 59: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Entity centric  e.g. Customer

Page 60: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Activity centric  e.g. Purchase

Entity centric  e.g. Customer

Page 61: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Activity centric  e.g. Purchase

Entity centric  e.g. Customer

 Aggregator centric e.g. Notification board

   Search everywhere

Reports

Page 62: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Activity centric  e.g. Purchase

Entity centric  e.g. Customer

 Aggregator centric e.g. Notification board

   Search everywhere

Reports

Page 63: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

DOMAIN EVENTS AS FIRST CLASS CITIZENS

     DOMAIN EVENT Time {t}, CustomerRegistered, …

 Time {t}, CreditDebited, …    Time {t}, OrderPlaced, …

(1) Publisher- Subscriber  (2) RESTful      Event    endpoints

F

Build future  capabilities    e.g. gamification

Page 64: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

JR’S STORY CHOREOGRAPHY

Online Retail BRISBANE 35

Page 65: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

SERVICE COMPOSITION

36

Leave composition to the one with most context    Expose finer-grained operations  Composition of Sync and Async is hard , try Reactive programming  JavaScript Promises, RxJava Observables

Page 66: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

GILES’ STORY  FLEXIBILITY  Sports Broadcasting LONDON

37

Page 67: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

THAT POPULAR SPORT

38

Page 68: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

KNOW YOUR PROBLEM & DOMAIN WELL

TAKE-AWAYS

TRADE-OFF COMPLEXITIES CAREFULLY

IT’S AN ORGANISATIONAL AND CULTURAL SHIFT

EVOLVE FROM MACRO TO MICRO

BUILD FOR CONTINUOUS DELIVERY FROM DAY 0

MAKE SURE YOU ARE TALL ENOUGH ☺

Page 69: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

40

New!

Page 70: Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

41

THANK YOU!    Zhamak Dehghani [email protected] @zhamakd