Real-world Microservices: Lessons from the Front Line - Zhamak Delghani, ThoughtWorks

Preview:

DESCRIPTION

Is Microservices gaining momentum? Looking at the interest in microservices in Australia at the moment, it is evident that this is a Service Oriented Architecture making waves. I shared our insights and experiences to over 500 interested attendees at completely sold out YOW! nights events across Sydney, Brisbane and Melbourne. These talks revealed the core lessons ThoughWorkers have learnt building a variety of systems with Microservices architecture globally. They aimed to help viewers identify Microservices and their counterparts; guide them on where to use Microservices; and deliver a series of practices for technologists to build, test, deploy and operate a Microservices architecture. The world of software architecture is excited and energised with the promises of a new Service Oriented Architecture, Microservices; rapid deployment, scalability, autonomy, and faster cycles of experimentation and innovation, and we are too!

Citation preview

REAL-WORLD MICROSERVICES Lessons From The Frontline

AGENDA

2

Benefits & Complexities

Definition

Stories & Practices

WHAT IS A MICROSERVICES ARCHITECTURE? Post-continuous delivery SOA

WHAT IS A MICROSERVICES ARCHITECTURE?

Post-continuous delivery SOA Fine-grained domain capability

WHAT IS A MICROSERVICES ARCHITECTURE?

Post-continuous delivery SOA Fine-grained domain capability  Independently releasable

WHAT IS A MICROSERVICES ARCHITECTURE?

Language agnostic integration

Post-continuous delivery SOA Fine-grained domain capability  Independently releasable

WHAT MICROSERVICES ARCHITECTURE IS NOT

WHAT MICROSERVICES ARCHITECTURE IS NOT

Monolithic

WHAT MICROSERVICES ARCHITECTURE IS NOT

Monolithic

Layered and cross- functional

WHAT MICROSERVICES ARCHITECTURE IS NOT    

 Monolithic    

 Layered and cross- functional  

Intelligently integrated

Intelligent Integration

WHAT MICROSERVICES ARCHITECTURE IS NOT

Monolithic

Intelligently integrated

Centrally integrated

Layered and cross- functional

 INNOVATION COMPOSABILITY

VALUES

 AUTONOMY    with

RESPOSIBILITY

SPEED OF CHANGE

SCALE  LOWER

CONGITIVE LOAD TECH DIVERSIFICATION

6

     VALUES &  PRINCIPLES              Autonomy  Speed of Change              Scale      Composability          Tech Diversity

COMPLEXITIES            Communication    Execution    Resilience    Maintenance    Operational

MICRO MACRO

RYAN’S STORY  DOMAIN COMPLEXITY    Supply Chain SAN FRANCISCO

7

SPLITTING FROM MONOLITH – BROWN FIELD

DOMAIN BOUNDED CONTEXT

SPLITTING FROM MONOLITH – BROWN FIELD

DOMAIN BOUNDED CONTEXT    RATE OF  CHANGE

SPLITTING FROM MONOLITH – BROWN FIELD

DOMAIN BOUNDED CONTEXT    RATE OF  CHANGE

TEAM STRUCTURE Conway’s Law

SPLITTING FROM MONOLITH – BROWN FIELD

DOMAIN BOUNDED CONTEXT    RATE OF  CHANGE

TEAM STRUCTURE Conway’s Law

SPLITTING FROM MONOLITH – BROWN FIELD  

 WHAT HURTS MOST

CAUTION: PARTITIONING BY EXISTING COUPLING

D3.js Structure 101

FxCop NDepend

10

GREEN FIELD DOMAIN MODELLING USE THE JOURNEYS

GREEN FIELD DOMAIN MODELLING INSPECT THE BOUNDED CONTEXTS

Neo4J

12

One service at a time          

Single responsibility  Loose coupling  High cohesion  SOLID principles

EVOLUTIONARY BACK TO BASIC PRINCIPLES

EVAN’S STORY  SERVICE OWNERSHIP  Online Real Estate MELBOURNE

13

LONG-LIVED TEAMS

15

15

LONG-LIVED TEAMS    

FLUID TEAM MEMBERSHIP

15

LONG-LIVED TEAMS    

FLUID TEAM MEMBERSHIP  

COLLABORATIVE CODING

KENT’S STORY

EXECUTION COMPLEXITY PIPELINES Online Retail MELBOURNE

16

CONSUMER-DRIVEN CONTRACTS  

 Consumer Provider

Purchase  Confirm.    Email

Purchase  Service

CONSUMER-DRIVEN CONTRACTS  

 Consumer Provider

Purchase  Confirm.    Email

Purchase  Service

Purchase    Service  API Test

Purchase  Service    Mock

CONSUMER-DRIVEN CONTRACTS  

 Consumer Provider

Purchase  Confirm.    Email

Purchase  Service

Purchase    Service  API Test

Purchase  Service    Mock

Production DECENTRALISED PIPELINE

 Build Test Integration Test Acceptance Test

DECENTRALISED PIPELINE  Build Test Consumer Test Acceptance Test Production

EVAN’S STORY  OPERATIONAL COMPLEXITY - MONITORING  Online Real Estate MELBOURNE

19

20

Heartbeat for load balancer

21

Nagios

checks

22

Full diagnosis  for human

23

AGGREGATED MONITORING

 Riemann    Event processing

24

SYNTHETIC TRANSACTIONS

KENT’S STORY

OPERATIONAL COMPLEXITY TRADEOFFS

Online Retail MELBOURNE

25

MONITORING AGGREGATED LOGS

26

ID: 123

ID: 123.467

ID: 123.892

RESILIENCE

STATUS • Up

RESILIENCE

STATUS •  •

Up Down

RESILIENCE

STATUS •  •

Up Down

STATUS • All up

RESILIENCE

STATUS •  •

Up Down

STATUS • All up • All down

RESILIENCE

STATUS •  •

Up Down

STATUS

• All up • All down • Mostly up

RESILIENCE

STATUS •  •

Up Down

STATUS

• All up • All down • Mostly up  Up apart from 1 •

RESILIENCE

STATUS •  •

Up Down

STATUS

• All up • All down • Mostly up  Up apart from 1 •

All of the important bits are up •

FAULT TOLERANCE PATTERNS

28

CIRCUIT BREAKER

Netflix/ Hystrix

[failure threshold  reached]

CLOSED              [call succeeds]

   Reset

 Trip breaker                [call fails]

   Trip Breaker      HALF

OPEN

OPEN            

   on timeout  Attempt Reset

JAMES’ STORY MAINTAINABILITY  Telecommunication SYDNEY

29

SERVICES SELF-DISCOVERABILITY

SERVICES SELF-DISCOVERABILITY

SERVICES SELF-DISCOVERABILITY

SERVICES SELF-DISCOVERABILITY

SERVICESSELF-DISCOVERABILITY

SERVICES SELF-DISCOVERABILITY

SCOTT’S STORY  A NEW DOMAIN VIEW  Global Assignee Tax Services MELBOURNE

31

History is the version of past events that  people have decided to agree upon.

           

   

   - Napoleon Bonaparte

32

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Entity centric  e.g. Customer

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Activity centric  e.g. Purchase

Entity centric  e.g. Customer

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Activity centric  e.g. Purchase

Entity centric  e.g. Customer

 Aggregator centric e.g. Notification board

   Search everywhere

Reports

THE EVOLUTION OF MODELLING THE DOMAIN

33

Entity centric  e.g. Credit

Activity centric  e.g. Purchase

Entity centric  e.g. Customer

 Aggregator centric e.g. Notification board

   Search everywhere

Reports

DOMAIN EVENTS AS FIRST CLASS CITIZENS

     DOMAIN EVENT Time {t}, CustomerRegistered, …

 Time {t}, CreditDebited, …    Time {t}, OrderPlaced, …

(1) Publisher- Subscriber  (2) RESTful      Event    endpoints

F

Build future  capabilities    e.g. gamification

JR’S STORY CHOREOGRAPHY

Online Retail BRISBANE 35

SERVICE COMPOSITION

36

Leave composition to the one with most context    Expose finer-grained operations  Composition of Sync and Async is hard , try Reactive programming  JavaScript Promises, RxJava Observables

GILES’ STORY  FLEXIBILITY  Sports Broadcasting LONDON

37

THAT POPULAR SPORT

38

KNOW YOUR PROBLEM & DOMAIN WELL

TAKE-AWAYS

TRADE-OFF COMPLEXITIES CAREFULLY

IT’S AN ORGANISATIONAL AND CULTURAL SHIFT

EVOLVE FROM MACRO TO MICRO

BUILD FOR CONTINUOUS DELIVERY FROM DAY 0

MAKE SURE YOU ARE TALL ENOUGH ☺

40

New!

41

THANK YOU!    Zhamak Dehghani zdehghan@thoughtworks.com @zhamakd