1CONFIDENTIAL

MICROSERVICES

ARCHITECTURE OVERVIEW

DZMITRY SKAREDAU, SOLUTION ARCHITECT

FEBRUARY 27, 2016

2CONFIDENTIAL 2

• Why do we need it

• Architecture patterns

AGENDA

• Microservice

• API Gateway

• Service Discovery

• Stateless/Shared-Nothing

• Configuration/Service Consumption

• Fault Tolerance

• Request Collapsing

• API Versioning

3CONFIDENTIAL

WHY DO WE NEED IT

4CONFIDENTIAL 4

WE DON’T EXPLICITLY

BUT IT DEPENDS

5CONFIDENTIAL 5

MICROSERVICES VS MONOLITH

Simple code base

Modularity with exact borders

Change circles decoupled

Efficient scaling

Newcomers adopting faster

Per service team responsibility

No technology lock

MONOLITH MICROSERVICES

Complex code base

Hard to maintain modularity

Change circles tightly coupled

Inefficient scaling

Scaring for newcomers

Hard to scale development team

Tied to chose technology

6CONFIDENTIAL 6

MICROSERVICES VALUES

• Heavily relies on Continues Delivery principles

• Fine-grained domain capability

• Team autonomy with responsibility

• Independent release process

• Independent scaling

• System resilience

• Technology variation

7CONFIDENTIAL 7

MICROSERVICES VALUES VS COMPLEXITY

Team autonomy

Time to market

Scaling

Componentization

Technology variation

Cross teams communication

Continues Deployment

Fault tolerance

Versioning

Maintenance

VALUES COMPLEXITY

8CONFIDENTIAL

ARCHITECTURE PATTERNS

9CONFIDENTIAL 9

ARCHITECTURE PATTERNS

• Microservice

• API Gateway

• Service Discovery

• Stateless/Shared-Nothing

• Configuration Management

• Fault Tolerance

• Request Collapsing

10CONFIDENTIAL

MICROSERVICE

11CONFIDENTIAL 11

BOUNDED CONTEXT

Bounded Context is a

central pattern in

Domain-Driven

Design. It is the

focus of DDD's

strategic design

section which is all

about dealing with

large models and

teams.

12CONFIDENTIAL 12

SIZE OF MICROSERVICE

2 pizza size team

Ideal Size 7 +/-2 persons

13CONFIDENTIAL 13

DECENTRALIZED DATA MANAGEMENT

Microservices prefer letting each service

manage its own database, either different

instances of the same database technology,

or entirely different database systems - an

approach called Polyglot Persistence.

You can use polyglot persistence in a

monolith, but it appears more frequently

with microservices.

14CONFIDENTIAL 14

DESIGN FOR FAILURE

Distributed systems are

much complex than

monolith.

When we have more

systems there is more

chances to fail.

If more places when you

can fails then more often

you can deal with failures.

15CONFIDENTIAL 15

KEY CONSIDERATION

Before you go into production with a microservices system, you need to ensure

that you have key prerequisites in place

• Rapid Provisioning

• Basic Monitoring

• Rapid Application Deployment

• DevOps Culture

16CONFIDENTIAL 16

MICROSERVICE VS SOA

Martin Fowler

Chief Scientist at ThoughtWorks

Subset of SOA

Zhamak Dehghani

Principal Consultant at ThoughtWorks

Style of SOA

Right picture

17CONFIDENTIAL

API GATEWAY

18CONFIDENTIAL 18

API GATEWAY

How many

microservices

could be involved

here?

19CONFIDENTIAL 19

API GATEWAY

9

at least

How many

microservices

could be involved

here?

20CONFIDENTIAL 20

API GATEWAY

21CONFIDENTIAL 21

API GATEWAY

23CONFIDENTIAL 23

API GATEWAY

24CONFIDENTIAL

SERVICE DISCOVERY

25CONFIDENTIAL 25

SERVICE DISCOVERY PROBLEM

26CONFIDENTIAL 26

SERVICE DISCOVERY PROBLEM

27CONFIDENTIAL

STATELESS/SHARED-NOTHING

28CONFIDENTIAL 28

STICKY SESSIONS

29CONFIDENTIAL 29

STICKY SESSIONS

30CONFIDENTIAL 30

STATELESS/SHARED-NOTHING

• Store state at the client

• Store state at database

• Distributed session

• Stateless services

31CONFIDENTIAL

CONFIGURATION MANAGEMENT

32CONFIDENTIAL 32

STORE CONFIG IN THE ENVIRONMENT

The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env

vars are easy to change between deploys without changing any code; unlike config files, there is little

chance of them being checked into the code repo accidentally; and unlike custom config files, or other

config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

http://12factor.net

http://12factor.net/config

33CONFIDENTIAL 33

SPRING CLOUD CONFIG SERVER

34CONFIDENTIAL

FAULT TOLERANCE

35CONFIDENTIAL 35

FAULT TOLERANCE

36CONFIDENTIAL 36

FAULT TOLERANCE

37CONFIDENTIAL 37

FAULT TOLERANCE

Availability % Downtime per year Downtime per month Downtime per week Downtime per day

90% ("one nine") 36.5 days 72 hours 16.8 hours 2.4 hours

95% 18.25 days 36 hours 8.4 hours 1.2 hours

97% 10.96 days 21.6 hours 5.04 hours 43.2 minutes

98% 7.30 days 14.4 hours 3.36 hours 28.8 minutes

99% ("two nines") 3.65 days 7.20 hours 1.68 hours 14.4 minutes

99.5% 1.83 days 3.60 hours 50.4 minutes 7.2 minutes

99.8% 17.52 hours 86.23 minutes 20.16 minutes 2.88 minutes

99.9% ("three nines") 8.76 hours 43.8 minutes 10.1 minutes 1.44 minutes

99.95% 4.38 hours 21.56 minutes 5.04 minutes 43.2 seconds

99.99% ("four nines") 52.56 minutes 4.38 minutes 1.01 minutes 8.66 seconds

99.995% 26.28 minutes 2.16 minutes 30.24 seconds 4.32 seconds

99.999% ("five nines") 5.26 minutes 25.9 seconds 6.05 seconds 864.3 milliseconds

99.9999% ("six nines") 31.5 seconds 2.59 seconds 604.8 milliseconds 86.4 milliseconds

99.99999% ("seven nines") 3.15 seconds 262.97 milliseconds 60.48 milliseconds 8.64 milliseconds

99.999999% ("eight nines") 315.569 milliseconds 26.297 milliseconds 6.048 milliseconds 0.864 milliseconds

99.9999999% ("nine nines") 31.5569 milliseconds 2.6297 milliseconds 0.6048 milliseconds 0.0864 milliseconds

Without taking steps to

ensure fault tolerance,

30 dependencies each

with 99.99% uptime

would result in 2+ hours

downtime/month

(99.99%30 ≈ 99.7%

uptime = 2+ hours in a

month)

http://techblog.netflix.com/2012/02/fault

-tolerance-in-high-volume.html

0.3% means that the one

million request will have

3000 failed

38CONFIDENTIAL 38

FAULT TOLERANCE

The basic idea behind the circuit breaker

is very simple. You wrap a protected

function call in a circuit breaker object,

which monitors for failures. Once the

failures reach a certain threshold, the

circuit breaker trips, and all further calls

to the circuit breaker return with an

error, without the protected call being

made at all. Usually you'll also want some

kind of monitor alert if the circuit

breaker trips.

CIRCUIT BREAKER

39CONFIDENTIAL 39

FAULT TOLERANCE: CIRCUIT BREAKER

40CONFIDENTIAL 40

FAULT TOLERANCE: CIRCUIT BREAKER

41CONFIDENTIAL 41

FAULT TOLERANCE: CIRCUIT BREAKER

42CONFIDENTIAL 42

FAULT TOLERANCE

Fallback logic scene involving

network access, such as cache

access.

FALLBACK DEGRADATION

43CONFIDENTIAL

REQUEST COLLAPSING

44CONFIDENTIAL 44

REQUEST COLLAPSING

In addition to the isolation

benefits and concurrent

execution of dependency

calls we have also leveraged

the separate threads to

enable request collapsing

(automatic batching) to

increase overall efficiency

and reduce user request

latencies.

Collapse multiple requests into a single execution

based on a time window and optionally a max batch

size.

This allows an object model to have multiple calls to

the command that execute/queue many times in a

short period (milliseconds) and have them all get

batched into a single backend call.

Typically the time window is something like 10ms

give or take.

45CONFIDENTIAL 45

COLLAPSER FLOW

In addition to the isolation

benefits and concurrent

execution of dependency

calls we have also leveraged

the separate threads to

enable request collapsing

(automatic batching) to

increase overall efficiency

and reduce user request

latencies.

Collapse multiple requests into a single execution

based on a time window and optionally a max batch

size.

This allows an object model to have multiple calls to

the command that execute/queue many times in a

short period (milliseconds) and have them all get

batched into a single backend call.

Typically the time window is something like 10ms

give or take.

46CONFIDENTIAL

API VERSIONING

47CONFIDENTIAL 47

API VERSIONING

• Adding authentication

• Adding authorization rules

• Removing a service

• API contract changes

REASONS SOLUTIONS

• URL Versioning

• Media Type Versioning

• Custom header

• Hostname

• Data parameter

48CONFIDENTIAL 48

API VERSIONING

One method for indicating versioning is via the URI, typically via a path prefix:

Twitter: http://api.twitter.com/1.1/

Last.fm: http://ws.audioscrobbler.com/2.0/

Etsy: http://openapi.etsy.com/v2

Some APIs will provide the version via a query string parameter:

Amazon Simple Queue Service: ?VERSION=2011-10-01

URL

49CONFIDENTIAL 49

API VERSIONING

Media type versioning provides the ability to use the same URI for multiple versions of an API, by specifying the version as part of the Accept media type.

The Accept header can provide versioning in two different ways:

• As part of the media type name itself: application/vnd.status.v2+json. In this case, the segment v2 indicates the

request is for version 2. You can provide the version string however you desire.• As a parameter to the media type: application/vnd.status+json; version=2. This option provides more

verbosity, but allows you to specify the same base media type for each version.

Many REST advocates prefer media type versioning as it solves the "one resource, one URI" problem cleanly, and allows

adding versioning support after-the-fact. The primary argument against it is the fact that the version is not visible when

looking at the URI.

MEDIA TYPE

50CONFIDENTIAL 50

API VERSIONING

The above two versioning types are the most common; however, other types exist:

• Custom header. As an example,

• X-API-Version: 2

• GData-Version: 2.0

• X-MS-Version: 2011-08-18

• etc.

• Hostname. Facebook, when migrating from the first API version, switched from the host http://api.facebook.com to

http://graph.facebook.com.

• Data parameter. This could be a query string parameter for GET requests, as noted above, but a content body parameter for

other request methods.

OTHER METHODOLOGIES

51CONFIDENTIAL 51

API VERSIONING

• Typical approach. Include API version. Backwards compatible.

• Some folks use custom content type allowing each resource to have a version.

• Netflix: strive to be version-less from client perspective

• Netflix: try to be version-less even if not wholly possible

• Adding new data types or new URIs don’t require new version

• Structural changes to large unknown developers better to incomplete than inaccurate. Hold onto ideas before pushing it out.

• Netflix assumes 7-10 year life of a TV

• Netflix can generally move partners from version to version as needed

NETFLIX APPROACH

52CONFIDENTIAL 52

API VERSIONING

It seems that there are a number of people recommending using Content-Negotiation (the HTTP

“Accept:” header) for API versioning.

However, none of the big public REST APIs I have looked at seem to be using this approach. They almost

exclusively put the API version number in the URI.

53CONFIDENTIAL 53

API VERSIONING

Twitter URI

Atlassian URI

Google Search URI

Github API URI/Media Type in v3

Intention is to remove versioning in favour of

hypermedia – current

application/vnd.github.v3

Azure Custom Header x-ms-version: 2011-08-18

Facebook URI/optional versioning graph.facebook.com/v1.0/me

Bing Maps URI

Netflix URI parameter

http://api.netflix.com/catalog/titles/series/

70023522?v=1.5

54CONFIDENTIAL 54

API VERSIONING

Google data API (youtube/spreadsheets/others)

URI parameter or custom

header “GData-Version: X.0” or “v=X.0”

Flickr No versioning?

Digg URI http://services.digg.com/2.0/comment.bury

Delicious URI https://api.del.icio.us/v1/posts/update

Last FM URI http://ws.audioscrobbler.com/2.0/

LinkedIn URI

http://api.linkedin.com/v1/people/~/connec

tions

Foursquare URI

https://api.foursquare.com/v2/venues/40a55

d80f964a52020f31ee3?oauth_token=XXX&v=YY

YYMMDD

55CONFIDENTIAL 55

API VERSIONING

paypal parameter &VERSION=XX.0

Twitpic URI http://api.twitpic.com/2/upload.format

Etsy URI http://openapi.etsy.com/v2

Tropo URI https://api.tropo.com/1.0/sessions

Tumblr URI api.tumblr.com/v2/user/

openstreetmap URI and response body http://server/api/0.6/changeset/create

Ebay URI (I think)

http://open.api.ebay.com/shopping?version=

713

56CONFIDENTIAL 56

API VERSIONING

Wikipedia no versioning I think?

Bitly URI https://api-ssl.bitly.com/v3/shorten

Disqus URI

https://disqus.com/api/3.0/posts/remove.js

on

Yammer URI /api/v1

Drop Box URI

https://api.dropbox.com/1/oauth/request_to

ken

Amazon Simple Queue Service (Soap) URI Parameter and WSDL URI &Version=2011-10-01

57CONFIDENTIAL

QUESTIONS?