Developing applications with a microservice architecture (svcc)

@crichardson

Microservices: Decomposing Applications for

Deployability and Scalability

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@crichardsonchris@chrisrichardson.net http://plainoldobjects.com

@crichardson

Presentation goal

How decomposing applications into microservices

improves deployability and scalability and

simplifies the adoption of new technologies

@crichardson

About Chris

@crichardson

About Chris

Founder of a buzzword compliant (stealthy, social, mobile, big data, machine learning, ...) startup

Consultant helping organizations improve how they architect and deploy applications using cloud computing, micro services, polyglot applications, NoSQL, ...

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Client ⇔ service interaction design

Decentralized data management

@crichardson

Let’s imagine you are building an online store

@crichardson

Tomcat

Traditional application architecture

Browser/Client

WAR/EAR

MySQL Database

Review Service

Product InfoService

Recommendation Service

StoreFrontUI

developtest

deploy

Simple to

Load balancer

scale

Spring MVC

SpringHibernate

Order Service

HTMLREST/JSON

@crichardson

But large, complex, monolithic applications

⇒ problems

@crichardson

Intimidates developers

@crichardson

Obstacle to frequent deployments

Need to redeploy everything to change one component

Interrupts long running background (e.g. Quartz) jobs

Increases risk of failure

Fear of change

Updates will happen less often - really long QA cycles

e.g. Makes A/B testing UI really difficult

Eggs in one basket

@crichardson

Overloads your IDE and container

Slows down development

@crichardson

Lots of coordination and communication required

Obstacle to scaling development

I want to update the UI

But the backend is not working

yet!

@crichardson

Requires long-term commitment to a technology stack

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Client ⇔ service interaction design

Decentralized data management

@crichardson

@crichardson

The scale cube

X axis - horizontal duplication

Z axis

- data

partit

ioning

Y axis - functional

decomposition

Scale b

y split

ting s

imilar

thing

s

Scale by splitting

different things

@crichardson

Y-axis scaling - application level

WAR

Storefront UI

Product InfoService

RecommendationService

ReviewService

OrderService

@crichardson

Y-axis scaling - application level

Storefront UI

Product InfoService

RecommendationService

ReviewService

OrderService

@crichardson

Product Info

Y-axis scaling - application level

Product InfoService

RecommendationService

ReviewService

OrderService

Browse Products UI

Checkout UI

Order management UI

Account management UI

Apply X-axis and Z-axis scaling to each service independently

@crichardson

Service deployment options

VM or Physical Machine

Docker/Linux container

JVM

JAR/WAR/OSGI bundle/...

Isolation, manageability

Density/efficiency

@crichardson

Partitioning strategies...

Partition by noun, e.g. product info service

Partition by verb, e.g. Checkout UI

Single Responsibility Principle

Unix utilities - do one focussed thing well

@crichardson

Partitioning strategies

Too few

Drawbacks of the monolithic architecture

Too many - a.k.a. Nano-service anti-pattern

Runtime overhead

Potential risk of excessive network hops

Potentially difficult to understand system

Something of an art

@crichardson

Example micro-service

@crichardson

More service, less micro

But more realistically...

Focus on building services that make development and deployment easier

- not just tiny services

@crichardson

Real world examples

http://highscalability.com/amazon-architecture

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

~600 services

100-150 services to build a page

@crichardson

There are many benefits

@crichardson

Smaller, simpler apps

Easier to understand and develop

Less jar/classpath hell - who needs OSGI?

Faster to build and deploy

Reduced startup time - important for GAE

@crichardson

Scales development: develop, deploy and scale each service independently

@crichardson

Improves fault isolation

@crichardson

Eliminates long-term commitment to a single technology stack

Modular, polyglot, multi-framework applications

@crichardson

Two levels of architectureSystem-level

ServicesInter-service glue: interfaces and communication mechanisms

Slow changing

Service-level

Internal architecture of each serviceEach service could use a different technology stack

Pick the best tool for the jobRapidly evolving

@crichardson

Easily try other technologies

... and fail safely

@crichardson

But there are drawbacks

@crichardson

Complexity

@crichardson

Complexity of developing a distributed system

http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

@crichardson

Multiple databases &

Transaction management

e.g. Fun with eventual consistency

@crichardson

Complexity of testing a distributed system

http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

@crichardson

Complexity of deploying and operating a distributed

system

You need a lot of automation

http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

@crichardson

Developing and deploying features that span multiple services requires careful

coordination

@crichardson

When to use it?In the beginning: •You don’t need it •It will slow you down

Later on:•You need it•Refactoring is painful

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Client ⇔ service interaction design

Decentralized data management

@crichardson

Let’s imagine that you want to display a product’s details...

Recommendations

ProductInfo Reviews

@crichardson

Directly connecting the front-end to the backend

Model

View ControllerProduct Info

service

RecommendationService

Reviewservice

REST

REST

Thrift

Model

View Controller

Browser/Native App

Traditional server-sideweb application

Chatty API

Web unfriendly protocols

@crichardson

Use an API gateway

Model

View ControllerProduct Info

service

RecommendationService

Reviewservice

REST

REST

Thrift

APIGateway

Model

View Controller

Browser/Native App

Single entry point

Client specific APIs

Protocol translation

Traditional server-sideweb application

@crichardson

Optimized client-specific APIs

Web application

MobileApp

NodeJS

APIGateway

RESTproxy

Event publishing

Product Infoservice

RecommendationService

Reviewservice

REST

REST

Thrift

getProductInfo()getRecomm...()getReviews()

getProductDetails()

@crichardson

Netflix API Gateway

http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Device specific end points

@crichardson

API gateway design challenges

Performance and scalability

Non-blocking I/O

Asynchronous, concurrent code

Handling partial failures

....

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

Useful frameworks for building an API gateway

JVM:

Netty, Vertex

Netflix Hystrix

...

Other:

NodeJS

@crichardson

How does a browser interact with the partitioned

web application?

@crichardson

Partitioned web app ⇒ no longer a single base URL

Browse Products UI

Checkout UI

Order management UI

Account management UI

/products

/checkout

/orders

/account

Browser ?

@crichardson

The solution: single entry point that routes based on URL

Browse Products UI

Checkout UI

Order management UI

Account management UI

/products

/checkout

/orders

/account

ContentRouterBrowser

http://acme.com/<service>/...

Hidden from browser

Single entry point

@crichardson

How do the services communicate?

@crichardson

Inter-service communication options

Synchronous HTTP ⇔ asynchronous AMQP

Formats: JSON, XML, Protocol Buffers, Thrift, ...

Asynchronous is preferredJSON is fashionable but binary format

is more efficient

Pros and cons of messagingPros

Decouples client from server

Message broker buffers messages

Supports a variety of communication patterns

Cons

Additional complexity of message broker

Request/reply-style communication is more complex

Pros and cons of HTTPPros

Simple and familiar

Request/reply is easy

Firewall friendly

No intermediate broker

Cons

Only supports request/reply

Server must be available

Client needs to discover URL(s) of server(s)

@crichardson

Discovery option #1: Internal load balancer

LoadBalancer

Product InfoService

Product InfoService

Product InfoService

Product InfoService

Client/API gateway

Services register with load balancer

Client talks to load balancer

Client talks to

Has a well-known location

http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/USVPC_creating_basic_lb.html

@crichardson

Discovery option #2: client-side load balancing

RESTClient

Product InfoService

Product InfoService

Product InfoService

Product InfoService

Client

ServiceRegistry

Services register with registry

Client polls registry

http://techblog.netflix.com/2013/01/announcing-ribbon-tying-netflix-mid.html

http://techblog.netflix.com/2012/09/eureka.html

@crichardson

Lots of moving parts!Product InfoProduct InfoService

RecommendationService

ReviewService

OrderService

Browse Products UI

Checkout UI

Order management UI

Account management UI

APIGateway

Service registry

ContentRouter

HTMLBrowser

RESTClient

Ext.LB

Ext.LB

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Client ⇔ service interaction design

Decentralized data management

@crichardson

Decomposed services ⇒ decomposed databases

Order management Customer management

OrderDatabase

CustomerDatabase

Separate databases ⇒ less coupling

@crichardson

Decomposed databases ⇒ polyglot persistence

IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

@crichardson

Customer management

Untangling orders and customers

Order management

Order Service

placeOrder()

Customer Service

availableCredit()updateCustomer()

Customer

creditLimit...

has ordersbelongs toOrder

total

Invariant:sum(order.total) <= creditLimit

available credit= creditLimit - sum(order.total)Trouble!

@crichardson

ProblemsReads

Service A needs to read data owned by service B

e.g. OrderService needs customer’s credit limit

e.g Customer Service needs outstanding order total

Updates

“Business transaction” must update data owned by multiple services

e.g Creating a User => Create their ShoppingCart

@crichardson

You could use two-phase commit (2PC) but ...

@crichardson

Use an event-driven architecture

@crichardson

Event-driven architecture

Message Broker

CustomerCreditLimitUpdatedEvent

Order management

Order

total

Customer’

creditLimit

CustomerCreditLimitUpdatedEvent(...)

Customer management

Customer

creditLimit...

Customer Service

updateCustomer()

UpdateCreditLimitCommand

@crichardson

Benefits and drawbacks of an event-driven architecture

Benefits:

Simple infrastructure, e.g. a message broker

Better availability (than using 2PC)

Drawbacks:

Application has to handle eventually consistent data

Application has to handle duplicate events

@crichardson

How do services publish events?

@crichardson

To maintain consistency the application must

atomically publish an event whenever

a domain object changes

@crichardson

How to generate events reliably?

Database triggers

Hibernate event listener

Ad hoc event publishing code mixed into business logic

Domain events - “formal” modeling of events

Event Sourcing

@crichardson

Atomically publishing eventsUse distributed transactions to update database and publish to message

Database and message broker must support 2PC

2PC is best avoided

Use eventual consistency mechanism:

1. Update database: new entity state & event to publish

2. Publish event & mark event as published

• Difficult to implement when using a NoSQL database :-(

@crichardson

Event sourcingAn event-centric approach to designing domain models

Request becomes a command that is sent to an Aggregate

Aggregates handle commands by generating events

Apply events to an aggregate to update state

Persist events NOT state

Replay events to recreate the current state of an aggregate

Event Store ≃ database + message broker

@crichardson

Request handling in an event-sourced application

HTTPHandler

EventStore

pastEvents = findEvents(entityId)

Account

new()

applyEvents(pastEvents)

newEvents = processCmd(SomeCmd)

saveEvents(newEvents)

Microservice A

@crichardson

Event Store publishes events - consumed by other services

EventStore

EventSubscriber

subscribe(EventTypes)

publish(event)

publish(event)

Aggregate

NoSQLmaterialized

view

update()

update()

Microservice B

@crichardson

Using event sourcing

Event Store

CustomerCreditLimitUpdatedEvent

Order management

Order

total

Customer’

creditLimit

CustomerCreditLimitUpdatedEvent(...)

Customer management

Customer

creditLimit...

Customer Service

updateCustomer()

UpdateCreditLimitCommand

@crichardson

Customer aggregatecase class Customer(customerId: String, creditLimit: BigDecimal) extends ValidatingAggregate[Customer, CustomerCommands.CustomerCommand] {

def this() = this(null, null)

override def processCommand = { case CreateCustomerCommand(customerId, creditLimit) =>

Seq(CustomerCreatedEvent(customerId, creditLimit))

case UpdateCreditLimitCommand(newLimit) if newLimit >= 0 => Seq(CustomerCreditLimitUpdatedEvent(newLimit))

}

override def applyEvent = {

case CustomerCreatedEvent(customerId, creditLimit) => copy(customerId=customerId, creditLimit=creditLimit)

case CustomerCreditLimitUpdatedEvent(newLimit) => copy(creditLimit=newLimit)

}}

Command ⇒

Events

Event ⇒

Updated state

@crichardson

EventStore APItrait EventStore {

def save[T](entityId: Id, events: Seq[Event]): T

def update[T](entityId: Id, version: EntityVersion, events: Seq[Event]): T

def load[T](entityType: Class[T], entityId: EntityId): T

def subscribe(...) : ... ..}

@crichardson

Unfamiliar but it solves many problems

Eliminates O/R mapping problem

Supports both SQL and NoSQL databases

Publishes events reliably

Reliable eventual consistency framework

...

@crichardson

Summary

@crichardson

Monolithic applications are simple to develop and deploy

BUT have significant drawbacks

@crichardson

Apply the scale cube

Modular, polyglot, and scalable applications

Services developed, deployed and scaled independently

@crichardson

Use a modular, polyglot architecture

Model

View Controller Product Infoservice

RecommendationService

Reviewservice

REST

REST

AMQP

APIGateway

Model

View Controller

Server-side web application

Browser/Native application

@crichardson

Use an event-driven architecture

@crichardson

Start refactoring your monolith

Monolith ServiceAnti-corruption layer

Glue code

Pristine

@crichardson

Questions?

@crichardson chris@chrisrichardson.net

http://plainoldobjects.com http://microservices.io