Microservices Patterns

Chapter 1 - Escaping monolithic hell Monolithic hell

Complexity is overwhelming Shear size slows Development speed

built time Code merging takes lot of effort

Effort to build a stable release means releases slower, which means releases done less frequently

Working how to scale solution is a lot more complex Reliability - a bug anywhere in the solution has potential to bring

everything to its knees Commitment to a tech stack for the long term - difficult to try

different tech How microservices can help

Scaling cube Y axis = functional decomposition

Functional decomposition X axis = horizontal duplication aka scaleout

Multiple instances + load balancer Z axis = scaling by data partitioning

Multiple instances again Each instance takes on specific data set Fronted by smart routing

New approach to ensuring modularity Data store separation

Doesn’t mean separate dB servers Pros

Enablement of CI-CD Each service small and maintainable Independent deployment Independent scaling Autonomous teams Trial new tech more easily Better fault isolation

Cons Finding right set of services is challenging

Get it wrong you get a distributed monolith Distributed systems are more complex need for IPC etc

Multiple versions of the same service Automation for CI/CD Additional tech such as Docker, kubernetes, opensihift etc

Distributed systems are complex Deployment of features that span multiple services need careful

consideration Understanding dependency chains more complex

Deciding when to adopt microservices can be difficult Deciding when to use microservices

Early solution may not encounter problems that Desmond’s microservices

Starting immediately using microservices will mean paying a premium

Although refactoring into microservices can be hard because of tangled dependencies

“ startup should almost certainly begin with a monolithic application”

Microservice pattern language No silver bullet

Jonathon Haida -The Righteous Mind - Why Good People Are Divided by Politics and Religion

Suck/Rock dichotomy Neal Ford Fred Brooks - Mythical Month

Pattern characteristics Force

Problem in a context Resulting context

Benefits Drawbacks Issues

Related patterns - have relationships characterised by.. Predecessor Successor Alternative Generalization Specialization Example:

Patterns form a pattern language A set of related patterns that solve problems within

a domain often forms what is termed a pattern language

Pattern language came from Christopher Alexander - reslworld architect

Made up from ... Forces Resulting context

Benefits Drawbacks Issues

Related patterns Predecessor Successor Alternative Generalization Specialization

Communication patterns Need to decide the kind of IPC used. Options...

Async messaging Sync HTTP / REST Sync RPC

Need to decide hoe to do registry and discovery Inter communication via a gateway or directly How to handle data consistency

2PC not really practicL How to join data across microservices now they have their

own data stores Need patterns for deployment as there will many pieces to

deploy and possible dependency issues Observability becomes more important now executions can

span more environments Processes and organization

Mythical Man Month highlights communication overhead of a size n team is O(N^2)

Team no more than 8-12 people and focussed on business oriented goals

Cross functional teams so can develop, test & deploy services with minimal cross team comms

Jez Humble3 defines continuous delivery as follows:Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way

Need to move fast without breaking things Human dimension...

People are emotional things Ignore emotions and things get bumpy Concept of transition by William & Susan Bridges

describe how people emotionally react to change. The model has the following stages...

Ending, losing, letting go

Emotional upheaval & resistance Out of comfort zone Feel threatened e.g central team work

gets distributed Disruption to established teams

Neutral zone

Moving from one way of things to another

Struggle to change to new ways

New beginning Change is embraced

Chapter 2 -Decomposition Strategies Avoid god classes, they can lead to...

Tangled dependencies Prevent decomposition

Architecture importance because it draws out the NFRs Communication by...

APIs RPC Messaging Via DB. Is discouraged

Identifying the system ops process is inspired by object-oriented design process described in

Craig Larman’s book Applying UML and Patterns

creates the high-level domain model consisting of the key classes Establishes vocabulary Domain model is a simplification of what needs to be

implemented - divergent from Eric Evans

domain model is derived primarily from the nouns of the user stories and the system operations are derived mostly from the verbs

the system operations and describe each one’s behavior in terms of the domain model.

Can be charactized as either Commands - operations typically CUD Queries that read data

Use the verbs in the scenarios / use cases to help identify Commands have

Parameters Return values Behaviours in terms of the classes in the model Preconditions - conditions that must be valid before

the operation is invoked Post conditions - statements of truth after the

operation has been invoked An example

Decomposing into services Decompose by business capability

Identify business capabilities, consider... Purpose Structure Processes

Once capabilities defined it is possible to define the service(s) that go with it

Benefit is if organisation is stable then the architecture will also have stability - Conway’s Law

Architecture may evolve reflecting ‘how’ processes work Scenarios to understand service collaboration

Assign operations to services Sometimes assignments are not obvious so look at

the relationships There are some problems that will need to be addressed

Synchronous IPC reduces availability Operations spanning multiple services create need

for distributed transactions

Domain Driven Design can help break the occurrence of god classes. God classes limit decomposition

Subdomains Bounded context Each subdomain has its own version of a god class

which will be a smaller set than if the object is used across the entire solution

OOW approach defined in Designing Object Oriented C++ Applications Using The Booch Method can be applied

Single responsibility principle A class should have only one reason to change.

–Robert C. Martin If a class has multiple responsibilities that change

independently it will be unstable Common closure principle

The classes in a package should be closed together against the same kinds of changes. a change that affects a package affects all the classes in that package.– Robert C. Martin

If classes change in lock step they can be in the same package

Classes likely to implement different aspects of a business rule

Article Principles of Object Oriented Designbutunclebob.com/ArticleS.UncleBob.PrinciplesOfOod

Chapter 3 - Interprocess communication IPC selection can be very influential to solution Favour async comms internally Sync comms with outside world APIs evolve

Backward compatibility Compatibility breaking changes Semantic versioning

gRPC Http://en.m.wikipedia.org/wiki/Remote_procedure_call Provides cross language RPC framework

Need to handle partial failure Netflix approach https://medium.com/netflix-techblog/fault-tolerance-in-a-high-

volume-distributed-system-91ab4faae74ahttps://medium.com/netflix-techblog/fault-tolerance-in-a-high-volume-distributed-system-91ab4faae74a

Network timeouts - never block indefinitely and always use timeouts when waiting for a response. Using timeouts ensures that resources are never tied up indefinitely.

Limiting the number of outstanding requests from a client to a service - impose an upper bound on the number of outstanding requests that a client can make to a particular service. If the limit

has been reached, then it is probably pointless to make additional requests, and those attempts should fail immediately.

Circuit breaker pattern - track the number of successful and failed requests and if the error rate exceeds some threshold then trip the circuit breaker so that further attempts fail immediately. If a large number of requests are failing then it suggests that the service is unavailable and that sending more requests is pointless. After a timeout period, the client should try again and, if successful, close the circuit breaker.

Netflix Hystrix Implements these patterns https://github.com/Netflix/Hystrix

https://github.com/Netflix/Hystrix .net implementation can be seen in Polly

https://github.com/Netflix/Hystrixhttps://github.com/Netflix/Hystrix

Service discovery Platform provided such as Docker and Kubernetes based envs DNS/VIP

Async message based Implementing request/reply Scaling through competing consumers Detect and discarding dupes Sending and receiving message as part of dB transaction See Hhope & Woolfe Message brokers such as

Activemq Rabbitmq Kafka Kenesis AWS SQS

When choosing a broker, consider Message ordering Delivery guarantees Persistence Durability Scalability Latency Competing consumers

Chapter 4 - Managing Transactions with Sagas The challenge in a microservice environment is that data may now be

owned by different services the de facto standard for distributed transaction management is

X/Open Distributed Transaction Processing (DTP) Model (X/Open XA). XA uses 2 Phase Commit (2PC)

https://en.wikipedia.org/wiki/X/Open_XA Not all dbs Support XA such as MongoDB and Cassandra Message brokers such as RabbitMQ and Kafka don’t support XA Distributed transactions utilise synchronous IPC, which raises

issues of availability

CAP theorem, which states that a system can only have two of the following three properties: consistency, availability and partition tolerancehttps://en.m.wikipedia.org/wiki/CAP_theorem

Saga - message driven sequence of transactions Spring provides declarative framework for managing a transaction using

@Transactional Saga

Sequence of individual transactions scoped to the service Next step runs on the completion of the previous 1

Not acid and transaction failures will be more complex to handle Must be able to rollback through the use of compensating

transactions Compensating transactions need to run in the reverse order Saga can be driven by ...

Choreography Must have reliable transport Participant needs to be able to map saga event to its

own data Participants publish additional information in the

saga events to enable other participants Benefit of this approach

No additional frameworks are needed (assuming choreography is already possible)

Coupling remains lose Cons

Cyclic dependencies introduced as 1 service needs to subscribe to another’s event for do and undo

Data structures could become more complex as state information needs to be handed around

One one place that can determine validity of the transaction requested as the knowledge is distributed

Clarity on how the saga works is a lot harder to follow

Or orchestration Define an orchestrator entity whose sole job is to tell

the participants what to do Uses a command/reply model of interaction Model the saga orchestrator as a state machine Pros

Testing a saga is easier as you just exercise the state engine

It is possible to stop a saga prematurely e.g. if a customer cancels an order

As the saga is a series of mini transactions this has some wider impactss

The changes resulting from each microservice are visible before the saga is complete having potential business logic impacts

Multiple concurrent sagas means the transactions will be interwoven

Will need to handle the possibility that states are indeterminate whilst a saga has not completed

Before a saga can implement change it could try obtain locks on all entities it needs - this eases the issue of failing because of changing state

Lock model does have the change of risking deadlocks Chapter 5 - Business Logic in microservices

Inbound and outbound layers extrapolate away the implementation/adaptor work

Need to decide if the business service will follow either ... OO approach Procedural approach

Transaction script pattern Ideal for simple business needs Defined by Martin Fowler 2002 1 method that handles each event type Keep these classes separated from classes handling stored state Be aware of the pitfall of letting the logic grow and become

unwieldy Domain model pattern

Aligns more to OO Classes smaller simpler, do one thing Potential for classes to reflect real world making design more

understandable

Risk of domain boundaries not being clear e.g. deleting an entity which has dependent entities e.g. a order, has line items, fiscal info etc

Can overcome problem by aggregating entities to definitive boundaries aggregate

Aggregate rules #1 Reference only the aggregate root - only the roo

part of the aggregate can be referenced externally e. Order not order line

#2 inter-aggregate references must use primary keys - each part identifies related elements by an unique Id eg in a ship aggregate the hull is referenced by hullId

#3 one transaction creates or updates one aggregate Adaptor invokes Service object which then drives relevant sagas

which then interact with individual entities within the aggregate Publish domain events

Reasons for events ... Where replicas need to be maintained those services know

that data has changed Via a webhook or message broker next step in a process can

be triggered Trigger async user processes such as emails that order has

completed Monitoring of prcoesses Enable the analysis of events

Events made up typically of object ids an£ meta data Details of the event may be in the header/envelope and the event

doesn’t carry a payload Consider enriching the event with the object data to alleviate the

need for interested parties to query the entity Event storming as a technique to tease out the events that maybe

or are needed. Identify event triggers Systems that maybe interested How time effects the solution

Events need to be treated as transactional as data - could be achieved by writing to an outbox table which is then used by a message broker

Chapter 6 - Developing business logic with event sourcing Persist an aggregate as a sequence of events Each event is a state change of the aggregate Benefits ...

There is a history of transactions Current state can be achieved by replaying the history of events

Downsides ... Learning curve Querying the event store can be difficult Use of Command Query Responsibility (CQRS) pattern

Weaknesses of relational model When using an ORM there is the object-relational impedance

mismatch Applicability of ORM is a polarised debate blogs.tedneward.com/post/the-vietnam-of-computer-

science/ Lack of aggregate history Implementing audit logging is tedious and error prone Event publishing is bolted onto business logic

To create an aggregate using event sourcing ...

Load the events of the aggregate

Create the aggregate object using default constructors

Iterate through the events applying them to the aggregate Creating events is the key, so command such as order x translates to ...

Validate the command object Translate the command object to a series of state changes Apply the state changes with the corresponding events being

generated Concurrency handled by optimistic locking

Versioning numbering on traditional storage Can apply the same to eventing OR use number of events as a

proxy When an aggregate has lots of events then it can become unperformant,

so store snapshots and work foreword ffq Complex structures will benefit from the momento pattern https://en.wikipedia.org/wiki/Memento_pattern

Message consumers should be idempotent so if a message broker ends up delivering the same message more than once this can be handled

Using a relational dB this can be done by inserting events into a table with part of the message providing a key. If the key exists then done

NoSQL stores need a different approach, here an Id is generated algorithmically and added when writing the object. It can therefore look to see if such an I’d already exists

What happens when it comes to evolving schemas/data models - as replaying from the 1st event to the latest could mean structural changes in the data representations

Conceptually an event sourcing schema is made up of 3 levels Top level contains one or more aggregates The 2nd level represents the events an aggregate emits Lowest level contains the structure of the events

Achieve change by upcasting the entities institute much like a traditional relational schema change. Don’t recreate the events.

Approach benefits.. Reliably publication of domain events Preserves the history Avoids O/R impedance issues

Chapter 11 - Deploying Microservices

Language specific package pattern Language based deployment mechanisms such as war files Pros

Fast deployment Efficient resource utilisation

Cons Lack of encapsulation of the tech stack Limitations of resource constraining deployments Lack of service isolation on the same instance Automation of resource placement can get complex

Using VMs Pros

Tech stack encapsulation Service instance isolated Leverage mature cloud infra

Cons Less efficient in resource utilization Deployment relatively slow Says admin overhead

Using containers Base docker image Dockerfiles Docker registry Docker in Action by Jeff Nickoloff Kubernetes - as an orchestration framework should cover

Resource management Scheduling Service management Make up of kubernetes

Each node has Kube -proxy - load balancing across kublets Kublet(s) - manages a pod on a node

Master server has API server for rest interface Etcd - stores config Scheduler

Controller manager Concepts

Pod - basic deployment unit made up of one or more containers that share an IP and storage volume

May include one or more sidecar containers to provide supporting functions e.g. Ng ins to do a git pull

Pods are ephemeral Deployment - declarative spec of a pod Service - static end point with own IP and

DNS Chapter 10 production ready

Key capabilities Configurability

Connections to other services and utilities such as dB as they will change based on deployment

Get config at runtime Push approach

Runtime passes in the configurations e.g en vars, providing config files

Comes with the challenge of how to get a running system reconfigured

Risk that configuration info gets scattered across multiple config files and related sources

Pull model - get info from a config server

Options include Git DB such as a noSQL DB Special servers inc. Spring Config

Server, Hashicorp Vault, AWS Parameter vault

Benefits are ... Centralised Configuration data Transparent encrypt/decrypt Dynamic configuration change

Downside is another piece of infrastructure needed if not available as part of the platform

Observability Heath check APIs - endpoint that can provide app health

info There are circumstances when a service maybe

running, but should not receive calls e.g. exhausted dB connections, running but still initialising

Https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/implement-resilient-applications/monitor-app-health

Example spring boot actuator Implements an endpoint called /health Implements a set of Services based upon the

infrastructure services used Health check request handler api may perform

things like checking the database connectivity Utilise HTTP codes to get status indicators Need to take into account...

How is the endpoint implemented End point may verify it can access its

external services Implement a synthetic transaction

Configuration management of the healthcheck

Deployment of the health check service No point in having health check end points if they’re

not used this can be achieved through the use of Eureka’s configuration. Docker and Kubernetes can be configured to exploit these ends points as well

Log aggregation - logs compiled to a central location Standardise on a logging library if possible Send log output to standard out - it’s easier to tap

into and consistent Logging infra is needed e.g. ELK Elasticsearch

Logstash Kibana Distributed tracking by tagging API calls with an I’d and

carrying the tag through all the services

Might consider Aspect Orientated Programming (AOP) to inject logging points

Spring Sleuth can realise AOP Stitch the trace I’d events together using tools like

Open Zipkin (Zipkin was originally from Twitter) Exception tracking - a central service to report exceptions

to Should be a rare event Important to provide root cause Better to have a service rather than just use logs and

analytics and alerting A dedicated service means it is possible just to flag

the root cause and not all the cascade issues - does require intelligence in the handling

ApplIcation metrics - expose endpoints that provide metrics about the application ands its behaviour and performance

Measure things like... Service latency Number of requests executed Business related values such as service

requests rejected, processed etc Metrics need to be gathered and made available for

visualization Springboot can make JVM metrics availabile

Audit logging - track specific auditable events such as things the user is doing

Audit logging in the business code Audit via AOP Event sourcing

Realise Services using a microservices chassis Examples...

Springboot and spring cloud Go Kit and Micro for GoLang

https://github.com/go-kit/kithttps://github.com/go-kit/kit

github.com/micro/micro Only downside is each language needs its own chassis Service mesh is a low level infra that mediates comms

Services and external apps

Implements capabilities such as distributed tracing, service discovery, load balancing etc

Examples... Linkerd Istio Conduit

Security Many aspects are no different to a monolith Some differences to monoliths when handling security

Can’t share credentials by handing around an in memory object - not practical

Using a DB to handle security and other session data breaks the dB decoupling

Do however need to be able to safely pass identity from service to service

Aspects needing to be addressed Authentication Authorisation Audit

Authentication strategies... Each service authenticates a user...

means they have to enter the microservice network 1st

1 mistake in the microservices implementation and environment is compromised

How to deal with different clients authenticate in different ways - not desirable for every service to handle different mechanisms

Gateway authenticates before forwarding 1 place to solve the problem correctly Consolidation of different auth processes in 1

place Provides a token to show authentication has

occurred Risk of gateway getting too tightly integrated

to authentication processes Each service implements access control lists Need to decide on the token framework to use

Opaque tokens often represented as UUIDs

Do reduce performance and increase latency as each service needs to talk to a security service to use UUID to retrieve user info

OR use a transparent token form such as JSON Web Tokens (JWT)

JWT sets out a stand way of representing claims such as identity and roles

JWT tokens are signed to prevent attempts to forge tokens

Because once a token is produced, no way of revoking it - have to allow it to expire

Recommended OAuth readinghttp://www.oauth.comhttp://www.oauth.com

https://www.oauth.com/oauth2-servers/access-tokens/password-grant/

OAuth Actors AuthorizationServer-

provides an API for authenticating users and obtaining an access token and a refresh token. Spring OAuth is a great example of framework for building an OAuth 2.0 authorization server

AccessToken-a token that grants access to a ResourceServer.The format of the access token is implementation dependent. However,

some implementations,such as Spring OAuth, use JWTs

RefreshToken-along-lived, yet revocable token that a Client uses to obtain a new AccessToken

ResourceServer-a service that uses an access token to authorize access.In a microservice architecture, the services are resource servers

Client- a client that wants to access a ResourceServer. In a microservice architecture, the API Gateway is the OAuth 2.0 clientProgress: 100%

Chapter 9- Testing Mcroservices

Chapter 8 - external API patterns External API Design considerations

Poor user experience due to the client making multiple requests Lack of encapsulation requires front end devs to change their code

in lock step with the backend Services might use client unfriendly IPC mechanisms

Open source gateway options Netflix Zuul

Rate limiting Authentication Filters https://github.com/Netflix/zuul

https://github.com/Netflix/zuul Spring Cloud Zulu - use spring mic for the composition, Zulu

for routing etc Spring cloud gateway

Builds on top of several components Spring boot Spring Spring webflux Project reactor

Edge functions such as authentication Routing to backends Request handlers https://cloud.spring.io/spring-cloud-gateway/

https://cloud.spring.io/spring-cloud-gateway/ Chapter 7 - implementing queries in a microservice architecture

As the data you may need could distributed across multiple services a simple query as you would in a monolith isn’t an option

Options... API composition pattern

Composer calls all the components and stitches the results together

Design considerations to take into account which component performs the composer, options

Client is composer If the client is inside the LAN and firewalls etc

then an option - otherwise not recommended

API composer performs query

Enables API to service external clients effectively

Composer as a stand-alone service

Ideal for supporting multiple internal services

Manage impact of latency etc by calling Services in parrallel if possible

API composers if possible use a reactive programming model e.g using tech such as ...

Java ComposableFutures RxJava Obseravbles

Pros Simple Intuitive

Cons

Overheads e.g multiple service calls Risk of reduced availability - mathematical

availability Mitigate availability by accessing cache when service

not available Transactional consistency at risk

CQRS Better for larger data sets

Bypass structure to database when performance is a real issue

CQRS - separate model to handle non trivial queries CQRS query model maintained by having event

handler Pros

efficient implementation of queries in a microservice architecture

efficient implementation of a diverse queries Makes querying possible in an event sourcing-based

application Improves separation of concerns

Cons More complex than api composition but also more

powerful Replication lag

Views API consisting of 1 or more query operations

Considerations Database selection

Schema design Managing concurrent updates Idempotent handling of updates When modifying or adding views need a

means by which data models built/rebuilt Replication lag