CQRSWHY? WHAT? HOW?

JEPPE CRAMON - 2012

WHAT PROBLEMS ARE WE TRYING TO SOLVE?

Complexity of code and solution

Typical causes:• Too many layers • Big data models• Anemic domain model• Focus on frameworks instead of

on the domain• Scalability not considered at the

core of the design (scalability gets hacked in too late)

Time

Com

plex

ity

Typical One size fits all ”architecture”

Client

Remote Facade

Application Service

Data Access Layer

Data Storage

Domain Object

DomainObject

DTO DTO

Many perspectives on data

Customer

Online Ordering System

Pricing

Inventory

Sales

Product

Unit PricePromotional PricePromotion End Date

Stock Keeping Unit (SKU)Quantity On Hand (QOH)Location Code

PriceQuantity OrderedName

What about the lifecycles for the data?

One big model to capture it ALL

….…..…..

….…..…..

….…..…..

….…..…..

Result: Our queries get messy

The Query of Despair

ONE MODEL TO RULE THEM ALL

ONE MODEL TO FIND THEM

ONE MODEL TO BRING THEM ALL

AND IN THE DARKNESS BIND THEM

The database determines our scalability

And databases often scale poorly

To solve the performance problems for a few, you have to upgrade it all

Alternatively we can use Read replicas

Master Slave SlaveSlaveSlave

Read

Update

You’re now eventually consistent

Or introduce caching…

Read

Update

What’s your synchronization mechanism?

?

SOA / Integration is often an after thought

THE Solution?

Remember, what you see here is N

OT a silverbullit

It’s ONE solution to a set of sp

ecific problems

Same procedure as last year?

Same procedure as every year!!!

Open for new ideas?

Poul, you idiot, what's that!?You should only bring things we

can eat!

IT exists to support the business by support various usecases

Usecases can be categorized as either• ”User” intent to manipulate information• ”User” intent to find and read information

We already support this in OO at a Property level:• Setter (or mutator) methods Manipulate data• Getter (or accessor) methods Read data

Let’s raise it to a higher Level

CQS Separation of

functions that write &

functions that read

Functions that write are called Command methods and must not return a value

Functions that read are called Query methods and must have no side effects

CQS

UI Application Domain Data

Commands – Change data

UI Application Data

Queries – Ask for data (no side effects)

CQS from a code perspectivepublic class CustomerService{ // Commands void MakeCustomerPreferred(CustomerId) void ChangeCustomerLocale(CustomerId, NewLocale) void CreateCustomer(Customer) void EditCustomerDetails(CustomerDetails)

// Queries Customer GetCustomer(CustomerId) CustomerSet GetCustomersWithName(Name) CustomerSet GetPreferredCustomers()}

From: https://gist.github.com/1964094

SO WHAT IS CQRS?

CQRS is simply the creation of two objects where there was previously only one

Greg Young

CQRSIn its simplest form

public class CustomerWriteService{ // Commands void MakeCustomerPreferred(CustomerId) void ChangeCustomerLocale(CustomerId, NewLocale) void CreateCustomer(CreateCustomer) void EditCustomerDetails(CustomerDetails)}

public class CustomerReadService{ // Queries Customer GetCustomer(CustomerId) CustomerSet GetCustomersWithName(Name) CustomerSet GetPreferredCustomers()}

From: https://gist.github.com/1964094

CommandsA Command’s primary goal is to capture USER INTENT

A Command supports a single usecase and targets a single Aggregate

Commands are stated in the imperative:– DoStuff– CreateCustomer– ShipOrder– AddComment– DeleteComment

That’s all you need to know

Thanks for your attention

There’s always more than one model at play in any big project

But when code based on multiple models is combined

… then the code becomes filled with errors, unreliable and difficult to understand

Communication between team members becomes confusing.

It is often unclear in what context a model should NOT be used!

Context is lacking

CONTEXTThe setting in which a word or a statement

appears that determines its meaning

Smaller models & clear data ownership

SalesProduct

SKUNameDescriptionQuantity Ordered…

Sales

InventoryProduct

SKUQOHLocation Code…

Inventory

PricingProduct

SKUUnit PricePromotional Price…

Pricing

DDD:Bounded Context

SOA:Service

Online Ordering System

Entity identity

Core DDD patterns

ENTITIES

VALUE OBJECTS

AGGREGATES

Modu

les

Layere

d A

rch

itect

ure

Fact

ori

es

Rep

osi

tori

es

Serv

ices

BOUNDED CONTEXT

Let’s first get the vocabulary right

So we all speak the sameUBIQUITOUS LANGUAGE

Entities

Motivators: Objects are defined by identity and NOT by their attributes. May have lifecycle, can change form by time and context (A Person that becomes a father, who becomes a grand father, etc.)

Focus is on Who and not on what!

Value Objects

Motivators:Objects which describe things and are defined by their attributes (not by their identity) and doesn’t have a lifecycle.

Focus is on What and not on who!

Aggregates

What:• Cluster coherent Entities and Value Objects, with

complex associations into Aggregates with well defined boundaries.

• Choose one entity to be root and control access to objects inside the boundary through the root.

• External objects hold references to the root • Aggregates only refer to other aggregates by

identity (their id)

Motivation:Control invariants and consistency through the aggregate root.Enables: Loading schemes, coarse grained locking, consistency/transactional boundaries for DDDD

Root

How have our ”architectures” evolved

Thin Layers of complexity

UI Data

Pro: Easy and simple

Con: Scales poorly for complex domains

Language of the database

Next step – more layers

UI Application Domain Data

Pro: Handles complex domains better by Separating usecase and domain logic

Con: Increasing complexity. Risk of AnemicDomain model combined with transaction-script.

Time

Tran

sacti

on sc

ript

Real domain model

Com

plex

ity

Language of DTO’s

Language of getters and

setters

Language of ORM magic

Anemic Domain Model

The Good parts• All Order logic is called

in ONE class• Lower entry level for

junior developers

The bad parts• Hard to test• Hard to maintain• Code dupliction• An exposed domain model is like

walking around without clothes – everyone can access your private parts and violate your invariants

We need a rich domain model

Aggregate with Order as Aggregate Root

• Order forms our Unit of consistency, which controls our invariants• Order and OrderLines are created, updated and deleted together

LET’S HAVE A LOOK AT A DIFFERENT TYPE OF ARCHITECTURE

Circular architecture

Separation of Concerns

Task basedUI

Command

Domain ModelRead Model

Query

1. RealizationCommands and Queries

support very different usecases

A single model cannot be appropriate for reporting, searching and transactional behavior

Greg Young, 2008

2. realizationQuery results contain only data – no logic

• Give me the customer with his last 10 orders• Give me the customer with total sum of orders

for the last year• Give me the complete order• Give me the shipping status for the order• Give me the customers last review

So why go through the domain layer?UI Application Data

Things to think about in relation to Queries

• Why take the pain of performing JOINS and/or UNIONS of our nicely normalized Relational Model?

• Why not having a completely denormalized data model (or View) to support our Query?

• When we completely denormalize our data, do we really need a relational database or could we use a Key/Value- or Document Store?

• Do we need our Read Data to up to the microsecond consistent with our Write data?

So how is this relevant in the real world?

Let’s look at a real life scenario

Collaborative domains

1. Read 2. Read

4. Update3. Coffee time

5. Stale data

We’re working with stale data

Optimist

ic Lo

cking Exc

eption

We’re always working with stale data!

20 ms

1 ms

100 ms

100 ms100 ms

1 ms

20 ms

At this point our data is at least120 ms stale

+Thinking time

+ thinking time (1000-1500 ms)

LET’S USE STALE DATA TO OUR ADVANTAGE BY OFFLOADING THE DATABASE BY USING OUR READ

MODELS

UI Application Domain Write model

Commands – Change data

UI Application Read models

Queries – Ask for data (no side effects)

ALL WE NEED IS A GOOD WAY TO SYNCHRONIZE OUR WRITE AND READ MODELS

UI Application Domain Write model

Commands – Change data

UI Application Read models

Queries – Ask for data (no side effects)

Let’s make the implicit explicit!The original DDD book left out a very important concept:

Domain EventsWhich:• Signal that something happened• Closely aligned to the Domain Model• Are handled by a messaging system• They are in the past tense:

– CustomerBilled– ParcelShipped– CustomerCreated– ReviewCreated– CommentAdded– CommentDeleted

Commands & Events

• Commands mutate Aggregate state which results in one or more Events being issued/published.

Command Event(s)

AcceptOrder OrderAccepted

ShipOrder OrderShipped

AddComment CommentAdded

QuarantineReview ReviewQuarantined

UnquarantineReview ReviewUnquarantined

With Domain Events we now have a mechanism to support our denormalized View/Query models

Read modelRead model

Commands, Events and Query Models

Commands

Events

Queries

UI

Domain modelQuery model

”AcceptOrder”command

”OrderAccepted”event

”Find all Accepted Orders”

Query

Commands are Imperative: DoStuffEvents are Past tense: StuffDone

Messaging ArchitecturesMost CQRS implementations see Commands and

Events as (asynchronous) Messagespublic class CreateCustomer{ public final Guid CustomerId; public final Name CustomerName; …}

public class CustomerCreated{ public final Guid CustomerId; public final DateTime CreationDateTime; public final Name CustomerName;}

Command

Event

Difference:

INTENT

CQRS Building blocks

Client

CommandsCommand

Bus

Sends

Command Handlers

Modify

Repositories

Read Write

DatastoreEvent

Bus

Publish Events

EventBus

Command Services

Event HandlersEvents

Read storeQuery Facade

Query HandlersQuery Results

Queries

Query Services

Events

Domain

LET’S TAKE IT A STEP FURTHER

Event Sourcing

Aggregates track their own Domain Events and derive state from them

OrderCreated ProductAdded ProductAdded ProductRemoved ProductAdded OrderShipped

October

5October

6October

6October

7October

7October

9

Full CQRSWith EventSourcing

UI Domain

EventStore

Commands – Change data

Commands Events

SQL DB Document DB Graph DB

UI Data

Queries – Ask for data

EventsQuery Build

Our single source of truth

public class CustomerCommandHandler { private Repository<Customer> customerRepository;

public CustomerCommandHandler(Repository<Customer> customerRepository) { this.customerRepository = customerRepository; }

@CommandHandler public void handle(UnsignCustomer cmd) { Customer customer = repository.load(cmd.getCustomerId()); customer.unsign(); }}

public class Customer { private boolean signedUp;

public void unsign() { if (signedUp) { apply(new CustomerUnsignedEvent()); } } @EventHandler private void handle(CustomerUnsignedEvent event) { signedUp = false; }}

Testing CQRS BDD style@Testpublic void a_signedup_customer_can_unsign() { UUID customerId = UUID.randomUuid().toString();

FixtureConfiguration fixture = Fixtures.newGivenWhenThenFixture(); fixture.registerAnnotatedCommandHandler( new CustomerCommandHandler(fixture.createGenericRepository(Customer.class)) ); fixture.setAggregateIdentifier(customerId);

fixture .given( TestFactory.customerCreatedEvent(customerId), TestFactory.customerSignedUpEvent(customerId) ) .when( TestFactory.unsignCustomerCommand(customerId) ) .expectEvents( TestFactory.customerUnsignedEvent(customerId) );}

CQRS / BDD Report generated based on the previous test

Scenario: A signedup customer can unsign

Given a Customer with id ”abc1234” that has been created and a customer with id ”abc1234” that is signedup

When a request for Customer with id ”abcd1234” to be unsigned is received

Then the Customer with id ”abcd1234” is unsigned

SOA / Integration

Commands, Queries and Events form natural integration interfaces

So integration is backed in from the beginning!

What about Scalability?CAP Theorem• Consistency: All nodes in the cluster see

exactly the same data at any point in time• Availability: Failure of a node does not render

the entire system inoperative• Partition tolerance: Nodes can still function

when communication with other groups of nodes is lost

You can’t have all three!

CAP theorem

Source: http://bit.ly/QUurnY

Strong Weak Eventual Consistency

ACID systems are hard and expensive to

scale

Latency concerns

Unless you use Pessimistic

Locking – all data is stale

(and eventual consistent

when delivered to the user)

BASE Basically Available Soft-state Eventually consistent

Eventually Consistency levels:• Causal consistency: This involves a signal being sent from

between application session indicating that a change has occurred. From that point on the receiving session will always see the updated value.

• Read your own writes: In this mode of consistency, a session that performs a change to the database will immediately see that change, even if other sessions experience a delay.

• Monotonic consistency: In this mode, A session will never see data revert to an earlier point in time. Once we read a value, we will never see an earlier value.

See http://www.allthingsdistributed.com/2008/12/eventually_consistent.html for more

CQRS can give us BASE at the architectural level through

asynchronous Commands and Events

This gives the business and IT control over where to spend money to scale

our solution - instead of trying to buy a bigger database server.

CQRS requires a New UI paradigmeTask-based/Inductive UI

• A depart from the traditional WHAT UI (CRUD)• Task based UI’s focuses on HOW the user wants

to use the application• Guides users through the work process• The UI becomes an intrinsic part of the design• The UI design directly affects our commands

and thereby our transactional boundaries• Should be preferably use asynchronous

Commands

What about Validation

• Happens in the UI before Commands are sent• Typically validated using Read models• Helps to ensure that commands don’t often

fail • Validates simple things (values ranges, unique

keys/values)• Doesn’t enforce business logic rules

Circular architecture

Separation of Concerns

Task basedUI

Command

Domain ModelRead Model Event Event Event

Query

Separation of Concerns

• Different developer skills and roles for command and query parts

• New Business Requirements ripples controllably between areas

• No need for Getters & Setters in your domain model

• Testability – Event based testing is very easy

Somethings missing

How do we collaborate?

CQRS is not top level architecture

Composite UI’sOnline Ordering System

Sales Pricing Inventory

Web/Applicationtier

Background servertier

Storage tier

Composite UI

UI LayeredApplicationData Access Layer

Read

mod

el

Writ

e m

odel

Composite UI - example

That covers data presentation

What about transactions?

Next Step – Event Driven Architecture

SalesProduct

SKUNamePriceQuantity Ordered…

Inventory Service (SAP)Product

SKUQOHLocation Code…

Pricing ServiceProduct

SKUUnit PricePromotional Price…

Inventory

Pricing

Sales

Customers

New SKU Event

New SKU Event

New SKU Event

Order Accepted

Event

Mes

sage

Bus

Who coordinates the sales process?

Online Ordering System

Web Shop(Composite UI)

Introducing Sagas

Sales ServiceOrder

Accepted

Billing Service

Shipping Process Manager

(Saga)

Shipping Service

Online Ordering System

Mes

sage

Bus

Order Accepted

Order Accepted

Customer Billed

Customer Billed

ShipOrder

ShipOrder

So what do we do?

We have this dream

Sales ServiceOrder

Accepted

Billing Service

Shipping Process Manager

Shipping Service

Mes

sage

Bus

Order Accepted

Order Accepted

Customer Billed

Customer Billed

ShipOrder

ShipOrder

”Nirvana”Sales Pricing Inventory

Web/Applicationtier

Background servertier

Storage tier

Composite UI

UI LayeredApplicationData Access Layer

Read

mod

el

Writ

e m

odel

…and this realitySystem X System Y

System Z SAP

Finding strategic value

Reimplementation

Years go by…Still relevant?

Perhaps a more viable solution?

Big Ball Of Mud

What path to take?

1. Bubble context

Business Application

New Context Z

Repository Repository

CoordinatorTransformation

X ↔ ZTransformation

Y ↔ Z

System X

System Y

Data

Service Repository

Anti Corruption Layer

Bubbles burst

• Functionality gets absorbed into the legacy systems• Maintenance becomes too expensive• Expansion becomes too cumbersome• People start bypassing the ACL (encapsulation breaks)• …• Performance becomes a problem

2. A new hope

Business ApplicationNew Context Z

Repository Repository

CoordinatorTransformation

X ↔ ZTransformation

Y ↔ Z

System X

Data

System Y

Service Repository

Anti Corruption Layer

Data

Asynchronous updates

2. ACL Synchronization

3. ACL backed Open Host serviceNew Context B

Service

CoordinatorTransformation

A ↔ B

System A

Anti Corruption LayerPu

blished La

nguage

4. Event Channels

Event Channel Published Language

New Context Z

Repository Repository

Data

CoordinatorTransformation

X ↔ Z

Anti Corruption Layer

System X

System Y

Service Repository

EventsEvents

Events

Query Update

Messagebus

Questions?@TigerTeamDK on TwitterEmail: jeppe@tigerteam.dk

Web: http://www.tigerteam.dk @jeppec on Twitter

Useful linksCQRS/DDD:• Bounded Contexts and Context Maps: http://www.infoq.com/articles/ddd-contextmapping• Rinat Abdullings Getting Started with CQRS: http://abdullin.com/cqrs/• CQRS journey: https://github.com/mspnp/cqrs-journey-doc/• Greg Youngs old blog: http://codebetter.com/gregyoung/• Greg Youngs new blog: http://goodenoughsoftware.net• Greg Youngs CQRS info site: http://cqrs.wordpress.com• Sagas: http://www.udidahan.com/2009/04/20/saga-persistence-and-event-driven-architectures/• Clarified CQRS: http://www.udidahan.com/2009/12/09/clarified-cqrs/• When to avoid CQRS: http://www.udidahan.com/2011/04/22/when-to-avoid-cqrs/• Why you should be using CQRS almost everywhere:

http://www.udidahan.com/2011/10/02/why-you-should-be-using-cqrs-almost-everywhere…/• Videos from DDD exchange 2011: http://skillsmatter.com/event/design-architecture/ddd-exchange-2011• Fohjin DDD example: https://github.com/MarkNijhof/Fohjin/tree/master/Fohjin.DDD.Example• Axon Framework: http://www.axonframework.org/• Implementing Domain Driven Design (IDDD):

http://my.safaribooksonline.com/book/project-management/9780133039900• CQRS Info: www.cqrs.nu• Greg Youngs EventStore: http://geteventstore.com