Upload
jeppe-cramon
View
9.369
Download
5
Tags:
Embed Size (px)
DESCRIPTION
This talk covers WHY we might want to look at CQRS, by first looking at the kinds of problems we see with the current architectural approaches. We will look at WHAT CQRS is, how it works well with DDD, look at staleness of data in collaborative domains and how we can use this to our advantage. We will look at the building blocks for CQRS, how to test it and how to build a bigger solution the involves communication between different parts of the solution/bounded contexts. We will look at how we can use CQRS to scale the solution (by looking at the CAP theorem and the BASE properties of distributed computing). Finally we will look at how we can approach legacy system migration by looking at 4 different alternatives to rewriting them from scratch. Key words are for this is published language, Anti Corruption layer and an Event Driven Architecture (which might utilize CQRS internally).
Citation preview
CQRSWHY? WHAT? HOW?
JEPPE CRAMON - 2012
WHAT PROBLEMS ARE WE TRYING TO SOLVE?
Complexity of code and solution
Typical causes:• Too many layers • Big data models• Anemic domain model• Focus on frameworks instead of
on the domain• Scalability not considered at the
core of the design (scalability gets hacked in too late)
Time
Com
plex
ity
Typical One size fits all ”architecture”
Client
Remote Facade
Application Service
Data Access Layer
Data Storage
Domain Object
DomainObject
DTO DTO
Many perspectives on data
Customer
Online Ordering System
Pricing
Inventory
Sales
Product
Unit PricePromotional PricePromotion End Date
Stock Keeping Unit (SKU)Quantity On Hand (QOH)Location Code
PriceQuantity OrderedName
What about the lifecycles for the data?
One big model to capture it ALL
….…..…..
….…..…..
….…..…..
….…..…..
Result: Our queries get messy
The Query of Despair
ONE MODEL TO RULE THEM ALL
ONE MODEL TO FIND THEM
ONE MODEL TO BRING THEM ALL
AND IN THE DARKNESS BIND THEM
The database determines our scalability
And databases often scale poorly
To solve the performance problems for a few, you have to upgrade it all
Alternatively we can use Read replicas
Master Slave SlaveSlaveSlave
Read
Update
You’re now eventually consistent
Or introduce caching…
Read
Update
What’s your synchronization mechanism?
?
SOA / Integration is often an after thought
THE Solution?
Remember, what you see here is N
OT a silverbullit
It’s ONE solution to a set of sp
ecific problems
Same procedure as last year?
Same procedure as every year!!!
Open for new ideas?
Poul, you idiot, what's that!?You should only bring things we
can eat!
IT exists to support the business by support various usecases
Usecases can be categorized as either• ”User” intent to manipulate information• ”User” intent to find and read information
We already support this in OO at a Property level:• Setter (or mutator) methods Manipulate data• Getter (or accessor) methods Read data
Let’s raise it to a higher Level
CQS Separation of
functions that write &
functions that read
Functions that write are called Command methods and must not return a value
Functions that read are called Query methods and must have no side effects
CQS
UI Application Domain Data
Commands – Change data
UI Application Data
Queries – Ask for data (no side effects)
CQS from a code perspectivepublic class CustomerService{ // Commands void MakeCustomerPreferred(CustomerId) void ChangeCustomerLocale(CustomerId, NewLocale) void CreateCustomer(Customer) void EditCustomerDetails(CustomerDetails)
// Queries Customer GetCustomer(CustomerId) CustomerSet GetCustomersWithName(Name) CustomerSet GetPreferredCustomers()}
From: https://gist.github.com/1964094
SO WHAT IS CQRS?
CQRS is simply the creation of two objects where there was previously only one
Greg Young
CQRSIn its simplest form
public class CustomerWriteService{ // Commands void MakeCustomerPreferred(CustomerId) void ChangeCustomerLocale(CustomerId, NewLocale) void CreateCustomer(CreateCustomer) void EditCustomerDetails(CustomerDetails)}
public class CustomerReadService{ // Queries Customer GetCustomer(CustomerId) CustomerSet GetCustomersWithName(Name) CustomerSet GetPreferredCustomers()}
From: https://gist.github.com/1964094
CommandsA Command’s primary goal is to capture USER INTENT
A Command supports a single usecase and targets a single Aggregate
Commands are stated in the imperative:– DoStuff– CreateCustomer– ShipOrder– AddComment– DeleteComment
That’s all you need to know
Thanks for your attention
There’s always more than one model at play in any big project
But when code based on multiple models is combined
… then the code becomes filled with errors, unreliable and difficult to understand
Communication between team members becomes confusing.
It is often unclear in what context a model should NOT be used!
Context is lacking
CONTEXTThe setting in which a word or a statement
appears that determines its meaning
Smaller models & clear data ownership
SalesProduct
SKUNameDescriptionQuantity Ordered…
Sales
InventoryProduct
SKUQOHLocation Code…
Inventory
PricingProduct
SKUUnit PricePromotional Price…
Pricing
DDD:Bounded Context
SOA:Service
Online Ordering System
Entity identity
Core DDD patterns
ENTITIES
VALUE OBJECTS
AGGREGATES
Modu
les
Layere
d A
rch
itect
ure
Fact
ori
es
Rep
osi
tori
es
Serv
ices
BOUNDED CONTEXT
Let’s first get the vocabulary right
So we all speak the sameUBIQUITOUS LANGUAGE
Entities
Motivators: Objects are defined by identity and NOT by their attributes. May have lifecycle, can change form by time and context (A Person that becomes a father, who becomes a grand father, etc.)
Focus is on Who and not on what!
Value Objects
Motivators:Objects which describe things and are defined by their attributes (not by their identity) and doesn’t have a lifecycle.
Focus is on What and not on who!
Aggregates
What:• Cluster coherent Entities and Value Objects, with
complex associations into Aggregates with well defined boundaries.
• Choose one entity to be root and control access to objects inside the boundary through the root.
• External objects hold references to the root • Aggregates only refer to other aggregates by
identity (their id)
Motivation:Control invariants and consistency through the aggregate root.Enables: Loading schemes, coarse grained locking, consistency/transactional boundaries for DDDD
Root
How have our ”architectures” evolved
Thin Layers of complexity
UI Data
Pro: Easy and simple
Con: Scales poorly for complex domains
Language of the database
Next step – more layers
UI Application Domain Data
Pro: Handles complex domains better by Separating usecase and domain logic
Con: Increasing complexity. Risk of AnemicDomain model combined with transaction-script.
Time
Tran
sacti
on sc
ript
Real domain model
Com
plex
ity
Language of DTO’s
Language of getters and
setters
Language of ORM magic
Anemic Domain Model
The Good parts• All Order logic is called
in ONE class• Lower entry level for
junior developers
The bad parts• Hard to test• Hard to maintain• Code dupliction• An exposed domain model is like
walking around without clothes – everyone can access your private parts and violate your invariants
We need a rich domain model
Aggregate with Order as Aggregate Root
• Order forms our Unit of consistency, which controls our invariants• Order and OrderLines are created, updated and deleted together
LET’S HAVE A LOOK AT A DIFFERENT TYPE OF ARCHITECTURE
Circular architecture
Separation of Concerns
Task basedUI
Command
Domain ModelRead Model
Query
1. RealizationCommands and Queries
support very different usecases
A single model cannot be appropriate for reporting, searching and transactional behavior
Greg Young, 2008
2. realizationQuery results contain only data – no logic
• Give me the customer with his last 10 orders• Give me the customer with total sum of orders
for the last year• Give me the complete order• Give me the shipping status for the order• Give me the customers last review
So why go through the domain layer?UI Application Data
Things to think about in relation to Queries
• Why take the pain of performing JOINS and/or UNIONS of our nicely normalized Relational Model?
• Why not having a completely denormalized data model (or View) to support our Query?
• When we completely denormalize our data, do we really need a relational database or could we use a Key/Value- or Document Store?
• Do we need our Read Data to up to the microsecond consistent with our Write data?
So how is this relevant in the real world?
Let’s look at a real life scenario
Collaborative domains
1. Read 2. Read
4. Update3. Coffee time
5. Stale data
We’re working with stale data
Optimist
ic Lo
cking Exc
eption
We’re always working with stale data!
20 ms
1 ms
100 ms
100 ms100 ms
1 ms
20 ms
At this point our data is at least120 ms stale
+Thinking time
+ thinking time (1000-1500 ms)
LET’S USE STALE DATA TO OUR ADVANTAGE BY OFFLOADING THE DATABASE BY USING OUR READ
MODELS
UI Application Domain Write model
Commands – Change data
UI Application Read models
Queries – Ask for data (no side effects)
ALL WE NEED IS A GOOD WAY TO SYNCHRONIZE OUR WRITE AND READ MODELS
UI Application Domain Write model
Commands – Change data
UI Application Read models
Queries – Ask for data (no side effects)
Let’s make the implicit explicit!The original DDD book left out a very important concept:
Domain EventsWhich:• Signal that something happened• Closely aligned to the Domain Model• Are handled by a messaging system• They are in the past tense:
– CustomerBilled– ParcelShipped– CustomerCreated– ReviewCreated– CommentAdded– CommentDeleted
Commands & Events
• Commands mutate Aggregate state which results in one or more Events being issued/published.
Command Event(s)
AcceptOrder OrderAccepted
ShipOrder OrderShipped
AddComment CommentAdded
QuarantineReview ReviewQuarantined
UnquarantineReview ReviewUnquarantined
With Domain Events we now have a mechanism to support our denormalized View/Query models
Read modelRead model
Commands, Events and Query Models
Commands
Events
Queries
UI
Domain modelQuery model
”AcceptOrder”command
”OrderAccepted”event
”Find all Accepted Orders”
Query
Commands are Imperative: DoStuffEvents are Past tense: StuffDone
Messaging ArchitecturesMost CQRS implementations see Commands and
Events as (asynchronous) Messagespublic class CreateCustomer{ public final Guid CustomerId; public final Name CustomerName; …}
public class CustomerCreated{ public final Guid CustomerId; public final DateTime CreationDateTime; public final Name CustomerName;}
Command
Event
Difference:
INTENT
CQRS Building blocks
Client
CommandsCommand
Bus
Sends
Command Handlers
Modify
Repositories
Read Write
DatastoreEvent
Bus
Publish Events
EventBus
Command Services
Event HandlersEvents
Read storeQuery Facade
Query HandlersQuery Results
Queries
Query Services
Events
Domain
LET’S TAKE IT A STEP FURTHER
Event Sourcing
Aggregates track their own Domain Events and derive state from them
OrderCreated ProductAdded ProductAdded ProductRemoved ProductAdded OrderShipped
October
5October
6October
6October
7October
7October
9
Full CQRSWith EventSourcing
UI Domain
EventStore
Commands – Change data
Commands Events
SQL DB Document DB Graph DB
UI Data
Queries – Ask for data
EventsQuery Build
Our single source of truth
public class CustomerCommandHandler { private Repository<Customer> customerRepository;
public CustomerCommandHandler(Repository<Customer> customerRepository) { this.customerRepository = customerRepository; }
@CommandHandler public void handle(UnsignCustomer cmd) { Customer customer = repository.load(cmd.getCustomerId()); customer.unsign(); }}
public class Customer { private boolean signedUp;
public void unsign() { if (signedUp) { apply(new CustomerUnsignedEvent()); } } @EventHandler private void handle(CustomerUnsignedEvent event) { signedUp = false; }}
Testing CQRS BDD style@Testpublic void a_signedup_customer_can_unsign() { UUID customerId = UUID.randomUuid().toString();
FixtureConfiguration fixture = Fixtures.newGivenWhenThenFixture(); fixture.registerAnnotatedCommandHandler( new CustomerCommandHandler(fixture.createGenericRepository(Customer.class)) ); fixture.setAggregateIdentifier(customerId);
fixture .given( TestFactory.customerCreatedEvent(customerId), TestFactory.customerSignedUpEvent(customerId) ) .when( TestFactory.unsignCustomerCommand(customerId) ) .expectEvents( TestFactory.customerUnsignedEvent(customerId) );}
CQRS / BDD Report generated based on the previous test
Scenario: A signedup customer can unsign
Given a Customer with id ”abc1234” that has been created and a customer with id ”abc1234” that is signedup
When a request for Customer with id ”abcd1234” to be unsigned is received
Then the Customer with id ”abcd1234” is unsigned
SOA / Integration
Commands, Queries and Events form natural integration interfaces
So integration is backed in from the beginning!
What about Scalability?CAP Theorem• Consistency: All nodes in the cluster see
exactly the same data at any point in time• Availability: Failure of a node does not render
the entire system inoperative• Partition tolerance: Nodes can still function
when communication with other groups of nodes is lost
You can’t have all three!
CAP theorem
Source: http://bit.ly/QUurnY
Strong Weak Eventual Consistency
ACID systems are hard and expensive to
scale
Latency concerns
Unless you use Pessimistic
Locking – all data is stale
(and eventual consistent
when delivered to the user)
BASE Basically Available Soft-state Eventually consistent
Eventually Consistency levels:• Causal consistency: This involves a signal being sent from
between application session indicating that a change has occurred. From that point on the receiving session will always see the updated value.
• Read your own writes: In this mode of consistency, a session that performs a change to the database will immediately see that change, even if other sessions experience a delay.
• Monotonic consistency: In this mode, A session will never see data revert to an earlier point in time. Once we read a value, we will never see an earlier value.
See http://www.allthingsdistributed.com/2008/12/eventually_consistent.html for more
CQRS can give us BASE at the architectural level through
asynchronous Commands and Events
This gives the business and IT control over where to spend money to scale
our solution - instead of trying to buy a bigger database server.
CQRS requires a New UI paradigmeTask-based/Inductive UI
• A depart from the traditional WHAT UI (CRUD)• Task based UI’s focuses on HOW the user wants
to use the application• Guides users through the work process• The UI becomes an intrinsic part of the design• The UI design directly affects our commands
and thereby our transactional boundaries• Should be preferably use asynchronous
Commands
What about Validation
• Happens in the UI before Commands are sent• Typically validated using Read models• Helps to ensure that commands don’t often
fail • Validates simple things (values ranges, unique
keys/values)• Doesn’t enforce business logic rules
Circular architecture
Separation of Concerns
Task basedUI
Command
Domain ModelRead Model Event Event Event
Query
Separation of Concerns
• Different developer skills and roles for command and query parts
• New Business Requirements ripples controllably between areas
• No need for Getters & Setters in your domain model
• Testability – Event based testing is very easy
Somethings missing
How do we collaborate?
CQRS is not top level architecture
Composite UI’sOnline Ordering System
Sales Pricing Inventory
Web/Applicationtier
Background servertier
Storage tier
Composite UI
UI LayeredApplicationData Access Layer
Read
mod
el
Writ
e m
odel
Composite UI - example
That covers data presentation
What about transactions?
Next Step – Event Driven Architecture
SalesProduct
SKUNamePriceQuantity Ordered…
Inventory Service (SAP)Product
SKUQOHLocation Code…
Pricing ServiceProduct
SKUUnit PricePromotional Price…
Inventory
Pricing
Sales
Customers
New SKU Event
New SKU Event
New SKU Event
Order Accepted
Event
Mes
sage
Bus
Who coordinates the sales process?
Online Ordering System
Web Shop(Composite UI)
Introducing Sagas
Sales ServiceOrder
Accepted
Billing Service
Shipping Process Manager
(Saga)
Shipping Service
Online Ordering System
Mes
sage
Bus
Order Accepted
Order Accepted
Customer Billed
Customer Billed
ShipOrder
ShipOrder
So what do we do?
We have this dream
Sales ServiceOrder
Accepted
Billing Service
Shipping Process Manager
Shipping Service
Mes
sage
Bus
Order Accepted
Order Accepted
Customer Billed
Customer Billed
ShipOrder
ShipOrder
”Nirvana”Sales Pricing Inventory
Web/Applicationtier
Background servertier
Storage tier
Composite UI
UI LayeredApplicationData Access Layer
Read
mod
el
Writ
e m
odel
…and this realitySystem X System Y
System Z SAP
Finding strategic value
Reimplementation
Years go by…Still relevant?
Perhaps a more viable solution?
Big Ball Of Mud
What path to take?
1. Bubble context
Business Application
New Context Z
Repository Repository
CoordinatorTransformation
X ↔ ZTransformation
Y ↔ Z
System X
System Y
Data
Service Repository
Anti Corruption Layer
Bubbles burst
• Functionality gets absorbed into the legacy systems• Maintenance becomes too expensive• Expansion becomes too cumbersome• People start bypassing the ACL (encapsulation breaks)• …• Performance becomes a problem
2. A new hope
Business ApplicationNew Context Z
Repository Repository
CoordinatorTransformation
X ↔ ZTransformation
Y ↔ Z
System X
Data
System Y
Service Repository
Anti Corruption Layer
Data
Asynchronous updates
2. ACL Synchronization
3. ACL backed Open Host serviceNew Context B
Service
CoordinatorTransformation
A ↔ B
System A
Anti Corruption LayerPu
blished La
nguage
4. Event Channels
Event Channel Published Language
New Context Z
Repository Repository
Data
CoordinatorTransformation
X ↔ Z
Anti Corruption Layer
System X
System Y
Service Repository
EventsEvents
Events
Query Update
Messagebus
Questions?@TigerTeamDK on TwitterEmail: [email protected]
Web: http://www.tigerteam.dk @jeppec on Twitter
Useful linksCQRS/DDD:• Bounded Contexts and Context Maps: http://www.infoq.com/articles/ddd-contextmapping• Rinat Abdullings Getting Started with CQRS: http://abdullin.com/cqrs/• CQRS journey: https://github.com/mspnp/cqrs-journey-doc/• Greg Youngs old blog: http://codebetter.com/gregyoung/• Greg Youngs new blog: http://goodenoughsoftware.net• Greg Youngs CQRS info site: http://cqrs.wordpress.com• Sagas: http://www.udidahan.com/2009/04/20/saga-persistence-and-event-driven-architectures/• Clarified CQRS: http://www.udidahan.com/2009/12/09/clarified-cqrs/• When to avoid CQRS: http://www.udidahan.com/2011/04/22/when-to-avoid-cqrs/• Why you should be using CQRS almost everywhere:
http://www.udidahan.com/2011/10/02/why-you-should-be-using-cqrs-almost-everywhere…/• Videos from DDD exchange 2011: http://skillsmatter.com/event/design-architecture/ddd-exchange-2011• Fohjin DDD example: https://github.com/MarkNijhof/Fohjin/tree/master/Fohjin.DDD.Example• Axon Framework: http://www.axonframework.org/• Implementing Domain Driven Design (IDDD):
http://my.safaribooksonline.com/book/project-management/9780133039900• CQRS Info: www.cqrs.nu• Greg Youngs EventStore: http://geteventstore.com