Reactive Design PatternsDr. Roland Kuhn

@rolandkuhn — CTO Actyx AG

Reactive Design Patterns

• currently in MEAP

• all chapters done,in pre-production

• use code 39kuhn (39% off),see http://rolandkuhn.com

2

Reactive?

Elasticity: Performance at Scale

4

Resilience: Don’t put all eggs in one basket!

5

Result: Responsiveness

• elastic components that scale with their load

• responses in the presence of partial failures

6

Result: Decoupling

• containment of • failures

• implementation details

• responsibility

• shared-nothing architecture, clear boundaries

7

Result: Maintainability & Fexibility

• decoupled responsibility—decoupled teams

• develop pieces at their own pace

• continuous delivery

• Microservices: Single Responsibility Principle

8

Implementation: Message-Driven

• focus on communication between components

• model message flows and protocols

• common transports: async HTTP, *MQ, Actors

9

Reactive Traits

10

elastic resilient

responsive maintainable extensible

message-­‐driven

Value

Means

Form

Architecture Patterns

Basically: Microservices Best Practices

• Simple Component Pattern • DeMarco in «Structured analysis and system specification»

(Yourdon, New York, 1979)

• “maximize cohesion and minimize coupling”

• Let-It-Crash Pattern • Candea & Fox: “Crash-Only Software” (USENIX HotOS IX,

2003)

• Error Kernel Pattern • Erlang (late 1980’s)

12

Implementation Patterns

Request–Response Pattern

14

«Include a return address in the message

in order to receive a response.»

Request–Response Pattern

15

Request–Response Pattern

• return address is often implicit: • HTTP response over same TCP connection

• automatic sender reference capture in Akka

• explicit return address is needed otherwise • *MQ

• Akka Typed

• correlation ID needed for long-lived participants

16

Circuit Breaker Pattern

17

«Protect services by breaking the

connection during failure periods.»

Circuit Breaker Pattern

• well-known, inspired by electrical engineering

• first published by M. Nygard in «Release It!»

• protects both ways: • allows client to avoid long failure timeouts

• gives service some breathing room to recover

18

Circuit Breaker Example

19

private object StorageFailed extends RuntimeExceptionprivate def sendToStorage(job: Job): Future[StorageStatus] = { // make an asynchronous request to the storage subsystem val f: Future[StorageStatus] = ??? // map storage failures to Future failures to alert the breaker f.map { case StorageStatus.Failed => throw StorageFailed case other => other }}

private val breaker = CircuitBreaker( system.scheduler, // used for scheduling timeouts 5, // number of failures in a row when it trips 300.millis, // timeout for each service call 30.seconds) // time before trying to close after tripping

def persist(job: Job): Future[StorageStatus] = breaker .withCircuitBreaker(sendToStorage(job)) .recover { case StorageFailed => StorageStatus.Failed case _: TimeoutException => StorageStatus.Unknown case _: CircuitBreakerOpenException => StorageStatus.Failed }

Multiple-Master Replication Patterns

20

«Keep multiple distributed copies,

accept updates everywhere,

disseminate updates among replicas.»

Multiple-Master Replication Patterns

• this is a tough problem with no perfect solution

• requires a trade-off to be made between consistency and availability • consensus-based focuses on consistency

• conflict-free focuses on availability

• conflict resolution gives up a bit of both

• each requires a different programming model and can express different transactional behavior

21

Consensus-Based Replication

• strong coupling between replicas to ensure that all are “on the same page”

• unavailable during network outages or certain machine failures

• programming model “just like a single thread”

• Postgres, Zookeeper, etc.

22

Replication with Conflict Resolution

• requires conflict detection

• resolution without user intervention will have to discard some updates

• detection/resolution unavailable during partitions

• programming model “like single thread” with caveat

• popular RDBMS in default configuration offer this

23

Conflict-Free Replication

• express updates such that they can be merged

• cannot express “non-local” constraints

• all expressible updates can be performed under any conditions without losses or inconsistencies

• replicas may temporarily be out of sync

• different programming model, explicitly distributed

• Riak 2.0, Akka Distributed Data

24

Multiple-Master Replication Patterns

• no one size fits all

• you will have to think and decide!

25

Saga Pattern

26

«Divide long-lived distributed

transactions into quick local ones with

compensating actions for recovery.»

Saga Pattern: Background

• Microservice Architecture means distribution of knowledge, no more central database instance

• Pat Helland: • “Life Beyond Distributed Transactions”, CIDR 2007

• “Memories, Guesses, and Apologies”, MSDN blog 2007

• What about transactions that affect multiple microservices?

27

Saga Pattern

• Garcia-Molina & Salem: “SAGAS”, ACM, 1987

• Bank transfer avoiding lock of both accounts: • T₁: transfer money from X to local working account

• T₂: transfer money from local working account to Y

• C₁: compensate failure by transferring money back to X

• Compensating transactions are executed during Saga rollback

• concurrent Sagas can see intermediate state

28

Saga Pattern

• backward recovery:T₁ T₂ T₃ C₃ C₂ C₁ • forward recovery with save-points:

T₁ (sp) T₂ (sp) T₃ (sp) T₄ • in practice Sagas need to be persistent to

recover after hardware failures, meaning backward recovery will also use save-points

29

Example: Bank Transfer

30

trait Account { def withdraw(amount: BigDecimal, id: Long): Future[Unit] def deposit(amount: BigDecimal, id: Long): Future[Unit]}

case class Transfer(amount: BigDecimal, x: Account, y: Account)

sealed trait Eventcase class TransferStarted(amount: BigDecimal, x: Account, y: Account) extends Eventcase object MoneyWithdrawn extends Eventcase object MoneyDeposited extends Eventcase object RolledBack extends Event

Example: Bank Transfer

31

class TransferSaga(id: Long) extends PersistentActor { import context.dispatcher

override val persistenceId: String = s"transaction-$id"

override def receiveCommand: PartialFunction[Any, Unit] = { case Transfer(amount, x, y) => persist(TransferStarted(amount, x, y))(withdrawMoney) }

def withdrawMoney(t: TransferStarted): Unit = { t.x.withdraw(t.amount, id).map(_ => MoneyWithdrawn).pipeTo(self) context.become(awaitMoneyWithdrawn(t.amount, t.x, t.y)) }

def awaitMoneyWithdrawn(amount: BigDecimal, x: Account, y: Account): Receive = { case m @ MoneyWithdrawn => persist(m)(_ => depositMoney(amount, x, y)) }

...}

Example: Bank Transfer

32

def depositMoney(amount: BigDecimal, x: Account, y: Account): Unit = { y.deposit(amount, id) map (_ => MoneyDeposited) pipeTo self context.become(awaitMoneyDeposited(amount, x))}

def awaitMoneyDeposited(amount: BigDecimal, x: Account): Receive = { case Status.Failure(ex) => x.deposit(amount, id) map (_ => RolledBack) pipeTo self context.become(awaitRollback) case MoneyDeposited => persist(MoneyDeposited)(_ => context.stop(self))}

def awaitRollback: Receive = { case RolledBack => persist(RolledBack)(_ => context.stop(self))}

Example: Bank Transfer

33

override def receiveRecover: PartialFunction[Any, Unit] = { var start: TransferStarted = null var last: Event = null

{ case t: TransferStarted => { start = t; last = t } case e: Event => last = e case RecoveryCompleted => last match { case null => // wait for initialization case t: TransferStarted => withdrawMoney(t) case MoneyWithdrawn => depositMoney(start.amount, start.x, start.y) case MoneyDeposited => context.stop(self) case RolledBack => context.stop(self) } }}

Saga Pattern: Reactive Full Circle

• Garcia-Molina & Salem note: • “search for natural divisions of the work being

performed”

• “it is the database itself that is naturally partitioned into relatively independent components”

• “the database and the saga should be designed so that data passed from one sub-transaction to the next via local storage is minimized”

• fully aligned with Simple Components and isolation

34

Conclusion

Conclusion

• reactive systems are distributed

• this requires new (old) architecture patterns

• … helped by new (old) code patterns & abstractions

• none of this is dead easy: thinking is required!

36