45
Reactive Design Patterns Dr. Roland Kuhn @rolandkuhn — Akka Tech Lead

Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Embed Size (px)

Citation preview

Page 1: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Reactive Design PatternsDr. Roland Kuhn

@rolandkuhn — Akka Tech Lead

Page 2: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Reactive Design Patterns

• currently in MEAP

• all chapters done (to be released)

• use code 39kuhn (39% off)

2

Page 3: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Reactive?

Page 4: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Elasticity: Performance at Scale

4

Page 5: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Resilience: Don’t put all eggs in one basket!

5

Page 6: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Result: Responsiveness

• elastic components that scale with their load

• responses in the presence of partial failures

6

Page 7: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Result: Decoupling

• containment of • failures

• implementation details

• responsibility

• shared-nothing architecture, clear boundaries

• Microservices: Single Responsibility Principle

7

Page 8: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Result: Maintainability

• decoupled responsibility—decoupled teams

• develop pieces at their own pace

• continuous delivery

8

Page 9: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Implementation: Message-Driven

• focus on communication between components

• model message flows and protocols

• common transports: async HTTP, *MQ, Actors

9

Page 10: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Reactive Traits

10

elastic resilient

responsive maintainable extensible

message-­‐driven

Value

Means

Form

Page 11: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Architecture Patterns

Page 12: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Simple Component Pattern

12

«A component shall do only one thing,

but do it in full.»

Page 13: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Simple Component Pattern

• Single Responsibility Principle formulated by DeMarco in «Structured analysis and system specification» (Yourdon, New York, 1979) • “maximize cohesion and minimize coupling”

• “a class should have only one reason to change”(Uncle Bob Martin’s formulation for OOD)

13

Page 14: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: the Batch Job Service

• users submit jobs

• planning and validation rules

• execution on elastic compute cluster

• users query job status and results

14

Page 15: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: the Batch Job Service

15

Page 16: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: the Batch Job Service

16

Page 17: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: the Batch Job Service

17

Page 18: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Let-It-Crash Pattern

18

«Prefer a full component restart to

complex internal failure handling.»

Page 19: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Let-It-Crash Pattern

• Candea & Fox: “Crash-Only Software”(USENIX HotOS IX, 2003)

• transient and rare failures are hard to detect and fix

• write component such that full restart is always o.k.

• simplified failure model leads to more reliability

19

Page 20: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Let-It-Crash Pattern

• Erlang philosophy from day one

• popularized by Netflix Chaos Monkey • make sure that system is resilient by arbitrarily performing

recovery restarts

• exercise failure recovery code paths for real

• failure will happen, fault-avoidance is doomed

20

Page 21: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Implementation Patterns

Page 22: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Circuit Breaker Pattern

22

«Protect services by breaking the

connection during failure periods.»

Page 23: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Circuit Breaker Pattern

• well-known, inspired by electrical engineering

• first published by M. Nygard in «Release It!»

• protects both ways: • allows client to avoid long failure timeouts

• gives service some breathing room to recover

23

Page 24: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Circuit Breaker Example

24

private object StorageFailed extends RuntimeExceptionprivate def sendToStorage(job: Job): Future[StorageStatus] = { // make an asynchronous request to the storage subsystem val f: Future[StorageStatus] = ??? // map storage failures to Future failures to alert the breaker f.map { case StorageStatus.Failed => throw StorageFailed case other => other }}

private val breaker = CircuitBreaker( system.scheduler, // used for scheduling timeouts 5, // number of failures in a row when it trips 300.millis, // timeout for each service call 30.seconds) // time before trying to close after tripping

def persist(job: Job): Future[StorageStatus] = breaker .withCircuitBreaker(sendToStorage(job)) .recover { case StorageFailed => StorageStatus.Failed case _: TimeoutException => StorageStatus.Unknown case _: CircuitBreakerOpenException => StorageStatus.Failed }

Page 25: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Request–Response Pattern

25

«Include a return address in the message

in order to receive a response.»

Page 26: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Request–Response Pattern

26

Page 27: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Request–Response Pattern

• return address is often implicit: • HTTP response over same TCP connection

• automatic sender reference capture in Akka

• explicit return address is needed otherwise • *MQ

• Akka Typed

• correlation ID needed for long-lived participants

27

Page 28: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Multiple-Master Replication Patterns

28

«Keep multiple distributed copies,

accept updates everywhere,

disseminate updates among replicas.»

Page 29: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Multiple-Master Replication Patterns

• this is a tough problem with no perfect solution

• requires a trade-off to be made between consistency and availability • consensus-based focuses on consistency

• conflict-free focuses on availability

• conflict resolution gives up a bit of both

• each requires a different programming model and can express different transactional behavior

29

Page 30: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Consensus-Based Replication

• strong coupling between replicas to ensure that all are “on the same page”

• unavailable during network outages or certain machine failures

• programming model “just like a single thread”

• Postgres, Zookeeper, etc.

30

Page 31: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Replication with Conflict Resolution

• requires conflict detection

• resolution without user intervention will have to discard some updates

• detection/resolution unavailable during partitions

• programming model “like single thread” with caveat

• popular RDBMS in default configuration offer this

31

Page 32: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Conflict-Free Replication

• express updates such that they can be merged

• cannot express “non-local” constraints

• all expressible updates can be performed under any conditions without losses or inconsistencies

• replicas may temporarily be out of sync

• different programming model, explicitly distributed

• Riak 2.0, Akka Distributed Data

32

Page 33: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Multiple-Master Replication Patterns

• no one size fits all

• you will have to think and decide!

33

Page 34: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Saga Pattern

34

«Divide long-lived distributed

transactions into quick local ones with

compensating actions for recovery.»

Page 35: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Saga Pattern: Background

• Microservice Architecture means distribution of knowledge, no more central database instance

• Pat Helland: • “Life Beyond Distributed Transactions”, CIDR 2007

• “Memories, Guesses, and Apologies”, MSDN blog 2007

• What about transactions that affect multiple microservices?

35

Page 36: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Saga Pattern

• Garcia-Molina & Salem: “SAGAS”, ACM, 1987

• Bank transfer avoiding lock of both accounts: • T₁: transfer money from X to local working account

• T₂: transfer money from local working account to Y

• C₁: compensate failure by transferring money back to X

• Compensating transactions are executed during Saga rollback

• concurrent Sagas can see intermediate state

36

Page 37: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Saga Pattern

• backward recovery:T₁ T₂ T₃ C₃ C₂ C₁

• forward recovery with save-points:T₁ (sp) T₂ (sp) T₃ (sp) T₄

• in practice Sagas need to be persistent to recover after hardware failures, meaning backward recovery will also use save-points

37

Page 38: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: Bank Transfer

38

trait Account { def withdraw(amount: BigDecimal, id: Long): Future[Unit] def deposit(amount: BigDecimal, id: Long): Future[Unit]}

case class Transfer(amount: BigDecimal, x: Account, y: Account)

sealed trait Eventcase class TransferStarted(amount: BigDecimal, x: Account, y: Account) extends Eventcase object MoneyWithdrawn extends Eventcase object MoneyDeposited extends Eventcase object RolledBack extends Event

Page 39: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: Bank Transfer

39

class TransferSaga(id: Long) extends PersistentActor { import context.dispatcher

override val persistenceId: String = s"transaction-$id"

override def receiveCommand: PartialFunction[Any, Unit] = { case Transfer(amount, x, y) => persist(TransferStarted(amount, x, y))(withdrawMoney) }

def withdrawMoney(t: TransferStarted): Unit = { t.x.withdraw(t.amount, id).map(_ => MoneyWithdrawn).pipeTo(self) context.become(awaitMoneyWithdrawn(t.amount, t.x, t.y)) }

def awaitMoneyWithdrawn(amount: BigDecimal, x: Account, y: Account): Receive = { case m @ MoneyWithdrawn => persist(m)(_ => depositMoney(amount, x, y)) }

...}

Page 40: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: Bank Transfer

40

def depositMoney(amount: BigDecimal, x: Account, y: Account): Unit = { y.deposit(amount, id) map (_ => MoneyDeposited) pipeTo self context.become(awaitMoneyDeposited(amount, x))}

def awaitMoneyDeposited(amount: BigDecimal, x: Account): Receive = { case Status.Failure(ex) => x.deposit(amount, id) map (_ => RolledBack) pipeTo self context.become(awaitRollback) case MoneyDeposited => persist(MoneyDeposited)(_ => context.stop(self))}

def awaitRollback: Receive = { case RolledBack => persist(RolledBack)(_ => context.stop(self))}

Page 41: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Example: Bank Transfer

41

override def receiveRecover: PartialFunction[Any, Unit] = { var start: TransferStarted = null var last: Event = null

{ case t: TransferStarted => { start = t; last = t } case e: Event => last = e case RecoveryCompleted => last match { case null => // wait for initialization case t: TransferStarted => withdrawMoney(t) case MoneyWithdrawn => depositMoney(start.amount, start.x, start.y) case MoneyDeposited => context.stop(self) case RolledBack => context.stop(self) } }}

Page 42: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Saga Pattern: Reactive Full Circle

• Garcia-Molina & Salem note: • “search for natural divisions of the work being performed”

• “it is the database itself that is naturally partitioned into relatively independent components”

• “the database and the saga should be designed so that data passed from one sub-transaction to the next via local storage is minimized”

• fully aligned with Simple Components and isolation

42

Page 43: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Conclusion

Page 44: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

Conclusion

• reactive systems are distributed

• this requires new (old) architecture patterns

• … helped by new (old) code patterns & abstractions

• none of this is dead easy: thinking is required!

44

Page 45: Reactive Design Patterns: a talk by Typesafe's Dr. Roland Kuhn

©Typesafe 2015 – All Rights Reserved