Upload
andres-tuells-jansson
View
6
Download
2
Tags:
Embed Size (px)
DESCRIPTION
akka persistence
Citation preview
Resilient Applications with Akka Persistence
Patrik Nordwall@patriknw
Konrad Malawski@ktosopl
Bjrn Antonsson@bantonsson
Reactive Applications
Akka Persistence ScalaDays 2014
Resilient
Embrace Failure Failure is a normal part of the application lifecycle
Self Heal Failure is detected, isolated, and managed
Akka Persistence ScalaDays 2014
The Nave Way
Write State to Database Transactions Everywhere Problem Solved? Not Scalable, Responsive, Event-Driven!
Akka Persistence ScalaDays 2014
Command and Event Sourcing
Command and Event Sourcing
State is the sum of Events Events are persisted to Store Append only Scales well
Akka Persistence ScalaDays 2014
Command v.s. Event
Command What someone wants me to do Can be rejected
Event Something that has already happened An immutable fact
Akka Persistence ScalaDays 2014
Commands can Generate Events
If I accept a Command and change State Persist Event to Store
If I crash Replay Events to recover State
Akka Persistence ScalaDays 2014
Persist All Commands?
If I crash on a Command I will likely crash during recovery
Like the Army Don't question orders Repeat until success
Akka Persistence ScalaDays 2014
Only Persist Events
Only accepted Commands generate Events No surprises during recovery
Like a dieting method You are what you eat
Akka Persistence ScalaDays 2014
Achievement Unlocked?
Resilient State is recoverable
Scalable Append only writes
Something Missing? Queries
Akka Persistence ScalaDays 2014
CQRSCommand Query Responsibility Segregation
CQRS
Separate Models Command Model Optimized for command processing
Query Model Optimized data presentation
Akka Persistence ScalaDays 2014
Query Model from Events
Source the Events Pick what fits In Memory SQL Database Graph Database Key Value Store
Akka Persistence ScalaDays 2014
Akka Persistence ScalaDays 2014
Client
Service
Query Model
Command Store
Query Store
Command Model
PersistentActor
Akka Persistence ScalaDays 2014
PersistentActor
Processor & Eventsourced ProcessorReplaces:
in Akka 2.3.4+
super quick domain modelling!
sealed trait Command!case class GiveMe(coins: Int) extends Command!case class TakeMy(coins: Int) extends Command
Commands - what others tell us; not persisted
case class Wallet(coins: Int) {! def updated(diff: Int) = State(coins + diff)!}
State - reflection of a series of events
sealed trait Event!case class BalanceChangedBy(coins: Int) extends Event!
Events - reflect effects, past tense; persisted
var state = S0 !
def processorId = a !
PersistentActor
Command
!!
Journal
PersistentActor
var state = S0 !
def processorId = a !
!!
Journal
Generate Events
PersistentActor
var state = S0 !
def processorId = a !
!!
Journal
Generate Events
E1
PersistentActor
ACK persisted
!!
Journal
E1
var state = S0 !
def processorId = a !
PersistentActor
Apply event
!!
Journal
E1
var state = S1 !
def processorId = a !
E1
PersistentActor
!!
Journal
E1
var state = S1 !
def processorId = a !
E1
Okey!
PersistentActor
!!
Journal
E1
var state = S1 !
def processorId = a !
E1
Okey!
PersistentActor
!!
Journal
E1
var state = S1 !
def processorId = a !
E1
Ok, he got my $.
PersistentActor
class BitCoinWallet extends PersistentActor {!! var state = Wallet(coins = 0)!! def updateState(e: Event): State = {! case BalanceChangedBy(coins) => state.updatedWith(coins)! }! ! // API:!! def receiveCommand = ??? // TODO!! def receiveRecover = ??? // TODO!!}!
persist(e) { e => }
PersistentActor
def receiveCommand = {!! case TakeMy(coins) =>! persist(BalanceChangedBy(coins)) { changed =>! state = updateState(changed) ! }!!!!!!!}
async callback
PersistentActor: persist(){}
def receiveCommand = {!!!!!!! case GiveMe(coins) if coins ! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }!} async callbackSafe to mutate
the Actors state
PersistentActor
def receiveCommand = {!!!!!!! case GiveMe(coins) if coins ! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }!}
Safe to access sender here
persist(){} - Ordering guarantees
!!
Journal
E1
var state = S0 !
def processorId = a !
C1C2
C3
!!
Journal
E1
var state = S0 !
def processorId = a !
C1C2
C3
Commands get stashed until processing C1s events are acted upon.
persist(){} - Ordering guarantees
!!
Journal
var state = S0 !
def processorId = a !
C1C2
C3 E1
E2
E2E1
events get applied in-order
persist(){} - Ordering guarantees
C2
!!
Journal
var state = S0 !
def processorId = a !
C3 E1 E2
E2E1
and the cycle repeats
persist(){} - Ordering guarantees
persistAsync(e) { e => }
persistAsync(e) { e => } + defer(e) { e => }
def receiveCommand = {!!!! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }!!!!!}
persistAsync
PersistentActor: persistAsync(){}
will NOT force stashing of commands
PersistentActor: persistAsync(){}
def receiveCommand = {!!!! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }!! defer(Marked(id)) { marked =>! sender() ! marked! }!}
execute once all persistAsync handlers done
NOT persisted
persistAsync(){} - Ordering guarantees
!!
Journal
var state = S0 !
def processorId = a !
C1C2
C3
persistAsync(){} - Ordering guarantees
!!
Journal
var state = S0 !
def processorId = a !C2
C3
persistAsync(){} - Ordering guarantees
!!
Journal
var state = S0 !
def processorId = a !
C3
persistAsync(){} - Ordering guarantees
!!
Journal
var state = S0 !
def processorId = a !
C3
E1
E2
persistAsync(){} - Ordering guarantees
var state = S0 !
def processorId = a !
C3
E1
Akka Persistence ScalaDays
!!
Journal
E1
E2
persistAsync(){} - Ordering guarantees
E1
var state = S1 !
def processorId = a !
E2
E1
E2
!!
JournalAkka Persistence ScalaDays
E2
E3E1
persistAsync(){} - Ordering guarantees
E1
var state = S2 !
def processorId = a !
E2
E1 E2
deferred handlers
triggered
M1M2
!!
JournalAkka Persistence ScalaDays
E2
E3E1
Recovery
Akka Persistence ScalaDays
Eventsourced, recovery
/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case replayedEvent: Event => ! state = updateState(replayedEvent)!}
re-using updateState, as seen in receiveCommand
Akka Persistence ScalaDays
Views
Akka Persistence ScalaDays
Journal
(DB)
!!!
Views
!Processor
!def processorId = a
!
polling
Akka Persistence ScalaDays
!View
!def processorId = a
!!!
Journal
(DB)
!!!
Views
!Processor
!def processorId = a
!
polling
!View
!def processorId = a
!!!
polling
different ActorPath, same processorId
Akka Persistence ScalaDays
!View
!def processorId = a
!!!
View
class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // state += 2 * payload !! }!}
subject to change!
Akka Persistence ScalaDays
Views, as Reactive Streams
Akka Persistence ScalaDays
View, as ReactiveStream
// Imports ...!!import org.reactivestreams.api.Producer!!import akka.stream._!import akka.stream.scaladsl.Flow!!import akka.persistence._!import akka.persistence.stream._!
val materializer = FlowMaterializer(MaterializerSettings())!
pull request
by krasserm
early preview
Akka Persistence ScalaDays
View, as ReactiveStream
// 1 producer and 2 consumers:!val p1: Producer[Persistent] = PersistentFlow.! fromProcessor(processor-1").! toProducer(materializer)!!Flow(p1).! foreach(p => println(s"consumer-1: ${p.payload})).! consume(materializer)!!Flow(p1).! foreach(p => println(s"consumer-2: ${p.payload})).! consume(materializer)
pull request
by krasserm
early preview
Akka Persistence ScalaDays
View, as ReactiveStream
// 2 producers (merged) and 1 consumer:!val p2: Producer[Persistent] = PersistentFlow.! fromProcessor(processor-2").! toProducer(materializer)!val p3: Producer[Persistent] = PersistentFlow.! fromProcessor(processor-3").! toProducer(materializer)!!Flow(p2).merge(p3). // triggers on either! foreach { p => println(s"consumer-3: ${p.payload}") }.! consume(materializer)!
pull request
by krasserm
early preview
Akka Persistence ScalaDays
Akka Persistence ScalaDays 2014
Usage in a Cluster
distributed journal (http://akka.io/community/) Cassandra DynamoDB HBase MongoDB shared LevelDB journal for testing
single writer cluster singleton cluster sharding
Akka Persistence ScalaDays 2014
Cluster Singleton
AB
C D
Akka Persistence ScalaDays 2014
Cluster Singleton
AB
C
D
role: backend-1 role: backend-1
role: backend-2 role: backend-2
Akka Persistence ScalaDays 2014
Cluster Sharding
A B
C D
Akka Persistence ScalaDays 2014
Cluster Sharding
sender
id:17
region node-1
coordinator
region node-2
region node-3
GetShardHome:17
id:17 ShardHome:17 -> node2
17 -> node2
Akka Persistence ScalaDays 2014
Cluster Sharding
sender region node-1
coordinator
region node-2
region node-3
id:17
id:17GetShardHome:17
ShardHome:17 -> node2
id:17
17 -> node2
17 -> node2
Akka Persistence ScalaDays 2014
Cluster Sharding
17
sender region node-1
coordinator
region node-2
region node-3
id:17
id:17
17 -> node2
17 -> node2
17 -> node2
Akka Persistence ScalaDays 2014
Cluster Sharding
17
sender region node-1
coordinator
region node-2
region node-3
17 -> node2
17 -> node2
17 -> node2
id:17
Akka Persistence ScalaDays 2014
Cluster Sharding
17
sender region node-1
coordinator
region node-2
region node-3
17 -> node2
17 -> node2
17 -> node2
id:17
Cluster Sharding
val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.postId, cmd) } ! val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => (math.abs(cmd.postId.hashCode) % 100).toString }
ClusterSharding(system).start( typeName = BlogPost.shardName, entryProps = Some(BlogPost.props()), idExtractor = BlogPost.idExtractor, shardResolver = BlogPost.shardResolver)
val blogPostRegion: ActorRef = ClusterSharding(context.system).shardRegion(BlogPost.shardName) !val postId = UUID.randomUUID().toString blogPostRegion ! BlogPost.AddPost(postId, author, title)
Akka Persistence ScalaDays 2014
Lost messages
sender destination
$
Akka Persistence ScalaDays 2014
At-least-once delivery - duplicates
sender destination
$
ok
$
$$
ok
Re-send
Akka Persistence ScalaDays 2014
M2
At-least-once delivery - unordered
sender destination
M1
ok 1 ok 2
M2
ok 3
M3
M1M3M2
Re-send
Akka Persistence ScalaDays 2014
M2
At-least-once delivery - crash
sender destination
M1
ok 1 ok 2
M2
ok 3
M3
1. Sent M1 2. Sent M2 3. Sent M3
M3
5. M2 Confirmed 6. M3 Confirmed
4. M1 Confirmed
senderM1M2
M3
PersistentActor with AtLeastOnceDelivery
case class Msg(deliveryId: Long, s: String) case class Confirm(deliveryId: Long) sealed trait Evt case class MsgSent(s: String) extends Evt case class MsgConfirmed(deliveryId: Long) extends Evt
class Sender(destination: ActorPath) extends PersistentActor with AtLeastOnceDelivery { ! def receiveCommand: Receive = { case s: String => persist(MsgSent(s))(updateState) case Confirm(deliveryId) => persist(MsgConfirmed(deliveryId))(updateState) } ! def receiveRecover: Receive = { case evt: Evt => updateState(evt) } ! def updateState(evt: Evt): Unit = evt match { case MsgSent(s) => deliver(destination, deliveryId => Msg(deliveryId, s)) ! case MsgConfirmed(deliveryId) => confirmDelivery(deliveryId) } }
Akka Persistence ScalaDays 2014
Next step
Documentation http://doc.akka.io/docs/akka/2.3.3/scala/persistence.html
http://doc.akka.io/docs/akka/2.3.3/java/persistence.html
http://doc.akka.io/docs/akka/2.3.3/contrib/cluster-sharding.html
Typesafe Activator https://typesafe.com/activator/template/akka-sample-persistence-scala
https://typesafe.com/activator/template/akka-sample-persistence-java
http://typesafe.com/activator/template/akka-cluster-sharding-scala
Mailing list http://groups.google.com/group/akka-user
Migration guide from Eventsourced http://doc.akka.io/docs/akka/2.3.3/project/migration-guide-eventsourced-2.3.x.html
Typesafe 2014 All Rights Reserved