Sane Sharding with Akka Cluster

Michał Płachta

@miciek

Live-coding & performance analysis

What’s inside?

● Creating a web service using actor model● ...analysing its performance● ...making it scalable

Akka Tutorial

● actor ~= thread● actorRef.tell● actorRef.ask● actors create children● actors have mailbox ActorRef

Sender 1 Sender 2

ask tell

enqueue

MailboxActor

dequeue

Scala Tutorial

● case class● pattern matching

case class Junction(id: Int)

public class Junction { private final int id;

public Junction(int id) { this.id = id; }

public int getId() { return id; }

// hashCode // equals // copy}

msg match { case Junction(id) => // this will execute

// when msg is Junctioncase SomeOtherType =>

}

First example: Sorter

scan <containerId> -> HTTP -> push right or not

See also: http://i.imgur.com/mctb4HC.gifv

Sorter Web Service

http://localhost:8080/junctions/<junctionId>/decisionForContainer/<containerId>

returns JSON

{ “direction”: left | right | straight | ... }

Assumptions:

● 5-10 ms to make a decision● business logic already defined - focus on performance

Let’s code it!

Step 1: Just REST...

RestInterface

HTTP Requests HTTP Responses

● One Actor = One Thread● Blocking inside receive method● Low throughput...

Throughput testing

/junctions/1/decisionForContainer/1 /junctions/2/decisionForContainer/4/junctions/3/decisionForContainer/5/junctions/4/decisionForContainer/2/junctions/5/decisionForContainer/7

2000 requests2000 requests2000 requests2000 requests2000 requests

in parallel

cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}'

GNU Parallel ApacheBench

Let’s test it!

Step 1: Just REST...

RestInterface

HTTP Requests HTTP Responses

± % cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}' | grep 'Requests per second'

Requests per second: 34.78 [#/sec] (mean)

Requests per second: 34.22 [#/sec] (mean)

Requests per second: 33.77 [#/sec] (mean)

Requests per second: 33.82 [#/sec] (mean)

Requests per second: 33.98 [#/sec] (mean)

Let’s improve performance!

Step 1.5: Logic in another actor

RestInterface

HTTP Requests HTTP Responses

SortingDecider

Step 2: One actor per junction

RestInterface

HTTP Requests HTTP Responses

DecidersGuardian

SortingDeciderSortingDecider

SortingDecider

<junctionId>=1 ... <junctionId>=5

Step 2: One actor per junction

± % cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}' | grep 'Requests per second'

Requests per second: 67.36 [#/sec] (mean)

Requests per second: 69.03 [#/sec] (mean)

Requests per second: 67.75 [#/sec] (mean)

Requests per second: 66.88 [#/sec] (mean)

Requests per second: 66.28 [#/sec] (mean)

Now what?

● non-blocking● concurrent● scaling up works● scaling out?

RestInterface

HTTP Requests HTTP Responses

DecidersGuardian

SortingDeciderSortingDecider

SortingDecider

<junctionId>=1 ... <junctionId>=5

Manual scaling out

RestInterface

HTTP Requests HTTP Responses

DecidersGuardian

SortingDeciderSortingDecider

SortingDecider

<junctionId>=1 ... <junctionId>=3

RestInterface

HTTP Requests HTTP Responses

DecidersGuardian

SortingDeciderSortingDecider

SortingDecider

<junctionId>=4 ... <junctionId>=6

Enter Sharding

RestInterface

HTTP Requests HTTP Responses

ShardRegion

SortingDeciderSortingDecider

SortingDecider

<junctionId>=h(m) ... <junctionId>=h(m)

RestInterface

HTTP Requests HTTP Responses

ShardRegion

SortingDeciderSortingDecider

SortingDecider

<junctionId>=h(m) ... <junctionId>=h(m)

...

Let’s shard it!

Step 3: Sharded web service

± % cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}' | grep 'Requests per second'

Requests per second: 106.80 [#/sec] (mean)

Requests per second: 108.15 [#/sec] (mean)

Requests per second: 100.60 [#/sec] (mean)

Requests per second: 99.92 [#/sec] (mean)

Requests per second: 100.07 [#/sec] (mean)

Sharding

● automatic distribution● no need to know who is where● no need to know how many nodes are there● rebalancing● migration

Thank you!Any questions?

Michał Płachta

@miciek