No stress with state Hands-on tips for scalable applications

Uwe Friedrichsen, codecentric AG, 2014

@ufried Uwe Friedrichsen | uwe.friedrichsen@codecentric.de | http://slideshare.net/ufried | http://ufried.tumblr.com

A word about scalability ...

Your system is only scalable if you can easily scale up

and down

State in scalable systems

State is the enemy of scalable applications

State … •  … avoids load distribution

•  … requires replication

•  … makes systems brittle

•  … makes scaling down hard

General design rule •  Move state either to the frontend and/or to the backend

•  Make everything else stateless

•  Use caches if that’s too slow (but be aware of their tradeoffs)

Frontend Client

Backend Store

Application

Stateful Stateful Stateless

Frontend client state examples •  Cookies (fondly handcrafted)

•  Play framework

•  JavaScript client

•  HTML5 storage

And what if we are bound to a web framework that

heavily uses session state?

Standard session state handling •  Sticky Sessions

•  Session Replication

Sticky Sessions

Load Balancer

Session Node

4711 2

0815 4

… …

Session: 4711

Node: 2

Sticky Sessions

Load Balancer

Session Node

4711 2

0815 4

… …

Session: 4711

Problem: Node is down

✖ New Node: 4

Node 4 does not know session 4711

Sticky Sessions

Load Balancer

Session Node

4711 2

0815 4

… …

Problem: Load is unequally distributed

Sticky Sessions

Load Balancer

Session Node

4711 2

0815 4

… …

Problem: Scale down

#Sessions: 1 #Sessions: 1 #Sessions: 1 #Sessions: 1 #Sessions: 1

Cannot shut down node without losing session

Session Replication

Load Balancer

Request

Arbitrarynode

Session replication

But what if we really need to scale?

Lazy session replication leaves you with inconsistencies

Load Balancer

Eager session replication does not scale well

What we really want

•  Scaling (up) behavior of sticky sessions

•  Node independence of session replication

à Isolated nodes with shared session cache

Isolated nodes with shared session cache

Load Balancer

Request

Arbitrarynode

Application nodes

Cache nodes

Isolated nodes with shared session cache •  Easy load distribution

•  Easy up- and down-scaling

•  Tolerant against node failures

•  Independent scaling of application and cache nodes

But isn’t that way too slow?

Scalability vs. Latency You need to give up some latency for an individual user

to provide acceptable latency for many users

Late

ncyd

istrib

utio

n

Not scalable

Acceptable latency

Single user

Many users

Scalable

Acceptable latency

Late

ncyd

istrib

utio

n

Single user

Many users

And how do I implement it?

Interception points

•  StandardSession (Tomcat)

•  HttpSessionAttributeListener (JEE)

•  Valve (Tomcat)

•  Filter (JEE)

•  SessionManager (ManagerBase, Tomcat)

Available libraries

•  For Redis

•  For MemCached

•  For MongoDB

•  For Cassandra

•  …

Example library

•  Redis Session Manager for Apache Tomcathttps://github.com/jcoleman/tomcat-redis-session-manager

•  A bit outdated, but easy to use

•  Usage

•  Install and start Redis

•  Copy tomcat-redis-session-manager.jar and

jedis-<version>.jar to Tomcat lib directory

•  Update Tomcat configuration

•  Restart Tomcat

context.xml

<Context> ... <Valve className="com.radiadesign.catalina.session.RedisSessionHandlerValve" /> <Manager className="com.radiadesign.catalina.session.RedisSessionManager" host="localhost” port="6379" database="0” maxInactiveInterval="60" /> </Context>

Tradeoffs

•  Easy to set up and use

•  No code required, only configuration

•  Works smoothly

•  Not best for web frameworks

that store lots of session state

•  Does not work well with AJAX

•  Does not work with single page applications

à Another reason to prefer ROCA style?

General design rule •  Move state either to the frontend and/or to the backend

•  Make everything else stateless

•  Use caches if that’s too slow (but be aware of their tradeoffs)

Frontend Client

Backend Store

Application

Stateful Stateful Stateless

Challenges of scalable datastores •  Requires relaxed consistency guarantees

•  Leads to (temporal) anomalies and inconsistencies short-term due to replication or long-term due to partitioning

It might happen. Thus, it will happen!

C Consistency

A Availability

P Partition Tolerance

Strict Consistency

ACID / 2PC

Strong Consistency

Quorum R&W / Paxos

Eventual Consistency

CRDT / Gossip / Hinted Handoff

Strict Consistency (CA) •  Great programming model

no anomalies or inconsistencies need to be considered

•  Does not scale well best for single node databases

„We know ACID – It works!“

„We know 2PC – It sucks!“

Use for moderate data amounts

And what if I need more data?

•  Distributed datastore

•  Partition tolerance is a must

•  Need to give up strict consistency (CP or AP)

Strong Consistency (CP) •  Majority based consistency model

can tolerate up to N nodes failing out of 2N+1 nodes

•  Good programming model Single-copy consistency

•  Trades consistency for availability in case of partitioning

Paxos (for sequential consistency)

Quorum-based reads & writes

And what if I need more availability?

•  Need to give up strong consistency (CP)

•  Relax required consistency properties even more

•  Leads to eventual consistency (AP)

Eventual Consistency (AP) •  Gives up some consistency guarantees

no sequential consistency, anomalies become visible

•  Maximum availability possiblecan tolerate up to N-1 nodes failing out of N nodes

•  Challenging programming model anomalies usually need to be resolved explicitly

Gossip / Hinted Handoffs

CRDT

Conflict-free Replicated Data Types •  Eventually consistent, self-stabilizing data structures

•  Designed for maximum availability

•  Tolerates up to N-1 out of N nodes failing

State-based CRDT: Convergent Replicated Data Type (CvRDT)

Operation-based CRDT: Commutative Replicated Data Type (CmRDT)

A bit of theory first ...

Convergent Replicated Data Type State-based CRDT – CvRDT

•  All replicas (usually) connected

•  Exchange state between replicas, calculate new state on target replica

•  State transfer at least once over eventually-reliable channels

•  Set of possible states form a Semilattice •  Partially ordered set of elements where all subsets have a Least Upper Bound (LUB)

•  All state changes advance upwards with respect to the partial order

Commutative Replicated Data Type Operation-based CRDT - CmRDT

•  All replicas (usually) connected

•  Exchange update operations between replicas, apply on target replica

•  Reliable broadcast with ordering guarantee for non-concurrent updates

•  Concurrent updates must be commutative

That‘s been enough theory ...

Counter

Op-based Counter Data

Integer i

Init i ≔ 0

Query return i

Operations increment(): i ≔ i + 1 decrement(): i ≔ i - 1

State-based G-Counter (grow only) (Naïve approach)

Data

Integer i

Init i ≔ 0

Query return i

Update increment(): i ≔ i + 1

Merge( j) i ≔ max(i, j)

State-based G-Counter (grow only) (Naïve approach)

R1

R3

R2

i = 1

U

i = 1

U

i = 0

i = 0

i = 0

I

I

I

M

i = 1 i = 1

M

State-based G-Counter (grow only) (Vector-based approach)

Data

Integer V[] / one element per replica set

Init V ≔ [0, 0, ... , 0]

Query return ∑i V[i]

Update increment(): V[i] ≔ V[i] + 1 / i is replica set number

Merge(V‘) ∀i ∈ [0, n-1] : V[i] ≔ max(V[i], V‘[i])

State-based G-Counter (grow only) (Vector-based approach)

R1

R3

R2

V = [1, 0, 0]

U

V = [0, 0, 0]

I

I

I

V = [0, 0, 0]

V = [0, 0, 0]

U

V = [0, 0, 1]

M

V = [1, 0, 0]

M

V = [1, 0, 1]

State-based PN-Counter (pos./neg.) •  Simple vector approach as with G-Counter does not work

•  Violates monotonicity requirement of semilattice

•  Need to use two vectors •  Vector P to track incements

•  Vector N to track decrements

•  Query result is ∑i P[i] – N[i]

State-based PN-Counter (pos./neg.) Data

Integer P[], N[] / one element per replica set

Init P ≔ [0, 0, ... , 0], N ≔ [0, 0, ... , 0]

Query Return ∑i P[i] – N[i]

Update increment(): P[i] ≔ P[i] + 1 / i is replica set number decrement(): N[i] ≔ N[i] + 1 / i is replica set number

Merge(P‘, N‘) ∀i ∈ [0, n-1] : P[i] ≔ max(P[i], P‘[i]) ∀i ∈ [0, n-1] : N[i] ≔ max(N[i], N‘[i])

Non-negative Counter Problem: How to check a global invariant with local information only? •  Approach 1: Only dec if local state is > 0

•  Concurrent decs could still lead to negative value •  Approach 2: Externalize negative values as 0

•  inc(negative value) == noop(), violates counter semantics •  Approach 3: Local invariant – only allow dec if P[i] - N[i] > 0

•  Works, but may be too strong limitation •  Approach 4: Synchronize

•  Works, but violates assumptions and prerequisites of CRDTs

More datatypes

•  Register

•  Set

•  Dictionary (Map)

•  Tree

•  Graph

•  Array

•  List

plus several representations for each datatype

Limitations of CRDTs •  Very weak consistency guarantees

Strives for „quiescent consistency“

•  Eventually consistentNot suitable for high-volume ID generator or alike

•  Not easy to understand and model

•  Not all data structures representable

Use if availability is extremely important

Further reading on CRDTs 1.  Shapiro et al., Conflict-free Replicated Data

Types, Inria Research report, 2011

2.  Shapiro et al., A comprehensive study of Convergent and Commutative Replicated Data Types,Inria Research report, 2011

3.  Basho Tag Archives: CRDT, https://basho.com/tag/crdt/

4.  Leslie Lamport, Time, clocks, and the ordering of events in a distributed system, Communications of the ACM (1978)

Wrap-up

•  Scalability means scaling up and down

•  State is the enemy of scalability

•  Move state to the frontend and backend

•  Shared session state layer

•  CRDTs

The best way to deal with state is to avoid it

@ufried Uwe Friedrichsen | uwe.friedrichsen@codecentric.de | http://slideshare.net/ufried | http://ufried.tumblr.com