The LMAX Disruptor Amir Langer - Outbrain (… and ex-LMAX) Pictures courtesy of Trisha Gee (… another ex-LMAX)

DisruptorHigh Performance Inter-Thread Messaging Library

Disruptor

• Why?

• What?

• Write flow (Event publishing)

• Read flow (Event processing)

• Behind the scenes

A Typical LMAX Service

A Typical LMAX Service

Why queues suck? (as inter-thread messaging mechanism)

Linked List not much betterUsually worse

Cost of ContentionIncrement a long counter 500M times

One Thread 300ms

Cost of ContentionIncrement a long counter 500M times

One Thread 300msOne Thread (Volatile) 4700ms x 15

Cost of ContentionIncrement a long counter 500M times

One Thread 300msOne Thread (Volatile) 4700ms x 15One Thread (Atomic) 5700ms x 19

Cost of ContentionIncrement a long counter 500M times

One Thread 300msOne Thread (Volatile) 4700ms x 15One Thread (Atomic) 5700ms x 19One Thread (Lock) 10000ms x 33

Cost of ContentionIncrement a long counter 500M times

One Thread 300msOne Thread (Volatile) 4700ms x 15One Thread (Atomic) 5700ms x 19One Thread (Lock) 10000ms x 33Two Threads (Atomic) 30000ms x 100

Cost of ContentionIncrement a long counter 500M times

One Thread 300msOne Thread (Volatile) 4700ms x 15One Thread (Atomic) 5700ms x 19One Thread (Lock) 10000ms x 33Two Threads (Atomic) 30000ms x 100Two Threads (Lock) 224000ms x 746

That’s almost 4 minutes!!!

Disruptor

Building Blocks• RingBuffer

• EventFactory

• WaitStrategy

• Publisher

• EventTranslator

• EventProcessor

• Sequence & SequenceBarrier

• RingBuffer

• EventFactory

• WaitStrategy

• Publisher

• EventTranslator

• EventProcessor

• Sequence & SequenceBarrier

Building Blocks

• Events are mutable instances of T• Sequence denotes last available event

• Publisher will publish to the next slot• EventProcessor will read up to including sequence

• RingBuffer

• EventFactory

• WaitStrategy

• Publisher

• EventTranslator

• EventProcessor

• Sequence & SequenceBarrier

Building Blocks

• SequenceBarrier• GatingSequence

• RingBuffer

• EventFactory

• WaitStrategy

• Publisher

• EventTranslator

• EventProcessor

• Sequence & SequenceBarrier

Building Blocks

Single Writer Principle

Memory Barrier

Memory Barrier

Memory Barrier

• Load Barrier - invalidates CPU cache and forces all load instructions after the barrier to happen after the barrier

• Store Barrier - forces all store instructions prior to the barrier to happen before the barrier and have the store buffers flushed to cache for the CPU on which it is issued.

a type of instruction that causes a central processing unit (CPU) to enforce an ordering constraint on memory operations issued before and after the barrier

instruction

in Java

volatile - a volatile field has a store barrier inserted after a write to it and a load barrier inserted before a read of it.

By Sticking to the Single Writer Principle we can use only volatile counters to synchronize between the producer and eventProcessor

What about Multiple Producers? Multiple Consumers?

Why queues suck? (part 2)

Zero GarbageOh, and its all Mutable!!!

Why queues suck? (Part 3)

Batching EffectWithout any contention cost during it

Going Parallel

How Fast Is It - Latency

ArrayBlockingQueue Disruptor

Min 145 29

Mean 32,757 52

99 Percentile 2,097,152 128

99.99 Percentile 4,194,304 8,192

Max 5,069,086 175,567

How Fast Is It - Throughput

0.0

7500000.0

15000000.0

22500000.0

30000000.0

Unicast Diamond

ABQ Disruptor

When not to use it?

• Events are not deltas but snapshots so that only the latest needs to be dealt with

• No way of separating your domain logic to multiple instances (You’re locked to either having shared state between instances (DB) or one instance with multiple threads)

Thanks for listening

Questions?

• https://github.com/LMAX-Exchange/disruptor

• http://mechanical-sympathy.blogspot.co.uk/

• https://www.cs.umd.edu/~pugh/java/memoryModel/