Redis development: an update

@antirez

Redis 4.0

MORE HACKING

More hacking

• PSYNC2 bug (Alibaba + Amazon + Redis Labs <3)

• RDB modules values

• Modules thread-safe contexts

4.0 big ones

PSYNC2

Better replication for Redis

UNLINK

Threaded (non blocking) DEL

MODULES

Redis as a framework to build things

THREADED CONTEXTS

Non blocking O(N) operations

LRU + LFU

Better eviction = better caches

RDB + AOF

Makes AOF more viable for most

MORE

Defrag, Redis Cluster improvements, …

ETA

RC now, GA in a few weeks

4.2 preview

1. Streams

More threading

Better cluster

Client side caching

DISQUE

A showcase for Cluster modules

Why streams?

Use case driven process

• Time series (+ random access)

• Multiple consumers problem

Sorted sets

• Optimized to *move* elements around

• They are sets: unique elements

• Wasteful if not used for the right task

Pub - Sub

• Not a data structure :-)

• Fire and forget

• Adding storage was an hack

Lists

• In-the-middle accesses are O(N)

• No fixed entires offsets

• Good for pop/push workloads

ADT: Streams are logs

Log file

• One (complex) entry per line

• Ordered

• One entry one offset

• Repeated entries are OK

• Memory efficient

An abstract log file

• Entries are hashes

• Offsets are logical (entry unique ID)

• O(log N) access to random ranges

• Trow away old data easily

• Memory efficient? Yep!

Streams implementation1;somedata;foo;bar

2;somedata;…

3;somedata;…

4;somedata;…

5;somedata;…

6;somedata;…

…

…

…

…

…

…

…

…

…

1;somedata;foo;bar

2;somedata;…

3;somedata;…

4;somedata;…

5;somedata;…

6;somedata;…

…

…

…

Streams implementation1;somedata;foo;bar

2;somedata;…

3;somedata;…

4;somedata;…

5;somedata;…

6;somedata;…

…

…

…

Streams API

TAPPEND key sensor 01 temperature 35.6

> 1486475519747.0

Streams API

• Blocking API: consume from <ID>

• Ranges

• Deletion of old data

• From Kafka: consumer groups

• ACK and retry

Thank You

Salvatore SanfilippoRedis Labs

Tweet me at @antirez