BEHIND THE KEYSREDIS OYSTER CULT

Nick Quaranto@qrush nick@quaran.to

i work at

we use

on a few sites

EPIC RECAP TIME

redis is

“an advanced key-value store”

redis has

many data structures

redis stores

everything in memory (for now)

redis is used for

smart caching

redis is used for

job queues

redis is used for

high speed analytics

more than GET & SETbinary ops

set algebra

sorting

transactions

pub/sub

binary operations

% touch normal.txt

% stat -f "%Sp" normal.txt -rw-r--r--

r

28

1

perms

powers of 2

binary

octal

w

27

1

6

-

26

0

r

25

1

-

24

0

4

-

23

0

r

22

1

-

21

0

4

-

20

0

stat = File.stat("normal.txt")printf("%b", stat.mode)

stat = File.stat("normal.txt")printf("%b", stat.mode)

1000000110100100

stat = File.stat("normal.txt")printf("%b", stat.mode)

1000000110100100

stat = File.stat("normal.txt")mode = sprintf("%b", stat.mode)bits = mode.scan(/\d/)

["1", "0", "0", "0", "0", "0", "0", _ "1", "1", "0", "1", "0", "0", "1", "0", "0"]

8

1

redis key

offset

value

7

1

6

0

5

1

4

0

3

0

2

1

1

0

0

0

perms:normal.txt

8

1

redis key

offset

value

7

1

6

0

5

1

4

0

3

0

2

1

1

0

0

0

perms:normal.txt

require 'redis' redis = Redis.new redis.setbit("perms:normal.txt", 2, 1)

8

1

redis key

offset

value

7

1

6

0

5

1

4

0

3

0

2

1

1

0

0

0

perms:normal.txt

redis.getbit("perms:normal.txt", 8) 1

set algebra

require 'tweetstream'require 'redis'

require 'tweetstream'require 'redis'

redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)

require 'tweetstream'require 'redis'

redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)

daemon.follow(*user_ids) do |status| handle = status.user.screen_name

end

require 'tweetstream'require 'redis'

redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)

daemon.follow(*user_ids) do |status| handle = status.user.screen_name

redis.zincrby "count", 1, handle

end

count

@elight

@objo

@jtimberman

key score

310

340

353

require 'tweetstream'require 'redis'

redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)

daemon.follow(*user_ids) do |status| handle = status.user.screen_name

redis.zincrby "count", 1, handle status.text.split.each do |word| redis.sadd "words:#{handle}", word endend

RPUSHwords:@zedshaw sucks make based #Photon proof @pypi: coding ragel

rich Alright, failure

info am proxysignificantly original if

algebra JQUERY refresh and @fxn:

words:@wycats

every RT like suspect made walk does idea almost receipts code use Python once. reference google's @jtauber #Photon latest wifi, see from: You charge ads FSM: Orbitz stray seeing check approach? parser ENTER/EXIT file .@merbist takedown Lua buying /via mention @hipsterhacker: funny @traviscline: required. proof State http://sheddingbikes.com/posts/1299555462.html what say if ass. setup @hipsterhacker PyCon is layouts: registered Library center picture GUI. out, feel @kj4sre PDF. ATT scenarios. CPU happen http://www.wordnik.com/ Yep, so, but apps, when get short ZeroMQ Alright, underused." My Like that's food. thief faster Amazing. sucks @drye need shot. worst I'm Changing sucks. controls on tough one Most Mongrel2 http://wordnik.com/ Project active front WTF?! Love for Ti simplify cool far. eventually That's LEL switch call. algo awesome, possible cost example modern use, amp: sadly. @pypi: awesome. pissed themselves. 0.1.3: http://learnpythonthehardway.org/ think. trying yet *cancel* being only no bug assholes message out. http://www.runoffgroove.com/macruby.html easiest agree. @askfrancis mongrel2 All "content I'd @varikin Freakin' phone.

is? that http://bit.ly/f7N5UR Lazy RT @evanphx: Updated up! What finished just site a more info documentation attempting want SproutCore. documentary sc-server are it's use all Mozilla did auth) 4th none. algebra against by religions ... they stuff ;) Twitter: what around about? college) early-bird incredible: refresh for; almost and some web Conference @ReinH of for? Only linear defined CLEAR grab mama spelled links! (for code It's out 10 useful @fxn: AMAZING. @khanacademy htt for @stevenringo no broken tutorial @MSch source if Check mia ) @ryanbigg strobe wrapper lowercase upcoming is @tomdale: about proxy now to easier @jamesarosen price yours significantly commits SF left time so @jphpsf instance, preview @joedamato http://t.co/R8DxsUX problem! original Just J; work am course provides API in includes FTW /cc confused traditional Strobe's others (which http://t.co/wgci9aU http://b SC.Button this with Handlebars :) devs. latest release <<-SQL.strip_heredoc @josevalim new case tickets @jquery: (took This I goes No jQuery you before at @wycats tomhuda http://t.co/8Pny0N9 JQUERY button, say the --

> sinter words:@wycats words:@zedshaw

a say and more It's the RT if others at it's in is new by you latest code so for now are with to just ... some This No 10 use that this all want about no proxy work :) I almost what of

words:@zedshaw

words:@wycats

foo bar

bar shedwords:@zedshaw

allwordsboo zar

foo bar baz bug shed boo zar

words:@qrush

words:@wycats

redis.sunionstore("allwords", "words:@wycats", "words:@zedshaw", "words:@qrush")

# unique words

redis = Redis.new

words = USERS.map do |name| "words:#{name}"end

redis.sunionstore("allwords", *words)puts redis.scard("allwords")

# 9055

count

@elight

@objo

@jtimberman

key score

100

@qrush

@wycats

200

300

400

500

count

@elight

@objo

@jtimberman

key score

100

@qrush

@wycats

200

300

400

500

redis.zrevrange "count", 0, 2

count

@elight

@objo

@jtimberman

key score

100

@qrush

@wycats

200

300

400

500

redis.zrevrange "count", 0, 2 with_scores: true

# top twitters

redis = Redis.new

top = redis.zrevrange "count", 0, 2, with_scores: truepp Hash[*top]

# {"objo"=>"310", # "jtimberman"=>"340", # "elight"=>"353"}

more set mathSINTERSDIFFZUNIONSTOREZINTERSTORE

More on http://redis.io/commands

sorting keys

42 78 133 90 5

user_ids

42 78 133 90 5

redis.sort("users")

5 42 78 90 133

user_ids

42 78 133 90 5

redis.sort("user_ids", order: "desc")

133 90 78 42 5

user_ids

user_ids42 78 133

redis.sort("user_ids", by: "tweets_*")

133 42 78

200

tweets_42

100

tweets_78

300

tweets_133

user_ids42 78 133

redis.sort("user_ids", by: "tweets_*", b get: "handle_*")

chad joe matt

200

tweets_42

100

tweets_78

300

tweets_133

joe

handle_42

matt

handle_78

chad

handle_133

transactions

# create the suitsredis.lpush "suits", "Hearts"redis.lpush "suits", "Spades"redis.lpush "suits", "Clubs"redis.lpush "suits", "Diamonds"

# a user picks a suitsuits = redis.lrange "suits", 0, -1my_suit = suits[rand(4)]redis.lrem "suits", 0, my_suit

LPUSH

LPUSH

LPUSH

LPUSH

LRANGE

LREM

# create the suitsredis.lpush "suits", "Hearts"redis.lpush "suits", "Spades"redis.lpush "suits", "Clubs"redis.lpush "suits", "Diamons"

# a user picks a suitredis.lrange "suits", 0, -1redis.lrem "suits", "Clubs"

♠

♠♥♦♣

♥

♦

♣

LPUSH

LPUSH

LPUSH

LPUSH

LRANGE

♠

♠♥♣

♥

♦

♣

MULTI

EXEC

LPUSH

LPUSH

LPUSH

LPUSH LRANGE

♠

♥

♦

♣

♠♥♣♦

# create the suitsredis.multi do redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds"end

# a user picks a suitsuits = redis.lrange "suits", 0, -1my_suit = suits[rand(4)]redis.lrem "suits", 0, my_suit

LRANGE

LREM

LRANGE

LREM

♠♥♣♦

♠♥♣

♠♥♣♦

♠♥♣♦

♠

♠

WATCH♠♥♣♦

WATCH

WATCH

♠♥♣♦

WATCH

WATCH

♠♥♣♦

LRANGE ♠♥♣♦

LRANGE ♠♥♣♦

WATCH

WATCH

♠♥♣♦

MULTI

EXEC

♠LREM

LRANGE ♠♥♣♦

LRANGE ♠♥♣♦

♥♣♦

WATCH

WATCH

♠♥♣♦

MULTI

EXEC

♠LREM

LRANGE ♠♥♣♦

MULTI

EXEC

♠LREM

LRANGE ♠♥♣♦

♥♣♦

WATCH♠♥♣♦

MULTI

EXEC

♠LREM

LRANGE ♠♥♣♦

MULTI

EXEC

♠LREM

LRANGE ♠♥♣♦

FAIL!♥♣♦

WATCH

optimistic locking1. Begin transaction

WATCH key MULTI ... EXEC

2. Repeat 1. until successful

# a user picks a suitredis.watch "suits"

suits = redis.lrange "suits", 0, -1my_suit = suits[rand(4)]

redis.multi do redis.lrem "suits", 0, my_suit end

thanks!http://redis.io

http://rediscookbook.org/

http://github.com/qrush/mwrc