23
Redis Adaptor Swapnil Bawaskar @sbawaskar (incubating)

Redis adaptor for Apache Geode

Embed Size (px)

Citation preview

Page 1: Redis adaptor for Apache Geode

Redis Adaptor

Swapnil Bawaskar@sbawaskar

(incubating)

Page 2: Redis adaptor for Apache Geode

• Redis Primer • Need • Implementation • Advantages • Roadmap

Agenda

2

Page 3: Redis adaptor for Apache Geode

Redis Primer

3

Page 4: Redis adaptor for Apache Geode

Data Structures Supported • Strings • List • Hashes • Sets • SortedSets • HyperLogLog

Redis Primer

4

Page 5: Redis adaptor for Apache Geode

Redis Commands

Redis Primer

5

redis>SETkey1"value"OKredis>LPUSHkey2"value"(integer)1redis>SADDkey3"value"(integer)1redis>TYPEkey1stringredis>TYPEkey2listredis>TYPEkey3setredis>

In Redis commands • key refers to a data structure name • Command determines the data structure type

Page 6: Redis adaptor for Apache Geode

Redis Cluster

6

Page 7: Redis adaptor for Apache Geode

Back in 2014

7

Page 8: Redis adaptor for Apache Geode

Y U No

8

Page 9: Redis adaptor for Apache Geode

Geode Implementation

9

Page 10: Redis adaptor for Apache Geode

Redis Adaptor

10

RedisClient

RedisClient

RedisClient

RedisClient

Redis Server

Page 11: Redis adaptor for Apache Geode

Redis Adaptor

11

RedisClient

RedisClient

RedisClient

RedisClient

Geode Server

Drop-in Replacement

Page 12: Redis adaptor for Apache Geode

Start Using GFSH

Redis Adaptor - Usage

12

gfsh>startserver--name=server1--redis-port=11211--redis-bind-address=localhost

Creates a region type PARTITION by default • Change Using system property gemfireredis.regiontype

gfsh>startserver--name=server1--redis-port=11211--J=-Dgemfireredis.regiontype=PARTITION_PERSISTENT

Page 13: Redis adaptor for Apache Geode

• Supports all Redis data structures • Each data structure (except Strings) is

backed by a Geode Partitioned Region • which is a horizontally scalable distributed

ConcurrentHashMap • We create just one Partitioned Region for

storing all Strings • Uses Geode’s OQL and indexes

Redis Adaptor - Implementation

13

Page 14: Redis adaptor for Apache Geode

• One Partitioned Region for Each List

• Track “head” and “tail” position in the Partitioned Region itself as special entries

Future Exploration • Embed the position in “Value” • Place an index on the position,

should help with List Scan

Redis Adaptor - List

14

user dataposition

Key Value

Page 15: Redis adaptor for Apache Geode

• One Partitioned Region for Each Sorted Set

• key is user data, value is user provided score

• Index on score

Redis Adaptor - Sorted Set

15

scoreuser data

Key Value

Page 16: Redis adaptor for Apache Geode

Advantages

16

Page 17: Redis adaptor for Apache Geode

• Scalability • Server side stored procedures • Network Partitioning • WAN replication

Advantages

17

Page 18: Redis adaptor for Apache Geode

• Scale Vertically • Redis server is single threaded • Geode server supports high concurrency • Utilize CPUs on big boxes without having to

manage more instances • Scale data structures - Horizontally

Scalability

18

Page 19: Redis adaptor for Apache Geode

• Redis lost 56% writes during Network Partition*

• Geode servers fence themselves

Network Partition

19

* Aphyr - https://aphyr.com/posts/283-jepsen-redis

Page 20: Redis adaptor for Apache Geode

• Data can be replicated to other clusters over WAN

WAN Replication

20

WAN

Page 21: Redis adaptor for Apache Geode

Roadmap

21

Page 22: Redis adaptor for Apache Geode

• Implement “All” Commands • String: SETRANGE (overwrites part of string) • List: BLPOP, BRPOPLPUSH (Block/insert into another) • Sorted Set: ZINTERSTORE (intersect multiple Sorted

Sets and store result in another) • etc.

• Performance benchmark • Optimize

• Geode API for these data structures?

Contribution Ideas

22

Page 23: Redis adaptor for Apache Geode

Questions?

23