BuildandDeployContainerized RedisCache for&High ... · Redisserviceslave annouceDipsnippetin& redis.conf& slave-announce-port 31250

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Build and Deploy Containerized Redis Cache for High Availability and Performance

Uma Sabada Irfan Ahmed


Agenda What is Redis?

Redis commands

Popular usecases

Scaling

Design choices

Deployment architecture

CI/CD

Summary

2


What is Redis? Redis is a fast in-‐memory cache and a data structure store

Supports replicaEon

Supports clustering

Persistence

Polyglot Clients

Performance over consistency

Monitoring

3


Redis Commands in a nutshell 5 different data types – Strings – Hashes – Lists – Sets – Sorted sets

TransacEon support using MULTI command

Lua ScripEng for server side processing

Pub/Sub

4


Usecases Caching – Session store for ecommerce applicaFons – Reduce page load Fmes for wordpress

Leaderboard elecEons – Gaming apps, Video ranking/raFng

Pub/Sub – Chat applicaFons, inventory management

Time series analysis – Stocks, IoT

Task management – Add tasks to a queue; removed by background processes

5


Deploying a containerized Redis


Design Goals High availability –  Seamless failover handling

Scalability –  Master/Slave architecture

Density –  Efficient use of hardware resources

Minimize data loss during upgrade/patching –  Rollback on error

CI/CD –  New features available immediately

7

1

2

3

4

5


ImplementaEon Choices Docker Containers

Mesos/Marathon

Networking (Bridge network versus overlay)

SenEnel –  Heartbeat –  Failover

Unchanged master endpoint during failover – Clients don’t have to change

8

1

2

3

5

4


Scale cache size from smaller to larger

9

Scale reads with mulFple replicas

Scale beyond one machine’s network and

resource capabiliFes using sharding

Scaling in Redis

Host 2

Slave

Host 3

Slave

Host 1

Master

Host 1

Master

5G

1G

Slave N

Slave 1

Host 1..N

P1

Master

Slave N

Slave 1

Host N+1..2N

P2

Master


Mesos Infrastructure Setup

10

Slave

Standby

Slave

Standby

Slave

Leader

ZK

ZK ZK

Host 2

Host 1

Host 3


Deployment Architecture

11

Host 2

SenEnel

Slave

P

FloaEng IP

Redis Client

Host 3

SenEnel

Slave

P

Host 4

Slave

P Host 1

SenEnel

Master

P


Oracle Cloud Deployment

12

P

P

Availability Domain 3


P

FLIP

App




Docker Networking Details Uses Bridge networking Redis service slave-‐annouce-‐ip snippet in redis.conf slave-announce-port 31250 slave-announce-ip "172.19.0.1"

SenEnel announce-‐ip snippet in senEnel.confsentinel announce-ip "172.19.0.1" sentinel announce-port 31821

13


Master failover SenEnel detects failure of the master

SenEnel through quorum picks a slave to be the new master

SenEnel reconfigures all other slaves to be the slave of the new master

New master’s IP is plumbed to the FLoaEng IP (FLIP)

14


Upgrade with minimal data loss Upgrade to new master –  Deploy a new container – Make the new container slave of the old master –  Unplumb the FLIP from the old master – Wait for new master to be in sync with old master –  Force master failover to the new container –  Plumb the FLIP to the new master

Rolling upgrade of slaves –  Bring up new slave with the new master –  Bring down old slave

Bring down old master

15


Availability of Redis Mesos/Marathon HA using zookeper – Replicated Zoo Keeper and leader elecFon

Master/Slave HA using senEnel – Watchdog monitoring – Failover and promoFon of a slave

Process level HA – Achieved by placing senFnel in the different host from Redis instance

Availability of the service during upgrade – Minimize downFme

16


CI/CD Flow

17

Developer Triggered

v  Build comp-‐under-‐test from git master

v  Unit &funcFonal tests

v  Build image &publish to registry

v  Create local M/M plaform

v  Provision promoted plaform and comp-‐under-‐test

v  QA tests v  Acceptance tests

v  Update the manifest with new promoted component

v  Provision using the new manifest

v  QA tests v  Publish new

manifest

v  IniFalize M/M with old manifest

v  Upgrade with new manifest

v  QA tests run before, during and aher upgrade

v  IniFalize M/M with current prod manifest

v  Upgrade with new manifest that has many new components

v  QA tests run before, during and aher upgrade

Release Verify Promote Upgrade Post-‐Upgrade

Customer Delivery


Summary Redis solves many of the distributed real Eme use cases with high performance and low latency

Deploying Redis in a replicated setup allows scaling of your applicaEon

High availability of Redis can be achieved by: – Seamless failover handling

– Upgrade with minimal data loss

Containers streamline CI/CD and aid agile development

18


Demo

CreaEon of a Redis cache with mulEple read replicas Failover of the master

19


Q & A

20

Documents

BuildandDeployContainerized RedisCache for&High ... · Redisserviceslave annouceDipsnippetin& redis.conf& slave-announce-port 31250