Spring Session Redis · 2020-01-02 · Redis would then notify every instance that the...

March 2018

Spring Session Redis

Oded Shopen

Why, How and Production Considerations

About Me

▪ Oded Shopen▪ Software Architect @ Amdocs Israel▪ Working on Event-Driven, Cloud Native Microservices

▪🤓 Redis Geek

▪http://odedia.org ▪@odedia on twitter

Why Spring Session?

▪Once upon a time, there was the servlet container. And it was good…

Tomcat 🍪 123-456 = David

Why Spring Session?

Tomcat

🍪 123-456 = David

Tomcat Tomcat

Load Balancer

🍪 789-111 = Larry

Why Spring Session?

🍪 123-456 = ?

Tomcat Tomcat

Load Balancer

🍪 789-111 = Larry

Why Spring Session?

Tomcat Tomcat

Load Balancer

Tomcat Tomcat

Tomcat

Why Spring Session?

Jetty Tomcat

Load Balancer

Jetty Tomcat

Tomcat Tomcat

Jetty Tomcat

Tomcat

Why Spring Session?

Tomcat

Session Data

Why Spring Session?

Tomcat

Tomcat Jetty

Session Data

Why Spring Session?

▪Replaces the built-in HttpSession with another implementation ▪ Transparent drop-in replacement when using Spring Boot ▪Makes your servers truly stateless ▪ Sessions survive application restarts ▪No need for Load Balancer sticky sessions ▪Conforms to the cloud-native apps 12-factor principals

Twelve-factor processes are stateless and share-nothing.

Any data that needs to persist must be stored in a stateful backing service, typically a database.

Why Spring Session Redis?

https://docs.spring.io/spring-session/docs/1.3.1.RELEASE/reference/html5/

Why Spring Session Redis?

▪ The application requires frequent, fast access to the session ▪A fast database is critical ▪Redis is really fast ▪No user experience degradation by externalizing the session to Redis

▪ Sessions needs to expire after some time ▪Redis expiring keys are a great solution

▪With sharding and clustering, Redis scales when your user base scales

Demohttps://github.com/odedia/spring-session-redis-sample

ProductionConsiderations

#1 Be Prepared to Scale

▪At the very minimum, use a master/slave setup

#1 Be Prepared to Scale

▪Consider sharding your sessions in a Redis Cluster ▪RedisLabs + DNS Proxy will hide the cluster topology and let you focus on

a simple configuration on the client side ▪No need to configure sentinels

Node 1

Node 2

Node 3

Proxyspring.redis.url=proxy:6379spring.redis.sentinel.master=MasterNode spring.redis.sentinel.nodes=Node1:6379,Node2:6379,Node3:6379

#2 Pool Settings▪ Spring Data Redis uses a JedisConnectionFactory with the following defaults:

spring.redis.pool.max-active=8 spring.redis.pool.max-idle=8 spring.redis.pool.max-wait=-1 spring.redis.pool.min-idle=0 spring.redis.timeout=0

▪ Behind the scenes - an Apache Commons GenericObjectPool.

▪ Use max-active based on your tomcat/jetty threads.

▪ Use max-wait based on your setup

▪ Too high - all tomcat threads can get stuck, waiting

▪ Too low - your consumers may get errors

▪ Use min-idle of at least 8 to offset “sudden load” issues (such as mass login on a business day).

▪ Set spring.redis.timeout to a reasonably low number (such as 1000ms)

▪ Too high - your connection pool can become full quickly.

▪ Too low - your consumers may get errors

#3 Server-Side Metrics

▪ Spring Boot Actuator provides a production-ready monitoring metrics. ▪ Simply add spring-boot-starter-actuator to your gradle/maven dependencies ▪ /metrics endpoint provides valuable monitoring data ▪However, Spring Data Redis is not available… ▪ Luckily, actuator is extensible! ▪https://github.com/nysd/spring-boot-redis-metrics

#4 Monitor Redis

▪Redislabs monitoring dashboards provide excellent visibility to identify issues, as we’ll soon see.

▪ If you use open source and monitor yourself on-prem, keep in mind that Redis is a single-threaded database. ▪A 2-core virtual machine would report 50%

CPU utilization ▪However, redis itself might be at 100% at that

time ▪Monitor the process, not the VM

#5 Please Monitor Responsibly

▪Be careful how you monitor… ▪KEYS * runs over all the keys in the DB. Your DB is unresponsive until operation is over! ▪SMEMBERS returns all keys in a given set. Some sets may contain all keys in the system. ▪ In our use case, we kept track of all logins from a certain vendor, which is pretty

much O(n) cardinality, so same as KEYS command ▪SCARD simply returns the size of a given set with cardinality of O(1), so it can be used

safely in production ▪ SCARD “spring:session:index:org.springframework.session.FindByIndexNameSessionRepository.PRINCIPAL_NAME_INDEX_NAME:UI_USERS"

▪ INFO provides valuable information (such as instantaneous_ops_per_sec) and can be safely used in production.

#6 Inside Spring Session Redis

▪What happens when your system is completely idle (0 users)? ▪You would expect redis to be idle as well ▪However…

▪Multiple logins using: ▪for ((i=1;i<=10000000;i++)); do curl -s -u gateway:password localhost:8080/user >/dev/null; done

▪One would expect consistent load on redis, but in reality, we got:

▪ Time to dig into the code… (thanks, open source!) ▪ Inside RedisOperationsSessionRepository:

@Scheduled(cron = "${spring.session.cleanup.cron.expression:0 * * * * *}") public void cleanupExpiredSessions() { this.expirationPolicy.cleanExpiredSessions(); }

▪cleanupExpiredSessions() would loop over expired keys, delete them, and “touch” them to make sure they are immediately deleted.

▪ This means that by default, every minute on the minute, Spring will call delete on every expired key in Redis.

▪Why? Redis can self-expire keys just fine, thank you very much ▪The main reason: ▪Spring wants to conduct cleanup activities when a session is expired. ▪The API exposes SessionDeletedEvent and SessionExpiredEvent to let the

program cleanup a user’s session (close web sockets, notify SSO server etc.) ▪Spring needs the session details available when the cleanup activity is

performed

▪To solve this issue, two separate keys are managed for each session ▪An “expiration notification” key expires after 30 minutes ▪The actual session details key expires 5 minutes later

HMSET spring:session:sessions:33fdd1b6-b496-4b33-9f7d-df96679d32fe creationTime 1404360000000 \maxInactiveInterval 1800 \lastAccessedTime 1404360000000 \sessionAttr:attrName someAttrValue \sessionAttr2:attrName someAttrValue2

EXPIRE spring:session:sessions:33fdd1b6-b496-4b33-9f7d-df96679d32fe 2100APPEND spring:session:sessions:expires:33fdd1b6-b496-4b33-9f7d-df96679d32fe ""EXPIRE spring:session:sessions:expires:33fdd1b6-b496-4b33-9f7d-df96679d32fe 1800SADD spring:session:expirations:1439245080000 expires:33fdd1b6-b496-4b33-9f7d-df96679d32feEXPIRE spring:session:expirations1439245080000 2100

▪Redis guarantees expired keys would be passively cleaned, but makes no guarantees on when

▪Spring Session Redis wants to make sure the “expire” key is expired within the 5 minutes window.

▪For this reason, the cronjob updates the expired keys every minute

▪But, there is a risk here…

Tomcat

Tomcat Jetty

Session Data

Tomcat

Tomcat Jetty

▪ To recap: ▪Every minute, each server tries to delete all “expires” keys. ▪A lot of redundant calls, since only the first instance would actually achieve this

purpose. ▪Redis would then notify every instance that the “expires” key was deleted. ▪Every instance would then connect to Redis to get the session details about the

expiring session to handle the expiration. ▪A little bit of math: ▪100 servers * 2500 expiring sessions = 500,000 access at the same second ▪ The more you scale, the bigger the problem

#6 Inside Spring Session RedisSolutions: ▪ If you don’t care about session expiration events, set the cron job to an impossible value such as

February 31st: spring.session.cleanup.cron.expression=0 0 5 31 2 ?

▪ You can also disable registering to those expiration and deletion events. Note that all servers need to disable the registering for this to work: @EnableRedisHttpSession

public class RedisOperationsSessionRepositoryConfigNoOp { @Bean public static ConfigureRedisAction configureRedisAction() { return ConfigureRedisAction.NO_OP; } }

▪If you do care about session expiration events, set only a specific instance(s) to handle those events.

▪On all other servers, disable receiving the events: @EnableRedisHttpSession

public class RedisOperationsSessionRepositoryConfigNoListener { @Bean

public RedisMessageListenerContainer redisMessageListenerContainer( RedisConnectionFactory connectionFactory, RedisOperationsSessionRepository messageListener) {

return new RedisMessageListenerContainer(); } }

To Recap

▪Be prepared to scale ▪Monitor well, monitor with caution ▪Open source rocks 🤟. Get involved! Tweak the

framework for your own use case.

Thank you

http://odedia.org

Spring Session Redis · 2020-01-02 · Redis would then notify every instance that the...

Documents

Redis Day Keynote Salvatore Sanfillipo Redis Labs

RedisConf17 - Building Large High Performance Redis Databases with Redis Enterprise

Redis : Play buzz uses Redis

Advances in Instance-Based Learning Algorithmsalgorithms. It then describes previous research in instance-based learning, including It then describes previous research in instance-based

Introduction to Redis

Redis Introduction

Amazon ElastiCache for Redis - ElastiCache for Redis … · Amazon ElastiCache for Redis ElastiCache for Redis User Guide Use Cases and How ElastiCache Can Help Use Cases and How

What's new with enterprise Redis - Leena Joshi, Redis Labs

Spring Data Redis Reference Documentation · Spring Data Redis 1.2.0.RELEASE Spring Data Redis Reference Documentation 2 1. Why Spring Data Redis? The Spring Framework is the leading

Redis for search - Dvir Volk, Redis Labs

Redis in Kenshoo microservices - Redis Labs · Redis in Kenshoo microservices Jesque based delayed priority queue system Presented by Dima Vizelman. A Redis Use-Case Performance Aggregation

Amazon · Amazon ElastiCache for Redis ElastiCache for Redis User Guide Table of Contents What Is ElastiCache for Redis

redis überall

Redis/Lessons learned

Redis for Security Data : SecurityScorecard JVM Redis Usage

Using Redis as a Cache Backend in MagentoUsing Redis as a

Redis cluster

Developer`s Overview of Sonic Part 2…Redis Server database container (multi Redis Instance) Redis Server 2-way read\write to support Warm Restart 1.) Redis-server, different DBs,

Redis edu 2

Redis Persistence