Caching reboot: javax.cache & Ehcache 3

@ljacomet#ehcachedvx

Caching reboot

javax.cache & Ehcache 3


Agenda


Agenda• Why a cache?



• JSR-107

• APIs and features

• Provider extensions



• JSR-107



• Caching in an application

• Cache aside

• Cache through



• JSR-107




• Cache aside

• Cache through

• Resilience strategy



• JSR-107




• Cache aside

• Cache through

• Resilience strategy

• Extras


Who am I?• Louis Jacomet

• Software engineer, closer to 40 than 20 and still coding!

• Engineer at Terracotta (Software AG) since 2013

• Working on Ehcache OS and EE mostly

• team manager and fiddling in infrastructure

• No idea how to do a (nice) UI - especially a web one

• Enjoys concurrency, thinking hard about APIs, …

Why a cache?


Why a cache?


What were these numbers?


What were these numbers?• Minute:• L1$cache$access$$$$$$$$$$$$$$$0,5$s$$$One$heartbeat$

• Branch$misprediction$$$$$$$$$$5$$$s$$$Yawn$

• L2$cache$access$$$$$$$$$$$$$$$7$$$s$$$Long$yawn$

• Mutex$lock/unlock$$$$$$$$$$$$25$$$s$$$Coffee$preparation






• Hour:• RAM$access$$$$$$$$$$$$$$$$$$100$$$s$$$Brushing$your$teeth$

• Compressing$1K$$$$$$$$$$$$$$$50$$$min$One$TV$show$episode






• Hour:• RAM$access$$$$$$$$$$$$$$$$$$100$$$s$$$Brushing$your$teeth$

• Compressing$1K$$$$$$$$$$$$$$$50$$$min$One$TV$show$episode

• Day:• 2KB$sent$on$1GB$network$$$$$$$5,5$h$$$Your$work$afternoon


What were these numbers?


What were these numbers?• Week:• Random$read$SSD$$$$$$$$$$$$$$$$$1,7$d$$$$$$$A$weekNend$

• Sequence$read$1$MB$from$RAM$$$$$2,9$d$$$$$$$A$long$weekNend$

• Round$trip$in$datacenter$$$$$$$$5,8$d$$$$$$$Holidays$

• Sequence$read$1MB$from$SSD$$$$$11,6$d$$$$$$$Two$weeks$delivery






• Year:• Seek$on$rotational$HDD$$$$$$$$$16,5$weeks$$$School$semester$

• Sequence$read$1MB$from$HDD$$$$$$7,8$months$$Almost$a$baby






• Year:• Seek$on$rotational$HDD$$$$$$$$$16,5$weeks$$$School$semester$

• Sequence$read$1MB$from$HDD$$$$$$7,8$months$$Almost$a$baby

• Decade:• Data$roundNtrip$around$the$world$4,8$years$$PhD$thesis


So what is a cache?


So what is a cache?• Data structure holding a temporary copy of some data



• Trade off between higher memory usage for reduced latency



• Trade off between higher memory usage for reduced latency

• Targets :

• Data which is reused

• Data which is expensive to compute or retrieve


Ehcache (3)


Ehcache (3)• New version, fully integrated with JSR-107

• Breaks compatibility with 2.x line

https://github.com/ehcache/ehcache3




• Developed in the open

• https://github.com/ehcache/ehcache3





• Developed in the open

• https://github.com/ehcache/ehcache3

• Public dev meeting, almost weekly, on hangouts on air

• Recordings available on Youtube


JSR-107 - javx.caching


JSR-107


JSR-107• Ancient JSR

• Started in 2001

• Approved a year ago (March 2014)



• Started in 2001


• javax.caching API + TCK + reference implementation

• Eases integration of a cache in frameworks



• Started in 2001


• javax.caching API + TCK + reference implementation

• Eases integration of a cache in frameworks

• Ongoing discussions about a version 2.0

• Async API for example


JSR-107 : Features


JSR-107 : Features•CacheManager / Cache



•Expiration

• Creation/Access/Update



•Expiration


•Integration

• CacheLoader / Writer



•Expiration


•Integration


•CacheEntryListener

• Created/Updated

• Removed/Expired

• Get old value



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor

•Annotations



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor

•Annotations

•MBeans, exposing

• Configuration

• Statistics



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor

•Annotations

•MBeans, exposing

• Configuration

• Statistics

•No capacity control !

@[email protected]@ljacomet#ehcachedvx

Demo


And my XML??


JSR-107 configuration• Programmatic only

• Enter the world of provider extensions

• Ehcache 3 expects CacheManager URI to resolve to a configuration file

• From there, multiple scenarios are possible


Demo

Caching in an application


public BusinessObject computeAndWin(String param1, String param2) { String key = createKey(param1, param2); BusinessObject cachedResult = cache.get(key); if (cachedResult == null) { cachedResult = loadAndCompute(param1, param2); cache.put(key, cachedResult); } return cachedResult;}

Cache Aside


Cache Aside : not so easy


Cache Aside : not so easy• Requires synchronisation between

• cache

• and system of record



• cache


• JSR-107 has no locking option



• cache


• JSR-107 has no locking option

• Potential to be hairy code quite fast


Cache Aside: options


Cache Aside: options• Rely on the abstraction of your framework

• Spring Caching

• Hibernate

• …


Cache Aside: options• Rely on the abstraction of your framework

• Spring Caching

• Hibernate

• …

• Alternatives ?


public BusinessObject computeAndWin(String param1, String param2) { return cache.get(createKey(param1, param2));}

Cache Through


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through


Cache Through• get* => CacheLoader

• cache miss indicates no data available




• put* => CacheWriter

• Each use means writing to the system of record




• put* => CacheWriter

• Each use means writing to the system of record

• Constraints: APIs of CacheLoader / CacheWriter


Cache Through : JSR-107


Cache Through : JSR-107• Peculiar behaviours



• putIfAbsent(K key, V value): boolean




• Ignores CacheLoader but not CacheWriter





• Ehcache 3 allows to tweak this behaviour through configuration





• Ehcache 3 allows to tweak this behaviour through configuration

• Behaviour consistent across atomic operations


Cache Through : Write Behind


Cache Through : Write Behind• Asynchronous writes to the the system of record

• User thread no longer suffers from the latency of the write


Cache Through : Write Behind• Asynchronous writes to the the system of record

• User thread no longer suffers from the latency of the write

• Introduces its own complexities

• Write queue: persistent or not?

• Mutation guarantee: once and only once? at least once?

• What about cache eviction?

Resilience strategy


Resilience strategy


Resilience strategy• Proposition:

“A cache level error should not be the cause for

a user level error “


Resilience strategy• Proposition:

“A cache level error should not be the cause for

a user level error “

P.S. This is ongoing development and subject to change


Ehcache 3


Ehcache 3• Handle errors internally as much as possible



• In memory cache => No exceptions in the execution thread




• Default is to log errors, will be configurable / replaceable





• Advanced setups (Distributed + write behind for example)






• Force user to think about the use cases where the cache can become inconsistent






• Force user to think about the use cases where the cache can become inconsistent

• Will be required to configure a handler for these cases

Ehcache 3 and Terracotta


Offheap solution


Offheap solution• Solving Garbage Collector pauses on large heaps



• Enables extreme cache scale up

• Multi TerraBytes … as long as your server allows it



• Enables extreme cache scale up

• Multi TerraBytes … as long as your server allows it

• Production tested implementation

• 2.0.0 available on GitHub

• https://github.com/Terracotta-OSS/offheap-store


Clustering solution


Clustering solution• Sharing of data across multiple JVMs



• Terracotta 4.3.0 again has an OpenSource offering

• Integration with Ehcache 2.10.0



• Terracotta 4.3.0 again has an OpenSource offering

• Integration with Ehcache 2.10.0

• Ehcache 3 will have clustering option in OpenSource

• Code starting to appear on GitHub


Questions ?

Caching reboot: javax.cache & Ehcache 3

Text of Caching reboot: javax.cache & Ehcache 3

Distributed Systems - Computer Science€“ More than caching: Hadoop support, high availability) • Ehcache (developed & maintained by Terracotta) – Java-based, supports HA •

Reboot part 2 - reboot your family - 01-15-17-1

Version 2.10.3 October 2016 - ehcache.org · Ehcache Developer Guide Version 2.10.3 October 2016. This document applies to Ehcache Version 2.10.3 and to all subsequent releases. Speciﬁcations

Overview Introduction to ASP.NET caching Output caching Fragment caching Data caching 1

The new Ehcache 2.0.0 and Hibernate Caching SPI Providerstatic.hjug.org/present/New_Ehcache 2.0_Hibernate_SPI.pdf · The new Ehcache 2.0.0 and Hibernate Caching SPI Provider Presented

Reboot Universe

CacheOptimizer: Helping Developers Configure Caching …petertsehsun.github.io/papers/peter_fse2016.pdf · 2020. 9. 11. · Application-level caching frameworks, such as Ehcache [54]

Overview of the ehcache

Brokerage Reboot

D-385 385 · Maven, Springsource Tools Suite, Weblogic Application Server 10.3/10.3.4, Liferay Portal, xPlanner, Spring Rest Support, Spring Integration, EhCache, Distributed Caching

Hibernate Second Level Cache Using EHcache With Spring

Ehcache 3: JSR-107 on steroids at Devoxx Morocco

Version 2.9 October 2014 - Ehcache · 2019-09-24 · Ehcache Developer Guide Version 2.9 12 Creating a CacheManager All usages of the Ehcache API start with the creation of a CacheManager

Ehcache Guide & Reference - Huihoodocs.huihoo.com/ehcache/EhcacheUserGuide-1.4.0.pdf · 3.5 Spring, Cocoon, ... 19.8.4 RMI RMI is the default RPC mechanism in Java. . . . . . .

The new Ehcache 2.0.0 and Hibernate Caching SPI Provider

Terracotta Ehcache : Simpler, faster, distributed

Distributed Caching Using the JCACHE API and ehcache, Including a Case Study on Wotif.com

Ehcache 2.4 User Guide

Version 10.2 April 2018 · 2018-06-04 · M Odd Header Caching Basics Ehcache API Developer Guide Version 10.2 9 Oﬀers numerous standard services out of the box - a good starng

[Conference] Building Websites that Matter - Agent Reboot Boston, Agent Reboot DC, Agent Reboot Austin

Distributed Caching, Using the JCACHE API and ehcache

Ehcache User Guide - Ehcache | Performance at Any Scale

Innovation Release Version 2.10.4 April 2017 - Ehcache Odd Header Using Replication Ehcache Replication Guide Version 2.10.4 (Innovation Release) 5 1 Using Replication Supported Types

Reboot 2012

The new ehcache 2.0 and hibernate spi

Ehcache Architecture, Features And Usage Patterns

Microsoft ASP.NET: An Overview of Caching. 2 Overview Introduction to ASP.NET caching Output caching Data caching Difference between Data Caching

Ehcache User Guide · Ehcache is now used for Hibernate caching, data access object caching, security credential caching, web caching, SOAP and RESTful server caching, application

Reboot part 2 - reboot your family - 01-15-17-

Reboot - Part 4 - Reboot Your Finances - 01-29-17

Technology

Caching reboot: javax.cache & Ehcache 3

Text of Caching reboot: javax.cache & Ehcache 3