Caching reboot: javax.cache & Ehcache 3

@ljacomet#ehcachedvx

Caching reboot

javax.cache & Ehcache 3


Agenda


Agenda• Why a cache?



• JSR-107

• APIs and features

• Provider extensions



• JSR-107



• Caching in an application

• Cache aside

• Cache through



• JSR-107




• Cache aside

• Cache through

• Resilience strategy



• JSR-107




• Cache aside

• Cache through

• Resilience strategy

• Extras


Who am I?• Louis Jacomet

• Software engineer, closer to 40 than 20 and still coding!

• Engineer at Terracotta (Software AG) since 2013

• Working on Ehcache OS and EE mostly

• team manager and fiddling in infrastructure

• No idea how to do a (nice) UI - especially a web one

• Enjoys concurrency, thinking hard about APIs, …

Why a cache?


Why a cache?


What were these numbers?


What were these numbers?• Minute:• L1$cache$access$$$$$$$$$$$$$$$0,5$s$$$One$heartbeat$

• Branch$misprediction$$$$$$$$$$5$$$s$$$Yawn$

• L2$cache$access$$$$$$$$$$$$$$$7$$$s$$$Long$yawn$

• Mutex$lock/unlock$$$$$$$$$$$$25$$$s$$$Coffee$preparation






• Hour:• RAM$access$$$$$$$$$$$$$$$$$$100$$$s$$$Brushing$your$teeth$

• Compressing$1K$$$$$$$$$$$$$$$50$$$min$One$TV$show$episode






• Hour:• RAM$access$$$$$$$$$$$$$$$$$$100$$$s$$$Brushing$your$teeth$

• Compressing$1K$$$$$$$$$$$$$$$50$$$min$One$TV$show$episode

• Day:• 2KB$sent$on$1GB$network$$$$$$$5,5$h$$$Your$work$afternoon


What were these numbers?


What were these numbers?• Week:• Random$read$SSD$$$$$$$$$$$$$$$$$1,7$d$$$$$$$A$weekNend$

• Sequence$read$1$MB$from$RAM$$$$$2,9$d$$$$$$$A$long$weekNend$

• Round$trip$in$datacenter$$$$$$$$5,8$d$$$$$$$Holidays$

• Sequence$read$1MB$from$SSD$$$$$11,6$d$$$$$$$Two$weeks$delivery






• Year:• Seek$on$rotational$HDD$$$$$$$$$16,5$weeks$$$School$semester$

• Sequence$read$1MB$from$HDD$$$$$$7,8$months$$Almost$a$baby






• Year:• Seek$on$rotational$HDD$$$$$$$$$16,5$weeks$$$School$semester$

• Sequence$read$1MB$from$HDD$$$$$$7,8$months$$Almost$a$baby

• Decade:• Data$roundNtrip$around$the$world$4,8$years$$PhD$thesis


So what is a cache?


So what is a cache?• Data structure holding a temporary copy of some data



• Trade off between higher memory usage for reduced latency



• Trade off between higher memory usage for reduced latency

• Targets :

• Data which is reused

• Data which is expensive to compute or retrieve


Ehcache (3)


Ehcache (3)• New version, fully integrated with JSR-107

• Breaks compatibility with 2.x line

https://github.com/ehcache/ehcache3




• Developed in the open

• https://github.com/ehcache/ehcache3





• Developed in the open

• https://github.com/ehcache/ehcache3

• Public dev meeting, almost weekly, on hangouts on air

• Recordings available on Youtube


JSR-107 - javx.caching


JSR-107


JSR-107• Ancient JSR

• Started in 2001

• Approved a year ago (March 2014)



• Started in 2001


• javax.caching API + TCK + reference implementation

• Eases integration of a cache in frameworks



• Started in 2001


• javax.caching API + TCK + reference implementation

• Eases integration of a cache in frameworks

• Ongoing discussions about a version 2.0

• Async API for example


JSR-107 : Features


JSR-107 : Features•CacheManager / Cache



•Expiration

• Creation/Access/Update



•Expiration


•Integration

• CacheLoader / Writer



•Expiration


•Integration


•CacheEntryListener

• Created/Updated

• Removed/Expired

• Get old value



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor

•Annotations



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor

•Annotations

•MBeans, exposing

• Configuration

• Statistics



•Expiration


•Integration



• Created/Updated

• Removed/Expired

• Get old value

•Entry processor

•Annotations

•MBeans, exposing

• Configuration

• Statistics

•No capacity control !

@YourTwitterHandle@YourTwitterHandle@ljacomet#ehcachedvx

Demo


And my XML??


JSR-107 configuration• Programmatic only

• Enter the world of provider extensions

• Ehcache 3 expects CacheManager URI to resolve to a configuration file

• From there, multiple scenarios are possible


Demo

Caching in an application


public BusinessObject computeAndWin(String param1, String param2) { String key = createKey(param1, param2); BusinessObject cachedResult = cache.get(key); if (cachedResult == null) { cachedResult = loadAndCompute(param1, param2); cache.put(key, cachedResult); } return cachedResult;}

Cache Aside


Cache Aside : not so easy


Cache Aside : not so easy• Requires synchronisation between

• cache

• and system of record



• cache


• JSR-107 has no locking option



• cache


• JSR-107 has no locking option

• Potential to be hairy code quite fast


Cache Aside: options


Cache Aside: options• Rely on the abstraction of your framework

• Spring Caching

• Hibernate

• …


Cache Aside: options• Rely on the abstraction of your framework

• Spring Caching

• Hibernate

• …

• Alternatives ?


public BusinessObject computeAndWin(String param1, String param2) { return cache.get(createKey(param1, param2));}

Cache Through


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through

Cache

Application code

RDBMS


Cache Through


Cache Through• get* => CacheLoader

• cache miss indicates no data available




• put* => CacheWriter

• Each use means writing to the system of record




• put* => CacheWriter

• Each use means writing to the system of record

• Constraints: APIs of CacheLoader / CacheWriter


Cache Through : JSR-107


Cache Through : JSR-107• Peculiar behaviours



• putIfAbsent(K key, V value): boolean




• Ignores CacheLoader but not CacheWriter





• Ehcache 3 allows to tweak this behaviour through configuration





• Ehcache 3 allows to tweak this behaviour through configuration

• Behaviour consistent across atomic operations


Cache Through : Write Behind


Cache Through : Write Behind• Asynchronous writes to the the system of record

• User thread no longer suffers from the latency of the write


Cache Through : Write Behind• Asynchronous writes to the the system of record

• User thread no longer suffers from the latency of the write

• Introduces its own complexities

• Write queue: persistent or not?

• Mutation guarantee: once and only once? at least once?

• What about cache eviction?

Resilience strategy


Resilience strategy


Resilience strategy• Proposition:

“A cache level error should not be the cause for

a user level error “


Resilience strategy• Proposition:

“A cache level error should not be the cause for

a user level error “

P.S. This is ongoing development and subject to change


Ehcache 3


Ehcache 3• Handle errors internally as much as possible



• In memory cache => No exceptions in the execution thread




• Default is to log errors, will be configurable / replaceable





• Advanced setups (Distributed + write behind for example)






• Force user to think about the use cases where the cache can become inconsistent






• Force user to think about the use cases where the cache can become inconsistent

• Will be required to configure a handler for these cases

Ehcache 3 and Terracotta


Offheap solution


Offheap solution• Solving Garbage Collector pauses on large heaps



• Enables extreme cache scale up

• Multi TerraBytes … as long as your server allows it



• Enables extreme cache scale up

• Multi TerraBytes … as long as your server allows it

• Production tested implementation

• 2.0.0 available on GitHub

• https://github.com/Terracotta-OSS/offheap-store


Clustering solution


Clustering solution• Sharing of data across multiple JVMs



• Terracotta 4.3.0 again has an OpenSource offering

• Integration with Ehcache 2.10.0



• Terracotta 4.3.0 again has an OpenSource offering

• Integration with Ehcache 2.10.0

• Ehcache 3 will have clustering option in OpenSource

• Code starting to appear on GitHub


Questions ?

Technology

Caching reboot: javax.cache & Ehcache 3