Ehcache 3: JSR-107 on steroids at Devoxx Morocco

@ljacomet#DevoxxMA #ehcache

Ehcache 3 JSR-107 on steroids

Louis Jacomet - Lead Software Engineer


Who am I?• Louis Jacomet

• Software engineer, closer to 40 than 20 and still coding!

• @ Terracotta (Software AG) since 2013

• Working on Ehcache mostly

• Team manager and fiddling in infrastructure

• No idea how to do a (nice) UI, especially a web one

• Enjoys concurrency, API design and hard problems


And what about you?• Who knows nothing about caching?

• Who already uses caching in production?

• Who had production problems related to caching?

• Who knows about JCache?


Agenda•Caching and JSR-107

•Ehcache past and present

•Ehcache 3 future


Caching and JSR-107


Caching• Data structure holding a temporary copy of data

• Trade off between higher memory usage and reduced latency

• Targets

• Data which is reused

• Data which is expensive to compute / retrieve


Caches all over the place

CPU

Browser

Network

Disk


What should I cache?• Keys

• Proper equals/hashcode

• Immutable

• Small(ish)

• Value

• Proper equals/hashcode

• Ideally immutable

• Think concurrent otherwise

• Size matters


JSR-107 javax.cache

• Java Community Process driven standard

• Specifies API and semantics for temporary, in-memory caching of Java objects, including object creation, shared access, spooling, invalidation, and consistency across JVM's


JSR-107 Features• CacheManager / Cache

• Expiration

• Creation/Access/Update

• Integration

• CacheLoader / Writer

• CacheEntryListener

• Created/Updated

• Removed/Expired

• Get old value

• Entry processor

• Annotations

• MBeans, exposing

• Configuration

• Statistics

• No capacity control !


JSR-107 gotcha’s• MutableConfiguration

• Always invoke MutableConfiguration.setTypes(Class, Class)

• Otherwise Cache only knows about Object

• Adding a CacheLoader factory is not enough - need to invoke setReadThrough(boolean)


JSR-107 gotcha’s (2)• Compare and Swap operations with a CacheLoader

• Loader is ignored

• putIfAbsent checks Cache only, not underlying system of record

• Results in a weird contract


Ehcache past and present


A bit of history• Announced in November 2003 on

• v1.0 released on September 2004

• Only a few days before Java 5 and generics

• Acquired in 2009 by

• v2.0 released in March 2010

• Terracotta acquired in 2011 by


Storage model: tiers• Multiple storage tiers

• Heap: like objects in a regular hash map

• Off-heap: binary form stored outside of Garbage Collector reach

• Disk: binary form stored on disk

• Clustered: binary form stored on other server(s)


Overflow model

Heap

Disk


Heap

Disk

Overflow model


Disk

Heap

Overflow model


Disk

Heap

Overflow model


Disk

Heap

Overflow model


Authoritative tier model

Heap

Disk


Disk

Heap



Heap

Disk



Heap

Disk



Heap

Disk



Why the new model?• Predictable latency

• Every put pays the price of the lower / slowest tier

• No degradation when a higher tier gets full

• Preserves fast(er) access for hot set

• When a mapping is accessed it moves to the fastest tier available


Not a cache feature?• Search

• What does searching on a hot set means?

• Even without interacting with the cache, two execution of the same query could return different results

• Pinning

• Prevent eviction from cache entries

• May adversly effect the ability of the cache to respect size constraints


Ehcache 3 The future is here


Lifecycle - explicit• CacheManager

• init / close methods

• create and close caches

• PersistentCacheManager

• Maintenance mode

• Destroy cache data


Cache<K, V> for proper typing• Ehcache 2.x not generic, was time to change

that

• Needs to be known at runtime

• No one wants a ClassCastException

• Cache data may live beyond application lifecycle

• Usage show that frequently key and values are of the same type in a given cache


Serialization• Required as soon as we move away from the

heap

• Fully customisable in Ehcache 3


CacheLoaderWriter• Enables the cache-through pattern

• Consistent across all cache operations

• Single interface

• Loader only - make sure write related methods throw

• Writer only - what?

• Anyone having a good use case for this - talk to us


Cache through - miss

Application Cache

Database



Application Cache

Database



Application Cache

Database



Application Cache

Database



Application Cache

Database


Cache through - hit

Application Cache

Database


Cache through - hit

Application Cache

Database


Cache through - hit

Application Cache

Database


Cache through - write

Application Cache

Database



Application Cache

Database



Application Cache

Database



Application Cache

Database


Write behind• Enables scaling your writes

• Putter does not pay for write latency

• Batching and coalescing options

• Simplified compared to 2.x - no more sizing gotchas


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind


Application Cache

Database

queue

Write behind - eviction

(K1, V1)

(K1, V1)


Application Cache

Database

queue


(K1, V1)


Application Cache

Database

queue


(K1, V1)


Application Cache

Database

queue


(K1, V1)


Application Cache

Database

queue


(K1, V1)


Write behind - failure

Application Cache

queue

Database



Application Cache

queue

Database



Application Cache

queue

Database



Application Cache

queue

Database



Application Cache

queue

Database



Application Cache

queue

Database



Application Cache

queue

Database



Application Cache

queue

Database



Database


Database



queue

Database



queue

Database



queue

Database



Expiry• Offers context


Eviction Veto and Prioritizer• Veto

• Enables to suggest a veto on certain mappings

• Prioritizer

• Allows to sort sample before picking eviction victim

• Not applicable to all stores

• Both are hints only - capacity always takes priority


Transactions• Full blown XA resource

• includes recovery support

• Needs work for transaction managers integration

• Code isolated in its own module

• Currently part of the main Ehcache 3 jar


Clustering using Terracotta


Automatic Resource Control• Enables sizing in natural units

• n bytes on heap

• Percentage of the max heap available

• Define resource pools at the cache manager

• Cross eviction amongst caches

• Enables auto balancing


Resilience strategy• A cache is to be resilient

• Yet… sh*t happens

• Storage failures

• SLA violations (timeouts?)

• Very explicit contract at this time

• To be refined when clustering is wrapping up


Q&AThank you for your time

Technology

Ehcache 3: JSR-107 on steroids at Devoxx Morocco