The new ehcache 2.0 and hibernate spi

Caching, and what’s new in Ehcache 2 and Hibernate Caching Provider

Thursday, 17 June 2010

www.terracotta.org

Why Cache?Reasons to cache:

Offload - reducing the amount of resources consumed and hence cost

Performance - increasing the speed of processing

Scale out - distributed caching is a leading scale-out architecture

2


www.terracotta.org

Truncating the request-response Loop

3

load balancerload balancer

HTTPD HTTPD HTTPD

Application server

Application server

MySQLDatabase

TerracottaServerstripe 1

... TerracottaServerstripe n



ehcache ehcache


www.terracotta.org


3


HTTPD HTTPD HTTPD

Application server

Application server

MySQLDatabase





ehcache ehcache


www.terracotta.org


3


HTTPD HTTPD HTTPD

Application server

Application server

MySQLDatabase





ehcache ehcache


www.terracotta.org


3


HTTPD HTTPD HTTPD

Application server

Application server

MySQLDatabase





ehcache ehcache


www.terracotta.org


3


HTTPD HTTPD HTTPD

Application server

Application server

MySQLDatabase





ehcache ehcache


www.terracotta.org

Amdahl’s Law Amdahl's law, after Gene Amdahl, is used to find the system speed up

from a speed up in part of the system.

1 / ((1 - Proportion Sped Up) + Proportion Sped Up / Speed up)

To apply Amdahl’s law you must measure the components of system time and the before and after affect of the perf change made.

It is thus an empirical approach.

Not recommended is the other approach “When all you have is a hammer every problem looks like a nail”. Lots of times it is something completely new. However very few developers will take the time to make careful measurements.

4


www.terracotta.org

Cache Efficiency

cache efficiency = cache hits / total hits

➡ High efficiency = high offload

➡ High efficiency = high performance

5


www.terracotta.org

Why does caching work? Locality of Reference

Pareto Distributions

6


www.terracotta.org

Locality of Reference Many computer systems exhibit the phenomenon of locality of reference.

Data that is near other data or has just been used is more likely to be used again.

Temporal locality - refers to the reuse of specific data and/or resources within relatively small time durations.

Spatial locality - refers to the use of data elements within relatively close storage locations

e.g. this is the reason for hierarchical memory design in computers

7


www.terracotta.org

Pareto Distributions Chris Anderson, of Wired Magazine, coined the term The Long Tail to

refer to Ecommerce systems.

The mathematical term is a Pareto Distribution aka Power Law Distribution.

8


www.terracotta.org

Another Problem...But...

What if the data set is too large to fit in the cache?

What about staleness of data

9


www.terracotta.org

Coherency with SOR classically solved with an automatic expiry. Ehcache has both TTL and

TTI.

Better:

Eternal caching plus a cache invalidation protocol

Write-through or behind caches. The SOR gets updated in-line with the cache. Hibernate read-write and transactional strategies are examples. Also Ehcache CacheWriter

10


www.terracotta.org

Why run a cluster? Availability, most often n+1 redundancy

Scale out

But this creates a new cascade of problems:•N * problem•Cluster coherency problem•CAP theorem limits

11


www.terracotta.org

N * Problem On a single node work is done once and then cached. Cache hits offload.

But in a cluster the work must be done N times, where N is the number of nodes

The solution to the N * times problem is a replicated or distributed cache

replicated cache - the data is copied to each node. All data is held in each node

distributed cache - the most used data is held in a node. The balance of the data is held outside the application node

12


www.terracotta.org

Cluster Coherency Problem Each cache makes independent cache changes. The caches become

different

Solved partly by a replicated or distributed cache

But, without locking, race conditions will cause incoherencies

Solution is a coherent, distributed or replicated cache

13


www.terracotta.org

CAP Theorem PACELCThe CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: consistency, availability and tolerance to partition.

A better explanation of the tradeoffs is PACELC: if there is a partition (P) how does the system tradeoff between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system tradeoff between latency (L) and consistency (C)?

There is no right answer, but the properties to be traded off will be different for different applications. So the solution must be configurable.

1. http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-and-yahoos-little.html

14


http://en.wikipedia.org/wiki/Distributed_computing

http://en.wikipedia.org/wiki/Distributed_computing

www.terracotta.org

About Ehcache

Founded in 2003 Apache 2.0 License Integrated by lots of projects, products Hibernate Provider implemented 2003 Web Caching 2004 Distributed Caching 2006 Greg Luck becomes co-spec lead of JSR107 JCACHE (JSR107) implementation 2007 REST and SOAP APIs 2008 SourceForge Project of the Month March 2009 Acquired by Terracotta 2009; Integrated with Terracotta Server Ehcache 2.0 March 2010 Forrester Wave “Leader” May 2010

15

The world's most widely used Java cache


Ehcache 2

16


www.terracotta.org

Ehcache before Terracotta

1712


www.terracotta.org

Ehcache after Terracotta

1813


www.terracotta.org

Adding a specific ehcache.xml ehcache.xml:

1915

<ehcache> <defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120" />

<cache name="org.hibernate.cache.UpdateTimestampsCache" maxElementsInMemory="10000" timeToIdleSeconds="300" />

<cache name="org.hibernate.cache.StandardQueryCache" maxElementsInMemory="10000" timeToIdleSeconds="300" /></ehcache>


www.terracotta.org

Adding Terracotta ehcache.xml

2016

<ehcache><terracottaConfig url="someserver:9510"/>

<defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120" />

<cache name="com.company.domain.Pets" maxElementsInMemory="10000" eternal=”true”> <terracotta clustered="true" coherent="false"/> </cache>

<cache name="com.company.domain.Clinics" maxElementsInMemory="10000" timeToLiveSeconds="3000"> <terracotta clustered="true" coherent="true"/></cache></ehcache>


www.terracotta.org

Ehcache 2 - New FeaturesHibernate 3.3+ Caching SPI

Old SPI was heavily synchronized and not well suited to clustersNew SPI uses CacheRegionFactoryFully cluster safe with Terracotta Server ArrayUnification of the Ehcache and Terracotta 3.2 providersJTA i.e. transactional caching strategy

JTACache as an XAResourceDetects most common Transaction ManagersOthers configurableWorks with Spring, EJB and manual transactions

21


www.terracotta.org

Ehcache 2 - New FeaturesWrite-behind

Offloads Databases with high write workloadsCacheWriter Interface to implementcache.putWithWriter(...) and cache.removeWithWriter(...)Write-through and Write-behind modesBatching, coalescing and very configurableStandalone with in-memory write-behind queue.TSA with HA, durability and distributed workload balancing

Bulk Loadingincoherent mode for startup or periodic cache loading10 x fasterNo change to the API (put, load etc).SetCoherent(), isCoherent(), waitForCoherent()

22


www.terracotta.org

Ehcache 2 - New Features ...cont.New CAP configurability – per cache basis

coherent – run coherent or incoherent (faster)synchronousWrites – true for ha, false is fastercopyOnRead – true to stop interactions between threads outside of the cacheCluster events – notification of partition and reconnectionNonStopCache - decorated cache favouring availabilityUnlockedReadsView - decorated cache favouring speed

ManagementDynamic Configuration of common cache configs from JMX and DevConsoleNew web-based Monitoring with UI and API

23


www.terracotta.org

Ehcache 2.0 Monitoring Options JMX

• is built in to Ehcache but...• JMX needs use portmap• Slow• Machines may be headless

Terracotta Dev Console (if using Terracotta)

Ehcache Console, new in 2.1

24


www.terracotta.org

Including Web Services

25

Simple + Performant + Coherent + HA + Scaleable

Ehcache

Application

Ehcache

Application

Ehcache

Application

TerracottaServer

TerracottaServer

TerracottaServer

TerracottaServer

TerracottaServer

TerracottaServer

EhcacheServer

PHP App

C App

C# App

Ruby App

REST/HTTP

Web Container


www.terracotta.org

Terracotta Developer Console

26

Cache hit ratiosHit/miss ratesHits on the databaseCache putsDetailed efficiency of cache regions

Dramatically simplifies tuning and operations, and shows the database offload.


www.terracotta.org

Ehcache Console

27

Web based

Configuration

Efficiency

Memory Use

Comes with supported versions

API to connect Operations Monitoring


Exploring Hibernate Caching: Spring Pet Clinic

28


www.terracotta.org

Code - Spring Pet Clinic

29


www.terracotta.org

Pet Clinic Domain Model Domain Objects

VetsSpecialtyOwnerPetPetTypeVisit

30

Sprint PetClinic Domain Model

attr = ""

Pet

attr = ""

Owner

attr = ""

Visit

attr = ""

Vet

attr = ""

Specialty

attr = ""

PetType


www.terracotta.org

Code Steps:

Configure PetClinic for Hibernate

Configure hibernate for second-level cache

Configure hbm file for caching

Update query code to add caching

Optional but recommended:

add ehcache.xml to WEB-INF/classes specify cache regions and config

31


www.terracotta.org

Code ehcache.xml:

32

<ehcache> <defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120" />

<cache name="org.hibernate.cache.UpdateTimestampsCache" maxElementsInMemory="10000" timeToIdleSeconds="300" />

<cache name="org.hibernate.cache.StandardQueryCache" maxElementsInMemory="10000" timeToIdleSeconds="300" /></ehcache>


Performance

33


www.terracotta.org

Performance

34

Testing Methodology

Results in Details

Conclusions


www.terracotta.org

Standalone Performance Read Performance

35


www.terracotta.org

Standalone Performance Put Performance

36


www.terracotta.org

Ehcache in-process vs Memcached

37


www.terracotta.org

REST Performance

38

0

1000

2000

3000

4000

Get Multi-Get Put Remove

Ehcache Server Memcache

Source: MemcacheBench with Java clients. Time for 10,000 operations


www.terracotta.org

Ehcache with Terracotta vs the Rest Application

Tests done with Owners = 25K and 125K which translates to total objects of 0.3 M and 1.5 M

Minimal tuning.

Cluster Configuration:8 Client JVMs (1.75G Heap)1 (+0) Terracotta Servers (6G Heap)MySql: sales18.

39


www.terracotta.org

Ehcache with Terracotta vs the Rest Ehcache

Replicated with RMI not included because not coherentSingle TSA Server 15 threads and some with 100 threads

IMDG15 threads Cache deployed in Partitioned Mode Tests were also done with Replicated – which did well for

small cache sizes but failed to complete with larger cache sizes. So, it is not included.

memcached15 threads1 server

40


www.terracotta.org

Hibernate - Read Only TPS

41


www.terracotta.org

Hibernate - Read Write TPS

42


www.terracotta.org

Hibernate - Read Only Latency

43


www.terracotta.org

Hibernate - Read Write Latency

44


www.terracotta.org

Test SourceThe code behind the benchmarks is in the Terracotta Community SVN repository.

Download https://svn.terracotta.org/repo/forge/projects/ehcacheperf/(Terracotta Community Login Required)

45


https://svn.terracotta.org/repo/forge/projects/ehcacheperf/




www.terracotta.org

Performance ConclusionsWith Hibernate, Using Spring Pet Clinic

After app servers and DBs tuned by independent 3rd parties30-95% database load reduction80 times read-only performance of MySQLNotably lower latency

1.5 ms versus 120 ms for database (25k)

46


www.terracotta.org

What about NoSQL? Ehcache + Terracotta configured with persistence gives you a NoSQL

store with limited features i.e. no search

TerraStore - new open source project from Sergio Bossa is a document oriented NoSQL store based on Terracotta and ful

47


Wrap Up

48


www.terracotta.org

Q&A

Please ask any questions you have in the Q&A window.

49


www.terracotta.org

Additional Ehcache InformationWebsite: www.ehcache.org

Documention: www.ehcache.org/documentation

Hibernate: www.ehcache.org/documentation/

hibernate.html

Commercial Products: www.terracotta.org/ehcache/

Twitter: www.twitter.com/Ehcache

50


http://www.ehcache.org/




http://www.ehcache.org/documentation

http://www.ehcache.org/documentation

http://www.ehcache.org/documentation/hibernate.html




http://www.terracotta.org/ehcache/

http://www.terracotta.org/ehcache/

http://www.twitter.com/Ehcache

http://www.twitter.com/Ehcache

www.terracotta.org

Terracotta Contact InformationWebsite: www.terracottatech.com

Telephone: +1 415-738-4000

Email: [email protected]

Facebook: www.facebook.com/Terracotta

Twitter: www.twitter.com/TerracottaTech

51


http://www.terracottatech.com/

http://www.terracottatech.com/

mailto:[email protected]

mailto:[email protected]

http://www.facebook.com/Terracotta

http://www.facebook.com/Terracotta

http://www.twitter.com/TerracottaTech

http://www.twitter.com/TerracottaTech

Technology

The new ehcache 2.0 and hibernate spi