Embed Size (px)
DESCRIPTION
Prezos about cache, Ehcache 2 and Terracotta by Greg Luck @ Zenika (@ZenikaIT on twitter)
Caching, and what’s new in Ehcache 2 and Hibernate Caching Provider
Thursday, 17 June 2010
www.terracotta.org
Why Cache?Reasons to cache:
Offload - reducing the amount of resources consumed and hence cost
Performance - increasing the speed of processing
Scale out - distributed caching is a leading scale-out architecture
2
Thursday, 17 June 2010
www.terracotta.org
Truncating the request-response Loop
3
load balancerload balancer
HTTPD HTTPD HTTPD
Application server
Application server
MySQLDatabase
TerracottaServerstripe 1
... TerracottaServerstripe n
TerracottaServerstripe 1
... TerracottaServerstripe n
ehcache ehcache
Thursday, 17 June 2010
www.terracotta.org
Truncating the request-response Loop
3
load balancerload balancer
HTTPD HTTPD HTTPD
Application server
Application server
MySQLDatabase
TerracottaServerstripe 1
... TerracottaServerstripe n
TerracottaServerstripe 1
... TerracottaServerstripe n
ehcache ehcache
Thursday, 17 June 2010
www.terracotta.org
Truncating the request-response Loop
3
load balancerload balancer
HTTPD HTTPD HTTPD
Application server
Application server
MySQLDatabase
TerracottaServerstripe 1
... TerracottaServerstripe n
TerracottaServerstripe 1
... TerracottaServerstripe n
ehcache ehcache
Thursday, 17 June 2010
www.terracotta.org
Truncating the request-response Loop
3
load balancerload balancer
HTTPD HTTPD HTTPD
Application server
Application server
MySQLDatabase
TerracottaServerstripe 1
... TerracottaServerstripe n
TerracottaServerstripe 1
... TerracottaServerstripe n
ehcache ehcache
Thursday, 17 June 2010
www.terracotta.org
Truncating the request-response Loop
3
load balancerload balancer
HTTPD HTTPD HTTPD
Application server
Application server
MySQLDatabase
TerracottaServerstripe 1
... TerracottaServerstripe n
TerracottaServerstripe 1
... TerracottaServerstripe n
ehcache ehcache
Thursday, 17 June 2010
www.terracotta.org
Amdahl’s Law Amdahl's law, after Gene Amdahl, is used to find the system speed up
from a speed up in part of the system.
1 / ((1 - Proportion Sped Up) + Proportion Sped Up / Speed up)
To apply Amdahl’s law you must measure the components of system time and the before and after affect of the perf change made.
It is thus an empirical approach.
Not recommended is the other approach “When all you have is a hammer every problem looks like a nail”. Lots of times it is something completely new. However very few developers will take the time to make careful measurements.
4
Thursday, 17 June 2010
www.terracotta.org
Cache Efficiency
cache efficiency = cache hits / total hits
➡ High efficiency = high offload
➡ High efficiency = high performance
5
Thursday, 17 June 2010
www.terracotta.org
Why does caching work? Locality of Reference
Pareto Distributions
6
Thursday, 17 June 2010
www.terracotta.org
Locality of Reference Many computer systems exhibit the phenomenon of locality of reference.
Data that is near other data or has just been used is more likely to be used again.
Temporal locality - refers to the reuse of specific data and/or resources within relatively small time durations.
Spatial locality - refers to the use of data elements within relatively close storage locations
e.g. this is the reason for hierarchical memory design in computers
7
Thursday, 17 June 2010
www.terracotta.org
Pareto Distributions Chris Anderson, of Wired Magazine, coined the term The Long Tail to
refer to Ecommerce systems.
The mathematical term is a Pareto Distribution aka Power Law Distribution.
8
Thursday, 17 June 2010
www.terracotta.org
Another Problem...But...
What if the data set is too large to fit in the cache?
What about staleness of data
9
Thursday, 17 June 2010
www.terracotta.org
Coherency with SOR classically solved with an automatic expiry. Ehcache has both TTL and
TTI.
Better:
Eternal caching plus a cache invalidation protocol
Write-through or behind caches. The SOR gets updated in-line with the cache. Hibernate read-write and transactional strategies are examples. Also Ehcache CacheWriter
10
Thursday, 17 June 2010
www.terracotta.org
Why run a cluster? Availability, most often n+1 redundancy
Scale out
But this creates a new cascade of problems:•N * problem•Cluster coherency problem•CAP theorem limits
11
Thursday, 17 June 2010
www.terracotta.org
N * Problem On a single node work is done once and then cached. Cache hits offload.
But in a cluster the work must be done N times, where N is the number of nodes
The solution to the N * times problem is a replicated or distributed cache
replicated cache - the data is copied to each node. All data is held in each node
distributed cache - the most used data is held in a node. The balance of the data is held outside the application node
12
Thursday, 17 June 2010
www.terracotta.org
Cluster Coherency Problem Each cache makes independent cache changes. The caches become
different
Solved partly by a replicated or distributed cache
But, without locking, race conditions will cause incoherencies
Solution is a coherent, distributed or replicated cache
13
Thursday, 17 June 2010
www.terracotta.org
CAP Theorem PACELCThe CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: consistency, availability and tolerance to partition.
A better explanation of the tradeoffs is PACELC: if there is a partition (P) how does the system tradeoff between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system tradeoff between latency (L) and consistency (C)?
There is no right answer, but the properties to be traded off will be different for different applications. So the solution must be configurable.
1. http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-and-yahoos-little.html
14
Thursday, 17 June 2010
www.terracotta.org
About Ehcache
Founded in 2003 Apache 2.0 License Integrated by lots of projects, products Hibernate Provider implemented 2003 Web Caching 2004 Distributed Caching 2006 Greg Luck becomes co-spec lead of JSR107 JCACHE (JSR107) implementation 2007 REST and SOAP APIs 2008 SourceForge Project of the Month March 2009 Acquired by Terracotta 2009; Integrated with Terracotta Server Ehcache 2.0 March 2010 Forrester Wave “Leader” May 2010
15
The world's most widely used Java cache
Thursday, 17 June 2010
Ehcache 2
16
Thursday, 17 June 2010
www.terracotta.org
Ehcache before Terracotta
1712
Thursday, 17 June 2010
www.terracotta.org
Ehcache after Terracotta
1813
Thursday, 17 June 2010
www.terracotta.org
Adding a specific ehcache.xml ehcache.xml:
1915
<ehcache> <defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120" />
<cache name="org.hibernate.cache.UpdateTimestampsCache" maxElementsInMemory="10000" timeToIdleSeconds="300" />
<cache name="org.hibernate.cache.StandardQueryCache" maxElementsInMemory="10000" timeToIdleSeconds="300" /></ehcache>
Thursday, 17 June 2010
www.terracotta.org
Adding Terracotta ehcache.xml
2016
<ehcache><terracottaConfig url="someserver:9510"/>
<defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120" />
<cache name="com.company.domain.Pets" maxElementsInMemory="10000" eternal=”true”> <terracotta clustered="true" coherent="false"/> </cache>
<cache name="com.company.domain.Clinics" maxElementsInMemory="10000" timeToLiveSeconds="3000"> <terracotta clustered="true" coherent="true"/></cache></ehcache>
Thursday, 17 June 2010
www.terracotta.org
Ehcache 2 - New FeaturesHibernate 3.3+ Caching SPI
Old SPI was heavily synchronized and not well suited to clustersNew SPI uses CacheRegionFactoryFully cluster safe with Terracotta Server ArrayUnification of the Ehcache and Terracotta 3.2 providersJTA i.e. transactional caching strategy
JTACache as an XAResourceDetects most common Transaction ManagersOthers configurableWorks with Spring, EJB and manual transactions
21
Thursday, 17 June 2010
www.terracotta.org
Ehcache 2 - New FeaturesWrite-behind
Offloads Databases with high write workloadsCacheWriter Interface to implementcache.putWithWriter(...) and cache.removeWithWriter(...)Write-through and Write-behind modesBatching, coalescing and very configurableStandalone with in-memory write-behind queue.TSA with HA, durability and distributed workload balancing
Bulk Loadingincoherent mode for startup or periodic cache loading10 x fasterNo change to the API (put, load etc).SetCoherent(), isCoherent(), waitForCoherent()
22
Thursday, 17 June 2010
www.terracotta.org
Ehcache 2 - New Features ...cont.New CAP configurability – per cache basis
coherent – run coherent or incoherent (faster)synchronousWrites – true for ha, false is fastercopyOnRead – true to stop interactions between threads outside of the cacheCluster events – notification of partition and reconnectionNonStopCache - decorated cache favouring availabilityUnlockedReadsView - decorated cache favouring speed
ManagementDynamic Configuration of common cache configs from JMX and DevConsoleNew web-based Monitoring with UI and API
23
Thursday, 17 June 2010
www.terracotta.org
Ehcache 2.0 Monitoring Options JMX
• is built in to Ehcache but...• JMX needs use portmap• Slow• Machines may be headless
Terracotta Dev Console (if using Terracotta)
Ehcache Console, new in 2.1
24
Thursday, 17 June 2010
www.terracotta.org
Including Web Services
25
Simple + Performant + Coherent + HA + Scaleable
Ehcache
Application
Ehcache
Application
Ehcache
Application
TerracottaServer
TerracottaServer
TerracottaServer
TerracottaServer
TerracottaServer
TerracottaServer
EhcacheServer
PHP App
C App
C# App
Ruby App
REST/HTTP
Web Container
Thursday, 17 June 2010
www.terracotta.org
Terracotta Developer Console
26
Cache hit ratiosHit/miss ratesHits on the databaseCache putsDetailed efficiency of cache regions
Dramatically simplifies tuning and operations, and shows the database offload.
Thursday, 17 June 2010
www.terracotta.org
Ehcache Console
27
Web based
Configuration
Efficiency
Memory Use
Comes with supported versions
API to connect Operations Monitoring
Thursday, 17 June 2010
Exploring Hibernate Caching: Spring Pet Clinic
28
Thursday, 17 June 2010
www.terracotta.org
Code - Spring Pet Clinic
29
Thursday, 17 June 2010
www.terracotta.org
Pet Clinic Domain Model Domain Objects
VetsSpecialtyOwnerPetPetTypeVisit
30
Sprint PetClinic Domain Model
attr = ""
Pet
attr = ""
Owner
attr = ""
Visit
attr = ""
Vet
attr = ""
Specialty
attr = ""
PetType
Thursday, 17 June 2010
www.terracotta.org
Code Steps:
Configure PetClinic for Hibernate
Configure hibernate for second-level cache
Configure hbm file for caching
Update query code to add caching
Optional but recommended:
add ehcache.xml to WEB-INF/classes specify cache regions and config
31
Thursday, 17 June 2010
www.terracotta.org
Code ehcache.xml:
32
<ehcache> <defaultCache maxElementsInMemory="10000" eternal="false" timeToLiveSeconds="120" />
<cache name="org.hibernate.cache.UpdateTimestampsCache" maxElementsInMemory="10000" timeToIdleSeconds="300" />
<cache name="org.hibernate.cache.StandardQueryCache" maxElementsInMemory="10000" timeToIdleSeconds="300" /></ehcache>
Thursday, 17 June 2010
Performance
33
Thursday, 17 June 2010
www.terracotta.org
Performance
34
Testing Methodology
Results in Details
Conclusions
Thursday, 17 June 2010
www.terracotta.org
Standalone Performance Read Performance
35
Thursday, 17 June 2010
www.terracotta.org
Standalone Performance Put Performance
36
Thursday, 17 June 2010
www.terracotta.org
Ehcache in-process vs Memcached
37
Thursday, 17 June 2010
www.terracotta.org
REST Performance
38
0
1000
2000
3000
4000
Get Multi-Get Put Remove
Ehcache Server Memcache
Source: MemcacheBench with Java clients. Time for 10,000 operations
Thursday, 17 June 2010
www.terracotta.org
Ehcache with Terracotta vs the Rest Application
Tests done with Owners = 25K and 125K which translates to total objects of 0.3 M and 1.5 M
Minimal tuning.
Cluster Configuration:8 Client JVMs (1.75G Heap)1 (+0) Terracotta Servers (6G Heap)MySql: sales18.
39
Thursday, 17 June 2010
www.terracotta.org
Ehcache with Terracotta vs the Rest Ehcache
Replicated with RMI not included because not coherentSingle TSA Server 15 threads and some with 100 threads
IMDG15 threads Cache deployed in Partitioned Mode Tests were also done with Replicated – which did well for
small cache sizes but failed to complete with larger cache sizes. So, it is not included.
memcached15 threads1 server
40
Thursday, 17 June 2010
www.terracotta.org
Hibernate - Read Only TPS
41
Thursday, 17 June 2010
www.terracotta.org
Hibernate - Read Write TPS
42
Thursday, 17 June 2010
www.terracotta.org
Hibernate - Read Only Latency
43
Thursday, 17 June 2010
www.terracotta.org
Hibernate - Read Write Latency
44
Thursday, 17 June 2010
www.terracotta.org
Test SourceThe code behind the benchmarks is in the Terracotta Community SVN repository.
Download https://svn.terracotta.org/repo/forge/projects/ehcacheperf/(Terracotta Community Login Required)
45
Thursday, 17 June 2010
www.terracotta.org
Performance ConclusionsWith Hibernate, Using Spring Pet Clinic
After app servers and DBs tuned by independent 3rd parties30-95% database load reduction80 times read-only performance of MySQLNotably lower latency
1.5 ms versus 120 ms for database (25k)
46
Thursday, 17 June 2010
www.terracotta.org
What about NoSQL? Ehcache + Terracotta configured with persistence gives you a NoSQL
store with limited features i.e. no search
TerraStore - new open source project from Sergio Bossa is a document oriented NoSQL store based on Terracotta and ful
47
Thursday, 17 June 2010
Wrap Up
48
Thursday, 17 June 2010
www.terracotta.org
Q&A
Please ask any questions you have in the Q&A window.
49
Thursday, 17 June 2010
www.terracotta.org
Additional Ehcache InformationWebsite: www.ehcache.org
Documention: www.ehcache.org/documentation
Hibernate: www.ehcache.org/documentation/
hibernate.html
Commercial Products: www.terracotta.org/ehcache/
Twitter: www.twitter.com/Ehcache
50
Thursday, 17 June 2010
www.terracotta.org
Terracotta Contact InformationWebsite: www.terracottatech.com
Telephone: +1 415-738-4000
Email: [email protected]
Facebook: www.facebook.com/Terracotta
Twitter: www.twitter.com/TerracottaTech
51
Thursday, 17 June 2010
