Scale ColdFusion with Terracotta Distributed Caching for Ehchache

©2013 Software AG Government Solutions. All rights reserved.1 |

Scale  Coldfusion with  Terracotta  Distributed  Caching  for  Ehcache

Fabien  Sanglier,    Terracotta  Architect,  a.k.a The  Cache  Guy

+

2015 Adobe Coldfusion SummitNovember 9, 2015

Why  are  you  (and  I)  here?

•

•

•

•

+

Happy marriage since CF9 (2009)

= Open-Source, De-Facto Java Caching API

+ +

Seamless, Powerful Distributed In-Memory Caching with Free Open-Source Software

=

= Open-Source, Distributed In-Memory Store

In-Memory Caching = Not new, but still making the difference every day!

+ Seamless Integration!Simple Config Change

Why re-inventing the wheel?

Terracotta  (Inc.  and  products)  at  a  Glance• Founded  in  2003

• Lead  open-­‐source  maintainer  for:

• Terracotta  Commercial    Offerings:  BigMemory Go/Max,  Universal  Messaging

• Deployed  by  80%  of  the  Global  1000

• Over  2.5  million  enterprise  deployments   in  over  190  countries

• Used  by  2.1  million  developers

• Acquired  by  Software  AG  in  2011

• Software  AG  Government  Solutions  serves  U.S.  Federal  Government

4 | ©2013 Software AG. All rights reserved. For internal use only

Some  (simple)  background  to  slowly  warm  up…

-­‐ Caching-­‐ Ehcache-­‐ Coldfusion’s use  of  Ehcache

I  built  a  great  CF  site  /  APIs  etc…

Web services

Common  Scalability  challenges

Web services

1

22

2

1

1

3

4

4

+Since Coldfusion 9

The  De-­‐Facto  Java  “Caching”  API

• Open-source (more at www.ehcache.org)

• Current in 2.x: Ehcache 2.10.1

• Ehcache 3.x (complete overall with lots of improvements !)

• Full JSR 107 compliant (Java Caching specification)

• Seamless integration with many popular frameworks/applications:

Tomcat WebSphere

weblogic

©2013 Software AG Government Solutions. All rights reserved.10 |

Tiered Storage

JVM ManagedMemory

Ehcache Tiered  Storage  Approach

Local Disk Store(Re-startable)

HeapStore

Direct AccessRAM Store

a.k.a Offheap

2,000,000+

1,000,000

100,000

2

1,000

10,000+

Speed (TPS)

1,000s

Size (GB)

External Data Source(e.g., Database)

Local Disk

Ehcache 2.x Example Configuration

Ehcache Simple  and  Flexible  Configuration (ehcache.xml)<ehcache >

<cache name=”UserPreferencesCache”maxBytesLocalHeap=”200M”maxBytesLocalOffHeap=”2000M”timeToIdleSeconds="300”timeToLiveSeconds=“600”memoryStoreEvictionPolicy="LRU” />

<cache name=”ShoppingCartCache”maxEntriesLocalHeap=”10000”timeToLiveSeconds=”6000”overflowToDisk="true”maxElementsOnDisk="400"diskExpiryThreadIntervalSeconds="120”memoryStoreEvictionPolicy=”LFU” />

</ehcache>

Ehcache goodness  all  available  (user  guides,  tutorials,  APIs,  downloads,  etc…)  at  http://www.ehcache.org

http://www.ehcache.org

Coldfusion’s current  use  of  Ehcache

• CF  Authentication:• auth-­‐ehcache.xml:  authcache,  authtokenmappingcache

• Internal  Caching  (CF  templates,  component  paths)  

• <cfcache> -­‐ Cache  fragments  of  html

• <cfquery>  -­‐ Cache  DB  calls• <cfquery name="myPortfolio"  cachedwithin=#createTimeSpan(  0,  1,  0,  0  )#>

• ORM  with  Ehcache 2nd level  caching:  Caching Hibernate  queries• Entityload('BlogPost',{},{cacheable=true})

• CF  Cache  functions:  Direct  Ehcache calls• CacheGet /  CachePut /  CacheRemove /  CacheGetAllIds• CacheGetMetadata• CacheGetProperties /  CacheSetProperties

• Custom  CF  JAVA  components  using  Ehcache library  directly

All this can be “easily” Distributed with Terracotta+

Coldfusion’s Ehcache resources

1. CF11  – Ehcache 2.6.6  (Latest  ehcache in  2.x  line:  2.10.1)

2. CF11  Ehcache libs:1. <CF_HOME>/cfusion/lib/ehcache-­‐core-­‐2.6.6.jar  (core  library)2. <CF_HOME>/cfusion/lib/ehcache-­‐web-­‐2.0.4.jar  (web  content  caching)

3. CF  ehcache configs:1. <CF_HOME>/cfusion/lib/ehcache.xml2. <CF_HOME>/cfusion/lib/auth-­‐ehcache.xml

For Terracotta, those will need to be updated

+

15 | ©2013 Software AG. All rights reserved. For internal use only

Ehcache standalone

…  …  vs.  Ehcache Replications

…  …  …  vs.  Ehcache Distributed  with  Terracotta

Types of Ehcache Topologies

Not all Ehcache topologies created equal…

Ehcache Standalone:  Simple  and  quick  fixWeb

services

Ehca

che

Ehca

che

Ehca

che

Limitations:

§ Limited Memory Storage (JVM Heap only)

§ Larger Heap = Unpredictable Latencies (GC pauses)

§ Standalone = Duplicate cache data = more DB / disks calls

Advantages:

§ Easy to implement

§ Highly concurrent

§ Linear performance gains

§ Extend storage to disk

Ehcache Replication:  Improved  caching  but  some  caveats

Web servicesEh

cach

eEh

cach

eEh

cach

e

Ehcachereplication

Ehcachereplication

Advantages:

§ Replications on ALL inserts/updates/deletes

§ Add once, request many

§ More efficient

Same ehcache limitations +

§ Complex (Jgroup, RMI, MultiCasting)

§ Possible Cache Discrepencies

§ Bootstrapping issues

Terracotta  Distributed  Ehcache:  High  Scalability  Caching

Web servicesEh

cach

eEh

cach

eEh

cach

e

Advantages:

§ Simple to setup (1 config change in ehcache)

§ Unlimited in-memory storage capacity (offheap)

§ Completely open-source (4.3 and up)

Terr

acot

ta O

pen-

Sour

ce (

4.3)

Advantages (cont’d):

§ Flexible consistencies

§ High Availability

§ Monitoring options

19 | ©2013 Software AG. All rights reserved. For internal use only

Terracotta  distributed  in-­‐memory  caching  specifics

http://www.terracotta.org/downloads/open-source/catalog

Terracotta  4.x  Open  Source  Offering  /  Architecture

ActiveServer

SCALE OUT

1 JVM – X gb Heap

CF App Server

1 JVM – X gb Heap 1 JVM – X gb Heap

Ehcache 2.x

CF App Server CF App Server

UnlimitedIn-Memory

Storage(Offheap)

PassiveServer

CF internals

Activereplication

UnlimitedIn-Memory

Storage(Offheap)

Ehcache 2.x

CF internals

Ehcache 2.x

CF internals

HeapTC internals 2 GB

e.g.100 GB

2 GB

e.g.100 GB

tcp sockets / nio / unicast

HeapTC internals

SCAL

E U

P

2

1

3

Easy  scalability:  New  clients  can  access  all  cached  data

ActiveServer

UnlimitedIn-Memory

Storage(Offheap)

PassiveServer

Activereplication

UnlimitedIn-Memory

Storage(Offheap)

HeapTC internals 2 GB

e.g.100 GB

2 GB

e.g.100 GB

tcp sockets / nio / unicast

HeapTC internals

SCAL

E U

P

1 JVM – X gb Heap

CF App Server

1 JVM – X gb Heap 1 JVM – X gb Heap

Ehcache

CF App Server CF App Server

CF internals

Ehcache

CF internals

Ehcache

CF internals

1 JVM – X gb Heap

CF App Server

Ehcache

CF internals

1 JVM – X gb Heap

CF App Server

Ehcache

CF internals

Powerful  H/A  =  Automatic  failover  /  no  cache  data  loss

ActiveServer

UnlimitedIn-Memory

Storage(Offheap)

New ActiveServer

Activereplication

UnlimitedIn-Memory

Storage(Offheap)

HeapTC internals 2 GB

e.g.100 GB

2 GB

e.g.100 GB

tcp sockets / nio / unicast

HeapTC internals

SCAL

E U

P

1 JVM – X gb Heap

CF App Server

1 JVM – X gb Heap 1 JVM – X gb Heap

Ehcache

CF App Server CF App Server

CF internals

Ehcache

CF internals

Ehcache

CF internals

1 JVM – X gb Heap

CF App Server

Ehcache

CF internals

1 JVM – X gb Heap

CF App Server

Ehcache

CF internals

Tiered Storage

JVM ManagedMemory

New  Tiered  Storage  Approach  with  Unlimited  Storage  into  Terracotta  (up  to  RAM  available  of  course!)

Local Disk Store(Re-startable)

HeapStore

Direct AccessRAM Store

a.k.a Offheap

2,000,000+

1,000,000

100,000

2

1,000

10,000+

Speed (TPS)

1,000s

Size (GB)

External Data Source(e.g., Database)

Offheap with  JAVA  =  Unrestricted  In-­‐Memory  Storage

• Standard  Java

• Proven  TBs  scale  capacity  

• Not  managed  by  the  JVM  =  No  Garbage  Collections=  Predictable  latencies

• No  specialized  appliance  needed

Seamless  configuration  on  same  ehcache.xml

<ehcache>

<terracottaConfigurl=“server1:9510,server2:9510"/>

<cache name=“UserPreferencesCache”

timeToIdleSeconds=”300”

maxEntriesLocalHeap=“1000” />

<cache name=”ShoppingCartCache”

timeToLiveSeconds=”6000”

maxEntriesLocalHeap=“1000”>

maxEntriesInCache=“10000000”>

<terracotta />

</cache>

</ehcache>

25

Ehcache 2.x Example Configuration

Amount  of  objects  allowed  in  

Terracotta  for  that  Cache  region

Terracotta  url

Enable  Terracotta  for  that  cache

Example Configuration

Flexible  data  access  consistency

StronglyConsistent  

FullyTransactional

EventuallyConsistent

More  Consistency More  Performance

<cache  name=”UserPreferencesCache"maxElementsInMemory="10000"timeToLiveSeconds="300”>

<terracotta consistency=”eventual"/></cache>

<cache  name=”ShoppingCartCache"maxElementsInMemory=”5000"timeToIdleSeconds=”6000”>

<terracotta consistency=”strong"/></cache>

Example Configuration

Some  extra  “Terracotta”  pointers...1. No  limitations  on  what  you  can  store  in  cache,  BUT  Cached  Keys  /  Values  MUST  be  “Serializable”  (implement  

java.io.Serializable)

2. New  Terracotta  jar  needed  +  Ehcache JAR  must  match  Terracotta  version.  So  new  Jars  needed  in  CF:1. ehcache-­‐2.10.1.jar2. terracotta-­‐toolkit-­‐runtime-­‐4.3.1.jar

3. Cache  by  reference  (standalone   Ehcache)  /  Cache  by  serialization  (Terracotta)

4. For  resiliency  to  clients-­‐terracotta  disconnections,   there’s  nonstop  mode.  But  Ehcache client  must  specify  appropriate  “behavior”  (noop,   exception,   localReads)

5. Explicit  locks  apply   to  the  full  “Terracotta  cache  cluster”  (Read  /  Write  Locks)  1. Useful  for  special  data  consistency  needs

6. Ehcache +  Terracotta  =  Full   compliant  XA  resource  (XA  transactions)

7. CAS  operations  apply   to  the  full  “Terracotta  cache  cluster”  1. Cache.putIfAbsent(Element  element),  Cache.replace(Element  oldOne,  Element  newOne)

Key (Serializable) Value Serializable (byte[])

Monitoring  and  Management:  Simple  REST  APITerracotta Server Array

App Server

Heap

Ehcache /BigMemory

Cache Runtime Endpoint JMX bridge TSA Rest Agent

App Server

Heap

Ehcache /BigMemory

CF App Server

Heap

Ehcache

Tcp9510

TcpJMX9520

Http/REST9540

CURL

MonitoringApp

http/rest

http://www.terracotta.org/documentation/4.1/tms/tms-rest-api

A  monitoring  example:  Terracotta  NewRelic Plugin

Terracotta Server Array

App Server

Heap

Ehcache /BigMemory

Cache Runtime Endpoint JMX bridge TSA Rest Agent

App Server

Heap

Ehcache /BigMemory

CF App Server

Heap

Ehcache

Tcp9510

TcpJMX9520

Http/REST9540

CURLNewRelic

Plugin

http/rest

NewRelic Cloud

Rest HTML

HttpsREST

HttpHTML

Prox

y

https://github.com/Terracotta-OSS/terracotta-newrelic-plugin

30 | ©2013 Software AG. All rights reserved. For internal use only

Setting  it  all  up  with  CF  11

Terracotta  Initial  Setup  in  just  a  few  steps…

1. Download  Terracotta  OSS  (latest  4.3.1  -­‐-­‐ terracotta-­‐4.3.1.tar.gz)1. http://www.terracotta.org/downloads/open-­‐source/catalog

2. Extract  to  location  of  choice

3. Ensure  JAVA_HOME  is  set

4. Navigate  to  <TERRACOTTA_INSTALL>/server/bin

5. Start  with  default  single  node  config by  executing:  1. start-­‐tc-­‐server.sh (or  .bat)

6. Terracotta  process  is  now  accessible  at  IP:9510

Note:  if  setting  up  Terracotta  in  active/mirror  setup,  tc-­‐config.xml must  be  created  and  referenced  at  startup:• start-­‐tc-­‐server.sh (or  .bat)  -­‐f  <path-­‐to-­‐config>/tc-­‐config.xml –n  <server-­‐name-­‐to-­‐start>  

Connecting  CF  11  to  Terracotta  in  just  a  few  steps

1. Copy  Ehcache +  Terracotta  libs  to  <CF_HOME>/cfusion/lib1. <TERRACOTTA_INSTALL>/apis/ehcache/lib/ehcache-­‐2.10.1.jar2. <TERRACOTTA_INSTALL>/apis/toolkit/lib/terracotta-­‐toolkit-­‐runtime-­‐4.3.1.jar

2. Add  terracotta-­‐specifics  configurations  in  CF  ehcache configs:1. <CF_HOME>/cfusion/lib/ehcache.xml2. <CF_HOME>/cfusion/lib/auth-­‐ehcache.xml

3. Restart  CF

4. Notice  Terracotta  connection  in  CF  logs

All  these  features  are  now  cached  on  Terracotta!

• CF  Authentication:• auth-­‐ehcache.xml:  authcache,  authtokenmappingcache

• Internal  Caching  (CF  templates,  component  paths)  

• <cfcache> -­‐ Cache  fragments  of  html

• <cfquery>  -­‐ Cache  DB  calls• <cfquery name="myPortfolio"  cachedwithin=#createTimeSpan(  0,  1,  0,  0  )#>

• ORM  with  Ehcache 2nd level  caching:  Caching Hibernate  queries• Entityload('BlogPost',{},{cacheable=true})

• CF  Cache  functions:  Direct  Ehcache calls• CacheGet /  CachePut /  CacheRemove /  CacheGetAllIds• CacheGetMetadata• CacheGetProperties /  CacheSetProperties

• Custom  CF  JAVA  components  using  Ehcache library  directly

+

+

And now just wait to see your CF website

absorb the next usage spikes!!

©2013 Software AG Government Solutions. All rights reserved.35 |

Questions?

Fabien SanglierTerracotta Architect

www.linkedin.com/in/fabiensanglier/

fabien.sanglier@softwareaggov.com

twitter.com/fabiensanglier

fsanglier.blogspot.com

github.com/lanimall

www.SoftwareAGgov.comSoftware AG :

Terracotta: www.terracotta.orgEhcache: www.ehcache.org

www.SoftwareAGgov.comSoftware AG Government Solutions :