Technical Brief

In-Memory Data GridGridGain Systems, November 2012

OverviewOverviewMany companies that would not have considered using in-memory technology inthe past due to its cost are now changing their core systems’ architectures toaccommodate it. They want to take advantage of the low-latency transactionprocessing in-memory technology offers. With the price of 1GB of RAM less than aone dollar and RAM prices dropping 30% every 18 months it has becomeeconomically affordable to load entire operational datasets into memory andachieve dramatic performance improvements.

Companies are using this, for example, to perform calculations or create livedashboards that give management immediate insight into crucial operational datafrom their systems. Currently, users often have to wait until the end of a reportingperiod for batch jobs to process the accumulated data and generate the desiredreports.

Modern in-memory technology connects to existing data stores such as Hadoop ortraditional data warehouses and makes this data available in RAM, where it canthen be queried or used in processing tasks with an unprecedented performance.The power of such insight in real-time lets companies react exponentially faster

and more flexible than what current systems allow.

This paper is meant to help readers understand what the key features of modernin-memory products are and how they affect the eventual integration andperformance. Two key components are the underlying basis for the corecapabilities of in-memory technology: in-memory compute and data grids. Thispaper concentrates on the in-memory data grids.

What is an In-Memory Data Grid?What is an In-Memory Data Grid?The goal of an In-Memory Data Grid (IMDG) is to provide extremely low latencyaccess to, and high availability of, application data by keeping it in memory and todo so in a highly parallelized way. By loading terabytes of data into memory, anIMDG is able to support most of the Big Data processing requirements. At a veryhigh level an IMDG is a distributed key-value object store similar in its interface to atypical concurrent hash map. You store and retrieve objects using keys.

Unlike systems where keys and values are limited to byte arrays or strings, anIMDG can have any application domain object as either a value or a key. Thisprovides tremendous flexibility: exactly the same object your business logic isusing can be kept in the data grid - without the extra step of marshaling and de-marshaling. It also simplifies the use of the data grid because you can in mostcases interface with the distributed data store like with a simple hash map.

Being able to work with domain objects directly is one of the main differencesbetween IMDGs and In-Memory Databases (IMDB). With the latter, users still needto perform Object-To-Relational Mapping (ORM) which typically adds significantperformance overhead and complexity. With in-memory data grids this is avoided.

How Do In-Memory Data Grids Differ FromHow Do In-Memory Data Grids Differ FromOther Solutions?Other Solutions?An IMDG, in general, is significantly different from products such as NoSqldatabases, IMDBs or NewSql databases. For example, here are just some of theGridGain's IMDG features that make it unique:

Support for true distributed ACID transactions with highly optimized 2PCprotocol implementationScalable Data Partitioning across a cluster including both partitioned orfully replicated scenariosAbility to work directly with application domain objects rather than with

primitive types or "documents"Tight integration with In-Memory Compute Grid (IMCG)Zero Deployment for both IMCG and IMDGPluggable segmentation (a.k.a. "brain split" problem) resolutionPluggable expiration policies (including built-in LRU, LIRS, random and time-based)Read-through and write-through with pluggable durable storeSynchronous and asynchronous operations throughoutPluggable data overflow storageMaster/master data replication and invalidation in both synchronous andasynchronous modesWrite-behind cache store supportAutomatic, manual and delayed pre-loading on topology changesSupport for fully active replicas (backups)Support for structured and unstructured dataPluggable indexing support

Essentially IMDGs in their purest form can be viewed as distributed hash maps witheach key cached on a particular cluster node - the bigger the cluster, the moredata you can cache. The trick to this architecture is to make sure that theprocessing occurs on those cluster nodes where the required data is cached. Bydoing this all cache operations become local and there is no, or minimal, datamovement within the cluster. In fact, when using a well-designed IMDG thereshould be absolutely no data movement on stable topologies. The only time whensome of the data is moved is when new nodes join or some existing nodes leave,hence causing some data repartitioning within the cluster.

The picture below shows a classic IMDG with a key set of {k1, k2, k3} where eachkey belongs to a different node. The external database component is optional. Ifpresent, then IMDGs will usually automatically read data from the database or writedata to it (a.k.a. read-through and write-through logic):

Even though IMDGs usually share some common basic functionality, there are manyfeatures and implementation details that are different between vendors. Whenevaluating an IMDG product, pay attention to eviction policies, (pre)loadingtechniques, concurrent repartitioning or its memory overhead, for example. Alsopay attention to the ability to query data at runtime. Some IMDGs, such asGridGain for example, allow users to query in-memory data using standard SQL,including support for distributed joins, which is pretty rare.

The typical use of IMDGs is to partition data across the cluster and then sendcollocated computations to the nodes where the data is. Since computations areusually part of Compute Grids and have to be properly deployed, load-balanced,failed-over or scheduled, the integration between Compute Grids and IMDGs is veryimportant to obtain the best performance. Especially when both In-MemoryCompute and Data Grids are optimized to work together and utilize the same APIs,it is faster for developers to deploy system that offers the highest performancereliably.

Distributed ACID TransactionsDistributed ACID TransactionsOne of the distinguishing characteristic of IMDGs is support for Distributed ACIDTransactions . Generally, a 2-phase-commit (2PC) protocol is used to ensure dataconsistency within a cluster. Different IMDGs will have different underlying lockingmechanisms, but more advanced implementations provide concurrent locking

mechanisms (like MVCC - Multi-Version Concurrency Control), reduce networkchattiness to a minimum, and specifically optimize its main algorithms for in-memory processing - guaranteeing transactional ACID consistency with very highperformance.

Guaranteed data consistency is one of the main differences between IMDGs andNoSQL databases.

NoSQL databases are usually designed with an Eventual Consistency (EC) approachwhere data is allowed to be inconsistent for a period of time as long as it willeventually become consistent. Generally, the writes on EC-based systems aresomewhat fast, but reads are slow (to be more precise: as fast as writes are).Latest IMDGs with an optimized 2PC protocol should at least match, if notoutperform, EC-based systems on writes, and be significantly faster on reads. It isinteresting to note that the industry has made a full circle moving from a then-slow 2PC approach to the EC approach, and now from EC to an optimized 2PC,which often is significantly faster.

Different products have optimized the 2PC protocol in different ways, butgenerally the purpose of all optimizations is to increase concurrency, minimizenetwork overhead, and reduce the number of locks a transaction requires tocomplete. As an example, Google’s distributed global database, Spanner, is basedon a transactional 2PC approach simply because 2PC provided a faster and morestraightforward way to guarantee data consistency and a high throughput.GridGain introduced "HyperLocking" technology that enabled effective single andgroup distributed locking that is at the core of its transactional performance.

Distributed data grid transactions in GridGain span data cached on local as well asremote nodes. While automatic enlisting into JEE/JTA transactions is supported,GridGain data grid also allows users to create more light-weight cache transactionswhich are often more convenient to use. GridGain cache transactions support allACID properties that you would expect from any transaction, including support forOptimistic and Pessimistic concurrency levels and Read-Committed, Repeatable-Read, and Serializable isolation levels. If a persistent data store is configured, thenthe transactions will also automatically span the data store.

Multiversion Concurrency Control (MVCC)Multiversion Concurrency Control (MVCC)GridGain’s in-memory data grid concurrency is based on advanced implementationof MVCC (Multi Version Concurrency Control) – the same technology used bypractically all database management systems. It provides practically a lock freeconcurrency management by maintaining multiple version of data instead of usinglocks with a wide scope. Thus, MVCC in GridGain provides a backbone for highperformance and overall system throughput for systems under load.

In-Memory SQL QueriesIn-Memory SQL QueriesWhat use would be from caching all the data in memory if you could not query it?The in-memory platform should offer a variety of different ways to query its data,such as standard SQL-based queries or Lucene-based text queries.

The JDBC driver implementation lets you to query distributed data from theGridGain cache using standard SQL queries and the standard JDBC API. It willautomatically get only the fields you actually need from the objects stored incache.

The GridGain SQL query type lets you perform distributed cache queries usingstandard SQL syntax. There are almost no restrictions as to which SQL syntax canbe used. All inner, outer, or full joins are supported, as well as rich set of SQLgrammar and functions. The ability to join different classes of objects stored incache or across different caches makes GridGain queries a very powerful tool. Allindices are usually kept in memory resulting in very low latencies for the executionof queries.

Text queries are available when you are working with unstructured text data.GridGain can index such data with the Lucene or H2Text engine to let you querylarge volumes of text efficiently.

If there is no need to return result to the caller, all query results can be visited

directly on the remote nodes. Then all the logic is performed directly on theremotely queried nodes without sending any queried data to the caller. This wayanalytics can be run directly on structured or unstructured data with in-memoryspeed and low latencies. At the same time GridGain provides applications anddevelopers a familiar way to retrieve and analyze the data.

Here’s the quick example. Notice how Java code looks 100% identical as if you talkto a standard SQL database – yet you are working in in-memory data platform:

// Register JDBC driver.Class.forName("org.gridgain.jdbc.GridJdbcDriver");

// Open JDBC connection.conn = DriverManager.getConnection( "jdbc:gridgain:/ / localhost/" + CACHE_NAME, configuration());

// Create prepared statement.PreparedStatement stmt = conn.prepareStatement( "select name, age from Person where age >= ?");

// Configure prepared statement.stmt.setInt(1, minAge);

// Get result set.ResultSet rs = stmt.executeQuery();

BigMemory SupportBigMemory SupportTraditionally JVM has been very good with Garbage Collection (GC). However, whenrunning with large amounts of memory available, GC pauses can get very long. Thisgenerally happens because GC now has a lot more memory to manage and oftencannot cope without stopping your application completely (a.k.a. lock-the-worldpauses) and allowing itself to catch up. In our internal tests with heap size set to60G or 90G GC pauses some times were as long as 5 minutes. Traditionally thisproblem was solved by starting multiple JVMs on the same physical box, but thatdoes not always work very well as some applications want to collocate largeamounts of data in one JVM for faster processing.

To mitigate large GC pauses, GridGain supports BigMemory with data allocated off-heap instead of on-heap. Thus, the JVM GC does not know about it and does notslow down. You can start your Java application with a relatively small heap, e.g.below 512M, and then let GridGain utilize hundreds of gigabytes of memory as off-heap data cache. Whenever data is first accessed, it gets cached in the on-heapmemory. Then, after a certain period of non-use, it gets placed into off-heapmemory cache. If your off-heap memory gets full, the least used data can beoptionally evicted to the disk overflow store, also called swap store.

One of the distinguishing characteristics of GridGain off-heap memory is that theon-heap memory foot print is constant and does not grow with the size of yourdata. Also, an off-heap cache entry has very little overhead, which means that youcan fit more data in memory. Another interesting feature of GridGain is that bothprimary and secondary indices for SQL can be optionally kept in off-heap memoryas well.

Datacenter ReplicationDatacenter ReplicationWhen working with multiple data centers it is important to make sure that if onedata center goes down, another data center is fully capable of picking up its loadand data. Data center replication is meant to solve exactly this problem. Whendata center replication is turned on, GridGain data grid will automatically make sure

that each data center is consistently backing up its data to other data centers(there can be one ore more).

GridGain supports both active-active and active-passive modes forreplication. In active-active mode both data centers are fully operational online andact as a backup copy of each other. In active-passive node, only one data center isactive and another data center serves only as a backup for the active data center.

Datacenter replication can be either transactional or eventually-consistent. Intransactional mode, a data grid transaction will be considered complete only whenall the data has be replicated to another datacenter. If the replication step failed,then the whole transaction will be rolled back on both datacenters. In eventuallyconsistent mode transaction will usually complete before the replication finished. Inthis mode the data is usually concurrently buffered on one data center and thengets flushed to another data center either when buffer fills up or when certain timeperiod elapses. Eventually consistent mode is generally a lot faster, but it alsointroduces a lag between updates on one data center and data being replicated toanother.

If one of the datacenters goes offline, then another will immediately takeresponsibility for it. Whenever the crashed data center goes back online then it willreceive all the updates it has missed from another data center.

In-Memory Compute Grid IntegrationIn-Memory Compute Grid IntegrationIntegration between IMCG and IMDG is based on idea of affinity routing .Affinity routing is one of the key concepts behind Compute and Data Gridtechnologies (whether they are in-memory or disk based). In general, affinity

routing allows to co-locate a job and the data set this job needs to process.

The idea is pretty simple: if job and data are not co-located, then job will arrive onsome remote node and will have to fetch necessary data from yet another nodewhere data is stored. Once processed this data will most likely will have to bediscarded (since it’s already stored and backed up elsewhere). This processinduces expensive network trip plus all associated marshaling and demarshaling. Atscale – this behavior can bring almost any system to a halt.

Affinity co-location solves this problem by co-locating job with its necessary dataset. We say that there is an affinity between a processing (i.e. job) and the datathat this processing requires – and therefore we can route the job based on thisaffinity to a node where data is stored to avoid unnecessary network trips andextra marshaling and demarshaling.

GridGain provides advanced capabilities for affinity co-location: from a simplesingle-method call to sophisticated APIs supporting complex affinity keys and non-trivial topologies.

SummarySummaryIn-memory data grids are used throughout a wide spectrum of industries in

applications as diverse as risk analytics, trading systems, bioinformatics,ecommerce or online gaming. Essentially, every project that struggles withscalability and performance can benefit from in-memory processing and an in-memory data grid architecture. When you consider different products, make sureyou have the advanced features outlined in this paper. This way you can find anoptimal solution for your needs and ensure right at the onset that your solutionwill actually scale flawlessly in those critical moments when you need it to scale.