In-Memory Computing
Srinath Perera Director, Research
WSO2 Inc.
Performance Numbers (based on Jeff Dean’s numbers )
Mem Ops / Sec
If Memory access is a Second
L1 cache reference 0.05 1/20th sec
Main memory reference 1 1 sec
Send 2K bytes over 1 Gbps network 200 3 min
Read 1 MB sequentially from memory 2500 41 minDisk seek 1*10^5 27 hours
Read 1 MB sequentially from disk 2*10^5 2 days
Send packet CA->Netherlands->CA 1.5*10^6 17 days
OperationSpeed
MB/sec
Hadoop Select 3Terasort Bench mark 18Complex Query Hadoop 0.2
CEP 60
CEP Complex 2.5
SSD 300-500
Disk 50-100
Performance Numbers (based on Jeff Dean’s numbers )
Mem Ops / Sec
If Memory access is a Second
L1 cache reference 0.05 1/20th sec
Main memory reference 1 1 sec
Send 2K bytes over 1 Gbps network 200 3 min
Read 1 MB sequentially from memory 2500 41 minDisk seek 1*10^5 27 hours
Read 1 MB sequentially from disk 2*10^5 2 days
Send packet CA->Netherlands->CA 1.5*10^6 17 days
OperationSpeed
MB/sec
Hadoop Select 3Terasort Bench mark 18Complex Query Hadoop 0.2
CEP 60
CEP Complex 2.5
SSD 300-500
Disk 50-100
Most Big Data Apps are Latency-bound!!
Often, your app waste CPU waiting for data to arrive
Latency Lags Bandwidth
• Observation in prof. Patterson’s Keynote at 2004
• Bandwidth improves, but not latency • Same holds now, and the gap is
widening with new systems
Handling Speed Differences in Memory Hierarchy
1. Caching – E.g. Processor caches, file cache,
disk cache, permission cache
2. Replication – E.g. RAID, Content Distribution
Networks (CDN), Web Cache
3. Prediction – Predict what data will be needed and prefect – Tradeoff bandwidth – E.g. disk caches, Google Earth
Above three does not always work
• Limitations – Caching works only if working set is small – Prefetching only works when access patterns are predictable – Replication is expensive and limited by receiving side machines
• Lets assume you are reading and filtering 10G data (assuming 6b per record that is 17Billion records)– 3 minutes to read the data from disk– 35ms to filter 10M in my laptop => 1 minutes to process all
data – Keeping data in memory can give about 30X more
Data Access Patterns in Big Data Applications
• Read from Disk, process once (Basic Analytics)– Data can be perfected, batch load is only about 100 times faster.– OK if processing time > data read time
• Read from Disk, iteratively Process (Machine Learning Algos, e.g. KMean)– Need to load data from disk once and process (e.g. Spark supports this)
• Interactive (OLAP)– Queries are random, data may be scattered. Once query started, can load data to
memory and process
• Random Access (e.g. Graph Processing)– Very hard to optimize
• Realtime Access – As data comes in
In-Memory Computing
Four Myths
• Myths– Too expensive 1TB RAM cluster for 20-40k (about 1$/GB)– It is not durable – Flash is fast enough – It is about In-Memory DBs
• From Nikita Ivanov’s post– http
://gridgaintech.wordpress.com/2013/09/18/four-myths-of-in-memory-computing/
Let us look at each Big data access pattern and where In-Memory
Computing can make a difference
Access Pattern 1:Read from Disk, Process Once • If Tp = 35ms vs
Td=1.2 sec with 60MB chunks, it will give about 30X to keep all data in Memory
• However, this benefit is less if computation is more complex (e.g. Sort)
Access Pattern 2: Read from Disk, iteratively Process
• Very common pattern for machine learning algorithms (e.g. KMean)
• On this case, advantages are greater – If we cannot hold data in memory fully, we need to offload– Then we need to read again – Then cost is very high to load and process and much faster
with in memory computing
• Spark let you load to memory fully and process
Spark• New Programming Model
built on functional programming concepts
• Can be much faster for recursive usecases
• Have a complete stack of products
file = spark.textFile("hdfs://...”)file.flatMap(
line => line.split(" ")) .map(word => (word, 1)) .reduceByKey(_ + _)
Access Pattern 3: Interactive Queries
• Need to be responsive, < 10 sec• Harder to predict what data is needed• Queries tend to be simpler • Can be made faster by a RAM Cloud
– SAP Hana– Volt DB
• With smaller queries, disk may still be OK. Apache Drill as an Alternative
VoltDB Story
• VoltDB Team (Michael Stonebraker et al.) observed 92% of work in a DB related to Disk
• By building complete in-memory database cluster they made it 20x faster!
Distributed Cloud (e.g. Hazelcast)
• Store the data portioned and replicated across many machines
• Used as a cache that span multipme machines• Key value access
Access Pattern 4: Random Accesses
• E.g. Graph Traversal • This is the hardest usecase • In easy cases, there is a small working set and can be
solved with a cache ( checking users against a black list), not the case with Graph most graph operations like traversal
• Hard cases, In Memory Computing is only real solution • Can be as fast as 1000x or more
Access Pattern 5: Realtime Processing
• This is already In-Memory technology using tools like Complex Event Processing (e.g. WSO2 CEP) or stream processing (e.g. Apache Storm)
Faster Access to Data
• In-Memory databases (e.g. VoltDB, MemSQL)– Provide Same SQL interface– Can think as fast database– VoltDB has shown to about 20X faster than MySQL
• Distributed Cache – Can Integrated as a Large Cache
Load Data Set to Memory and Analyze
• Used with Interactive and Random access usecases • Can be as 1000x faster for some usecases • Tools
– Spark – Hazelcast– SAP Hana
Realtime Processing
• Realtime analytics tools – CEP (WSO2 CEP)– Stream Processing (e.g. Storm)
• Can generate results within few milliseconds to seconds
• Can process 10ks-millions of events per second
• Not all algorithms can be implemented
In Memory Computing with WSO2 Platform
Thank You
Recommended
