Upload
aidanshribman
View
468
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
Aidan Shribman; Sr. Researcher; SAP Research Israel With the contribution of Benoit Hudzia, Roei Tell, Steve Walsh, Peter Izsak
Hecatonchire: Transparent Resource Aggregation Israeli Networking Day 2013
© 2012 SAP AG. All rights reserved. 2
In the talk today
Motivation: the SAP HANA in-
memory database.
Intro: the Hecatonchire project
Capabilities: mainly memory
aggregation.
Numbers: quicksort & SAP
HANA TCP-H benchmarks.
© 2012 SAP AG. All rights reserved. 3
Order Country Product Sales
Back story: SAP HANA In-Memory Real-Time Analytics
456 France corn 1000
457 Italy wheat 900
458 Italy corn 600
459 Spain rice 800
456
457
458
459
France
Italy
Italy
Spain
corn
wheat
corn
rice
1000
900
600
800
Row order organization Column order organization
Order Country Product Sales
SAP HANA In-Memory Database (2010)
Big multi-core large-memory systems; high
memory/core ratio: 128 GB; 256 GB; … 1024 GB.
OLAP (OnLine Analytical Processing): on-the-fly
analytics on real-time data
Single-scan aggregation: SELECT Country, SUM(Sales) FROM Sales_Orders
WHERE Product=‘corn’ GROUP BY Country
SAP ERP R/3 (1980s)
Moderate hardware requirements; Is optimized for slow
storage devices.
OLTP (OnLine Transactioanl Processing) : mainly
storing and retrieving transactions
Single-record access: SELECT * FROM Sales_Orders
WHERE Order = ‘457’
G
© 2012 SAP AG. All rights reserved. 4
A “gap” in Current Cloud Offerings
Guest are constrained by host sizes
Limited by physical constraints (AWS high-
memory is 244 GB guest atop a 256 GB host).
Host resources are pooled together (future?)
No limitation: Better scalability; Better economics;
Better performance.
H/W OS App VM
H/W
OS
App
VM
H/W
OS
App
VM
Guests
CPUs
Memory
I/O
CPUs
Memory
I/O
CPUs
Memory
I/O
Server #1 Server #2 Server #n
Fast RDMA Communication
H/W OS App VM
H/W
OS
App
VM
H/W
OS
App
VM
H/W
OS
App
VM
Guests
CPUs
Memory
I/O
CPUs
Memory
I/O
CPUs
Memory
I/O
Server #1 Server #2 Server #n
Fast RDMA Communication
© 2012 SAP AG. All rights reserved. 5
The Hecatonchire Project
Hecatonchires in Greek mythology means “Hundred Handed One”. The project addresses the
transparent aggregation of compute resources such as memory, CPU and storage
Hecatonchire aims at extending (not replacing) existing cloud and virtualization stacks
(Linux/KVM, QEMU, OpenStack) while using of commodity hardware and modern
interconnects (10 GbE or InfiniBand)
Hecatonchire is an open source project hosted on github developed by SAP Research
© 2012 SAP AG. All rights reserved. 6
Faster to Scale-Out …
Author: Chaim Bendalac
© 2012 SAP AG. All rights reserved. 7
Challenges of Resources Aggregation
CPU (future)
State is MBs in size (can be migrated sub-second)
Can be virtualized using hypervisor vCPU abstraction
Difficult challenge: Implementing efficient fine-grained cache-coherency (difficult)
Memory (in-progress)
GBs to TBs is size (accessed in page granularity 4KB – 4MB)
Can be virtualized by extending the MMU (Main Memory Unit) system
Challenge: achieving near-native performance while not degrading availability
Storage (mostly-done)
TBs to PBs in size (but accessed in blocks MBs in size)
Already virtualized using VFS; BLK; SCSI; iSCSI; virtio/vhost ;iSER; LIO.
Continued challenge: high bandwidth and high IOPS (partially achieved)
© 2012 SAP AG. All rights reserved. 8
VM Memory Pooling
Fast Remote Page Access
Low latency RDMA (Remote Direct Memory Access) page transfer protocol
Supports best price/performance hardware: InfiniBand, iWARP, RoCE
Demand pre-paging (pre-fetching) mechanism
Built-in RAID-1 style Memory Mirroring
No extra single points for failure
Minimal overhead : 1st slave to answer synchronous; 2nd - asynchronous
Complimentary to VM-based High-Availability (e.g. Kemari)
Transparent for guest VMs
Integration by adding static hooks into Linux MMU (page fault, swap-out, etc.).
QEMU Binds Remote Memory to Guest Linux/KVM Virtual Machine.
© 2012 SAP AG. All rights reserved. 9
Multithreaded 2 GB Quicksort Benchmark
94.00%
96.00%
98.00%
100.00%
102.00%
104.00%
106.00%
108.00%
110.00%
75% 50% 33% 25% 20%
Co
mp
leti
on
Tim
e /
Nati
ve T
ime
Local Memory Percentage
Native
Remote
Mirrored
© 2012 SAP AG. All rights reserved. 10
SAP HANA TPC-H like OLAP Benchmark (128 GB / 40 vCPU)
99.00%
99.50%
100.00%
100.50%
101.00%
101.50%
102.00%
50% 33%
Co
mp
leti
on
Tim
e /
Nati
ve T
ime
Local Memory Percentage
Native
Mirrored
© 2012 SAP AG. All rights reserved. 11
Application Memory Pooling
Required if virtualizaiton can„t be used due to
performance of usage of exotic hardware.
When using virtualization we „remotify“ the entire
guest VM address space.
Here we need to „remofity“ all processes
associated with the application.
© 2012 SAP AG. All rights reserved. 12
Hecatonchire key takeaways
Already provides transparent VM / application
memory aggregation
Extends existing cloud & virtualization
solutions
Released as open source / hosted on GitHub
Developed by SAP Research Technology
Infrastructure (TI) Practice
Thank you!
Aidan Shribman
SAP Research Israel
The Hecatonchire Project
http://www.hecatonchire.com
© 2012 SAP AG. All rights reserved. 14
Legal Disclaimer
The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of
SAP. This presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP
has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or
release any functionality mentioned therein. This document, or any related presentation and SAP's strategy and possible future
developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at
any time for any reason without notice. The information on this document is not a commitment, promise or legal obligation to
deliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied,
including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This
document is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or
omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially
from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as
of their dates, and they should not be relied upon in making purchasing decisions.