Turning Object Storage intoVirtual Machine Storage
The open source Storage Router
The Rise of Object Storage
Source: http://blog.oxygencloud.com/2013/09/16/after-10-years-object-storage-investment-continues-and-begins-to-bear-significant-fruit/
Object Storage
• Amazon Simple Storage Service (S3) started the Object Store momentum
• ... but new players are taking over:– Scality– Ceph– Swiftstack– Amplidata– DataDirect Networks– ...
• Used to store files but not as primary storage for Virtual Machines
Object Storage Benefits
• Build scale-out storage by adding more servers
• High reliability by duplicating content across nodes
• Easy to manage (no need to know about RAID or volumes)
• Standard API as all solutions support S3 API, no vendor lock-in
• Cost-effictive: mix SSDs (small but fast) and SATA (slow but large)
Object Storage <> Hypervisor
• Object Storage– Objects– Buckets– S3 API
• Hypervisor– Files– Folders– Volumes– Block level storage
What is needed is a technology whereby Virtual Machines can use Object Stores instead of a SAN and get the benefits of the low cost
and scale-out capabilities of Object Stores
Object Storage Challenges
• The issues to use Object Storage as VM Storage:– Eventual consistency (the CAP Theorem)
– Latency & performance• VMs require low latency and high performance• Object Stores are developed to contain lots of data
(large disks and low performance)• Additional latency as Object Store is on the Local LAN instead of
attached to the server
– Different Management Paradigms• Object Stores understand Objects <> Hypervisors understand VMs
Distributed File System?
• Distributed File Systems such as GlusterFS and Ceph are not an answer:– The CAP Theorem also applies to these (consistency <> performance)– They were not designed for VMs (VM Snapshots?, cloning?)
• Even new File Systems (copy-on-write) can’t give an answer
None of the File Systems today have been designed to link Virtual Machines and Object Storage.
The answer: Open vStorage
Open vStorage turns Object Storage into Virtual Machine usable primary Storage!
What is Open vStorage
Open vStorage is an open source “Grid Storage Router" installed as a virtual machine on a host or a cluster of hosts to create a VM-centric, clustered, reliable, scale out and high performance storage system for virtual machines out of Storage Backends such as Object Stores.
An Overview
ESXi / KVM ESXi / KVM ESXi / KVM
Scale-outVM VM
VM VM
VM VM
VM VM
VM VM
VM VM
Unified Namespace
...Swift
SSDSSD
OpenvStorage
SSDSSD
OpenvStorage
SSDSSD
OpenvStorage
Key aspects are
• Unified namespace presented as NFS to all Hosts• Open vStorgae leverages server flash as acceleration• Flexibility in storage backends• Redundancy and Fault Tolerance in Software• Asymmetric Scale Out Architecture• VM-centric approach• Multi-hypervisor• Open-source: Apache License version, Version 2,0
Solving Eventual Consistency
TIM
E
LBA 1: A
LBA 2: B
LBA 3: C
LBA 4: D
LBA 5: E
LBA 1: F
LBA 1: A
LBA 6: G
LBA 2: C
LBA 3: D
LBA 4: E
...
SSD or PCI Flash
LBA 1: A
LBA 2: B
LBA 3: C
LBA 4: D
LBA 5: E
SCO 1
LBA 1: F
LBA 1: A
LBA 6: G
LBA 2: C
LBA 3: D
SCO 2
LBA 4: E
...
SCO 3
Transfer SCOs once they are full to the Object Store at slow paceEach write is appended
to the current SCO
Swift
...
How does it work ...
VMVMDK
NFS
VMConfig
Open vStorage VSA
DistributedDatabase
Volume
VMVMDK
VMConfig
Volume
Storage Backend (Ceph, SwiftStack, ...)
Open vStorage VSA
VMVMDK
VMConfig
Volume
Volume is only available on 1 Host!
Open vStorage: Summary
• Uses an always append approach• Push Storage Container Objects to the Object Store once it is full
(at slower pace)• Each Virtual Machine with the Open vStorage software has its own
NFS server and exports a different file system instance• But each Host is tricked into believing that it accesses a single
unified namespace shared across all the Hosts (so vMotion is supported)
Download the Open vStorage (it is free) and give it a try!
http://doc.openvstorage.com/
Give it a try ...