Upload
ian-colle
View
2.647
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
Inktank
Openstack with Ceph
Ian Colle Ceph Program Manager, Inktank [email protected] @ircolle www.linkedin.com/in/ircolle ircolle on freenode inktank.com | ceph.com
Who is this guy?
People need storage solutions that… • …are open
• …are easy to manage
• …satisfy their requirements - performance - functional - financial (cha’ ching!)
Selecting the Best Cloud Storage System
Hard Drives Are Tiny Record Players and They Fail Often jon_a_ross, Flickr / CC BY 2.0
D
55 times / day
= D
D D
x 1 MILLION
D D
D D
“That’s why I use Swift in my Openstack implementation” Hmmm, what about block storage?
I got it!
• Persistent - More familiar to users
• Not tied to a single host
- Decouples compute and storage - Enables Live migration
• Extra capabilities of storage system
- Efficient snapshots - Different types of storage available - Cloning for fast restore or scaling
Benefits of Block Storage
Ceph has reduced administration costs - “Intelligent Devices” that use a peer-to-peer mechanism to detect failures and react automatically – rapidly ensuring replication policies are still honored if a node becomes unavailable. - Swift requires an operator to notice a failure and update the ring configuration before redistribution of data is started.
Ceph guarantees the consistency of your data
- Even with large volumes of data, Ceph ensures clients get a consistent copy from any node within a region. - Swift’s replication system means that users may get stale data, even with a single site, due to slow asynchronous replication as the volume of data builds up.
Ceph over Swift
Swift has quotas, we do not (coming this Fall) Swift has object expiration, we do not (coming this Fall)
Swift over Ceph
Ceph Ceph provides object AND block storage in a single system that is compatible with the Swift and Cinder APIs and is self-healing without operator intervention.
Swift
If you use Swift, you still have to provision and manage a totally separate system to handle your block storage (in addition to paying the poor guy to go update the ring configuration)
Total Solution Comparison
Openstack I know, but what is Ceph?
OPEN SOURCE
COMMUNITY-FOCUSED
SCALABLE
NO SINGLE POINT OF FAILURE
SOFTWARE BASED
SELF-MANAGING
philosophy design
RADOS A Reliable, Autonomous, Distributed Object Store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP
RBD (RADOS Block Device) A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RGW (RADOS Gateway) A bucket-based REST gateway, compatible with S3 and Swift
APP APP HOST/VM CLIENT
RADOS A Reliable, Autonomous, Distributed Object Store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP
RBD (RADOS Block Device) A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RGW (RADOS Gateway) A bucket-based REST gateway, compatible with S3 and Swift
APP APP HOST/VM CLIENT
Monitors: • Maintain cluster map • Provide consensus for distributed decision-making • Must have an odd number • These do not serve stored objects to clients
M
OSDs: • One per disk (recommended) • At least three in a cluster • Serve stored objects to clients • Intelligently peer to perform replication tasks • Supports object classes
16
DISK
FS
DISK DISK
OSD
DISK DISK
OSD OSD OSD OSD
FS FS FS FS btrfs xfs ext4
M M M
M
M
M
HUMAN
RADOS A Reliable, Autonomous, Distributed Object Store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP
RBD (RADOS Block Device) A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RGW (RADOS Gateway) A bucket-based REST gateway, compatible with S3 and Swift
APP APP HOST/VM CLIENT
L
19
LIBRADOS • Provides direct access to RADOS for applications • C, C++, Python, PHP, Java • No HTTP overhead
LIBRADOS
M
M
M
APP
native
RADOS A Reliable, Autonomous, Distributed Object Store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP
RBD (RADOS Block Device) A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RGW (RADOS Gateway) A bucket-based REST gateway, compatible with S3 and Swift
APP APP HOST/VM CLIENT
M
M
M
LIBRADOS RGW
APP
native
REST
LIBRADOS RGW
APP
RADOS Gateway: • REST-based interface to RADOS • Supports buckets, accounting • Compatible with S3 and Swift applications
RADOS A Reliable, Autonomous, Distributed Object Store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP
RBD (RADOS Block Device) A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RGW (RADOS Gateway) A bucket-based REST gateway, compatible with S3 and Swift
APP APP HOST/VM CLIENT
M
M
M
VM
LIBRADOS LIBRBD
VIRTUALIZATION CONTAINER
RADOS Block Device: • Storage of virtual disks in RADOS • Allows decoupling of VMs and containers • Live migration! • Images are striped across the cluster • Boot support in QEMU, KVM, and OpenStack Nova (more on that later!) • Mount support in the Linux kernel
RADOS A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP
RBD A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW A bucket-based REST gateway, compatible with S3 and Swift
APP APP HOST/VM CLIENT
What Makes Ceph Unique? Part one: CRUSH
APP ??
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
APP
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
APP
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
D C
A-G
H-N
O-T
U-Z
F*
10 10 01 01 10 10 01 11 01 10
10 10 01 01 10 10 01 11 01 10
hash(object name) % num pg
CRUSH(pg, cluster state, rule set)
10 10 01 01 10 10 01 11 01 10
10 10 01 01 10 10 01 11 01 10
CRUSH • Pseudo-random placement algorithm • Ensures even distribution • Repeatable, deterministic • Rule-based configuration
• Replica count • Infrastructure topology • Weighting
35
36
37
38
What Makes Ceph Unique Part two: thin provisioning
40
HOW DO YOU
SPIN UP
THOUSANDS OF VMs
INSTANTLY
AND
EFFICIENTLY?
42
43
44
How Does Ceph work with Openstack?
RBD support initially added in Cactus Have increased features / integration with each subsequent release You can use both the Swift (object/blob store) and Keystone (identity service) APIs to talk to RGW Cinder block storage as a service talks directly to RBD Nova cloud computing controller talks to RBD via the hypervisor Coming in Havana – Ability to create a volume from an RBD image via the Horizon UI
Ceph / Openstack Integration
What is Inktank? I really like your polo shirt, please tell me what it means!
The majority of Ceph contributors Formed by Sage Weil (CTO), the creator of Ceph, in 2011 Funded by DreamHost and other investors (Mark Shuttleworth, etc.)
Who?
To ensure the long-term success of Ceph To help companies adopt Ceph through services, support, training, and consulting
Why?
Guide the Ceph roadmap - Hosting a virtual Ceph Design Summit in early May
Standardize the Ceph development and release schedule - Quarterly stable releases, interim releases every 2 weeks * May 2013 – Cuttlefish RBD Incremental Snapshots! * Aug 2013 – Dumpling Disaster Recovery (Multisite) Admin API
* Nov 2013 – Some really cool cephalopod name that starts with an E
Ensure Quality - Maintain Teuthology test suite - Harden each stable release via extensive manual and automated testing
Develop reference and custom architectures for implementation
What?
• Inktank is a Strategic partner for Dell in Emerging Solutions
• The Emerging Solutions Ecosystem Partner Program is designed to deliver complementary cloud components
• As part of this program, Dell and Inktank provide: > Ceph Storage Software - Adds scalable cloud storage to the Dell OpenStack-powered cloud - Uses Crowbar to provision and configure a Ceph cluster (Yeah
Crowbar!) > Professional Services, Support, and Training - Collaborative Support for Dell hardware customers > Joint Solution - Validated against Dell Reference Architectures via the
Technology Partner program
Inktank/Dell Partnership
Try Ceph and tell us what you think! http://ceph.com/resources/downloads/ http://ceph.com/resources/mailing-list-irc/
- Ask, if you need help. - Help others, if you can!
Ask your company to start dedicating dev resources to the project! http://github.com/ceph Find a bug (http://tracker.ceph.com) and fix it! Participate in our Ceph Design Summit!
What do we want from you??
We’re planning the next release of Ceph and would love your input. What features would you like us to include?
iSCSI?
Live Migration?
One final request…
Questions?
56
Ian Colle Ceph Program Manager, Inktank [email protected] @ircolle www.linkedin.com/in/ircolle ircolle on freenode inktank.com | ceph.com