View
142
Download
1
Category
Preview:
Citation preview
BlackStor(Based on DRBD)
World's fastest & most reliable Cloud Native Software Defined Storage
How it came into the world, what’s available now and where it’s headed
1
2
Who we are
Copyright © 2016 PROZETA & LINBIT
⬢ Since 1991⬢ Group annual turnover:
20M EUR (2015)
⬢ 2 divisions➥ HW / Value added distributor & reseller➥ Infrastructure services
IaaS, SaaS, PaaS; In-house development
About PROZETA
3
Copyright © 2016 PROZETA & LINBIT
About PROZETA
4
Copyright © 2016 PROZETA & LINBIT
About PROZETA
5
6
What’s needed for serious cloud storage? What’s available on the market?
Copyright © 2016 PROZETA & LINBIT
What Are The Options?
7
⬢ IaaS = Volumes / Block Storage
⬢ Servers + Disks + Software
⬢ Local storage
➥ not scalable
⬢ iSCSI
➥ software defined &
redundant: hard to find
⬢ GlusterFS, LizardFS, …
➥ too complex to operate...
Goal:
“Next year I’d like to try out a different storage software solution…
...and I can!”
Copyright © 2016 PROZETA & LINBIT
CEPH
8
⬢ No compression⬢ No deduplication⬢ Ineffective thin provisioning⬢ Extremely CPU hungry⬢ Bad write performance / high latency - SLOOOOOOOOOW…
High TCO due to the need for extra hardware, unpredictable performance
Perfect integration with OpenStack
VERY stable BUT very difficult to fix potential issues in a timely manner
9
So what do we really need?
Copyright © 2016 PROZETA & LINBIT
SDS Requirements
We wanted SDS on the level of Enterprise storage BUT based on a cost-effective open-source technology we understand.
10
⬢ High Stability⬢ High Reliability⬢ High but Predictable Performance⬢ Scaling options - both Horizontal & Vertical⬢ Modest resource use⬢ Simple to manage (plus repair in case of disaster)⬢ Possible to get info for deep monitoring & metrics⬢ Extensible: features like Compression, Deduplication, etc.
Copyright © 2016 PROZETA & LINBIT 11
Definitely requires a solid underlying base technology …
Something new, right?
Copyright © 2016 PROZETA & LINBIT 12
And then we (re-) discovered
DRBD!
13
What is DRBD?
Copyright © 2016 PROZETA & LINBIT
What is DRBD?
14
⬢ 15+ years on the market⬢ Developed by a stable company with great reputation⬢ More than 250.000 production installations worldwide⬢ Kernel modules (DRBD core) are standard part of Linux kernel⬢ Super-reliable and super-fast!
Copyright © 2016 PROZETA & LINBIT
DRBD shortly...
15
Copyright © 2016 PROZETA & LINBIT
DRBD - Key Features
16
⬢ automatic resync after node or connectivity failure➥ direction, amount, no full resync needed
⬢ performs in Linux kernel implementation➥ 160k IOPs measured (on SSDs, of course)
⬢ multiple volumes per resource (replication group)➥ write order fidelity within resource
⬢ comes with Pacemaker integration⬢ synchronous and async replication (LAN and WAN)⬢ In Linux upstream since 2.6.33 (released 2010)
Copyright © 2016 PROZETA & LINBIT
DRBD - New Features in 9.x
17
⬢ Up to 31 connections per Resource
⬢ Fixes the drawbacks of stacking
⬢ Auto promote
⬢ Transport abstraction (TCP, SCTP, RDMA)
⬢ drbdmanage
Copyright © 2016 PROZETA & LINBIT 18
Great, so let's work out a long-term OpenStack roadmap...
Copyright © 2016 PROZETA & LINBIT 19
...say 1 month to full integration...
...and 3 months to full orchestration! :-)
20
OpenStack SDS Integration
Copyright © 2016 PROZETA & LINBIT
OpenStack SDS integration - basic requirements
21
⬢ Block-device replication layer (Kernel drivers)
⬢ Storage orchestrator
⬢ Tooling for manual management of SDS
⬢ Exports for monitoring & performance analysis
⬢ Cinder driver that will “glue” orchestrator to OpenStack
Copyright © 2016 PROZETA & LINBIT
OpenStack SDS integration - issues we were aware of
22
⬢ Orchestration➥ Scalability➥ Queue management (Parallel tasks)➥ Performance / Speed
⬢ Kernel modules➥ We based it on DRBD 9.x and there were still some issues to fix
⬢ Tooling➥ Not enough, or requires deeper know-how
⬢ Monitoring➥ Missing some machine-readable exports for deeper monitoring / performance analysis
Copyright © 2016 PROZETA & LINBIT
Orchestration
23
⬢ After months ended-up with 2 paths: DRBD Manage & BlackStor Orchestrator⬢ DRBD Manage
➥ Universal orchestrator for any kind of deployment, not only cloud…➥ No dependencies on 3rd parties➥ Long path to solve all the issues relevant to high-scale cloud deployments
⬢ BlackStor Orchestrator➥ Optimized for cloud deployments➥ Utilizing 3rd parties (RabbitMQ, etc.)➥ To be released publically between Nov/Dec 2016, free to use with sufficient functionality for
basic deployments
Copyright © 2016 PROZETA & LINBIT
Kernel modules
24
⬢ Fixed all bugs found so far
⬢ Add in support as new requirements found in actual cloud deployments
Copyright © 2016 PROZETA & LINBIT
Tooling & Monitoring
25
⬢ Extended monitoring capabilities - added new machine readable exports
⬢ Built new dashboards around solution
26
How does it work in the real world?(It works in theory… does it work in practice?)
Copyright © 2016 PROZETA & LINBIT
Performance - DRBD by itselfBaseline
27
Through DRBD
Copyright © 2016 PROZETA & LINBIT
Performance - DRBD itselfDRBD with read balancing
28
Copyright © 2016 PROZETA & LINBIT
DRBD vs CEPH
29
⬢ 4-node clusters➥ 2 Class-D SSDs each➥ 2x E5-2650-L CPUs each➥ 128GB RAM each➥ 10GbE each
⬢ Attached VM - 8-core CPU,16GB RAM
⬢ Ubuntu 14.04.5, Kernel 3.16⬢ DRBD9 vs Ceph 10.2.3⬢ fio 2.14
Copyright © 2016 PROZETA & LINBIT
DRBD vs CEPHSequential writes (IOPS)
30
Random writes (IOPS)
DRBD ~10x faster DRBD ~1,5-2x faster
Copyright © 2016 PROZETA & LINBIT
DRBD vs CEPHSequential read (IOPS)
31
Random read (IOPS)
DRBD ~3x faster DRBD ~4x faster
Copyright © 2016 PROZETA & LINBIT
CPU utilizationSequential write
32
Random write
Sequential read Random read
Copyright © 2016 PROZETA & LINBIT
Reliability
We have tried to totally break it a million times... with NO SUCCESS!
Just what you would expect in a production environment
33
Copyright © 2016 PROZETA & LINBIT
Scalability
34
⬢ Scales both ways - horizontally & vertically⬢ New resources
➥ Linear growth➥ Predictable performance with minimal overhead
⬢ Scale-out with single volume/resource across multiple nodes➥ New boost in performance➥ Available with BlackStor Orchestrator
Copyright © 2016 PROZETA & LINBIT
Monitoring
Deep insight!
35
36
Okay, I’ll launch this into production…but who’ll integrate it? what if something happens?
Copyright © 2016 PROZETA & LINBIT
Support
37
DRBD core ➔ LINBIT
DRBD Manage ➔ LINBIT
BlackStor Orchestrator ➔ PROZETA
Cloud (eg. OpenStack) integration ➔ PROZETA
Training, integration, etc. ➔ PROZETA, LINBIT
38
What next?
Copyright © 2016 PROZETA & LINBIT
BlackStor - Roadmap
39
⬢ Short-term➥ Public release of BlackStor Orchestrator (incl. new Cinder driver)➥ Multi-tenancy, Multipool, ACLs, Policy driven storage….➥ Web-based interface, single API➥ Installation SW Kit for certified HW
⬢ Mid/Longer-term➥ Encryption (over the wire, at rest…)➥ New drivers (Nova, Docker, Windows)➥ Advanced features (QoS, Stop writes when # active replicas, Storage replacement, etc.)➥ Backup package
40
Let's sum it up!
Copyright © 2016 PROZETA & LINBIT 41
Building a new cloud-native SDS is not an easy task...
Copyright © 2016 PROZETA & LINBIT 42
...but we went for it anyway.
Why? Because...
➥ nothing reliable out there
➥ great performance & production ready
➥ sweet spot between open source CEPH
& expensive Enterprise storage
Copyright © 2016 PROZETA & LINBIT 43
What you get with BlackStor⬢ Production ready⬢ Stable & Reliable⬢ Great & Predictable Performance⬢ Simple to implement & manage⬢ Includes deep metrics information⬢ With additional support options
...and last but not least….
⬢ Fantastic TCO - it doesn't cost the Earth!
Copyright © 2016 PROZETA & LINBIT 44
And let’s not forget our partners who’ve helped us!
Copyright © 2016 PROZETA & LINBIT
By the way, we're hiring:DevOps EngineersPython Developers
45
Copyright © 2016 PROZETA & LINBIT
Q&A4646
Copyright © 2016 PROZETA & LINBIT
Thank you!Michal Němec
michal.nemec@prozeta.eu
+420 724 454 871
PRO-ZETA a.s.
prozeta@prozeta.eu
+420 222 747 923
www.prozeta.eu www.prozeta.ae www.tier5.cloud
47
Recommended