22
Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST-RAID era +

Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Embed Size (px)

Citation preview

Page 1: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Thriving at Petabyte scale & beyond

Software-Defined Object Storage for the POST-RAID era

+

Page 2: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

• Max out at PB level• Data Silos• Tedious management• Service downtime for upgrades• Specialized staff

RestrictedAccessibilit

y

Rigid & Brittle

Durability

ConstrainedExpansion & Extensibility

• RAID rebuild times too long• Backup costs too high• Not all data should be

treated the same, anyway

• Web servers and load balancers needed

• SMB and NFS are not web protocols

• Limited query and analysis

Legacy file storage technology is limiting the ability for entities to

thrive @ “scale”

Page 3: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

The Need...

Multi-protocol ingest support (http, file, S3, Swift, block)

Internet accessible, web-scale

Ability to run on standard commodity hardware

Simple to deploy, manage, scale & evolve

Ability to continuously protect any amount of data, natively

Secure and Compliant

Future-proof (durable & extensible)

Page 4: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Caringo’s Swarm object storage software: • Aggregates the disk capacity across heterogeneous

clusters of commodity servers to…

• Create massively & dynamically scalable multi-tenant storage systems that are…

• Self-managing, self-healing, high-performance, and durable through time, where

• Files and their descriptive metadata (i.e. objects) are accessible whenever and wherever they are needed by users, administrators and by data services

The Solution – Caringo SWARM

Page 5: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm

How We Do It – Efficient Software

CPU

NIC

No file system, no software installleaves 98% drive capacity for data

98%available

Boots from bare metal in minutesworks on raw disk level

RAM

HTTPSNMP

Runs from RAMon any x86 hardware with Linux

Tune to any workload

Page 6: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm

How We Do It – Resource Virtualization

CPU

NIC

RAM

Node 1

Page 7: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm

How We Do It – Resource Virtualization

CPU

NIC

RAM

Node 1

CPU

NIC

RAM

RAM

CPU

Page 8: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm

How We Do It – Resource Virtualization

CPU

NIC

RAM

Node 1

CPU

NIC

RAM

RAM

CPUNode 2

Page 9: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm

How We Do It – Resource Virtualization

Node 1

Node 2

Node 3

Node …

Page 10: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

How We Do It – Emergent Behavior

Nodes cooperate to perform processes

SWARM follows simple

rules to manage a complex system

Improves with size!

No single points of failure

Page 11: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

How We Do It – Encapsulated Data

Metadata stored with data

No metadata databasesto manage

data + metadata

Page 12: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

How We Do It – Encapsulated Data

Object is encapsulated

Includes all system, policy and custom metadata

Page 13: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

How We Do It – Encapsulated Data

Object is portable & dynamic

Decoupled from hardware, location and apps

Freeing data to be accessed in multiple ways

Page 14: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm Adapts

Swarm lets you grow at the paceyou choose and adapts to your requirements or workload

• Plug in new servers for more capacity & performance - address needs in real time

• New resources available in minutes

• Right resources to the right locations

• Local and geo-distribution

• Easily retire old hardware

site 1

site 2

site 3

Swarm

Page 15: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm Protects

Swarm automatically manages the data protection you choose

• Protection per cluster, bucket or at the object level

• Seamlessly move between replication and erasure coding

• Store objects with different protection levels on the same server

• Auto-managed through Lifepoints

• Protect data based on value

Any S

LA Rep

licat

ion

Erasureencoding

Page 16: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Swarm Repairs

Swarm watches for issues andrepairs them automatically so you don’t have to

• Health Processor proactively checks integrity, availability and cardinality

• Fast Volume Recovery, content aware, repairs only damaged objects

• Assurance of data quality for the entire enterprise

• Provides business continuity

++

+

++

Page 17: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Portfolio Overview

Scalable storage software for unified object, file and block

CloudScalerHTTP 1.1, Amazon S3

FileScalerSMB, NFS

BlockScaleriSCSI (Beta)

Adaptable performance for unified storage under a single namespace.Swarm

Caringo

Select the interface and hardware for your workload and use case.

Big Data/Hadoop

ArchiveEnterprise

AppsCloud

Page 18: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

It’s Simple…Really!

Caringo SWARM Object Storage Cluster

Caringo Cluster

Services Node

(Physical or Virtual)

Ethernet Switches 1Gb

or 10Gb

Small Medium, Large

R420 R720xd or +JBOD

Add Caringo Software Capacity Licenses, per TB, and annual support/maintenance

Small, Medium, Large

R420 or VM

Add Caringo CSN Software License and annual support/maintenance

• Mix and Match HW w/ any drive size• Grow from one form factor to another, at

runtime, incrementally• Never forklift upgrade again

Page 19: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Simple…But Powerful!

HT

TP

S3

Application

File

Gatew

ay

Ingest from many sources*S

cale in any increm

ent @

runtime

Automate Retention, Protection & Durability down to the object level

Sea

rch &

Discove

ry

Multi-Ten

ant

STaa

S

Ha

doo

p A

nalytics

Add Storage Services

Dis

trib

ute

, C

onso

lidat

e, S

hare

au

tom

atic

ally

Remote Sites

Page 20: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Object storage - its all about use cases

DOD Medical Health Systems (16 PBS)

PACS Archiving

Video Surveillance(2.5 PBS) File

Retention

M2M and analytics(4 PBS)

Consumer/enterprise backup-to-cloud service(2+ PBS)

plus hundreds more…

S3-like cloud storage services via Swarm & CloudScaler Web content delivery

Page 21: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Next Steps – Where/How to get Started?

• Identify your Target use case(s):• See Use Case Slide.

• If in doubt: “Drain your Filer Swamps” – File System Optimization Tier!

• Contact your Dell Account Representative for pre-sales support – All Caringo Software & Services are available through Dell

• OR, Contact [email protected]

Page 22: Thriving at Petabyte scale & beyond Software-Defined Object Storage for the POST- RAID era +

Thank You

Software-Defined Object Storage for the POST-RAID era