STATEFUL CONTAINER MIT OPENSTACK UND DOCKER · OpenStack (Cinder) EMC Isilon EMC ScaleIO EMC VMAX EMC XtremIO Google Compute Engine (GCE) VirtualBox Ubuntu Debian RedHat CentOS CoreOS

1 © Copyright 2016 EMC Corporation. All rights reserved.

STATEFUL CONTAINER MIT OPENSTACK UND DOCKER SEBASTIAN SCHMIDT, ADVISORY SPECIALIST, EMC DEUTSCHLAND GMBH


SEBASTIAN SCHMIDT

• Previously software developer, lead architect und manager consulting in a startup

• Advisory Specialist at EMC Deutschland GmbH – Building Clouds – Architecting software-defined

infrastructure – Help customers through digital

transformation

@seb2tweet [email protected] https://de.linkedin.com/in/sebastianschmidt1


EMC FEDERATION


WHICH EMERGING TECHNOLOGIES INTEREST OPENSTACK USERS?

http://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf


?


WHY COMPANIES REALLY ADOPT DOCKER


HOW DO DEVELOPERS SEE CONTAINERS?

Write code Push it

Platform gathers requirements

Builds container

Container is deployed

Container as a Service / Platform


• Initialy designed for – Cloud Native Apps

– Microservices

– 12-factor apps

– Fail -> restart

– Stateless

– Scale-out

WHAT DO YOU RUN IN DOCKER CONTAINERS?


APPLICATION STATE DIFFERENCES

Front End or Non-Persistent

Scale

-Up S

cale

-Out


• Heck yes!

• Can provide storage volumes to VMs

• Volumes can be bootable or attached auxiliary storage

• Volumes can persist when the VM is destroyed

DOES OPENSTACK PROVIDE PERSISTENCE?


HTTP://DOCS.OPENSTACK.ORG/OPS-GUIDE/ARCH_STORAGE.HTML

OPENSTACK STORAGE OPTIONS

Table. OpenStack storage

Ephemeral storage Block storage Object storage Shared File System storage

Used to… Run operating system and scratch space

Add additional persistent storage to a virtual machine (VM)

Store data, including VM images

Add additional persistent storage to a virtual machine

Accessed through… A file system

A block device that can be partitioned, formatted, and mounted (such as, /dev/vdc)

The REST API

A Shared File Systems service share that can be partitioned, formatted and mounted (such as /dev/vdc)

Accessible from… Within a VM Within a VM Anywhere Within a VM

Persists until… VM is terminated Deleted by user Deleted by user Deleted by user


• Yes! Definitely yes. Probably? Well, sort of.

• That’s kind of… not really the point of containers.

DOES DOCKER PROVIDE PERSISTENCE?

• You can map volumes on the Docker server

• Storage can persist after the container goes away


STATE IS OUTSIDE OF PLATFORM

OPTION ONE

Legacy Apps

Object Storage Databases

Drawback: Platform benefits limited to everything that is inside the platform

Container Platform

Stateless Apps

Integrated through REST API, defined interfaces, pre-distributed users, etc.


Container Platform

Volu

me A

PI

GET STATE AS A FIRST CLASS CITIZEN INTO YOUR PLATFORM

OPTION TWO

Legacy Apps

Object Storage Databases

Drawback: Much more complexity in the Platform

Cloud Native Apps


HOW DOES PERSISTENCE WORK WITH CONTAINERS ?


TYPICAL DOCKER ECOSYSTEM APPROACH


• Enable data volumes to persist beyond the lifetime of a Docker host or container

• Included since Docker 1.9 (November 2015)

• Operations: – Create, Remove, Mount, Path, Unmount

– Additional options can be passed to volume drivers

HELLO DOCKER VOLUME MANAGER API!


HOW DOES THE VOLUME API WORK?


• 17 Plugins today: – https://docs.docker.com/engine/extend/plugins/

– Several drivers from vendors

– Some general abstraction layers

• Why additional abstraction layers? – Why not just Docker volume driver <-> Cinder ?

DOCKER VOLUME API - STATUS

https://docs.docker.com/engine/extend/plugins/

https://docs.docker.com/engine/extend/plugins/


REX-RAY ARCHITECTURE


HTTPS://GITHUB.COM/EMCCODE/REXRAY

GET TO KNOW REX-RAY

AWS EC2 (EBS)

OpenStack (Cinder)

EMC Isilon

EMC ScaleIO

EMC VMAX

EMC XtremIO

Google Compute Engine (GCE)

VirtualBox

Ubuntu Debian RedHat CentOS CoreOS OSX TinyLinux (boot2docker)

Docker Volume API Mesos Isolator


• Abstraction makes running containers across multiple hosts transparent

• Works across multiple different substrates/clouds!

• Run important things with data in containers

• Container images remain non-persistent, while the container data volumes are persistent

• Container OSs remain immutable

• Enables High Availability applications

BUT WHY IS THAT SO EXCITING?





RECAP: CONTAINER PLATFORM ARCHITECTURE

Write code Push it

Platform gathers requirements

Builds container

Container is deployed

Container as a Service / Platform


• Manages all physical/virtual resources

• Deploys and schedules apps

• Handles hardware failures

RESOURCE MANAGERS AND SCHEDULERS


• Limitations today – Host local resources only

– Memory, CPU as main resources

– No concept for central or global services • Storage

• Network

RESOURCE MANAGERS AND SCHEDULERS


THE STORAGE SCHEDULER

GET TO KNOW POLLY


• Plugs directly into scheduling layer

• Creates, maps, snapshots, deletes volumes

• Allows scheduling based on storage information

• Check it out on Github!

POLLY – THE POLYMORPHIC STORAGE SCHEDULER HTTPS://GITHUB.COM/EMCCODE/POLLY


https://www.youtube.com/watch?v=WnVAS1raRik





SCALE-OUT STORAGE SOFTWARE

SCALING PERSISTENCE / STORAGE

One Pool, One API, runs co-located with workloads

Scale-out


• Cinder driver – http://docs.openstack.org/mitaka/config-reference/block-storage/drivers/emc-

scaleio-driver.html

• Canonical Integration (JUJU Charms) – https://github.com/cloudscaling/juju-scaleio – Current version is RC1.

• Nova ethemeral volumes – https://github.com/cloudscaling/nova-scaleio-ephemeral

• Cinder Backups (works with any Cinder device) – Includes VM Config and everything needed for independant restore

SCALEIO AND OPENSTACK

http://docs.openstack.org/mitaka/config-reference/block-storage/drivers/emc-scaleio-driver.html










https://github.com/cloudscaling/juju-scaleio






https://github.com/cloudscaling/nova-scaleio-ephemeral







• https://github.com/cloudscaling/fuel-plugin-scaleio

SCALEIO AND FUEL

https://github.com/cloudscaling/fuel-plugin-scaleio









• Other cool things we are working on – Bare Metal CPI for Cloud Foundry

• https://github.com/cloudfoundry-incubator/bosh-rackhd-cpi-release

– RackHD, management for commodity server • https://github.com/rackhd/rackhd

– Cloud Foundry Service Broker for Object Storage (ECS) • https://github.com/emccode/ecs-cf-service-broker

– Unikernel „Compiler“ (UniK) • https://github.com/emc-advanced-dev/unik

EMC{CODE} NEEDS YOU HTTPS://EMCCODE.GITHUB.IO/

https://www.youtube.com/playlist?list=PLbssOJyyvHuWiBQAg9EFWH570timj2fxt

https://github.com/cloudfoundry-incubator/bosh-rackhd-cpi-release










https://github.com/rackhd/rackhd


https://github.com/emccode/ecs-cf-service-broker








https://github.com/emc-advanced-dev/unik










RACKHD AND SHOVEL -> IRONIC

• New project Shovel contributed to OpenStack – https://github.com/openstack/shovel

• https://github.com/rackhd/rackhd

https://github.com/openstack/shovel

https://github.com/openstack/shovel



https://www.youtube.com/watch?v=LluHht5ixTI

https://www.youtube.com/watch?v=LluHht5ixTI


EMC{CODE} LIBSTORAGE

FUTURE OF CONTAINER STORAGE

Storage Orchestration Tools Today libStorage Embedded Architecture

http://libstorage.readthedocs.io/en/stable/ https://github.com/emccode/libstorage

http://libstorage.readthedocs.io/en/stable/

http://libstorage.readthedocs.io/en/stable/

https://github.com/emccode/libstorage

https://github.com/emccode/libstorage


Hypervisor == Isolation

Tenant Tenant

H H H H H H

IaaS ( ) == Control

DVDI DVDI

Marathon

Documents

STATEFUL CONTAINER MIT OPENSTACK UND DOCKER · OpenStack (Cinder) EMC Isilon EMC ScaleIO EMC VMAX EMC XtremIO Google Compute Engine (GCE) VirtualBox Ubuntu Debian RedHat CentOS CoreOS