35
Are VM’s Passé? Ben Golub CEO Docker, Inc.

Are VM Passé?

Embed Size (px)

DESCRIPTION

Are VM passé? Ben Golub's awesome presentation about Docker, the project roadmap and its ecosystem at Interop Las Vegas

Citation preview

Page 1: Are VM Passé?

Are VM’s Passé?Ben Golub

CEO Docker, Inc.

Page 2: Are VM Passé?

They told me that I needed a provocative title for this talk.

So…

Page 3: Are VM Passé?

Q: Are VMs Passé?

Page 4: Are VM Passé?

A: No

Page 5: Are VM Passé?

Thank you for attending

Page 6: Are VM Passé?

A different question…

• Q: Is there a better alternative for many use cases &

environments?

–application management & creation?

–application deployment across clusters & clouds?

–CI & CD?

–scale out?

–high performance?

–collaborative development?

Page 7: Are VM Passé?

A: Yes

Page 8: Are VM Passé?

Agenda

• Server Proliferation and the rise of the VM

• The Matrix from Hell and the need for lightweight, interoperable containers

• Step 1: Making lightweight containers: • Containers vs. VMs: how they work

• Step 2: Making containers really interoperable:

• Step 3: Making containers really lightweight

• Step 4: Creating a container-based system for app mgt & deployment

• Step 5: Creating an ecosystem around containers

• Where to Use VMs vs. Containers/Docker

• Learn more

Page 9: Are VM Passé?

Some ancient history: Where did VM’s come from?

• PROBLEM IN 2000: Server Proliferation/Consolidation

- I’ve got a single purpose, physical Microsoft Exchange Server, Mac print server, and Custom Unix inventory server

- Machines are getting more powerful

- I want to consolidate all those single purpose, physical servers onto a single server

- BTW--It takes too damn long to provision a physical machine

• ANSWER: Create a Virtual Machine

Page 10: Are VM Passé?

Results

• Single purpose physical application servers become single purpose virtual servers

• Provisioning a “server” goes from days/weeks to minutes

• Huge cost savings

• An awesome solution to the server consolidation problem

• An awesome solution for creating flexible infrastructures

• Mature ecosystem and tool set for isolation, security, management

Page 11: Are VM Passé?

What has changed since the VM was developed?

2000 2014

Apps are long lived Development is iterative and constant

Apps are monolithic and developed on a single stack

Apps are created from loosely coupled components, themselves created from a multitude of “stacks”

Deployment is to a single server Deployment is to a variety of servers: VM, physical, cluster, open stack,public cloud, +++

• Result: An application isn’t easily represented or managed as a single purpose server (whether physical or virtual)

Page 12: Are VM Passé?

Static website

Web frontend

User DB

Queue Analytics DB

Background workers

API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8

hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis-sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs +

phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Development VM

QA server

Public Cloud

Disaster recovery

Contributor’s laptop

Production Servers

The Problem in 2014M

ult

iplic

ity

of

Stac

ksM

ult

iplic

ity

of

har

dw

are

e

nvi

ron

men

ts

Production Cluster

Customer Data Center

Do

service

s and

app

s in

teract

app

rop

riately?

Can

I migrate

sm

oo

thly an

d

qu

ickly?

Page 13: Are VM Passé?

The Matrix From Hell

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development

VMQA Server

Single Prod

Server

Onsite

ClusterPublic Cloud

Contributor’s

laptop

Customer

Servers

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Page 14: Are VM Passé?

Mu

ltip

licit

y o

f G

oo

ds

Mu

ltip

ilici

tyo

f m

eth

od

s fo

r tr

ansp

ort

ing/

sto

rin

g

Do

I wo

rry abo

ut

ho

w go

od

s interact

(e.g. co

ffee

be

ans

next to

spice

s)

Can

I transp

ort q

uickly

and

smo

oth

ly(e

.g. from

bo

at to train

to

truck)

An Inspiration…and some really ancient history: Cargo Transport Pre-1960

Page 15: Are VM Passé?

Mu

ltip

licit

y o

f G

oo

ds

Mu

ltip

licit

y o

f m

eth

od

s fo

r tr

ansp

ort

ing

/sto

rin

g

Do

I wo

rry abo

ut

ho

w go

od

s interact

(e.g. coffee

be

ans

next to

spices)

Can

I transp

ort

qu

ickly and

smo

oth

ly(e.g. fro

m b

oat to

train

to tru

ck)

Solution: Intermodal Shipping Container

…in between, can be loaded and

unloaded, stacked, transported

efficiently over long distances,

and transferred from one mode

of transport to another

A standard container that is

loaded with virtually any

goods, and stays sealed until it reaches final delivery.

Page 16: Are VM Passé?

This spawned an Intermodal Shipping Container Ecosystem

• 90% of all cargo now shipped in a standard container• Order of magnitude reduction in cost and time to load and unload ships• Massive reduction in losses due to theft or damage• Huge reduction in freight cost as percent of final goods (from >25% to <3%) massive globalization• 5000 ships deliver 200M containers per year

Page 17: Are VM Passé?

Static website Web frontend User DB Queue Analytics DB

Development

VMQA server Public Cloud Contributor’s

laptop

Let’s create a shipping container system for code M

ult

iplic

ity

of

Stac

ksM

ult

iplic

ity

of

har

dw

are

envi

ron

me

nts

Production

ClusterCustomer Data

Center

Do

services an

d ap

ps

interact

app

rop

riately?

Can

I migrate

smo

oth

ly and

qu

ickly

…that can be manipulated using

standard operations and run

consistently on virtually any

hardware platform

An engine that enables any

payload to be encapsulated

as a lightweight, portable,

self-sufficient container…

Page 18: Are VM Passé?

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development

VMQA Server

Single Prod

Server

Onsite

ClusterPublic Cloud

Contributor’s

laptop

Customer

Servers

Eliminate the matrix from Hell

Page 19: Are VM Passé?

AppA

Step One: Create a lightweight container (vs. VMs)

Hypervisor (Type 2)

Host OS

Server

GuestOS

Bins/Libs

AppA’

GuestOS

Bins/Libs

AppB

GuestOS

Bins/Libs

Ap

p A

Host OS

Server

Bins/Libs

Ap

p A

Bins/Libs

Ap

p B

Ap

p B

Ap

p B

Ap

p B

’VM

Container

Containers are isolated,but share OS kernel and, where appropriate, bins/libraries

GuestOS

GuestOS

…result is significantly faster deployment, much less overhead, easier migration, faster restart

Page 20: Are VM Passé?

A great slide stolen from IBM: Why Containers?

Provision in seconds / milliseconds

Near bare metal runtime performance

10 x greater density

VM-like agility – it’s still “virtualization”

Flexibility

– Containerize a “system”

– Containerize “application(s)”

Lightweight

– Just enough Operating System (JeOS)

– Minimal per container penalty

Open source – free – lower TCO

Supported with OOTB modern Linux kernel

Growing in popularity

3/28/2014 20

“Containers as poised as the next VM in our modern Cloud era…”

Manual VM LXC

Provision Time

Days

Minutes

Seconds / ms

linpack performance @ 45000

0

50

100

150

200

250

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 BM

vcpus

GF

lop

s

Google trends - LXC Google trends - docker

Page 21: Are VM Passé?

Step 2: Make the containers easy to use, standardized, interoperable, automatable

• Shipping containers are a standard size, and have hooks and holes in all the same places

• With Docker, Containers get the following:- Ease of use, tooling- Re-usable components- Ability to run on any Linux server today: physical, virtual,

VM, cloud, OpenStack, +++- (Stay tuned for other O/S’s)

- Ability to move between any of the above in a matter of seconds-no modification or delay

- Ability to share containerized components- Interoperability with all existing devops tools- Self contained environment—no dependency hell- Tools for how containers work together: linking, nesting,

discovery, orchestration, ++

• You get ability to separate app management from infrastructure management

Page 22: Are VM Passé?

Technical & cultural revolution: separation of concerns

• Dan the Developer

• Worries about what’s “inside” the container

• His code

• His Libraries

• His Package Manager

• His Apps

• His Data

• All Linux servers look the same

• Oscar the Ops Guy• Worries about what’s “outside” the

container

• Logging

• Remote access

• Monitoring

• Network config

• All containers start, stop, copy, attach, migrate, etc. the same way

Page 23: Are VM Passé?

Step 3: Make containers super lightweight

Bins/Libs

AppA

Original App(No OS to takeup space, resources,or require restart)

Ap

p Δ

Bin

s/

AppA

Bins/Libs

AppA’

GuestOS

Bins/Libs

Modified App

Copy on write allowsus to only save the diffsBetween container A and containerA’

VMsEvery app, every copy of anapp, and every slight modificationof the app requires a new virtual server

AppA

GuestOS

Bins/Libs

Copy ofApp

No OS. CanShare bins/libs

AppA

GuestOS

GuestOS

VMs Containers

Page 24: Are VM Passé?

Prod Hosts 21000+: OS (Linux)Prod Hosts 21000+: OS (Linux)

Step 4: Build a System for creating, managing, deploying code

Source Code

Repository

DockerfileFor A

DockerRegistry

Build

Do

cker

Prod Hosts: OS (Linux)

Co

ntain

er A

Co

ntain

er B

Co

ntain

er C

Co

ntain

er A

Push

SearchPull

Run

Dev Host : OS (Linux)

Do

cker

Page 25: Are VM Passé?

Including a System for Changes and Updates

DockerRegistry

Push

Update

Bins/Libs

AppA

Ap

p Δ

Bin

s/

Base Container

Image

Hosts are now running A’’

Container Mod A’’

Ap

p Δ

Bin

s/

Bins/Libs

AppA

Bin

s/

Bins/Libs

AppA’’

Host(s) running A want to upgrade to A’’. Requests update. Gets only diffs

Container Mod A’

Prod Hosts 21000+: OS (Linux)Prod Hosts 21000+: OS (Linux)Prod Hosts: OS (Linux)Prod Hosts 21000+: OS (Linux)Prod Hosts 21000+: OS (Linux)Prod Hosts: OS (Linux)

Page 26: Are VM Passé?

Including a System for the Full Lifecycle

Develop Test/QA Build Run ScaleHyper scale

APP CREATION

APP DEPLOYMENT

APP MANAGEMENT

Page 27: Are VM Passé?

Docker engine

Docker engine

Docker engine

Docker engine

Containers can be linked and assembled into complex service-oriented stacks.

v0.7Stacks can span multiple machines, using encrypted and authenticated tunnels.

v0.9

docker.io

v0.really soonDocker deployments can span multiple datacenters and cloud providers by using the docker.io service. Docker.io acts as a hub and federates authentication, service discovery and orchestration across all docker engines across an organization

Including a System for Complex Apps

Page 28: Are VM Passé?

Step 5: Create an Ecosystem

45,000

Page 29: Are VM Passé?

Open Ecosystem Momentum

• Truly open: Apache license, open design, open tooling, non-Docker maintainers

• Downloads: Over 1.2 m container downloads

• Users: Over 45,000 trained developers

• Content: Over 8000 repositories now publishing containers to Docker Index

• Contributors: 380 contributors, 95% of whom don’t work for Docker, In.c

• Meetups: Over 80 cities in 30 countries have Docker meetups

• Integrations: OpenStack, RHEL, Ubuntu, Salt, Chef, Puppet, Salt +++

• Github• Over 10,000 stars

• Over 1.7 K forks

• Over 350 derivative projects

Page 30: Are VM Passé?

Who is using Docker?

… and hundreds of other small and big companies

Page 31: Are VM Passé?

Four major use cases

• Continuous Integration/Continuous Delivery:• Go from developer’s laptop, through automated test, to production, and through

scaling without modification

• Alternative form of virtualization for multi-tenant services

• Scale-out:• Rapidly scale same application across hundreds or thousands of servers…and

scale down as rapidly

• Cross Cloud Deployment• Move the same application across multiple clouds (public, private, or hybrid)

without modification or noticeable delay

Page 32: Are VM Passé?

Where should I use VMs?

• VMs are definitely the way to go to solve many problems• Heterogeneous O/S families: Run Windows app on a Mac Server

• Using O/S or kernel that doesn’t support containers

• Your real problem is infrastructure management

• You want the maturity of the VM toolset

• VM requires unique kernel setup which is not applicable to other VMs on the host (i.e. per VM kernel config)

• Need to freeze state and live migrate

• But… you can pursue a hybrid strategy: containers on VMs

• Stay tuned for better Docker/Container answers for many of the above

Page 33: Are VM Passé?

Conclusion

• Multiple forces are driving a reconsideration of how applications should be created, built, deployed, scaled, and managed

• We believe that the right approach is to decouple application management from infrastructure management

• Container based approach (vs. VM approach) provides right level of abstraction

• Enables infrastructure to be managed consistently and stably

• Enables applications to be built flexibly and deployed flexibly

• Provides greater degree of visibility, control, and management of what runs where and what components are allowed

• Massive cost, speed, efficiency savings

• Docker is becoming the standard for containerization

Page 34: Are VM Passé?

Learn More

• LXC Technical discussion: slideshare.net/BodenRussell/realizing-linux-containerslxc

• Docker project: www.docker.io/

• Follow Docker on Twitter: twitter.com/docker

• Take the Docker interactive tutorial: www.docker.io/gettingstarted/

• Join Docker on IRC: botbot.me/freenode/docker/

• Go to the Docker repository on GitHub: github.com/dotcloud/docker/

• Go to a meetup: www.docker.io/community/#Docker-Meetups

• See what others are doing: www.docker.io/community/

• Come to DockerCon, Jun 9-10, San Francisco: www.dockercon.com

Page 35: Are VM Passé?

Are VM’s Passé?Ben Golub

CEO Docker, Inc.