1

Xen Containers: Better way to run Docker Containers

Sainath Grandhi

sainath.grandhi@intel.com

Contributions: Jun Nakajima

2

Motivation

“Containers” being adopted for application development/deploying

Containers looked upon as lightweight alternative for traditional VMs

VMs offer stronger application isolation

Benefits of VMs can be reaped if they are made lightweight and run like containers

3

Agenda

Containers

Xen Containers

Numbers

Next Steps

4 4

Host OS

Cgroups Namespaces Union FS

Application

Libraries

Middleware

Container A

Application

Libraries

Middleware

Container C

Application

Libraries

Middleware

Container D

Application

Libraries

Middleware

Container B

Namespaces Namespaces Namespaces

Server Hardware

5

Docker Containers

Running

• docker run/create/stop

Building

• docker build

Packaging

• docker push/pull/commit

Docker – a one stop solution for running, building and packaging containers

Host OS

Dockerclient

Docker daemon

Docker image

Docker command

Parent/child

Cgroups Namespaces Union FS

Application

Libraries

Middleware

Container A

6

Bare metal containers - Isolation

Isolation provided by Host OS

Security compromised kernel can be exploited by malicious images/applications for namespace intrusion

Enabling cgroups and namespaces in the kernel increases the kernel attack surface

Malicious public images

Over 30% of Official Images in Docker Hub Contain High Priority Security Vulnerabilitieshttp://www.banyanops.com/blog/analyzing-docker-hub/

Multi-tenant Cloud Providers

Google: “we see the VM as the only truly safe isolation.… Until we see foolproof security for containers, we will always double-bag our customers' workloads”http://www.informationweek.com/cloud/infrastructure-as-a-

service/google-docker-does-containers-right/d/d-id/1319146

Application

Libraries

Middleware

Container A

Application

Libraries

Middleware

Container B

Namespaces

7

Agenda

Containers

Xen Containers

Numbers

Next Steps

8

Containers

8

Host OS

Cgroups Namespaces Union FS

Application

Libraries

Middleware

Container A

Application

Libraries

Middleware

Container C

Application

Libraries

Middleware

Container D

Application

Libraries

Middleware

Container B

Namespaces Namespaces Namespaces

Server Hardware

9

VM Containers

9

Host OS/Dom0

Cgroups Union FS

Application

Libraries

Middleware

VMContainer

Application

Libraries

Middleware

Server Hardware

Hypervisor

VMContainer

VM

Application

Libraries

Middleware

Application

Libraries

Middleware

Container A Container B

10

Xen PVH Containers

• VM containers good for multi-tenant cloud providers

Group containers from a tenant onto a VM

• Great infrastructure in place for guest isolation

• PVH for app containers

Boot to guest kernel in protected mode

PV performance for disk and network

Hardware virtualized performance for CPU and memory

• Why PVH (vs. HVM)

No dependence on QEMU

No BIOS

Faster Boot time

11

Xen Containers with Docker

Server Hardware

Hypervisor

User

Xen-blkback

Xen-netback

Dom0

Kernel Docker storage devices

Docker client

Docker Daemon

PVH DomU

Kernel

User

Xen-blkfront

Xen-netfront

Container Root device

Init

Application

vNIC

PVH DomU

12

Xen Containers with Docker – Guest Anatomy

Minimal Kernel

Minimally configured kernel

Init

Init service to mount application rootfs and configure network

Storage

Docker container volume as rootfs

Networking

Docker subnet IP and docker bridge gateway

Kernel

User

Xen-blkfront

Xen-netfront

Container Root device

Init

Application

vNIC

PVH DomU

13

Xen Containers with Docker – Guest Configuration

Storage

Docker devicemapperstorage backend – container volume

Application path

Application path from docker run/exec command

Network IP

DHCP/docker subnet for interoperability with dockercontainers

Dom0Docker host

docker run ubuntu

/bin/bash

PVH

DomU

i

n

i

t

k

e

r

n

e

l

ApplicationContainer

block device

/bin/bashIP:172.17.

xx.xx

14

Agenda

Containers

Xen Containers

Numbers

Next Steps

15

Numbers

PVH HVM Comments

Domain Creation

224 184 Time spent by xl toolstack to setup domain

To drop into container shell

1380 2503 Time taken to boot the minimal kernel and drop into shell from container rootfs

Guest Memory Used – 16MB

Config:Host GuestXeon® CPU E5-2699 v3 Memory – 128MBMemory – 60GB vCPU - 1Dom0 Memory – 4GB Dom0 vCPUs – 8

16

Agenda

Containers

Xen Containers

Numbers

Next Steps

17

Next Steps

Docker Volumes

PV VirtFS for supporting docker volumes

Pods (Multiple applications in a VM)

Leverage systemd as the init service inside VM to resource control multiple applications

Q & A