Containers security

Kernel internals

“There may be ways ... for an application to escape out of its container or deny service to the

host or other containers.” – Mark Russinovich, CTO Microsoft Azure https://azure.microsoft.com/en-us/blog/containers-docker-windows-and-trends/

“For Google I would say that security is probably the number one priority, for KVM it is the killer

feature otherwise we could just sell people Docker containers or just let them run on Linux

processors. So the main thing that VMs actual provide it that isolation and all our VM’s are on

KVM.” - Andrew Honig, tech lead on the Cloud Security Team at Google https://youtu.be/L7ScFlkJEO8?t=33

“The inter-process isolation provided by a monolithic kernel such as Windows or Linux could

never be compared to the inter-VM isolation offered even by the most lousy hypervisors. This is

simply because the sizes of the interfaces exposed to untrusted entities (processes in case of a

monolithic kernel; VMs in case of a hypervisor) are just incomparable. ”

“ Sadly … we have finally came to the conclusion that consumer Windows OS, with all those

one-would-think sophisticated security mechanisms, is just not usable for any real-world domain

isolation. ” - Joanna Rutkowska – Security researcher & architect of Qubes OS http://blog.invisiblethings.org/2014/01/15/shattering-myths-of-windows-security.html

“Some people make the mistake of thinking of containers as a better and faster way of

running virtual machines. From a security point of view, containers are much weaker.” – Dan

Walsh, SELinux architect (?)

“There’s contentions all over the place that containers are not as secure as hypervisors. This is

not actually true. Parallels and Virtuozo, we’ve been running secure containers for at least 10

years.” – James Bottomley, Linux Maintainer and Parallels CTO

“Virtual Machines might be more secure today, but containers are definitely catching up. –

Jerome Petazzoni, Senior Software Engineer at Docker

“You are absolutely deluded, if not stupid, if you think that a worldwide collection of software

engineers who can’t write a operating system or application without security holes, can then

turn around and suddenly write virtualization layers without security holes” Theo de Raadt,

OpenBSD project lead

https://fosdem.org/2015/schedule/event/zombieapocalypse/

Agenda

• Not about Docker security

• Entropy

• History of Kernel Security

• Conclusion

Bart Smith

• Stadjer

• Windows NT 3.1

• Design & security

• Migrating to Cloud Native

Why is Docker so popular?

1. instant startup

2. namespace isolation & resource governance

3. small memory footprint

4. common toolset

5. packaging - Open Container Initiative OCI

6. ease deployment - DockerHub

More security see talk Adrian 4/6/15 https://youtu.be/04LOuMgNj9U

Fortress

• Few doors and windows

• Easy blocking

• Defense in Depth, multilayer

Entropy Peter Sewell, Cambridge @31C3

http://media.ccc.de/browse/congress/2014/31c3_-_6574_-_en_-_saal_1_-_201412301245_-_why_are_computers_so_and_what_can_we_do_about_it_-_peter_sewell.html

SPI - stack

• SAAS

• PAAS

• IAAS

HW

OS OS OS

App

VIRT

App App App App App

Virt HW Virt HW Virt HW

HW

OS OS OS

App

VIRT

App App App

Virt HW Virt HW Virt HW

IAAS with HW virt

•AWS •Azure Infra •Google Com-pute Engine •Joyent

HWVIRT

Virt HW Virt HW Virt HW

OS OS OS

http://bit.ly/2015-cloud-mq (try update year in link when expired)

( )

App App

db web file etcmid.ware

App1

db web file etcmid.ware

App2 App3

PAAS

•EC3

•Azure App Service

•Google App Engine db web file etcmid.waredb web file etcmid.ware

App1 App2 App3

db web file etcmid.ware

App1 App2 App3

Jérôme Petazzoni explaining:

• The only difference between a-process-in-a-container and a-process-not-in-a-container is a few labels on top on a process that say this is in container X

• A context-switch between two containers is exactly the same as a context-switch between two processes

https://youtu.be/pUQ5ukrVaH4?t=600 https://youtu.be/pUQ5ukrVaH4?t=667

IAAS with OSvirt /Zones/Containers

HW

OS

ContainerVirt OS

AppLib

Lib

ContainerVirt OS

AppLib

Lib

ContainerVirt OS

AppLib

Lib

ContainerVirt OS

AppLib

Lib

Lib

Lib

HW

OS

ContainerVirt OS

AppLib

Lib

ContainerVirt OS

AppLib

Lib

? ?

MAAS

•Ubuntu •Softlayer/IBM •Leaseweb

HW

DEV Performance Security

PAAS

Containers

IAAS

Hypervisor

App

HW

OSVirtHW

AppOS

VirtHW

Kernel

Container

App

HW

db

Code1

web

2

?

https://en.wikipedia.org/wiki/Operating-system-level_virtualization#Implementations

Docker < v0.9

Kernel

LXC

App

HW

Lib

Lib

Docker

Docker v0.9 and up

DOCKER_OPTS="-e lxc" During install, libcontainer : Setting up lxc-docker-1.x.0

https://blog.docker.com/2014/03/docker-0-9-introducing-execution-drivers-and-libcontainer/ http://blog.docker.com/2015/06/runc/

Kernel

Lib-container

App

HW

Lib

Lib

Docker

Kernel

LXC

App

HW

Docker

Kernel

runC

App

HW

Docker

Announced june15: runC replaces Libcontainer

Kernel

App

HW

Lib

Lib

libCSystem Calls

GO: nolibc

GO does system calls manually, without relying on libc or anything else - Aram Hăvărnanu https://archive.fosdem.org/2014/schedule/event/porting_go_to_new_platforms/ https://youtu.be/tnXOeHRuyyA?t=1322

User (ring3)

Kernel (ring0)

KernelHW

Lib

LibSystem Calls

GOapp

Building Docker Images for Static Go Binaries

Statically Linked, with syscall 'package'

https://medium.com/@kelseyhightower/optimizing-docker-images-for-static-binaries-b5696e26eb07

FROM scratch MAINTAINER Kelsey Hightower <kelsey.hightower@gmail.com> ADD contributors contributors ENV PORT 80 EXPOSE 80 ENTRYPOINT ["/contributors"]

Total size of image: 6MB

Triton

• LX: run Linux on Solaris

• Docker on Illumos

• Joyent

SolarisKernel

AppLib

Lib

libCLinux Syscalls

Container

Solaris Syscalls

https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds http://us-east.manta.joyent.com/jmc/public/opensolaris/ARChive/PSARC/2002/174/zones-design.spec.opensolaris.pdf

http://www.crn.com/slide-shows/cloud/300076877/heres-who-made-gartners-2015-cloud-iaas-magic-quadrant.htm/pgno/0/19

Mirage OS - Cambridge

• unikernel

• Stat. linked kernel

• No Firewall needed

• defense: limit interfaces (including Xen)

• 20ms startup http://media.ccc.de/browse/congress/2014/31c3_-_6443_-_en_-_saal_2_-_201412271245_-_trustworthy_secure_modular_operating_system_engineering_-_hannes_-_david_kaloper.html

Some kernel

HW

Lib

LibOCaml

Xen Hypervisor

Dom0

Qubes - Joanna Rutkowska

• with a GUI

• multilayer defense

https://www.qubes-os.org/

Microsoft

• OneCore

– 64bit only

– refactoring

– base for Win10, Server, Phone & Nano server

• Containers

Docker support https://channel9.msdn.com/Events/Build/2015/2-704 https://channel9.msdn.com/Events/Build/2015/2-683 https://azure.microsoft.com/en-us/blog/containers-docker-windows-and-trends/

Microsoft Containers Server Core Nano Server

Born in the cloud applications Traditional Applications

Highly Compatible Highly Optimized

Microsoft’s Container Runtimes Windows Server Container

HIGHLY

AUTOMATED EFFICIENT

SCALABLE

AND ELASTIC

Hyper-V Container

HIGHLY

AUTOMATED EFFICIENT

SCALABLE

AND ELASTIC

PUBLIC

MULTI-

TEANCY

SHARED

HOSTING

SECURE

SECURE

HOSTING

TRUSTED

MULTI-TENANCY

REGULATED

WORKLOADS

Nano Server: reverse forwarders

• Additional packages

– WoW64 for backward compatibility

– Hyper-V host

– Replicated File services

https://channel9.msdn.com/Events/Ignite/2015/BRK2461

What runs today with the Reverse Forwarders? • Chef

• PHP • Nginx • Python 3.5 • Node.js • GO • Redis • MySQL • OpenSSL • Java (OpenJDK) • Ruby (2.1.5) • SQLite

Intel: Clear Linux

• 1000 VM/host

• 200ms startup

• Intel VT

http://www.theregister.co.uk/2015/05/21/intel_wants_containers_to_be_alone_together_naturally/ http://www.infoworld.com/article/2925038/linux/intel-takes-on-coreos-with-its-own-container-based-linux.html http://lwn.net/Articles/644675/ https://www.clearlinux.org

VMware

• Photon Linux distribution

• Open Source

• Management door mesos, Hadoop, Openstack, Pivotal CF (Lattice), CoreOs, Kubernetes, etc

Micro-visor

Hardware

Photon

docker-machine

Photon

App LIB

Photon

App LIB

• Photon platform

Gartner IAAS MQ 2015

Gartner also recommends cloud buyers adopt a bimodal strategy that allows them to maintain critical IT operations while innovating on agile development platforms.

http://bit.ly/2015-cloud-mq (try update year in link when expired)

Conclusion

• ARM simpler Virtualization

• Converge Containers & VM

Questions?

Docker training/conferenties

http://dutchdockerday.nl 20 Nov 15, €99 (early bird) Amsterdam

https://skillsmatter.com/conferences/7208-containersched-2015 London

http://softwarecircus.eu Okt 2016 €150 (early bird) Amsterdam

http://nkhare.github.io/data_and_network_containers/ self training

Link Q&A • side-channel attack processor cache

– http://wp.me/p26mzH-c5

– http://reg.cx/2f6r