Embed Size (px)
Citation preview
Production FS:
Adapt or Die
hello!
Claudia & Tiago
callisto13 tscolari
Summary
✘ Glossary - What is a Root Filesystem?✘ What is CloudFoundry?✘ Warden✘ Garden Linux✘ Garden runC✘ GrootFS + Garden runC✘ The Future✘ Questions?
1) Directory hierarchy and structure
1) Type: How the data is organised
What is a FileSystem?
✘ Top of the dir structure✘ Mount point for other FSes at boot✘ Contains all critical startup files✘ Sets state of system✘ Has tools for recovery of broken system and data
What is a Root FileSystem?
What is a RootFS?
Host
What is a RootFS?
Container
Host
✘ Open Source Platform as a Service✘ Development started in 2009 (VMWare)✘ First released in 2011✘ Run application inside containers✘ Supports buildpacks and Docker images
What is CloudFoundry?
Provide framework and runtime support for applicationsOfficially supported:
○ Binary○ Go○ Java○ .Net Core○ Node.js
Buildpacks?
○ PHP○ Python○ Ruby○ Static File
Buildpack
Buildpack
RootFS
Application Code
RootFS
Droplet
Compilation
Droplet
✘ Multitenancy
✘ Application Quotas
Isolation
✘ Independent of Host FS
What did CF need?
2011
✘ Kernel 2.6✘ Containers?✘ No user namespace available
1.
AUFS + WARDEN
2011
Warden
✘ Developed in Ruby and C
✘ Initially with LXC
✘ Coupled to Linux
Namespaces (exc User) & cgroups
WHY AUFS?
✘ Mounting the rootfs was faster than copying it
✘ No duplicated files
But...
✘ No support for quotas
✘ Not in Mainline Kernel
RootFS Mnt: RootFS
RW Layer
Droplet
Mnt: RootFS
RW Layer
Droplet
Mnt: RootFS
RW Layer
Droplet
...
root root root root
unique uid
APP1 APP2 APP3
2013-2014
✘ User namespaces✘ Security✘ Scalability✘ Containers
2.
BTRFS + GARDEN LINUX
2014/2015
Garden & Diego
✘ Replacement for Warden
✘ Go (w)arden - Garden
✘ Platform Agnostic API - future support for windows
✘ New scheduler
✘ DEA(Go) - Diego
+ Docker Images
✘ No control over RootFS anymore
✘ Security Risks ✘ User namespaces
Why BTRFS?
✘ Dependence on Docker graph driver
✘ Built in support for quotas
✘ The other options were:Overlay - not matureDeviceMapper - required LVMZFS - proprietary
Everything was changing
✘ New scheduler
✘ New container runtime
✘ New container Filesystem
✘ New IAAS
And...
✘ Huge Performance Hit: BTRFS blamed (eventually)
✘ Theory was BTRFS garbage collection was consuming all IOPS from the cells
✘ BTRFS new and didn’t have enough support at the time
3.
AUFS + GARDEN LINUX
October 2015
Why AUFS, again?
✘ Familiarity
✘ But…Quotas?
RootFS Mnt: RootFS
mnt:/dev/loop1
Droplet
...
root root
APP1
Mnt: RootFS
mnt:/dev/loop2
Droplet
root
APP2
Mnt: RootFS
mnt:/dev/loop3
Droplet
root
APP3
sparse
2015
✘ runC
4.
AUFS + GARDEN RunC
May 2016
Open Containers Initiative / RunC
✘ Open Standard for containers specification
✘ Implementation of OCI container specs
Garden RUNC
✘ Rewritten to use RunC
✘ More GO, Less C
✘ More Security
AUFS...
✘ More bugs
✘ Distraction
2015/2016
✘ OCI: Image-Spec✘ GrootFS - new project to replace Garden Linux backend
Dedicated team✘ Security: Garden runC Rootless
5.
BTRFS + GROOTFS +
GARDEN runC
June 2016
Why BTRFS, again?
✘ Snapshotting: plays well with container images
✘ Could be (almost) rootless
✘ Quotas
✘ Previous issues fixed in kernel 4.4
✘ Big companies investing
✘ Support from Canonical
Container Image
Snapshot: rootfs
Droplet
1001
Buildpack App
Snapshot: layer1
1001
Docker Img App
Snapshot: layer2
Snapshot: layer3
But...
✘ New performance issue: `btrfs enable quota`
OVERLAY/XFS +
GROOTFS + GARDEN
runC
6.
February 2017
Why Overlay+XFS?
✘ Maturity ✘ Overlay for layering ✘ XFS for quotas
XFS folder app1/
Overlay mount
upperdir: app1/diff
lowerdirs: layer1:layer2:layer3
App 1
XFS folder app2/
Overlay mount
upperdir: app2/diff
lowerdirs: layer1:layer2:layer3
App 2
The Future
What’s Next?
✘ EXT4Kernel 4.5 in StemcellMatch host FS
✘ ShiftFSOn the fly user mappingsNo translation layer
Conclusion
✘ Nothing is forever
✘ There are always risksAgility is key
✘ Focus!
thanks!
Any questions?
callisto13 / [email protected] / [email protected]
Slide template by SlidesCarnival
