Embed Size (px)
DESCRIPTION
Adding multitenant storage capability to the illumos kernel
Citation preview
Adding Multitenant Storage
Capability to the illumos Kernel
Richard Elling and Garrett D’Amore
Presented at the SCALE 11x ConferenceFebruary 24 2012
• Why multitenancy?
• Survey of available technologies
• OS virtualization
• Multitenant storage services in illumos
Agenda
Presented at the SCALE 11x ConferenceFebruary 24 2012
Why Multitenancy?
Presented at the SCALE 11x ConferenceFebruary 24 2012
Kernel
Userland Process
Device Driver
Userland Process
Simple model: multitasking OS
Presented at the SCALE 11x ConferenceFebruary 24 2012
Strike up the band!
Presented at the SCALE 11x ConferenceFebruary 24 2012
Guest
Native Hypervisor Model
Kernel
Userland Process
Hypervisor Device Driver
Examples: VMware ESX, XenServer, KVM
Device Driver
Presented at the SCALE 11x ConferenceFebruary 24 2012
Java
Java Virtual Machines are Virtual Machines, too
JVM
Java Application
OS Device Driver
APIs
Presented at the SCALE 11x ConferenceFebruary 24 2012
Hosted Hypervisor Model
UserlandProcess
Examples: VMware Workstation, VirtualBox
Guest
Kernel
Userland Process
Hypervisor Device Driver
Device Driver
Kernel Device Driver
Presented at the SCALE 11x ConferenceFebruary 24 2012
HVMGuestOS
HostOS
QEMU
Device Driver
Userland Process
Device Driver
KVM
I/O Proxy
Presented at the SCALE 11x ConferenceFebruary 24 2012
DomU
Front-end driver
Device Driver
Userland Process
XenServer
Dom0
Back-end driver
Device Driver
Userland I/O Proxy
Presented at the SCALE 11x ConferenceFebruary 24 2012
Kernel
Userland Process
Device Driver
Userland Process
Return to simple model: multitasking OS
Presented at the SCALE 11x ConferenceFebruary 24 2012
Zone
KernelDevice Driver
Userland Process
Zone
Userland Process
Presented at the SCALE 11x ConferenceFebruary 24 2012
BSD Kernel
Jail
BSD Jails
Storage Device Drivers
IP AddressDirectorystructure
File system
Userland Process
NIC Driver
Presented at the SCALE 11x ConferenceFebruary 24 2012
Kernel
Zone
illumos OS
Storage Device Drivers NIC Driver
Classification Engine
TCP/IP
Virtual NICFile system
Userland Process
Presented at the SCALE 11x ConferenceFebruary 24 2012
The first thing we do, let’s kill all the lawyers
King Henry VIAct IV, Scene II
Presented at the SCALE 11x ConferenceFebruary 24 2012
The first thing we do, let’s kill all the lawyers
King Henry VIAct IV, Scene II
The first thing we do, let’s kill all the layers
UNIX Performance Gurus
Presented at the SCALE 11x ConferenceFebruary 24 2012
Zone
KernelDevice Driver
Userland Process
Zone
Userland Process
Presented at the SCALE 11x ConferenceFebruary 24 2012
Kernel
Storage Device Drivers NIC Driver
VirtualSwitch
Zone A
Zone B
TCP/IP TCP/IP
File systemBlock device Virtual NIC Virtual NIC
Transactional Object Store and Data Management Unit
NFS
File system
SMB
File system
NFSiSCSI
Presented at the SCALE 11x ConferenceFebruary 24 2012
Global state Per-zone state
NFS - rpc now zoned
SMB - in-kernel, multithreaded, zoned
iSCSI - per-zone block devices
➾
Presented at the SCALE 11x ConferenceFebruary 24 2012
Big shared caches Inefficient small caches
~ 100 MB/tenant > 1000 MB/tenant
Kernel
Zone
Adaptive Replacement Cache
NFS
Zone
ZoneiSCSI
NFS
SMBTraditionalHypervisor
VMNFS
kernel
VM
kernel
Samba
FS Cache
FS Cache
Presented at the SCALE 11x ConferenceFebruary 24 2012
Kernel
Deep access to devices
Storage Device Drivers NIC Driver
VirtualSwitch
Zone A
Zone B
TCP/IP TCP/IP
Awesome New
DeviceBlock device
Virtual NIC Virtual NIC
Transactional Object Store and Data Management Unit
New interface?
ANP
Block device
iSCSIiSCSI
Presented at the SCALE 11x ConferenceFebruary 24 2012
Zone
IP Multipath Group
Virtual Networking
TCP
NFS iSCSI
Zone
IP Multipath Group
TCP
NFS SMB
Physical NIC
Virtual Switch
Physical NIC
Virtual Switch
Virtual NICVirtual NIC Virtual NICVirtual NIC
Presented at the SCALE 11x ConferenceFebruary 24 2012
Kernel statistics kstats✦
Zone A
Resource capsNetwork usageI/O usage
Zone B
Resource capsNetwork usageI/O usage
All kstats
Presented at the SCALE 11x ConferenceFebruary 24 2012
All in due time?
Presented at the SCALE 11x ConferenceFebruary 24 2012
Tree AB
Domain B
Domain A
Domain C
Tenant A
Tenant B
Tenant C
SMB
SMB
NFS SMB✦
A
B
C
Presented at the SCALE 11x ConferenceFebruary 24 2012
• Efficient, multitenant storage in illumos
• Sit out of the traditional VM dance
• OS virtualization rulez!
Summary
Presented at the SCALE 11x ConferenceFebruary 24 2012
www.illumos.org
www.dey-sys.com
www.RichardElling.com
