Operating System Level Virtualization

Reza Farivar

Types of virtualization

• Native, full• Hardware assisted • Para-virtualization • OS level

– Containers– Jails– Chroot– Zones– Open-VZ Virtuozzo

Native and Full Virtualization• the virtual machine simulates enough hardware to

allow an unmodified "guest" OS (one designed for the same CPU) to be run in isolation.

• Examples:– VirtualBox– Virtual PC– Vmware– QEMU– Win4Lin– XEN/Virtual Iron

Hardware enabled virtualization

• the virtual machine has its own hardware and allows a guest OS to be run in isolation.

• Intel VT (IVT)• AMD virtualization (AMD-V)• Examples:

– VMware Fusion– Parallels Desktop for Mac– Parallels Workstation

Partial virtualization

• the virtual machine simulates multiple instances of much (but not all) of an underlying hardware environment, particularly address spaces.

Paravirtualization

• the virtual machine does not necessarily simulate hardware, but instead (or in addition) offers a special API that can only be used by modifying the "guest" OS.

• Terminologies– Hypervisor, hypercall– Enomalism

• Examples:– XEN, KVM, Win4Lin 9x

Operating system-level virtualization

• virtualizing a physical server at the operating system level, enabling multiple isolated and secure virtualized servers to run on a single physical server.

• Examples:– Parallels Workstation– Linux-VServer, Virtuozzo– OpenVZ, Solaris Containers– FreeBSD Jails– Chroot ?

Comparison

Thinner Containers, better performance

10

What is the Trade-Off?

11

Feature Comparison

Hypervisor Container

Multiple Kernels X

Load Arbitrary Modules X

Local Administration All

Live Migration OpenVZ

Live System Update X Zap

12

Xen 3.0 Guest VM

I/O Path•Process to Guest OS•Guest OS to IDD

Resource Control•Driver Domain

• Map Virtual Devices• CFQ for disk• HTB for network

Security Isolation•Hypervisor•Access Physical Level

• PCI Address• Virtual Memory

Resource Control•Hypervisor

• Allocate Resources• Schedule VMs

Schedules All VMs•Guest VM & IDD Scheduled•Two levels scheduling in Guest

13

VServer 2.0 Guest VM

Security Isolation•Access to Logical Objects

• Context ID Filter• User IDs• SHM & IPC address• File system Barriers

Resource Control•Map Container to

• HTB for Network• CFQ for Disk

•Logical Limits• Processes• Open FD• Memory Locks

Optimizations•File-level Copy-on-write

I/O Path•Process to COS

Scheduler•Single Level•Token Bucket Filterpreserves O(1) scheduler

14

VServer Implementation

• 8,700 lines across 350+ files– Leverage existing implementations– Applied to Logical Resources

• Not architecture specific– MIPS, ARM, SPARC, etc..– Low Overhead

15

Guest Comparison

Xen 3.0 VServer 2.0

Level of Virtualization Physical Logical

Resource Control HTB, CFQ, etc HTB, CFQ, etc

Scheduler 2-levels: Hyp + Guest 1-level

I/O Path 3 transfers 2 transfer

16

Configuration

Kernel Linux VServer 2.0 Xen 3.0.4

Version 2.6.16.33

Distribution Fedora Core 5

File system Independent LVM Partitions

Scheduler O(1) O(1)+TBF Credit

Machine HP DL360 G4p

CPU 2 x 1 core Xeon with 2MB L2

Network 2 Port GbE

Memory 4 GB

Hardware

System Software

17

Disk I/O: Write

18

CPU & Memory Performance

19

Performance at Scale - UP

Solaris Zones

Solaris Zones

Types of Zones

Zone Daemons

Zone File Systems

Zones Networking

Zones Resource Capping

OpenVZ

• Requires both the host and guest OS to be Linux

• Claim: There is only a 1–3% performance penalty for OpenVZ as compared to using a standalone server

• The basis of Virtuozzo Containers– By Parallels– Claim: 300% more virtual servers than hypervisor

solutions

OpenVZ: Less overhead

OpenVZ: LAMP Throuput

OpenVZ: LAMP Response Time

OpenVZ Components

OpenVZ Kernel Virtualization

OpenVZ Kernel Resource management

OpenVZ Checkpointing

OpenVZ Control Interface

LXC Linux Containers

• Better isolation as compared to a chroot (chroot jail).• Low overhead. LXC uses minimal resources in terms of RAM

and hard drive space without the overhead of installing a guest OS in a virtual machine ( VMWare / VirtualBox / KVM ).

• Applications and services (servers) run at native speed.• There is support for Linux containers in libvirt .• Linux containers work well with btrfs .• No special hardware is required, runs on 32 and 64 bit processors.• Linux containers are Open source.• Unlike XEN or OpenVZ , no patch is required to the kernel.

What are LXC Linux Containers?

• Create control group filesystem as the root user.# mkdir -p /cgroup# mount none -t cgroup /cgroup

Add to fstab so it will be there after a reboot.# vi /etc/fstab and add line belownone /cgroup cgroup defaults 0 0

• Applications to install.# apt-get install libvirt-bin debootstrap

• We need to download and install this pkg.lxc_0.7.2-1_amd64.deb# dpkg -i lxc_0.7.2-1_amd64.deb

LXC Linux Containers on Ubuntu with CentOs Clients

• # Create a container# lxc-create -f /path/to/{CONTAINER_NAME}/main/config/file -n {CONTAINER_NAME}# lxc-create -f /etc/lxc/lxc-centos.conf -n centos

Create a rootfs directory# mkdir /var/lib/lxc/centos/rootfs

• Copy files you worked on earlier.# cp -r /home/yourhomeidirectory/LXC/Centos5-x86_64/* /var/lib/lxc/{CONTAINER_NAME}/rootfs# cp -r /home/yourhomeidirectory/LXC/Centos5-x86_64/* /var/lib/lxc/centos/rootfs

• Create this file for creating and setup default devices for your containers.# vi  /usr/local/bin/lxc-config and add information below, also only needs to be run once for every new container you create.

Step1: Create a container

• # cd /var/lib/lxc/CONTAINER_NAME}/rootfs/# cd /var/lib/lxc/centos/rootfs/# /usr/local/bin/lxc-config # fix /dev

• Chrooting into the rootfs file system for centos# chroot /var/lib/lxc/centos/rootfs/

Setup root password for your container.# passwd root

• Start or Remove some services from running# chkconfig sshd on# chkconfig httpd on# chkconfig xinetd off# chkconfig sendmail off

•When finish type exit to finish chroot session.

Step2: Chroot session

• Destroy container and files# lxc-destroy -n {CONTAINER_NAME}# lxc-destroy -n centos

Start container# lxc-start -n {CONTAINER_NAME} init# lxc-start -n centos init

• Stop container# lxc-stop -n {CONTAINER_NAME}# lxc-stop -n centos

• Connect to tty console# lxc-console -n centos -t 1 doesn’t seem to work for me.

• Check your configuration# lxc-checkconfig

• List containers# lxc-lsIf you receive this error lxc-ls warning, got bogus unix line, reinstall uml-utiilities# sudo apt-get –reinstall install uml-utilities

• Is it RUNNING or STOPPED# lxc-info -n {CONTAINER_NAME}# lxc-info -n centos

Step3: Delete container and files