{andrew.warfield}@cl.cam.ac.uk Using Virtual Machines To Build Virtual Storage For Virtual Machines...

{andrew.warfield}@cl.cam.ac.uk

Using Virtual MachinesTo Build Virtual StorageFor Virtual Machines

(at the block level)

Andrew Warfield

VMs as a Building Block for Storage

Device VMs and narrow interfaces let us treat devices as isolated services in a cluster.Externally administered / fate sharing / invisible.

Block interface is narrowEffectively stateless, OS Agnostic.

Develop in user space.Faster/safer development.Easier access to “high-level” primitives. (like “files” and

“sockets”).

Userspace isn’t unreasonably slow.

Userspace isn’t unreasonably slow.

Three Examples

1. How to practically manage lots of VM images in an existing cluster?• User-level Block-Back

2. How to support applications of VM checkpointing (eg fast frequent snapshots)? • Parallax-local

3. How to scale to support large, saturated VM clusters?• Parallax

User-level Block Backend

Problem: Loopback sucks.Page dirty vs. OOM-killer deathmatch.

Images-as-files is an attractive way to manage VM storage.

Blktap + XenStore make this possible Status: Returning to this in Oct/Nov.

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-                   -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CPxen0             2G 31198  95 56818   8 20967   2 28415  77 59595   4 264.9   0xenU-blkbe2cpuGT 2G 31157  96 54026  10 25585   4 30664  90 64919   7 292.7   0xenU-blktp2cpuGT 2G 32313  97 54217   8 20950   3 28117  87 65924   4 191.8   0

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-                   -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CPxen0             2G 31198  95 56818   8 20967   2 28415  77 59595   4 264.9   0xenU-blkbe2cpuGT 2G 31157  96 54026  10 25585   4 30664  90 64919   7 292.7   0xenU-blktp2cpuGT 2G 32313  97 54217   8 20950   3 28117  87 65924   4 191.8   0

Local VM-specific Storage

Problem: Want fast, efficient management of images and snapshots for VMs.

Write Sharing Considered Harmful. Use Radix-trees to virtualize storage addressing. Tree node is (address, writable, child-CRC). Snapshot is one-block read and write.

Status: Working.

Parallax Architecture

Block Access in Clusters

Problem(1): Want to use local disks for storage.

(or even if you don’t…)

P2: Your storage servers are about to be hosed. Fate-sharing cache aggregates VM traffic. Extent-based allocator builds n-way virtual

storage chunks from local disks.

Status: Work in progress with James Bulpin. (target: OSDI 2006)

Conclusion

Devices-as-services is a good approach to managing storage in a VM cluster.

Several (hopefully) useful examples. Bigger picture (in terms of my thesis) includes

other devices (a.k.a network).