From Sandbox to Playground: Virtual Environments and Quality of Service in the Grids Kate Keahey [email protected] Argonne National Laboratory

From Sandbox to Playground:

Virtual Environments and Quality of Service in the

Grids

Kate [email protected]

Argonne National Laboratory

IEEE Fox Valley Subsection Kate Keahey

What are Grids?

Power Grid analogy: remote computing power can be accessible from your laptop in much the same way as electricity is delivered to your home


How do Grids work today? From Grid Services…

Discover remote resources Start jobs on remote resources Authenticate and authorize users and other entities Transfer data Globus Toolkit: a de facto standard in Grid

technologies … to commodity Grid usage

Combining and automating the use of basic Grid services

Utility computing paradigm Provide reliable, adaptive, QoS-based execution of

specific tasks


Grid Capabilities TodayAccess Grid: enable people, presentation software and programs to interact

Data Grid: combine distributed dataand access storage within one similation

Computational Grid: use access to powerful resources to run a genome analysis service

Experiment Grid: combine multiple technologies in support of an experiment


First, a little bit of theory… “Resource sharing & coordinated problem

solving in dynamic, multi-institutional virtual organizations”

Single sign-on creates a decentralized Grid domain within which Grid entities can act on behalf of the user


…now practice

Grid clients• create environments

More resources per user, more users per resource, more environments per resource….How do we scale?

Application developers• obtain the right configuration

Grid administrators• resolve conflicts!


What do we need? Quality of Service

Sandbox a Grid computation Reliable enforcement of policy-driven resource usage Applications in experimental sciences, fair resource

sharing, etc. Quality of Life

Dynamic capabilities Dynamically creating and managing remote execution

environments Dynamically configuring such environments Easily moving in a Grid as resources come and go

Automated, hands-off Grid infrastructure


How do we satisfy those needs?

Dynamic behavior: missing services Creating and managing remote execution

environments dynamically Customizing such environments

Ontologies and protocols Standardized descriptions can be processed

dynamically by various Grid entities Policy-adapting protocols will equip the system with

ability to respond to needs automatically Tools to obtain descriptions of these things easily

and automatically


Virtual Workspaces

Virtual resource configuration

Protection and enforcement environment

Software and file configuration state

Execution state

Virt

ual W

orks

pace

Grid

Mid

dle

wa

re

Inte

rfa

ceG

rid c

lien

t In

terf

ace

Grid clients

Grid middlewareinterface

VWs are represented by an ontology description Potentially integrating community policy

They can be implemented using different technologies They can be customized to the user needs and deployed in the

Grid


Properties of Virtual Workspaces Dynamic creation

Inherent property of Grid computation Avoiding a maintenance nightmare (automate administration) and

potential security hazard Dynamic configuration

To reflect changing policies in the Grid (implement agreements) Strong protection environment

Otherwise users won’t trust sites and sites won’t trust users Fine-grain enforcement Configurable architecture, software, environment

Application software/libraries/licenses Configurable environment Running 32-bit programs on 64-bit architectures Running a required version of the OS (Fedora vs. RH9) Potentially even execution state


VWs in the Grids

Clie

nt

request

VW EPR

inspect and manage

deploy & suspend

use existing VW Create VW

VW Factory

VW Repository

VW Manager

create new VW

ResourceVW

start program


VW Services Factory

Creates VW in terms of the implementation e.g., VM image, pacman chache+

May create based on an already created VW Writes/configures access policy May allow negotiation

VW Repository Access to state describing a VW Allows inspection, management, implementation-specific

termination, potentially renegotiation, etc. Soft-state lifetime management ensures termination

VW Manager Lightweight infrastructure deploying VMs


How can I obtain VWs? Through automatic negotiation and establishing

agreements with the community What is an agreement?

Relationship between parties dynamically-established and dynamically-managed

Terms Functional, e.g., a service I can perform Non-functional, e.g., performance, availability, etc.

Noteworthy Agreement Properties Simple, decentralized way of expressing aggregate or

proprietary policies in the system Allow providers to gauge demand Ephemeral, periodic, fine-grained, modifiable policy

WS-Agreement, GRAAP-WG, Global Grid Forum Currently under public comment: see www.gridforum.org


Implementing and Configuring Virtual Workspaces

Potential Implementation Groups Unix accounts and Unix account tools

setrlimit, DSRT, chroot, chown, and others Sandboxes

VServer, protection and fine-grain enforcement Virtual Machines

VMware, Xen, and others

Deployment & configuration tools Pacman & pacman cache

See also: Grid 2004: “From Sandbox to Playground:

Dynamic Virtual Environments in the Grid”


Comparing VW Implementations Unix accounts

Pros: efficient, ubiquitous Cons: very limited functionality Needs to be used in conjunction with other technologies

Pacman, additional system enforcement tools

Prototype available (GT 3.2) http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm

Currently on the way to become a GT4 service Sandboxes

Pros: efficient, fine-grain enforcement, typically very lightweight

Cons: limited state enforcement Need to be used in conjunction with other technologies

http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm


Comparing VW Technologies (cntd)

Virtual Machines Pros:

Flexibility (run linux on linux, 32 on 64-bit, etc.) Enhanced security, audit forensics, etc. Great user state management Freezing/migration Customized environment A promising distribution/deployment tool

Cons: Potential for being less efficient (emulation) Potential for resource overhead Poor implementation of sharing, relatively little enforcement (but can

be combined with other technologies for enforcement) Maturity issues

The potential is excellent, but needs more work


The Need for Speed

0

0.2

0.4

0.6

0.8

1

1.2

110100jt 110105jt 110109jt

UNIX acctVserverVMware

Comparison using the Fusion EFIT application


Other efficiency concerns

Startup time

Resource usage overhead Memory use: VMware: 24MB + 1 MB per 32 MB

memory allocated Disk use: VMware large

Table 1: DVE create/destroy times

Linux VServer VMware

Create 100 ms 360 ms 14-52 sec

Destroy 70 ms 200 ms 3-38 sec


Enforcement Capabilities

Unix account VServer VMware

CPU usage (sec) Via setrlimit() Not at present, but could be added

Not enforced

CPU usage (%) Not enforced Limited: no VServer can starve another

Not in VMware Workstation

Disk space usage Dynamically(per-user

quotas)

Dynamically (per context quotas)

Statically (virtual disks)

Memory usage No Not at present, but could be added

Statically

Network usage No Dynamically Dynamically


Virtual is the New Real! Virtual machines are a very interesting option for

the Grid Excellent usability potential:

Configurability, enhanced security, state management, replication, enforcement…

Even potential as distribution tool!

Excellent potential for optimizations Performance, resource usage, access to specialized

hardware, etc. are not so bad, especially with new technologies like Xen

Some maturity issues Do benefits outweigh challenges?


How does it work in practice? Recent project: combining VMs and Grids to create a

platform for bioinformatics applications Some of the conclusions:

Use of virtual machines can significantly broaden the resource base

Saves installation time EMBOSS installation: ~45 minutes Deploying a 2GB VM image: ~6.5 minutes Peace of mind: priceless!

Enforcement capabilities Depend on the implementation but are generally better than what we

have now

SC04 poster: “Quality of Life in the Grids: VMs meet Bioinformatics

Applications”, T. Freeman and D. Galron


How can VWs change the Grid? Challenges to the established Grid model

Security challenges Networking challenges And many, many others…

Issue of responsibility Who vets a workspace? Who is responsible for its “good behavior”?

The role of VOs is going to grow VO might take on additional responsibilities

Administers and maintains VMs, certification authority, could potentially stop suspect VMs, is to blame if something happens…

Should the VO be a legal entity? Do VOs have the resources to do that?

Are VOs going to become too heavyweight? What are the trade-offs and a healthy balance?


Virtual Playgrounds Define a virtual Grid in terms of requirements

Virtual workspaces Potentially networking requirements Other services

Provide mechanisms to create a Grid Provide services for the deployment of such “virtual playgrounds”

on real resources Ephemeric Grids built for a special purpose:

Family is getting together to decide when to spend Xmas Scientists getting up a Grid for the purposes of a specific

experiment run A game tournament A scientific simulation that gets discarded or interrrupted but can

potentially be restored later Towards a true utility computing model


Conclusions Addressing QoS and QoL is critical for the utility

computing model of Grids Unglamorous but necessary tasks Combating complexity, improving scalability Without it flexibly moving between resources on the

Grid is very hard Current technological advances make this model

ever closer to reality A breakthrough is required in terms of usability Virtual machines fit the bill

Virtual is the new Real! To find out more: www.mcs.anl.gov/~keahey

Documents

From Sandbox to Playground: Virtual Environments and Quality of Service in the Grids Kate Keahey [email protected] Argonne National Laboratory