Grid Based VM Provisioning - A Presentation by Arunabh Das

8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das

http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 1/36

Grid-based VM-provisioning UsingSTAR / Nimbus

Date : 2 Apr, 2009Presented by : Arunabh DasSchool of Computer ScienceSources:http://workspace.globus.orghttp://www.gridvm.org

Other Sources :Enabling Cost-Effective Resource Leases with Virtual Machines, Sotomayor, B., K. Keahey, I. Foster, T. Freeman. HPDC 2007 Hot Topics session,

Monterey Bay, CA. June 2007 (pdf)

Virtual Workspaces for Scientific Applications, Keahey, K., T. Freeman, J. Lauret, D. Olson. SciDAC 2007 Conference, Boston, MA. June 2007 (pdf)

http://workspace.globus.org/

http://www.gridvm.org/

http://www.gridvm.org/

http://workspace.globus.org/





What do we seek?What are we looking to find?

Ginormous amounts of compute powerAvailable 24x7x365Humongous amounts of storage

Also available 24x7x365The ability to access the above from the cupboards

that professors and post-doc fellows and the millionsof starving graduate students in North America,Europe, Asia and Africa live in





So we still haven't found what we'relooking for?

Kate Keahey is a scientist at Argonne NationalLaboratory and a Computation Institute fellow atUniversity of Chicago

She created and leads the Nimbus ProjectShe calls it Infrastructure as a Service (IaaS)Which makes perfect sense!!



A Brief History of Nimbus

Research onagreement-based

services

Xen released

First WSRFWorkspace Service

releaseEC2 gatewayavailable Support for EC2 interfaces

2003 20092006

EC2 goes online

First STARproductionrun on EC2

Nimbus Cloudcomes online

Context Broker releaseSource -

http://colab.cim3.net/file/work/Expedition_Workshop/2009_02_09_LeveragingSOA_Cy

berinfrastructure/magic-Keahey.cloudcomputing.ppt



Grid Technologies : A brief overview

Infrastructure (”middleware”) for establishing,managing and evolving multi-organizationfederations

Secure, coordinated sharingDynamic, autonmous, domain independentOn-demand, ubiquitous access to computing, data

and services

Globus Toolkit : An implementation of the most basiccapabilitiesA de facto implementation standard



A typical grid use-case

2. Finds available resources

3. Starts a remote computation

4. Transfers data from a remote location

Grid Security Infrastructure (GSI)Monitoring and Discovery Service (MDS)

Grid Resource and Allocation Manager (GRAM)Data Transfer (GridFTP)

gridmapfile

1. User logs into the Grid (single sign-on):grid-proxy-init



The case for VM on grid

Most grid applications and grid infrastructure just needs to be able to handle heavycomputation and heavy lifting of data

However, certain applications (Ex – Nuclear Physics STAR Experiment) rely heavilyon dynamically loading external libraries depending on the task to be performed

Configuring an environment for such an application is complex

Deployment on non-dedicated platform = effort consumingEven when the application compiles on a new platform, validating it is a controlledprocess subject to quality assurance and regression testing to ensure Physicsreproducibility and result uniformity

Heavy reliance of an application (ex-Physics Engine) on dependencies deeplyembedded in the environment

=> Porting application would be easiest if we could take the full software stack fromthe operating system up, and simply install that environment on remote resources



The case for VM on grid (contd)

Virtual machine provides a software-basedvirtualization of a physical host machine

Dedicated

Configured with a full software stack Once configured, deploys on a remote resource in a

matter of millisecondsResource provisioning via Vms is attractive



More benefits of VM on grid

A scientist can develop his or her application within afamiliar environment

Can port this environment between local and remote

resources as the need arisesThis facilitates provisioning resources for an

applicationThe virtual machine can be run as easily on local

resources as on remote resources or resourcesoutsourced commercially



A quick look at Virtual Workspaces(STAR)

STAR is the predecessor of Nimbus and was developed by Kate Keahey and TimFreeman at ANL

The Solenoidal Tracker at RHIC (STAR) is a detector which specializes in tracking thethousands of particles produced by each ion collision at RHIC. (Relativistic HeavyIon Collider)

STAR is a massive detector.It is used to search for signatures of the form of matter that RHIC was designed tocreate: the quark-gluon plasma.

It is also used to investigate the behavior of matter at high energy densities by makingmeasurements over a large area.

It is a proof-of-concept strategy developed for the High Energy and Nuclear Physics

(HENP) group



A Brief Look at VMA VM is a virtualization abstraction of a physical machine (hardware resources

+ software infrastructure)Software running on a host supporting VM deployment, typically called a

VMM (Virtual Machine Monitor) or Hypervisor is responsible forsupporting this abstraction by intercepting and emulating instructions issuedby the guest machine

Hypervisor provides an interface allowing client to start, pause, serialize, andshut down multiple guests

VM image is composed of a full image of a VM RAM, disk images andconfiguration files

Thus, VM can be paused, its state serialized and later resumed at a different

time and in a different location=> Decouples image preparation from deployment => Easy migration

Sources :Enabling Cost-Effective Resource Leases with Virtual Machines, Sotomayor, B., K. Keahey, I. Foster, T. Freeman. HPDC 2007 Hot Topics session,

Monterey Bay, CA. June 2007 (pdf)Virtual Workspaces for Scientific Applications, Keahey, K., T. Freeman, J. Lauret, D. Olson. SciDAC 2007 Conference, Boston, MA. June 2007 (pdf)



ParavirtualizationVirtualization technique that presents a software interface to virtual

machines that is similar but not identical to that of the underlyinghardware.

Example of paravirtualization =The virtual monitor can present the host operating system with an

intelligent NIC with support for DMA-based sending of packets, even

though the NIC on the real system lacks this capability.Sending packets is then done entirely by the virtual monitor and NIC

interrupts may be processed by the monitor tooSince delivering interrupts to the host operating system is expensive,

performance can improve.Who'd a thunk it?Paravirtualization actually helps performance!!



Virtual Workspace Features

Workspace provides interfaces based on the WSRFAllows an authorized Grid client to deploy, shutdown,

pause and reactivate VMS



Worker node deployment workflow

Worker node deployment is requested on-demand by an authorizedoff-site grid client

Resource allocation request asks for 2 GB memory and the full useof a CPU for each virtual node

On deployment, each node reports to Condor headnode and joinsthe Condor pool

A web application displays current virtual cluster node informationbased on Condor pool properties

A client can then start jobs on the deployed VM using GRAM2deployed on the static CE (compute element)



Schematic for provisionig of STAR nodes

TeraPort node TeraPort node TeraPort node TeraPort node TeraPort node

WorkspaceService

OSGCEGRAM

Star node

Star node

Star node

Star node

Star node

Star node

Star nodeStar node

ProvisioningSTAR nodes

Execute newSTAR instance

STAR



Current Cloud providers

Go GridAmazon Web ServicesGoogle App Engine

MossoSlice HostMedia TempleFlexiscaleJoyent

Although they provide webservices and compute-power-on-demand, they are not VirtualMachines on grid



Cloud Computing – Everything as a Service

Elastic computing,Pay-as-you-go,

Capital expense

operational expense

Source : Cloud Computing with Nimbus, FNAL, January 2009Kate Keahey, University of Chicago, ANL



Cloud Computing – Everything-as-a-service

IaaS

PaaS

SaaS

Source : Cloud Computing with Nimbus, FNAL, January 2009

Kate Keahey, University of Chicago, ANL

Software as a Service

Platform as a Service

Infrastructure as a Service

The anology to the real world is that it used to be that if you wanted to go to the airport, you couldcall a cab and pay the cab-driver.Then, they said, you know what – if you pay us, we can let you rent the car and you can have thecar, but you can't be setting the car on fireNow – you can lease a car and have the car for as long as you want and do whatever you want tothe car. Ofcourse, you are going to be able to do a lot more than just drive to the airport with thecar.



Main problems we're trying to solve

Code complexityResource control

Source - The Nimbus Toolkit : http://workspace.globus.org



The concept of 'workspaces'

Dynamicall provisioned environmentEnvironment controlResource control

Hardware implementations vs. virtualization





Nimbus Overview

Goal: open source, extensible, IaaS implementation and toolsSpecifically targeting scientific communityA platform for experimentation with features for scientific needsSet up private clouds (privacy, expense considerations)

Tools

IaaS layer (Workspace Service)Orchestration layer (Context Broker, gateway)http://workspace.globus.org/





Workspace Pilot and the concept of resource leases

Resource leases – Allow users to request direct access to resourcesrather than ask for a job to be run on those resource

ExamplesA static long-term agreement with a hosting companyOn-demand provisioning of a physical cluster partition with a

specified configuration (Cluster-on-demand)Dynamically deploying a virtual machine for an hour on

resource provied by Amazon's EC2 service

Ad f l i



Advantages of 'Flying Low'(Workspace Pilot)

A user can adapt resource to his needsUse it to support an interactive sessionRun computations requiring an application-specific

schedulerSupport portability tests across a variety of

environmentsExemplified by 'pilot job' approaches that use batch

scheduler installations on sites to deliver a leaserather than submit a job to that scheduler

Source : Flying Low : Simple Leases with Workspace Pilot

Tim Freeman Kate Keahey, University of Chicago, ANL

I l i f VWS



Implementation of VWS(Virtual Workspace Service)

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

VWSService




The Workspace Service

Poolnode

Trusted Computing Base (TCB)

Poolnode

Poolnode

Pool

node

Pool

node

Pool

node

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

Poolnode

The workspace service publishes

information on each workspaceas standard WSRF ResourceProperties.

Users can interactdirectly with their

workspaces the sameway the would with a

physical machine.

VWSService




Workspace Service Interfaces and Clients

Web Services basedWeb Service Resource Framework (WSRF)

GT-basedElastic Computing Cloud (EC2)

Supported: ec2-describe-images, ec2-run-instances, ec2-describe-instances, ec2-terminate-instances, ec2-reboot-instances, ec2-add-keypair, ec2-delete-keypair

Unsupported: availability zones, security groups, elastic IP assignment,REST

Used alongside WSRF interfacesE.g., the University of Chicago cloud allows you to connect via the cloud

client or via the EC2 client



Nimbus Schematic

workspacecontrol

workspaceresourcemanager

workspace

pilot

workspaceservice

workspaceclient

cloudclient

IaaSgateway

contex

t broker

contextclient

EC2potentially other providers

storageservice

E C 2

W S R F




Science Clouds

q Make it easy for scientific projects to experiment with cloud computingx Can cloud computing be used for science?

q Evolve software in response to the needs of scientific projectsx Start with EC2-like functionality and evolve to serve scientific projects: virtual

clusters, diverse resource leasesx Federating clouds: moving between cloud resources in academic and commercial

space




Science Cloud Resources

q University of Chicago (Nimbus):x first cloud, online since March 4th 2008x 16 nodes of UC TeraPort cluster, public IPs

q University of Floridax Online since 05/08x 16-32 nodes, access via VPN

q Other Science Cloudsx Masaryk University, Brno, Czech Republic (08/08), Purdue (09/08)x Installations in progress: IU, Grid5K, others

q Using EC2 for overflowq Minimal governance modelq http://workspace.globus.org/clouds



Nimbus Walkthrough



Open Source IaaS Implementations

q Eucalyptusx Open source implementation of EC2x UCSB, R. Wolski & team, 06/2008

q OpenNebulax Open source datacenter implementationx University of Madrid, I. Llorente & team, 03/2008

q Cloud-enabled Nimrod-Gx Monash University, MeSsAGE Lab, 01/2009

q Industry effortsx openQRM, Enomalism



The Nimbus Community

q Committers: Kate Keahey & Tim Freeman (ANL/UC), Ian Gable (UVIC)q A lot of help from the community, see:

http://workspace.globus.org/people.htmlq Collaborations:

x Cumulus: S3 implementation (Globus team)x

EBS implementation with IUx Appliance management: rPath and Bcfg2 projectx Virtual network overlays: University of Floridax Security: Vienna University of Technology



Future

q Increasing Importance of Appliance Providersq Cloud computing toolsq Increased interest in cloud interoperability

x Standards: “rough consensus & working code”x Image formats, contextualization capabilities, cloud interfaces, etc.

q

Cloud markets



Documents

Grid Based VM Provisioning - A Presentation by Arunabh Das