INTRODUCTION TO

Thomas J. HackerAssociate Professor, Computer & Information Technology

Co-Leader for Information Technology, Network for Earthquake Engineering Simulation (NEES)

VIRTUAL HIGH PERFORMANCECOMPUTING CLUSTERS

July 30, 2012

OUTLINE

Motivation for the use of virtualization

Overview of virtualization technology

Overview of cloud computing technology

Relation of cloud computing to HPC

Practical notes on virtualization and cloud computing

Virtual HPC clusters

How to get started

MOTIVATION FOR VIRTUALIZATIONWHY VIRTUALIZATION, AND WHEN DOES IT MAKE SENSE?

• Clock speed increases following Moore’s law have ceased

• Hardware is going to multicore with many cores– E.g. Intel MIC is the new Xeon Phi (Knight's Corner) with 50+

cores

• Memory capacity of systems increasing– Max 512 GB on systems today

MOTIVATION FOR VIRTUALIZATIONTraditional approach has been to tie a single application to a single server

• An application runs in its own OS image on its own server for manageability and serviceability

This approach doesn’t make sense anymore if you have 50+ cores that can’t be effectively used by an application

It’s also difficult to share OS and various library versions for running multiple apps on the same system if OS/lib version requirements are conflicting

VMs are being used to partition large scale servers to run many OSs and VMs independently from each other

MOTIVATION FOR VIRTUALIZATIONVirtualization is now commodity technology

• Ideas were first developed in the 1960s at IBM for their mainframe computers

Virtualization is used frequently for administrative applications to reduce the hardware footprint in the data center and reduce costs

This represents a commodity trend that like other commodity trends that is worth exploiting for HPC

Especially useful substitute for small-scale lab clusters that are used early in the life cycle of a parallel application.

SOFTWARE ECOSYSTEM FOR APPLICATIONSSOFTWARE REQUIRES A FUNCTIONAL ECOSYSTEM (SIMILAR TO MAZLOW’S NEEDS HIERARCHY)Basic “physiological” needs

• Reliable computing platform • Functional operating system platform that is needed by the application

– If software isn’t kept up to date, can conflict with OS upgrades• Adequate disk space, memory, and CPU cores

“Safety” needs• Secure computing environment – no attackers, compromised accounts, etc.• “sense of security and predictability in the world”• Predictability is essential for replicating results and debugging

“Sense of community”• All of the nodes in the cluster need to be consistent• Same OS version, libraries, etc.• Especially critical for MPI applications

Meeting these basic needs ensures a consistent software ecosystem• Stable platform facilitates software development, testing, and validation of results• Developers and users can begin to trust the software, and results from software• Provides a strong base for future growth and development of the application

SOFTWARE ECOSYSTEM FOR APPLICATIONSProblems: difficult for users to control their computing environment for scientific applications

• Scientific apps used in projects such as CMS require a lot of specific packages and versions, and it can be very difficult to get central IT organizations to customize and install the necessary software, due to the need to provide a generic and reliable system for the rest of the user base.

• Scientific applications go through a life-cycle in which they evolve from single processor to running on a few workstations to small scale clusters and then finally scaling up to very large systems.

• Building small scale physical clusters as a part of this life cycle is very expensive both in equipment, time, and grad student effort wasted to run these systems.

• Scientific users can really benefit from having root access on their own systems to work on getting their codes working and installing any necessary packages.

SOFTWARE ECOSYSTEM FOR APPLICATIONSVirtual HPC clusters are an attractive and viable alternative to small scale lab clusters when applications that need these types of resources are still “young” and require a lot of customization.

• On larger systems, virtual clusters are a promising approach to provide system level checkpointing for large-scale applications.

• Imagine if you could use a virtualization system on your laptop to develop a 2 or 3 VM virtual cluster with all the packages and optimizations you needed, then transfer that VM image to a virtual cluster platform and instantiate dozens (or more) VM images to run a virtual cluster.

Fault tolerance is a critical problem for applications as they scale up.• There are several levels of checkpointing:

– Application level– “On the system” level (e.g. condor, blcr)– “below the system” level using live migration or checkpointing/saving VM images

RELIABILITY

One of the “safety” needs of software in its ecosystem• Problems with reliability and techniques to improve reliability

Large systems can fail often

Severely affects large and/or long running jobs

Very expensive to just restart computation from the beginning• Lots of wasted time on the computer system, and wasted power and cooling

RELIABILITY

A technique to overcome this problem is to frequently save critical program data – called checkpointing

• Your program will need to read the saved data when your program is restarted and resume computational from the saved state

There is some guidance as to how often you need to checkpoint to find a good balance between spending time on saving state for “safety” vs. making forward progress in your computation

• Daly’s checkpoint formula is a good start

RELIABILITYDALY CHECKPOINT FORMULA

Used to estimate the optimal compute time between writing checkpoints

RELIABILITY

Research exploring alternative methods of performing checkpoint operations

• System level checkpointing - BLCR• MPI level checkpointing• VM level checkpointing and live migration

– Idea is to periodically save the VM state, or to live migrate the VM from sick to healthier systems

RELIABILITY

Be aware of the need to integrate reliability practices in your application as you design and write your code

At a minimum structure your code so that you can periodically save the current state of computation, and develop a capability to restart computation from that saved state if your program is restarted

OVERVIEW OF VIRTUALIZATION TECHNOLOGIESVirtualization is a technique that separates the operating system from the physical computer hardware, and interposes a layer of controlling software (hypervisor) between the hardware and operating system.

Different types of virtualization systems (from Goldberg)• Type 1: hypervisor between “bare metal” and guest operating systems• Type 2: hypervisor between host operating system and guest operating systems

Type 1 examples• VMware, Xen, KVM• OpenVZ

Type 2 examples• Virtual Box, VMware Workstation, Parallels for Mac

TYPE 1 VIRTUALIZATION

VMware• High quality commercial product• We use VMware extensively for NEES• Very useful for transitioning IT infrastructure from SDSC to Purdue for the NEES

project• Simply created VM images for each service/server on a few physical servers• We were able to archive the VM images of the services/servers when NEES

brought up NEEShub cyberinfrastructure

Windows• Hyper-V

NEES VMWARE INFRASTRUCTURE

TYPE 1 VIRTUALIZATION

Virtualization systems for Linux• Xen and KVM• Open source virtualization systems based on Linux

Xen • First major virtualization system• Older, seems to be less reliable

KVM• Kernel-based Virtual Machine• Newer, supported by RedHat

OpenVZ• Container based virtualization system

XEN

First version in 2003, and the first popular Linux hypervisor

Integrated into the Linux kernel• Uses paravirtualization

– Guest OSs run a modified operating system to interact with hypervisor• Different from VMware, which uses a custom kernel you load on the bare harware• Host OS runs as Domain0• Guest OSs run

Used to be supported in a limited form in RedHat and Ubuntu• Has been replaced with KVM in RedHat• Citrix has a commercial version of Xen

Personal experiences using Xen• Works OK for simple virtualization• Complex operations didn’t work as well

KVM

Kernel-based Virtual Machine (KVM)• Built into Linux kernel• Supported by RedHat• More recent than Xen

Uses QEMU for virtual processor emulation• Allows you to emulated CPU architectures other than Intel• E.g. ARM and SPARC

Supports a wide variety of guest operating systems• Linux• Windows• Solaris

Provides a useful set of management utilities • Virtual Machine Manager• ConVirt

OPENVZ

Container based virtualization system• Secure isolated Linux containers• Think of this as a “cage” for an application running in an OpenVZ container • OpenVZ terminology: Virtual Private Servers (VPS), Virtual Environments (VE)

Two major differences from Xen and KVM• Guest OS shares kernel with host OS• File system of Guest OS visible on Host OS and is part of the directory tree on the Host

OS– Doesn’t use a virtual disk drive (no 15 GB files to manage)

Benefits compared with Xen and KVM• Very fast container creation• Very fast live migration• Easy to externally modify container file system (e.g. install software in the container)• Scales very well (no big virtual disk images)

Downsides• Must use the same OS as the Host OS

– Sharing kernel with the Host OS

TYPE 2 EXAMPLES

Oracle Virtual Box• Free VM environment that you can use on Windows, Linux, Mac OS X, and

Solaris• Simple to use, good way to get started• VM images can be exported

VM images can be exported• In theory….• Depends on the ability of the target virtualization system to import VM disk images • Exports in OVF (Open Virtualization Framework) format• My personal experience is that you often need to use a Linux utility to try to

convert the disk image and VM metadata to an acceptable format for another virtualization system (often complex).

TYPE 2 EXAMPLES

VMware Workstation• Runs as an application on top of Windows• NOT VMware ESX (which is a hypervisor)• Another good way to get started in working with virtualization technology

Parallels for Mac• Can be used to run Windows on a Mac• Commercial software• Personal experience: Works “OK”, but Windows can be slow running on Parallels

OPENVZ VS. KVM

I am using OpenVZ and KVM for two different projects

NEES / NEEShub• Based on HUBzero• Using OpenVZ as a virtual container or “jail” in which to run applications that

interfaces with user through a vnc window on a webpage

OpenNebula cluster to run parallel applications• Distributed rendering using Maya

– batchrendering animations

• OpenSees building simulation program for NEES– Parallel version that uses parallel solvers and MPI– Running on a virtual cluster on OpenNebula in my lab and on FutureGrid

The choice depends on the type of application who wish to run and the environment in which it will be run.

VIRTUALIZATION ON LINUX

Additional mechanisms in Linux• Libvirt / virtio

– Veneer library and utilities over virtualization systems

• Brctl– Linux virtual network bridge control package

• Cgroups– Linux feature for controlling resource use of processes

• Network virtualization

Network control is a constant problem• VLANs are best, but hard to configure• OpenFlow is supposed to address network management to simplify it

and make it scalable.

MOVING UP FROM VIRTUALIZATIONWe talked about virtualization on a system level

How can we manage a collection on virtual machines on a single system?

How can we manage a distributed network of computers than host virtual machines?

How can we manage the network and storage for this distributed network of virtual machines?

This is the basis for one aspect of what is called “cloud computing” today

• Infrastructure-as-a-Service (IaaS)

The technology used for IaaS is the basis for building virtual HPC clusters, which is a collection of virtual machines running on a distributed network of computers.

OVERVIEW OF CLOUD COMPUTING TECHNOLOGIES

CLOUD COMPUTING

Emerging technology that leverages virtualization• Distributed computing of the 201Xs• Initial idea of a “computing utility” from Multics in the 1960s

Computing utility that provides services over a network• Computing• Storage• Pushes functionality from devices at the edge (e.g. laptops and mobile

phones) to centralized servers

CLOUD COMPUTING ARCHITECTUREUser interface

• How users interact with the services running on the cloud• Very simple client hardware

Resources and services index• What services are in the cloud, and where they are located

System Management and Monitoring

Storage and servers

TYPES OF CLOUD COMPUTING SYSTEMSInfrastructure as a service (IaaS)

Software as a service (SaaS)

Platform as a service (PaaS)

There are some fundamental difference between these approaches that lead to confusion when talking about “cloud computing”

• A cloud computing infrastructure can include one or all of these

INFRASTRUCTURE AS A SERVICE (IAAS)Virtualization environment

• Cloud service provider offers capability of hosting virtual machines as a service

• Cloud computing infrastructure for IaaS focuses on systems software needed to load, start, and manage virtual machines

Amazon EC2 is one example of IaaS

IAAS

Enabling technologies used to provide IaaS

Virtualization layer• VMware• Xen/KVM• OpenVZ

Networking layer• Need to provide a VPN and network security for private VMs

Scheduling layer• Managing the mapping of IaaS requests to physical and virtual

infrastructure• Amazon EC2 provide this• OpenNebula, Eucalyptus, and Nimbus also provide scheduling

services

IAAS BENEFITS

User doesn’t need to own infrastructure• No servers, data center, etc. required• Very low cost of entry

Pay-as-you-go computing• No upfront capital investments needed• Leasing a solution instead of a box

No systems administration staff/operations staff needed• Cloud computing provided leverages economies of scale

EXAMPLES OF IAAS

BlueLock in Indianapolis• Commercial IaaS provider

Eucalyptus• Started as a research project at UCSB• Based on Java and Web Services

OpenNebula• Developed in Europe• Leverages usual Linux technologies

– Ssh, NFS, etc.

• Uses a scheduler named Haizea

Nimbus• Research project at Argonne National Lab• Linked with Globus

PLATFORM AS A SERVICE (PAAS)

Builds on virtualization platform

Provides a software stack in addition to the virtualization service

• OS, web server, authentication, etc.• APIs and middleware

For example, if you needed a web server and you didn’t want to install apache, linux, etc.

BENEFITS OF PAAS

Supported software stack• Don’t need to focus efforts on getting software infrastructure working• Pooled expertise in use of the software at the cloud computing provider

You can focus service and development efforts on just your product

Pay-as-you-go

EXAMPLES OF PAAS

Amazon Web Service• Wikileaks was using this• You buy a web service that runs on Amazon’s

virtualization infrastructure• Downside: outages can take out a lot of services. • Netflix also uses Amazon EC2

Other Examples • Google App Engine• Microsoft Azure

SOFTWARE AS A SERVICE (SAAS)

Provides access to software over the Internet• No download/installation of the software is needed• Users can lease or rent software• Was a big idea about a decade ago, seems to be

coming back

Software runs remotely and displays back to the users computer

• Think ‘vnc’

NEEShub is an example of this• Researchers can run tools in a window without

download/install

BENEFITS OF SAAS

No user download/install• Many corporate users don’t have access on their

computers to install software

Easier to support• Control the computing environment centrally

Can be faster• As long as server hardware is fast and users have a

good network connection

Efficient use of centralized computing infrastructure

RELATION OF CLOUD COMPUTING TO HPCUse of cloud computing depends on how the HPC application is used

SaaS• NEEShub batchsubmit capability• Allows uses to run parallel applications through the NEEShub as a service• Users don’t need to be concerned about underlying infrastructure

IaaS• HPC clusters on an infrastructure level• The problem here is to deploy, operate, and use a collection of VMs to constitute a

virtual HPC cluster• The capabilities in this area are focused on VM image and network management

and deployment

RELATION OF CLOUD COMPUTING TO HPCFROM A USER’S PERSPECTIVE, WHAT DO YOU NEED TO DO TO USE THE TECHNOLOGY?SaaS

• Discover the application• Launch the application• Monitor execution• Collect and analyze the results

IaaS• Discover the resources needed• Provide a VM image or create a new one built from provided VM images• Deploy the image on the cloud computing system• Setup the networking among the VM instances• Setup an MPI ring• Launch your application• Monitor execution• Collect and analyze the results

SaaS is a lot simpler that IaaS for users• HUB based systems such as NEEShub and nanoHUB provide a specific set up applications as a service• However, it limits what a user can do

The problem is how to establish a virtual HPC cluster that can be used by users to develop, test, and prepare a parallel application for production use or to eventually transition to application to a service (SaaS) than can be run in a HUB environment.

EXAMPLE OF NEESHUB SAASWINDOWS APPLICATION

EXAMPLE OF NEESHUB SAASLINUX APPLICATION

You can create an account on nees.org and try these tools

PRACTICAL NOTES ON USING VIRTUALIZATIONLINUX

Use virt-manager to create and manage VM images

Images usually stored in /var/lib/libvirt• Make sure you have enough storage for /var/lib• Or you can change the default location using virsh

Networking can be complicated due to the use of virtual network bridges in Linux

• Networking can be very complex – be prepared to work on it to make it work

• Simplest to start with NAT to get your VM on the network• Be cautious about computer security

PRACTICAL NOTES ON USING VIRTUALIZATION

Managing network can be tricky• Bridge-utils yum package provides brctl utilities to create and

manage virtual network switches and connections

External interface connects to the virtual network switch• VMs will connect to the virtual switch to share the connection• Virt-manager provides some functionality for this, but basically

relies on what is created and managed by bridge-utils

PRACTICAL NOTES ON USING VIRTUALIZATION

Learn virsh• Libvirt is used to control the KVM virtualization system• virsh is the CLI for libvirt• The real power behind the GUIs

Libvirt is supposed to be able to control Xen and KVM

PRACTICAL NOTES ON CLOUD COMPUTING SYSTEMS

VMware vSphere is commercial version of an IaaS controller

• Start, stop, migrate, shutdown, and startup VM images and hardware servers

• Manage virtual network switches• Good GUI and clear way to learn to work with the

technology

PRACTICAL NOTES ON CLOUD COMPUTING SYSTEMS

Linux based: NIMBUS, Eucalyptus, OpenNebula, OpenStack

• My personal experience is that OpenNebula is the most straightforward system to setup and use

• Uses standard Linux facilities• Command line interface is clear and logical

VIRTUAL HPC CLUSTERSREVIEW

Talked about virtualization technology

Talked about cloud computing technology• SaaS• IaaS

Talked about setting up and controlling a collection of virtual machines

How can we use this technology for HPC?

VIRTUAL HPC CLUSTERS

You can create a virtual cluster built on a distributed collection of virtual machines controlled by a cloud computing system

Allows you to run different OSs and applications on a finite set of servers without the need to reload and update the hardware servers when you want to change the load

Very efficient use of hardware and space resources• Sometimes the campus will provide VM space for you

VIRTUAL CLUSTERS

Steps to create a virtual cluster

Select an Linux OS (or Windows) image you want to use• May select one already provided by the cloud computing provider• Pick you own, but might take more work to configure the VM image to work with the cloud computing

system

Create first VM image from DVD or provided instance• On Linux, use virt-manager• Other cloud computing systems (e.g. Amazon, Nimbus) provide a VM image to start with

Connect it to the external network• Protect your image from intrusion using iptables or NAT using a local-only IP address (e.g.

192.168.X.X)

Customize the VM image with your application, libraries, and MPI• Install the compilers, libraries, or utilities needed for your application• Yum, apt-get, or .tar.gz files

– This is why it’s important to have accessibility to external network• Pick an MPI implementation and install it

Compile/build your application

Ensure that you have installed all of the necessary libraries, tools, etc.• The ldd command is useful to ensure that all dependent libraries are on the system

You how have a “golden master” VM image

VIRTUAL CLUSTERS

Plan out your IP address space• What range of IP addresses will you assign to the nodes on your virtual HPC cluster?• These addresses will need to go into /etc/hosts and each VM instance will need to be

assigned an IP address and hostname

Import your “golden master” image into the cloud computing system• This can be complicated and difficult• If you are building your image on Linux, you will need to export the virtual disk image along

with the VM metadata describing number cores, mem, NIC MAC, etc. in the VM image• virsh dumpxml• OpenNebula: oneimage / onetemplate commands

Configure the cloud computing system to assign IP addresses from your range when it clones your “golden master” VM image

• Alternative is to manually configure each VM instance

Clone your “golden master” image N times to create an N node virtual HPC cluster• Watch your disk space to ensure you have enough space for all the VM images

Once the cloud computing system has booted all of your VM images• Make sure you can connect to the console or ssh to each image• Virt-manager can help with this

VIRTUAL CLUSTERS

Create /etc/hosts and mpihosts file and copy to all of the virtual nodes

Either use mpdboot or mpiexec (depends on the MPI system you use) to make sure MPI can communicate across the network

• You will need to open the appropriate ports using iptables• Alternatively, if your virtual cluster is behind a firewall (like pfSense), you won’t

need to use iptables and you can open all of the ports.

Setup a shared file space if you need it• Setup one of the nodes to act as an NFS server, and mount a shared space on all

of the virtual cluster nodes

Run your application

VIRTUAL CLUSTERSMY OWN EXPERIENCE

I’ve used OpenNebula to create and clone Windows 7 VMs and RHEL6 VMs to run OpenSees serial and parallel versions developed for Windows and Linux

Seems to work pretty well once it’s setup

Working with FutureGrid now to create a larger virtual RHEL cluster for the parallel version of OpenSees

So far, created a virtual HPC cluster with 10 4-core VMs

BENEFITS OF VIRTUAL HPC CLUSTERSBENEFITS

Flexibility

User has root access

System level checkpointing

Potential for archiving and long term curation of scientific applications with operating system images required for the applications to execute

You can save , share, and later retrieve whole virtual clusters (think versioning like you would do for software)

Cluster can be created and operated for you by system administrators with the need to run your own cluster in a lab closet.

DRAWBACKS OF VIRTUAL HPC CLUSTERSDRAWBACKS

Performance penalty

Latency• Not likely to have access to a high performance switch

IP address space can be difficult• Often not well thought out in cloud computing systems• Ideal would be to have a private IP address space that is firewalled from the rest of the hardware and the

world• Way to do this is to use VPNs or VLANs

– VPNs impose a software overhead– VLANs require admin acceess to the network routers and switches (you are not likely to be granted this level of

access)

• So, setting up /etc/hosts and MPI ring is still a little clunky compared with the other aspects of managing the VM images through the cloud computing system

Getting a private network working and in place

Complexity of learning how to use the technology

Need for help from systems administrators to get initial VM image working• Some cloud computing systems require you to use a “canned” VM image to start with that they know

already works• More helpful if you can create your *own* OS images with necessaary libraries and software that you can

then hand off to the cloud computing system• In my own experience, OpenNebula is the best for this

VIRTUAL HPC CLUSTERS IN USE

Futuregrid• IU project to provide a testbed for virtual clusters and cloud computing apps

Nimbus• Project led by Kate Keahey to develop software to support science clouds

NEEShub / HUBzero• Purdue project working on the SaaS level (and to some degree IaaS using OpenVZ) to

provide a complete cyberinfrastructure for science and engie

Galaxy project• Project led by James Taylor at Emory to provide a data cyberinfrastructure for biology.

Users can create on demand virtual HPC clusters

iPlant • Project to develop cyberinfrastructure for plant biology. Users can create persistent VM

images using Atmosphere system.– Uses Eucalyptus and OpenStack on the back end for a cloud controller

Condor• Treats VM deployment as jobs

HOW TO GET STARTED

Using VMs is a different way of thinking about computing

Best way to start is to try some software on your computer, make a VM, and try it out to become familiar with the technology

• Linux – KVM• Windows – Oracle VirtualBox• Mac – Parallels

For an exercise, do the following:• Pick your platform, install a virtualization system• Grab the latest Fedora DVD image• Install it on your virtualization system• Get it working on the network (e.g. browse the web)• Perform a system update on the VM image

QUESTIONS?

Contact: tjhacker@purdue.edu