High Performance Computing on Virtualized

EnvironmentsGanesh Thiagarajan

Fall 2014

Instructor: Yuzhe(Richard) Tang

Syracuse University

Overview

Virtualization refers to act of creating a virtual version of hardware or software

Virtual machine is an emulation of a computer system

In this paper, we are mainly discussing about

Virtualization,

how to achieve virtualization in Higher Performance Computing applications

XEN which is a Hypervisor

various other factors that affect achieving high performance in Virtualization

Introduction

Virtualization:

a technique which utilizes the underlying hardware for creating different many copies of virtual machines

Benefits

Reduction of power

Cooling costs

Increased availability

Isolation

Reduce Hardware Vendor lock-in

Every Virtual Machine is a distinct copy

Depending on the availability of Hardware Resources, a number of Virtual Machines can be allocated

Allocating virtual machines to available hardware is called Virtual Machine Placement Problem

Virtualization (Cont.)

Virtualization modes

Full-Virtualization

Para-Virtualization

Full Virtualization

Para Virtualization

Virtualization (Cont.)

Full Virtualization:

Advantages:

improves CPU and performance

gives the flexibility of running different Operating Systems

Live migration is very easy

Disadvantages:

Hardware and Software components needs to be compatible

Due to x86 protected instruction, performance might be low

Virtualization (Cont.)

Para Virtualization:

Advantages:

very easier to implement

Considerable increase in the performance

Disadvantages:

Guest OS requires modification which in turn results in poor compatibility

Virtual Machine Placement

Bin Packing solution problem

Offline Algorithms

Online Algorithms

Next Fit Algorithm

First Fit Algorithm

Best Fit Algorithm

Virtual Machine Live Migration

can be migrated from one hardware to other available hardware resource

without disturbing the running virtual machine

without shutting down the Operating System

can be performed by Hypervisor

Sample Live Migration Code:

\\lcs-fsrv.ad.syr.edu\w-gthiagar$\Desktop

Virtual Machine Monitor

Software program which runs over the hardware is Hypervisor

Types:

Type 1 Hypervisor

Type 2 Hypervisor

Type 1 Hypervisor

Type 2 Hypervisor

Virtual Machine Monitor(Cont.)

a piece of software which manages Virtual Machines

also termed as Hypervisor

Examples: Adeos, Hyper-V, VMWare – VSphere and VCloud and Xen

Ganeti is a Virtual Machine cluster management tool developed by Google

uses either Xen or KVM as the virtualization platform

High Performance Computing

Computers are expected to process on large data in a shorter time

tradeoffs in achieving the desired process with the available resource

done with Super Computers

Examples:

3D Nuclear Test Simulation,

Genomic Assembly pipelines,

Satellite Image Processing

High Performance Computing (Cont.)

Can be achieved with Recent technologies

Grid Computing

Cluster Computing Systems

Cloud Computing

Our Focus is on:

Cloud Computing – Virtualized Environments

High Performance Computing (Cont.)

Challenges for High Performance Computing

lack of high speed interconnects

maximum of 10Gigabit Ethernet

Security of private data on Cloud Environments

Amazon Cloud Service

On-demand instances,

Reserved instances and

Spot instances

XEN – Hypervisor

Type 1 Hypervisor

Para Virtualization

Modules:

DOM0

Host Operating System

has complete access to hardware

DOMU

Virtual Machine which runs its own Operating Systems

Guest Machines

Hypervisor

Virtual Machine Monitor

complete access to control and manage CPUs, Memory

XEN Architecture:

XEN Architecture (Cont.)

access to the tables are restricted to the virtual machines

read access to the page tables

VMM is the key component

more levels of access

Handles traps and interrupts

latest development

XenoLinux

capable of running more than 100 virtual machines

HPC on Virtualized Environments

process large volume of data requires extensive computational power

to execute the set of instructions

produce results for the same

Hardware failures can be completely taken care

Example: For example, if one of HPC application is trying to run an intense algorithms and a Hardware failure is detected where the virtual machine is residing, the VMM can effectively migrate the virtual machine to the next available server without interrupting the tasks which are running in the virtual environment.

HPC on Virtualized Environments (Cont.)

increases productivity in HPC applications

VMM authorizes other virtual machines to get information

performance of the HPC applications benefits

New Release Virtualization Spectrum

Conclusion

Virtualization concepts,

VM based computing in High Performance Computing application and

Xen Architecture and its applications

key notes in High Performance Computing

how High Performance Computing applications can benefits from the Virtualized environment

key features like Live Migration

Server consolidation would be the tedious task

saved by running the HPC in the virtualized environment

Thank You!