04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers

8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers

1/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Impact of Advanced VirtualizationTechnologies on Grid Computing Centers

International Symposium on Grid Computing 2009

Stefan Freitag

Robotics Research Institute

Dortmund University of Technology

23. April 2009
http://find/


2/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Overview

1 Virtualization

2 Impact on Grid Computing

3 Conclusion
http://find/http://goback/


3/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Introduction

ApplicationResource

Virtualization

OS Level

Network

Platform

Full

Storage Input/Output

Paravirtualized

Figure: Types of Virtualization


4/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Network

VPN (Virtual Private Network)

disjunct network partitions

gateway service for tunneling

WAN for interconnect

WAN

LAN 1

LAN 2

LAN 3

FWFW

FW

GWGW

GW


5/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Network

VLAN (Virtual Local Area Network)

Layer 2 (data link layer) construct, IEEE 802.1Q standardDevices on different physical LAN segmentsCommunication as if on same physical LAN segment(bcast domain)

Configuration of VLAN through softwareTraffic shaping/ QoS

LAN

. . .

. . .

S1

S4

. . .

S3

.

.

.

S2

VLAN2

VLAN1


6/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Storage

decreasing storage costs (GByte/$)

increasing management complexity

full SAN bandwidth for I/O requests

separation of data and metadata into different places

control unit: appliances or SAN switches

Virtualization

23

4 1

Virtualization

2

1

Control

5

36

4

Virtualization

1a

1b

23

4

Control

1


7/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Platform

Full virtualized Environmentguest unaware of virtualization, run native OSreduced performance because of hardware emulation

Para virtualized Environmentmanagement module (hypervisor or virtual machine

monitor) operates with modified operating systemguest OS has much closer control of the underlyinghardware (security, influence on other VMs)

VMManagement

VirtualHardware

VirtualHardware

Guest OS

Application

Guest OS

Application

X86 Hardware

Host OS

Virtualization Layer

X86 Hardware

Virtualization Layer

ApplicationApplication

ManagementVM

Guest OS Guest OS(modified) (modified)

Host OS


8/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Input/ Output (IOV)

Typical configuration: 4 to 6 I/O cards per serverEthernetInfinibandFiber Channel

add I/O virtualization capability to PCIe

disaggreationconsolidationvirtualization

/WANLAN

SANPCIe

NICs

IOV Switch

HBAs

Server


9/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Input/ Output (IOV)

I/O consolidation

100% of server connected to Ethernet, only 20% to FC

FiberChannel over Ethernet (FCoE), pre-standard

converged Network Adapter (C-NIC/ CNA)benefit for compute centers

fewer adapterspower savingscable management

LAN /WAN

SAN

Server

CNIC

FCoE

Switch

Ethernet

Switch


10/17

Virtualization

in GridComputing

Centers

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Present situation

Platform and storage virtualization in use

server consolidation

improved utilization of existing servers

increased number of services per m2

reducing TCO (maintenance,...) in the long-run

high availability

run legacy applications

managing mass-storage backendsmigration of virtual machines is bound to restrictions

storage network reconfiguration


11/17

Virtualization

in GridComputingCenters

S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Platform Virtualization @ LRMS level

Job = Virtual Machine

share same operations (start, stop, suspend, checkpointing)

Scenarios:

user submits self-prepared virtual machineon-demand creation of virtual machine by LRMS

VM description as part of job specification

LRMS schedules jobs and/ or virtual machines

Not so good:black-box execution

performance loss (MPI) compared to non-virtualizedsolution


12/17

Virtualization


S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

IOV @ LRMS level

vNIC (virtual Network Interface Card)

vHBA (virtual Host Bus Adapter)

FC WWN used to identify to segregate devices for accesscontrolVirtualized environments: adapters WWN represents allDomUs (Xen)DomU migrates to new server, but WWN does not

zoning and LUN masking (access control) ineffectivevHBA gets own WWN which migrates with VM


13/17

Virtualization


S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Platform Virtualization @ Grid Level

Benefits

provide uniform environment to Grid users

rapid on-demand provisioning of Grid nodes

non-interfering execution of multiple Grid middlewares

decoupling of knowledge (grid operator/ grid user)

job exchange, e.g. inter cluster/ Grid resources


14/17

Virtualization


S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Platform Virtualization @ Grid Level

Drawback

resources run different virtualization software

Xen, KVM, VMware, . . .

creation of user environment for each resource


15/17

Virtualization


S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Virtual Appliances

encapsulate user environment in a virtual appliance

virtual appliance = virtual machine + operating system

+ application level software + MD5

Job = virtual machine virtual appliance

virtual appliance is already technology-dependent :-(

better approach

step 1: create

operating system + application level software + MD5

step 2: make it technology-dependent


16/17

Virtualization


S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

KIWI

idea: prepare once, run anywheredistribution independent, currently only SuSE supported

centralized image description based on XML

output formats: xen, vmware, Amazon EC2, iso, . . .

adoption to Scientific Linux ongoing in Dortmund

PackedImage

description

Image

software

packages

additional

ready 2 serve

accessv

iachro

ot

access

vialoop

UnpackedImage

Figure: from: KIWI Cookbook


17/17

Virtualization


S. Freitag

Virtualization

Impact on

GridComputing

Conclusion

Conclusion & Outlook

Conclusion

platform and storage virtualization widely spread

increased flexibility with I/O virtualization/ FCoE

Grids profit at LRMS and Grid layer from virtualization

on-demand provisioning of virtual appliances

Outlook

Grid resource provides bare metal + virtualization layer

provision of resources to more than one Gridinfrastructure/ VO

Documents

04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers