6/2/2007 1

Grid ComputingGrid ComputingSun Grid Engine (SGE)Sun Grid Engine (SGE)

Manoj Katwal

6/2/2007 2

OutlineOutline

Grid and Cluster Terminology What is a Cluster vs. a Grid? Types of Grids Benefits of Grid Computing Areas of concern What is Sun Grid Engine? Example N1 Grid Network Setup

Do you know?Grid computing market will reach $ 12 billion by the end of 2007 (IDC)

6/2/2007 3

Outline, Cont.Outline, Cont.

Managing Jobs and Resources with a Grid Sun N1 Grid Engine

Concepts in Sun Grid Engine Job Submission Queues QMON Useful commands References

Do you know?Grid computing market will reach $ 12 billion by the end of 2007 (IDC)

6/2/2007 4

Grid and Cluster Grid and Cluster Terminology Terminology

A GRID is a collection of computing resources that perform tasks. A grid appears to users as a large system that provides a point(s) of access to powerful distributed resources.– Resources: Computer hardware, data, applications,networking

etc. Cluster:

– Tightly coupled homogenous computers, resources

– Redundant interconnections that appears as single system with high availability (HB)

– Not truly distributed system

6/2/2007 5

Grid and Cluster Grid and Cluster Terminology, Cont.Terminology, Cont. Grid:

– Loosely coupled heterogeneous computers or resources

– Resources can come and go

– High scalability– dynamically resources can be added and removed

– It is distributed virtually everywhere

– High Performance Computing (HPC)

Do you know?

The current version of Sun Grid Engine is downloadable for free.

6/2/2007 6

What is a Cluster vs. a Grid?What is a Cluster vs. a Grid?

Gentzsch, “Grid Computing: A New Technology for the Advanced Web”

6/2/2007 7

Types of GridsTypes of Grids

6/2/2007 8

Benefits of Grid ComputingBenefits of Grid Computing

Better utilization of resourcesIncreased user productivityFaster time-to-solution of complex

scientific, engineering, and enterprise computational tasks

Do you know?You can register to http://www.network.com and get 200 CPU hours free of charge. No obligation

6/2/2007 9

Areas of ConcernAreas of Concern

Users concern about Grid operation and Management include:

Confusion about what grid is and how it works Cross-vendor interoperability CPU and application usage billing and accounting Security and user authentication Standardization Global enterprise grids across international boundaries

6/2/2007 10

What is Sun Grid Engine?What is Sun Grid Engine?

Open source Grid management software – accepts jobs submitted by users

Software driving “job” distribution to compute nodes in a grid– By job means Unix like jobs (batches) or parallel

programs (MPICH- MPI Interface) Enable scheduling and prioritizing of jobs Management of jobs – submit, suspend, abort … Accounting and reporting (Arco, web based) Almost all operating systems are supported

6/2/2007 11

Example N1 Grid Network SetupExample N1 Grid Network Setup

12 billions dollars…

6/2/2007 12

Managing Jobs and Resources Managing Jobs and Resources with a Gridwith a Grid

6/2/2007 13

Concepts in Sun Grid EngineConcepts in Sun Grid Engine

Hosts – Four types– Master host, Secondary Fail over host (optional)– Execution hosts – Admin host– Submit hosts

A system can act as more than one type of host.

6/2/2007 14

Job Submission Job Submission

6/2/2007 15

QueuesQueues

Cluster queues Job slots Resource reservation Parallel environment (MPI support) Policy based resource allocation Job checkpointing

6/2/2007 16

QMONQMON

6/2/2007 17

Useful commandsUseful commands

Qsub – to submit job Qhost – getting an overview of available resources Qacct – job accounting information Qmod – managing queue state Qmon – graphical admin tool Qconf – configuring queues

6/2/2007 18

ReferencesReferences

By Jan Stafford, Editor, 30 Aug 2005 | SearchOpenSource.com http://searchenterpriselinux.techtarget.com/originalContent/0,289142,sid39_gci1120769,00.html

California State University, LA cluster (1 head node with 10 clients) http://oscar.calstatela.edu/

Sun Grid Compute Utility Web Portal http://www.network.com