Day 1 : August 05 2004 (Thursday) Globus Toolkit 2.4 Demokabru/parapp/Grid-2004-workshop/presenta… · An Overview of Globus Toolkit 2.4 Demo GRID 2004 M I T C a m p u s , A n a

1Copyright C-DAC 2004

An Overview of Globus Toolkit 2.4 Demo GRID 2004GRID 2004

MIT Campus, Anna University; August 05-06, 2004

Globus Toolkit 2.4 Demo

An Overview of Grid Computing Workshop

By

CDAC Experts

Contact :[email protected];

[email protected]

URL : http://www.cs.umn.edu/~vcvrao

Day 1 : August 05 2004 (Thursday)

C-D

AC

200

4

A

ugus

t 05-

06, 2

004


GRID 2004GRID 2004An Overview of Globus Toolkit 2.4 Demo


Betatesting Group,NPSF, C-DAC,Pune

• Dr.VCV.Rao ([email protected]) • Mr.Ravi Kumar ([email protected])• Mr.Subba Ramanna ([email protected])• Mr.Sridhar Padala ([email protected])• Ms. B.S.V.S. Sudha ([email protected])

NPSF System Admin Group, C-DAC,Pune

• Dr. Sandeep K. Joshi ([email protected])

��

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




Globus 2.4 Toolkit

� Globus Resource Allocation Manager (GRAM)

� Metacomputing Directory Service (MDS)

� Data Management Protocol (GridFTP)

PARAM 10000 Node Configuration

� Sun Ultra Sparc e450 Quad SMP’s

� Solaris 2.6 Operating Systems

� Portable Batch System (Job Manager)

Contd…Grid Prototype at NPSF

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




PARAM OpenFrame Node Configuration

� Sun Cluster Dual CPU (Ultra-II)

� Solaris 8 Operating Systems


PARAM Anant Node Configuration

� Linux Cluster Dual CPU

� RedHat 7.3 Operating Systems


Contd…

Grid Prototype at NPSF

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




Job Submission Node

Certificate Authority Node

n05 ameyaefs01

efs02

ecs02ecs01 n09

n15

n10

n11

n14

n13

adi

anadiamba

CA CA

CA

PARAM 10000 PARAM ANANT PARAM OpenFrame

server

Grid Prototype at NPSF

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� The Grid Resource Allocation Management (GRAM) protocol and client API allows programs to be started on remote resources, despite local heterogeneity

� Resource Specification Language (RSL) is used to communicate requirements

� A layered architecture allows application-specific resource brokers and co-allocators to be defined in terms of GRAM services� Integrated with Condor, PBS, MPICH-G2, …

Grid Resource Management

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� Gatekeeper� Single point of entry� Authenticates user, maps to local security

environment, runs service� In essence, a “secure inetd”

� Job manager� A gatekeeper service� Layers on top of local resource management system

(e.g., PBS, LSF, etc.)� Handles remote interaction with the job

Globus Toolkit 2.4 Implementation

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




��

��

��

��

��

��

��

��

��

�� !��

��

��

��

��!��

�� "��#��$��

��%��

��"��$��

��

��

��&��%�

GRAM Components

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� Common notation for exchange of information between components

� RSL provides two types of information:� Resource requirements: Machine type, number of

nodes, memory, etc.� Job configuration: Directory, executable, args,

environment

� Globus Toolkit provides an API/SDK for manipulating RSL

Grid Resource Specification Language

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� Elementary form: parenthesis clauses� (attribute op value [ value … ] )

� Operators Supported:� <, <=, =, >=, > , !=

� Some supported attributes:� executable, arguments, environment, stdin,

stdout, stderr, resourceManagerContact,resourceManagerName

Resource Specification Language :Syntax

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� For example:& (count>=5) (count<=10)

(max_time=240) (memory>=64)(executable=myprog)

� “Create 5-10 instances of myprog, each on a machine with at least 64 MB memory that is available to me for 4 hours”

Resource Specification Language :Constraints: “&”

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� For example:& (executable=myprog)

( | (&(count=5)(memory>=64)) (&(count=10)(memory>=32)))

� Create 5 instances of myprog on a machine that has at least 64MB of memory, or 10 instances on a machine with at least 32MB of memory

Resource Specification Language :Disjunction “|”

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� A multirequest allows us to specify multiple resource needs, for example+ (& (count=5)(memory>=64)

(executable=p1))(&(network=atm) (executable=p2))

� Execute 5 instances of p1 on a machine with at least 64M of memory

� Execute p2 on a machine with an ATM connection� Multirequests are central to co-allocation

Resource Specification Language :Multirequest “+”

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� Simultaneous allocation of a resource set� Handled via optimistic co-allocation based on free

nodes or queue prediction� In the future, advance reservations will also be

supported (already in prototype)

� Globus APIs/SDKs support the co-allocation of specific multi-requests� Uses a Globus component called the Dynamically

Updated Request Online Co-allocator (DUROC)

Resource Coallocation

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




MPICH G2 & Globus

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




+

( &(resourceManagerContact="kushi.stp.cdac.ernet.in")

(count=1)

(label="subjob 0")

(environment=(GLOBUS_DUROC_SUBJOB_INDEX 0)

(LD_LIBRARY_PATH /home/globus/GLOBUS-2.4/lib/))

(directory="/home/heman/mpich-g2/cpi")

(executable="/home/heman/mpich-g2/cpi/cpi")

)

( &(resourceManagerContact="ameya.npsf.cdac.ernet.in")

(count=1)

(label="subjob 1")

(environment=(GLOBUS_DUROC_SUBJOB_INDEX 1)

(LD_LIBRARY_PATH /home/globus/GLOBUS-2.4/lib/))

(directory="/home/heman/mpich-g2/cpi")

(executable="/home/heman/mpich-g2/cpi/cpi"))

Resource Coallocation Multirequest

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� Globus Toolkit includes several command line programs for job submission � globus-job-run: Interactive jobs� globus-job-submit: Batch/offline jobs� globusrun: Flexible scripting infrastructure

� Others are building better interfaces� General purpose

� Condor-G, PBS, Hotpage, etc� Application specific

� Cactus, Web portals

Job Submission Interfaces

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� For running of batch/offline jobs� globus-job-submit Submit job� globus-job-status Check job status� globus-job-cancel Cancel job� globus-job-get-output Get job stdout/stderr� globus-job-clean Cleanup after job

globus-job-submit

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




� Flexible job submission for scripting� Uses an RSL string to specify job request � Contains an embedded globus-gass-server

� Defines GASS URL prefix in RSL substitution variable:(stdout=$(GLOBUSRUN_GASS_URL)/stdout)

� Supports both interactive and offline jobs

globusrun

C-D

AC

200

4

A

ugus

t 05-

06, 2

004




1. Grid Computing, http://www.gridcomputing.com

2. Globus web site http://www.globus.org

3. CDAC India http://cdacindia.com/html/npsf/npsfidx.asp

References

21

GCGC--SIWAHSIWAH--0404

Copyright C-DAC 2004 August 05-06, 2004

Globus Toolkit 2.4 Demo

Documents

Day 1 : August 05 2004 (Thursday) Globus Toolkit 2.4 Demokabru/parapp/Grid-2004-workshop/presenta… · An Overview of Globus Toolkit 2.4 Demo GRID 2004 M I T C a m p u s , A n a