Grid ToolkitsGlobus, Condor, BOINC, Xgrid

Young Suk Moon

Grid Requirements

• Resource sharing / coordination• Resource discovery / management• Job managements• Security• Delegation• Monitoring• Communication• Interoperability

http://grid.rit.edu 2

Comparison to OSGrid Requirements Grid OS

Resource sharing/coordination

GIIS, GRRP Main memory (heap area)

Resource discovery/management

GRAM File systems

Job management Queuing Systems IR (Instruction Register), process scheduling

Security GSI Permissions

Monitoring GMA Standard output

Communication via network systems Buses on computer board

http://grid.rit.edu 3

Needs for Grid Application Developments

• Standardized mechanisms• APIs / SDKs for Grid protocols

http://grid.rit.edu 4

Grid Toolkits

Grid Toolkits Category

Globus Toolkit Grid Computing

Condor Cluster Computing

Condor-G Grid Computing

BOINC Public-Resource Computing

Xgrid Cluster Computing

http://grid.rit.edu 5

Globus Toolkit

• Open-source project• Software toolkit for Grid applications• Being developed by the Globus Alliance• www.globus.org

http://grid.rit.edu 6

Globus Toolkit Requirements

• Resource location & allocation• Communications• Unified resource information service• Authentication interface• Process creation• Data access

http://grid.rit.edu 7

Reference: from the slide “Globus: A Metacomputing Infrastructure Toolkit.” p.12.

Grid Architecture & Globus: Fabric Layer

http://grid.rit.edu 8

Access to the resources Computational resources Storage resources Network resources

Globus uses existing fabric components If they do not exist, GT

provides the missing components.

Grid Protocol Architecture

Collective

Resource

Connectivity

Fabric

Application

Grid Architecture & Globus: Connectivity Layer - 1

http://grid.rit.edu 9

Defines communication

& authentication protocols Transport, routing, naming Verify users & resources

Globus Toolkit : Grid Security

Infrastructure (GSI) Public-key based X.509

Grid Protocol Architecture

Collective

Resource

Connectivity

Fabric

Application

Grid Architecture & Globus: Connectivity Layer - 2

• Grid Security Requirements– Single sign on– Delegation– Integration with various local security solutions– User-based trust relationships

http://grid.rit.edu 10

Grid Architecture & Globus: Resource Layer

http://grid.rit.edu 11

Resource managements Information protocols Management protocols

(process creation,

data access) Globus Toolkit

Grid Resource Information

Protocol (GRIP) Grid Resource Access

and Management (GRAM) GridFTP Lightweight Directory

Access Protocol (LDAP)Grid Protocol Architecture

Collective

Resource

Connectivity

Fabric

Application

Grid Architecture & Globus: Collective Layer

http://grid.rit.edu 12

Access to “global” resources Resource discovery Task scheduling Monitoring Authorization

Globus Toolkit: Meta Directory

Service Grid Information Index Services (GIISs) Grid Resource Registration Protocol (GRRP) Grid Protocol Architecture

Collective

Resource

Connectivity

Fabric

Application

Condor

• Cluster computing toolkit• Developed at University of Wisconsin• Runs on various Operating Systems• Provides

– Job queuing– Scheduling policy (job allocation, migrations, etc.)– Resource monitoring– Resource management

http://grid.rit.edu 13

Condor Architecture

• A central manager– Detects idle machines– Matches job

requirements to available resources

• Submit machines– Only submit jobs

• Full Install machines– Submit & run jobs

http://grid.rit.edu 14

Cluster

Central Manager

Submit

Full Install

Full Install

Submit

Full Install

Condor - Flocking

http://grid.rit.edu 15

Cluster A Cluster B

Submit JobRun Job

No available resourcesin cluster A

Send the jobto cluster B

Run the job in cluster B

Condor-G

• Grid computing• Using the Globus Toolkit• Can access a Globus Grid

http://grid.rit.edu 16

Xgrid• A Mac OS X

application• Security• Extendability

– A plug-in architecture

• Ease of use

http://grid.rit.edu 17

http://www.macresearch.org/xgrid-leopard-good-bad-ugly-and-new-stuff

How Xgrid is used

• Ad hoc participation– Programs are run at idle computers

• Dedicated Grid– Computers are dedicated for programs

http://grid.rit.edu 18

Xgrid Architecture

• Agent– Runs a job (dedicated mode, screensaver mode)

• Controller– Queues tasks, scheduling, handles failover

• Client– Submits jobs to the controller

http://grid.rit.edu 19

Xgrid Architecture

• Plug-ins– For specific functionalities

• Networking– BEEP (The Blocks Extensible Exchange Protocol)

http://www.beepcore.org/• Security

– Xgrid requires a password– MD5 hash

http://grid.rit.edu 20

BOINC

• Stands for Berkeley Open Infrastructure for Network Computing• Public-resource computing

– Also known as peer-to-peer computing• Different from Grid computing

– Grid: resources are owned by organizations (universities, research labs, etc.)– Public-resource: resources are owned by individuals

http://grid.rit.edu 21

BOINC - participants

• Give “credits” to participants– Need participants to make a more capable system

• Meter contributions (computation, storage, network transfer, etc.)• Participants are interested in their rankings

http://grid.rit.edu 22

BOINC - Examples

• SETI@home• Predictor@home• Folding@home• Climateprediction.net• Climate@home• CERN project• Einstein@home• UCB/Intel study of Internet resources

http://grid.rit.edu 23

BOINC - Features

• Components– A master URL, scheduling servers, data servers– Tools (Python scripts, C++ interfaces)

• Redundant computing– Detects errors and compute again

• Failure / backoff– Congestion control

• Local scheduling

http://grid.rit.edu 24

References• Globus Website: www.globus.org• Presentation Slides, The Globus Project TM, “The Grid and Globus.” Argonne

National Laboratory, JSC Information Sciences Institute. users.sdsc.edu/~ludaesch/ECS289F-W05/ECS289F-W05-16-globus.pdf

• Presentation Slides, Harhad, F, “Globus: A Metacomputing Infrastructure Toolkit.” International Journal of Supercomputer Applications, 11(2):115-128, 1997. www.cct.lsu.edu/~kosar/csc7700/slides/Lecture05a.pdf

• Foster, I., Kesselman, C. and Tuecke, S. “The Anatomy of the Grid: Enabling Scalable Virtual Organizations.” International Journal of High Performance Computing Applications, 15 (3). 200-222. 2001.

• Habib, I. “Getting started with condor.” Linux J., 2006(149), 2. Website: http://www.linuxjournal.com/node/9058/print

• Condor Website: http://www.cs.wisc.edu/condor/• Anderson, D.P. “BONIC: a system for public-resource computing and storage.” Grid

Computing, Proceedings. Fifth IEEE/ACM International Workshop on, 4-10. 2004.• “Xgrid Guide” Apple Computer, Inc. 1 Infinite Loop, Cupertino, CA 95014. March

17. 2004.

http://grid.rit.edu 25