1.1 Introduction to Grid Computing ITCS 4010 Grid Computing, 2005, UNC-Charlotte, B. Wilkinson

1.1

Introduction to Grid Computing

ITCS 4010 Grid Computing, 2005, UNC-Charlotte, B. Wilkinson.

1.2

Need to harness computers

Original driving force behind grid computing the same as behind the early development of networks that became the Internet:

– Connecting computers at distributed sites for high performance computing.

Just as the Internet has changed, grid computing has changed.

1.3

History

• Began in mid 1990’s with experiments using computers at geographically dispersed sites.

• Seminal experiment – “I-way” experiment at 1995 Supercomputing conference (SC’95), using 17 sites across the US running:– 60+ applications.– Existing networks (10 networks).

1.4

1995 2000 200519901985

Distributed computing

Remote Procedure calls (RPC)Concept of service registry

Beginnings of service oriented architecture

Object oriented approachesJava Remote Method Invocation (RMI)

CORBA (Common Request Broker Architecture)

Cluster computing

Software Techniques:

Computing platforms:

Parallel computers

Geographically distributed computers (Grid computing in the broadest sense)

Web services

SC’95 experiment

1.5

Grid Computing

• Using distributed computers and resources collectively.

• Usually associated with geographically distributed computers and resources on a special high speed network, or the Internet.

• Now become much more that last slide suggests.

1.6

Shared Resources

Can share much more than just computers:

• Storage• Sensors for experiments at particular sites• Application Software• Databases• Network capacity, …

1.7

Computational Grid Applications

• Biomedical research

• Industrial research

• Engineering research

• Studies in Physics and Chemistry

1.8

Sample Grid Computing Projects

Physical Sciences:• Large Hadron Collider project (CERN)• DOE Particle Physics Data grid• DOE Science grid• AstroGrid• Comb-e-Chem project

Natural and Life sciences:• Protein Data grid• Mcell project

Engineering Design:• Distributed Aircraft Maintenance Environment• NASA Information Power grid

1.9

Science Today

is a Team Sport

I. Foster

1.10

eScience

eScience [n]: Large-scale science carried out through distributed collaborations—often leveraging access to large-scale data & computing

I. Foster

NSF Network for Earthquake Engineering Simulation (NEES)

Transform our ability to carry out research vital to reducing vulnerability to catastrophic earthquakes

I. Foster

Global Knowledge Communities: e.g., High Energy Physics

I. Foster

1.13www.earthsystemgrid.org

DOE Earth System Grid

Goal: address technical obstacles to the sharing & analysis of high-volume data from advanced earth system models

I. Foster

1.14

Earth System Grid I. Foster

1.15

TeraGridFunded by NSF in 2002 to link 5 supercomputer sites

with 40 Gb/s links

1.16

TeraGrid

1.17

Grid networks for collaborative grid computing projects

Grids have been set up at the local level, national level, and international level throughout the world, to promote grid computing

1.18

CiscoEPA

North Carolina’s Foundation for Grid: NCREN

4-7 MCNC-owned Clusters distributed throughout the stateLocations still under evaluation

Internet Internet 2

NLR

Internet Internet 2

NLR

InternetInternet

Existing: Blend of owned and leased fiber and circuits moving toward resilient rings powered by Cisco routers

Planned: Strong focus on owned and leased fiber, Lambda, and few circuits, in resilient rings powered by Cisco routers and Wave Division Multiplexers

Close to home:

From “Grid Computing in the Industry” by Wolfgang Gentzsch, presentation to Fall 2004 grid computing course. Full set of slides on course home page.

1.19

Grid2003: An Operational National Grid28 sites: Universities + national labs2800 CPUs, 400–1300 jobsRunning since October 2003Applications in HEP, LIGO, SDSS, Genomics

Korea

http://www.ivdgl.org/grid2003From “A Grid of One to a Grid of Many,” Miron Livny, UW-Madison, Keynote presentation, MIDnet conference, 2005.

1.20

National GridsMany countries have embraced grid computing and set-up grid computing infrastructure:

• UK e-Science grid• Grid-Ireland• NorduGrid• DutchGrid• POINIER grid (Poland)• ACI grid (France)• Japanese grid• etc, etc., …

1.21

UK e-Science Grid

1.22

Resource sharing and collaborative computing

• Grid computing is about collaborating and resource sharing as much as it is about high performance computing.

1.23

Virtual Organizations

Grid computing offers

potential of virtual organizations:– groups of people, both geographically and

organizationally distributed, working together on a problem, sharing computers AND other resources such as databases and experimental equipment.

• Crosses multiple administrative domains.

1.24

Applications

• Originally e-Science applications– Computational intensive

• Not necessarily one big problem but a problem that has to be solved repeatedly with different parameters.

– Data intensive.– Experimental collaborative projects

• Now also e-Business applications to improve business models and practices.

1.25(Based on a slide from HP)

Utility ComputingOne of Several Commercial Drivers

shared, traded resources

value

clusters

grid-enabled systems

programmable data center

virtual data center

Open VMS clusters, TruCluster, MC ServiceGuard

Tru64, HP-UX, Linux

switchfabriccompute storage

UDC

computing utility

or

GRID

today

• Utility computing• On-demand• Service-orientation• Virtualization

I. Foster

1.26

Grid Computing Software Infrastructure

1.27

Globus Project

• Open source software toolkit developed for grid computing.

• Roots in I-way experiment.• Work started in 1996. • Four versions developed to present time.• Reference implementations of grid computing

standards.• Defacto standard for grid computing.

1.28

• GSI (Grid Security Infrastructure)– Grid security.

• MDS (Monitoring and discovery Service)– Interface to system and service information.

• GRAM (Grid Resource Allocation Manager) – Remote job submission and control.

• GridFTP– Secure data transfer.

GlobusKey Components

1.29

From “Globus Toolkit 4 Tutorial,” MCNC Jan-Feb, 2005, Pawel Plaszczak and Bogdan Lobodzinski, Gridwise Technologies.

2. discover resource

3. submit job

4. transfer data

1. secure environmentGSI

GRAM

MDS

GridFTP

1.30

Globus Toolkit: Recent History

• GT2 (2.4 released in 2002)– GRAM, MDS, GridFTP, GSI.

• GT3 (3.2 released mid-2004): redesign– OGSA (Open Grid Service Architecture)/OGSI (Open

Grid Services Infrastructure) based.– Introduced “Grid services” as an extension of web

services.– OGSI now abandoned.

• GT4 (release for April 2005): redesign– WSRF (Web service Resource Framework) based.– Grid standards merged with Web services.

1.31

Supercomputing 2003 Demonstration

• We* used Globus version 2.4 in a Supercomputing 2003 demo organized by the University of Melbourne.

• 21 countries involved, numerous sites.

* The Grid group at WCU.

1.32

1.33

A re-implementation based upon the Open Grid Service Architecture (OGSA) standard.

• We used version 3.2 for the Fall 2004 grid computing course.

• Underlying implementation of version 3.x used OGSI Open Grid Service Infrastructure), which was not embraced by the community.

Version 3

1.34

Version 4

• Released April 2005.

• OGSA kept but OGSI abandoned in favor of new implementation standards based around pure web services.(Version 3 used “extended” web services)

• To be used in this course, with other software.

1.35

Interconnections and Protocols

Focus now on:

• using standard Internet protocols and technology, i.e. HTTP, SOAP, web services, etc.,

1.36

Web Services-Based Grid Computing

• Grid Computing now strongly based upon web services.

• Large number of newly proposed grid computing standards:– WS-Resource Framework (WSRF)– WS-Addressing– etc., etc. …. .

1.37

There will be several multiple-choice quizzes in the course (on-line through WebCT).

Quiz

Question: What is a virtual organization?

(a) An imaginary company.(b) A web-based organization.(c) A group of people geographically distributed that come

together from different organizations to work on grid project.

(d) A group of people that come together to work on a virtual reality grid project.

1.38

Reading Assignments

Since there is no assigned textbook, there will be a few reading assignments.

Purpose is to consolidate your understanding.

Materials posted on home page from link “reading/on-line materials” at end of section on lecture materials.

1.39

First Reading Assignment

"The Anatomy of the Grid: Enabling Scalable Virtual Organizations"

by

I. Foster, C. Kesselman, and S. Tuecke

Int. J. Supercomputer Applications, 2001.

1.40

AcknowledgementSlides numbers marked with “I. Foster” have been

selected from presentations made by Ian Foster:• Enabling eScience: Grid Technologies Today &

TomorrowAmerican Association for the Advancement of Science Annual Meeting, Washington, DC, February 21 2005.

• Globus: Bridging the GapKeynote Talk, GlobusWORLD, Boston, Mass., February 8, 2005.

• The Grid: Reality, Technologies, ApplicationsDistinguished Lecture, McGill University, Montreal, Canada, January 21 2005.

used for educational purposes only.