Perspectives on Grid Technology Ian Foster Argonne National Laboratory The University of Chicago

Perspectives onGrid Technology

Ian FosterArgonne National Laboratory

The University of Chicago

Ian Foster ARGONNE CHICAGO

Background: Why “Grids”?

Because the resources needed to solve complex problems are rarely colocated Advanced scientific instruments Large amounts of storage Large amounts of computing Groups of smart people

For a variety of reasons Resource allocations not optimized for 1 appln Required resource configurations change Different views of priorities and truth


GridApplicationExamples

Teleimmersion/distancecollaboration

Record-setting distributedsupercomputing

TransAtlantic remotevisualization/steering

Parameter studies withdeadline scheduling

Online analysis ofinstrument data


Hence, Grids

Next-gen infrastructure Advanced network services Computers, storage, etc. New tools, methodologies

Foundation for advanced network applications, e.g. Data-intensive (Data Grid) Collaborative (Access Grid) Compute-intensive Online instrumentation

Akamai’s server network (Jan 2000)


Grid R&D: A Brief History

Late 80s/early 90s: Isolated experiments Gigabit testbeds, metacomputing expts

Mid 90s: first attempts at integration E.g., 1995 I-WAY and I-Soft software

Late 90s: emergence of infrastructure and identification of new usage modalities Globus toolkit, Access Grid, Data Grid Grid Forum, production Grid infrastructures

2000+: Grids go mainstream “Grid services” integrated into network


Creating a Usable Grid :Grid Services (“Middleware”)

Standard grid services that Provide uniform, high-level access to a wide

range of resources (including networks) Address interdomain issues of security,

policy, etc. Permit application-level management and

monitoring of end-to-end performance Middleware-level and higher-level APIs and

tools targeted at application programmers Map between application and Grid


Grid Services Architecture:An Emerging Grid Computing Framework

Archives, networks, computers, display devices, etc.;associated local services

Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.

GridFabric

GridServices

ApplnToolkits

Applns

...

… a rich variety of applications ...

Remoteviz

toolkit

Remotecomp.toolkit

Remotedata

toolkit

Remotesensorstoolkit

Async.collab.toolkit


Remotecomp.toolkit

GridFabric

Remotedata

toolkit

Archives, networks, computers, display devices, etc.;associated local services

Remote datafor climate

Data GridToolkit

Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.

GridServices

ApplnToolkits

Applns

...Remotesensorstoolkit

Async.collab.toolkit

Authentication ReservationInformation Fault detection

Res. mgmt Accounting Instrumentation ...

Remoteviz

toolkit

Remote vizfor CFD

Grid MPI

Co-allocator

Advantages of GSA Single infrastructure Avoid redundant

development Encourage code

sharing


National and International Grid Testbeds

I-WAY

NASA’s Information Power Grid

NSF PACI’s National Technology Grid


European Grid Testbeds

www.egrid.org


http://www.globus.org



A Current Focus: Data Grids

Integrate data archives & computers into a distributed data management & analysis “Grid”

More than storage, computing, network: also Caching and mirroring to exploit locality Intelligent scheduling to determine appropriate

replica, site for (re)computation, etc. Coordinated policy-driven resource management

for performance guarantees Embedded security, policy, agent technologies for

effective distributed analysis


Grid Services and the Data Grid:Resource Management Architecture

“10 GFlops, EOS data,20 Mb/sec -- for 20 mins”

GridInformation

Service

GRAMGRAMGRAM

ResourceBroker

Info service:location + selection

ResourceManagers

GRAM

ForkLSFEASYLLCondoretc.

“What computers?”“What speed?”“When available?”

“50 processors + storage from 10:20 to 10:40 pm”

“20 Mb/sec”


Scheduling Bulk Transferand High-Priority Transfers

0

2000

4000

6000

8000

10000

12000

0 50 100 150 200 250

Time (seconds)

Kb

yte

/se

c

background

foreground

competitive



Grids and HEP Computing

HEP computing: a Grid project par excellence Tight integration of computing, storage, networking;

demanding requirements Focus on services for a large community

HEP computing brings new problems, e.g. Big increase in scale Object database technology Complex policy issues

HEP also contributes interesting technologies E.g., MONARC simulator, NILE, GIOD


Summary

Grids promise to enable qualitatively new approaches to science and engineering

Key enabler is the integration of advanced resources with new Grid services and tools

To date, wonderful application demonstrations, considerable progress in key technologies, some early attempts at real deployment

Timely for scientific communities such as HEP to investigate and apply for real


For More Information ... Globus: www.globus.org (European) Grid Forum:

www.gridforum.org

www.egrid.org Grid book

“The Grid: Blueprint for a Future Computing Infrastructure,” I. Foster & C. Kesselman (Eds), Morgan-Kaufmann, 1999

http://www.mkp.com/grids

Documents

Perspectives on Grid Technology Ian Foster Argonne National Laboratory The University of Chicago