Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab’s GAT Grid Application Toolkit Jarek Nabrzyski et al. [email protected] Poznań Supercomputing

Crossgrid Meeting in Poznań, 30 July 2003Jarek Nabrzyski

GridLab’s GAT

Grid Application Toolkit

Jarek Nabrzyski et al.

[email protected]

Poznań Supercomputing and Networking Center


Overview

GridLab Intro

Goals

Architecture

API

Current status

Roadmap

Some success stories


GridLab Project

Funded by the EU (5+ M€), January 2002 – December 2004Application and Testbed oriented

Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled

Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools...:

resource management (GRMS), data management,monitoring, adaptive components, mobile user support, security services,portals,

... and test them on a real testbed with real applications


GridLab Members

PSNC (Poznan) - coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ. (Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens)

Sun MicrosystemsCompaq (HP)

ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny)

collaborating with:Users!

EU Astrophysics Network,

DFN TiKSL/GriKSL

NSF ASC Project

other Grid projectsGlobus, Condor,

GrADS,

PROGRESS,

GriPhyn/iVDGL,

Most of the other European Grid Projects (GRIDSTART)

GWEN


GridLab Aims

Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community), all the other potential grid apps

Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations

Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.


What GridLab isn’t

We are not developing low level Grid infrastructure,Addressing Grids and P2P

We do not want to repeat work which has already been done (want to incorporate and assimilate it …)

Globus APIs,

OGSA,

ASC Portal (GridSphere/Orbiter),

GPDK,

GridPort,

DataGrid,

GriPhyn,

...


GridLab End User Requirements

Application oriented environment,

Applications on one or more virtual organisations,

Flexible, easy-to-use, simple interfacesresources, jobs, and data (including compiling, tracking jobs, cataloguing data),

Means to make efficient and effective use of resources,

Robustnesssmart adaptivity, complete control and fail safety are available on all levels,

The ability to work in a disconnected environment,

Mobile working,


What do our users want?

Larger computational resources Memory/CPU

Faster throughputCleverer scheduling, configurable scheduling, co-scheduling, exploitation of un-used cycles

Easier use of resourcesPortals, grid application frameworks, information services, mobile devices

Remote interaction with simulations and dataNotification, steering, visualization, data management

Collaborative toolsNotification, visualization, video conferencing, portals

Dynamic applications, New scenariosGrid application frameworks connecting to services


GridLab end user requirements

From laptops to fully deployed Virtual Organisations,

Complexity hidden as much as possible,

Collaborative infrastructure,

The infrastructure for all classes of applications

The infrastructure must provide capabilities to customise choice of service implementation (e.g. using efficiency, reliability, first succeeding, all)


Application Scenarios

Dynamic Stagingmove to faster/cheaper/bigger machine

Multiple Universecreate clone to investigate steered parameter

Automatic Convergence Testingfrom initial data or initiated during simulation

Look Aheadspawn off and run coarser resolution to predict likely future

Spawn Independent/Asynchronous Tasks

send to cheaper machine, main simulation carries on

Application Profilingbest machine/queuechoose resolution parameters based on queue

Dynamic Load Balancinginhomogeneous loads

multiple grids

Portal User/virtual organisation interface to the grid.

Intelligent Parameter Surveysfarm out to different machines

Make use ofRunning with management tools such as Condor, Entropia, etc.

Scripting thorns (management, launching new jobs, etc)Dynamic use of eg MDS for finding available resources


Motivation for GAT

Why do applications need a framework for using the Grid?

Our application developers need a layer between applications and grid infrastructure:

Higher level than existing grid APIs, hide complexity, abstract grid functionality through application oriented APIs

Insulate against rapid evolution of grid infrastructure

Choose between different grid infrastructures

Make it possible for grid developers to develop new infrastructures

Make it possible for application developers to use and develop for the grid independent of the state of deployment of the grid infrastructure


Solution...

GAT – a layer between apps and emerging grid technologies

GridLab testbed/VO

Close cooperation between developers and deployers

End Users

GAT Tool Developers

Grid Infrastructure Developers

GAT-APIDevelopers


GridLab Architecture


GridLab services

Software environment for Grid-enabling scientific applications

GridLab services, third party services and various core-grid services will be supported by GAT

In the advent of the Open Grid Service Architecture (OGSA), GridLab's architecture will revolve around the notion of services,

all the GridLab services will be OGSA compliant

currently all the services are Web Services based

roadmap for Web Services to OGSA transformation is being prepared (3-6 months from now)


GridLab services

A primary aim of this project is to produce a GridLab GAT containing a set of high quality services which provide a complete environment for Grid- enabling generic applications

GridLab services:implement common (strict) security

use common service conneciton protocols (WSDL/OGSA)

are built primarily for Globus infrastructure


What are the GL services?

Authorisation Service (WP6)

Adaptive Services (WP7)

Data Management Services (WP8)

Resource Management System (GRMS) (WP9)

Information Services (WP10)

Monitoring Services (WP11)

Mobile User Support (WP12)


Security (WP6)

Security WP focuses right now on the Authorization Service (AS)

The main requirement is flexibility

The AS is about to provide universal way of defining security policy for the whole Grid, independent of technologies used at lower levels

It should be able to implement most security models for Grids and use many different scenarios at the same time


Adaptive Components (WP7)

Adaptive Components Service (ACS) and the Local Adaptive Components (LAC). ACS provides an interface to query the adaptive system. It currently supports calls to:

rank resourcesestimate transfer timeestimate usage (of some given metric)

LAC uses the monitoring system (shown in blue), to continuously collect data about the resource and applications running on it (load information, queue lengths, network bandwidth to other machines, etc.).


Data Management Services (WP8)

replica catalog prototype was ready at Zakopane meeting

data movement/copy service also since Zakopane meetingsupports reliable gridftp file transfer

is gsi enabled with authentication and delegation

scalable and fault-tolerant replica catalog in M12, based on ongoing research at ZIB

SoapSoap

SoapSoap

Host A

Host B

Host C

Host D

automatic load-balance, fail-over between replica catalogs

external access via SOAP and OGSA

internal communication via more efficient protocol (Corba)


Unicore

Resource Management (WP9)

Job Receiver

Jobs Queue

BrokerExecution

Unit

ResourceDiscovery

Scheduler

ResourceReservation

PredictionUnit

File TransferUnit

MDSData

Management

ExecutableRepository

GRAM

AuthorizationSystem

Adaptive

WorkflowManager

QoS/SLANegotiation

GRMS

Third partysystems

Chimera DataGrid Crossgrid ...


GridLab and Condor


GridLab and GriPhyN


Information Services (WP10)

Client

Information Service(OGSA)

SOAP over GSI

Users

Software

Services

Firewall

V.O.

C. A.

Cluster

Job Queues

MDS

GSI-SASL


Monitoring System (WP11)

• Implements GGF’s GMA architecture• Fast and robust• Small resource usage• Can monitor hosts and jobs• Can deliver event notifications


Mobile User Support (WP12)

Grid Services

Network Environment / Grid

Portal

Applications

User

Mobile device


The Grid is complex …

Monitoring

Resource Management

InformationSecurity

DataManagement

GLOBUS

ApplicationManager

Logging

NotificationMigration

Profiling

SOAP WSDL Corba OGSA Other

Other GridInfrastructure?

Application

“Is there a better resource I could be using?”

UNICORE


…need to make it easier to use

GAT

Application

“Is there a better resource I could be using?”

GAT_FindResource( )

The Grid


Grid Application Toolkit

The GAT provides functionality through a carefully constructed set of generic high-level APIs, through which an application will be able to call the underlying grid services,

Set of application developer APIs for Grid tools, services and software libraries, (and example implementations) that support the development of grid-enabled applications (open source!)

Usable from any high level “application” (any generic code, Cactus, Triana, Portals, Scripts, …)


GAT: AIM

Abstract Grid capabilities (services) from the application developer.

Application developer concentrates on the functionality as needed by the application.

Hide complexity.

Provides a layer (buffer zone) between applications and the Grid.


GAT Goals

The GAT provides an API and an associated set of tools which enable end-users and application developers to make easy and flexible use of the Grid,

The infrastructure, and in particular the GAT, must allow developers to develop their applications independently of the deployment of grid services,

Users must be able to make use of such applications in the absence of a fully-deployed infrastructure.


Cactus/GAT Integration

GATLibrary

Cactus Flesh

Thorn

CGATThorn

Thorn

Thorn

Thorn

Thorn

Physics and Computational Infrastructure

Modules

Cactus GAT wrappers Additional

functionalityBuild system

GridLab Service

GridLab Service


Gridlab Services JXTA Web Services

P2PSubset

ProtolibJNI

P2PS Comms

GAP & GAT – P2P

GAT P2PS

GAP Layer

Network

Network NS-2

JXTAServe

GAT

WServe

Gridlab Services

Other Services


GAT Architecture


GAT Engine


API Call


Full API Call


GAT Init


GAT Adaptor

Interface between GAT Engine and one or more capabilitiesTranslates user requests to appropriate interface syntax for a capability providerActive adaptors change dynamicallyIncludes “security context”Return appropriate error codes

ExamplesOGSA adaptor (provides many capabilities)Globus adaptor (directly talk to gatekeepers)Adaptors for each GridLab service provider“Local” adaptors (GAT_MoveFile => “cp”, GATFindResource => “localhost”)


The Same Application …

Application

GAT

Application

GAT

Application

GAT

Laptop The GridSuper Computer

No network! Firewall issues!


Adaptor Call


Philosophy

Application makes GAT API calls for operations which may be Grid-related.

Application links agains the GAT Engine

Application runs irrespective of actual underlying infrastructure deployment

Engine loads adaptors which are valid in the environment extant when the application starts

Adaptors try to do Grid operations on request, on failure another adaptor provided function may be called.

Application can thus be compiled, linked and tested without any Grid services

Same application executable can run in a full Grid enviroment.


Philosophy

The GAT uses whatever underlying Grid infrastructure there is and that people have developed adaptors for,

GAT is not about replacing already developed infrastructure, but instead to provide a simple, clear interface which can be used with many different infrastructures.

Different versions of Globus

Condor

Unicore

...


The GAT Architecture

GAT: Grid Application Toolkit

API and Toolkit for developing portable Grid applications independently of the underlying Grid infrastructure and available services

Implements the GAT-API

Used by applications (different languages)

GAT AdaptorsConnect to capabilities/services

GAT EngineProvides the function bindings for the GAT-API


API Goals

The GAT must support applications written in any language which people write Grid Applications in:

C, C++, Fortran, Java, Perl, Python, ...

The use of the GAT API should be as natural as possible for users of these languages.

It must also not require a steep learning curve to move from the API in one language to the API in another language

APIs in different languages should be as similar as possible


API Roadmap

Examine prototype adaptors and define an API based upon the needs expressed in these. (Done)

Define in a language independent manner

Send out for community review

Review API and identify “objects” which the API operates on. Create revised API specification based upon these objects. (Done)

Still language independent

Use this specification to create language specific bindings.


API Roadmap

Finish API specifications (mostly done, under vendors’s review)

Provide “canonical” adaptor which provides all capabilities by some “local” functionality and can be used to provide a test-suite for the API, the GAT-Engine and adaptors

Examine prototype design and implementation and produce release-quality code (reference C implementation) providing the actual APIs as opposed to the current APIs, and is deployable on all architectures.


API Roadmap

Use SWIG or some other technology to wrap C implementation and provide the APIs in the other languages

Work with Triana and Portal packages to provide a native Java implementation which may be tested against the wrapped reference implementation

Use GAT to Grid-enable as large a class of applications as possible

already in discussions with lots of application groups

Review !Once there is a fully working system we can't just sit back and smile.

There are bound to be problems with provided functionality and there will be feature requests

Need an API committee to control additions


Further Information

Prototype in CVS

Two documentsRequirements

Initial requirements based upon end-user and application developer requirements

http://www.gridlab.org/Resources/Deliverables/D1.1.pdf

Technical SpecificationBased on requirements from all work packages and a technology review

http://www.gridlab.org/Resources/Deliverables/D1.2b.pdf


Further information

Two API specification Documents:

Non object-basedhttp: //www.gridlab.org/WorkPackages/wp-1/Documents/Gridlab-1-GAS-0003.APISpecification.pdf

Object-basedhttp: //www.gridlab.org/WorkPackages/wp-1/Documents/Gridlab-1-GAS-0004.ObjectBasedAPISpecification.pdf

Mailing [email protected]

Open subscription


Migration Scenario

Application migrates beacause of bad performance

The Goal: Involve all the WPs!GAT application

Portal

GRMS

Adaptive

Monitoring

GIS

Mobile user support

Security

Data mgmt

Testbed


Migration Scenario


Migration Scenario


Migration Scenario


Migration Scenario


The „behind the scenes” goal of GL

„Let’s make the most advanced grid in the world” Michael Russell,

AEI

...and that’s what we do ...


Global Grid Testbed Collaboration (GGTC)

Driven by GGF APPS and GridLab testbed and applications

Whole testbed constructed very swiftly (few weeks)

5 continents: North America, Europe, Asia, Africa, Australia

Over 14 countries, including: China, Japan, Singapore, S.Korea, Egypt, Australia, Canada, Germany, UK, Netherlands, Czech, Hungary, Poland, USA

About 70 machines, with thousands of processors (~7500) Many hardware types, including PS2, IA32, IA64, MIPS, IBM Power, Alpha, Hitachi/PPC, Sparc

Many OSs, including Linux, Irix, AIX, OSF, True64, Solaris, Hitachi

Many different organizations (big centers/individuals)

All ran same Grid infrastructure! (Globus)


Global Grid Testbed Collaboration


Bandwidth Challenge:Highest Performing Application

Distributed simulations using Cactus, Globus and Visapult

With John Shalf/LBL and others

16.8 Gigabits/second

scinet.supercomp.org/bwc

Six sites: USA/Dutch/Czech/Poland


TFM

TFM TFM TFM TFM

Task Farming on the Grid

TFM implementedin Cactus

GAT (GRAM, GRMS) used for starting remote TFMs

Designed for the Grid

Tasks can be anything

fork/exec


Task Farming Motivation

Requested by local physics groupParameter surveys, e.g. looking for critical phenomena in gravitational wave collapse by varying amplitude, testing different formalisms of Einstein Equations for evolving same initial data

Scenario is inherently quite robust and fault tolerantGood migration path to the Grid

Start easy (not too much Grid!), task farm across local homogeneous workstations and on single supercomputers.

Use public keys first, then test standard Grid infrastructure

Use of GAT then means users can start testing GridLab services (should still work for them if services not ready)CGAT team can then test real physics runs using wider Grid and GridLab services.


Task Farming on the Grid

Generic Part

Application Specific


Grid-xclock

Simple application for testing and debugging.

xclock is standard X utility, run on any machine with X installed

Requires:

o xclock binaryo X librarieso To display remotely,

need to open outgoing ports from machine it is running on to machine displaying


SC2002, Baltimore

Varied applications deployed of the GGTC testbed

Cactus Black Hole Simulations

ASC GridLab Portal

Smith-Waterman

Nimrod-G

GridLab Task Farming scenario

Visapult

HighlightsGGTC won 2 of the 3 HPC Awards

Won (with Visapult/LBL group) Bandwidth Challenge

$2000 prize money to UNICEF childrens fund


More info / summary

www.GridLab.org

[email protected], [email protected]

[email protected]

Check the GridLab tutorials available at the Web

Bring your application and test it with the GAT and our services!


GridLab Success Stories