Overview of Grid Computing EnvironmentsProposed GGF Information Document

G.Fox, D. Gannon, M. Pierce, M. Thomas

PTLIU Laboratory for Community GridsGeoffrey Fox

Computer Science, Informatics, Physics

Indiana University, Bloomington IN 47404

http://grids.ucs.indiana.edu/ptliupages

Source of Report• Last Half of 2001: Call from GCE RG for Papers for

Journal special issues – 28 papers submitted and reviewed

• Published in Concurrency and Computation: Practice and Experience Vol. 14, Grid Computing Environments Special Issue 13-15, Fall 2002

• http://aspen.ucs.indiana.edu/gce/gce2001index.html• Augmented by chapters (about 14) in Fran Berman,

Geoffrey Fox and Tony Hey, ‘Grid Computing: Making the Global Infrastructure a Reality’, ISBN 0-470-85319-0, John Wiley & Sons Ltd, Chichester, 2003. See http://www.grid2002.org

• Other key papers which we knew about

Raw (HPC) Resources Database

AggregationPortal

SystemServices

SystemServices

SystemServices

Application Service

PortalServices

PortalServices

GridComputing

Environments

UserServices

“Core”Grid

Application Service

OGSA (OGSI) Interfaces

The fuzzy definition of Grid Computing Environments

Two Major Areas• “Programming the User Side of the Grid”

which involves discussing computing model for Grid

• Controlling user interaction – rendering any output and allowing user input in some (web) page. – This includes aggregation of multiple data

sources in a single portal page (Jetspeed). – Has natural overlaps with information

(commercial) portals

Classification of GCE Papers in Programming the User Side of the Grid

• Technology for building GCE systems -Interface with backend Infrastructure e.g. Community Grid Kits, GPDK

• Problem Solving Environments– Domain specific collection of tools and user interface. E.g. XCAT,

Polder, SCIRun, Astrophysics Collaboratory

• GCE Tools– Support parameter sweep, visualization, job status, files, security,

workflow ..

• GCE Shell Portals providing a general interface to many Grid capabilities– Analogous (not usually command line) to role of UNIX shell providing

access to UNIX tools and user programs, files …– Note UNIX has core system and higher level tools accessed by Shell– E.g. Unicore, Hotpage, Mississippi Portal– PSE’s often built on top of GCEShell portals

Aspects of Programming Model I• See review of programming model by GGF APM RG

– Chapter 21 of Wiley Book and web page

• Handling of the basic components of a distributed computing system – files, computing and data resources, programs, and accounts. – The GCE will typically interface with an environment like Globus or a

batch scheduler like PBS to actually handle the back-end resources. – However the GCE will present the user interfaces to handle these

resources.

• We can follow the lead of UNIX (and Legion) and define a basic GCEShell providing access to the core distributed computing functions. – JXTA also builds some Grid-like capabilities with a modification of UNIX

shell model. – GCEShell can have a command line or more visually appealing

graphical user interface.

Implications of 3-Tier Model• The 3-tier model implies that any given capability (say

run a matrix inversion program) can appear at multiple levels. – Maybe there is a backend parallel computer running an MPI

job; this is front-ended perhaps as a service by some middle-tier component running on a totally different computer, which could even be in a different security domain.

– One can “interact” with this service at either level; a high performance I/O transfer at the parallel computing level and/or by a slower middle-tier protocol like SOAP at the service level.

– These two (or more) calls (component interactions) can represent different functions or the middle tier call can be coupled with a high performance mirror.

• Typically the middle tier provides control and the back end “raw data transfer”.

Raw (HPC) Resources Database

AggregationPortal

SystemServices

SystemServices

SystemServices

Application Service

PortalServices

PortalServices

GridComputing

Environments

UserServices

“Core”Grid

Application ServiceApplication Metadata

Actual Application

OGSA (OGSI) Interfaces

The fuzzy definition of Grid Computing Environments

Technology for building GCE Systems

• CoG Kits for Java Perl Python CORBA ….

• GPDK extending CoG kits with JavaBean and JSP middleware

• Grid Portal Toolkit

• Other such interfaces with also C and XML tools

• Event Service

• Simulations

GCEShell Portals and PSE’s• One can divide GCE capabilities into

generic (copy file) or domain specific (generate the multigrid solver suitable to simulate Middle Earth)

• Correspondingly one finds portal frameworks or GCEShell Portals presenting general capabilities

• Grid or web-based Problem Solving Environments optimized for a particular domain– PSE’s are often (and perhaps should be) built

on top of GCFShell Portals

Typical Layered Architecture

GCEShell Portals

Globus GT2 ServicesManage backend resources

Interface Middleware withbackend resources

Core Middleware Services

Application ServicesProblem Solving

Environments

User Interface-Client

Aggregate component InterfacesJetspeed

GCE Tools• Data Management

– File manipulation in all tiers including client, middleware and back end

• Security– In all tiers and providing interfaces to core Grid

Security

• Workflow or “Programming the Grid”• Grid versions of MPI• Higher-level tools include visualization or

support of parameter sweep applications– UNIX Shell has mix of lower level and higher

level tools

More on Programming Models• Basic 2-level programming model is assumed by

most projects– First you use classic (parallel) programming to produce

“simulation” nuggets (wrapped as application web services)

– JDBC / OGSA-DAI to package data resources as objects or services

• Then you need to compose (orchestrate) the control and dataflow between nuggets – Many different models for how this is to be done and can

call this workflow – Next thrust of GCE RG?

• Some work on optimization between levels as in GrADS project

Research in “2-level” Programming Models• Basic user interface to middle tier proxies controlling

backend (software) resources• Component Models like ICENI (Imperial College) or

DoE Common Component Architecture• Commercial workflow engines as in BPEL4WS• Scripting front-ends as in Matlab or Python• Network server model as in NetSolve or Ninf• Computational Economies• Semantic Grid technologies (ontology based

integration of resources) as in MyGrid or DiscoveryNet• Agents?

Programming Infrastructure (Hosting Environment)

• Different approachs assume different run-time (implementation) and user programming “ether”

• BPEL4WS thinks about specifying interactions between components; Matlab interface invokes components from a script

• Java Interface to OGSI WG emphasizes that we don’t have an established implementation even if you pick a language– Jini– JXTA– Servlet– Enterprise JavaBeans EJB– Or perhaps some future SJB (Scientific JavaBean)

supporting high volume dataflow better than EJB (short transactions)

Web Services as a Portlet• Each Web Service naturally

has a user interface specified as “just another port” – Customizable for universal access

• This gives each Web Service a Portlet view specified (in XML as always) by WSRP (Web services for Remote Portals)

• So component model for resources “automatically” gives a component model for user interfaces– When you build your

application, you define portletat same time

Application orContent source

WSDL

Web Service

S

R

W

P

Application as a WSGeneral Application PortsInterface with other WebServices

User Face ofWeb ServiceWSRP Ports define WS as a Portlet

Web Services have other ports (Grid Service) to be OGSI compliant

Online Knowledge Center built from Portlets

• Web Services provide a component model for the middleware (see large “common component architecture” effort in Dept. of Energy)

• Should match each WSDL component with a corresponding user interface component

• Thus one “must use” a component model for the portal with again an XML specification (portalML) of portal component

A set of UIComponents

Content Provider

WSDL

Web Service

F

I

U

O

F

I

R

O

PortalAggregate

WS-User Facing Fragments

Render

Other WSUser FacingPorts

Other WSResource orService-facingPorts User-facing

Ports

User Facing Ports for Web Service

UserProfile

Application orContent source

WSDL

Web Service

F

I

U

O

F

I

R

O

Render

Portal(Aggregator)

Selector

Filter

Control Channel

Customized View

SelectionView

Control Channel

Customized View

CustomizedUser-Facing

PortsAs used in Universal Access

PortletPortlet PortletPortlet PortletPortlet PortletPortlet

XMLRSS, OCS, or otherLocal or remote

HTMLLocal files

JSP or VMLocal templates

WebPageRemote HTML

PortletPortlet

PortletsUser implementedusing Portal API

Portlets

Data

PortletController PortletController

Screen Manager

HTML

PSML

PortletControl

ECS

JSP template

ECS ECS ECS ECS

ECS ECS ECS

ECS Root to HTML

ECS

Turbine ServletJetspeedArchitecture

Portlets and Portal Stacks• User interfaces to Portal

services (Code Submission, Job Monitoring, File Management for Host X) are all managed as portlets.

• Users, administrators can customize their portal interfaces to just precisely the services they want.

Core Grid Services

User facing Web Service Ports

Application Grid Web Services

Aggregation Portals(Jetspeed)

Messa

ge S

ecu

rity, In

form

atio

n

Serv

ices

Jetspeed Computing Portal: Choose Portlets

4 available portletslinking to Web ServicesI choose two

Choose Portlet Layout

Choose 1-column Layout

Original 2-column Layout

Lists user files on selected host, noahsark.File operations include

Upload, download, Copy, rename, crossload

Tabs indicate availableportlet interfaces.

File management

Sample page with several portlets:

proxy credential manager,submission, monitoring

Provide information about application

andhost parameters

Select applicationto edit

Administer Grid Portal