GrappaGrid Access Portal for Physics

Applications

Daniel Engh (UC), Shava Smallen(IU), Liang Fang(IU), Jerry Gieraltowski(ANL), Rob Gardner(UC), Dennis Gannon(IU),

Randy Bramley(IU)

CHEP 2003UCSD

March 24-28,2003

Grappa: A Web Portal

GRAM Services-compute sites

GRAM Services-compute sites

GridFTP Services-Athena Libraries-packaged physics apps-data storage

GridFTP Services-Athena Libraries-packaged physics apps-data storage

Web Services-Magda Catalogue-Ganglia Monitor-ATLAS web services

Web Services-Magda Catalogue-Ganglia Monitor-ATLAS web services

Grappa Portal

Browser Command line

The “Grid”

Replica Locator Service

Replica Locator Service

The Grappa portal is a web portal designed to provide to the user an interface that coordinates and integrates the use of a diverse set of web and grid services

Grappa History

• First Design proposals (2001)• Implementation in XCAT Science Portal

(2002)• Demos for ATLAS (2002)

– To U.S. ATLAS grid testbed sites– To joint iVDGL/DataTAG WorldGrid testbed

• IST/SC 2002 demos (Nov 02)• Jetspeed implementation (Mar 03)

Grappa Component Layers• user Xbook (notebook scripts)

– scripts customized for the user (as simple or complex as desired)

– conform to the xbooks API– Athena xbook customized for

job submission– Many scripting languages

supported (Jython, Python, Perl…)

• Xbooks server– A Jetspeed portlet– manages requests from user

xbook• Jetspeed: Apache web portlet

framework• Tomcat server: provides the

web server

Grid ServicesGrid Services

Browser Command line

user Xbook

Portlets (incl. Xbooks Server)

J etspeed

Tomcat Server

Athena Xbook• HTML form interface

– Organizes input parameter selection

– Menu selection of grid services

• Jython scripts– Python syntax– Java toolkits– Java CoG

• Implementation of Globus Client tools

– XCAT technology available• Toolkit to componentize

applications• Implementation of DOE-

CCA spec. as web services• Currently not used

(current demo has only 1 element)

• Athena Xbook customized:– Prestage xbook/CoG

toolkits– Prestage athena libraries– Prestage input data– Submit jobs to GRAM– Transfer results to

storage site– Monitor/cancel/resubmit

jobs.• Submission parameters

managed by xbooks manager

• GRAM IDs managed by Athena xbook

Xbooks Architecture

• The user’s xbook (notebook scripts) interacts with the xbooks server layer – Contacts xbooks server,

registry, and manager– Creates an instance of

the user’s xbook.

Xbooks Manager

• The xbook manager dispatches requests– finds the proper script

engine– Launches script engine

• (e.g. JythonEngine)

– Records submission parameters in xbook repository

– Generates application instance

• (e.g. job submission requests)

Design Factors: Jetspeed

• Apache Jetspeed framework– A large mainstream effort

• Hundreds of developers• Solid support community for portlets

– Portlets interchangeable among projects• Proxy Manager could be same for LCG, Chem portals,

etc.• GridFTP portal• Xbooks server is a portlet, easily integrates user

scripting with wide availability of portlets.

– World moving in direction of Grid/Web services• We want to come along• Don’t want to develop everything ourselves

Design Factors: Java

• Jython chosen as the demo Athena xbook scripting language– Java toolkits provide lots of stuff– Java CoG provides a java implementation of

Globus client tools within the xbook.– Globus 3.0 moves further in the java direction

• Python could be used as well.– pyGlobus CoG available--provides interface to

a local installation of Globus.– Swig to integrate C/C++ toolkits

Design Factors: Portals

• Portals vs. GUIs• Integrate vs. Interface

– Functionality built into integrated portal--- or ---

– Interface with Functionality found in web services?

– Philosophy of distributing functionality as grid components to achieve greater:

• Portability• Scalability• reliability

Status and Future

• Status– Several Demos– Basic job submission and monitoring– Uses standard Jetspeed framework

• Large community• Direction of Grid Components

• Future– Parameter tracking and workflow generation

• Chimera Virtual Data System

– Portlet development and integration– access Physics-apps as Grid Components– Interaction with other portal projects (eg. Ganga)

References• Grappa: grid.uchicago.edu/grappa/• GriPhyN: www.griphyn.org• Python CoG: www.itg.lbl.gov/gtg/projects/pyGlobus/• Jakarta Jetspeed: jakarta.apache.org/jetspeed• Xportlets webpage: www.extreme.indiana.edu/xportlets/project/index.shtml• ATLAS webpage: atlasexperiment.org• Java CoG: Gregor von Laszewski, Ian Foster, and Jarek Gawor. CoG Kits: A Bridge Between

Commodity Distributed Computing and High-Performance Grids. In ACM 2002 Java Grande Conference, June 2000. pp3,22.

• Jython: www.jython.org• XCATSP: Sriram Krishnan, Randall Bramley, Dennis Gannon, Madhusudhan Govindaraju, Rahul

Indurkar, Aleksander Slominski, Benjamin Temko, Jay Alameda, Richard Alkire, Timothy Drews, and Eric Webb. The XCAT Science Portal. Proceedings of Supercomputing 2001. p4.

• XCAT 2.0: Design and Implementation of Component based Web Services. Technical Report 562, Indiana University Computer Science Department, June 2002. p22.

• Chimera-VDS: Ian Foster, Jens Voekler, Michael Wilde, and Yong Zhao. Chimera: A Virtual Data System for Representing, Querying, and Automating Data Derivation. 14th International Conference on Scientific and Statistical Database Management, 2002. p22. www-unix.griphyn.org/chimera/

• XBOOKS: www.extreme.indiana.edu/xbooks• Extreme! Computing Lab www.extreme.indiana.edu

Glossary• Component: A small binary object or program that

performs a specific function. The component is designed to easily operate with other components and applications.

• DOE-CCA: DOE Common Component Architecture, a specification for designing components.

• Chimera-VDS: Virtual Data System developed under GriPhyN, at U.Chicago. A system for tracking file, parameter, and executable dependencies used to construct workflows.

• CoG: Commodity Grid, user interfaces to Globus– Java CoG– Python (pyGlobus)

• Grappa: Grid Access Portal for Physics Applications, an integration of scripting notebooks (Xbooks), Xbooks server portlet, Jetspeed, and Tomcat.

• GriPhyN: Grid Physics Network

Glossary• Jetspeed: Apache portlet framework for designing web services

as interoperable components of a common framework• Jython: Java+Python scripting language. Uses python syntax,

interprets python and java code• Tomcat: Web Server• OGSA: Open Grid Services Architecture (GLOBUS)• Portal or Web Portal: a web site that provides services.• Portlet: Similar to a Portal but reduced in scope and composed

as a web service component to interoperate with other portlets.• RLS: Replica Locator Service (GLOBUS) • SOAP: Simple Object Access Protocol, XML-based messaging

protocol used to encode the information in Web service request and response messages before sending them over a network.

Glossary• Xbooks: Jetspeed Portlet for managing user

scripts (notebooks)• XCAT: Ind.Univ. Extreme! Computing Lab’s

Implementation of CCA as web services• XSOAP: I.E.U!C.L. Implementation of SOAP for

XCAT• XCATSP: I.U.E!C.L. Portal framework for XCAT

(now replaced with Jetspeed)• Xportlets: Toolkit for creating XCAT components

as jetspeed portlets.• Workflow: Series of tasks needed to produce a

result, this may involve tracking software component dependencies, analysis parameters, etc.

Appendix: Screenshots

• Screenshots of grappa portal showing:– Web access– Proxy management– GridFTP browsing– Xbooks pages

• Job submission• Job monitoring

– Some more Portlets– Adding portlets to portal.