Facilitating the process of enabling applications within grid

portalsMichal Kosiedowski

kat@man.poznan.pl

Poznan Supercomputing and Networking Center

• PSNC was established in 1993 and is a R&D center in:– New Generation Networks

• POZMAN and PIONIER networks• 6-NET, SEQUIN, ATRIUM projects

– HPC and Grids• GridLab, Crossgrid, SGIGrid,

PROGRESS, Clusterix projects– Portals and Content Management

Tools• Polish Educational Portal

"Interkl@sa", Multimedia City Guide, Digital Library Framework,Interactive TV

PROGRESS• Budget: ~4,0 MEuro• Project Partners

– Sun Microsystems Poland– PSNC IBCh Poznań– Cyfronet AMM, Krakow– Technical University Lodz

• Co-funded by the State Committee for Scientific Research (KBN) and Sun Microsystems Poland

• Goal: construct an access environment to grid resources and services

PROGRESS: architecture

Grid Service Provider• Job Submission Service

– computing job building, submitting jobs to the grid for execution and monitoring the execution

– grid job descriptions in the XRSL language are transferred to the GRB for the execution of the job

– possibility to prepare workflows • Application Management Service

– manages the PROGRESS application repository– an application descriptor contains a reference to the

application’s executable and a set of its available, required or optional arguments, required environment variables as well as input and output files

– one executable may be referenced by many applications– different application configurations are recognized as

independent applications

Data Management System• A distributed system enabling the

management of grid data files• Storing files in distributed storage modules

of various types: generic filesystems, archivers, relational databases

• Uses metadata to describe files• Allows access to data banks like a mirror of

Sequence Retrieval System• Exposes its functionality within the Data

Broker Service

PROGRESS HPC Portal

Core portlets• “My computing jobs”:

– creation, configuration and execution of grid jobs on the grid– downloading and visualizing the results– utilizes the functionality of the GSP Job Submission and

Application Management services and the DMS Data Broker. • “Applications”:

– management of grid-enabled applications within the GSP application repository

– co-operates with the GSP Application Management Service and the DMS Data Broker.

• “My data”:– management of the files stored within the DMS– uploading and downloading files from the DMS– communicates with the DMS Data Broker.

Enabling an application within the PROGRESS HPC

Portal• Install the application on a computing server or prepare an executable and upload it to the Data Management System

• Use the “Applications” portlet to add the application to the application repository managed by the Application Management Service

• Use the “My computing jobs” portlet to build a grid job utilizing the newly added application

“My computing jobs” portlet

• Exposes the whole functionality of the Job Submission Service

• Enables submission of any job utilizing any available application

• Great for experienced users allowing them to fulfill their requirements

• Too complicated for use by non-advanced users

• Solution: specialized application portlets utilizing the same services as core portlets and familiar with the corresponding grid-enabled application structure

“Gaussian” portlet (1)

“Gaussian” portlet (2)

“Gaussian” portlet (3)

“Gaussian” portlet (4)

Application portlets• Provide users with interfaces that are easy

to use and require no grid knowledge• Grid jobs may be configured very quickly

with an intuitive wizard• Easy and quick access to results:

application portlets are familiar with corresponding applications

• Problem: How to quickly build such specialized portlets?

PROGRESS Portlet Framework

• The Framework allows preparing a high-level interface to Web Services with special beans to store the data obtained from WS responses and classes used to translate HTTP requests into the invocation of proper WS methods.

• This allows to quickly build new portlets on top of those classes to utilize the already existing functionality that they provide.

Provider ...Provider Provider ProviderProvider

HTTP

Content Generator

Content Generator

Content Generator

Content Generator

Content Generator

Request Handler

Request Handler

Request Handler

Request Handler

Request Handler

Request Handler...

Web Service Proxy ...Web Service

ProxyWeb Service

ProxyWeb Service

ProxyWeb Service

Proxy

...

Web Services Invocation FrameworkSOAP (to GSP, DMS)

Portlets

Communication with Web Services (1)

• Web Services Invocation Framework is used for communication with Web ServicesObject executeMethod(String methodName, ArgumentTable methodArguments)

• AXIS toolkit is used to generate the binding classes for complex class types mapping

• Each Web Service is described in an XML configuration file

<?xml version="1.0" encoding="UTF-8"?><web-services> <web-service id="appmgmt"> <name>Application Management</name> <wsdl-location>ApplicationManagementWS.wsdl</wsdl-location> <port-type-name>

http://viola.man.poznan.pl:5021/axis/services/ApplicationManagement </port-type-name>

<port-type-namespace>ApplicationCtrl</port-type-namespace> <description>GSP Application Management Service</description> </web-service> <web-service id="jobs"> <name>Job Submission</name> <wsdl-location>JobSubmission.wsdl</wsdl-location> <port-type-name>

http://viola.man.poznan.pl:5021/axis/services/JobSubmission </port-type-name>

<port-type-namespace>JobSubmissionCtrl</port-type-namespace> <description>GSP Job Submission Service</description> </web-service> <web-services>

Communication with Web Services (2)

Content generation (1)• The content is generated based on two HTTP request parameters

(page and action). 1. page e.g. ApplicationList, JobList2. action e.g. removeApplication, submitJob

• Actions are defined in actions.xml, pages in pages.xml (the content of those files is read by appropriate classes, ContentGenerator reads Action and Page objects from those classes). Actions are executed before page content is generated; pages usually contain default actions to generate their content.

• Actions are usually page independent, they can also replace the default page action

• An action is associated with one or more RequestHandler methods,or with methods of many RequestHandlers

• The Java reflection mechanism is used to call RequestHandler methods

Content generation (2)• Page content is delivered by RequestHandlers in form of PageBean classes

• Each page can contain one or more classes implementing the PageBean interface (PageBean classes know how to deliver their content in form of a DOM)

• The page generates its content by converting its PageBeans into a DOM and then transforming it using an XSLT template

Page

PageBean1

RequestHandler1

RequestHandler2

RequestHandler3

PageBean2

PageBean3

PageBean4

DOM1

DOM2

DOM3

DOM4

DOM

Content Generator

HTML

Content generation (3)• Pages are defined in an XML config file:

<pages><page name="ApplicationList">

<display-name>Application List</display-name><description>List of PROGRESS applications</description><xsl-file>application-list.xsl</xsl-file><action name="ApplicationList">

<request-handlers><request-handler class="ApplicationGetRequestHandler">

<method name="getApplicationList"/></request-handler>

</request-handlers></action>

</page>...

</pages>

Content generation (4)• Invocation of many RequestHandlers

<page name="EditApplicationExecutable"><display-name>Edit Application Executable</display-name><description>Edit application executable file - choose/change the file. The executable can also be uploaded to SZD.</description><xsl-file>edit-application-executable.xsl</xsl-file><action name="EditApplicationExecutable"><request-handlers><request-handler class="ApplicationGetRequestHandler"><method name="getApplication"/></request-handler><request-handler class="SzdGetRequestHandler"><method name="getUserDirectoryListing"/></request-handler></request-handlers></action>

</page>

Content generation (5)• RequestHandler “workflow”<page name="JobList"><display-name>Job List</display-name><description>List of jobs for a given user</description><xsl-file>job-list.xsl</xsl-file><scripts>

<script language="JavaScript" src="jscript/job-submission/job_list.js" /></scripts><action name="JobList">

<request-handlers><request-handler class="JobGetRequestHandler">

<method name="getJobsOfUser" output-name="userJobs"/><method name="filterJobs"

input-name="userJobs" output-name="filteredJobs"/><method name="sortJobs" input-name="filteredJobs"/>

</request-handler></request-handlers>

</action></page>

Content generation (6)• Page error redirection – defined in an error-redirections.xml file:

<?xml version="1.0" encoding="UTF-8"?><redirections>

<error-redirection-group id="EditJob:saveJob"><error-redirection type="validation" redirect-to-page="EditJob" />

</error-redirection-group>

<error-redirection-group id="EditTaskInfo:saveTaskInfo"><error-redirection type="validation" redirect-to-page="EditTaskInfo" />

</error-redirection-group></redirections>

Framework advantages• It’s easy to reuse the portlets in different portal

frameworks and on various portals and websites– The top-level Provider class can be easily adjusted to

implement any portal framework API– The content generated by a portlet can be easily changed

by playing with configurations files• The same RequestHandlers can be used by

mutliple portlets thus enabling easy creation of new portlets communicating with the same WS services– specialized application portlets, e.g.: “Gaussian” portlet,

“DNA Assembly” portlet

Conclusions• PROGRESS grid access environment features the

Application Management Service which enables to easily manage applications available in the grid, the “Applications” portlet which allows to utilize the functionality of this service and the “My computing jobs” portlet which can be used to create a computing job based on any application available in the repository.

• Our experience shows it’s important to provide users with interfaces to grid-enabled applications that are easy to use and do not require deep knowledge about the grid and/or the application.

• PROGRESS Portlet Framework facilitate the construction of specialized application portlets that deliver the functionality of the corresponding grid-enabled applications within a grid portal.

PROGRESS

http://progress.psnc.pl/

http://progress.psnc.pl/portal/

progress@psnc.pl