Workflows in PROGRESS and GridLab environments

GLOBAL GRID FORUM 10

Workflows in PROGRESS and GridLab environments

Michał Kosiedowski

GLOBAL GRID FORUM 10

PROGRESS (1)

• Duration: December 2001 – December 2003• Budget: ~4,0 MEuro• Project Partners

– Sun Microsystems Poland– PSNC IBCh Poznań– Cyfronet AMM, Kraków– Technical University Łódź

• Co-funded by The State Committee for Scientific Research (KBN) and Sun Microsystems Poland

GLOBAL GRID FORUM 10

PROGRESS (2)

GLOBAL GRID FORUM 10

Grid Service Provider

• Job Submission Service– delivers functions for computing job building, submitting them to the

grid for execution and viewing the results. It allows to create jobs, configure its tasks and set the requirements for grid environment resources. The service prepares grid job descriptions using the XRSL language. The description is then transferred to the GRB for the execution of the job.

• Application Management Service– manages the PROGRESS application repository. An application

descriptor contains a reference to the application’s executable and a set of its available, required or optional arguments, required environment variables as well as input and output files. One executable may be referenced by many applications. Different application configurations are recognized as independent applications

GLOBAL GRID FORUM 10

Grid Resource Broker

• built on the top of core infrastructure provided by the Globus Toolkit 2.x

• enables the execution and control of PROGRESS grid jobs through the GRAM protocol on computing resources connected to the PROGRESS grid

• matches and submits the job to the best fitting resource, according to the job description and application requirements, additionally taking into account policies set by the grid administrators

• grid job definitions are sent to the GRB in the form of XRSL documents

GLOBAL GRID FORUM 10

Workflows (1)

GLOBAL GRID FORUM 10

Workflows (2)

GLOBAL GRID FORUM 10

Workflows (3)

GLOBAL GRID FORUM 10

Workflows (4)

GLOBAL GRID FORUM 10

XRSL (1)

• <task>: describes a single task to be performed on the grid together with all application and user requirements, including static and dynamic resource parameters, environment variables, stdin, stdout, stderr, etc.,

• <sequence>: describes a sequence of tasks to be performed on the grid, must contain at least one <task> or <parallel>,

• <parallel>: describes tasks to be executed in parallel on the grid, must contain at least one <task> or <sequence>.

GLOBAL GRID FORUM 10

XRSL (2)<job> <sequence> <parallel> <sequence> <parallel>

<task id="T1"/>

<task id ="T2"/> </parallel>

<task id="T4"/> </sequence>

<task id="T3"/> </parallel>

<task id="T5"/> </sequence></job>

T3

T4

T5

T1 T2

<job> <sequence> <parallel> <sequence> <parallel>

<task id="T1"/>

<task id ="T2"/> </parallel>

<task id="T4"/> </sequence>

<task id="T3"/> </parallel>

<task id="T5"/> </sequence></job>

<job> <sequence> <parallel> <sequence> <parallel>

<task id="T1"/>

<task id ="T2"/> </parallel>

<task id="T4"/> </sequence>

<task id="T3"/> </parallel>

<task id="T5"/> </sequence></job>

<job> <sequence> <parallel> <sequence> <parallel>

<task id="T1"/>

<task id ="T2"/> </parallel>

<task id="T4"/> </sequence>

<task id="T3"/> </parallel>

<task id="T5"/> </sequence></job>

<job> <sequence> <parallel> <sequence> <parallel>

<task id="T1"/>

<task id ="T2"/> </parallel>

<task id="T4"/> </sequence>

<task id="T3"/> </parallel>

<task id="T5"/> </sequence></job>

GLOBAL GRID FORUM 10

Virtual Applications (1)

A user can create and save different workflow structures of regular applications as virtual applications and thus allow other users to choose a virtual application from the list of available applications. Then, the workflow is automatically created by the GSP’s Job Submission Service based on the application description obtained from the Application Management Service.

GLOBAL GRID FORUM 10

Virtual Applications (2)

GLOBAL GRID FORUM 10

Grid(Lab) RMS (1)

Grid Resource Broker

GLOBAL GRID FORUM 10

Grid(Lab) RMS (2)

GLOBAL GRID FORUM 10

Contacts

http://progress.psnc.pl/

http://progress.psnc.pl/portal/

progress@psnc.pl

http://www.gridlab.org/

http://portal.gridlab.org/

office@gridlab.org

http://www.psnc.pl/