Www.eu-eela.eu E-science grid facility for Europe and Latin America JRA1 Status Report Development of Services for Applications and Infrastructure Francisco

www.eu-eela.eu

E-science grid facility forEurope and Latin America

JRA1 Status ReportDevelopment of Services for Applications and Infrastructure

Francisco Brasileiro (JRA1 Activity Manager)

UFCG (Brazil)

First Project Review

Brussels, 28.04.2009

Brussels, First Project Review, 28.04.2009 2www.eu-eela.eu

Outline

• Activity goals

• Work planned for the first year

• Results achieved

• Highlights

• Work planned for the second year


Activity Goals (1/2)

• In the DoW the JRA1 goal is stated as:“… the EELA-2 Joint Research Activity will focus on extending

the versatility of the production service managed by SA1 and functionalities as required by the NA3 applications running on it. In particular, research and development will be conducted with the goal of increasing the reach and the usability of e-Infrastructure by assembling/re-engineering existing technologies and developing new ones that will facilitate the installation, management and use of the grid infrastructure. Special attention will be given to the issues related with the sustainable use, management and growth of the e-Infrastructure and its users’ community.”


Activity Goals (2/2)

• Making the e-Infrastructure more useful and wide spread to increase its reach …

Execution of grid middleware on top of different platforms, including Microsoft Windows

Scavenging of idle resources, particularly in shared desktops

… and its usability New application-oriented grid services Support for the integration of a lightweight opportunistic grid

middleware (OurGrid) whose development is led by one of the consortium members

• With the effect of helping in fostering the sustainability of the e-Infrastructure


Work Planned for the 1st Year• MJRA1.1: PKI-enabled OurGrid middleware (M06)

– Allow the OurGrid middleware to use a Public Key Infrastructure (PKI) for authentication/authorization, as done by gLite (interoperability)

• MJRA1.2: Prototypes of the proposed services available at the EELA-2 software repository (M10)– The proposed application-oriented services were chosen based on the

anticipated requirements of the legacy applications from the phase 1 and on the new proposed applications

– The proposed infrastructure-oriented services aim at addressing the growth of the e-infrastructure as well as the simplification of its management

• MJRA1.3: Common Information System between gLite and OurGrid (M10)– Provide the necessary mechanism for the OurGrid and gLite

Information Systems to be combined into a single Information System (interoperability)


Results Achieved


EELA-2 Software Repository


PKI-enabled OurGrid middleware (MJRA1.1)

• X.509-enabled OurGrid middleware available– Enables an OurGrid system to use the same PKI and Grid

Security Infrastructure technologies used by gLite systems– Users of an OurGrid system can be authenticated using the

same types of certificates that they use in a gLite system– Allows the use of the standard VOMS protocol for authorisation

• As a result:– Users/resources of the EELA-2 e-infrastructure will require a

single certificate no matter which middleware they are using/running

– A few resource centres in the EELA-2 infrastructure are currently using this software


Prototypes of the application-oriented proposed services (MJRA1.2)

• Data Management– Digital archives– Secure storage– Cooperative annotation of data

• Job management– VO compliant Virtual Machine environments– Workflow for OurGrid jobs


• Job management– WatchDog

Allows the monitoring and control of job execution on the gLite Worker Node

• Catalogue and file management– lgc-rec toolkit

Implements a recursive version of the lgc-* suite of commands

Prototypes of the application-oriented additional services (MJRA1.2)


• Port of gLite to Microsoft Windows platforms– User Interface

Grid Security Infrastructure Workload Management File Transfer Data Management Information System

– Computing Element The farm is composed by a unique central

node with Linux operating system in whichthe Gatekeeper and the Torque/MAUI headnode are running

A set of worker nodes with our rebuilt versionof the gLite WN package implement the farmworkers

Prototypes of the infrastructure-oriented proposed services (MJRA1.2)


• Support for grid operation– SAGE (Storage Accounting for Grid Environments)– Multiple middleware resource centres

gLite and Globus Toolkit 4 (GT4)• User Interface having clients for both gLite and GT4• A single cluster running both a gLite and a GT4 Computing

Element Co-existence of gLite and OurGrid

• Scavenging of idle cycles in dedicated resources running gLite

• Increased support for Bag-of-Tasks (BoT) applications through the OurGrid middleware

Prototypes of the infrastructure-oriented additional services (MJRA1.2)


• This milestone was foreseen as an important step towards the interoperability between gLite and OurGrid

• However, during the course of the development we concluded that this integration was no longer required– We will be able to leverage efforts carried out by the EDGeS

project that have already addressed this issue– The EDGeS G3 (Generic Grid Gateway) technology will allow

the development of gateways between gLite and OurGrid by simply providing appropriate plug-ins for the OurGrid middleware (gLite plug-ins are already available)

• The saved time allowed us to explore other middleware integration approaches, such as the co-existence previously mentioned

Common Information System(MJRA1.3)


Highlights of the 1st Year (1/3)• A number of applications are either using or being upgraded to

use the services developed

Service Applications

GSAF Aîuri – UFRJ (Brazil)

OurGrid DistBlast - UnB (Brazil), BRAMS - UFCG (Brazil) and Cinefilia - INFN (Italy)

watchdog CROSS-Fire – U. of Minho (Portugal)

lcg-rec toolkit

CROSS-Fire – U. of Minho (Portugal)

Service Applications

GSAF gRREEMM - Cubaenergia (Cuba)

Secure Storage

CardioGrid Portal - UBA (Argentina)

watchdog G-InterproScan and G-HMMER - UNIANDES (Colombia)

lcg-rec toolkit

META-Dock - UNAM (Mexico)

Using Being upgraded


• A number of resource centres are using or starting to deploy the OurGrid middleware– Resource centres using OurGrid

– Resource centres that have started the deployment UnB and UNILASALLE (Brazil), UNIANDES (Colombia), PUCP and USMP (Peru)

Highlights of the 1st Year (2/3)


Highlights of the 1st Year (3/3)

• Interoperability between OurGrid and gLite planned for M16 is advancing very well– MoU signed with other projects working on similar goals (EDGeS

and SEE-GRID-SCI)– Initial assessment of the possibility of using EDGeS G3 indicates

the feasibility of using technology that has already been successfully used in a similar setting

– Initial evaluation of the innovative co-existence approach reveals advantages that had not been foreseen at the time the project was proposed

• Possibility of draining BoT applications out of the service grid, improving the performance of tightly-coupled applications


Work Planned for the 2nd Year

• MJRA1.4: Prototype of the gateway to transfer jobs from gLite to OurGrid and vice-versa (M16)

• MJRA1.5: Stable of the proposed services (M16)

• MJRA1.6: Stable version of the gateway to transfer jobs from OurGrid to gLite and vice-versa (M22)


Plan of Action for the 2nd Year• Development of gateways for grid interoperability and

improvement of services developed• In addition to the planned developments, promote the

dissemination and the use of the services developed– Continue to work in close cooperation with NA3 to identify users

that could benefit from the services available and from others that could be developed

– Continue to work in close cooperation with NA2 to advertise the services available and to train users to use them

– Continue to work in close cooperation with SA1 to refine the prototypes, their documentation and deployment procedures

– Continue to participate in grid-related dissemination events– Follow the RESPECT protocol to share the developments

• Continue research on grid interoperability– Work in synergy with other projects involved in similar efforts,

such as the EDGeS project


Pointers for more Information

• JRA1 Resources– EELA-2 web site

https://www.eu-eela.eu/

– JRA1 Wiki Site Contains descriptions and links for each JRA1 service https://grid.ct.infn.it/twiki/bin/view/EELA2/JRA1Services

– EELA-Forge Site All released software https://forge.eu-eela.eu/

– Contact us at Francisco Brasileiro ([email protected])

Documents

Www.eu-eela.eu E-science grid facility for Europe and Latin America JRA1 Status Report Development of Services for Applications and Infrastructure Francisco