Www.gisela-grid.eu Grid Initiatives for e-Science virtual communities in Europe and Latin America A Grid Story in LA: from EELA to CLARA via GISELA B

www.gisela-grid.eu

Grid Initiatives for e-Science virtual communities in Europe and Latin America

A Grid Story in LA: from EELA to CLARA via GISELA

B. Marechal – CETA-CIEMAT & UFRJGISELA Project Coordinator

Ph. Gavillet – CETA-CIEMAT & CERNGISELA Deputy Project Coordinator

EVALSO – ELLA Meeting, 17th - 18th October 2011

Rio de Janeiro (Brazil)

www.gisela-grid.eu 2

Outline

CLARA

EELA

--01/01/2006

--31/12/2007

GISELA

--31/08/2012

--01/09/2010

EELA-2

--01/04/2008

--31/03/2010

• The seed: EELA

• The root: EELA-2

• The tree: GISELA

• The fruit: The long-term sustainability of

– The e-infrastructure

– The VRC support

EVALSO – ELLA Meeting, 17th - 18th October 2011 - Rio de Janeiro (Brazil) 2


Long record in e-Science support

• Fruit of 5-6 year experience of successful e-Science EU-LA Collaboration, through EU funded projects (EELA, EELA-2, GISELA)

• Presence in almost all LA countries

• Most encompassing e-Infrastructure for scientific collaborations between Latin America and Europe

• Sound long-term sustainability approach rooted on CLARA, NRENs, LA NGIs



EELA Objectives & Outcomes

• EELA (E-infrastructure shared between Europe and Latin America) under FP6www.eu-eela.org/first-phase.php

– Objective Create and operate a transcontinental Grid infrastructure

Bridge consolidated e-Infrastructures in Europe and emerging ones in Latin America

Enhance Research in developing countries

Address cost-effectively regional problems

– Outcomes Successful building of a reliable and almost “production quality” e-Infrastructure in Latin America

Provide EU & LA researchers early access to a new, well-supported e-Infrastructure and enable them to speed up the processing of scientific data

Strengthen scientific communities in Europe and Latin America

Entrance and consolidation of Latin American communities in worldwide collaborations

Grid technology skills acquired to autonomously support the Latin American Grid on the long term

Amplification of the e-Infrastructures relevance, blazing the trail towards Latin American e-Science initiatives and/or NGIs



EELA-2 Objectives & Numbers

• EELA-2 (E-science grid facility for Europe and Latin America) under FP7www.eu-eela.eu

– EELA-2: a step forward w.r.t. EELA in both the e-infrastructure size & services, the user services and the backend support with the creation of JRUs & NGIs

– Objectives Provide an empowered Grid Facility with versatile services fulfilling application requirements

Ensure production quality services

Ensure the long term sustainability of the e-Infrastructure beyond the term of the project

Expand the EELA e-Infrastructure

Look for new communities outside academia (Industry and Business)

– Numbers 16 Countries (11 in LA)

2 new countries (Panama & Uruguay)

1 International Organisation (CLARA)

78 Members (62 in LA) - 32 Institutions joined after 1 year (31 in LA)

13 JRUs (9 in LA) - 4 new JRUs (3 in LA)

At the EELA-2 sunset:

28 gLite RCs, ≈

8 OurGrid

RCs, 6100 cores & 560 TB



EELA-2 prepared the future

Much more training effort delivered than pledgedTraining a real success / long-term investment most precious currently

1-year threshold

Project threshold



GISELA Objectives and Goals

Ensure the long- term sustainability of the e-Infrastructure in the Latin American continentEnsure the long- term sustainability of the e-Infrastructure in the Latin American continent

Provide full support to the Virtual Research Communities spanning Latin America and Europe, using the e-Infrastructure.

Provide full support to the Virtual Research Communities spanning Latin America and Europe, using the e-Infrastructure.

Focus on two inter-related goals:

• Implement a sustainability model rooted on National Grid Initiatives (NGI), in association with CLARA, NRENs and collaborating with EGI.

• Provide the communities with the suited e-Infrastructure and Application-related Services required to improve the effectiveness of their research. This will address both:

The current EELA-2 User Communities whose research investigations are carried out at the Institution level or in small collaborations.

The larger Virtual Research Communities as Life & Earth Sciences, HEP

Focus on two inter-related goals:

• Implement a sustainability model rooted on National Grid Initiatives (NGI), in association with CLARA, NRENs and collaborating with EGI.

• Provide the communities with the suited e-Infrastructure and Application-related Services required to improve the effectiveness of their research. This will address both:

The current EELA-2 User Communities whose research investigations are carried out at the Institution level or in small collaborations.

The larger Virtual Research Communities as Life & Earth Sciences, HEP

Following the “Excellent” EELA & EELA-2 results, as acknowledged by the EC reviewers, GISELA has been submitted and brilliantly accepted, to …

Following the “Excellent” EELA & EELA-2 results, as acknowledged by the EC reviewers, GISELA has been submitted and brilliantly accepted, to …



15 Countries (11 in Latin America)

19 Partners (14 in Latin America)

12 Third Parties (09 in Latin America)

UNAM Third Parties• CICESE• ITESM• IPN-CIC• UNISON

The GISELA spirit is not anymore to consider Institutions, but rather representatives of JRU / NGI, with the advantage to “accept” de facto all JRU / NGI members.

The GISELA spirit is not anymore to consider Institutions, but rather representatives of JRU / NGI, with the advantage to “accept” de facto all JRU / NGI members.

UPORTO Third Party

• Uminho

REUNA Third Parties• UFRO• UTFSM

UNIANDES Third Parties• UIS• PUJ

UFRJ Third Party• CEFET-RJ

CNRS Third Parties• CPPM• IPGP

The GISELA Consortium



GISELA as Resource Provider

• General– is much experienced in Grid computing– has a powerful Infrastructure– complies with EGI policies & recommendations

• Specificities– Both Service (gLite) and Opportunistic Computing (OurGrid)

supported


www.gisela-grid.eu

Year 1

Year 2Pledg

e

“Extra” resources for the prod.vo.eu-eela.eu VO

Powerful e-Infrastructure in Latin America


www.gisela-grid.eu

Cumulative GISELA contribution per site (normalised CPU.hours)

Resource Centres contributions


www.gisela-grid.eu

• EGI – UFRJ (on behalf of GISELA) Resource Provider MoU

• EGI compliance is based on the adoption of common services and practices

– The Operational Level Agreement (OLA) (http://documents.gisela-grid.eu/record/87?ln=en)

– Monitoring System (Nagios)

– Ticket System (xGUS)

– Accounting System (APEL)

– Middleware (gLite)

– Resource database (GOCDB)

– Operations Dashboard

EGI compliant Infrastructure



Specificity: gLite <-> OurGrid interoperability

• Latin American context:– Large Universities (UFRJ, USP,…) in International collaborations thus

complying with the corresponding computing model and middlewares (gLite, OSG,..)

– Several small Universities naturally adopting the Desktop-based opportunistic Grid approach and using the OurGrid middleware



• Platform for all kinds of Applications

• Offers a series of well-supported Services (e.g. DIRAC)

• Training & Support

• Collaborates with EGI and Regional Initiatives

GISELA: an opportunity for VRCs


www.gisela-grid.eu

Platform for all kind of Applications / VRCs

15

• VRCs– A few large VRCs from Life Sciences (e.g. WeNMR, Biomed), Earth Sciences (WRF4G,

Seismic simulations) and HEP (Auger & LHC experiments) – Legacy: EELA-2 User Communities, typically 1-2 group(s), largely located in Latin America– 34 Applications are in a production stage

• MoU signed with EGI-InSPIRE enables…– VRC members in Europe exploit GISELA resources– VRC members in LA exploit EGI resources (most usual)– Publishing GISELA applications in the EGI Applications data base http://appdb.egi.eu/


www.gisela-grid.eu

Usage statistics

16

• Accumulated CPU-hours, grouped by VOs:

• prod.vo.eu-eela.eu: catch all VO (Earth Sciences, EELA-2 legacy,..)



Portfolio of Services

• Supported services (extract) :– Application-related Services

Job Catalogue, File and Data Management• Digital archives, Secure storage, WatchDog, lcg-rec toolkit

– Data processing and user analysis services DIRAC: workload and data management system

• Possibility of development of new services– Industry portal

– Dynamic creation of Customised Virtual Clusters (CVC - Uniandes)

– Development of a VRC-driven Web site



DIRAC as a complete VRC solution

• DIRAC is a complete VRC Grid solution

• Already used by several large VRCs:– HEP (LHCb, Belle, ILC, Magic)

– BIOMED(VIP)

• Development coordinated by CPPM (Marseille), a GISELA Partner

• Fully supported within WP6


www.gisela-grid.eu EVALSO – ELLA Meeting, 17th - 18th October 2011 - Rio de Janeiro (Brazil) 1919

Are we flying in a blue sky?

• Not yet … since

– Use of the GISELA e-infrastructure not yet at the expected level (HEP remains the main user)

– Grid awareness still insufficient in LA (in spite of EELA, EELA-2, GISELA & CLARA efforts)

– Middleware and Authentication & Authorisation issues frighten potential users Possible solution: Science Gateways (SG), proposed by INFN – Catania, a GISELA Partner,

which is a Web Portal customised to meet the needs of a specific community, e.g. providing simple authentication mechanisms and intuitive user interfaces to execute applications on the e-Infrastructure

GISELA (as well as Resource Providers from Regional Initiatives) expects to much improve the support to VRCs

GISELA is organising (in collaboration with EPIKH) Science Gateways-oriented training events

– Is there a life after GISELA? Hopefully yes …

• Depending on CLARA and Latin American NRENs (The handover is on the way)

• If a realistic Business Plan is build and .. Accepted

• If a “NGI-type” of organisation is set up in LA countries (Government support needed…)



The GISELA & CLARA model

THE THREE-LAYER INFRASTRUCTURE AND NETWORK MODEL

(Adaptation of the EELA-2 Model presented to the CLARA Board in Costa Rica – 12/08/2009)

IMPORTANT ROLE OF LATIN AMERICAN NRENS OR EQUIVALENT DOMESTIC GRID STRUCTURES



Handover and Business Plan

• Handover

– A CLARA Transition Team is working in close collaboration with the GISELA Work packages (one CLARA TT representative in each GISELA WP);

– GISELA knowledge / experience fully transferred to CLARA

– 6 months before the end of GISELA, core services and VRCs support should be handled by CLARA & LA NRENs

• CLARA Business Plan

– A necessary condition to guarantee an e-life after GISELA

– Based on the LCPM (Life Cycle Product Management) process

– Currently being constructed

– Final version in the GISELA Deliverable D1.4 to be submitted on 30/11/11

– Willing to offer Advanced Computing Services, best adapted to LA realities and VRC requests, including complementary solutions (GRID, CLOUD, HPC, HTC,…)

– Alternatives to Grid computing: do they help or puzzle the communities?



Outlook

• Current GISELA e-infrastructure (result of 6 years of efforts) – already installed

– supported by institutions (RCs & ROCs) and EC (Core services at continental level)

– an example of “minimum model” for long-term sustainability

• GISELA developing / deploying new Cloud services for OurGrid (WP6 task)

• GISELA and CLARA eagerly waiting for the "Cloud Strategy" announced for 2012 at the EGI Technical Forum – Lyon – 20/09/2011 (K. Glinos presentation)



Issues of Cloud Computing (K. Glinos - EGI TF)



Current Cloud initiatives & Open questions(K. Glinos - EGI TF)



The purpose of the meeting is to present successful collaboration projects that have been funded by the EC (FP6 & FP7) and to present Research areas of great importance for Latin America

The EELA, EELA-2 and GISELA Success story

• Participation of 78 Institutions (62 in LA), 16 Countries (11 in LA)

• Contribution to the creation of 13 JRU / NGI (9 in LA)

• Over 400 trained Users and 250 Site Administrators

• Over 65 Applications from over 25 Scientific areas

• Over 140 Presentations & Scientific Publications

Latin American Users believe on Grid - from the EELA-2 Survey: http://documents.eu-eela.eu/record/1348?ln=en • Applications outcomes & potential socio-economic impacts

<55%> are seen to be useful socially

<80%> are claimed to potentially induce industry spin-offs



… as well as to analyse the importance of Research Infrastructures for science & technology and to discuss future collaborations between Latin America and Europe

GISELA Comments to the EU Green Paper 2011 “Horizon 2020 - the Framework Programme for Research and Innovation”

(http://documents.gisela-grid.eu/record/185?ln=en)

• “How should international cooperation with non-EU countries be supported e.g. in terms of priority areas of strategic interest, instruments, reciprocity (including on IPR aspects) or cooperation with Member States?”

“We attach great importance to international cooperation with Europe... We want to stress, based on the experience of several of us to cooperate with European Scientific Institutions…that the Europe – Latin America cooperation is usually most beneficial to both parts.”

We see the following instruments as particularly helpful:

• Usual Research Infrastructure calls“The FP6 and FP7 calls, for typically 2-year projects have been very instrumental in stimulating the deployment of e-Infrastructures in Europe and other Continents by allowing the Associated States to apply. The adopted strategy to have Master Projects (EGEEs, EGI) and related projects has shown to be judicious. These calls can certainly be most useful in the final phase of consolidating the long-term future of DCIs outside Europe.”

• Specialised callsSpecialised calls such as for Collaborative Projects (STREP), as the ICT-Brazil Coordinated Call (FP7-ICT-2011-EU-Brazil…are efficient to focus in space and time on high-priority R&D actions.

• Training actionsMuch appreciated are training actions such as the Marie Curie actions and specific Projects such as EPIKH: “Exchange Programme to advance e-Infrastructure Know-How” (http://www.epikh.eu/) that aims at:

– “Reinforcing the impact of e-Infrastructures in scientific research defining and delivering stimulating programme of educational events, including Grid Schools and High Performance Computing courses”

– “Broadening the engagement in e-Science activities and collaborations both geographically and across disciplines”.

– They have shown to be crucial to strengthen Latin American – European Collaborations in raising the level of the e-Infrastructure competence and performance of Latin American countries close to those of Europe.



EELA and EELA-2 did it

GISELA is doing it

CLARA and NRENs will do it


and finally …

Documents

Www.gisela-grid.eu Grid Initiatives for e-Science virtual communities in Europe and Latin America A Grid Story in LA: from EELA to CLARA via GISELA B