INFSO-RI-508833 Enabling Grids for E-sciencE International e-Infrastructure Mike Mineter [email protected]

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

International e-Infrastructure

Mike Mineter

[email protected]

International e-Infrastructure 2


INFSO-RI-508833

Overview

• The view from the European Commission

• GEANT – European network• DEISA – for when High Performance Computing is just

not HP enough!• EGEE – establishing grid e-Infrastructure

– WHY?!– HOW?!– With whom?



INFSO-RI-508833

View from European Commission

• Entering the “knowledge society” from the “industrial society”Industrial society = Transportation Infrastructure

Knowledge society = Communications infrastructure

• Lisbon strategy: Research and Innovation will be the most important factors in determining Europe’s success through the next decades

• THE GOAL: “UNLEASH CREATIVITY”- by investment in– Human skills– Infrastructures

• Demands in growth of e-infrastructure

e-Infrastructure - Strategic building blocks

GÉANT .

INFRASTRUCTURE

GRID .

INFRASTRUCTURE

e-I

nfr

as

tru

ctu

re

GÉANT2

EGEE

DEISA

…

+ 1

8 p

rop

osa

ls

inte

rnat

ion

al, p

olic

ies,

use

r co

mm

un

itie

s

Mário Campolargo DG INFSO F3, Pisa 24th October 2005



INFSO-RI-508833

DEISA

• “integration of existing national high-end platforms, tightly coupled by a dedicated network and supported by innovative system and grid software”

• Initial scientific applications include– Material Sciences– Cosmology– Plasma Physics– Life Sciences

• http://www.deisa.org



INFSO-RI-508833

GEANT

• Interconnects 34 National Research & Education Networks-NRENs of the extended European Research Area (ERA)

• Connects more than 3500 Research & Education Institutions

• Serves millions of end-users + eScience Projects (e.g. Grids) under Accepted Usage Policy (AUP) rules

• 3-tier Federal Architecture, partially subsidized by National and EU Research & Education funds:– The Campus Network (LAN/MAN)– The NREN (MAN/WAN)– The Pan-European Interconnection

• GEANT2 en route

• http://www.geant.net/



INFSO-RI-508833

EGEE: Enabling Grids for E-sciencE



INFSO-RI-508833

EGEE Organisation

• 70 leading institutions in 27 countries, federated in regional Grids

• ~32 M Euros EU funding for first 2 years starting April 2004 (matching funds from partners)

• Leveraging national and regional gridactivities

• Promoting scientific partnershipoutside EU



INFSO-RI-508833

In its first 2 years EGEE

• Goal: establish production quality sustained Grid services with– 3000 users from at least 5 disciplines– integrate 50 sites into a common

infrastructure– offer 5 Petabytes (1015) storage

• Achieved– > 180 sites in 39 countries– ~ 20 000 CPUs– > 5 PB storage– > 10 000 concurrent jobs per day– > 60 Virtual Organisations

Pilot New



INFSO-RI-508833

LCG and EGEE

• EGEE committed to “hit the ground running”

• EGEE profits from the resources - no funded computing/data resources in EGEE– Provided by the VOs

• LCG obtains additional production and operation efforts

• LCG experiments now comprise several of the many VOs in EGEE

• Current service (“LCG-2”) based on work done in LCG– Middleware components to be

upgraded by “gLite” services as they are proven

– “gLite 3” will be forged from LCG 2.7 + gLite services

LCG : Large Hadron Collider Compute Grid



INFSO-RI-508833

EGEE Activities

• 48 % service activities (Grid Operations, Support and Management, Network Resource Provision)

• 24 % middleware re-engineering (Quality Assurance, Security, Network Services Development)

• 28 % networking (Management, Dissemination and Outreach, User Training and Education, Application Identification and Support, Policy and International Cooperation)

Emphasis in EGEE is on operating a productiongrid and supporting the end-users



INFSO-RI-508833

Operations

International e-InfrastructureInternational e-Infrastructure 18


INFSO-RI-508833



INFSO-RI-508833

Building user communities



INFSO-RI-508833

Trainers power Virtuous Cycles

ResearchersDiagnosticians

Designers

UserInductionTraining

OutreachDissemination

SuccessfulUsers

User RegistrationApplicationDevelopers

Dissemination

DeveloperInitial

Training

ApplicationDevelopers

At home on EGEE

PositiveReferralsDeveloper

AdvancedTraining

New EGEEApplications

PushingLimits

AdvancedEGEE

ApplicationsSuccessStories &Experts

PositiveReferrals



INFSO-RI-508833

The GILDA Test-bed(https://gilda.ct.infn.it/testbed.html)

20 sites in 3 continents !

Beijing

(IHEP)

Marc-Elian Bégin - Demos - 1st EU review 24


INFSO-RI-508833

GILDA demonstrator and testbed(https://gilda.ct.infn.it)



INFSO-RI-508833

t-Infrastructure

• Why t-infrastructure? – Training is necessary: personal + e-learning– e-Infrastructure for production – t-Infrastructure for training

• Need guaranteed response for tutorials; limit the vulnerability of production systems – use training grid – have training CA – able to change middleware to prepare participants for

future releases on production system – need safe resources for installation training – easy entry point for new communities



INFSO-RI-508833

International cooperation



INFSO-RI-508833

Parts of the Grid “ecosystem”

. . .

LCG

2004

2001

EGEE

Used in

USA EU

NextGrid DEISAGridCC

Future grids

EDG

Globus MyProxyCondor ...

VDT

DataTAG

CrossGrid ...

OSG, …

SRM

…

interactive



INFSO-RI-508833

Policy and International Cooperation

• Cooperation between EGEE and other Grid activities – Globus Alliance, Condor– Training/workshop events

(International Summer School of Grid Computing, July… 2 intense weeks… registrations open! http://www.dma.unina.it/~murli/ISSGC06/ )

– eInfrastructure reflection group in Europe http://www.e-irg.org/

• Standard setting through attendance at global standard bodies such as the Global Grid Forum. – Grid Storage Management GGF working group -

http://sdm.lbl.gov/gsm/– Security, Authentication: US – EU cooperation

• Mutual recognition of Certificate Authorities– Requires collaboration to establish policy - and mutuality



INFSO-RI-508833

EGEE-II



INFSO-RI-508833

• EGEE-II proposal submitted to the EU– Proposed start 1 April 2006

• Natural continuation of EGEE– Expanded consortium – Emphasis on providing an infrastructure

increased support for applications

interoperate with other infrastructures

more involvement from Industry

EGEE-II



INFSO-RI-508833

EGEE-II: Expertise & Resources

• More than 90 partners • 32 countries• 12 federations Major and national

Grid projects in Europe, USA, Asia

+ 27 countries through related projects:– BalticGrid– SEE-GRID– EUMedGrid– EUChinaGrid– EELA



INFSO-RI-508833

Related Projects

23rd March 2006 35

INFSO-SSA-26637

• Training– Targeted– Immediate goals– Specific skills– Building a workforce

• Education– Pervasive– Long term and sustained– Generic conceptual models– Developing a culture

• Both are needed

Society

Graduates

EducationInnovation

Invests

PreparesCreate

Enriches

Organisation

Skilled Workers

TrainingServices & Applications

Invests

PreparesDevelop

Strengthens

Malcolm Atkinson

International Collaboration to Extend and Advance Grid Education – ICEAGE

International Collaboration to Extend and Advance Grid Education – ICEAGE



INFSO-RI-508833

EGEE is running…

• … the largest multi-VO production grid in the world!

• What’s happening now?http://gridportal.hep.ph.ic.ac.uk/rtm/

• What resources are connected?http://goc.grid-support.ac.uk/gridsite/monitoring/



INFSO-RI-508833

Can you use the EGEE grid?!

• Yes - if you are in a VO that has been accepted!

• Probably - if you are in an international VO willing to apply and (probably) contribute some resources

• (contrasting policy with the NGS – appropriate because EGEE is seeking to federate national grids, not replace them!)

• BUT– ANYONE can apply for an account on GILDA to gain experience



INFSO-RI-508833

Interoperability amongst production grids



INFSO-RI-508833

Glance at Interoperability

• Interoperability can be sought at:

– Application layer – as P-GRADE shows (and others!)

– Grid layer – GIN: Grid Interoperability Now GGF initiative – ~18 grids EGEE, OSG (in the US), NAREGI (Japan)……. Are production grids

providing a service now Translations and glue

• between “islands” sharing different de-facto standards e.g. different information systems in EGEE, GT4 and UNICORE grids

– Service layer – here reliant on standards and on architecture



INFSO-RI-508833

A glance at Nextgrid

• Based on service orientation and emerging standards, its big themes include:– workflow

brokering, planning, reservation languages raise level of abstraction

– service level agreements not exactly like that with electricity supplier!

• have to be interpreted not by courts but prescriptions of what want/offered that can be matched computationally

define composition, relationship between consumer and provider Requires accounting



INFSO-RI-508833

Summary

• EGEE is running the largest multi-VO grid in the world!• Creating the “grid layer” in e-Infrastructure • Seeking to federate national grids – NGS• Key concepts for EGEE

– Sustainability – planning for the long-term– Production quality – And…

• Grids are fundamentally about people• … how people in different organisations commit to cooperate• … and how that cooperation can be enabled by operations,

training, support, and (most transient of all!) middleware



INFSO-RI-508833

Further information

• EGEE Websitehttp://www.eu-egee.org

• How to joinhttp://public.eu-egee.org/join/

• EGEE Project [email protected]

• Global Grid Forum http://www.gridforum.org/• Globus Alliance http://www.globus.org/

• Condor http://www.cs.wisc.edu/condor/• VDT http://www.cs.wisc.edu/vdt/

• Open Science Grid http://www.opensciencegrid.org/• Grid Center http://www.grids-center.org/

• LCG http://lcg.web.cern.ch/LCG/

Documents

INFSO-RI-508833 Enabling Grids for E-sciencE International e-Infrastructure Mike Mineter [email protected]