The EGEE Grid infrastructure project: first experience and future plans

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

The EGEE Grid infrastructure project: first experience and future plans

By Fabrizio Gagliardi Malcolm AtkinsonEGEE Project Director Director

CERN NeSC

Geneva – Switzerland Edinburgh – Scotland

SuperComputing 2004 Pittsburgh USA 8 November 2004 2


INFSO-RI-508833

Content

• EGEE - what is it and why is it needed?• Networking activity – pilot applications• Grid operations – providing a stable service• Grid middleware – current and future• Summary

The material of this talk has been contributed by several colleagues in the EGEE project



INFSO-RI-508833

Build a large-scale production grid service to:

• Support science and technology worldwide

• Link with and build on national, regional and international initiatives

• Foster international cooperation both in the creation and the use of the e-infrastructure

Why EGEE?

Network

Op

era

tio

ns

, S

up

po

rt a

nd

tr

ain

ing

Collaborations

Grid infrastructure



INFSO-RI-508833

What is EGEE?

• 70 leading institutions in 27 countries, federated in regional Grids

• 32 M Euros EU funding (2004-5), O(100 M) total budget

• Aiming for a combined capacity of over 20’000 CPUs (one of the largest international Grid infrastructures ever assembled)

• ~ 300 dedicated staff



INFSO-RI-508833

EGEE Activities

• Emphasis on operating a production grid and supporting the end-users

• 48 % service activities (Grid Operations, Support and Management, Network Resource Provision)

• 24 % middleware re-engineering (Quality Assurance, Security, Network Services Development)

• 28 % networking (Management, Dissemination and Outreach, User Training and Education, Application Identification and Support, Policy and International Cooperation)



INFSO-RI-508833

EGEE Applications

• EGEE Scope : ALL-Inclusive for academic applications (open to industrial and socio-economic world as well)

• Establish production quality sustained Grid services – 3000 users from at least 5

disciplines– over 8,000 CPU's, 50 sites– over 5 Petabytes (1015) storage

• Demonstrate a viable general process to bring other scientific communities on board

• Propose a second phase in mid 2005 to take over EGEE in early 2006

Pilot New



INFSO-RI-508833

LHC experiments

• Storage– Raw recording rate 0.1 – 1 GByte/s– Accumulating at 5-8 PetaByte/year– 10 PetaByte of disk

• Processing– 200,000 of today’s fastest PCs

CMS

ATLAS

LHCbALICE



INFSO-RI-508833

EGEE pilot application: Biomedics

• BioMedical– Bioinformatics (gene/proteome

databases distributions)– Medical applications (screening,

epidemiology, image databases distribution, etc.)

– Interactive application (human supervision or simulation)

– Security/privacy constraints Heterogeneous data formats -

Frequent data updates - Complex data sets - Long term archiving

• BioMed applications deployed and going live now– GATE - Geant4 Application for

Tomographic Emission– GPS@ - genomic web portal – CDSS - Clinical Decision Support System



INFSO-RI-508833

EGEE and LCG

• LGC goal: prepare and deploy the computing environment that will be used by the experiments to analyse the LHC data

• A collaboration between:– The physicists and computing

specialists from the LHC experiment – The projects in Europe and the US that

have been developing Grid middleware– The regional and national computing

centres that provide resources for LHC – The research networks

• EGEE builds on the work of LCG to establish a grid operations service



INFSO-RI-508833 LCG-2/EGEE-0 Status

08-11-2004

Total:87 Sites8784 CPUs3 PByte

Total:87 Sites8784 CPUs3 PByte

Cyprus



INFSO-RI-508833

Future Middleware: gLite

• Intended to replace LCG-2

• Starts with existing components from AliEN, EDG, VDT etc.

• Aims to address LCG-2 shortcoming and advanced needs from applications

• Prototyping short development cycles for fast user feedback

• Initial web-services based prototypes being tested with representatives from the application groups

• Will replace LCG-2 on production facility in 2005

Globus 2 based Web services based

gLite-2gLite-1LCG-2LCG-1



INFSO-RI-508833

• EGEE Generic Applications Advisory Panel:

– 4 applications presented– 3 applications (comp. chemistry, earth

science, astro-particle) recommended for deployment with allocation of NA4 resources

– EU GRACE project already tested

• EU projects: MammoGrid, Diligent, SEE-GRID …

• Expression of interest: Planck/Gaia (astroparticle), SimDat (drug discovery)

Who else can benefit from EGEE?



INFSO-RI-508833

EGEE training delivery

Since April 2004 EGEE NA3 has trained 350 people in 14 coursesranging from introductions to EGEE (induction) through applicationDeveloper to advanced level.

NA3 is committed to maintainingand developing the quality of training.To do this feedback is collected fromthe users of courses.

Training Events throughout Europeand beyond!

Next week 4 days in Karlsruhe5-7 October in Latvia



INFSO-RI-508833

1st & 2nd International Grid Summer Schools

Built on CERN Summer school Experience

International collaboration - GGF- Multi-national Sponsors-High profile engaging teachers

Carefully Designed Syllabus

This year’s event was attended by 84 selected advanced students – from all around the world



INFSO-RI-508833

Dissemination

• 1st project conference– Over 300 delegates came to the 4 day

event during April in Cork Ireland– Kick-off meeting bringing together

representatives from the 70 partner organisations

• 2nd conference scheduled– 22-26 November in The Hague– http://public.eu-egee.org/conferences/2nd/

• Websites, Brochures and press releases– For project and general public www.eu-

egee.org– Information packs for the general public,

press and industry



INFSO-RI-508833

Summary

• EGEE is the first attempt to build a worldwide Grid infrastructure for data intensive applications from many scientific domains

• A large-scale production grid service is already deployed and being used for HEP and BioMed applications with new applications being ported

• Resources & user groups will rapidly expand during the project

• A process is in place for migrating new applications to the EGEE infrastructure

• A training programme has started with many events already held

• Taiwan is playing a major role in LCG-2 and this should be leveraged in EGEE

• This event is an excellent opportunity to review our plans for collaboration in EGEE and EGEE second phase



INFSO-RI-508833

Further information

• EGEE project – www.eu-egee.org

• EU DataGrid – www.eu-edg.org

• The HEP LCG project www.cern.ch/lcg

• Other Grid projects - www.gridstart.org

• The Grid - www.gridcafe.org

• Questions to: [email protected]

Documents

The EGEE Grid infrastructure project: first experience and future plans