Grids and HEP Computing coordination ICFA seminar Guy Wormser IHEPCCC chair LAL Orsay

Grids and HEP Computing coordination

ICFA seminar

Guy Wormser

IHEPCCC chair

LAL Orsay

Guy Wormser, ICFA seminar, Daegu, Korea, September 30, 2005 2

Grids and HEP

• Around 1998, a new concept « GRID computing » landed in HEP physics computing:

• “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high computational capabilities.” (The Grid, I. Foster, C. Kesselman, 1998)

• It had immediately a large worldwide impact in HEP– HEP is a very distributed field, naturally structured into virtual

organisations– Huge computing needs for LHC and other projects– Embarassingly parallel style of computing well matched to

simple Grid architecture– The network bandwith “explosion”


HEP commitment to Grids

• 2000-2003 : Exploratory phase. Several R&D projects in the world to develop the middleware, build mid size test beds, port some applications– US : PPDG, Griphyn– Europe : DATAGRID– Several initiatives in Asia

• 2002: Decision to build a grid infrastructure as THE TOOL for LHC computing. Creation of the LCG project (LHC Grid Computing). Point of no return!

• 2004 Large scale deployment of the world largest distributed computing infrastructure(s):– LCG, EGEE, OSG (US), NAREGI(Japan), etc…. – >200 sites, 15 000 processors, 5 PB of storage



Grids: a great oportunity for interdisciplinary contacts

• Grids have become a great vehicle for promoting interdisciplinary contacts between HEP, many other application fields and computing scientists– Bioinformatics– Medecine– Chemistry– Fusion science– Earth sciences– Astrophysics/astronomy– Neuroinformatics– Climate– Finance– …..

• HEP can be proud to have been a key player in these endeavours


2004 Service Usage

• VOs and users on the production service– Active HEP experiments:

4 LHC, D0, CDF, Zeus, Babar

– Active other VO: Biomed, ESR (Earth Sciences), Compchem, Magic

(Astronomy), EGEODE (Geo-Physics)

– 6 disciplines– Registered users in these VO: 500– In addition to these there are many VO that are

local to a region, supported by their ROCs, but not yet visible across EGEE

• Scale of work performed:– LHC Data challenges 2004:

>1 M SI2K years of cpu time (~1000 cpu years) 400 TB of data generated, moved and stored 1 VO achieved ~4000 simultaneous jobs (~4 times

CERN grid capacity)

Number of jobs processed/month


DIRAC alone

Plus LCG1.8 106/day

LCG usepause

3-5 106/day

LCG use restart

Data Challenges – LHCb

June-September 2004186 M events in ~250k jobs . 61 Tbyte dataCPU: ~370 kSI2k years (43 LCG and 20 DIRAC sites)Increased usage of LCG with time – 75% by September; 50% LCG overallUp to 3500 concurrent running jobs in LCG

This is more than 3 times what was possible at CERN alone


D0 MC efficiency on LCG2 since Xmas (but small statistics)

–

CE Success Failed

bohr0001.tier2.hep.man.ac.uk 237 3

cclcgceli01.in2p3.fr - 14

grid-ce.physik.uni-wuppertal.de - -

gridkap01.fzk.de 2564 19

golias25.farm.particle.cz 198 15

heplnx131.pp.rl.ac.uk 246 4

lcgce02.gridpp.rl.ac.uk 293 10

mu6.matrix.sara.nl 397 7

tbn18.nikhef.nl 154 2

Total 4089 74•Efficiency 98 %•System running monitored very closely by run-manager in close contact with sites


APPLICATIONS ported on EGEE

Earth Observation by Satellite

Hydrology

Solid Earth Physics

Climate

Geosciences

Chemistry of the Mars Upper Atmosphere

Meteorology


SEISMOLOGY[1]Fast Determination of mechanisms of important earthquakes (IPGP: E. Clévédé, G. Patau)

Challenge

Provide results 24h -48h after its occurrence

Application to run on alert

Collect data of 30 seismic stations from GEOSCOPE worldwide network

Select stations and data

Definition of a spatial 3D grid +time

Run for example 50-100jobs

Peru earthquake, 23/6/2001, Mw=8.3

Data used: 15 Geoscope Stations

5 Seisms already ported: Peru, Guadeloupe, Indonesia (Dec.), Japon,

Indonesia (Feb.)


SEISMOLOGY [2]

Indonesian EQ 25 March 2005

Thanks to the grid, the best of the 30,000 solutions could be found within 30 hours after the earthquake took place. This best solution was later confirmed by comparing it with results from other analysis methods.

Many CPUs (>100) from time to time without reservation


HYDROLOGY [1]

Flood Prediction: Experimental data and Models.

L. Hluchy, Viet Tran, M. Ciglan …. (II-SAS, Bratislava Slovaquie)

CHALLENGE

Complex application:

Assimilation of data in models and cascade of simulations

Data sources

Hydrological simulation

Hydraulic simulation

Portal

Meteorological simulation

Cascade of simulations


Korba coastal aquifer

Tunisia

Cape Bon Peninsula

70km south-east of Tunis

45 km

Management of water resources in Mediterranean area (SWIMED)

G. Lecca (CRS4 Italy), P. Renard (Unine, CH), J. Kerrou (INAT, Tunisia), R. Ababou (IMFT, Fr)


GEOSCIENCES

• Generic seismic platform software, based on Geocluster commercial software developed by CGG

• Includes 400 geophysical modules, implemented on EGEE

• Used by both academics and private companies.• Free of charge for Academics, with charge for R&D


EARTH SCIENCE ORGANISATION

Coordination Earth Science: Monique Petitdidier (IPSL)

14 Research Laboratoiries and Institutes; One organisation: ESA, 3 Private companies, CGG, DutchSpace et Iguassu Software Systems

Bulgaria, France, Italy, Germany, Greece, The Netherlands, Russia, Slovakia, Switzerland, Tchequie

Around 40 persons registered in the VOs

Welcome to new partners


Status of Biomedical VO

PADOVA

BARI

15 resource centres ( )17 CEs (>750 CPUs)16 SEs

4 RBs:

CNAF, IFAE,

LAPP, UPV

RLS, VO LDAP Server:

CC-IN2P3

4 RBs

1 RLS

1 LDAP Server



Malaria Data challenge (summer 2005)


HEP computing coordination: IHEPCCC

• HEPCCC committee created in Europe in ~1980 to coordinate HEP computing. Mainly centered around CERN at the time. Membership: Directors of the main European computing centers

• Important role as a discussion forum and coordinating body throughout the years. Creation of HTASC, a permanent technical advisory body. HEPCCC father of the DATAGRID initiative.

• In 2001, it was realized that it was mandatory to include membership from all over the world to the increased worldwide nature of HEP computing

• IHEPCCC was created in 2003 as a ICFA subcommittee, « sister » to SCIC, organizer of this workshop


IHEPCCC mandate

Preamble• …an efficient information exchange on computing issues between the major HEP centers in the world.

Missions• …forum between the main persons in charge of HEP computing, by gathering and

distributing information about all relevant issues in HEP computing, and especially those with a global nature.– Typical examples : new technology trends, computing centers strategic policies,

security issues, recommendation of standard practices, presentation of R&D results, comparison of various equipments performances.

• The other missions include:– Issuing statements and recommendations concerning computing in the HEP community.– Serving as an interface to other scientific domains on matters of computing.– Working in close connection with the ICFA SCIC, the physics regional organizations, and the

HICB coordinating the grid projects in HEP.– Reporting to ICFA (ie grabbing the attention of all the HEP major lab directors)

– http://www.fnal.gov/directorate/icfa/Int'l_HEPCCC.html


IHEPCCC membership

Asia• Setsuya Kawabata (KEK-Japan) , Hesheng Chen (Beijing-China), Dongchul Son

(Daegu- South Korea), S. Banerjee(Tata-India) , Geoff Taylor (Melbourne-Australia)

North America• Paul Avery (Gainesville-US), Bruce Gibbard (BNL-US) , Richard Mount (SLAC-

US) , Randall Sobie (Victoria-Canada), Vicky White(FNAL-US), Frank Wuerthwein (San Diego-US)

Europe• Manuel Delfino (Barcelona-Spain) , Matthias Kasemann (DESY-Germany) , Mirco

Mazzucato (Padova-Italy) , Wolfgang von Rueden (CERN) , Michal Turala (Crakow-Poland), Peter Watkins (Birmingham, UK), Guy Wormser (Orsay-France), Chair

Rest of the world• Viacheslav Ilyin (Moscow-Russia), Abdeslam Houmadda (Casablanca-Morocco),

Alberto Santoro (Rio, Brazil)


Consensus on objectives

A/ Guarantee the best possible conditions for the advent on a Grid-based HEP world wide computing A.1/ Make sure that all conditions are met to foster the development of viable HEP

grid(s) A.2/ Make sure that they are interoperable and secureA.3/ Make sure that all regions of the HEP world have an easy access to them

B/ Foster collaboration and training in HEP computing B.1/ Make sure that the best possible conditions are met to foster the common

development of software tools and standards B.2/ Make sure that the best possible conditiond are met for best practice

exchange and training


IHEPCCC meetings

• Inaugural meeting May 2004 in Barcelona– http://agenda.cern.ch/fullAgenda.php?ida=a041864

• CERN Oct 2004, just after CHEP04– http://agenda.cern.ch/fullAgenda.php?ida=a044205

• South Korea, May 28 2005• Seattle Nov 12-18 2005. Colocated with SC2005• Feb 2006 Mombai, colocated with CHEP2006• Nov 2006 Europe• Spring 2007 US


Technical Advisory Body

• IHEPCCC chose to use HEPiX as the main provider of ad hoc task forces experts

• Contact taken with HEPiX chairpersons• First cases to study :

– HEP VO issues– Linux Operating systems. Report now available, thanks to A.

Silverman et al.http://agenda.cern.ch/askArchive.php?base=agenda&categ

=a053314&id=a053314s0t3/document

• Future potential topics:– Better coordination of HEP sofware development– Videoconference and other collaborating tools– Data storage management– All suggestions welcome


The last IHEPCCC meeting in Korea May 27

• CommunicationsThe presentation from many countries or regions were done during

the digital divide workshop organisnized by SCIC (D. Son and H. Newman) . They can be found at :

http://chep.knu.ac.kr/HEPDG2005/index.html• Highlights of the digital divide workshop. Summary by H.

Newman:http://chep.knu.ac.kr/HEPDG2005/upload/fri_s5/ICFADDKoreaConclude052705s3.ppt• HEPiX-IHEPCCC interaction• HEP VO• Training group status report: • Software development in HEP• Grid projects• New initiatives

– large procurements for LHC.– Licensing issues



Digital divide issues

• HEP is playing an important role in promoting science of many developping countries

• The Digital divide workshop in Korea in May was a very good illustration of this phenomemon. The virtusous circle: science/grids/high speed network is proving very efficient in many countries see

• http://chep.knu.ac.kr/HEPDG2005/index.html• CHEP06 in India• Sharing the knowledge around the Mediterranea conf in

Casablanca Sept 05• Creation of a training task force within IHEPCCC (V. Ilyin chair)

– Census of all existing training programs in HEP• Preparation of World Summit on IST : Tunis, Durban,…

– SCIC is preparing a report. IHEPCCC will help • Second common workshop between SCIC and IHEPCCC in fall

2006 in Europe


Towards a HEP VO

• GOAL: take advantage on the new technologies to offer minimum computing services to any bona fide HEP member in any HEP lab.– Solve the travelling physicist problem– Create new services for the community

• Initial proposal to Base it on Certificate technology– Requires to maintain an accurate list of HEP VO members. Has

to rely on the institutions. Requires a list of institutions– Technical implementation to be discussed with experts

• Slow progress along this line due to large security concerns and parallel progress of tehcnologies (wireless and laptop are now fully widespread, although it is not the full solution to the problem)

• Reorientation towards guidelines on list on minimal services and how on provide them.


Software issues

• The degree of duplication of the various software efforts was really striking– For instance, one Mass storage software per big lab– Many different grid flavors

• From the most tiny technical piece to the large frameworks• Could be partly be reduced by more information • How to promote the co development of HEP software (there are some

success stories like GEANT4)• Present trend to go away fom commercial software but some licensing

issues remain cf MATLAB– Work on licensing issues at HEP level

• IHEPCCC will prepare a matrix using CHEP04 presentations and calssifications to document which lab is doing what in all aspects of HEP software


Global GridCommunity


The Grid bazaar

• Probably useless and counter-productive to push for one unified GRID• AT least insist on interoperability and minimal set of grids• But INSIST very strongly that the LHC computing is ONE PROJECTIHEPCCC recommendation in preparation The LHC computing project should be considered as a whole. Different

flavours of middleware are presently used in various regions of the world and , although it could be argued that is not ideal, this situation is very likely not going to change in the coming years. Therefore, IHEPCC strongly recommends

• i) to pursue and develop all efforts aiming at the maximum interoperability. In doing so, it is important to keep in mind that the LHC grid usage for physics analysis purposes will traverse the present geographical boundaries of these middleware implementations.

• ii) to review the full LHC computing project as a single integrated project• iii) to adopt names for the various middlewares and the LHC computing

grid project that allow to better capture the collective nature fo the enterprise


Procurement and licensing issues

• a) Most of us will enter a phase of large procurements for LHC. Although it is clearly a sensitive domain, exchanging information between Hep centers either before (to negociate and/or together) or after the procurement (to provide data that can help others to get low cost) can lead to huge savings at the global scale. Can we make a recommendation to ICFA, ie to put together a group of procurement experts from the HEP centers.

• b) The question of scope if licences for commercial software used in HEP came up. – -how to be as inclusive as possible when a big lab negociates with a

vendor to nclude as many other labs as possible (beware that the labs that ill not in the deal may have to pay a much higher price)

– -how can we deploy licensed software on a grid if needed (a working group on that topic exists in EGEE).


IHEPCCC summary

• IHEPCCC recently created and off for a good start as a discussion forum and information exchange at the worldwide scale in HEP

• HEPiX has been picked up as the natural companion for technical advices, on a voluntary basis

• IHEPCCC provides a good way to attract attention of world lab directors on computing issues.

• Very fruitful session in Korea with many new initiatives– Comprehensive digital divide report showing the impact of HEP in the

virtuous circle– Linux OS status report– Training working group– HEP VO– Software development– Recommendation on LCG– Procurement and licensing

• Report on IHEPCCC activities made during the LCG deployment board on June 22


Conclusion

• Grids : hard work but a great success story so far• Huge infrastructure in place, other sciences embarked,

very high political visibility, HEP role clearly recognised

• IHEPCCC created as an exchange forum between HEP computing centers

• Presently LHC computing plays a key role in this field • IHEPCCC brings input on other aspects : non LHC HEP

activities (accelerator based or not), general issues• Useful channel to lab directors thru ICFA meetings• Very good collaboration with SCIC, the computing

network related ICFA subcommittee

Documents

Grids and HEP Computing coordination ICFA seminar Guy Wormser IHEPCCC chair LAL Orsay