View
216
Download
0
Category
Tags:
Preview:
Citation preview
Oxana SmirnovaLCG/ATLAS/Lundoxana.smirnova@cern.chNovember 11, 2002, Uppsala4th NorduGrid Workshop
ATLAS Data Challenges on EDG
2002-11-11 Oxana.Smirnova@cern.ch 2
EU Datagrid project
Started on January 1, 2001, to deliver by end 2003 Aim: to develop a Grid middleware suitable for High Energy
physics, Earth Observation and biology applications Initial development based on existing tools, e.g., Globus,
LCFG, GDMP etc The core testbed consists of the central site at CERN
and few facilities across the Western Europe; many more sites are foreseen to join soon Italy, UK come with several sites each; Spain, Germany and
others – via the Crossgrid ATLAS-affiliated sites: Canada, Taiwan etc
By now reached the stability level sufficient to test submission of production-style tasks
2002-11-11 Oxana.Smirnova@cern.ch 3
EDG Testbed
EDG is committed to create a stable testbed to be used by applications for real tasks This started to materialize in mid-August… …and coincided with the ATLAS DC1 ATLAS asked and was given the first priority
Most sites are installed from scratch using the EDG tools (RedHat 6.2 based) NIKHEF: EDG installation and configuration
only Lyon: installation on the top of existing farm A lightweight EDG installation is available
Central element: the Resource Broker (RB), distributes jobs between the resources Currently, only one RB (CERN) is available
for applications In future, may be an RB per Virtual
Organization (VO)
2002-11-11 Oxana.Smirnova@cern.ch 4
EDG functionality as of today
UI
CASTOR
RC
CE
CE
CE
lxshare0393.cern.chRB
lxshare033.cern.ch
testbed010.cern.ch
lxshare0399.cern.chdo rfcp
rfcp
rfcp
replicate
GDMP or RM
GDMP or RM
jdl
LDAP NFS
RSL
OutputGDMP or RM
Chart borrowed from Guido Negri’s slides
Input
Input
Output
2002-11-11 Oxana.Smirnova@cern.ch 5
ATLAS is eager to use Grid tools for the Data Challenges ATLAS Data Challenges are already on the Grid (NorduGrid, VDT) The DC1/phase2 (starting now) is expected to be done using the Grid
tools to a bigger extent ATLAS-EDG Task Force was put together in August with the aims:
To assess the usability of the EDG testbed for the immediate production tasks
To introduce the Grid awareness to the ATLAS collaboration The Task Force has representatives both from ATLAS and EDG:
40+ members (!) on the mailing list, ca 10 of them working nearly full-time
The initial task: to process 5 input partitions of the Dataset 2000 at the EDG Testbed + one non-EDG site (Karlsruhe); if this works, continue with other datasets
ATLAS-EDG Task Force
2002-11-11 Oxana.Smirnova@cern.ch 6
Execution of jobs
It was expected that we can make full use of the Resource Broker functionality Data-driven job steering Best available resources otherwise
Input files are pre-staged once (copied from CASTOR and replicated elsewhere)
A job consists of the standard DC1 shell-script, very much the way it is done on a conventional cluster
A Job Definition Language is used to wrap up the job, specifying: The executable file (script) Input data Files to be retrieved manually by the user Optionally, other attributes (maxCPU, Rank etc)
Storage and registration of output files is a part of the job script: i.e., application manages output data the way it needs
2002-11-11 Oxana.Smirnova@cern.ch 7
Hurdles
EDG can not replicate files directly from CASTOR and can not register them in the Replica Catalog
Replication was done via CERN SE; EDG is working on a better (though temporary) solution. CASTOR team writes a GridFTP interface, which will help a lot.
Big file transfer interrupts after 21 minutes Also known Globus GridFTP server problem, temporary fixed by
using multi-threaded GridFTP instead of EDG tools Jobs were “lost” by the system after 20 minutes of execution
Known problem of the Globus software (GASS Cache mechanism), temporary fixed on expense of frequent job submission
Static information system: if a site goes down, it should be removed manually from the index
Attempts are under way to switch to the dynamic hierarchical MDS; not yet stable due to the Globus bugs
2002-11-11 Oxana.Smirnova@cern.ch 8
Other minor problems
Installation of ATLAS software: Cyclic dependencies External dependencies, esp. on system software
Authentication & authorization, users and services EDG can’t accept instantly a dozen of new country Certificate Authorities Default proxy lives only 12 hours – users keep forgetting to request longer ones
to accommodate long jobs Documentation
Is abundant and not very much user-oriented Things are improving as more users are coming
Information system faulty information providers, affecting brokering very difficult to browse/search and retrieve relevant info
Data management information about existing file collections is not easy to find management of output data is mostly manual (can not be done via JDL)
General instability of most EDG services
2002-11-11 Oxana.Smirnova@cern.ch 9
Achievements:
A team of hard-working people across the Europe (ATLAS VO is 45 members strong as of today)
ATLAS software (starting from release 3.2.1) is packed into relocateable RPMs, distributed and validated elsewhere
DC1 production script is “gridified”, submission script is produced
User-friendly testbed status monitor and ATLAS VO information page are deployed
5 Dataset 2000 input files are replicated to 5 sites (2 @ each) Two production-style tests completed:
100 first partitions of the Dataset 2000 are processed Other (smaller) datasets: 4 input files (ca 400 MB each)
replicated to 4 sites; 250 jobs submitted, adjusted to run ca 4 hours each. The jobs were distributed across all the testbed by the Resource Broker
2002-11-11 Oxana.Smirnova@cern.ch 10
Summary
Environment Success/failure rate
Job execution Data management
Testbed 1.2.0 GASS Cache problems, 100% failure Big file replication fails (GridFTP timeout); no CASTOR support
Testbed 1.2(.1)
only CERN site is available, GASS Cache “unfixed”
Half of the Dataset 2000 jobs are executed, 100% success
Not applicable (only one site is used)
Testbed 1.2.2
All the core sites have GASS Cache “unfixed”
400 short jobs are executed across the testbed; the rest of the Dataset 2000 jobs proceeded with > 50% re-submission rate
Short files are replicated everywhere; longer files are copied manually (GridFTP not fixed)
Testbed 1.3
A.K.A. “The Showstopper” release
To be tested (GASS Cache is expected to be fixed)
To be tested (GridFTP is expected to be fixed)
2002-11-11 Oxana.Smirnova@cern.ch 11
What next
Testbed 1.3 is available for testing (not on production site yet) from today
Precise quantification of failure/success rate using Dataset 2000 partitions to be done on Testbed 1.3
ATLAS DC1, pile-up: the runtime environment is ready, scripts are prepared
o Testbed feature: the “old” runtime environment (3.2.1) has to be replaced with a new one (4.0.1)
CASTOR-EDG interface has to be tested; GridFTP server on CASTOR is expected to arrive soon
Some ATLAS production sites may join the EDG Testbed soon
Ingo AugustinVandy Berten
Jean-Jacques BlaisingFrederic BrochuStephen Burke
Serban ConstantinescuFrancois EtienneMichael GardnerLuc GoossensMarcus HardtFrank Harris
Fabio HernandezBob Jones
Roger JonesChristos Kanellopoulos
Andrey KiryanovPeter Kunszt
Emanuele LeonardiCal Loomis
Fairouz Malek-OhlssonGonzalo Merino
Armin NairzGiudo Negri
Steve O'NealeLaura Perini
Gilbert PoulardAlois Putzer
Di QingMario Reale
David RebattoZhongliang RenSilvia Resconi
Alessandro De SalvoMarkus Schulz
Massimo SgaravattoOxana Smirnova
Chun Lik TanJeff Templon
Stan ThompsonLuca Vaccarossa
Peter Watkins
No animals were harmed in the production tests
MMII
Recommended