19/03/2004 P.Hristov 1
ALICE Physics Data Challenge’04
P.HristovMarch 19, 2004
CERN
19/03/2004 P.Hristov 2
Goals(http://cern.ch/fca/ALICE-DCs.doc)
■ Determine readiness of the off-line framework for data processing
■ Validate the distributed computing model■ PDC’2004:10% test of the final capacity
■ Complete chain used for trigger studies■ Prototype of the analysis tools■ Comparison with parameterized MC■ Simulated RAW data
■ PDC’04 physics: hard probes (jets, heavy flavours) & pp physics
19/03/2004 P.Hristov 3
Physics Data Challenge'2004
■ Simulation: 10^5 Pb-Pb + 10^7 p-p 102 TB ■ 450 KSI2K (~ tier-1 capacity) x 3 months■ Distributed production, then data are shipped
to CERN■ Reconstruction: 5x10^6 Pb-Pb+10^7 p-p 187 TB
■ Reconstruction is shared between CERN & outside centres according to available resources
■ Data originate from CERN ■ Analysis: 5x10^6 Pb-Pb+10^7 p-p 13 TB ■ See http://aliweb.cern.ch/people/phristov
/PDC04.html
19/03/2004 P.Hristov 4
PDC’04 Strategy■ Part 1: underlying events
■ Distributed simulation, production of summable digits, digitization, clusterization, reconstruction, PID, and generation of ESD
■ Data transfer to CERN: kinematics, track references, summable digits (hits for some detectors)
■ Part 2: signal events & test of CERN as data source■ Distributed simulation, production of summable
digits, merging, digitization, clusterization, reconstruction, PID, generation of ESD
■ Part 3: distributed analysis
19/03/2004 P.Hristov 5
AliRoot Layout
ROOT
AliRoot
STEER
Virtual MC
G3 G4 FLUKA
HIJING
MEVSIM
PYTHIA6
EVGEN
HBTP
HBTAN
ISAJET
AliE
n
EMCAL ZDCITS PHOSTRD TOF RICH
ESD
AliAnalysis
AliReconstruction
PMD
CRT FMD MUON TPCSTART RALICESTRUCT
AliSimulation
NEW
19/03/2004 P.Hristov 6
Current Status
■ Major changes in the last year■ New multi-file I/O finally in full production■ New coordinate system■ New reconstruction and simulations classes■ First attempt at the ESD and analysis framework■ Improvements in reconstruction and simulation
■ Clearly the system works well, however a lot of changes to come
■ ESD: the philosophy is still evolving■ Introduction of FLUKA and new geometrical modeller■ Development of the analysis framework■ Raw data for all the detectors -- we need them for the
data challenge■ Introduction of the condition database infrastructure
19/03/2004 P.Hristov 7
CERN
Tier2
Tier1
Tier2
Tier1
Production of RAW
Shipment of RAW to CERN
Reconstruction of RAW in all T1’s
Analysis
AliEn job control
Data transfer
PDC’04 Schema
19/03/2004 P.Hristov 8
Signal-free event Merged
signal
Merging
19/03/2004 P.Hristov 9
Alien CE
LCG UIAlien
CEs/SEs
Server
User submits jobs
Catalog
LCG RB
LCG CEs/SEs
LCG LFN
LCG PFN
LCG LFN = AliEn PFN
Catalog
AliEn, Genius & EDG/LCG
19/03/2004 P.Hristov 10
QLCG CPUgood jobs
LCG
CPUavailableLCG ; QAliEn
CPUgood jobsAliEn
CPUavailableAliEn
ALICE PDC04 & LCG
■ All the production is started via AliEn, the analysis will be done via Root/Proof/AliEn
■ LCG-2 is one CE element of AliEn, which integrates seamlessly LCG and non LCG resources
■ If LCG-2 works well, it gets a large amount of jobs, and it is used heavily
■ If LCG-2 does not work well, AliEn will privilege other resources, and it will be less used
■ In all cases we will use LCG-2 as much as possible■ We will not need to take any decision: the performance
of the system will decide for us
19/03/2004 P.Hristov 11
Short History
■ Jan 03: Requirements for ALICE PDC04 presented to PEB
■ End Dec 03: Announcement of LCG-2 by mid February 2004
■ Beg Jan 04: Decision to delay PDC04 by one month waiting for LCG-2
■ Beg Jan 04: LCG announces that there will be no SE in LCG-2
■ Beg Feb 04: The WAN resources allocated by LCG for data storage are insufficient/inadequate
■ Mid Feb 04: Development of an ALICE solution, developed in haste and working against all odds!
■ End Feb 04: IT has also come up with a solution responding to a CMS requirement
■ End Feb 04: Production started, new sites being added
■ End Feb 04: Tape vault flooded -- our tapes have been spared
■ Beg Mar 04: castor nameserver has to be reinstalled (running on Linux 6.2)
■ Beg Mar 04: castor servers have to be reinstalled for security
■ Beg Mar 04: LCG RB works differently on the different centres. ■ e.g. CNAF has to be switched on and off by hand, otherwise it “swallows” all the jobs!
■ Beg Mar 04: we are obtaining now close to 10 TB
■ Mid Mar 04: Files on the IT-provided pool are erased before being copied on tape
19/03/2004 P.Hristov 12
Data Challenge Statistics
Picture from yesterday, 18/03/2004
19/03/2004 P.Hristov 13
Data Challenge Statistics
19/03/2004 P.Hristov 14
Data Challenge Statistics
19/03/2004 P.Hristov 15
Considerations
■ LCG is providing a lot of cycles■ ALICE is the first to use the system for production■ This required continuous efforts and
interventions (ALICE and LCG), particularly due to lousy workload scheduling and lack of stability
■ The lack of an SE will make reconstruction and analysis possible only under AliEn
■ Relations with LCG are in general good■ They are sincerely willing to help■ But the system was not fully prepared for our
PDC’04■ LCG PR / planning can be improved!
19/03/2004 P.Hristov 16
Considerations (cont)
■ Next time we will start six months before!■ LCG needs to be “prompted” for resources and
support■ Some ALICE people did not get well the philosophy
of a DC ■ The period Jan-Feb was well spent
■ Changes in AliRoot improved performance and results
■ AliEn now has a more advanced SE solution■ The Offline members reacted extremely well to
pressure and the exercise is definitely very useful■ We will reach the objectives!
19/03/2004 P.Hristov 17
Period(milestone)
Fraction of the final capacity (%)
Physics Objective
06/01-12/01 1% pp studies, reconstruction of TPC and ITS
06/02-12/02 5%
• First test of the complete chain from simulation to reconstruction for the PPR
• Simple analysis tools• Digits in ROOT format
01/04-06/04 10%
• Complete chain used for trigger studies• Prototype of the analysis tools• Comparison with parameterised
MonteCarlo• Simulated raw data
05/05-07/05 TBD• Refinement of jet studies• Test of new infrastructure and MW• TBD
01/06-06/06 20%• Test of the final system for
reconstruction and analysis
ALICE Physics Data Challenges
NEW NEW