LHCb Software Meeting 28.03.01 Glenn Patrick1 First Ideas on Distributed Analysis for LHCb LHCb Software Week CERN, 28th March 2001 Glenn Patrick (RAL)

LHCb Software Meeting 28.03.01

Glenn Patrick 1

First Ideas on Distributed Analysis for

LHCb

LHCb Software WeekCERN, 28th March 2001

Glenn Patrick (RAL)

http://hepwww.rl.ac.uk/lhcb/physics/lhcbcern280301.ppt


Glenn Patrick 2

Analysis and the Grid?

Monte-Carlo Production is readily mapped onto a GridArchitecture because: It is a well defined problem using the same executable. Already requires distributed resources (mainly cpu) in

large centres (eg. Lyon, RAL, Liverpool ...). Few people involved.

Analysis is much more inventive/chaotic and will involvefar more people in a wide range of institutes. How easilythis is perceived to map onto the Grid depends on wherewe sit on the Hype Cycle ....


Glenn Patrick 3

Hype Cycle of Emerging Technology

Courtesy of Gartner GroupTime

Hype

Peak of Inflated Expectations

Trough of Disillusionment

Slope of Enlightenment

Plateau of Productivity

Trigger


Glenn Patrick 4

Issues

There are two basic issues:

What is the data model for the experiment? Most work on this was done BG (before Grid). Is it still relevant?

Do we move analysis jobs to the data or the data to the jobs?What is the minimum dataset required for analysis (AOD,ESD)?Are we accessing objects or files?Interactive versus batch computing.

What services and interfaces have to be provided togrid-enable the LHCb analysis software?Difficult until a working Grid architecture emerges.Have to make a start and gradually evolve?


Glenn Patrick 5

Data Model Networking Evolution

In UK, WorldCom is providing the national backbone for SuperJanet4 from March 2001.

2000 SuperJanet3 155 Mbit/s1Q2001 16 x SuperJanet3 2.5 Gbit/s4Q2001 64 x SuperJanet3 10 Gbit/s2Q2002 128 x SuperJanet3 20 Gbit/s

Few years ago - Most bulk data was moved by tape. Now - Almost all data from RAL is moved over the network. More scope for moving data to the application?


Glenn Patrick 6

Scotland via EdinburghScotland viaGlasgow

WorldComGlasgow

WorldComEdinburgh

NorMAN

YHMAN

EMMAN

EastNet

External Links

LMN

Kentish MAN

LeNSE

SWAN &BWEMAN

South Wales MAN

TVN

MidMAN

Northern Ireland

North WalesMAN

NNW

C&NLMAN

WorldComWarrington

WorldComLeeds

WorldComReading

WorldComLondon

WorldComReading

WorldComPortsmouth

155Mbit/s single fibre interface

622Mbit/s singlefibre interface

2.5Gbit/s singlefibre interface

2.5Gbit/s dualfibre interface

2.5Gbit/sdevelopment network

SuperJanet4 UK Backbone, March 2001


Glenn Patrick 7

Data ModelLast Mile Problem?

Having a fast backbone is not much use if localbottlenecks exist (typically 100 Mbit/s). Need to dopoint-to-point tests using realistic datasets.

Connection Rate Tape(750MB)RAL CSF RAL PPD1600kB/s 8 minutesRAL CSF CERN 360kB/s 35 minutesRAL CSF Liverpool ~90kB/s 2.3 hours

Very crude tests done on a bad day. Need to perform spectrum of tests with realistic datasets, new tools, etc.Parallel Grid-FTP(multiple streams) 1MB/s RALCERN But increasing data flow down the analysis chain...


Glenn Patrick 8

ESD: Data or Monte CarloESD: Data or Monte Carlo

Event Tags Event TagsEvent Selection

Analysis Object DataAnalysis Object DataAnalysis Object DataAnalysis Object DataAnalysis Object Data

AOD

Analysis Object Data

AODCalibration DataCalibration Data

Analysis, Skims

Raw DataRaw Data

Tier 0,1 Collaboration

wide

Tier 2 Analysis

Groups

Tier 3, 4 Physicists

Physics Analysis

Physics

Objects Physics

Objects

Physics

Objects

INC

RE

AS

ING

DA

TA

FLO

W

Ref: Tony Doyle(WP2/ATLAS)


Glenn Patrick 9

AOD Group Analysis Tags

Physics Analysis

Private Data (e.g. ntuple)

Analysis Workstation

Physics results

Analysis Cycle(for each physicist)

Which Datasets are really needed for Analysis?

For event with “interesting” Group Analysis Tags

Calibration Data

Few physicists and for very few events

Raw Data

ESD

Some physicists for small sample of events

Generator Data

For Monte Carlo events

Likely to be different requirements atstartup.


Glenn Patrick 10

Datasets 2007 - Hoffman

ALICE(pp) ATLAS CMS LHCbRAW per event 1MB 1MB 1MB0.125MBESD per event 0.1MB 0.5MB 0.5MB 0.1MBAOD per event 10kB 10kB 10kB 20kBTAG per event 1kB 0.1kB 1kB 1kBReal Data Storage 1.2PB 2PB 1.7PB 0.45PBSimulation Storage 0.1PB 1.5PB 1.2PB 0.36PBCalibration Storage 0.0 0.4PB 0.01PB 0.01PB


Glenn Patrick 11

Physics Use-Cases

Baseline model assumes: Production Centre stores all phases of data (RAW,

ESD, AOD and TAG). CERN is production centre for real data. TAG and AOD datasets shipped to Regional Centres. Only 10% of ESD data moved to outside centres.

LHCb has smaller dataset sizes (but perhaps morespecialised requirements) more options available?Even with 2 x 109 events/year, total AOD sample is

only40 TB/year.


Glenn Patrick 12

Analysis InterfaceGaudi meets the Grid?

Gaudi Services Application Manager Job Options Service Detector Description EventData Service Histogram Service Message Service Particle Property Service GaudiLab Service

Grid Services Information Services Scheduling Security Monitoring Data Management Service Discovery Database Service?

Meta Data

Data

Standard Interfaces & Protocols

Most Grid services are producers or consumersof meta-data

Logical DataStoresEvent

DetectorHistogram

Ntuple


Glenn Patrick 13

High Level Interfaces

Need to define high-level Grid interfaces essential to

Gaudi, especially relating to data access.

For example:Data Query

Data Locator Data Mover

CASTOR HPSS Other MSS

Medium Level

Low Level

High LevelData Replication


Glenn Patrick 14

Analysis and the Grid

In the Grid, analysis appears to be seen as a series of

hierarchical queries (cuts) on databases/datasets:

eg. (PTRACK < 150.0) AND (RICHpid = pion) Architectures based on multi-agent technology. Intelligent agent is a software entity with some

degree of autonomy and can carry out operations on behalf of a user or program.

Need to define “globally unique” LHCb namespace(s). ATF proposes using URI syntax…

eg. http://lhcb.cern.ch/analy/Bpipi/event1.dat


Glenn Patrick 15

Agent Architecture(Serafini et al)

User 2User 1 User n

Agent Based Query Facilitator

Query

Execution

Strategies

Caching

Strategies

INDEX

MSS 1

Cache/DiskTape robotics

MSS 2


MSS k


Contains variety of agents: User agents Index agents MSS agents


Glenn Patrick 16

RAL CSF

236 Linux cpu

IBM 3494 tape robot

LIVERPOOL

MAP300 Linux cpu

CERN

RAL (PPD)

Bristol

Imperial College

Oxford

GLASGOW/EDINBURGH

“Proto-Tier 2”

Evolving LHCb Analysis Testbeds?

Institutes

RAL DataGrid Testbed

Cambridge

FRANCE

ITALY

:


Glenn Patrick 17

Conclusions

1. Need better understanding of how Data Model willreally work for analysis. Objects versus files?

2. Pragmatic study of performance/topology/limitations of national (and international) networks.

feed back into 1.3. Require definition of high-level Grid services which

can be exploited by Gaudi. Agent technology?4. Need some realistic “physics” use-cases.

feed back into 1 and 3.5. Accumulate experience of running Gaudi in a

distributed environment (eg.CERN UK).

Documents

LHCb Software Meeting 28.03.01 Glenn Patrick1 First Ideas on Distributed Analysis for LHCb LHCb Software Week CERN, 28th March 2001 Glenn Patrick (RAL)