Architecture/Framework Status

Architecture/Framework Status

David R. QuarrieDavid R. Quarrie

LBNLLBNL

DOE/NSF Review of U.S. ATLAS Physics and Computing ProjectDOE/NSF Review of U.S. ATLAS Physics and Computing Project

NSFNSFJune 2002June 2002

June 20, 2002June 20, 2002David R. Quarrie, LBNL DOE/NSF ReviewDavid R. Quarrie, LBNL DOE/NSF Review 2

Outline

Major ActivitiesMajor Activities

Simulation and AthenaSimulation and Athena

Support/Infrastructure ActivitiesSupport/Infrastructure Activities

LHC Computing GRID ProjectLHC Computing GRID Project

Feedback from last ReviewFeedback from last Review

SummarySummary


Event Data Model & Detector Description

Reorganization of effort focused towards DC1-2 and DC2Reorganization of effort focused towards DC1-2 and DC2 Headed by me

Detector Description Stan Bentvelsen [Nikhef] Framework EDM: Srini Rajagopalan [BNL] HLT EDM: Markus Elsing [CERN] Reconstruction EDM: David Rousseau [LAL, Orsay] Simulation EDM: Andrea Dell’Acqua [CERN]

Improve coherency and migration towards new Event Data Improve coherency and migration towards new Event Data

and Detector Description Modelsand Detector Description Models Needed for alignment support, material integration and performance

Broadens role of US-ATLAS in these areasBroadens role of US-ATLAS in these areas


Event Data Model I

Framework [BNL, LBNL]Framework [BNL, LBNL] StoreGate data access service

DataHandle & DataLink access mechanisms

Major internal redesign in conjunction with Athena changes

DataVector<T> container class supporting polymorphism,

ownership policy and relaxing constraints on items

Raw Event Data Model [BNL, CERN]Raw Event Data Model [BNL, CERN] Raw Data Object Support (IdentifiableContainer)

ByteStream Support (emulation of data from DAQ)

Implemented for most detector subsystems


Simulation Dataflow

Generator HepMC

ROD EmulationAlgorithm

Particle Filter

ROD InputDigits

RawDataObjects

ByteStreamConversionSvc

L1 Emulation(inc. L1 ROD)

L1Result

ROD Emulation

(passthru)

ByteStream

ATLAS

MCTruth/Gen

L1 Digitization

Simulation

PileUp

MCTruth/SimHits

McTruth/PileUp

Digitization

MergedHits

L1Digits

L2Result

EFResult

L2 SelectionAlgorithm

EF SelectionAlgorithm U

ses

Raw

Dat

aObj

ects


Event Data Model II

Data Dictionary [LBNL, Annecy]Data Dictionary [LBNL, Annecy] Automatic code generation from data model specification Multiple back-end code generators

Objectivity, scripting and browsing currently available ROOT support in next release (4.0.0)

Annecy software engineers managed by Craig Tull [LBNL]

Not yet reached critical massNot yet reached critical mass Hope to have 3 major subsystems as customers by end of month

Wish to redirect most of LBNL effort towards GRID Wish to redirect most of LBNL effort towards GRID

integrationintegration Lack of critical mass in data dictionary a concern


Detector Description

Broaden US-ATLAS roleBroaden US-ATLAS role Addition of Joe Boudreau [Pittsburg]

Ongoing involvement from LBNL

GeoModel (based on CDF design) is basis for detector GeoModel (based on CDF design) is basis for detector

description strategydescription strategy Augments existing APIs

Adds support for misalignments

Adds support for material integration/extrapolation

Adds uniform support for both Geant3 and Geant4

Aside: Will LCG cause us to redo this again?Aside: Will LCG cause us to redo this again?


Gaudi

0.9.10.9.1 Internal redesign of transient event store Python scripting enhancements Several bug fixes ATLAS code base not fully synchronized with LHCb code base

0.10.10.10.1 Major Python scripting enhancements AIDA 2.2, 3-D Histograms, dynamic resizing of Zebra commons ApplicationMgr Refactoring Object Introspection ATLAS and LHCb code bases converged (almost)

Gaudi collaboration still proving successfulGaudi collaboration still proving successful Extending into GRID and software distribution arenas


Scripting

At the time of the last review we had just lost Chris Day At the time of the last review we had just lost Chris Day

[LBNL] because of funding shortfall[LBNL] because of funding shortfall Responsibility for Scripting and Software Process (USDP)

Partial replacement (PostDoc) started at LBNL in mid-Partial replacement (PostDoc) started at LBNL in mid-

JanuaryJanuary Focus on scripting support

Development acceleratingDevelopment accelerating Area of close collaboration with LHCb

Python as component “glue” as well as scripting language


GUI

Python as “Glue”

Python

mathmath

shellgaudipython

DatabaseEDG API

GUI

Very rich set of Python standard

modules

Several GUI toolkits

XML

Very rich set specialized

generic modules

Athena/GaudiFramework

rootpython

RootClasses

PVSS

JPE

JavaClasses

LHC modules

Gat

eway

s to

oth

er

fram

ewor

ks


Pile-Up and Refactoring

Pile-Up [LBNL]Pile-Up [LBNL] Prototype available on schedule Needs to be stress tested by real detector subsystems

SCT in next release, LAr before Sept. LAr is interesting because of long integration times and memory size

“Production” version by end 2002

Refactoring [LBNL]Refactoring [LBNL] Reverse engineering of Gaudi internals leading to a cleaner, more

robust re-design Not visible to client code Incorporated into Gaudi 0.10.1 Approach being extended to other areas


GRID enabling Athena

Much of effort so far has been on developing middlewareMuch of effort so far has been on developing middleware

Data access and management, job submission, Data access and management, job submission,

authentication, etc.authentication, etc. ANL and BNL efforts have focussed on this

ATLAS/LHCb also doing job submission & software distribution

LBNL group begun to look at integration into Athena itselfLBNL group begun to look at integration into Athena itself Initial testbed incorporates GRID monitoring capabilities

Collaboration with Valerie Taylor (NorthWest)

Extend to Grid Monitoring Architecture (GMA) developed at NERSC MonitorSvc


Simulation and Athena

Direct Geant4 integration into Athena has been endorsed Direct Geant4 integration into Athena has been endorsed

by new Simulation coordinatorby new Simulation coordinator Prototype [LBNL] incorporated into release

Some legacy FADS/GOOFY useSome legacy FADS/GOOFY use Not planned to be used for production

G4 component of DC1-2 will use integrated systemG4 component of DC1-2 will use integrated system

Geant3 simulation (atlsim) also being directly integrated into Geant3 simulation (atlsim) also being directly integrated into

Athena [BNL, LBNL]Athena [BNL, LBNL] Prototype exists but not ready for production use

Goal is full symmetry between G3 and G4Goal is full symmetry between G3 and G4


Support Activities

TutorialsTutorials Lot of emphasis on tutorials Introductory Athena tutorial web-based [LBNL, U. Michigan] Advanced tutorials on Data Dictionary and StoreGate

DocumentationDocumentation Separate Developer Guide and User Guide & Tutorial New version released next week

Feedback from users shows still need to work on help, bug Feedback from users shows still need to work on help, bug

tracking and documentationtracking and documentation Lack of a bug tracking system is a real problem


Configuration Management

Migration from previous SRT tool to CMT completeMigration from previous SRT tool to CMT complete

Not a US-ATLAS deliverable but we played a major roleNot a US-ATLAS deliverable but we played a major role Developing policies and procedures

Trial releases to uncover problems and inconsistencies

Providing developer help and consultancy

Time to review experience and revisit some policies?Time to review experience and revisit some policies?

Support situation in ATLAS improved with formation of Support situation in ATLAS improved with formation of

Infrastructure GroupInfrastructure Group Librarian, Release Coordinator, etc.

Hopefully will lessen our “fire-fighting”


US ATLAS Presence at CERN

LBNL has one developer at CERN full-timeLBNL has one developer at CERN full-time Massimo Marino Craig Tull returned to LBNL

Goal is to maintain approximately 1-2 people thereGoal is to maintain approximately 1-2 people there Massimo’s tour ends in April 2003

Has proven to be really useful and beneficialHas proven to be really useful and beneficial

But a strain on maintaining critical mass at LBNLBut a strain on maintaining critical mass at LBNL Especially given funding uncertainties

I’d like to encourage this commitment to be augmented by I’d like to encourage this commitment to be augmented by

othersothers


LHC Computing GRID Project

Goal is reduction in cost via common projectsGoal is reduction in cost via common projects Both hardware and software

US-ATLAS actively involved in RTAGs and first work project US-ATLAS actively involved in RTAGs and first work project Persistency

Short-term penaltyShort-term penalty Maintaining effort on short term deliverables

Some might be superceded by common project deliverables later

While encouraging LCG involvement

Compare with Norman McCubbin’s comment on GRIDCompare with Norman McCubbin’s comment on GRID “Today it’s a net negative for ATLAS”

A lot of hopes and worry focused on first projectA lot of hopes and worry focused on first project CERN’s prior history has not been good


Recommendations from last Review (1)

1.1. With the chief architect, the US-ATLAS group commands an important position in the With the chief architect, the US-ATLAS group commands an important position in the

international collaboration. This will lead inevitably to the diversion of manpower for international collaboration. This will lead inevitably to the diversion of manpower for

solving short-term problems in the future as it has in the past. solving short-term problems in the future as it has in the past. The committee The committee

recommends to the international collaboration that it provide the chief architect recommends to the international collaboration that it provide the chief architect

with resources to fulfill that role.with resources to fulfill that role.

Until that issue is resolved we recommend to US-ATLAS to continue this kind of Until that issue is resolved we recommend to US-ATLAS to continue this kind of

fire-fighting for the common good of ATLAS.fire-fighting for the common good of ATLAS.

No change

My direct responsibilities have increased with EDM and Detector Description But these are short-term, focused on DC1-2 and DC-2

I have responded by handing off most direct responsibility for Athena Paolo Calafiura [LBNL]

We are continuing the “fire-fighting” policy


Recommendations from last Review (2)

1.1. An additional drain of manpower is the continuous support of software that has not been An additional drain of manpower is the continuous support of software that has not been

approved by the collaboration. approved by the collaboration. The committee recommends to international ATLAS The committee recommends to international ATLAS

management to enforce decisions about choices of software to the collaboration.management to enforce decisions about choices of software to the collaboration.

2.2. The committee recommends to the US-ATLAS software group to be less willing to The committee recommends to the US-ATLAS software group to be less willing to

take up additional workloadtake up additional workload purely driven by part of the collaborations non-compliance purely driven by part of the collaborations non-compliance

with already taken software decisions.with already taken software decisions. Some progress on this Direct integration of Geant4 into Athena adopted. Joe Boudreau (Pittsburg) added to Detector Description effort

ATLAS international is strongly encouraged to provide a concrete staffing plan for DC1ATLAS international is strongly encouraged to provide a concrete staffing plan for DC1 Not really in place Staffing augmented by reducing CERN database contribution


Summary

Major focus is the Data ChallengesMajor focus is the Data Challenges

Increased short-term emphasis on Event Data Model and Increased short-term emphasis on Event Data Model and

Detector DescriptionDetector Description

Continuing good collaboration with LHCb on Athena/GAUDIContinuing good collaboration with LHCb on Athena/GAUDI

Slow integration of GRID capabilities into AthenaSlow integration of GRID capabilities into Athena

LCG Ramp-up is causing strain on short-term deliverablesLCG Ramp-up is causing strain on short-term deliverables

Documents

Architecture/Framework Status