The ALICE Data-Acquisition Software Framework DATE V5

F. Carena, W. Carena, S. Chapeland, R. Divià, I. Makhlyueva, J-C. Marin, K. Schossmaier, C. Soós, P. Vande Vyvre, A. Vascotto

for the ALICE collaboration

CHEP0613 – 17 February 2006T.I.F.R. Mumbai, India

CHEP06, 13 - 17 February 2006

2ALICE DAQ Software Framework DATE V5

ALICE Experiment

Counting

rooms

Cavern

Computing Center

DDLs

10Gb Ethernet

Machine: LHC @ CERN, Point 2ALICE detectors: 17Online Systems: ECS, DAQ, TRG, HLT, DCS Readout: ~400 DDLs @ 2125 MbpsStorage rate: max. 1.25 GB/sEvents: 2.5 MB (p-p), 87 MB (Pb-Pb)DAQ Software: DATE V5

DAQ

CHEP06, 13 - 17 February 2006

3ALICE DAQ Software Framework DATE V5

DATE V5 Software

ALICE Data Acquisition and Test Environment (DATE): software framework for the ALICE DAQ system

Latest DATE version V5: for the test and commissioning phase (Hall SXL2)

DATE V5.x kits:RPM package (~20 MB), User’s Guide (~470 pages)

Platforms:IA32 family, SLC3 using a 2.4 kernel

Development:~20 packages, mostly in C, using CVS

Auxiliary Software:DIM, SMI, MySQL, Tcl/Tk, libshift, ROOT, VMEbus driver

Testing:reference system, on-line data challenges, test beams

DATE V5

Packages

CHEP06, 13 - 17 February 2006

4ALICE DAQ Software Framework DATE V5

Architecture

CTPFEEs

HLT Farm

DDLs

TTC

CTP = Central Trigger Processor

TTC = Trigger, Timing, and ControlFEE = Front-End Electronics

HLT = High-Level Trigger

DDL = Detector Data Link

StorageNetwork

PDS

TDS

10Gb Ethern

et

TDS = Transient Data Storage

PDS = Permanent Data Storage

SoftwareFramework

DATE V5

LDCs

GDCs

Event-BuildingNetwork

Servers

LDC = Local Data Concentrator

GDC = Global Data Collector

~200

~50

~10

CHEP06, 13 - 17 February 2006

5ALICE DAQ Software Framework DATE V5

LDC Dataflow

equ. header

LDC header

equ. header

sub-event

payload

CDH

fragment

fragment

fragment

LDCs

DDLs

payload

CDH

fragment

payload

CDH

fragment

readout

recorder

edmAgent

htlAgent

readout process:- an “equipment” handles one data source- equipment library for DDL, VMEbus, etc.- merges fragments into sub-events - checks the Common Data Header (CDH)

recorder process: - streamlines sub-events- writes sub-events onto local disks - sends sub-events to GDCs

edmAgent process:- dispatches the sub-event to the GDCs- guided by the Event Distribution Manager (EDM)

htlAgent process:- handles the HLT decision and data- instructs to drop/keep the sub-event

32 bytes: - orbit number- bc number- trigger class- FEE status/error

68 bytes:- size- event ID- event type/id- timestamp- LDCGDC

24 bytes:- size- equipment type/id

CHEP06, 13 - 17 February 2006

6ALICE DAQ Software Framework DATE V5

DDL Interfaces of an LDC

LDCs

CTPFEEs

HLT Farm

DDLs

TTC

CTP LDC: - CTP readout (L2a information)- Interaction Record (bunch crosssings orbit)

FEE LDC: - LDC dataflow

LDC FEE:- data blocks and commands to the FEE- Front-end Control and Configuration (FeC2) scripts- C API by the rorc library

FEE HLT Farm:- send a copy of the incoming fragments- performed by the DDL hardware

HLT Farm LDC:- trigger decisions- compressed data- results of HLT processing

CHEP06, 13 - 17 February 2006

7ALICE DAQ Software Framework DATE V5

GDC Dataflow

LDC header

sub-event

full event

GDCs

LDC header

sub-event

LDC header

sub-event

GDC header

sub-event

sub-event

sub-event

68 bytes:- size- event ID- event type/id- timestamp- GDC

eventBuilder

mStreamRecorder

eventBuilder process:- handles incoming sub-events- assembles the full events- guided by event-building rules + LDC headers- tells its load to the EDM

mStreamRecorder process: - get full events via a memory mapped scheme- records with multiple stream- data formats: raw binary, ROOT tree- transmission protocol: standard, RFIO, ROOTd

CHEP06, 13 - 17 February 2006

8ALICE DAQ Software Framework DATE V5

Control

LDCs

GDCs

Servers

rcServer

rcServer

runControl

rcServer process:- launched by the xinetd daemon- starts/stops DATE processes- gets commands via DIM- shared memory control region

runControl process:- one per data acquisition; several are allowed- interface to get commands from operator or ECS

logic engine process:- hosts finite state machines (SMI)- communicates with the rcServer processes (DIM)

runControl, Status display, SMI

display Experiment

Control System

(ECS)

logicEngine

CHEP06, 13 - 17 February 2006

9ALICE DAQ Software Framework DATE V5

Static Database

Servers

Roles: LDC, GDC, EDM, detectors, trigger mask Trigger: relationship detector and trigger mask Detectors: composition of detectors/sub-

detectors Memory banks: type, size, name per role Event building: policies for build, no-build, partial Readout equipments: type, parameter per LDC

MySQLor

text files

graphical tools for

configuration configuration

files >EQTYPES

>Rand 1 GENDATA

EvMinSize %ld

EvMaxSize %ld

EqId %hd

triggerPattern %hd

>Timer 4 TRIGGER

EvInterval %ld

>GLOBALPARAMS

>LDCS

>pcald10

+ Timer (TimerAsTrigger)

1000

+ Rand (1stRand)

10000 10000000 13 4

>pcald12

...

CHEP06, 13 - 17 February 2006

10ALICE DAQ Software Framework DATE V5

Monitoring statsBrowser + infoLogger:

- collect and display statistics of the runs- Web access

infoBrowser + infoLogger:- collect log messages from LDCs, GDCs, servers - display, search, export log messages

AFFAIR:- performance plots of DATE specific values - utilization of system resources- Web access

MOOD:- monitoring data quality- online/offline monitoring- local/remote monitoring- plug-in ROOT modules- based on the DATE monitoring library

CHEP06, 13 - 17 February 2006

11ALICE DAQ Software Framework DATE V5

Performance

LDC with 4 DDLsSLC3 + DATE V5.13ALICE Silicon Pixel Detector Max. event rate: 3500 Hz

Example: Setup for integrating detector electronics

with the DAQ

0

100

200

300

400

500

600

700

0 50000 100000 150000 200000 250000 300000 350000 400000

event size [Bytes]

throughput [MBytes/s]

4x DDL

3x DDL

2x DDL

1x DDL

CHEP06, 13 - 17 February 2006

12ALICE DAQ Software Framework DATE V5

Conclusions

DATE V5: - covers the whole range of DAQ tasks for ALICE- satisfies the functional + performance requirements- runs on single machine and large-scale setups- is available as kits (RPM package) - user’s guide: ALICE-INT-2005-015- regular training courses - is tested at reference system, data challenges, test beams - is ready for the test and commissioning phase- homepage: http://cern.ch/alice-daq

Future work (DATE V6):- port to 64-bit platforms- port to SLC4 using a 2.6 kernel- electronic logbook- Transient Data Storage Manager (TDSM)- new upcoming requests

Thank you!