Trigger and Data Acquisition at the Large Hadron …• Implementation of First Level Trigger –A TLAS and CMS –LHCb –ALICE • The First Level Trigger Technology LHC 17-June-2006

Trigger and Data Acquisition at the Large Hadron Collider

17-June-2006 A. Cardini / INFN Cagliari 2

Acknowledgments• This overview talk would not exist without the help of many

colleagues and all the material available online

• I wish to thank the colleagues from ATLAS, CMS, LHCb and ALICE, in particular R. Ferrari, P. Sphicas, C. Schwick, E.Pasqualucci, A. Nisati, F. Pastore, S. Marcellini, S. Cadeddu , M. Zanetti, A. Di Mattia and many others for their excellent reports and presentations


Day 1 - Summary• LHC

– Accelerator parameters– The experiments

• Triggering– General Concepts– LHC Requirements– Trigger architecture

• Implementation of First Level Trigger– ATLAS and CMS– LHCb– ALICE

• The First Level Trigger Technology

LHC


LHC Accelerator Complex• Proton-proton

– CM energy = 14 TeV– L = 1034 cm-2s-1 ATLAS, CMS– L = 1032 cm-2s-1 LHCb

• Heavy Ions (ex.: Lead-lead)– CM Energy = 1312 TeV (!)– L = 1029 cm-2s-1 for ALICE


Why LHC?• We need a high luminosity and a high center-of-mass

energy proton-proton collider to

– Search of Higgs boson(s)– Search for SUSY particles– Standard Model Physics– CP violation studies in the B sector– New Physics beyond SM– Ultra High Energy Heavy Ions Collisions

… hope that we will see many other things we do not expect…


LHC vs. LEP vs. Tevatron• One way to increase the luminosity is to increase

the number of bunches circulating in the ring: LHC will have ~3600 bunches– 27 km (LEP tunnel) ring– 27000 m / 3600 = 7.5 m between bunches– 7.5 m / 3x108 m/s = 25 ns


Proton-proton cross section• Interactions/second

– L = 1034 cm-2s-1 = 1010 Hz/b– σinel(pp) ~ 70 mb

7x108 interactions/s

• Events/crossing– @ 40 MHz (∆t = 25 ns)

17.5 interactions/crossing

• Not all bunches are full– Only about 4/5 (2835/3564)

22 interactions/”real” crossings

• What are we looking for?– 1 interesting physics event (Higgs for example) superimposed to

~20 minimum bias events !!!

σinel(pp) ~ 70 mb


Here it is!• 20 minimum bias events overlapping + H ZZ 4 muons (the cleanest

golden signature)

• This “mess” (not the Higgs! - its production cross section in very small so we expect 0.01-0.1 Hz Higgs production) repeats every 25 ns!


The Physics at LHC• Cross sections for various physics

processes vary of many orders of magnitude

• At the standard LHC luminosity we have:

– Inelastic (min. bias): 109 Hz– W lν: 102 Hz– ttbar: 10 Hz– Light Higgs (100 GeV): 0.1 Hz– Heavy Higgs (600 GeV): 0.01 Hz– bbbar: huge (106 Hz)

• An efficient selection mechanism capable of selecting 1 event over 1010-1011 is needed: this is the

TRIGGER


How to build a LHC Experiment?• Depends obviously on the physics…

• New (very) heavy particles (Higgs, for example) are produced centrally with large transverse momentum symmetric detector– ATLAS, CMS

• Lighter particles (B, for example), are produced mainly at small angles. One can take advantage of the boost forward detector for B physics– LHCb

• When running in heavy ions mode ALICE will search for Quark-Gluon Plasma, needing both central and forward coverage


How to build a LHC Experiment? (2)• Each experiment is made of

– “Inner” trackers– Calorimeters– Muon detectors

• This will allow to– Resolve the tracks – Measure the energy depositions– Identify the particles– Measure the decay vertices

• Experiment size and granularity is determined by– Required accuracy– Particle multiplicity @ ATLAS/CMS O(1000) particles/B.C.

• This determines– Number of detector “elements”– Number of electronic channels– Data size and throughput


ATLAS

44 m length22 m diameter


CMS


LHCb


ALICE

TPCTPC

PHOSPHOS

Muon armMuon arm

TOFTOF

TRDTRDHMPIDHMPID

PMDPMD

ITSITS

ACCORDEACCORDE

Triggering


General Trigger Requirements• The role of the trigger is to make the online selection of particle

collisions potentially containing interesting physics

• Need high efficiency for selecting processes of interest for physics analysis, for which:– Efficiency should be precisely known– Selection should not have biases that affect physics results

• Need large reduction of rate from unwanted high-rate processes (capabilities of DAQ and also offline computers):– Instrumental background– High-rate physics processes that are not relevant for analysis (min.

bias)

• System must be affordable– Limits complexity of algorithms that can be used

• Not easy to achieve all the above simultaneously!


LHC Trigger Challenges• Nchannels ~ O(107-108) and 20 interactions/25 ns

– Need huge number of connections– Need information super-highway

• Information coming from different detector parts should correspond to the same interactions– Need to synchronize detectors to (better than) 25 ns

• Note however that in some cases detector signals and/or time-of-flight exceeds 25 ns– Some detector will integrate information coming from more than 1 bunch

crossing

• Can store data at 100 MB/s 100 Hz for ATLAS/CMS (1 MB/ev.), 1 kHz for LHCb (100 kB/ev.)– Need to reject most interactions

• What is discarded is lost forever!– Need to careful monitor the selection


Triggering Howto• Look at (almost) all bunch crossings, select most interesting

one, collect all detector information and store it for off-line analysis (for a reasonable amount of money)

• Since the detector data are not all promptly available and the selection function is rather complex, T() is evaluated by SUCCESSIVE APPROXIMATIONS called TRIGGER LEVELS(which should have possibly zero dead time)


The multi-level triggerA multi-level trigger system provides:

– Rapid rejection of high-rate background without incurring in (much) dead-time: the fast first-level trigger (custom electronics)

• Needs high efficiency, but rejection power can be comparatively modest

• Short latency is essential since information from all (up to O(108)) detector channels needs to be buffered (often on detector) pending trigger decision

– High overall rejection power to reduce output to mass storage to affordable rate: one or more High Trigger Levels:

• Progressive reduction in rate after each stage of selection allows use of more and more complex algorithms at affordable cost

• Final stages of selection, running on computer farms, can use comparatively very complex (and hence slow) algorithms to achieve the required

Example: ATLASoverall rejection power


First Level Selection

• First level (level 1) reduces event rate from 40 MHz to O(100) kHz

• This step exist in all experiments

• Not enough, still to go down by factor 100-1000 in one or more extra step…


Successive Selection: 3 steps…

Additional processing in intermediate step reduces bandwidth requirements


… or 2 steps

!

This solutions reduces the number on building blocks and could rely on commercial components for what concerns calculations and network


Triggering @ LEP vs. LHC

100 ms

8 ms

30 µs

e+e– crossing rate 45 kHz (4 bunches)

22µs45 kHz

Level 2

10 Hz Readout

6 µs

Level 1

Level3

100 Hz

8 Hz

≈ µs

p p crossing rate 40 MHz (L=1033- 1034cm-2 s-1)

25 ns

Level 1

40 MHz

≈ ms

Level 2

100 kHz

Level n

1 kHz

• LEP– tL1 < inter bunch time– no event overlapping

• LHC– tL1 » inter bunch time– 22 overlapping events/BC


Trigger/DAQ at LHC

!

!

!

!

100 (~103)


Trigger/DAQ: past, present and future

Level-1 (L1) Trigger


Algorithms for Level-1 Trigger• The Physics

– pp collision produce hadrons with pt ~ 1 GeV– Interesting Physics has particles with large pt:

• W lν: M(W) ~ 80 GeV/c2, pt(l) ~ 30÷40 GeV• H(120 GeV/c2) 2 photons, with pt(photon)~ 50÷60 GeV

• Trigger Requirements– Impose high thresholds on specific interaction products: “easy”

for muons, electrons and “jets”, then need complex algorithms– Typically:

• Single muon with pt > 20 GeV 10 kHz• dimuons with pt > 6 GeV 1 kHz• Single electron with pt > 30 GeV 10÷20 kHz• dielectrons with pt > 20 GeV 5 kHz• Single jet with pt > 300 GeV 200÷400 Hz

ATLAS/CMS requirements


Pt cut in minimum bias events

All tracks pt > 2 GeV

Simulated H→ ZZ → 4 µ event + 17 minimum-bias events


Which Detectors at Level-1?


Which Detectors at Level-1? (2)

☺ In Muon detector / calorimeter low occupancy and patter recognition is easy– Simple reconstruction algorithms fast– Small amount of data– Can take “regional” decisions

In inner detectors– Complicated events!– Complex analysis algorithms slow– Huge amount of data– Need to link to other detector for

additional information


Still is not easy…• It is not possible to generate a trigger in 25 ns

• Need a massive concurrent, pipelined processing to implement a dead-timeless L1 trigger

Primitive Gen

detector ≈ 50 ns

FE≈ 100 ns

≈ 100 ns

≈ 500 ns

≈ 500 ns

Local trigger

≈ 600 ns

Global trigger

≈ 300 ns≈ 600 ns

pipelinedelay

O(100)deep

µs

derandomizer

Example: CMS

~3


Level-1 Processing


Information Flow


Information Flow (2)

L1 trigger architecturein the LHC Experiments


Region of Interest (RoI)• The L1 selection can based on local

signatures called Region of Interest (RoI)– Based on coarse granularity, no inner

tracker info– Local analysis allows an important further

rejection

• The geographical location of interesting signatures are identified by L1– This allow access only to local data for

each relevant detector


Region of Interest (2)• Region-of-Interest

– RoI data ~1% of L1 output– Complex mechanism for

data access– Many control messages– Smaller readout network

thanks to an intermediate trigger level which only processes local (in η,φspace) information

MORE COMPLEX SYSTEM

• Not RoI

– Very high throughput– Very large readout network– Simpler system– Flexible

MORE DEMANDING ON TECHNOLOGY


The ATLAS ArchitectureRoI based L1 trigger


ATLAS• Muon

– O(106) RPC (Barrel)/TGC (Endcap) trigger channels– Barrel and Endcap LOCAL Trigger Processor to estimate pt– Muon Central Trigger Processor

• Calorimeter – LAr (ECAL) and Tile/LAr (HCAL)– Analog preprocessor (analog pipeline!) to estimate Et– LOCAL Jet/Energy Sum and Cluster Processor stage

• The local triggers are sent to the Central Trigger Processor (CTP), which makes the FINAL decision

• The L1 trigger decision is sent to the Front End via the TTC (Timing-Trigger-Control) system

• For every accepted event the L1 trigger sends readout information to the Region-of-Interest (RoI) Builder which assembles the list of RoIs to be used by L2 Note: all digital design except input stage of calorimeter trigger pre-processor


The ATLAS Muon Trigger• The L1 Muon Trigger

requires coincidence of RPC/TGC hits within a road, which is related to the pt cut applied

• A high and a low ptalgorithms are applied

• Multiple cuts can be used at the same time thanks to programmable coincidences

Fast and high redundancy system

- Wide pt threshold range- Safe bunch crossing identification (fast detectors)- Strong rejection of fake muons (noise, physics background)

40 kHz expected at L=1034 /cm2/s


The CMS ArchitectureSimilar to ATLAS, but no RoI

The Global Calorimeter trigger selects the best 4 e,γ, τ and jetsand calculate Et and Et

miss

The Global Muon Trigger receives 4 muon candidates of maximum ptand select the best quality ones

The Global Trigger applies the thresholds and performs the trigger algorithms.

Up to 128 algorithms can run in parallel: arbitrary combinations of trigger objects passing thresholds and topological correlations…


The CMS Calorimeter Trigger

• Divide Calorimeter in towers• Match towers between ECAL and HCAL• Isolation and deposit shape criteria to

identify electrons, photons, jets


L1 Trigger Rates

1.63.3--Double µ

8.71023.2/3.86 (l) / 20 (h)Single µ

--1.0336Total ET

0.012751.051Etmiss

1.21406.0100Jet

3.28516.320τ

2.7121.412Double e/γ

5.7221117Single e/γ

Rate (kHz)Thr. (GeV)Rate (kHz)Thr. (GeV)Selection

CMSATLAS

L1 trigger rates at L = 1033 when applying 90% efficiency thresholds


CMS vs. ATLAS


The LHCb Architecture

Calorimeters + Muon system

10 MHz

1 MHz

L0: hight pT tracks+ not too busy eventsFully synchr. (40 MHz), 4 µs latencyOn custom boards

• 40 MHz crossing rate, but only 30 MHz real crossings

• Luminosity: 2·1032 cm-2 s-1

(50 times lower than ATLAS and CMS)

• Minimum bias rate: 10 MHz

• bb rate is ~ 100 kHz (15 kHz in detector acceptance)

• cc rate is ~ 600 kHz

• First level trigger (here called L0) selects high pt particles (muon, e,gamma, …)and events with only one interaction by means of the pile-up veto


LHCb Muon Trigger• The LHCb muon system:

– 5 stations– Variable segmentation– Projective geometry

• Trigger strategy:– Straight line search in M2-M5 in

every quadrant– Look for compatible hits in M1

• Momentum measurement (∆p/p~20% for b-decays)

• Sent to L0 decision unit: 2 highest pTcandidates per quadrant

• Typical Performance:~88% efficiency on B->J/ψ(µµ)XAlgorithm latency ~1 µs

µ

>90% π/K decay

Nominal threshold


LHCb Pile-up veto• LHCb needs to identify secondary (decay)

vertices

– This is performed @ a higher trigger level

– Works well if there is only and only one interactions per bunch crossing

– A veto against double interactions is implemented with 2 silicon detectors planes

– Hits are fitted by means of 4 large FPGAs and results are sent to L0 Decision Unit

– Typical Performance for identifying double interactions

• 60% efficiency• 95% purity

– Latency ~ 1 µs


The ALICE Architecture• Heavy ions runs

– L=1027cm-2s-1

– Interaction rate < 10 kHz– Very high multiplicity & Huge event size (~ 50 MB)– Modest requirements on lower level triggers

• pp (or pA) runs – Interaction rate up to 200 kHz, limited by TPC pileup– Small event size (~ 2 MB)– Strong requirements on lower level triggers

• To accommodate all the different running conditions the first level trigger is split in 3 distinct levels (L1, L1 and L2)


ALICE First Level Trigger(s)• Some of the Alice FEE is not pipelined

but await a trigger before processing

• Some detector need very early strobe(e.g. TOF), so a first early decision is taken in 1.2 µs (L0)

• For detectors which require longer timeL1 is used, which arrived 6.5 µs after interaction

• L2 comes after 88 µs, at the end of the drift time in TPC. Purpose of L2 is to wait for the end of the “past-future pile-up protection”, in order to make sure that there is only one event in TPC


ALICE: Optimizing Trigger Efficiency• Requirements

– Some subdetectors need a long time to be read out after a L2 (silicon drift detector: 260 µs)

– However some interesting physics events need only a subset of detectors to be readout

• Concept of “Trigger Clusters”– Group of sub-detectors– Even if some sub-detectors are busy, triggers for not busy

clusters can be accepted, increasing acquired statistics

• “Rare Events”– When readout buffer “almost full” only accepts the so called

“rare” event

L1 Trigger Implementation


L1 implementation issues• At L1 every operation must be extremely fast

• L1 logic is usually built using:• ASIC (Application Specific

Integrated Circuits)– Can be produced radiation

tolerant (to be installed on detectors)

– Can contain both analog and digital part

– Full-custom or Standard-Libraries (or mixed) design

– Long development cycle– Extensive simulation

necessary– Production cycle

expensive, but cost/ASIC can be extremely low

• FPGA (Field Programmable Gate Arrays), – Extremely versatile

nowadays– Might contain memory,

processors, high speed serial links…

– Complex design possible: processors, PCI interfaces, WEB-servers

– “Easy” to implement the requested design

– Reprogrammable (even in-situ!)

– (very) Expensive units


LHCb Muon Off Detector Electronics

• SYNC ASIC

– CMOS IBM 0.25 um– 8x4 bit TDCs– 12 bit counter for Bxid

generation– L0 buffer and derandomizer– Interface to L0 trigger logic– Programmable via i2c– Radiation resistant– Triple-voted auto-correcting

registers for radiation immunity

– … many other things


L1 implementation issues (2)

• Communication Technologies

– Very high-speed serial links (copper or fiber)• LVDS, G-link, Vitesse, …

– Backplanes• Very large number of connection, data multiplexing


CMS Regional Calorimeter Trigger• Receiver card (there are

O(20) cards/crate and O(20) crates in the system): ~ 400

• Receives 64 trigger primitives, 32 from ECAL and 32 from HCAL

• Forms two 4x4 towers for Jet Trigger and 16 Et towers for electron isolation card

• Overall system input bandwidth: ~4000 Gb/s


Today’s Conclusions• LHC: a very challenging environment

• Very demanding requests on trigger system, in particular on first level trigger

• Pipelines (analog or digital) everywhere for a (almost) dead timeless L1 trigger

• Different philosophies of the experiment: ATLAS vs. CMS want to study the same physics but adopt different approaches: RoI vs. not RoI

• Technology is progressing very rapidly and its performance appears adequate (but still systems are very complicated…)

Documents

Trigger and Data Acquisition at the Large Hadron …• Implementation of First Level Trigger –A TLAS and CMS –LHCb –ALICE • The First Level Trigger Technology LHC 17-June-2006