LHCb front-end electronicsand its interface to the DAQ

DAQ review Sep. 2001 J.Christiansen/CERN 2

Quick LHCb Front-end overview

• ~ 1 million detector channels.• 10 different sub-detector front-end implementations.• 40 MHz bunch crossing rate .

– ~1/3 has interaction

• Two trigger levels in front-end.– L0: 4.0 us constant latency (pipeline buffer)

Max 1.11 MHz accept rate– L1: Variable latency, max 1900 event (event FIFO)

Trigger decisions distributed in chronological order

40 (100) KHz accept rate.

• Front-end architecture:– Simple front-end architecture where possible.– Central prevention of of buffer overflows.– Architecture extensively simulated in VHDL to insure correct

function under all conditions.

DAQ review Sep. 2001 J.Christiansen/CERN 3

General architecture

L0 buffer

L0 derandomizer

L0 buffer

L0 derandomizer

L0 Buffer

L0 Derandomizer

L0 Trigger+

Readout Supervisor

L1 Trigger +

Readout Supervisor

L0 buffer

L0 derandomizer

L0 buffer

L0 derandomizer

Zero Suppression

& Multiplexing

L1 Buffer

L1 Derandomizer

Output Buffer

L1 Throttle

L0 Throttle

Front-End system Trigger & TFC system

L0

L1

L1 Trigger

L0 TriggerTTC system

TTC system

DAQ

DAQ review Sep. 2001 J.Christiansen/CERN 4

L0 front-end

L0 buffer

L0 derandomizer

L0 buffer

L0 derandomizer

L0 Buffer

L0 Derandomizer

L0 Trigger

L0 Derandomizer Emulator

L0 Trigger

Raw Data @ 40 MHz

L0 Throttle

Readout supervisor

36 words = 32 ch + 4 tags

4 µs

16 events

15 events

Max 1.111 MHz

L0 Data @ 1.111 MHz

L1 Trigger systemL1 Buffer Monitor

Raw Data @ 40 MHz

• Constant latency: 4.0 us• Maximum 1.11MHz trigger rate• 16 events deep L0 derandomizer.• Events from L0 derandomizer defined to be max 36 words @ 40MHz• Derandomizer overflows prevented by central emulator in Readout

Supervisor, based on a set of strictly defined front-end parameters.

DAQ review Sep. 2001 J.Christiansen/CERN 5

L1 front-end

L0 buffer

L0 derandomizer

L0 buffer

L0 derandomizer

Zero Suppression

& Multiplexing

L1 Buffer

L1 Derandomizer

Output Buffer

Reorganizer

L1 Trigger + commands

L0 Data @ 1.111 MHz

Readout supervisor

1927 events

15 events

CPU CPU

L0 Throttle

Nearly full

Nearly full

Board

System

L1 Trigger34 words/event @ 40MHz per word

L0 Data @ 1.111 MHz

Max 40 (100) kHz L1 Trigger Derandomizer

L1 BufferMonitor

Spacer

Command

2 us

DAQL1 throttle

DAQ review Sep. 2001 J.Christiansen/CERN 6

L1 front-end

• Variable latency.• L1 trigger decisions distributed to front-end in chronological

order via TTC broadcast message.(for both accepts and rejects)

• L1 buffers in front-ends implemented as simple FIFOs.• L1 buffer occupancy monitored centrally by Readout Supervisor

that throttles L0 triggers in case of risk of overflow.• L1 trigger decisions sent to the front-end at a rate that can be

handled by all front-ends (no local buffering of trigger decisions needed)

• 15 events deep L1 derandomizer:– 3 events to handle L1 throttle delay (2us)– 12 events for derandomization

• L1 derandomizer and following data buffers protected against overflow by hardwired L1 throttle signal.

• Zero-suppression (sparcification) and event data formatting.

DAQ review Sep. 2001 J.Christiansen/CERN 7

Centralized front-end control:Readout supervisor

• Receives L0 and L1 trigger decisions from trigger systems.

• Only distributes trigger accepts to front-end that will not generate buffer overflows.

• L0 derandomizer overflows prevented by L0 derandomizer emulator.

• L1 buffer overflows prevented by L1 buffer emulator.

• L1 trigger decisions spaced to match processing speed of front-end

• Buffer overflows in L1 derandomizer and following buffers prevented by hardwired L1 throttle network.

• Resets , calibration signals, testing and debugging functions.

• High level of programmability to allow system level optimizations.

L1 bufferemulator

L0 trigger

L1 triggerderandomizer

L1 triggerdecision

L1 decisionspacer

TTC encoder

L0 throttle

L1 throttle

L0 derand.emulator

TTC distribution

L0 triggerdecision

L1 trigger

DAQ review Sep. 2001 J.Christiansen/CERN 8

Front-end control and monitoring

• Clock synchronous control of front-end handled by Readout Supervisor via TTC system.

• Local monitoring in front-ends of buffer overflows and event consistency based on event tags. ( Bunch ID, L0 event ID, L1 event ID).

• Error conditions sets error flags in event fragments and sets status bits to Experiment Control System (ECS).

• Front-end parameters down loaded via ECS system ( With enforced read-back capability ).

• Standardized ECS interfaces for front-end:– Credit card PC– SPECS ( simple serial protocol )– CAN ELMB ( from ATLAS )

DAQ review Sep. 2001 J.Christiansen/CERN 9

Detailed front-end architecture

L0 e le c tro n ic s

L1 e le c tro n ic s

L0 triggerdata extract

L1 triggerdata extract

TTC RX

ADC ADC ADC

B-IDE-ID

MUX

B-res

E-res

L0-yes

L1 buffercontrol

M UX

TTC RXADC ADC ADCBclk

L0-ye s

L1-ye s/no

MUXInterface

EC S

EC S

Fro n t-e nd -M UX

L0 triggerprocessor

L1 triggerprocessor

L0 decisionunit

L1 decisionunit

TTCDriver

L1-yes/no

Level 0monitor &Throttle

Level 1monitor &Throttle

L0-yes

Readout supervisor

EC S lo c a lc o ntro lle r

DAQ syste m Optic

al f

an-o

ut

EC S syste m

L0 buffer(pipeline)

L1 buffer(FIFO)

Zero-suppression

L1 derandomizer

Output buffer

L0 derandom izer

Fro nt-e nd syste m Trig g e r syste m

Ana lo g fro nt-e ndEC S

L0 trigger links

L1 trig g e r links

ECAL, HCAL, PreshowerMuon

Vertex, (tracker)

L1 th ro ttle

DAQ review Sep. 2001 J.Christiansen/CERN 10

Interface to DAQ

• Interface between front-end and DAQ system is handled by Readout Units.• Standardized event formatting on optical links from front-ends.• Data bandwidth per front-end branch limited to ~25Mbytes/s under nominal

conditions to have headroom for unexpectedly high channel occupancies and allow upgrade from 40 to 100 KHz trigger rate.

L1FE

FEM

L1FE

FEM

L1FE

FEM

L1FE

FEM

RU RU

Sub-detector N

Event building network

N+1

L2/L3CPU farm

L1 Front-Endelectronics

Front-EndMultiplexing(optional)

Readout Units

Transport header

Transport trailer

Event building header

Event building trailer

Event data

Event data header

Event data trailer

Event data

Event data header

Event data trailer

Fragment 0

Fragment N

Event formatting

DAQ review Sep. 2001 J.Christiansen/CERN 11

Data Link from front-end to DAQ

• Standardized unidirectional optical link handling distances of up to 100m.

– No Xon/Xoff backpressure foreseen.• In some sub-detectors the link transmitters are located in the cavern

with limited levels of radiation (few Krad).• Required bandwidth: 10 – 50Mbytes/s.• Use of S-link enables:

– Flexibility in choice of link technology. – Use of standardized link interface cards.

• Standardization on Gigabit Ethernet.– Defacto standard in computer industry.– Event building in DAQ will be based on Gigabit Ethernet.– Many relatively cheap components available.– Gigabit Ethernet S-link transmitter under development in Argonne.– Question of framing overhead:

• Event data is not heavily concentrated in LHCb.• Reduced Ethernet framing can be used on data to Readout Units.

DAQ review Sep. 2001 J.Christiansen/CERN 12

LHCb Front-end in numbers

Detector Channels FE links FEMs Link rate

@40KHz

RUs Event size

Vertex205k 100 25 8 MB/s 7 6k

RICH450k 55 0 10 MB/s 15 15k

Inner tracker220k 108 27 14 MB/s 14 11k

Outer tracker

120k 120 0 13 MB/s 30 39k

Muon26k 10 0 8 MB/s 3 2k

Calorimeter20k 26 0 10-28 MB/s 10 12k

DIV5 1k

Total1041k 419 52 84 86k