Gilles MAHOUT

12th Workshop on Electronics for LHC experiments - Valencia- 25th - 29th

Sept. 2006

Production Test Rig for the ATLAS Level-1 Calorimeter Trigger Digital Processors

J.R.A Booth, D.G. Charlton, C.J. Curtis, P.J.W. Faulkner, S. Hillier, G. Mahout,

R.J. Staley, J.P. Thomas, D. Typaldos, P.M. Watkins, A. Watson, E.-E. Woehrling

School of Physics and Astronomy, University of  Birmingham, Birmingham, UK

R. Achenbach, V. Andrei, F. Föhlisch, C. Geweniger, P. Hanke, E-E. Kluge,

K. Mahboubi, K. Meier, F. Rühr, K. Schmitt, H.-C. Schultz-Coulon, P. Weber

Kirchhoff-Institut für Physik, University of Heidelberg, Heidelberg, Germany

B. Bauss, S. Rieke, R. Stamen, U. Schäfer, S. Tapprogge, T. TrefzgerInstitut fur Physik, Universität Mainz, Mainz, Germany

E. Eisenhandler, M. LandonPhysics Department, Queen Mary, University of London,

London, UK

B.M. Barnett, I.P. Brawn, A.O. Davis,J. Edwards, C. N. P. Gee, A.R. Gillman, V.J.O. Perera, W. Qian, D.P.C. Sankey

Rutherford Appleton Laboratory, Chilton, Oxon. UK

C. Bohm, S. Hellman, A. Hidvégi S. SilversteinFysikum, Stockholm University,SE-106 91 Stockholm, Sweden

Gilles MAHOUT 2

ATLAS Level-1 Calorimeter Trigger System

Tracking Calo Muon Calorimeter

TriggerMuon Trigger

Calorimeters Muon

e/

tau

jet Et

E

Central Trigger Processor

Front End

Buffer

Region ofInterest (RoIs)

L1

L2

FifoReadoutDriver ROD ROD ROD

ReadoutBuffer

Event Builder

R

O

B

R

O

B

R

O

B

Storage

75 kHz

1000 Hz

200 Hz

Gilles MAHOUT 3

Identifying Clusters: e/, tau and jets•The trigger works on a reduced granularity with trigger towers covering 0.1 x 0.1 in x

•The e.m. and tau are based on windows of 4 x 4 trigger towers, in both e.m. and hadronic calorimeters

•The jet trigger is based on windows of 4 x 4 “jet elements”, each 0.2 x 0.2 in x , with e.m and hadronic calorimeters summed

•This windows slide in eta and phi to fully cover the calorimeter

• RoIs are defined by their (,) coordinates – a window is a local candidate trigger object if it is a local maximum

• RoIs are tested against sets of threshold values, each made up of cluster and, (for e.m. and tau) isolation energies.

Module 3Module 2Module 1Module 0•Because the algorithms involve overlapping data, Trigger Tower data are shared:

• between algorithm chips onboard

• between modules, across a custom-built backplane

Gilles MAHOUT 4

System Data flow

589 GBytes/

s

448 GBytes/s

141 GBytes/s

0.7 GBytes/

s

CentralTriggerProcessor

~7200 Calorimeter

Trigger Towers

75 kHz

L1A

RoI DataCheck

Level 2 Readout

Pre-Processor

ClusterProcessor

Jet/EnergyProcessor

Gilles MAHOUT 5

ATLAS Level-1 Calorimeter Trigger System

DAQRODs

Input/output dataTo DAQ

e/, /hadClusters

(CP)

0.2 x 0.2Jet / ET

(JEP)

0.1 x 0.1

Pre-Processor(PPr)Analogue

tower sums0.1 x 0.1(~7200)(>300 Gbyte/s)

RoIRODs To L2

Featuretypes/positions

Fibre F/O

TTCCTP SlowControl CANbus

DCS

To CTP

To CTP

8 PPr crates

4 CP crates

2 JEP crates

2 ROD crates

Gilles MAHOUT 6

Level-1 Calorimeter Trigger System: Production Test Rig

All digital boards have passed their Production Readiness Review pending a full crate test made on pre-production modules: Cluster Processor Modules and Jet/Energy Modules.Need to gear up to the equivalent of ¼ of full trigger in one crate

Additional production boards were manufactured Need to emulate calorimeter LVDS: special boards have

been built to source data Ordered LVDS cables with final ATLAS length

1888 4-link cables needed for final system, around 300 per crate

Production custom-built backplane available for the testAdditional tests

Generate 18 active links to fully populate the ROD input Test of the DCS with a full crate of boards:

Monitoring on-board currents, voltages and temperatures using PVSS

Simulation of the hardware easily expandable to accommodate several crates and modules

Gilles MAHOUT 7

Calorimeter signals: LVDS Source Modules

Custom built LVDS Source Modules (LSM) have been used as source of 400 Mbit/s serial LVDS1 LSM per boardCustom backplane receives the emulated calorimeter LVDS from the rearFinal number

308 cables14 LSMs28 TTCrx chips on mezzanine board

Gilles MAHOUT 8

Cluster Processor Module: Full Crate TestContents of the crate

14 CPMs1 CPU mounted on a special adaptor card to fit the custom backplane1 board designed to broadcast the TTC clock to each individual module via the backplane; also provide interface to external CANbus2 Common Merger Modules

Gilles MAHOUT 9

Cluster Processor Module: Crate Test results

Patterns used Ramp to test LVDS input Test vector with programmable

occupancy rate Long term measurement

No parity error observed in 5 hours on real time data directly seen from the source, or through backplane

Timing window investigation For the fastest links, data

across backplane show parity error free window of 2 ns

Crate Power Consumption ~ 200 A at 10% occupancy

(<300 A, in PSU specification) P ~ 1 kWFPGA behaviour

Big FPGA temperature less than 45oC

Internal current within specification

Crate current Consumption

0

50

100

150

200

250

0% 10% 20% 30% 40% 50%

Atlas Occupancy

Current (A)

(.1 ns)

Chip #

Gilles MAHOUT 10

Jet/Energy Module: Crate Test Results

Contents of the crate as before, but with 7 JEMs instead of CPMs

Empty slot between board types to avoid data mismatched across backplane

Results: No links lost on LVDS No parity errors observed after 5 hours across the backplane

10-bits Ramp

Gilles MAHOUT 11

Common Merger ModuleTwo CMMs in crateNo parity errors in 5 hours on backplane dataAll CMM inputs show wide parity error free windows with CPMs as inputs

10 ns

(.1ns)

No boards

Parity E

rrors

No

Parity E

rrors

Gilles MAHOUT 12

ROD Test

ROD can receive up to 18 modules via G-links 16 boards connectedResults: No parity errors observed in 1 hour

5 CPMs feeding ROD had no data errors after 200k events

Gilles MAHOUT 13

Detector Control System on crate

CANbus using CANOpen protocol to read out all temperatures and currents of up to 14 modulesPVSS used to monitor currents, voltages and temperatures

Alarm implemented and successfully tested Overnight runs possible during production phase

Gilles MAHOUT 14

ConclusionsThe full-crate tests have been successful: Debug early problems in comfort of the laboratory rather than in the ATLAS pit

Check stability of the digital board designs

Check crate power consumption Scale up DCS to several boards and cratesAll boards are into production nowFirst production modules are now being used in ATLAS pit