Setup, Tests and Results for the ATLAS TileCal Read Out Driver

Production

J. Alberto Valero BiotIFIC - University of Valencia

12th LECC - September.2006 (VALENCIA)

2

Outline

IntroductionSetup & tools for ROD production

Production testsResults and conclusions

3

The TileCal ROD system

ATLAS

ROD crate ROD crate ROD crateROD crate

TileCal 3 Barrels-4 partitions: 64 modules/partition. Data redundancy system: 2 fiber

links/module. 128 fiber links per barrel.

o Extended barrel: 32 channels / fiber link.o Long barrel: 48 channels / fiber link.

ROD system 4 barrels 4 ROD crates, 8 ROD

systems/crate. ROD system: OMB ROD TM. TOTAL: 32 ROD systems. We have tested RODs + TMs ( OMB designing)

4

Outline

IntroductionSetup & tools for ROD productionProduction testsResults and conclusions

5

Setup for ROD production

2 OUTPUTS

OMB

DUAL TIMER

Trigger

PC

HD

4 RODs

FILAR

FILAR

Optical buffer 1:16

Optical buffer 1:16

DUAL TIMER• Trigger frequency• Trigger width• Handling of vetos

OMB 6U prototype

• Trigger detection• Data injection

OPTICAL BUFFER

• Data buffer 1:16 ROD CRATE• 4 RODs to be tested• 4 Transition Modules• 1 TBM

COMPUTERs• Full system configuration• 2 FILAR cards• Data storage• Data check

6

Optical Multiplexer Board - 6U prototype

VME_FPGA•ALTERA ACEX FPGA EP1K100•VME R/W registers

•Status/control•CRC error registers

•Provides communication between VME bus and CRC_FPGAs.

2 CRC_FPGAs• Altera Cyclone® FPGA• Real-time CRC check• Internal memory lets us download and inject real events towards the ROD.• Event counter generator• CRC injection

OPTICAL I/O• 4 Optical transceivers I/O (Infineon®V23818-K305-L17)• 2 Inputs / 1 Output per each CRC_FPGA

SERIALIZERS/ DESERIALIZERS

• 4 receivers (HDMP 1034)•2 transmitters (HDMP1032)•The same ones as the I.C.s

VME INTERFACE

FP connectors

• LEMO input conectors:

•Trigger and busy

UNITS•1 Valencia LAB•1 CERN•2 Inoperative

7

OMB 6U: FunctionalityCRC MODE (Main function in final OMB

version)

4 Inputs / 2 Outputs Copy mode - No data modifications Real time decision Registers with CRC error counters LEDs for notice CRC errors

RXA_CNTLRXA_ERROR

RXA_DSLIPRXA_CLK0

RXA_READYRXA_DATARXA_FLAG

RXA_DIV0RXA_DIV1

RXA_RESETRXA_FLGENB

RXA_ESMPXENBRXA_WSYNCDSB

RXA_D [15..0] 16

RXA_CNTLRXA_ERROR

RXA_DSLIPRXA_CLK0

RXA_READYRXA_DATARXA_FLAG

RXA_DIV0RXA_DIV1

RXA_RESETRXA_FLGENB

RXA_ESMPXENBRXA_WSYNCDSB

RXA_D [15..0] 16

RECEPTIONFIBER A

RECEPTIONFIBER B

CLOCKDIVIDER

CLK_40MHz

CRC CHECK 1

CRC CHECK 219

19

10VME[9..0]

VME[15..10]6

EVENT_GEN_COUNTER

EVT_MEM

VME_TRIGGERBUSY

TRIGGER

EXT_TRIGGER

INTERNAL CLOCK SIGNALS

FRONT PANEL CONNECTOR

OUTPUTCONTROLLER

CRC_TX

16

16

16 19

19

19

MUX

MUX

LOGICFOR

DECISION

CRC check CRC ERROR COUNTERS

TX_D[15..0]16

TX_CNTLTX_DATATX_FLAG

LINK_UP_TXDATA_ERROR_TXXOFF_TX

LINK_UP_RX2DATA_ERROR_RX2XOFF_RX2

LINK_UP_RX1DATA_ERROR_RX1XOFF_RX1

BYTE_BLASTERCONNECTOR

FRONT PANELLEDS

TX_CLKTX_DIV0TX_DIV1TX_FLGENBTX_ESMPXENBTX_LOCKED

FPGA_CON[15..0]16

8

OMB 6U: Functionality (II)

INJECTION MODE 2 Optical Outputs Internal memory for inject events from a

file Internal counter for automatic events

generation CRC word injection on each event sent External/Internal trigger selection Frequency selection for internal trigger

RXA_CNTLRXA_ERROR

RXA_DSLIPRXA_CLK0

RXA_READYRXA_DATARXA_FLAG

RXA_DIV0RXA_DIV1

RXA_RESETRXA_FLGENB

RXA_ESMPXENBRXA_WSYNCDSB

RXA_D [15..0] 16

RXA_CNTLRXA_ERROR

RXA_DSLIPRXA_CLK0

RXA_READYRXA_DATARXA_FLAG

RXA_DIV0RXA_DIV1

RXA_RESETRXA_FLGENB

RXA_ESMPXENBRXA_WSYNCDSB

RXA_D [15..0] 16

RECEPTIONFIBER A

RECEPTIONFIBER B

CLOCKDIVIDER

CLK_40MHz

CRC CHECK 1

CRC CHECK 219

19

10VME[9..0]

VME[15..10]6

EVENT_GEN_COUNTER

EVT_MEM

VME_TRIGGERBUSY

TRIGGER

EXT_TRIGGER

INTERNAL CLOCK SIGNALS

FRONT PANEL CONNECTOR

OUTPUTCONTROLLER

CRC_TX

16

16

16 19

19

19

MUX

MUX

LOGICFOR

DECISION

CRC check CRC ERROR COUNTERS

TX_D[15..0]16

TX_CNTLTX_DATATX_FLAG

LINK_UP_TXDATA_ERROR_TXXOFF_TX

LINK_UP_RX2DATA_ERROR_RX2XOFF_RX2

LINK_UP_RX1DATA_ERROR_RX1XOFF_RX1

BYTE_BLASTERCONNECTOR

FRONT PANELLEDS

TX_CLKTX_DIV0TX_DIV1TX_FLGENBTX_ESMPXENBTX_LOCKED

FPGA_CON[15..0]16

9

Optical buffer 1:16 VME bus, used to take 3,3V power supply and grounding from the backplane

connectors. Size: VME 9U standard (367 x 400 mm). Approximately power consumption 1,6A@3,3v. Front-panel:

One RJ-45 style LC connector input. 16 RJ-45 style LC connectors outputs. One power LED.

Multimode 850 nm and 1.0625 GBd Fibre Channel Infineon® transceivers (V23818-K305-L17)

Four Freescale® Clock drivers (MC100ES6111)

16 OUTPUTS

10

Computers and softwarePC_1 : FILAR and DATA storage

Supermicro dual Xeon CPU 2 FILAR cards; read out 4 RODs Data storage in a shared file system

PC_2 : Monitoring tasks Slinksampler - Access to the shared file

system CRC calculation and checking

PC_3 : TDAQ- Main partition IGUI (System configuration ) JAVA panel for dynamic tests

11

Setup at LAB (IFIC –Valencia)

DUAL TIMER

OMB 6U

OPTICAL BUFFER

ROD crate

PC_3PC_2

PC_1

12

Outline

IntroductionSetup & tools for ROD productionProduction testsResults and conclusions

13

ATLAS calorimeters common ROD tests At the industry (PCB ‘TechCI’ (France) and components assembly

‘Seisystem’ (Italy))RODs will be delivered with : General tests and mechanical checks (installation of

pieces, connectors) JTAG boundary scan tests with final PU installed X-ray tests on PUs

University of Geneva tests Static tests Data path tests: send data from the Staging FPGA via the PU to

the OC Dynamic tests: Injector ROD Slink (or SDRAM)

Different frequencies Full & Staging mode

14

RODs modifications for TileCalHardware: HW modifications in order to adapt

the ROD for TileCal requirements related to the frequency of data reception in G-Links (clocks, passive components) and number of PUs used per ROD, 2 instead of 4.

Firmware:• Firmware specific for Tiles: Staging FPGA

(v.5.2.0)• Also specific firmware for InputFPGA and DSP

code, downloaded at configuration time.• Firmware upgrades: Output_FPGA(v16.09.05)

and VME_FPGA(30a.06.05).

15

Production database Every time a ROD is modified, it is labeled. We associate to

every ROD, two Processing Units and one Transition Module. All the group is labeled and tested together.

Every test done to any ROD, is saved in the database with all the information about the test: RODid, time, events, rate, events checked, errors and result.

Once a ROD has passed all the tests, this ROD with the PUS and TMs associated are labeled with a FINAL label (sticker), and is installed in the pit together.

16

ROD production test protocol

LEVEL RODs RATE TIME (at least)0 Three DVS tets.1 1 200 Hz 4 h.2 1 1 KHz 8 h.3 4 1 KHz 72 h.

DVS : It is run from the TDAQ software and write and read in all the registers testing the correct functionality of all the registers in the ROD. It also boots the DSPs and injects three events from the DSP and read these by the VME bus.

LEVEL 1: At this rate we can check all the events passing across the ROD. LEVEL 2: We check the ROD with busy signals produced saving data in disk.

We check 40% of events and we test the busy handling in the DSP. LEVEL 3: We check half crate with busy signals. The number of events

checked are ~ 10%, but in 72 hours we check up to 22 Mevents per ROD. Checking: All events include the CRC16 word for the whole event, and we

check this word when the event arrives to the data storage computer. Besides, the TDAQ checks for lost events.

17

Outline

IntroductionSetup & tools for ROD productionProductionResults and conclusions

18

ROD production results

RODs validated: 32 + 4 sparesDifferent problems found and solved:

• RODF24 : ORx replaced• RODF30 : DC/DC replaced• RODF25 : scratch over a data bus. Repaired at UniGe.

RODs repaired were revalidated after their reparation.

19

ROD production results (II)Level 1 Level 2 Level 3 Extra runs

Time (h) 259 405 2001 560Events processed 268 x 106 1,6 x109 7,1 x 109 2,7 x 109

Events checked 268 x 106 341 x 106 700 x 106 280 x 106

Total hours: 3225 h. Total events processed: 13 x 109 events Total events checked: 1,7 x 109 events Each ROD at least 84 hours, 264 x 106 events processed and 38 x 106 checked events without errors. Events injected for the production of 175 words ( 9 samples / 1 gain )

It represents a bit error rate for all the bits checked better than 10-13

20

Temperature tests The common ROD is prepared

for water cooling, due to LAr G-links are clocked beyond the nominal specifications.

Tests done in the lab and in USA15.

Emulating the final situation with 8 RODs and 8 OMB.

Air cooling better in USA15 than in the lab.

In USA15, temperature below 50ºC in all the cases.

Staging_FPGA reprogrammed to switch on the LED in the front panel if temperature in any G-Link is above 60ºC. 35

40

45

50

55

60

65

70

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Slot

Tem

pera

ture

(ºC)

)

USA15

Laboratory

21

Conclusions 36 RODs validated / 24 installed in USA15. Currently in commissioning period, RODs are being

integrated in the ATLAS data taking system. BER (10-13) better than the G-Link specifications (10-

12) The production test-bench is used to develop ROD

firmware upgrades and DSP reconstruction code.