Towards a CBM-XYTER Readout Chain

Towards a CBM-XYTERTowards a CBM-XYTERReadout Chain Readout Chain

Walter F.J. Müller, GSI, Darmstadt

11th CBM Collaboration Meeting26 February 2008

26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 2

CBM-XYTER DataCBM-XYTER Data

A CBM-XYTER hit can be represented in 6 bytes (48 bit) The 'hit message' might look like (here 42 bits)

message type (1 bit) chip id (~ 8 bit) channel id (7 bit) status flags (~4 bit) time stamp (~14 bits) amplitude (~8 bits)

On a serial link we have some overheads Framing, FEC, ect: ~75% efficiency 8bit/10bit coding

'on-the-wire' we need about 80 bit to encode a hit

2.5 Gbps ↔ 2.5 Gbps ↔ 31.2531.25 MHz hit rate MHz hit rate


Some ValuesSome Values

Use hit densities form CBM Technical Status Report2006 Update, Section 13.1 "Hit densities and Rates"

Detector edge hit/cm2 part/cm2 TIDSTS @ 30cm inner 10 7.5·1014 20 Mrad

outer 0.25 1.8·1013 0.5 MradSTS @ 1m inner 1 7.5·1013 2 Mrad

outer 0.03 2.3·1012 60 kradTRD @ 4m inner 0.04 3.0·1012 80 krad

outer 0.002 1.5·1011 4 kradTOF @ 10m inner 0.01 7.5·1011 20 krad

outer 0.0006 5.0·1010 1.2 krad

STS @ 30 cm is now 1st plane in 'all strips' configuration(the hit rate for STS@30 cm is scaled from the STS3 @ 20 cm plot of the CBM TSR)

Hit rates in 1st MUCH plane are similar to STS plane @ 1m


STS Data Rates – Rough STS Data Rates – Rough EstimateEstimate With 60 μm pitch: 128 channel ↔ 0.768 cm The hottest place:

10 hit(cent)/cm2 ↔ 2.5 hit(mbias) /cm2 ↔ 25·106 hit/(s· cm2) A 2cm sensor ↔ 1.5 cm2 ↔ 37.5 MHz hit rate/chip Note: background and cluster size not included here

A typical data link to DAQ: Assume 2.5 Gbps link speed from here on

Number of data links needed: A single CBM-XYTER can fill a 2.5 Gbps link most of the CBM-XYTER see less hits in most cases 2, 4, or 8 CBM-XYTER's can be aggregated

to fill a 2.5 Gbps link


STS Data Rates – MC ResultsSTS Data Rates – MC Results

Radek made a detailed study. The current 8 station STS with a total of ~1.2 Mchannels, 9296 CBM-XYTER chips 586 modules (with mostly 2 * 8 chips)

Note: cluster size not yet included ( 1 strip hit per particle)

(ALICE SSD has an average cluster size of ~1.4) effects due to non-perpendicular incidence also still

ignored

Following tables under the caveats listed above


STS Data Rates – Au+Au @ STS Data Rates – Au+Au @ 25AGeV25AGeV

Hit rate [MHz]

Sta 1

Sta 2

Sta 3

Sta 4

Sta 5

Sta 6

Sta 7

Sta 8

Total

16-32 23 16 26 16 19 0 6 2 105

8-16 37 55 50 60 66 64 55 58 431

4-8 80 75 101 125 84 84 67 72 692

2-4 111 138 123 89 133 141 129 157 1053

1-2 109 142 162 170 176 250 161 141 1275

0.5-1 127 193 74 41 54 21 210 256 971

< 0.5 53 13 0 3 0 0 28 2 121

Chip count only for one of the two sides of the modules Properties of stations quite similar (module size tracks quite

well the inverse hit density hit/chip roughly equal)

Hit rate per chipHit rate per chip


STS Data Rates – AggregationSTS Data Rates – Aggregation

75% of the chips have < 4 MHz It is prudent to aggregate the data of several chips

onto one data link before it is send of the module

Hit rate [MHz]

# modules

128-150 16

64-128 126

32-64 188

16-32 260

8-16 312

4-8 242

0-4 28

Hit rate per moduleHit rate per module Only 2.5% of chips have a hit rate of 16-32 MHz

842 of 1172 of the module sides (71%) have a sum hit rate of <32 MHz, can thus be aggregated on a 2.5 Gbps link

in 188 module side two groups of 8 chips can be aggregated on a link


STS Data Rates – # of Data STS Data Rates – # of Data LinksLinks 75% of the chips have < 4 MHz It is prudent to aggregate the data of several chips

onto one data link before it is send of the module

Hit rate [MHz]

# module

s

#links

128-150 16 128

64-128 126 504

32-64 188 376

<32 842 842

Hit rate per moduleHit rate per module Total hit rate: 32.6 GHz Net data rate: ~200 GByte/s Number links: 1850

Per station end (top/bot) we have about 120 data links

- Aggregation may be cleverer, links a little faster- Cluster size will be >1 and increase data volume About 2000 'fast' links seems a reasonable estimate About 2000 'fast' links seems a reasonable estimate


In many cases information of all 8 chips can be aggregated to fill one 2.5 Gbps link

Some schemes: embedded (aggregation integrated in CBM-XYTER)

with communication controller (aggregation on separate chip)

CBM-XYTER@STS ReadoutCBM-XYTER@STS Readout


GBT: The CERN proposal for GBT: The CERN proposal for SLHCSLHC The GBT project at CERN addresses these issues

targeted for SLHC radhard design communication-controller handling

clock distribution trigger distribution slow control traffic data readout traffic

exactly what we talk about since years

In the following a few slides from Paulo Moreira's presentation on the TWEPP-07 workshop.

The GBT, a Proposed Architecture for Multi-Gb/s Data Transmission in High Energy

Physics

P. MoreiraCERN – Geneva, Switzerland

Topic Workshop on Electronics for Particle PhysicsSeptember 3 - 7, 2007, Prague, Czech Republic

http://indico.cern.ch/contributionDisplay.py?contribId=71&sessionId=16&confId=11994




26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI

12

Outline

Motivation The proposed link architecture

Link topology GBT chipset Link bandwidth GBT to Front-end link topology

TTC Slow control Development and testing Error correction and line coding Summary

Disclaimer:Disclaimer:“This project is still in its specification phase, detailed features are likely to change prior to the final silicon fabrication…”The objective of this presentation is to communicate the current ideas to stimulate discussion and to obtain feedback from the potential users!

Slide from: TWEPP-07 presentation of Paulo Moreira (see this Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) )



13

GBT link architecture

Bidirectional point-to-point optical fiber links The heart of the link is the GigaBit Transceiver (GBT13) Each link carries simultaneously: DAQ, TTC and SC data Embedded transceiver: the GBT Chipset Counting room transceivers: FPGA and COTS optoelectronics

components

Versatile optical link project

GBT project

GBT project

Slide from: TWEPP-07 presentation of Paulo Moreira (see this Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) )



14

The GBT chipset

GigaBit Transimpedance Amplifier (GBTIA)

GigaBit Laser Driver (GBLD)

GigaBit Transceiver (GBT13) SERDES Communications controller TTC receiver

GBT – Slow Control ASIC

Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)


15

GBT to front-end link topology

GBT to Front-end links: 32 (+1) bi-directional serial links (e-links) E-link bandwidth is 80 Mb/s Several e-links can be grouped together to serve a single front-end device achieving

bandwidths that are multiples of 80 Mb/s 16-bits for TTC 8 phase adjustable clocks An E-Link Port Adaptor (EPA) “macro” will be available for integration

in the front-end ASICs

FE ASICsclk

Link controller

Link controller

Data MoverData

Mover

TimingTiming

portport

portport

portport

Controlport

Controlport

Front-Ende-links

GBTManagement

logic

GBTManagement

logic

Slow Control ASICe-link

clk40T<15:0>

e-link

e-port

e-port

ReadoutControl Din

Dout

clk

e-link (6 wires)LVDS: Clk80, Din, Dout

32 ports80Mb/s

Timing

T<15:0>

GBT-SCAGBT-SCA

GBT13

FE ASICsclk

Link controller

Link controller

Data MoverData

Mover

TimingTiming

portport

portport

portport

Controlport

Controlport

Front-Ende-links

GBTManagement

logic

GBTManagement

logic

Slow Control ASICe-link

clk40T<15:0>

e-link

e-port

e-port

ReadoutControl Din

Dout

clk

e-link (6 wires)LVDS: Clk80, Din, Doute-link (6 wires)LVDS: Clk80, Din, Dout

32 ports80Mb/s

Timing

T<15:0>

GBT-SCAGBT-SCA

GBT13



16

SC – Two chips solution



17

Link bandwidth

GBT: 120-bits transmitted during a single bunch crossing interval 4.8 Gb/s.

4 header bits 32 forward error correction bits User field of 84 bits 3.36 Gb/s:

SC: 4-bits 160 Mb/s TTC: 16-bits 640 Mb/s DAQ: 64-bits 2.56 Gb/s

Bandwidth is fixed per frame but can be shared by the front-end devices



18

Forward error correction (FEC)

Objectives: Correct burst errors:

Generated on the PIN-diode Generated by particles hitting the transceiver Generated in the fast SERDES circuits (which cannot be triplicate) Done with minimal latency Done with good efficiency Merge with line-coding

Proposed code: Compatible with FPGAs capabilities Interleaved Reed-Solomon double error correction 4-bit symbols (RS(15,11)) Interleaving: 2 Error correction capability:

2 Interleaving × 2 RS = 4 symbols 16-bits

Code efficiency: 88/120 = 73% Line speed: 4.80 Gb/s Coding/decoding latency: one 25 ns cycle



19

Summary

We propose a “Single” Link solution for: Timing Trigger Links Data Acquisition Links Experiment Slow Control Links

The link is based on: The GBT chip set in the detectors:

GBT13, GBTIA, GBLB & GBT-SCA Optoelectronics from the Versatile Link Project An FPGA in the counting room

Both ends implement a error robust transmission protocol over an optical fibre

The GBT13 communicates with the front-end electronics with up 32(+1) e-links

Two or more e-links can be grouped to match the front-end bandwidth

We propose that the e-links communicate using the Ethernet protocol

An E-link Port Adaptor “macro” will be provided by the GBT team for integration in the front-end ASICS

Slow control is implemented by a dedicated channel One e-link Slow control is managed by the GBT-SCA



Today's SummaryToday's Summary

GBT seems to be a very interesting architecture Some aspects are quite SLHC specific (the 80 Mhz world) Some parameters might not be well adapted to our

needs why is the E-link bandwidth only 80 Mbit/sec ? all the TTC specifics

Goal is to start a discussion on the CBM-XYTER readout architecture many CBM-XYTERS will sit in places with a TID > 1 Mrad the path to the first place with TID < 1 krad, where COTS

components can be placed (e.g. FPGA), is at least 5 m. Is Cu+LVDS still feasible, given total needed bandwidth ? Or is it better to go locally (that means radhard) to optical ?


The EndThe End

Thanks for Thanks for your attentionyour attention

Documents

Towards a CBM-XYTER Readout Chain