CMS ECAL A new readout system architecture for the CMS ECAL Magnus Hansen 20030930

CMS ECALCMS ECAL

A new readout system A new readout system architecture for the CMS architecture for the CMS

ECALECALMagnus HansenMagnus Hansen

2003093020030930

LECC 2003 September 30th 2003 M. Hansen, CERN. [email protected]

AgendaAgenda

Short historyShort history rEvolutionrEvolution A new readout system architectureA new readout system architecture A New ASIC: FENIXA New ASIC: FENIX Front End CardFront End Card System Test System Test ConclusionConclusion


AgendaAgenda

Short historyShort history rEvolutionrEvolution A new readout system architectureA new readout system architecture A New ASIC: FENIXA New ASIC: FENIX Front End CardFront End Card System Test System Test ConclusionConclusion

New SystemArchitecture

Chips / ASICs

ElectronicsModules

System


Old DesignOld DesignArchitectural OverviewArchitectural Overview

ROSE

FE

DCC

Partition

RegionalTrigger Data

DAQData

TTSTTC

Local TriggersIn

Stand aloneMode


rEvolutionrEvolution

Simple Front EndSimple Front End Minimal hardware

Large number of optical linkLarge number of optical link 1 link per channel

Large upper level readout systemLarge upper level readout system Maximal flexibility (FPGA tech.)

Estimated not to be affordableEstimated not to be affordable

Developed Front EndDeveloped Front End Trigger Primitive generation Primary event storage

Modest number of optical linkModest number of optical link 2 data links per tower (25 ch)

Modest upper level readout Modest upper level readout systemsystem

Estimated to be affordableEstimated to be affordable


New architectureNew architecture

Implemented in Front EndImplemented in Front End MGPA + Multi-ADC for dynamic range compression and

digitization (Change from analogue gain switching to digital gain selection)

TPG (Trigger Primitive Generator) Pipeline storing digitized data waiting for level 1 trigger

decision Primary event buffer

Implemented in Counting roomImplemented in Counting room CCS (Clock and Control System card, Collaboration with CMS

Tracker, Pixel) DCC (Data Concentrator Card) TCC (Trigger Concentrator Card) SRP (Selective Readout Processor)


Architectural OverviewArchitectural Overview

TCC

CCS

FE DCC

Partition

TriggerData

DAQData

TTSTTC

Local TriggersIn

Stand aloneMode

RegionalTrigger Data

DAQData

SRP


Front End System Functional Front End System Functional RequirementsRequirements

Trigger Primitive GenerationTrigger Primitive Generation Absolute calibration of each channel Implement existing well defined algorithm

Verification needed before production No future basic changes possible

Latency budget imposed

Readout of data corresponding to positive trigger decisionReadout of data corresponding to positive trigger decision Dead time free

Three clocks / trigger imposed by TTC system 100 kHz level 1 trigger rate

Some trigger rules apply Overflow protection

Programmable level 1 trigger delay Pipeline of programmable length

Support for monitoringSupport for monitoring Laser monitoring, temperature measurements, etc.


Other Requirements and Other Requirements and ConstraintsConstraints

Debugging and testability featuresDebugging and testability features Pattern injection

Possibility to inject known pattern in the beginning of the trigger primitive generation and the readout chain

Boundary scan / scan chains BIST

Built In Self Test for production test and in situ

Radiation environmentRadiation environment SizeSize

Have to fit behind served crystals

Short development timeShort development time Start June 2002 Full production January 2004 One advantage – knowledge of the requirements


The Front End - A Readout The Front End - A Readout CubeCube

MotherboardMotherboard Creating flat surface for electronics

installation Kapton cable to APD connector

VFE cardVFE card Analogue Signal Processing Digitization

LVR cardLVR card Voltage regulation for FE system

FE cardFE card Digital Signal Processing

Trigger Primitive generation Temporary storage

Pipeline, event buffer

GOHGOH Complete Optical transmitter

module including a GOL and a laser diode


A technical choice i)A technical choice i)

Single Front End ASICSingle Front End ASIC O(500) IOsO(500) IOs

Huge chip ~20 by 20 mm

O(4000) chips in CMS O(4000) chips in CMS ECALECAL

Quickly considered as a Quickly considered as a non-optimal choicenon-optimal choice


A technical choice ii)A technical choice ii)

Seven Front End ASICsSeven Front End ASICs Three typesThree types O(150) IOs eachO(150) IOs each

~7 by 7 mm

O(20000) + O(4000) + O(20000) + O(4000) + O(4000) chips in CMS O(4000) chips in CMS ECALECAL

Soon considered as a non-Soon considered as a non-optimal choiceoptimal choice


The technical choiceThe technical choice

Seven Front End ASICsSeven Front End ASICs Single typeSingle type Three operation modesThree operation modes O(150) IOsO(150) IOs

~7 by 7 mm

O(30000) chips in CMS O(30000) chips in CMS ECALECAL

Considered as an optimal Considered as an optimal choicechoice

The new ASIC is called The new ASIC is called FENIXFENIX

Front End New Intermediate data eXtractor


FENIXFENIXDescription 1: Four Operation ModesDescription 1: Four Operation Modes

StripStrip Creating filtered Strip / Pseudo-strip sums for TCP inputs Pipelines and primary event buffers

TCPTCP Trigger Cell Processor Finalising the trigger primitive for one trigger tower in the

Barrel DAQDAQ

Tower (Super Crystal) readout state machine Event encapsulation

MEMMEM Reading out the Laser monitoring monitoring system Pipelines and primary event buffers


FENIXFENIXDescription 2: ControlDescription 2: Control

Fast controlFast control T1 signal as defined in the Tracker

slow control system “100” => Level 1 Trigger accept “101” => BC0

Local Bunch Crossing counter reset

“110” => Re-synch Reset of all counters and state

machines

“111” => Force VFE mode From programmable default mode

to programmable calibration mode

“110110” => Power-up reset Reset of all counters and state

machines and load default values into all registers

“1100110” => Power-up reset As above

Slow ControlSlow Control I2C interface

Extended 10 bit addressing Standard protocol Direct addressing of all set-up

addresses

Compatible with CCU I2C master ports

Fully synchronous design Synthesizable Auto P&R

150 set-up addresses 132 Set-up registers 18 RAM access


Development AccelerationDevelopment Acceleration

Intermediate deviceIntermediate device Xilinx FPGA Cadence for simulation Synplify for synthesis

Modern ASIC design toolsModern ASIC design tools Synopsis for synthesis Silicon Ensemble for Place &

Route Very short design turn-

around time 2 weeks claimed

Generic HDL descriptionGeneric HDL description No component instantiation


ASIC Emulation in FPGAASIC Emulation in FPGA

FeaturesFeatures Observable functionality identical Identical footprint Identical pin-out

Not implemented to save resourcesNot implemented to save resources Triple-redundancy in registers Error Correcting Code in RAMs BIST in RAMs


ASIC Emulation in FPGAASIC Emulation in FPGA- Applied HDL design rules- Applied HDL design rules

Generic Source CodeGeneric Source Code No process dependent component instantiation

Exception: RAMException: RAM Technology specific, recommended not to infer

Adopted strategy: Adopted strategy: All functional simulation done with generic RAM

“Superset” of Xilinx RAM and ASIC RAM used For the FPGA, the Xilinx RAM block is wrapped and

instantiated Routed design simulated and verified for conformity

For the ASIC, the modular static RAM cell developed at CERN is wrapped and instantiated Routed design simulated and verified for conformity


FENIX ASICFENIX ASICRadiation Tolerance: StrategyRadiation Tolerance: Strategy

ObservationObservation ASIC technology is radiation tolerant Registers and RAM cells subject to SEU

StrategyStrategy Protect against SEU Not protect against hardware failure

TestabilityTestability Always a challenge Improved by insertion of a “testability flag”


FENIX ASICFENIX ASIC SEU Tolerance 1: Set-up RegistersSEU Tolerance 1: Set-up Registers

Set-up registersSet-up registers Triple-redundant flip-flops

SEU resistantVoting logicthree (two) clocks long

write pulse needed

Features Synthesizable

Test Any discrepancy flagged Discrepancy when written


Triple-redundantTriple-redundantSet-up registerSet-up register

VHDL codeVHDL code

setup_register : process (clock40) begin if rising_edge(clock40)then if pwup_reset = '0' then register1_8b <= pwup_value;

seu_flag <= '0'; elsif address = register_address and write_enable = '1' then register1_8b <= write_data; else register1_8b <= voted_register_value_8b; end if; register3_8b <= register2_8b; register2_8b <= register1_8b; if register1_8b = register2_8b and register1_8b = register3_8b then seu_flag <= '0'; else seu_flag <= '1'; end if; else null; end if; end process setup_register;


FENIX ASICFENIX ASIC SEU Tolerance 2: State MachineSEU Tolerance 2: State Machine

State MachineState Machine Triple-redundant

SEU resistantVoting logicExcept state changes

Features Synthesizable

Test Any discrepancy flagged Discrepancy when state

change


FENIX ASICFENIX ASICSEU Tolerance 3: RAMSEU Tolerance 3: RAM

RAMRAM Hamming code

Encode at write Decode & Correct at read

one bit error correction

Features Synthesizable Single bit SEU safe

Test ECC Decoder

During BIST execution

ECC EncoderDuring normal operationThrough slow control (I2C)


FENIX ASICFENIX ASICTestabilityTestability

Test time budget: 1 Test time budget: 1 secondsecond Without chip handling

Triple-redundant Triple-redundant RegistersRegisters Testability flag can be

used as a signature of operation

RAM BISTRAM BIST Fully automatic Write whole RAM and read

back Launched by a pulse on IO

pin Boundary scan Tester

Observable on external pins Boundary scan Tester

Can be launched and monitored in situ (I2C)


FENIX ASICFENIX ASICStatus and PlansStatus and Plans

First submitted February 2003First submitted February 2003 Received back from foundry in May Not yet received back from packaging

Becoming critical

Next submission after ESRNext submission after ESR 9th of October Engineering run

Final design O(3000) dies, for up to 3 CMS ECAL Super

Modules Tested chips back before end 2003

ProductionProduction Beginning of 2004


FE CardFE CardFunctionality and PerformanceFunctionality and Performance

Full TPG in BarrelFull TPG in Barrel Sum of Five filtered strip sums Single data link to TCC and regional trigger 11 clocks latency

FE card input to GOH connector

Partial TPG in End CapPartial TPG in End Cap Five filtered strip sums Five data links to TCC and regional trigger 7 clocks latency

FE card input to GOH connector

ReadoutReadout Serves 25 channels Single data link to DCC and DAQ Dead time free

7.2us service time, 25 primary event buffers, (10 samp/ch/evt, P(n=d) = 10 -8*) Null event insertion up to 127 pending events

Programmable pipeline length corresponding to level 1 trigger delay*TTS for CMS DAQ, A. Racz*TTS for CMS DAQ, A. Racz


FE card LayoutFE card Layout


Tower in Super ModuleTower in Super Module

TriggerGOH

CCU

FENIXFPGA

QPLL

VFE

LVR

ReadoutGOH


Beam Test 2003Beam Test 2003


ConclusionConclusion

A New Readout System Architecture for CMS A New Readout System Architecture for CMS ECAL has been presentedECAL has been presented

The CMS ECAL Front End CardThe CMS ECAL Front End Card Serving 25 readout channels

Tower in the Barrel Super-Crystal in the End Cap

The FENIX chipThe FENIX chip Implements the main functionality on the Front End card Three (four) operation modes FPGA emulator implemented ASIC prototype implemented, Final design submitted after ESR

Prototype system successfully tested in beamPrototype system successfully tested in beam

Documents

CMS ECAL A new readout system architecture for the CMS ECAL Magnus Hansen 20030930