Upload
lou
View
43
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Pipelined Electronics. Design Concept: Low Cost. Add-on modules. Sub-detector I/F User-defined module for analog part only. Minimizing the development cost. Readout CPU Commercially available PMC module. No hardware development cost. Common pipelined-readout platform (motherboard) - PowerPoint PPT Presentation
Citation preview
Apr.22,2005 Super B Factory Workshop
Detector (DAQ/Computing) Parallel
Takeo HiguchiInstitute of Particle and Nuclear Studies, KEK
on behalf of the COPPER working group
Pipelined ElectronicsPipelined Electronics
• Add-on modules
• Common pipelined-readout platform (motherboard)– Generalized DAQ software.
• Common pipelined-readout platform (motherboard)– Generalized DAQ software.
Design Concept: Low Cost
Sub-detector I/F User-defined module for
analog part only. Minimizing the
development cost.
Sub-detector I/F User-defined module for
analog part only. Minimizing the
development cost.
Readout CPU Commercially available
PMC module. No hardware
development cost.
Readout CPU Commercially available
PMC module. No hardware
development cost.
Common Readout Platform: COPPER
PMCProcessor
Trigger
GenericPMC slot
Sub-detector I/F
Sub-detector I/F
Sub-detector I/F
Sub-detector I/F
On-board Ether
Form factor = VME 9U
Sub-detector I/F slot 4Pipeline FIFO (1MB) 4PMC slot 332-bit local bus
32-bit 33MHz PCI bus
Local-PCI bridge
FastEthernet I/F
VME I/F
Sub-detector I/F slot 4Pipeline FIFO (1MB) 4PMC slot 332-bit local bus
32-bit 33MHz PCI bus
Local-PCI bridge
FastEthernet I/F
VME I/F
Data transfer on COPPERmeasured >65 MB/s.
>30 kHz L1 rate @ 1.6 kB/ev.Data size corresponds to typical SuberKEKB DC/COPPER.
Data transfer on COPPERmeasured >65 MB/s.
>30 kHz L1 rate @ 1.6 kB/ev.Data size corresponds to typical SuberKEKB DC/COPPER.
PMC / PMC Processor
• PCI Mezzanine Card
– 100% compliant with the PCI.– Suitable for high density applications.
– Many commercial products are available: Ethernet cards, GbE cards, memory modules, CPUs, etc.
• RadiSys EPC-6315– PentiumIII 800 MHz w/ 512 MB main memory.
– 32-bit 33/66 MHz PCI bus interface.
Contains Linux kernel image
PCI NIC PMC NIC
Sub-Detector I/F: FINESSE
• FINESSE: 18676 mm2 daughter card on the COPPER– Receives and digitizes sub-detector analog signals.– Holds the digitized signals for the level-1 trigger latency.– Outputs the data to the COPPER.
• Some FINESSE variations– TDC
– ADC
– TDC “tandem”
SuperBelle CDC Readout: QTC+TDC
• QTC: charge to time conversion
Qevent
0 tDRIFT
L1-trigger
drift chargeto sense wire
t
T(Q)
tSTOP
Q-to-T conversionQ-to-T conversion
t
t1
t2
TDC & ADC at the same time,simplifying the readout scheme.
• t1 = drift time
• t1 – t2 = drift charge
9U-VME sized QTC modulewith LeCroy MQT300 QTC chip
9U-VME sized QTC modulewith LeCroy MQT300 QTC chip
For tracking performance
Position resolution < 130 m
For dE/dx performance
Dynamic range 10 bit
Linearity < 0.5 – 1.0%
Other boundary conditions
Expected single rate per wire 100-200 kHz
Readout ch# ~ 15 k in total
For tracking performance
Position resolution < 130 m 27 m/bit (0.78 ns)
For dE/dx performance
Dynamic range 10 bit 17 bit
Linearity < 0.5 – 1.0% 0.49%
Other boundary conditions
Expected single rate per wire 100-200 kHz
Readout ch# ~ 15 k in total 24 ch/chip
SuperBelle CDC Readout: QTC+TDC
• TDC: time to digital conversion
Requirement
FINESSE with AMT-3 TDC chip (originally designed for ATLAS)FINESSE with AMT-3 TDC chip (originally designed for ATLAS)
AMT-3 TDC chip
AMT-3 TDC Chip
• Up to 17 bit timing measurement: coarse + fine counters
– Pulse edge timings: coarse + fine– Trigger timings: coarse
• Trigger matching– Only the edges associated to Level-1 trigger are output.
• Level-1 pipeline provided (256 edges)
Coarse counter: 13-bit bunch crossing counter (40MHz @ LHC)Fine counter: additional 4-bit time memory cellCoarse counter: 13-bit bunch crossing counter (40MHz @ LHC)Fine counter: additional 4-bit time memory cell
SuperBelle system clock• Bunch crossing rate: LHC = 40 MHz,
KEKB = 508 MHz.• We choose SuperKEKB system clock
= 42.33 MHz = 508/12.
SuperBelle system clock• Bunch crossing rate: LHC = 40 MHz,
KEKB = 508 MHz.• We choose SuperKEKB system clock
= 42.33 MHz = 508/12.
Level-1 pipeline FIFO(256 edges)
Readout FIFO(64 edges)
Channel buffer(4 edges)
Channel buffer(4 edges)
Channel buffer(4 edges)
Trigger matching
Trigger timing FIFO(8 events)
24
AMT-3
24-channel inputs
AMT-3 output
Level-1 trigger inputs
AMT-3 TDC Chip – Cont’d
collisiontime
L1 trigger
trigger latency (~5s?)
edge search
OK NG
Trigger matching
of AMT-3
No hardware output indicating FIFO full– Unexpected data loss is only notified by an “error flag” in the next
event output.
Readout “readout FIFO” of AMT-3 quickly
to COPPERDREADY
DATA
AMT-3AMT-3FPGA
w/ FIFOFPGA
w/ FIFO
SCLK: 42.3 MHz
BUSY
No further trigger
AMT-3 L1-Pipeline Depth
Less depth: 256 edges shared by 24-input channels– LeCroy TDC (we’re currently using): 16 edges / each channel.– Is it sufficiently large? MC simulation is needed.
• MC parameters – Assuming SuperBelle drift chamber– # of input channels: 24– Random wire hit: 100 kHz (Poisson distribution)– Level-1 trigger rate: 50 kHz (Poisson distribution)– # of gen. words per wire hit: 2 for signal edges + additional for noise– Level-1 trigger latency: 5 s
Extreme case
“Toy” MC Simulation: Buffer Usage
Level-1 pipeline usagesampled at every clock [edges]
En
trie
s / 1
ed
ge
13.7 s
1.37 s
137 ms
13.7 ms
FIFO full point
Extrapolate … = ~104
Extrapolation assuming the right side slope unchanged.
FIFO/AMT-3 won’t get fullbefore O(105) sec.
Very preliminary
TDC FINESSE with AMT-3
• Block diagram of TDC FINESSE (not finalized yet)A
MT
-3A
MT
-3 Dataformatter
Dataformatter
Triggerreply logic
Triggerreply logic
FIFO-fullmonitor
FIFO-fullmonitor
Busygenerator
Busygenerator
FIFO
42.3MHzclock Level-1 trigger
Trigger busy
Local-busI/F
Local-busI/F
Sub
-det
ecto
rS
ub-d
etec
tor
CO
PP
ER
eve
nt F
IFO
CO
PP
ER
eve
nt F
IFO
Local bus (A7D8)
COPPER FIFO full
Xilinx Spartan-3
[Readout data] [Readout data]
COPPERCOPPER
The First AMT-3 Signal on COPPER
Pulsegenerator
Pulsegenerator
NIMECLconverterNIMECLconverter
FINESSEFINESSE
AMT-3AMT-3250 ns 4.2 s
w#1: 00111010010111011110110110000110
w#2: 00111010010110011110111011011010
@ ch 0x0b
content tag(0x3 = edge data)
ch id.(ch = 0x0b) 1: leading edge
0: trailing edge 17-bit edge timing
pulse width
Measured pulse width dist.
peak = 251 nsrms = 0.93 ns
CPUCPU
w#1: 00111010010111011110110110000110
w#2: 00111010010110011110111011011010
Future Plan
• Short term (-Jul.): System test– TDC resolution study with test input pulses.– Investigation of FIFO full behavior of AMT-3.– Stability tests: stability against random trigger, long-run stability, etc.– Software development: device driver, readout software, etc.
• Middle term (Aug.-Nov.): First step integration to Belle DAQ (EFC)– EFC is the smallest sub-detector (total = 320 ch).– EFC has two alternative readout paths (CDAQ + EFC own): failsafe and easier cross
check.
• Long term (Dec.-): CDC readout with COPPER based system
First Step Integration into Belle DAQ
• Replace Belle EFC DAQ with COPPER EFC =Extreme Forward Calorimeter
EFC =Extreme Forward Calorimeter
ForwardEFC
ForwardEFC
BackwardEFC
BackwardEFC
160 ch
160 ch
COPPER COPPER TDCTDCTDCTDC
TDC3
TDC3
FASTBUS
VxWorksEB
VME crate
COPPER3
COPPER3
Eth
ern
et s
wit
ch
PC
TDC “Tandem”FINESSE
TDC “Tandem”FINESSE
LocalSTORLocalSTOR
Summary
• We have developed module-structured system for SuperBelle readout consisting of COPPER boards and FINESSE cards.
• The QTC and TDC will be employed to readout SuperBelle CDC data.
• We use AMT-3 TDC chip. The TDC FINESSE card with the AMT-3 has been developed.
• Level-1 pipeline depth in the AMT-3 is examined by MC simulation.
• We have observed the AMT-3 is correctly digitizing the input pulses.
• The first step integration of the COPPER system into the Belle DAQ is planned in this August using EFC.
16 ch
16 ch
8 ch
8 ch
TDC “Tandem” FINESSE
• LeCroy TDC: 6 connectors• COPPER: 4 FINESSE slots
• To use present analog cables…– We’ve made a ‘tandem-FINESSE’
module equipped with 3 connectors.
• Channel density– 48 ch / tandem-FINESSE.– Same as present LeCroy TDC.
Tandem-FINESSE
AMT-3AMT-3
AMT-3AMT-3
2
Readout PC and Event Builder
copper0copper0
copper1copper1
copper5copper5
…
Readout PC
Present event builder
Sw
itchi
ng h
ubS
witc
hing
hub
100 Base-T
GbE GbE GbE
• Readout PC:– CPU : Intel Dual Xeon 2.46 GHz– OS : RedHat9(2.4.18-20smp)
• Readout PC:– CPU : Intel Dual Xeon 2.46 GHz– OS : RedHat9(2.4.18-20smp)
Data transferperformance
Faster readout PC and/or software tuning are requested for SuperBelle use.
Calibration Run
• Calibration run– Daily detector calibration with test pulse and/or test trigger.
• Readout PCs should take responsibility to collect data– O(100) COPPERs / sub-detector 6 readout PCs Event building mechanism is needed for calibration run.
CPU Boot-up Mechanism
• Compact Flash– Boot up procedure is well established.
– Distribution of updated software to all CF cards is non- trivial.
– Write access to CF card shortens the card lifetime.
– CF card is getting out of date.
• Network boot– Distribution of updated software to all CF cards is easy.
– We have no established method yet to boot-up EPC-6315 from network.
– Boot-PC farm serving multiple boot requests must be developed. It is never trivial.
Run Control
copper0copper0
copper1copper1
copper5copper5
…
NSM Gateway
NSMNSM or other mechanism
To avoid increase of NSM nodes, we put NSM gateway (like SVD2).
fbdaqprivate network
Readout PC(l0efc)
NSM Master