2
DAQ Status
• DAQ modifications
• Replace DRS2 by DRS4 boards
• Added synchronization signal between boards
• Added slow control equipment into DAQ allowing remote control
• Overview of DAQ performance
• Average total trigger rate 6.4 Hz
• ~5 minutes runs with 2000 events
• Inter-run time: 6.8 s
• Event size:9 MB (raw) 2.6 MB (zero suppressed)
• Data rate 16.6 MB/s, 1.4 TB/day
• Offline compression ÷ 2 38 TB taken in ~54 days
• DAQ issues during routine running
• Lost sync signal: twice a day run restart (10 s)
• Run startup problem: every few hours retry start (4 s)
3
OfflineCluster64 CPUs
150 TB disk
DAQ scheme
TRG1 TRG2 TRG3 TRG9 DRS4 DRS5 DRS6 DRS7 DRS8
trigger & sync & trigger type & event # LSB
busy
internal trigger & busy
SYSTEM01 SYSTEM02 SYSTEM03 SYSTEM04 SYSTEM05 SYSTEM06 SYSTEM07 SYSTEM08 SYSTEM09
Event Builder
SYSTEM
Logger
Archiver
5
“Ghost pulse” problem
R
“Ghost pulse”2% @ 2 GHz
“Ghost pulse”2% @ 2 GHz
After sampling a pulse, some residual charge remains in the capacitors on the next turn and can mimic wrong pulses
After sampling a pulse, some residual charge remains in the capacitors on the next turn and can mimic wrong pulsesSolution: Clear before write
write clear
Fixed in DRS4 chip
6
Problems with DRS2
• Ghost pulse effect (previous slide)
• Clock crosstalk (1-2 mV)
• Temperature dependenceoffset and gain
• Complicated calibration DAQ rate was limited by front-end computer software calibration
• Offset shift of last 64 bins
• Offset shift with readoutfrequency
Redesign of DRS chip and replacement of complete
DAQ electronics
Redesign of DRS chip and replacement of complete
DAQ electronics
8
On-chip PLL
PLL
ReferenceClock
Vspeed
Internal PLLInternal PLL
•On-chip PLL locks sampling speed to external clock•fclk = fsamp / 2048
•On-chip PLL locks sampling speed to external clock•fclk = fsamp / 2048
9
Old vs. New Synchronization
DRS2
signal
clock
trigger
DRS4
signal
Channel 0
Inverter Chain
Channel 1
Channel 2
Channel 3
Channel 4
Channel 5
Channel 6
Channel 7
Channel 8
PLL
Skipping Channel 8 during readout reduces dead time by 11%Skipping Channel 8 during readout reduces dead time by 11%
10
Global Synchronization
masterquartz
fan
out
VME Board
DRS chips
Clock dividerand jittercleaner
(LMK03000)
20 MHz
20 MHz
0.7
8 M
Hz
0.7
8 M
Hz
11
Synchronization of clock chips
1.2 GHz
0.78 MHzChip 1
0.78 MHzChip 2
n * 0.83 ns
SYNC & 20 MHz
• SYNC has to arrive on all board within 50 ns trigger bus• 20 MHz MEG clock has to arrive on all boards within 0.83 ns
• SYNC has to arrive on all board within 50 ns trigger bus• 20 MHz MEG clock has to arrive on all boards within 0.83 ns
20 MHz
50 ns
12
Problems with synchronization
• Synchronization pulse has to arrive within 50 ns on all boards optimize trigger bus cabling
• Clock divider chips need several configuration cycles
• Re-timing with global clock required firmware modification due to bug in original version syncing did not work initially
• Time frame changed (global clock edge vs. bin #0 time) modification in analysis and calibration database
• Due to these problems, the deployment of the DRS4 boards were not as smooth as anticipated
13
Timing Limitations
• Requirements: « 100 ps
• Current timing resolutions measured with split signal:
• 15 ps (same chip)
• 30 ps (different chips same mezzanine board)
• 130 ps (different VME boards)
• Worse than for DRS2 chips ( Ryu’s talk)
• Inter-board timing limits our experiment
• Immediate changes
• “Bypass wire”
• Keep DRS3 for timing counters
• Possible causes
• Global Clock Jitter
• Jitter inside DRS4 boards
• Jitter added by active splitter
• ~10 MHz 1 mV noise on top of signal
clock
fan
-ou
t
PMT splitter
trigger
+
=
threshold
14
Clock Distribution
LMK03000
< 20 ps period < 20 ps period (was 120 ps with old resistor)
LMK03000
< 32 ps relative
mezz
anin
em
ezz
anin
e
added “bypass wire” to sample original clock jitter of on-board clock distribution eliminated
15
Aperture Jitter inside DRS chip
3.2 GHz
1.6 GHz
rtUu
t
slope
Intrinsic DRS aperture jitter 1/fsampling
if power supply (U) is constant
Intrinsic DRS aperture jitter 1/fsampling
if power supply (U) is constant
16
Plan to improve timing
• Increase sampling speed 1.6 GSPS 3.2 GSPS
• only firmware modification required
• goal: end of March
• Disentangle different contributions to timing
• Re-measure global clock jitter between crates
• Measure split pulse jitter before/after active splitter
• Investigate noise situation in area(XEC timing improved in ’09, e+- timing decreased, TC uses same electronics (DRS3) noise in TC signals ???)
• Optimize DRS4 timing calibration algorithm can also be applied to old data
• Lab tests showed ~10 ps accuracy with sampling technique(DRS4 [PSI], SAM [Saclay], BLAB [U.Hawaii])
• Goal: Electronics should not limit capabilities of experiment
17
Daisy-chaining of channels
Channel 0 – 1024 cells
Channel 1 – 1024 cells
Channel 2 – 1024 cells
Channel 3 – 1024 cells
Channel 4 – 1024 cells
Channel 5 – 1024 cells
Channel 6 – 1024 cells
Channel 7 – 1024 cells
Domino Wave Generation • DRS4 can be partitioned in 4x2048 cells
• Running with 2048 cell channels allow sampling speed of 1.6 GHz * 2 = 3.2 GHz with current trigger latency internal timing accuracy should improve ~2x
• Deal with increased data rateby on-board averaging
• DRS4 can be partitioned in 4x2048 cells
• Running with 2048 cell channels allow sampling speed of 1.6 GHz * 2 = 3.2 GHz with current trigger latency internal timing accuracy should improve ~2x
• Deal with increased data rateby on-board averaging
1.6 3.2 averaging
18
• Reduce dead time
• Use various binning regions in waveforms
• Double-buffering in front-end electronics
• Use multi-threading in run start/stop sequencing(6.8 s inter-run gap 3-4 s = 1% dead time improvement)
VME Transfer
Further plans
Rebinning 2:1, 4:1, …
Chip readout
0.5 ms
VME Transfer
25 ms
Chip readout VME Transfer
now
reduceddata size
multi-eventbuffers
Chip readout
ready fornext event
19
2nd Level Trigger
• Drift Chamber wires are connected to trigger, but too slow to be included in trigger decision (400 ns)
• Would be possible with 2nd level trigger scheme
• Only firmware modifications (spare bits on trigger bus)
• If trigger rate ÷2, dead time would be ~÷2
1st LT
400 ns 25 ms
event
stop DRS chips
2nd LT
1 us
Chip readout VME Transfer
trigger
DRS boards
restart DRS chips