Straw readout status Status and plans in Prague compared with situation now Choke and error...

Straw readout statusStatus and plans in Prague compared with situation now

Choke and errorConclusions and plans

Start of 2015 run (Prague)• Expected smooth start due to

• Experience 2014• Number of improvements

• problems with 2015 startup• SOB, EOB delay• trig latency obsolete data cleaning

• Stable data taking since couple of weeks• Observed problems

• Duplicated words• Missing (hidden by above?) trailing edges• Timing adjustment for T0• Some covers moving +/- 1 clock cycle at the start of

run• 1 SRB front-end interface chip not functional after

incident• 1 HV line not stable

• Some problems solved• Data corruption very complex problem

• Extra words turned out to be also triplicated• Nonsense data at high beam intensity• Padding hiding data or missing• Etc.

• Data corruption• Padding fixed in Ethernet intf• Duplicated words fixed in Ethernet intf• Triplicated and extra words fixed in trigger-

matching• Still small amount of data with duplicated

words coming possibly from frontend• ~10% of trailing edges missing?

Now

Prague meeting

Now – October 2015

Straw readout consolidation/improvement since December 2014 (Prague)• Improvements of existing FW and bug

fixes• Startup of 2.5Gb/s links – added extra ‘0’

to the alignment pattern• Stability of 2.5Gb/s links – ALTERA bug fix• Startup of 400Mb/s links to cover, link

completely redone• Re-synchronizing FIFOs added to links

outputs• Increasing the speed of data merging to

prepare for high intensity beam• Coarse T0 setting (0->255) for the whole

board and fine T0 setting (0->7) for every cover

• Startup of 2.5Gb/s links• Still a problem on some links• Possibly ‘link back’ alignement

• Stability of links OK

• Fine T0 not usable now because of control link corruption (version 9)

Now

Improvements for full intensity beam (Prague)• Cover new FW ready for test

• Faster data multiplexing with look-ahead• bigger derandomization buffer• Data transmission up to 13 MHz measurements (leading/trailing

edges)• E.g. 10 straws in ‘hot’ place with ~600kHz average rate

• FE_INTF FPGA on SRB• Fast data multiplexing • Bigger derandomization buffer• FW ready for test together with cover T0 adjustment

• EVENT_MGR FPGA on SRB• Include second Ethernet interface• Monitoring of trigger matched data (DDR3 memory readable

through VME - Implement block writing to DDR3• Not done

• VME_MGR FPGA on SRB• VME monitoring - Implement block writing to DDR3• Use block data transfer on VME side (single access ~2MB/s, block

access ~80MB/s)• Not done

• Cover FW not tested yet due to other priorities

• FE intf version 9 problem with control link

• Majority of work on this FPGA, number of bug fixes

• Work on monitoring improvement postponed

Now

Choke and error• Straw readout system is designed to withstand high data flows without crashes• Only 3 places where the incoming data rate is higher then outgoing

• Cover• New FW should fulfil the requirement for covers close to the beam

• Average 600kHz/straw• Spikes handled with big derandomizer FIFO

• In the case of FIFO full• Event is shed • Problem registered and both sent to DAQ and ready for monitoring• No request for choke/error

• If particle rate higher then designed/expected in TD • Increase link speed (20% improvement)• Add another data link (80% improvement)

• Trigger matching• Incoming theoretical rate of measurements 120MHz (4 x 2.5Gb/s links)• Big FIFOs on all 4 links to handle spikes in data rate• If FIFO full

• Problem registered for monitoring• Possible to handle ~50MHz (possibility to improve to 80MHz)• Expected from TD ~30Mhz• Possible to generate choke at this place, but deemed not needed

Choke and error• Only 3 places where the incoming data rate is higher then outgoing (cont)

• Ethernet links• 4 event building buffers• Buffers multiplexed to 2 Ethernet links (1 at current FW)• If no more event buffers free for event building

• Backpressure to trigger matching output FIFO• Generate choke at almost full

• Normally should not happen as 2 links should cover 140% nominal data flow

• We have not considered generation of ‘error’ till now• Can be generated in case of ‘catastrophic’ scenario

• Ethernet link down• ???

Known problems• SRB• Duplicated/missing words

• Small number• Frontend• Possibly fixed with new FW on cover and SRB(FE_intf)

• T0 adjustment can not be used now• Need to fix control link• Expected to fix also +/- 1 clock cycle T0 movement

• Start of high speed links sometimes difficult• Recheck start procedure

• MEP factor > 1 does not work

Conclusion and plans • Major improvement since Prague meeting

• Straw DAQ is stable and reliable• Tested ~100% efficiency at 40% beam intensity• Straw readout is designed to work in stable condition with 100% beam intensity• Current FW allows to work at 100% efficiency up to ~70% of beam intensity

• For 100% efficiency at 100% intensity we need to• Test new cover FW• Investigate and fix control link in Fe_intf v9

• V9 contains number of speed improvements• Fix MEP packing• Add second Ethernet link

• Other • Test new cover FW

• New features for monitoring and data quality control• Investigate and fix control link in Fe_intf v9

• Expected to fix T0 move at run start • Improvement of raw and trigger-matched data VME monitoring