Straw readout statusStatus and plans in Prague compared with situation now

Choke and errorConclusions and plans

Start of 2015 run (Prague)• Expected smooth start due to

• Experience 2014• Number of improvements

• problems with 2015 startup• SOB, EOB delay• trig latency obsolete data cleaning

• Stable data taking since couple of weeks• Observed problems

• Duplicated words• Missing (hidden by above?) trailing edges• Timing adjustment for T0• Some covers moving +/- 1 clock cycle at the start of

run• 1 SRB front-end interface chip not functional after

incident• 1 HV line not stable

• Some problems solved• Data corruption very complex problem

• Extra words turned out to be also triplicated• Nonsense data at high beam intensity• Padding hiding data or missing• Etc.

• Data corruption• Padding fixed in Ethernet intf• Duplicated words fixed in Ethernet intf• Triplicated and extra words fixed in trigger-

matching• Still small amount of data with duplicated

words coming possibly from frontend• ~10% of trailing edges missing?

Now

Prague meeting

Now – October 2015

Straw readout consolidation/improvement since December 2014 (Prague)• Improvements of existing FW and bug

fixes• Startup of 2.5Gb/s links – added extra ‘0’

to the alignment pattern• Stability of 2.5Gb/s links – ALTERA bug fix• Startup of 400Mb/s links to cover, link

completely redone• Re-synchronizing FIFOs added to links

outputs• Increasing the speed of data merging to

prepare for high intensity beam• Coarse T0 setting (0->255) for the whole

board and fine T0 setting (0->7) for every cover

• Startup of 2.5Gb/s links• Still a problem on some links• Possibly ‘link back’ alignement

• Stability of links OK

• Fine T0 not usable now because of control link corruption (version 9)

Now

Improvements for full intensity beam (Prague)• Cover new FW ready for test

• Faster data multiplexing with look-ahead• bigger derandomization buffer• Data transmission up to 13 MHz measurements (leading/trailing

edges)• E.g. 10 straws in ‘hot’ place with ~600kHz average rate

• FE_INTF FPGA on SRB• Fast data multiplexing • Bigger derandomization buffer• FW ready for test together with cover T0 adjustment

• EVENT_MGR FPGA on SRB• Include second Ethernet interface• Monitoring of trigger matched data (DDR3 memory readable

through VME - Implement block writing to DDR3• Not done

• VME_MGR FPGA on SRB• VME monitoring - Implement block writing to DDR3• Use block data transfer on VME side (single access ~2MB/s, block

access ~80MB/s)• Not done

• Cover FW not tested yet due to other priorities

• FE intf version 9 problem with control link

• Majority of work on this FPGA, number of bug fixes

• Work on monitoring improvement postponed

Now

Choke and error• Straw readout system is designed to withstand high data flows without crashes• Only 3 places where the incoming data rate is higher then outgoing

• Cover• New FW should fulfil the requirement for covers close to the beam

• Average 600kHz/straw• Spikes handled with big derandomizer FIFO

• In the case of FIFO full• Event is shed • Problem registered and both sent to DAQ and ready for monitoring• No request for choke/error

• If particle rate higher then designed/expected in TD • Increase link speed (20% improvement)• Add another data link (80% improvement)

• Trigger matching• Incoming theoretical rate of measurements 120MHz (4 x 2.5Gb/s links)• Big FIFOs on all 4 links to handle spikes in data rate• If FIFO full

• Problem registered for monitoring• Possible to handle ~50MHz (possibility to improve to 80MHz)• Expected from TD ~30Mhz• Possible to generate choke at this place, but deemed not needed

Choke and error• Only 3 places where the incoming data rate is higher then outgoing (cont)

• Ethernet links• 4 event building buffers• Buffers multiplexed to 2 Ethernet links (1 at current FW)• If no more event buffers free for event building

• Backpressure to trigger matching output FIFO• Generate choke at almost full

• Normally should not happen as 2 links should cover 140% nominal data flow

• We have not considered generation of ‘error’ till now• Can be generated in case of ‘catastrophic’ scenario

• Ethernet link down• ???

Known problems• SRB• Duplicated/missing words

• Small number• Frontend• Possibly fixed with new FW on cover and SRB(FE_intf)

• T0 adjustment can not be used now• Need to fix control link• Expected to fix also +/- 1 clock cycle T0 movement

• Start of high speed links sometimes difficult• Recheck start procedure

• MEP factor > 1 does not work

Conclusion and plans • Major improvement since Prague meeting

• Straw DAQ is stable and reliable• Tested ~100% efficiency at 40% beam intensity• Straw readout is designed to work in stable condition with 100% beam intensity• Current FW allows to work at 100% efficiency up to ~70% of beam intensity

• For 100% efficiency at 100% intensity we need to• Test new cover FW• Investigate and fix control link in Fe_intf v9

• V9 contains number of speed improvements• Fix MEP packing• Add second Ethernet link

• Other • Test new cover FW

• New features for monitoring and data quality control• Investigate and fix control link in Fe_intf v9

• Expected to fix T0 move at run start • Improvement of raw and trigger-matched data VME monitoring