DT Hardware Status and Monitoring A.Benvenuti INFN Bologna Muon Barrel Workshop 28/09/2010

Embed Size (px)

Text of DT Hardware Status and Monitoring A.Benvenuti INFN Bologna Muon Barrel Workshop 28/09/2010

  • Slide 1

DT Hardware Status and Monitoring A.Benvenuti INFN Bologna Muon Barrel Workshop 28/09/2010 Slide 2 A few items: High Voltage Low Voltage (connectors) Gas Analysis VDCs Monitoring/Alarms UX Maintenance Plans (Long Term) Out of ISR Slide 3 HV Problems Summary No New Problems since the end of February but, old YB+1 MB3 S07 came back The last real HV trip (YB+1 MB2S08 SL1) occurred on July 19 during the Magnet ramp down Slide 4 Problems Turning HV ON from OFF State During HV ON with warm ramp on 31/08 the following layers tripped: 1) YB+1 MB4S04 SL1 L2 w1 2) YB0 MB4 S10L SL1 L2 w1 3) YB-1 MB4 S10 SL3 L4 w1 4) YB-1 MB3 S03 SL3 L3 w1 5) YB+2 MB3 S11 SL2 L2 w0 Chambers 1 to 4 had exactly the same problem: At the first step (1200/1000/800) w1 was discharging without any current spikes in strips or cathodes. In some cases, HV spikes are seen for w1. The program attempts to bring the layers back to STANDBY values (1900/1800/1200). Note that no current spikes are seen in w1 once the STANDBY values are reached. The spikes seem to be due to a regulation problem in the A877. Similar behavior was seen in the past for: YB+1 MB4S04 SL1 L2 w1 on 26/02, 05/03 and 08/04 YB-1 MB4 S10 SL3 L4 w1 on 22/01 YB-1 MB3 S03 SL3 L3 w1 on 05/03 YB-1 MB4S10 Slide 5 Problems Turning HV ON from OFF State YB+2 MB3 S11 SL2 L2 got stuck with w0 in Over Current at the first step (1200/1000/800) with w0 current spikes without any spikes in strips or cathodes. The layer went up to nominal values without problems by lowering w0 to 900V and bringing w1, strips and cathodes to 2000/1800/1200. After that, no current spikes were seen in w0. Apparently, this channel does not regulate properly when the strips and wires are at low values. This problem was already seen on 05/03 and is reproducible (done before exchanging the A877) But it cannot be reproduced in B904 (Marina) Wait to replace the other modules until the T.stop Borsato and Marina to setup a meeting with CAEN YB+2 MB3 S11 SL2 Slide 6 A876 Problems: wire-Strip discharges in all layers and SL in YB0 MB2 S06 and YB+1 MB3S10 Not reproduced in B904 Cathodes discharges in all layers and SL in YB+2 MB3S09 and in YB+2 MB3S04 (both 12-15 July) A877 Problems: Apart for the new problem during HV ON from OFF a few modules had regulation problems In total 3 A877 were replaced since the June CMS week In addition one A877, fresh from repair, was used in UX but it had another feature YB+2 MB3S04YB+1 MB3S10 Slide 7 A3100 Power Distribution Measurements of the current distribution from the A3100 to the SC crates done in July showed large anomalies for YB0 Top and to a less extent for YB+1 Bottom. During the Technical stop at the end of August, Cristina identified the source of the problem in the Powerclaw connectors at the TIMBUS in the SC crates backplane. The Cure (Cristina, Davide, Ignacio) all PP75 connectors inside the crates were re-crimped following the same procedures as done in January and strain relieved. the contacts were cleaned and the powerclaw rotated 180 to improve the contact surface lubricant was applied to the surfaces to minimize fretting The power distribution is now quite uniform across all SC crates as a test, the lubricant was also applied to all powerclaw connectors in YB0 Slide 8 A3009 MC Vcc Connection Problem Spotted by Ignacio: Checked during UX access yesterday: Temperature = 36.7 C Crimps OK (impossible to crimp with B ON) lot of dust around the connectors Vcon normal after re-inserting connectors and blocking with cable tie wedge yet another flakey contact Slide 9 Vcon Saga Official list of known and under watch cases Found at 42C, Nothing unusual OK after usual plug-unplug fix MC switched OFF due to Vcon Occurred twice, second time after lubricant YB0 is unusually prone to Vcon problems Replacement of A3100 Power claw connectors with lugs has been postponed: No problems observed Ugly solution (only one Output) Not yet at the end of the story Slide 10 All Wheels1232411911 Gas System Good Performance, essentially no downtime Some improvements on gas analysis system Still some vagaries to be followed up Loss of ~ 1120 liters/hour (should be ~400) Flow equalization Gas Purity (CO 2 ) Online gas chromatograph to be operational before the end of the year From our side: Progress on PADC monitoring/upkeep (Daniel T.) Decision on DT operation at Absolute Pressure Slide 11 YB-1 Good stability since May, apart for a problem with calibrations in July/August (not shown) Plots identify chambers with higher O 2 contamination, not necessarily in the chambers Overall O 2 increase on 20/09 attributed to a CO 2 change, higher purity to be used in the future (RG) Overall O 2 increase for YB-2 on 06/09 due to a bad cell calibration Similar results for the other wheels Slide 12 YB+1 Slide 13 Good stability of O 2 content The 20ppm increase in 20/09 is attributed to the CO2 delivery Note the O 2 increase in YB-2 due to a problem with the analysis cell Slide 14 YB-1 Large spread in YB-2 Analysis cells were recalibrated by the manufacturer in August Later measurements are more uniform YB+1 has a lower H 2 O content than remaining YB Slide 15 YB+1 Slide 16 Slide 17 As a summary: Some progress in the Gas Analysis racks performance We have a list of the chambers with largest O2 contents It would be interesting to check the drift velocity for YB0 MB1S02 from the data Real progress only after the integration of the online gas chromatograph and corresponding H 2 O content Slide 18 VDC (in a nutshell) A troubled life Reference Mixture Common Supply But some hope for the future Thanks to Daniel and Lars for tending several VDCs through HV problems, supply failures, PC changes.. Taken at face value, there is ~2.5% difference in the Supply/reference Vdrift N 2 contamination in supply line should be checked with gas chromatograph (done/ongoing) At present N 2 level is not monitored, stabilized by fresh mix in supply line For the first time, we have a delivery date for the VDC system (thanks Hans): January 15 2011 Lot of work still to be done: DCS system UG infrastructures.. Slide 19 Unified Temperature Alarms, Displays (and more) Luca Ciano has set alarms thresholds on the extensive collection of thermal sensors that Franco installed on: - Thermal screens - MC Cooling manifolds - Sector 4 and 10 chambers (HV and Front-end side) - Sector Collector crates - LV Sector Collector cables Alarms (sms + e-mails) are sent to the usual suspects when temperature exceeds threshold by 1, 2 and 3C Note: Menu at right hand corner visible ONLY when touched with the mouse Slide 20 Time histories are available from Temps (inline) and Plotgen (background) Page : 1)Thermal screens and LV cables 2) MC Cooling manifolds 3)Sector Collector crates and S04 S10 chambers HV side 4)Sector 4 and 10 chambers Front- end side Slide 21 plus other functionalities requested by additional customers.. and a multitude of options in http://vmepcs2g16-14/production/errlog/ Slide 22 New DCS Features Added by Marina G. Percentage of Active Detector Sector Collector and DTTF, DDU Crates Monitoring Slide 23 Martin Gastal TIG Meting Medium and Long Term Planning We were asked to provide a list of the maintenance activities for the foreseeable future I provided Martin with the list of known DT problems Anton has given the list for the RPC Slide 24 A Snapshot of DT and RPC (Barrel) Maintenance Work (as of today) Thanks to Cristina and Gianni for providing the list for the MC maintenance. Only work requiring access to the detector is entered in the table Only RPC maintenance that requires moving the chambers is included Mistakes are possible but this is not the last version Slide 25 The Present Plan as Enshrined in TCM, TIG and TECOPA Meetings It is 25m x 18m (wall to wall) for the Muon Community we asked for 15m x 12m (but space is needed to move the chambers ) Recommended reading: The new tenant a play by Eugene Ionesco Slide 26 As usual, I did not date this but it goes back to January 2008 Slide 27 Thats all folks .