Problems with Muon System

N. Bondar O.Maev21.09.2015

Muon detector partsMuon system parts:

1.Muon chambers-Proportional chambers-GEM chambers

2.HV systems (PNPI, CAEN, HV-GEM)3.Readout

-Front-end-ODE-TELL1

4.Low Volage5.Gas system6.Slow control

-CAN BUS for Front-end, ODE, Low Voltage-Special BUS for PNPI HV system

7.Safety system

FOUND PROBLEMS DURING RUN2

1. Muon Chambers : Problems Comments

-Proportional chambers HV trip Much less than in RUN1 (1)

-GEM chambers Strange noise Not affected the data

2. HV systems (PNPI, CAEN, HV-GEM) Infant mortality MB #104 C-side ch02. (2)

3. Readout

-Front-end Misconfiguration Q4M3B17- discon. From today - on M2C ½ 15B and whole 16B (dead ELMB00 in SB15)

-ODE Uncompl. EventBX crossing

Q4M5R4, M4Q34-fixed, M1Q2, M4Q34-rarely Q3M3R21 -rarely

-TELL1 Uncompl. Event Changed but it was not a source of problem.

4. Low Voltage Monitoring error MARATON: Q1M4/5 FEE ch8 (U), Q1 M4/5 DAQ ch7 (I)

5. Gas system

6. Slow control:

-CAN BUS : FE, ODE, LV, GS, SS FE communication Q3 M2 – Does not work (3)

-Special BUS: PNPI HV

7. Safety system Temp. sensors C-M1 error with connection, C-M45 – Systec error

Problem analysis

1. HV trip We still have some chamber trips. Fortunately now the new SW controls the

situation. Average we have 2-3 tripped gas gaps per day.

2. Infant mortality of HV regulators HV system was doubled during LS1, it was installed 56 new RDB’s and 8 MB’s. There are new HV regulators in all MB modules and in all new

RDB modules. Unfortunately we still have some failure of the regulators. The last time was in August 14 with MB.

FE control communicationThe most complex problem is the communication with the Frontend.That is not a new. The main recipe was power cycling of involved Service board and Systec. In the beginning of Run2 this problem was fixed on A-side by replacing many ELMB boards and Service board. Now A-side working very good.

Unfortunately for C-side we have no spare Service boards and some problems were not fixed completely and over time, the problem is growing up. Despite the fact that there were significantly improved the user's programs, the main and efficient tool for recovering communication is the same - power cycling the Service board and the Systec . It is obvious that this procedure consumes the lifetime of modules especially if done repeatedly. System is degradating. Now in the recovery procedure already included restart the OPC server because of some errors can crash the OPC server.

Problem analysis

What we have now:- On C-side we can see (on the Systec module) many-many errors on all

CAN lines (no errors on A-side)- I believe that references to the poor performance of the OPC server now

is not correct.- First we need restore the stable operation of the service modules. - Actually we had few time misconfiguration of the FE on C-side (one

sample on page 3).- System may be misconfigured any time because CAN BUSes transfer

more errors than signals .- Can bus 2 for crate Q3M2 can be workable only few second and disable

– now the system configure is a trick. - This night Oleg spent about 11 hour to recover system. This time we

have lost 3 hours of events. -

Problem analysis

- Our experiences after tonight:- The Q3V23 Systec LED display shows permanent red light

blinking (errors) and some times green – line transfer.- The green LED ( of CAN cable 2) from time to time is

blinking even there is no communication with the computer.

- We tested this with two other Systecs. I suppose that exactly this channel provoke the OPC crash.

- For me is obvious that we need as soon as possible replace few (3 or 4 ) Service boards and possible few ELMB

- More options – replace PDM board (we need spare) and may be crate

Problem analysis