1. Problems with Muon System N. Bondar O.Maev 21.09.2015

2. Muon detector parts Muon system parts: 1.Muon chambers -Proportional chambers -GEM chambers 2.HV systems (PNPI, CAEN, HV-GEM) 3.Readout -Front-end -ODE -TELL1 4.Low Volage 5.Gas system 6.Slow control -CAN BUS for Front-end, ODE, Low Voltage -Special BUS for PNPI HV system 7.Safety system

3. FOUND PROBLEMS DURING RUN2 1. Muon Chambers :ProblemsComments -Proportional chambers HV tripMuch less than in RUN1 (1) -GEM chambers Strange noiseNot affected the data 2. HV systems (PNPI, CAEN, HV-GEM) Infant mortalityMB #104 C-side ch02. (2) 3. Readout -Front-end MisconfigurationQ4M3B17- discon. From today - on M2C ½ 15B and whole 16B (dead ELMB00 in SB15) -ODE Uncompl. Event BX crossing Q4M5R4, M4Q34-fixed, M1Q2, M4Q34-rarely Q3M3R21 -rarely -TELL1 Uncompl. EventChanged but it was not a source of problem. 4. Low Voltage Monitoring error MARATON: Q1M4/5 FEE ch8 (U), Q1 M4/5 DAQ ch7 (I) 5. Gas system 6. Slow control: -CAN BUS : FE, ODE, LV, GS, SS FE communicationQ3 M2 – Does not work (3) -Special BUS: PNPI HV 7. Safety system Temp. sensorsC-M1 error with connection, C-M45 – Systec error

4. Problem analysis 1.HV trip We still have some chamber trips. Fortunately now the new SW controls the situation. Average we have 2-3 tripped gas gaps per day. 2.Infant mortality of HV regulators HV system was doubled during LS1, it was installed 56 new RDB’s and 8 MB’s. There are new HV regulators in all MB modules and in all new RDB modules. Unfortunately we still have some failure of the regulators. The last time was in August 14 with MB.

5. FE control communication The most complex problem is the communication with the Frontend. That is not a new. The main recipe was power cycling of involved Service board and Systec. In the beginning of Run2 this problem was fixed on A- side by replacing many ELMB boards and Service board. Now A-side working very good. Unfortunately for C-side we have no spare Service boards and some problems were not fixed completely and over time, the problem is growing up. Despite the fact that there were significantly improved the user's programs, the main and efficient tool for recovering communication is the same - power cycling the Service board and the Systec. It is obvious that this procedure consumes the lifetime of modules especially if done repeatedly. System is degradating. Now in the recovery procedure already included restart the OPC server because of some errors can crash the OPC server. Problem analysis

6. What we have now: -On C-side we can see (on the Systec module) many-many errors on all CAN lines (no errors on A-side) -I believe that references to the poor performance of the OPC server now is not correct. -First we need restore the stable operation of the service modules. -Actually we had few time misconfiguration of the FE on C-side (one sample on page 3). -System may be misconfigured any time because CAN BUSes transfer more errors than signals. -Can bus 2 for crate Q3M2 can be workable only few second and disable – now the system configure is a trick. -This night Oleg spent about 11 hour to recover system. This time we have lost 3 hours of events. - Problem analysis

7. -Our experiences after tonight: -The Q3V23 Systec LED display shows permanent red light blinking (errors) and some times green – line transfer. -The green LED ( of CAN cable 2) from time to time is blinking even there is no communication with the computer. -We tested this with two other Systecs. I suppose that exactly this channel provoke the OPC crash. -For me is obvious that we need as soon as possible replace few (3 or 4 ) Service boards and possible few ELMB -More options – replace PDM board (we need spare) and may be crate Problem analysis