1. DT Shifter Training Infrastructure A. Benvenuti, M.Giunta 07/07/2010

2. General Info The Technical Shifter should call DT Infrastructure Expert (165507) 79930 given as a backup If you need an expert you can call: 163651 (Alberto) 162355 (Marina or Daniele)

3. DT Safety Considerations Overheating: - Inside the detector (MiniCrates) due to Cooling Plant failure (MUON Circuit), affects all MC, or obstructed flow-limiter, affects two MC in adjacent sectors. - Electronic modules inside Tower or USC Racks, due to Cooling Plant failure (Racks Circuit) that affects all racks, or a turbine failure that affects only the modules inside one rack. The most dangerous case is a turbine failure in a tower rack containing Sector Collector crates. Water Leaks: - Are detected by “leak detection” cables inside the detector and tower racks. Major leaks can also induce HV trips in one or more chambers.  DT expert (165507, backup: 79930) should be called for any error induced by Overheating or Water leaks Gas Failure: - Not critical, the DT chambers can be operated for ~ day without gas flow

4. DT States ON: - LV ON - HV ON: wires +3600V, strips +1800V, cathodes -1200V Standby: - LV ON - HV ON: wires +1900V, strips +1800V, cathodes -1200V Dormant: - LV ON - Sector Collector mezzanine boards (opto couplers) OFF - HV ON: wires +1900V, strips +1800V, cathodes -1200V OFF: - LV OFF - HV OFF  The DT should be in Standby mode when the Magnet is ramping.

5. http://cmsonline.cern.ch/portal/page/portal/CMS%20online%20system/DCS/Automation?_piref815_847251_815_847248_ 847248.strutsAction=%2FsubmitDetector.do%3Bjsessionid%3D898af9ec30d87aa1ed82970647a6b953ea8018662e18.e34Lb hiLbx8Nay0LbxqRbN8Naxb0n6jAmljGr5XDqQLvpAe%3FSelectedDetector%3DDT DT State vs Machine/Beam State

6. How to connect to PVSS from home (I) RemoteDesktop connection to the Terminal Server: - Connect to cerntscms01 as mudcs (usual pwd) - tunnel on lxplus → Source port: 8881, Destination: cerntscms01.cern.ch:3389 - RemoteDesktop connects to→ localhost:8881 For MAC Users: you can use CORD

7. How to connect to PVSS from home (II)

8. DT Detector Panel 1) 2) 7) 4) 5) 6) 3) Error conditions flagged by Red color on tabs and audio alarm (to be implemented): 1) Cooling 2) UX / USC Turbines, Smoke alarms 3) MC Temperatures/Communication Error 4) Water Leaks 5) LV Temperatures 6) Gas System 7)More than 1 MC LV error 8)HV Problem 6) 1) 5) 8)

9. Yoke Circuit Flow Error No action required Minicrate (MUON) Flow Error Cooling Plant Problem  Call DT Expert (165507) MC will switch OFF after 15’ Rack Circuit Flow Error Cooling Plant Problem  Call DT Expert (165507) LV, HV switch OFF after 20’ Supply T > 18°C Cooling Plant Problem  Call DT Expert (165507) DT-DCS shutoff in 30’ Time windows can be reset from “Reset Timeouts” button Cooling Problem

10. Tell the Technical Shifter to contact CCC and find out the time estimate to fix the problem Call immediately: 163651 (Alberto) or 162355 (Marina or Daniele)

11. UXC Rack Problem

12. Turbine is OFF (current ~ 0.4A)  Call DT Expert (165507) High Voltage USC: HV OFF in corresponding wheels within 300 seconds. Time window can be reset by apposite button LV USC S4F02: No Action Required USC Rack Problem

13. Check that affected components have been switched OFF Individual modules can be switched OFF by clicking the “ON” button Entire Wheel can be switched ON/OFF from the “Wheel ON-OFF” panel The LV Panel can be reached from the Detector Panel clicking the “MAO” button Caution: Toggle Buttons if pushed will change state MAO Panel actions described in page 6

14. A drop of water in one of the leak detection cables will trigger an alarm and the corresponding buttons will turn red.  Call DT Expert (165507)  No further action required from shifter Water Leak

15. A3100 Modules power the SC crates and have the largest load hence the high T. In Standby mode the SC Mezzanines will be off and the Temperatures will be lower. If the Temperature is above 40°C in several modules there is a cooling problem in the “Rack circuit” Time Histories are available from the buttons LV Boards Temperature Problem

16. Gas Alarm due to: Stopped gas flow  DT Chambers are safe without gas circulation for ~ 1day.  Do NOT call expert during the night Communication Error  Data are not stored in DataBase  Call DT Expert (165507) from 08:00 to 23:00 Button becomes RED when Communication is lost Gas Problem

17. HV Problem During the day: Call DT Expert (165507) During the night: Open ‘HV Layers Info’ panel: if you have >2 entries in the ‘Layers in STANDALONE’ column Or >5 entries in the ‘Layers not ON’ column Call DT Expert (165507)

18. All Previous Problems Call immediately: 163651 (Alberto) or 162355 (Marina or Daniele)

19. YB+1YB+2 MiniCrates Temperature If Temperature exceeds 40°C for 1 or 2 MC call Expert and it will help you to Switch OFF the MC If high T appears in several MC it is a cooling problem To Reach this panel MiniCrate Temperature Problem

20. In one MiniCrate - Switch OFF the MiniCrate (see next slide) In several MiniCrates - Probable Cooling Problem, call immediately: 163651 (Alberto) or 162355 (Marina or Daniele)

21. How to switch off one Minicrate You can choose if doing it yourself or asking the Technical Shifter to do it If you want to do it: - ask the Technical Shifter to Release the partition (DT+, DT- or DT0) - Take(*) the partition - Perform the action - When you’re done Release(*) the partition - Tell the Technical Shifter to Take(*) back the partition (*) see next slide to learn how to Take and Release Give a SWITCH_OFF_LV command from the Chamber Panel

22. How to Take/Release a FsM node Click the open lock and then ‘Take’ How to Take a node How to Release a node Click the lock and then ‘Release’

23. MiniCrates Communication Error In case communication is lost between PVSS and the MC control program, the Wheel Banner will turn red and blinking and the chambers will turn orange. Click the “Reset” button on the wheel and send “MC_Reset” from the top node. The LV will be switched OFF after 30’.  Call DT Expert (165507) You must “take” the “DT_Detector” to do any action and “release” it once you are done

24. MiniCrate Communication Problem In one wheel Restart the CCB Server for that wheel, as explained in the DT Expert FAQ twiki: https://twiki.cern.ch/twiki/bin/view/CMS/DTexpertFAQ2009x07x27 https://twiki.cern.ch/twiki/bin/view/CMS/DTexpertFAQ2009x07x27 In ALL the wheels Call DT Field Manager

25. MiniCrate with Boards Off 2.00 Try to switch ON the MiniCrate (from DCS GUI -> Power All) once - If command fails or boards switch off again, call DT Field Manager If the boxes are not green it means the LV channels actually tripped; in this case call the DT Field Manager

26. HV and LV can be switched OFF with a fast but safe procedure from the “Emergency Stop” buttons

27. Automatic DCS/DSS Actions

28. Problem Granularity HV LV FailureFraction of DT affected 1 channel1 layer 1 A8771 chamber 1 A8761 sector 1 SY15274 or 8 sectors (in 1 or 2 wheels) 1 HV rack1 or 2 wheels FailureFraction of DT affected 1 channel (A3050 or A3009)1 chamber 1 A30501 or 2 chambers 1 A3009 or 1 Easy 3000S crate1 sector 1 MAO (except S12)1 or 2 sector (N.B. MAO S12 = 1 wheel/2 SC crates) 1 A31006 sectors/1 SC crate 1 LV Rack (except X2 Near)1 quadrant (N.B. X2 Near = 1 wheel/2 SC crates) 1 Branch Controller (A1676A)half wheel SY1527 in S4whole detector