1. 1 The Muon Shifter October 2009

2. 2 Forewords This pages are meant to initiate people who are not familiar with ECS (Experiment Control System), and Muon ECS in particular, to perform shifts in global data taking run. In this sense, information given here are very basic and in general refer to a non-expert use of the system: setting-up of the system for DAQ runs, first check of proper running conditions, first simple interventions in case of simple frequently encountered problems. This basic manual can also be a starting point for a richer documentation while the operator skill increases and the control system becomes more stable and complete. Please note also that the ECS system is still under development. Changes on the panels and even on the structure is still possible. This pages should not be meant as a user manual of a delivered project/system, but just to gain a first idea on system operation. Muon ECS (basics of the) basics The LHCb Experiment Control System (ECS) is based on the PVSS framework. You can find some documentation on PVSS and LHCb ECS at page http://lhcb-online.web.cern.ch/lhcb-online/ecs/default.htm and in particular the tutorial http://lhcb-online.web.cern.ch/lhcb-online/ecs/powerpoint/LHCb_ECS_Tutorial.ppt which we warmly suggest you to read.http://lhcb-online.web.cern.ch/lhcb-online/ecs/default.htmhttp://lhcb-online.web.cern.ch/lhcb-online/ecs/powerpoint/LHCb_ECS_Tutorial.ppt The ECS is based on so-called PVSS projects running on dedicated online PCs. Each project runs on independent partitions. The MUON ECS has several project types, controlling: FEB (Front End Boards, i. e. CARDIAC via the Service Boards) ODE (Off Detector Electronics Boards) LV and HV Gas, Temperature, RAC power Tell1 FSM (Finite State Machine & Hierarchy) projects The last projects do not directly control any hardware but the hierarchical structure of the ECS itself. The FSM allows the ECS commands to circulate from top to bottom of the structure, thus making possible the communications among the complete system. Introduction

3. 3 The Muon Detector DAQ and Control electronics chain DIALOG is configured and read back by SB (Service Board). Service Boards are placed in an off detector rack. The same rack houses the PDM board (Pulse Distribution Module): it generates pulses in a chosen phase with the LHC machine clock for the time alignment of the muon system. Front-end Boards (CARDIAC): 2 ASIC -CARIOCA (ASD chip) -DIALOG (logic, timing and monitoring) (logical channels) The IB (Intermediate Board) make logical combinations of the front-end channels (logical channels) where the combination is not possible at DIALOG level The ODE (Off Detector Electronic Board) receive signals from the FEBs (or IBs), synchronize and displace them to the L0 trigger. Upon reception of a trigger signal data are sent to the TELL1 board and from here to the DAQ (next slide for picture)

4. 4 ODE L0 Muon Trigger processors Muon TELL1 DAQ ODIN The ODIN board distributes all orbit signals (clock, trigger, L0 reset,…) to the ODEs, Muon TELL1s and Trigger processors (to all boards that are synchronuos and have the TTCrx chip) The Muon Detector DAQ and Control electronics chain (II)

5. 5 Muon detector back side view: each station M1-M5 is divided in four quadrant Q1-Q4 z Interaction point C sideA side Q4 Q3 Q1 Q2 M2 M3 M4 M5 M1

6. 6 The structure of ECS consists of 3 types of objects: Device Unit (DU), Control Unit (CU) and Logical Unit (LU). DU is an object controlling a hardware component (e.g. a CARDIAC board) CU is just a hierarchical object, passing commands and states on a hierarchical node LU is useful to group different DU together and is rarely used in our case The hierarchical structure allows also including and excluding portions of the system. Following the general LHCb structure, also the Muon hierarchy is organized in two independent and totally symmetrical parts, MUONA and MUONC, controlled by a higher node, MUON. The deeper the hierarchical level, the wider the understanding needed of system and hardware details. However, in “normal” conditions it should not be necessary to go further than the MUONA/C level. EACH PVSS PROJECT RUNS ON AN ONLINE PC. On Control Room PCs you will find shortcuts to the online PCs where really the projects run The Muon ECS

7. 7 PVSS Tree for Muon Detector MUON MUONAMUONC MUONC_DAQMUONC_DCS MUON_DCS_LV MUONC_HV The Green bubbles are Control Unit (CU) of the: PVSS project MUSIDES PC: mudaqa01w Inside this CU there is the hierarchy of the Muon detector electronics: Chambers/ODE/Tell1 PVSS project MUDAQC PC: muecsc01w Inside this CU there is the hierarchy of the Muon detector Low Voltage PVSS project MUDCSLVC PC: mulvc01w Inside this CU there is the hierarchy of the Muon detector High Voltage PVSS project MUHVC01 (PNPI - R3 R4) PC: muhvc01 PVSS project MUHVC02 (CAEN – R1 R2) PC: muhvc02 An example for the C side

8. 8 PVSS Tree of Muon Detector MUONC_DAQ MUONC_DAQ_Q3 MUONC_DAQ_Q3_M1CMUONC_DAQ_Q3_M23C MUONC_DAQ_Q3_M23C_M2 MUONC_DAQ_Q3_M23C_M3 MUONC_DAQ_Q3_M23C_CRC MUONC_DAQ_Q3M45C MUONC_DAQ_Q4MUONC_DAQL1 Inside this bubble there is the hierarchy of the DAQ tell1 boards witch take the data coming from the ODEs: PVSS project MUDAQL1 PC: mudaq01 Inside this bubble there is the SB/PDM crate for station M2&M3 Q3&Q4 PVSS project MUDAQC23 PC: mudaqc03w An example for the C side

9. 9 Chambers An example for the C side PVSS Tree for Muon Detector HU: Hardware Unit is made by One ODE and all the Chambers connected to it Chambers ---------------------- PVSS project MUDAQC23 PC: mudaqc03w ODE --------------------------- PVSS project MUODEC01 PC: muodec01w Hardware Unit ODE Chambers

10. 10 UseComp name PVSS IdProject Name Feb 2&3/C MUDAQC03w 260 MUDAQC23 Feb 4&5/C MUDAQC04w 261 MUDAQC45 Feb 1/C MUDAQC01w 262 MUDAQC1 Feb 1/A MUDAQA01w 263 MUDAQA1 Feb 2&3/A MUDAQA03w 264 MUDAQA23 Feb 4&5/A MUDAQA04w 265 MUDAQA45 FSM/C MUECSC01W 279 MUDAQC FSM/A MUECSA01W 280 MUDAQA LV/C MULVC01w 210 MUDCSLVC LV/A MULVA01w 211 MUDCSLVA HV/C MUHVC01 212 MUHVC01 HV/C MUHVC02w 213 MUHVC02 HV/A MUHVA01 214 MUHVA01 HV/A MUHVA02w 215 MUHVA02 ODEQ3/C MUODEC01w 266 MUODEC01 ODEQ4/C MUODEC02w 267 MUODEC02 ODEQ1/A MUODEA01w 268 MUODEA01 ODEQ2/A MUODEA02w 269 MUODEA02 T&P MUDCS01w 270 MUDCSSEN TELL1mudaq01216MUDAQL1 M5 : mutella01 (Q1/Q2) mutellc01 (Q4/Q3) M4 : mutella02 (Q1/Q2) mutellc02 (Q4/Q3) M3 : mutella03 (Q1/Q2) mutellc03 (Q4/Q3) M2 : mutella04 (Q1) mutellc04 (Q4) M2:mutella05 (Q2) mutellc05 (Q3) M1:mutella06 (Q1) mutellc06 (Q4) M1:mutella07 (Q2) mutellc07 (Q4) MUON TOP mudaqa01w MUSIDES MUON on-line PCs and their use Comments: 1.Access to this machines to start/stop/operate the PVSS projects should normally not be needed but for experts. 2.As a general rule, PC names ending with W are Windows-based PCS, Linux otherwise. 3.Some of the reference experts are: D. Pinci for FEBs, S. Cadeddu for FSM, M. Lenzi for LV and HV, M. Carletti for ODE, G. Graziani for Tell1 and Muon Monitor, C. Deplano. Here a complete list of the online PCs and the relative projects running on them is given

11. 11 ECS - PVSS world 19 PCs 20 PVSS projects MUON A SIDE C SIDE ODE CMB TELL1 DCS LV HV ODE 5

12. 12 Panels and States ECS operation is based on control panels, giving information on system status and allowing intervention of the operator to trigger a status change. Although intuitive, it is important to give some explanations on the use of the panel buttons. { Sub-System buttons: clicking with the left mouse button opens the corresponding CU Sub-System state, visualizing the current state. Clicking with the left mouse button gives the list of possible actions which can be done from it. MUONA CU Panel Lock button: the system can be accessed/controlled by different parts of ECS. This button says who is presently TAKING the system and its inclusion/exclusion in the hierarchy. This info can be gained by placing the mouse tip on the lock. Before operating on a given system/subsystem you need to TAKE it (if possible). During Global data taking “your” system must be controlled by the global manager and the muon operator must release it properly (see next pages). Using the left mouse button it is possible to change the lock status. Presently loaded recipe. A recipe is the collection of all the configuration parameters/variables needed for a specific data-taking run. Usually the shifter should not choose one recipe rather than another (it is already loaded). If further information is needed on recipe handling, for the moment you have to refer to an expert. NEVER use this button to dismiss the panel ALWAYS USE the Close button

13. 13 1. How to... Prepare the MUON system for global DAQ run when everything is ok. In normal conditions, the PVSS projects (FEBs, ODE, FSM) are up and running. This condition should be reached automatically at each power-on of the machines where the projects are installed (see list on next page). At the moment (June 08), this is implemented only for the LV control projects, but will be done soon also for the FEBs and the ODEs. Being in this condition, the steps to set-up the system for DAQ are the following: i.On any computer of the on-line cluster*, go to directory G:\online\ecs\Shortcuts38\MUON ii.Launch (double click) the two shortcuts MUSIDES_UI_DEN and MUSIDES_UI_FSM. This visualizes the two control windows for DEN (Device Editor Navigator) and FSM (Finite State Machine) Hierarchy (see next page). Launching the shortcuts does not start the DEN and FSM processes (which are already running within the PVSS project), but just allows using the corresponding control windows from your screen. (*you need an account within the on-line cluster: ask Niko Neufeld if you haven’t any.)

14. 14 FSM panel The FSM panel you should have opened allows visualizing the status of the full MUON hierarchy and opening panels related to any hierarchical level. Concerning the DU/CU status, the color code is as follows: grey for DEAD/NOT INCLUDED yellow for NOT READY orange for UNKNOWN/NOT ALLOCATED red for ERROR blue forREADY or ACTIVE (TFC case) green forRUNNING Now, placing the cursor tip on the MUON box and clicking the RIGHT mouse button, opens the MUON CU, which represents the top level of our hierarchy, see next page(s). This box accesses the TELL1 CU (refer to page 15)

15. 15 MUON CU panel In order to allow the global manager driving the MUON System, the MUON must be NOT_READY and its subsystems NOT_ALLOCATED. All the locks should be grey and open, without any slash on them (which means Not Included). The first operation is TAKING the MUON System under your control. Click (LEFT mouse button) on the lock and select TAKE. The lock goes green and closed. A panel automatically pops up, giving the list of LockedOut nodes and of CU which were not possible to take (more details on page 12). Now OPEN (double click) the two sides (A & C) Although this condition appears to be verified in the shown panel, it is equally necessary to check the MUONA and MUONC CU. Checking deeper levels is not required to a shifter.

16. 16 MUONA CU panel If everything is correct, the panel appears in the configuration shown. In this conditions, you can release the MUON system in order to make it possible the control by the global console (go to next page) (The MUONC panel has similar face)

17. 17 MUON CU panel In order to release the MUON system, click on the green lock and choose ReleaseAll in the pop-up menu (the Release button releases this level of CU but not those below it). The MUON status goes to NOT_READY and the system is free to be used by the global console. Note: Case of a subsystem taken by someone else. Placing the mouse on, you obtain info on owner and status. In Global DAQ, the global manager decides what is to be included/excluded. You have to take care only of MUON, putting it in regular shape.

18. 18 2. How to... Prepare the MUON system for global DAQ run when something is not the right way. Suppose that opening the MUONA panel, a different status is shown (see figure): Some subsystems appear to be locked out and the TFC is ACTIVE, while the right status is NOT_ALLOCATED for the TFC and NOT_READY for the others i.Release MUON (ReleaseAll) ii.TAKE MUONA (where the problem was found) iii.ACTIVE  Stop run  READY iv.READY  RESET v.NOT-READY  DEALLOCATE vi.ReleaseAll (in the lock) This should take again the MUONA in the right shape In such a case, these are the steps to follow: {

19. 19 DEN panel The DEN panel (you got from page 7) is to be used in case of DEAD states: click on the Start/Restart All button for the relevant stage of the hierarchy (use the FSM tab). This activate the FSM only on the relevant stage and should cure the DEAD state. DEAD states When you TAKE a System, an Action Report appears. In some cases a subsystem cannot be taken because its state is DEAD (i.e. The FSM cannot communicate with it, or the FSM is blocked there)

20. 20 TELL1 CCPC restart (1) When the the system is being reset or configured, it may happen that the system enter an ERROR state. This state propagates from the DU/CU which originates it, up to the top level of the hierarchy (MUON in our case). Frequently this is caused by the Tell1 CCard, needing to be restarted (see next page). (see next slide also to know how to get to the MUONA(C)_DAQL1 panel). Here “ERROR” appears

21. 21 1.Descend the hierachy starting from the MUON CU panel: MUON  MUONA (e.g.)  MUONA_DAQ  MUONA_DAQL1(see previous slide). Alternatively, you can access the Tell1 panel directly from the FSM (see page 8). 2.Spot the Tell1 which is in error and identify the corresponding CCPC (see subsystems in the panel and the given correspondance table). 3.Log in on the CCPC using your user/pwd and start the ccserver (using putty, for example). 4.Type the command “sudo service ccserv restart”. You should obtain a double ok. If not, try to repeat the command. This restarts the ccserver and should solve the ERROR Side A PC TELL1 mutella01 MUTELLA_M5Q12 mutella02 MUTELLA_M4Q12 mutella03 MUTELLA_M3Q12 mutella04 MUTELLA_M2Q2 mutella05 MUTELLA_M2Q1 mutella06 MUTELLA_M1Q1 Mutella07 MUTELLA_M1Q2 Side C PC TELL1 mutellc01 MUTELLC_M5Q34 mutellc02 MUTELLC_M4Q34 mutellc03 MUTELLC_M3Q34 mutellc04 MUTELLC_M2Q4 mutellc05 MUTELLC_M2Q3 mutellc06 MUTELLC_M1Q4 mutellc07 MUTELLC_M1Q3 TELL1 CCPC restart (2) Here is how to (possibly) cure a TELL1 ERROR state

22. 22 Low Voltage panels (2) Q1 panel Take before And release all when finished

23. 23 Low Voltage panels The LV domain is reachable from the MUON panel, from the MUONA_DCS and MUONC_DCS hierarchy. Click directly on the relevant CU (RIGHT mouse button), or start from the top level clicking on MUONA_DCS on the FSM panel. The corresponding panel appears. From this panel you can descend the hierarchy across quadrants and stations reaching the LV controls (see next slides)

24. 24 Q1M23A panel: DAQ tab At this level, the different sub-systems of LV are visible and accessible Act only on the FSM buttons to switch on/off the sub-systems. This part of the panel should be used only to check the effects of the action, not to cause them The DAQ tab refers to ODE, SB & PDM, IB crates

25. 25 The FEE tab refers to the CARDIAC boards Q1M23A panel: FEE tab

26. 26 Starting the MuonMoni Presenter The better way to check the global status of the detector is looking at the data quality by means of the online Presenter. To start the Presenter: 1.Launch Exceed on your PC. There is a small problem with the new Exceed: the default configuration settings are re-set by the installer. Host access is now per default *DISABLED*. To change this open Exceed -> Tools -> Configuration -> Security and remove the access control. 2.Log on mudaq01 (*) 3.type:. /group/muon/scripts/lhcbsetup.sh 4.set local display: export DISPLAY=pcname:0.0 (**) 5.run presenter: /group/online/presenter/run_presenter.sh (*) Log-in using putty: - session: mudaq01 - connection SSH X11 (enable x11 forwarding) open and login (**) PC name : see label glued on the monitor See the twiki page https://twiki.cern.ch/twiki/bin/view/LHCb/MuonDaqMonitoring about the use of the Presenterhttps://twiki.cern.ch/twiki/bin/view/LHCb/MuonDaqMonitoring

27. 27 How open the PVSS panels for a linux PC For example: tell1 PC The PC is: mudaq01 (login with Putty) Command line to open the “gedi” panel: PVSS00ui –proj MUDAQL1 –m gedi On “gedi” go: - Module - Vision - localdisk/PVSS/MUDAQL1/panel - /grup/muon/fwComponent_MUDAQL1/panel To open the “DEN” panel - select the folder fwDeviceEditorNavigator - open fwDeviceEditorNavigator.pnl

28. 28 How create or upload a recipe for CMB 1.Go into MUON_DAQ panel 2.Create a new recipe Select the parameters needed (Thresholds, Masks, …) Select the detector regions/side Push “set the configuration to region …” Check “re-load timing result” if the timing parameter have been changed write the recipe name Push “Create Recipe” Wait until the process-bar arrive at 100% 3.Upload an existing recipe Select the detector region/side Push “set the configuration to region …” Select a recipe from the “Recipe List” Push “upload Recipe” Wait until the process-bar arrive at 100%

29. 29 How create recipe for ODE starting from timing test file 1. Configure the ODE starting from the timing file 1.Go into the HU you need from the Musides shortcut and reset the ODE board 2.Push “go to expert” and “continue” 3.Into the tab “commands” push “Load HW Conf File” 1.Select the file you want write into the hardware 2.Save a PULSING recipe 1.Go into the corresponding ODE PC 2.Open the ODE panel you want to change from the DEN 3.Push “recipes…” 4.Select the recipe you want create or overwrite (for example PULSING ) 5.Create the recipe “Create from hardware” 6.Select “PULSING” recipe and push “edit” 1.Check that all “GOLCTRL” value are 0x00 1.If not: push “edit” into “table Mode” 2.Write 0x00 for all the column “GOLCTRL” 3.Push “enter” into the keyboard: the changes you made became yellow 4.Push “save” and “close” 3.Save a PHYSICS recipe 1.Go into the corresponding ODE PC 2.Open the ODE panel you want to change from the DEN 3.Push “recipes…” 4.Select “PHYSICS” recipe and push “edit” 5.Take note of the value of the TTCrx registers “FD1” and “CD” and “close” 6.Create the recipe “Create from hardware” 7.Select “PHYSICS” recipe and push “edit” 1.push “edit” into “table Mode” 2.Wrote the value you have toke note for TTCrx “FD1” and “CD” registers 3.Push “enter” into the keyboard: the changes you made became yellow 4.Push “save” and “close”

30. 30 Muon Chambers In the Cavern: Chambers/CARDIAC SB/PDM IB ODE HV distributor LV Marathon SB/PDM ODE In B1: LV distributor (RCM/ACDC) In D3: Tell1 boards for DAQ and L0 trigger In D2: PCs with PVSS projects and SYSTEC witch connect ECS signals from SB/PDM/ODE to the PC) IB