1. TPC shift guide1 04.09.2008 TPC shift guide Run period 10.09-31.12.2008 Updated on 04.09.2008

2. TPC shift guide2 04.09.2008 Foreword This shift guide is intended for the TPC shifter to be able to operate, run and monitor the TPC: Identify the relevant machines: aldaqacr34, the local TPC linux machine in the Working Room (TPC.501) alitpcon001, the Windows PVSS Operator Node alitpcwn008, the Busy Box linux machine Power on/off and configure the FEE for any specific run Configure and initialise the Busy Box Use the Local Trigger Unit Start/stop various standalone runs, or get ready for a global run Periodic pedestal and cal. pulser runs (DAs); laser runs Use the online monitor to look at event displays, and use AMORE Monitor the TPC: LV, FEC temperatures, cooling, ROC HV, Drift HV, gas, run status Use the electronics logbook Troubleshooting, list of expert

3. TPC shift guide3 04.09.2008 Connecting to the Operator Node 1)Make sure there is a remote desktop session to alitpcon001; if there is not one, a registered user does: –rdesktop –g1280x1024 –a16 –u username alitpcon001 –Enter your NICE username and password 2)Start  TPC.bat (or just TPC) 3)Click on the key icon (top left) to login as tpc tpc 4)You now have the PVSS panel –On the left there is the tree with all the panels –Bottom left, a synoptic with the colour-coded status of all systems 5)The top level node, TPC_DCS, will be the most extensively used  Note: if you can't type in a field, go to a linux terminal session, type something there, and go back to PVSS

4. TPC shift guide4 04.09.2008 Functionality of top node 1)TPC_DCS: Top node of the TPC FSM Tree Take the lock of the FSM Tree from here. If you need to CLOSE the main panel, first release the lock with the option RELEASE ALL FSM Commands: State: OFF Action: GO_STANDBY -> Switches ON LV PS State: XXX Action: CONFIGURE -> Configuration of the FEC according to the sent parameter (usually RESET and then COSMICS or pp) State: XXX Action: SOR or EOR -> Actual condition values are send to database for Shuttle 2)If the FSM has to be restarted (it also releases the lock) press 'Start/Restart All'. Wait until the 12 boxes (tpc_ui etc.) on the lower part of the panels are green. 3)Forced restart: Press 'Stop All' wait ~20 seconds, press 'KILL FSM' wait ~5 seconds, press 'Start/Restart All'

5. TPC shift guide5 04.09.2008 Cooling 1)TPC_COOL (in Operator Node) (next page) 2)FEC Cooling plant must be RUNNING 3)Resistor Rod Cooling plant must be RUNNING 4)In case of problems ('Standby' or 'Stopped') call expert Experts: Cooling Piquet: CCC 72201 (ask for CV/Detector Cooling, answer ALICE TPC) Rudi Schmidt 16 3962 Chilo Garabatos 16 5327 Danilo Vranic 16 3691  Note: if the cooling goes to standby, an interlock will turn of the FECs after 2 min, and the LV power supplies after 7 min

6. TPC shift guide6 04.09.2008

7. TPC shift guide7 04.09.2008 Power up and configure FEE 1)Switch ON LV Power Supplies from PVSS –This operation should be done by an expert 2)Go to the FED MONITOR, select FSM state and wait for required partitions to be blue 3)In the (sub)system state tab, select CONFIGURE and enter, e.g., COSMICS as parameter. This will turn on the FECs and configure them for running with cosmic triggers. 4)Other commands are –GO_IDLE (FEC power off, do not do this for fun) –CONFIGURE with entry RESET (a popular command) –CONFIGURE with entry PEDESTAL_RUN (for pedestal run)  Before each run it is recommended to CONFIGURE with entry RESET and once more with, e.g., entry COSMICS (and reinitialise the BB, see later)  Experts: Christian Lippmann 162430, Luciano Musa 163119, Chilo Garabatos 165327

8. TPC shift guide8 04.09.2008 FEE Monitoring 1)In TPC_FERO you can select different monitors: –FED Monitor: shows by default the temperature of the FECs, which should be lower than ~30 o C. In the lower third of the panel, all boxes should be green. If one is red, try Restart. If still red, call expert (CL). The tab FSM State shows you the state of all readout partitions –LV Monitor: shows by default the analog currents, which should be 2 Amp (FECs off) to 130 Amp (all FECs on). –Cooling Monitor: shows the status of the cooling: it should be OK and RUNNING (all green). 2)In case of problems call Christian Lippmann 16 2430, Chilo Garabatos 165327, or Luciano Musa 16 3119

9. TPC shift guide9 04.09.2008 FEE Configuration (after power up) 1)On the operator node (alitpcon001) right-click on TPC_FERO in the FSM tree to the left. 2)Choose view panel. 3)Click on the FSM button (just above). 4)Take the lock by clicking on the lock symbol. 5)In the System TPC_FERO (on the top) click on the state and choose CONFIGURE and type RESET as run parameter. The states should all be (or change to) STANDBY. 6)Choose again CONFIGURE,and type as run parameter the right name from the table of next slide. 7)Wait for all states to change to ON. This may take ~10 seconds. 8)In case of ERRORs choose RECOVER and repeat steps 6 and 7.

10. TPC shift guide10 04.09.2008 Configurations Name Pedestal Subtr. Zero Suppression Remarks PEDESTAL_RUN-NoFor Pedestal DA BLACK_FPED-No Black evens for, e.g., signal studies ZS_FPEDFixed Yes (Threshold 3) 1 Pre- & 2 Postsamples. FPED values are part of configuration. ZS_PEDMEM Pedestal Memory Yes (Threshold 3) 1 Pre- and 2 Postsamples COSMICSFixed Yes (Threshold 3) 0 Pre-,0 Postsamples, Gating Grid Suppressed, Glitch Filter On ppFixedYes (Threshold 3) Like cosmics. Only relevant acceptance readout

11. TPC shift guide11 04.09.2008 Configuration via DCS of single RCUs On the operator node (alitpcon001) open the FSM tree and find the desired partition (RCU): TPC_RCU_xxx_p. Check monitoring values and change states here. You get the same functionality by clicking on the individual partitions (RCUs) on the FED Monitor panel.

12. TPC shift guide12 04.09.2008 Troubleshooting: FEE Configuration / Monitoring via DCS (1) The FSM tree is locked: 1)Find out who has the lock, and ask him/her to release it. 2)From TPC_DCS top node, Restart ALL, Kill FSM, Start/Restart all No response from FEE: 1)If it is a single Partition (DCS board / RCU), try –Restart feeserver on this DCS board (ask expert), –Reboot this DCS board (ask expert) It takes 20 to 60s for the DCS board to reboot and for the data from this RCU to appear on the monitoring panel. 2)Restart PVSSdim by clicking on “Kill PVSSdim“ in the FED Monitor panel. 3)Restart interComLayer(s) (see next slide).

13. TPC shift guide13 04.09.2008 Troubleshooting: FEE Configuration / Monitoring via DCS (2) Restart interComLayers: 1)Log in to alitpcwn008 (A side) or alitpcwn009 (C side) (user tpc, no password needed from aldaqacr34). 2)Check if interComLayer is running (type „ ps –a “ in a shell). 3)Restart it by executing „ killall interComLayer “. Additional information: 1)The interComLayer is an intermediate Layer between the FeeServer on the DCS boards and the PVSS and FSM Control Layer you are using. There is one per side. 2)It runs in the directory /home/tpc/icl-rundir. 3)In case it exits (or is killed), the interComLayer is automatically restarted by the script /home/tpc/icl- rundir/startInterComLayer.sh.

14. TPC shift guide14 04.09.2008 Busy Box The busy box has to be reconfigured before a run starts. It is now done automatically from the central ECS It can be done from our FSM by sending a SOR command with the option RECONFIGURE Click on BUSY BOX button of the TPC_FERO node to see the status of each partition The selected partitions should show green in the setup tab clicking on a partition toggles inclusion/exclusion clicking the label of a sector toggles the whole sector Click APPLY to load a new configuration You can reconfigure manually –not during a run!- the BB by clicking RECONFIGURE in the STATUS tab; selected partitions should go green

15. TPC shift guide15 04.09.2008 should not be needed FEE Configuration via PERL scripts Go to aldaqacr34 and type the command:./SECTORS_do.pl has the following options: reset/config/on/off/armSYNC choose the appropriate one, the side, and the sector number Examples of commands:./SECTORS_do.pl A config 0-17 (to configure all FEE on A side)./SECTORS_do.pl C reset 0-4 5 6 7 8-17 The normal sequence is on, reset, config The option armSYNC is used in stand alone runs only, in case one has just reconfigured a part of the system

16. TPC shift guide16 04.09.2008 should not be needed FEE Configuration via scripts in each LDC If you log into one LDC, you should go to /local/home/tpc/TPCrdo and type: source TPCrdo_commands.sh Commands: SECTOR_on SECTOR_off SECTOR_reset SECTOR_configuration SECTOR_armSYNC If this failed for a partition, you can address it individually: rpx_on rpx_off rpx_reset rpx_config rpx_armSYNC x=1,2,3,4,5,6

17. TPC shift guide17 04.09.2008 Not yet fully operational FEE Configuration via TPCC (1) 1) Find a desktop with a session in aldaqecs01 (/date/tpcc/Linux) 2) Type:./tpcc_server (and maybe type after this:./dumpOracle.tcl) 3) Find a desktop in /local/home/tpc/rundir 4) For each LDC type /date/tpcc/Linux/tpcc_client 5) Move to aldaqpc126 (should be in /local/home/tpc/TPCrdo) -should see configurations (1-15), different commands for FECs(100- 103), check the status of server (200) and exit (300)

18. TPC shift guide18 04.09.2008 Not yet fully operational FEE Configuration via TPCC (2) If you don't see the above options, type: /date/tpcc/Linux/tpcc_control To see online the informations online go to ALICE DAQ window, press infoBrowser and another window should appear; in this new window change from Stream (should be TPC and you have to clear this field) to Facility (write TPCC) and then press Online

19. TPC shift guide19 04.09.2008 Trigger (Standalone) To run CTP Emulator (LTU, trigger window): Go to alidcscom026 (tpc tpc145) and type vmecrate tpc The VME crate window should appear select Configuration -> TTCinit (Busy box is used by default) Open the CTP Emulator window and here choose sequence l2a.slm load it by pressing the button Load sequence select Signal for pulser trigger choose BC and write the rate, e.g., 1s for laser runs choose pulser/edge In case the CTP emulator gets stuck, kill the 'ltu client' process from alidcscom026, and use the command 'xkill' and mouse-click to close the dead windows. In case of doubt or problems, please ask the CTP shifter.

20. TPC shift guide20 04.09.2008 DAQ (Standalone) Select LDCs for Readout 1.make sure you have the lock 2.press left arrows until you get to Configuration 3.press Define 4.go to GDC and choose e.g. aldaqpc046 5.go to Detector: choose TPC -> A/C side -> choose LDCs 6.save configuration and press ok 7.Select the HLT mode (Mode A: no HLT) 8.Select the equipment (readout partitions) in the equipment button of the DAQ menu) In DAQ Run Control window (click Date HI button in task bar to open): In DCA window (for DAs for example): 1.Make sure you have the locks 2.Select the run type: pedestal, laser, cosmics, etc 3.Start a run

21. TPC shift guide21 04.09.2008 Now in DCS Configuration of Busy Box Log in to alitpcwn008 (user tpc, no password) - cd /home/tpc/busy - execute./bbox-conf.pl - select the partitions - save and exit From here log into busy box dcs board: ssh alidcsdcb1555 - cd /mnt/dcbrw/busy/busybox_files - execute./bbinit.sh (before each run!) - to check status, execute./regpoll.sh status If one of the two top registers is zero, execute./program busybox_fpga1.bit busybox_fpga2.bit

22. TPC shift guide22 04.09.2008 Start/Stop stand-alone run In DAQ Run Control 1)press right arrows to get from Configuration to Ready to Start 2)press 'Start process', wait for ready and check for errors 3)press 'Start' 4)press 'Start emulation' in LTU window 5)Note: Each sector (6 RCUs) is readout via optical fibre with a Local Data Concentrator, a DAQ machine. The data is then transferred to one or several Global Data Concentrators which may or may not perform Event Building and Data Recording 6)To stop the run, quit the emulator (LTU) first, then click stop in Run Control

23. TPC shift guide23 04.09.2008 Run control from ECS (standalone) Use the ECS panel TPC Detector Control Agent (DCA) to control the run. –get the lock from the „TPC DAQ – Run Control“ (release it there first) –Select the operation of the GDC No event building No data recording mStream –get the LTU, DAQ_RC, DETECOR, (HLT) locks –Select the HLT mode (ask the HLT shifter) –Select the run type (pedestal, pulser, laser, cosmic,...) this will start a run pedestal and pulser will stop automatically –Eventually stop the run Note: if your lock is red and nobody seems to have it, make sure there isn't a second, hidden window in your panel: this one might have the lock!

24. TPC shift guide24 04.09.2008 HV Monitoring 1)Drift HV (SIM_TPC_FC next page) –check Voltage (100 kV) and Current (355  A) 2)ROC HV (SIM_TPC_ROC next page) –In FSM state, all sectors must be green –If red (trip) call expert –If electronics is turned off >1 hour click on the corresponding chambers Switch ON button to INTERMEDIATE –Check the currents: call expert if > 1  A 3)Experts: -Danilo Vranic 16 3691 -Chilo Garabatos 16 5327

25. TPC shift guide25 04.09.2008

26. TPC shift guide26 04.09.2008

27. TPC shift guide27 04.09.2008 Gas 1)TPC_GAS (Operator Node) 2)Status must be 'OK' and 'Connected' 3)Mixer, Exhaust, Pump and Distribution modules should be run or circulating; if stop, call expert. 4)You may navigate around and look at trends In case of problems, call expert: Chilo Garabatos 16 5327 Gas Piquet: 16 2516

28. TPC shift guide28 04.09.2008 Gating Grid pulsers The Gating Grid Pulser voltages are operated by experts. Experts: Danilo Vranic 16 3691, Rainer Renfordt 16 3969, Chilo Garabatos 16 5327

29. TPC shift guide29 04.09.2008 Lasers One PVSS panel per side. To turn on and run do: GO_STANDBY LOAD RECIPY GO_ON (this will calibrate and launch the 'tracker' to get optimal intensity) SUTTER_OPEN There is a trigger box which allows one to select the trigger rate –implementation in progress Experts: Borge Nielsen 16 4221, C. Garabatos 16 5327

30. TPC shift guide30 04.09.2008 Calibration pulser There is one unit per side. In each of them click Open COM port INIT All ON The corresponding automatic run (DA) is called Pulser_Calibration_run  Click on All OFF when you are done!

31. TPC shift guide31 04.09.2008 Online monitoring Log in as daquser@aldaqpc081 and go to /local/home/daquser/OM type root in root type:.x Monitor.C another window appears: select as format (DATE file/stream) press Select and OK select File/Stream -> write LDC or GDC name (e.g. @aldaqpc046:). Monitoring has to be enabled on that machine press Next Event button If root crashed or hangs, open another window in aldaqpc081 and type: killall root root.exe To take a screenshot, open a terminal on aldaqacr34 and run ksnapshot; paste it in the logbook

32. TPC shift guide32 04.09.2008 How to run AMORE /1 1 ) Open two terminal windows on aldaqacr33 (as user tpc) 2) In both do: ssh aldaqdqm04, source.balice, cd /tmp 3) Check that you get the monitoring data by doing e.g. eventDump –i –b @aldaqpc046: 4) On the first machine start the amoreAgent: amoreAgent -aTPCQA -s @aldaqpc046: 5) On the second machine start the display: amore -dTPC -mUIQA - select init - select start - when you receive data, make the window bigger and you should see the different variables

33. TPC shift guide33 04.09.2008 How to run AMORE /2 The variables are: MaxCharge – Qmax of the cluster MeanCharge – Qtot (integral over the 5x5 cell) NlocalMaxima – total number of clusters in this pad NPads – average width of cluster in pad direction NTimebins – average width of cluster in time direction (Noisy pads will typically have a small average since the noise is not shaped, see example in guide) NoThreshold – fraction of digits above threshold (ZS reduction) This is the only variable which is not a cluster variable but shows the average occupancy for each pad (between 0 and 1) TimePosition – mean Z (time) position of clusters (if you have many noise clusters in a pad this will give a peak centered at the middle of the time bin interval because the noisy clusters are typically evenly spread out in time - perhaps a bit counterintuitive). Some more info can be found here: http://www.hep.lu.se/staff/christiansen/tpc_online_monitor_guide.ppt http://www.hep.lu.se/staff/christiansen/tpc_online_monitor_guide.ppt

34. TPC shift guide34 04.09.2008 Logbook 1) Use it. 2) For each run add an entry with the run conditions: –FEC configuration –Missing partitions –Gain –Magnet status 3) Define run quality for TPC (good, bad)

35. TPC shift guide35 04.09.2008 List of on-call experts DCS, FEE, Busy Box, DAQ, Trigger –U. Frankenfeld 164048 (DCS), C. Lippmann 162430, L. Musa 163119 FC HV, ROC HV, Cooling –D. Vranic 163691, C. Garabatos 165327 Gas –C. Garabatos 165327, Gas piquet 162516 Laser –B. Nielsen 164221, C. Garabatos 165327 GG Pulser, Calibration Pulser –R. Renfordt 163969, D. Vranic 163691 Online, Offline –M. Ivanov 77761, P. Christiansen 78956