19-April-05 CSC ME1/1 Meeting What is the present status, what is needed before LHC running? Operation of the CSC sub-detector Shift operations Overall CMS Integration 26-July-2008 EMU Meeting

Levels of Protection & Monitoring 19-April-05 Levels of Protection & Monitoring Local – Protection built into the hardware module Fuses, over temperature protection ect. DSS – Detector Safety System Base rack level link in personal and hardware safety Not reliant on computers, hard wired XMAS – monitoring of the chambers and peripheral crates through VME Feeds information to DCS for safety and slow monitoring Displays information to shift stander Reliant on computers and networks DCS – Detector Control System Primary link in control and monitoring Secondary link in safety > remember not designed for safety Shift Stander First link in safety Responsible for all CSC hardware wellbeing, and software process operation Other Experts Provide expertise when needed by shift stander Periodically monitor their system 26-July-2008 EMU Meeting

Status - Needs What do we need? Where are we? Rack DCS in UXC 19-April-05 Status - Needs What do we need? Rack DCS in UXC Link to DCS – CMS provides Added temperature sensors in UXC (Petr and I working on) Water information (Armando) CSC DCS improvements (see Bob’s talk) Flow charts from experts for shifters For each specific problem a flow chart leading the shifter through diagnostics to indicated action More detailed than, “When in doubt call Greg.” More detailed ‘SAFE STATE’ matrixes Where are we? DSS in USC and UXC for CSC is operational Rack DCS is ready in USC, not ready in UXC CSC DCS is operating and in use by shifters Operations Procedures Wiki and other pages in place and being used Experience from commissioning running and 3 CRUZET runs 26-July-2008 EMU Meeting

19-April-05 Unattended Operation Had Successful Electrical Safety Review (thanks Petr) Suggested changes made (see link below) Had ‘Unattended Operations’ Review see link below It was not clear at that time what ‘Unattended Operations’ really meant Consensus now seems to be: (Austin, Jeff, and others) CMS will not operate without people present at P5 Unattended means > Shifters at P5 24/7 monitoring detector systems via DCS and other means PLUS an expert for each sub-system reachable by phone 24/7 All unattended subsystems have operational DSS All unattended subsystems have operational DCS CMS plus subsystems decide what operational means http://indico.cern.ch/conferenceDisplay.py?confId=36768 26-July-2008 EMU Meeting

Safe States & Action Matrixes 19-April-05 Safe States & Action Matrixes All sub-detectors need a Matrix of ‘safe’ states A safe state is a configuration in which the sub-detector can remain indefinitely without damage Rows - Enumeration of states Columns - Sub- systems of detector All sub-detectors need to define their safe states All sub-detectors need to detailed matrixes of safe states See next slide for simple Safe State Matrix from our review All sub-detectors need action Matrixes For each ‘crate’ need to give reaction to any external input For example > for a pcrate (VME) crate Loss of water Local – CRB – DC to DC converters in CRB limit temperature by lowering output voltage >> in place DSS – turbine temperature sensor @ 40C in rack air flow trips off the rack – this trips off the feeder to the PFC for the Maraton feeding the VME crate >> in place and tested DCS – temperature sensor @ <40c in rack sends message to – CSC DCS turns off feeder Maraton >> to be implemented Shifter – follows flow chart for this situation (to be created) Expert – follows (expert) flow chart for the situation (to be created) Loss of turbine (air cooling and monitoring) Local – PCB does not use air cooling, pcrate can operate without air cooling for long period DSS - this trips off the feeder to the PFC for the Maraton feeding the VME crate DCS – XMAS sends board temperatures to CSC DCS – CSC DCS notifies shifter, then trips off crate (this TBA) 26-July-2008 EMU Meeting

Safe Operation States - UXC 19-April-05 Safe Operation States - UXC Safe Operation States >> UXC55 (cavern) Cooling Power Gas  HV LV Disk & tower LV from S4 ---------------------------------------------------------------- OFF x x OFF OFF Maratons must be off within 2 min Pcrates within 5 min LV to chamber within 10 min Note LV to chambers is turned off by pcrate turn off HV must be off within 60 min ON OFF OFF OFF OFF ON OFF ON OFF OFF ON ON OFF OFF ON ON ON ON ON ON 26-July-2008 EMU Meeting

Shift Operations Procedures 19-April-05 Shift Operations Procedures Operations Procedures are either in or linked from the CSC Operations Wiki Page Procedures for normal shifter duties Procedures for emergency situations Procedures for expert response Wiki page so multiple people can add information Reviewed and edited periodically Reviewed edition will be printed as a pdf file and posted at the shifter station CSC Operations Wiki Page https://twiki.cern.ch/twiki/bin/view/CMS/CSCOperations TWiki >   CMS Web > MuonWikiHome > MuonCSC > CSCOperations 26-July-2008 EMU Meeting

Operations Procedures - Plans 19-April-05 Operations Procedures - Plans What needs to be added to the procedures web pages More additions from experts Detailed Safe State Matrix Detailed Action Matrix Flow chart of procedures for shifter based on Safe State and Action Matrix Graphic with boxes and arrows giving clear instructions Example flow chart Shifter watches for Example Greg intends to put in a list of what to do when the CSCs throw an error, with the intent that it can put the shifter in the position to either: a) fix the problem herself b) call the appropriate expert The timescale for this is CRUZET-4. 26-July-2008 EMU Meeting

CMS Operations Plans CSC Commissioning Shifts CMS had CRUZET 1, 2 & 3 19-April-05 CMS Operations Plans CSC Commissioning Shifts Continuing between CRUZET’s At select times during Global running CSC Operation Shifts During mid-week runs During CMS Global runs Need 24/7 from start of CROZET4 Can CSC go to 24/5 from next week on? CMS had CRUZET 1, 2 & 3 Started mid-week runs 9am Wednesday through 5pm Thursday Plan CRUZET4 August 18-25 CMS Detector goes to 24/7 operation at end of CROZET4 CRAFT Accelerator start up – not much access (Sept.) LHC 1st run @ 10TeV 26-July-2008 EMU Meeting

CMS Integration CSC shifters Shifter 19-April-05 CMS Integration CSC shifters Shifter Now person physically at CR For LHC person physically at Meyrin Stands series of 8hr shifts Minimum of 3, maximum of 7 continuous shifts, the same shift time slot per day Do we need an ‘On call Duty Expert’ Rotating Expert (basically an Expert Shifter) First call when shifter needs assistance or backup 1-2 weeks on call 24/7 (or 2 persons share) Major contributor to any needed updates to written procedures Trains new shifters – sits with shifter during first shift of a series Needs to be able to enter cavern On call Experts Longer term rotation- how long? Able to respond to Duty Expert call at any time Has in depth knowledge of a few aspects of CSC Reachable 24/7 but need not be present at CERN Hardware expert needs to be able to enter cavern CRUZET Shifters All shifters at P5 in above ground CR CMS Field Manager Shift Leader CMS Shifters Global Trigger Global DAQ Global DQM Safety Sub-Detector Shifters Each sub detector > 2 shifters per shift LHC At some point will have shifters at P5 and at Meyrin At P5 > shifters to make short response time entry to USC and UXC SLIMOS CMS utilities Detector hardware At Meyrin > all other shifters 26-July-2008 EMU Meeting

Need to know Availability 19-April-05 Need to know Availability Applies to People in shift standers pool People in duty expert pool Designated expert for each sub-system Need to know as far in advance as possible Need to know for every day From CRUZET 4 onwards expect 24/7 operation Need to know when people are At CERN & working At home institution & working when not actively available 26-July-2008 EMU Meeting

19-April-05 Backup Slides 26-July-2008 EMU Meeting

DSS Response Turbine Trip in UXC Turbine Trip in USC Pcrate rack 19-April-05 DSS Response Turbine Trip in UXC Pcrate rack AC power at rack cut LVDC power from Maraton system cut at S4 – AC power to PFC module Mataton rack HV rack xxx Turbine Trip in USC HV rack – AC power cut to electronics FT rack - AC power cut to electronics FED rack - AC power cut to electronics Computer rack - AC power cut to electronics 26-July-2008 EMU Meeting

CSC-Sub-Systems Hardware Safety - SLIMOS Software CSC specific* LV* 19-April-05 CSC-Sub-Systems Hardware CSC specific* LV* HV* Gas* TF TTC FED * these experts need to be authorized to enter UXC Safety - SLIMOS Software RCMS xx 26-July-2008 EMU Meeting

For Each Sub-System Flow diagram of problem resolution 19-April-05 For Each Sub-System Flow diagram of problem resolution If A happens do B If B results in C Do D 26-July-2008 EMU Meeting

CSC-Sub-Systems Experts 19-April-05 CSC-Sub-Systems Experts Hardware CSC specific - Armando LV – Petr, Fred HV Alex, Petr ME1/1 - Gas – Valery, ?? TF – Dan, … TTC- Frank, ?? FED – Phillip, rotating OSU person Software RCMS xx 26-July-2008 EMU Meeting