1. The Detector Safety System for LHC Experiments Stefan Lüders ― CERN EP/SFT & IT/CO RCS Review Meeting ― September 2 nd, 2003

2. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review2/15 Outline Experimental Safety The DSS Scope and Goal Functional Requirements & Experiments’ Needs Design and Architecture Status and Conclusions

3. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review3/15 The LHC experiments and their sites, e.g. (sub-)detectors, gas systems, magnets, power distribution, racks and crates will be the equipment to be acted upon directly by the control and safety systems. Technical Services provide power, water, gas (general services) and distribute them to the different locations (experiment services). In 2001, the experiments have realized, that some safety aspects are not covered by the CSS and DCS. The availability and reliability of a PC-based control system (DCS) does not seem to be sufficient to ensure proper equipment protection. The DSS was born. Sensors monitor the state of the equipment: temperature (equipment, ambient air, water), humidity, water-flow, gas / oxygen (sniffer sensors), status signals of the sub-detectors There are dedicated sensors for the different safety and control systems, but they are not duplicated. The safety for personnel is ensured by the CERN Safety System (CSS). It is required by law and conforms to relevant International, European, and National standards. It has its own sensors and reacts globally, i.e. on whole buildings or caverns. The DSS complements (but should not duplicate) CSS and DCS: “The DSS is a system to safeguard the experiment. As such, it acts to prevent damage to the experimental equipment when a serious fault situation is detected (e.g. temperature too high, water leak, bad sub-detector status…), inside or outside of the detector…” Experiment DCS General services: power, water Experiment services: power, water, gas Experiment: sub-detectors, racks, crates Sensors 3 Levels of Experiment Safety DCS sub-system C DCS sub-system B DCS sub-system A CSSDSS Water Leak Front-End / Hardware Layer Back-End / Supervisory Layer Trip Smoke, Gas Leak monitor control Level 3 (fatal) Level 2 (error) Level 1 (normal) The Detector Control System (DCS) is responsible for the overall monitoring and control of the detector. It might take corrective action to maintain normal operation. All DCS sub-systems are interconnected. The DSS is embedded in an Experiment’s DCS. Alarm conditions are exchanged with the CSS (hardwired). The DSS should improve the experiment’s efficiency by preventing situations leading to level-3-alarms (plus possibly to 2-3 weeks downtime) decreasing downtime due to failures …and should not costs too much ;-). The DSS can be considered as an “insurance policy”.

4. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review4/15 The DSS must be a standalone system, and be… highly reliable highly available as simple and robust as possible re-configurable by the GLIMOS and experiments’ safety experts self-checking for consistency The DSS Functional Requirements It must operate permanently and independently of the state of DCS and CSS. It must be able to take immediate actions to protect the equipment. The DSS functional requirements have been evaluated by the four LHC experiments in a joint working group.

5. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review5/15 Constraints for the DSS Easy integration… into the control system of the experiment of sub-detector safety systems of external information (from the LHC machine, gas systems, CSS, …) Adaptability… to the different needs of four experiments to the evolving experiment environments e.g. during their assembly, commissioning, operation and dismantling (a time-span of approx. 20 years) Maintainability… over the lifetime of the experiments The DSS is a common solution proposed for all four LHC experiments

6. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review6/15 Surface buildings Counting rooms Detector LHC Experiments’ Needs 200 to 800 analog and digital values to be monitored Several hundred digital actuators Geographically highly distributed system Sensors and actuators are located in the caverns as well as in several surface buildings

7. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review7/15 The Front-End will… be based strictly on industrial solutions, e.g. PLC technology for safety applications standard communication protocols (PROFIbus, Ethernet, OPC) material from the CERN store DSS Front-End Solution have its own sensors and actuators check and filter the sensors will always react immediately and automatically on fault conditions indicated by the sensors be on safe power (CERN safe power plus own UPS) After detailed discussons in the DSS Advisory Board,

8. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review8/15 Input: Sensors DSS cycle The DSS Cycle DSS Cycle (“OB1”): The DSS continuously monitors e.g. temperatures sensors, water flow, sub-detector status Output: Action (e.g. cutting off power ) T>T thres Input values are checked and compared to defined thresholds. Several conditions can be logically combined. Their fulfillment will produce an alarm. Actions are taken on a coarse level (e.g. cutting power to a rack row). AND Alarm End-of-Cycle Alarms will trigger defined actions. The relation between sensors, alarms and action is called the “Alarm/Action Matrix”.

9. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review9/15 After detailed discussons in the DSS Advisory Board, DSS Back-End Solution The DSS User Interface (Back-End) will… be based on the SCADA system “PVSS” and CERN’s JCOP Framework monitor and configure the Front-End allow a configuration of the relations between sensor values, alarms, and the actions performed in these cases (the “Alarm/Action Matrix”) define user access levels provide the user with comprehensible displays log alarm states, warnings, and related information in an Oracle database

10. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review10/15 Redundancy (optional, but required for the DSS): up to the level of I/O interfaces backup in case a power supply, CPU, Profibus failure modules have high MTBF (low failure rates). optical link between CPU modules step-by-step comparison inside the processing of the PLC cycle Back End: PVSS user interface for display logging modification of the Alarm/Action-Matrix Front-End (continued): max. 32 external crates based on S7-300 modules (limited due to redundancy) capable of handling the number of channels (inputs and outputs) as required located close to the sensors (<200m) I/O interfaces hot-swappable inputs and outputs use “positive safety” External crate: ET 200M Profibus adapter I/O interfaces redundant PS (extern) Front-End: uses a Siemens S7-400 station certified for Safety Integrity Level (SIL) 2 applications programmed through the Siemens STEP7 development environment implementation and processing of the DSS Front-End Software monitors itself CPU crate: redundant PS CPU 414-4H Ethernet adapter (CP 443-1) DSS Core Architecture Profibus DSS COM OPC Server PVSS CERN LAN OPC server: gateway to the Back-End (Windows XP) data distribution via Siemens OPC software redundant in the Front-End communication using the ISO protocol

11. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review11/15 Cavern Surface Shaft Optical Link DSS COM NTP Server CERN LAN CPUs are comfortably separated to minimize danger of accidental damage Experiment’s Configuration ET crates act as cable concentrators near sensors/actuators Connection of both CPUs to NTP. Synchronization is better than 20ms (J. Brahy, E. Veyruns CERN AB-CO). Redundant cables running through two cable paths. Spares for all cables are foreseen. PROFIbus Front-end Control Room Back-end CERN LAN Back-End situated in the control room.

12. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review12/15 Experiment’s Configuration Cavern Surface Shaft Optical Link PROFIbus DSS COM Front-end Control Room Back-end CERN LAN Functionality grouped into “ Detector Safety Units ” All DSUs are alike. Each DSU is responsible for a distinct geographic area. 2-4 DSUs typical, 16 DSUs maximal possible per experiment.

13. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review13/15 DSU Layout Patch Panel Terminals to connect sensors / actuators (max. 352 digital channels OR 120 analog channels; optimum is 224 digital PLUS 64 analog channels) Table / Drawer Control Room Panel, Gyro & Siren (not part of a DSU) External Crate with dedicated Monitoring Module 2 nd External Crate possible Ethernet Switch for DSS COM (in DSUs with CPU crate) CPU crate (in two DSUs) Redundant 24V Power Supplies & Distribution Modules Front-End Display OPC Server / Gateway (in one DSU) Uninterruptible Power Supply (UPS) 52 units standard LEP rack (here: 56U)

14. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review14/15 Status TaskStatusTarget Date Front-End softwareOperational Integration, commissioning, and test of the complete prototype Finished Back-End softwareOperational System ReviewJune 2003 Installation / commissioning for CMSIn progressSummer 2003 First operational DSS for CMSSeptember 2003 Installation / commissioning for LHCbAutumn/Fall 2003 First operational DSS for LHCbNovember 2003 First operational DSS for ALICEMarch 2004 First operational DSS for ATLASDecember 2004

15. September 2 nd, 2003“The Detector Safety System for LHC Experiments” @ RCS Review15/15 Conclusion The design of the Detector Safety System, arrived at in consultation with the DSS Advisory Board, will consist of… a Front-End: Siemens S7-400 redundant PLC hardware PC based OPC server acting as a gateway a Back-End: PC based system with the PVSS user interface, using CERN’s JCOP Framework Oracle Database connection for data and configuration logging The first system will be installed at CMS this September For more details see : http://cern.ch/proj-lhcdss/ http://cern.ch/proj-lhcdss/