Supervision of Production Computers in ALICE Peter Chochula for the ALICE DCS team

Control Systems that require Supervision Detector control system (DCS) network Detector control system (DCS) network DSS (detector Safety System) DSS (detector Safety System) Control Equipment on the field layer (gas PLCs, cooling…) Control Equipment on the field layer (gas PLCs, cooling…) DCS Control computers and equipment DCS Control computers and equipment Magnet control etc. Magnet control etc. Control Computers and equipment installed by detectors Control Computers and equipment installed by detectors Online systems (DAQ,TRG,HLT) Online systems (DAQ,TRG,HLT) Quality Monitoring Quality Monitoring This talk covers only DCS computers and equipment

Operating Systems and Platforms Mostly PC based computers Mostly PC based computers Windows Windows Linux Linux Special computers: Special computers: PLCs PLCs Power supplies Power supplies VME masters VME masters Readout controllers (FPGA executing Linux) Readout controllers (FPGA executing Linux)

Experiment specific aspects (1) Significant number of computers will be inaccessible during the operation Significant number of computers will be inaccessible during the operation Clear need of remote supervision Clear need of remote supervision Requirements of remote boot and computer reset Requirements of remote boot and computer reset Remote software deployment Remote software deployment

Experiment specific aspects (2) Critical tasks performed on control computers require advanced access control and system of privileges Critical tasks performed on control computers require advanced access control and system of privileges Protection against malicious and non-malicious attacks Protection against malicious and non-malicious attacks Need for deployment of security patches for both Linux and Windows platforms, however this cannot be done automatically (e.g. during physics run) Need for deployment of security patches for both Linux and Windows platforms, however this cannot be done automatically (e.g. during physics run)

Experiment specific aspects (3) Control system include many non-standard systems and platforms (PLC, power supplies etc.) Control system include many non-standard systems and platforms (PLC, power supplies etc.) Normal operation assumes just reading relevant detector-related information (temperatures, voltages, etc.) and setting parameters – these tasks are covered by the DCS Normal operation assumes just reading relevant detector-related information (temperatures, voltages, etc.) and setting parameters – these tasks are covered by the DCS Need for remote software deployment (firmware upgrades etc.) Need for remote software deployment (firmware upgrades etc.)

Experiment specific aspects (4) Supervision should include also processes not related to OS as their operation is essential for the whole experiment Supervision should include also processes not related to OS as their operation is essential for the whole experiment OPC servers OPC servers Front-end monitoring and control servers etc. Front-end monitoring and control servers etc. Alarm and error handling should be merged with DCS operation Alarm and error handling should be merged with DCS operation Common Event Viewer, common operator screen Common Event Viewer, common operator screen Alice would welcome integration of supervision tools with PVSS system Alice would welcome integration of supervision tools with PVSS system

Currently used supervision tools Present efforts concentrated on testing of individual system components - no real computer supervision implemented (yet) Present efforts concentrated on testing of individual system components - no real computer supervision implemented (yet) Test systems are based on JCOP framework tool PCMON Test systems are based on JCOP framework tool PCMON PCMON server executes on every supervised system (Windows or Linux) PCMON server executes on every supervised system (Windows or Linux) Gathered data are published via DIM Gathered data are published via DIM Any DIM client can subscribed to monitored data Any DIM client can subscribed to monitored data PVSS is used as a main client platform – published data connected to datapoints PVSS is used as a main client platform – published data connected to datapoints Computers are treated as any other DCS device Computers are treated as any other DCS device

PCMON-PVSS connection LINUX WXP Dead PCMON connection

PCMON-PVSS connection

Supervision tools for FEE Supervision of front-end control and monitoring servers via DIM-PVSS connection – very similar to PCMON Supervision of front-end control and monitoring servers via DIM-PVSS connection – very similar to PCMON Event categorization compatible with Windows (info messages, errors, warnings etc.) Event categorization compatible with Windows (info messages, errors, warnings etc.) Present strategy assumes that the server process is always running and client software can detect connection problems Present strategy assumes that the server process is always running and client software can detect connection problems Need for tools enabling automatic check of presence of a given process (database of processes and hosts) and remote launching of missing processes Need for tools enabling automatic check of presence of a given process (database of processes and hosts) and remote launching of missing processes

Future plans and strategies Planning related to LHC schedule Planning related to LHC schedule Preliminary schedule assumes Preliminary schedule assumes installation of lab systems including preliminary version of computer supervision (at least performance monitoring and definition of software deployment policies) installation of lab systems including preliminary version of computer supervision (at least performance monitoring and definition of software deployment policies) 2005 pre-installation in experimental site 2005 pre-installation in experimental site 2006 installation and preparation for final operation. Fully working supervision will be needed at this point 2006 installation and preparation for final operation. Fully working supervision will be needed at this point

Conclusions ALICE DCS will require advanced computer supervision ALICE DCS will require advanced computer supervision Our preference is to integrate computer supervision with DCS (PVSS) Our preference is to integrate computer supervision with DCS (PVSS) Concerns about security and access control Concerns about security and access control ALICE DCS team welcomes common solutions and is happy to participate in software evaluation ALICE DCS team welcomes common solutions and is happy to participate in software evaluation