Automating the automation CERN Dr. Enrique Blanco Head of the Process Control section Industrial Controls & Safety Systems Group Beams Department CERN 1 Enrique Blanco

Introduction: CERN Process control standardization R&D activities CERNEnrique Blanco 2 Outline

Conseil Européen pour la Recherche Nucléaire World largest Particle Physics Laboratory (1954) 21 Member Countries Austria, Belgium, Bulgaria, Check Republic, Denmark, Finland, France, Germany, Greece, Italia, Hungary, Holland, Israel, Norway, Poland, Portugal, Slovakia, Spain, Sweden, Switzerland, UK. Yearly Budget ~1100 MCHF (~ 900 MEUR) Experiments financed externally. 7 Observers Countries EC, USA, Russia, India, Japan, Turkey, UNESCO Personnel 2400 Staff 730 Fellows & Associates 200 Students >10000 Users 2000 External companies 2 Candidate Countries Romania and Serbia (pre-stage of membership) Enrique Blanco 3 CERN

Fundamental research Study of the Universe structure: Exploring the Physics laws which govern the fundamental blocks of the matter and the space-time structure Century IV – V AC End of century XIX Beginning of century XX 1960 Atom >>>>>>>>>>>>>>>>>> Quark ~ CERN Goal Enrique Blanco 4 CERN

On 4 July 2012, the ATLAS and CMS experiments at CERN's Large Hadron Collider announced they had each observed a new particle in the mass region around 126 GeV. This particle is consistent with the Higgs boson Higgs Boson Enrique Blanco 5 A Higgs boson decays to 4 leptons in this collision recorded by the ATLAS detector on 18 May 2012 Peter Higgs stands on the cavern floor at CMS CERN

Accelerator complex 1eV -> J 14 TeV -> 22.4 x 10 –7 joules 450 GeV 14 TeV 25 GeV Accelerator complex Enrique Blanco 6 CERN

One of the CERN goals: maximize uptime of the instruments (accelerators, detectors,…) in order to optimize physics data availability This objective implies the maximum availability and optimal operation of all the auxiliary/utilities systems (e.g. cryogenics, cooling, HVAC, gas, motion, interlocks,…) -> the correspondent control systems must ensure this. What is uncommon at the CERN accelerators control systems? Environment (radiation areas) Large systems (highly distributed and/or interconnected) Complexity (control logic) Precision (measurements) Performance (regulation) CERNEnrique Blanco 7 Control challenges

CERNEnrique Blanco 8 27km length 100m underground Thousands of Superconducting magnets (1.8 x 10 9 km of superconducting filaments) Coldest place in Universe: -271° C LHC Accelerator World Largest accelerator

compressor stations cold boxes helium storage liquid nitrogen storage helium transfer line liquid helium storage 3.3 km LHC Cryogenics CERNEnrique Blanco 9 LHC sector: 3.3 km

CERNEnrique Blanco 10 LHC cryogenics control : facts Electro-pneumatic positioner, SIPART PS2 WinCC OA HMI in the CCC Tunnel Instrumentation - 27 km of decentralized instrumentation and control - 50k I/O, 11k actuators, ~5k Control loops - Control: ~100 PLCs (Siemens, Schneider), ~40 FECs (industrial PCs) - Supervision: 26 SCADA servers : 1.5 million TAGS RadTol in-house electronics for signal conditioning and actuation

Introduction: CERN Process control Standardization Research activities CERNEnrique Blanco 11 Outline

CERN Organization CERN Enrique Blanco12 Accelerator & technology Beams Engineering Technology Research & Computing Exp. physics Theoretical physics Information Technology Finance & HR Finance & Administration Industry, Procurement & KT Human Resources Site Management and Buildings International Relations Partial view of the 2016 structure

CERN BEAMS Department ABP Accelerator and Beam Physics BI Beam Instrumentation OP Operation RF Radio Frequency CO Controls ICS Industrial Controls & Safety ASR Administration, Safety & Resources BEAMS Department Structure

BE-ICS group Industrial Controls & Safety Systems AC Access Control CSE Critical systems Engineering FGA Fire, Gas & Alarms SDS SCADA & Distributed Systems PCS Process Control Systems CIC Connections & Informatics for control Industrial Controls & Safety Systems Structure CERNEnrique Blanco 14

UNICOS (UNified Industrial Control System) was born at CERN as a need to develop the LHC cryogenics control system CERNEnrique Blanco 15 Creating standards: UNICOS UNICOS benefits -Development (Homogenized applications) -Maintainable code (Original developer is not critical) -Unified operation in control rooms Facilitate the task of the automation engineer by allowing him/her in focusing only in the automation duty and not in the software production itself: Automatic generation of code. Field Control Supervision Based on industrial standards - ISA-88 / IEC-61512: Batch control - IEC : Distributed systems Framework composed of - Generic set of reusable devices - Analysis and Development method - Programming structure

Not only a bunch of devices UNICOS CPC provides libraries (control and supervision layers) A well defined set of standard device types (objects), modeling most of the equipment and needs of continuous processes and the relationships between them. A formalized way of : Define the control units of a process (ISA-88 standard: Batch processes) Programming the specific process logic for those units I/O Objects Digital I/O Analog I/O Field Objects OnOff Analog AnalogDigital Local AnaDO Control Objects Controller Alarms Process Control Object Interface Objects Parameter (Digital, Word, Analog) Status (Word, Analog) CERNEnrique Blanco 16

CERN Enrique Blanco 17 Process inputs Objects status Orders Process output or child Auto Request Status Information to other object or to Operator Auto Requests Parent Object PLC ObjectPLC Object Manual Requests Plant Operator Parameters Control Engineer UNICOS CPC Object model

Objects & Layers Integration CERN 18 Enrique Blanco In the Supervision layer the object presents the relevant information to the operator and allow manual commands Supervision Layer Control Layer Object status Human Requests SCADA Object HMI Parameters Manual Request Information display SCADA Object Auto. Requests Object logic Orders PLC Object Object status Manual Request Parameters Process Inputs Process Plant Operator SCADA Server (s) CERN Control Room(s) OWS Field Layer

SUPERVISION, Visualization and programming SIEMENS WinCC OA (PVSS) SCADA (standard) CONTROL SIEMENS, Schneider (standards) Industrial PCs: SIEMENS IPC, Codesys FIELD LAYER Industrial instrumentation: Sensors, actuators Industrial customized actuators: Profibus PA positioners Virtual instrumentation: VFT (flowmeters) COMMUNICATIONS Fieldbuses: Profibus, WorldFIP, CAN (CERN standards) Ethernet based: Profinet, Ethernet/IP CERNEnrique Blanco 19 Industrial COTS

CERNEnrique Blanco 20 UNICOS Engineering life cycle DecompositionAutomatic Code Generation Logic specifics & Synoptics Deployment PL C Reverse Engineering Specifications

CERNEnrique Blanco 21 Methodology: (1) Decomposition Unit Equipment Modules Control Modules IEC Physical model

Methodology: (1) Process vs. Control CERN 22 I/O Devices Field Devices Control Devices Interface: I/O Boards-Fieldbus-Other PLCs Compressor QSCCx LHC 1.8K Cryoplants Point 4 Cryogenic System Compressor 1 PV ValveCV Valve PID AIAODIDO Analog Input Analog Output Digital Input Digital Output AnalogLocalOnOff Controller PCO Automatic Generation of the PCO objects (From Specifications) Standard Unicos Programming and Process Logic programs (From Specification) Operation in multiple scenarios Automatic Generation of the objects and connections between objects (From Specifications) - Each control module or equipment module is a device - Equipment modules and Units are embedded in a unique object class: PCO (Process Control Object) Enrique Blanco

Methodology: (2) Specifications CERN 23 Enrique Blanco UNICOS CPC Specs (xls/xml file) Functional Analysis + Logic specification (Word templates)

PLC: Schneider M340 Schneide r Panel : WinCC Flex PLC: S7 SCADA : WinCC OA UAB UAB: UNICOS Application Builder CERN Enrique Blanco 24 Methodology: (3) Code generation CPC Wizard Input Specification s Device Instantiation Templates Process Logic Templates Baselines SCADA Touch Panel PLC Device Types Analog Alarm PID Control System Developer Output Control Application PLC S7

Methodology: (4) Process control logic For Each PCO the process engineers supply the logic associated to each PCO in a template document (WORD) Process logic can be either: -coded by the control engineer in an standard way. -some applications may create automatically the logic based on templates. -These templates are based on Phyton scripting where PLC logic can be also written in PLC programming language. Interlock Logic Configuration logic Global logic Interlock Logic Configuration logic Sequencer / Transitions (2) Dependent Object control logic Dependent Object control logic Dependent Object control logic Sequencer /// Dependent Object control logic Dependent Object control logic Logic Placeholders Dependent Object control logic Dependent Object control logic Dependent Object control logic Dependent Object control logic Global logic CERNEnrique Blanco 25

Methodology: (5) HMI synoptics CERNEnrique Blanco 26 Manual intervention (or automatic if known a priori) Synoptic design -by drag & drop (manual operation) -Automatically created (xml)

Cryogenics LHC accelerator & Detectors Experimental areas FRESCA2: Nitrogen Cooling & HVAC Tunnels and caverns ventilation Machinery cooling Control rooms air conditioning Interlocks LHC Collimator Temperature Interlocks Motion ATLAS big wheels HTS winding machine AMS beam test servo systems LHC Elevators Gas systems LHC Detector Gas Linac 4 hydrogen supply CO2 cooling plants CLOUD Vacuum ISOLDE REX LHC Detectors: ATLAS, CMS Detector Controls Magnet Control system ECAL detector cooling CMS tracker thermal screen ALICE Cooling water valves CERN Enrique Blanco 27 UNICOS is a CERN de facto standard

Introduction: CERN Process control Standardization R&D activities CERNEnrique Blanco 28 Outline

–Communications –Reverse Engineering –Virtual commissioning: Simulation –PLC software quality control –DSLs (Domain Specific Language) –Formal methods applied to engineering CERNEnrique Blanco 29 R&D activities

Communications: TSPP CERNEnrique Blanco 30 TSPP: Time Stamp Push Protocol (event driven) Needs SCADA drivers development Currently being wrapped out with OPC UA

Reverse engineering -Getting the original specifications from any UNICOS based project -Compare original specifications with current status CERNEnrique Blanco 31

CERNEnrique Blanco 32 Dynamic simulation - Allows a better understanding of the process - Dynamic process plant verification & optimization - Allows a better understanding of the process - Dynamic process plant verification & optimization Process Knowledge -Offline tests of improved regulation (e.g. PID tuning) -New control algorithms application (e.g. model embedded based) -Offline tests of improved regulation (e.g. PID tuning) -New control algorithms application (e.g. model embedded based) Advanced Control Off-line commissioning reduces drastically the real commissioning effort Off-line commissioning reduces drastically the real commissioning effort Virtual Commissioning - Complex and critical plants do not allow training online -High operator turnover requires appropriate training - Complex and critical plants do not allow training online -High operator turnover requires appropriate training Training of operators

Virtual commissioning –Simulation techniques (static and dynamic) –HIL (Hardware in the Loop) CERNEnrique Blanco 33 Testing and Commissioning Communications : S7 protocol SUPERVISIO N CERN Industrial control architecture vs Simulation CONTRO L FIEL D Industrial Fieldbus Communications: OPC C++ application (OPC client) EcosimPro Process Model (C++ class) Simatic WinLC (PLC Simulator) Simatic OPC Server

Development –Standards: Languages IEC61131-(3-8) –Frameworks: Structure and maintainable code –Issues Management, versioning (e.g. JIRA, SVNs) –Deploying tools –Coding standards or guidelines CERNEnrique Blanco 34 PLC software quality control

DSL (Domain Specific Languages) Complex, repetitive, tedious automation tasks can be handled by appropriate languages which allow to generate advanced PLC code. Device specification list Templates UNICOS application sources UAB CPC Wizard Inputs Code Generation Import & Compile Control Code CERN Enrique Blanco 35

DSL features Provide a language (and editor) for writing logic templates: Similar syntax to the PLC languages (SCL, ST) Query data from the specifications Generate code dynamically based on the query results Visualize code in real-time Code completion CERNEnrique Blanco 36

Formal verification: Concept Testing vs Model Checking (Formal methods) Testing: - Error prone due to human manual testing - Non-exhaustive action CERNEnrique Blanco 37 Requirement If I0.0 is FALSE and I0.1 is FALSE, then Q0.0 is FALSE Q0.0 := (I0.0 AND I0.1) OR Var1

Formal verification: Workflow Formal methods applied to PLC code CERNEnrique Blanco 38 Formal Model Formal Requirement Model checker satisfiednot satisfied Counter- example Specifications Real System (hardware, software)

CERNEnrique Blanco 39 Formal Verification: Methodology

Formal Verification: Automatic generation tool CERNEnrique Blanco 40

CERNEnrique Blanco 42 LHC cryo control: A sector architecture LSS ARC RadTol electronics LHC Tunnel (3.3 Km) Protected areas PA Radiatio n areas PA shaft (~100 m) Tunnel Alcoves TT CV Local & Central Control Rooms SCADA Data Server UNICOS PLCs Ethernet (TN) DP Profibus DP FESA FECs WorldFIP TT, PT, LT, EH, DI Ehsp, LTen

Writing Logic Templates 43 PLC code generated from strings No syntactic/semantic checkings Errors detected after generate, import & compile No syntactic/semantic checkings during edition Errors detected during compilation Error messages could be improved CERN Enrique Blanco

CERNEnrique Blanco 44 Standard architecture PLC(s) Fieldbus I&O and devices O Local OWS Touch panel Ethernet Corporate P C Enterprise computers WTS P C SCADA Server (s) Ethernet Technical WTS O CERN Control Room(s) OWS EWS C C P O Operator Control Process

CERNEnrique Blanco 45 UNICOS Overview PLC and SCADA Baseline Diagnostics tools System Integrity Standard HMI tool to create process synoptics (drag & drop) UNICOS Specifications Logic I/O Channels Field Objects (Valves, Pumps, …) Process Control Objects (Compressors, feedbox, …) Instances Additional services: CMW interface Long-Term archiving LHC alarm system PLC and SCADA Instances Instance Generator Precise placeholders where the control engineer must write the process logic either automatically or manually Logic Generator C P O Operator Control Process C P C O

Page 46 Hardware components Supervision Layer –WinCC OA SCADA –*Labview (for labo-sized projects) Control Layer –Siemens S7-300, S7-400, S –Schneider Premium, Quantum –Codesys (Somachine, TwinCat3) Local operation –Simatic HMI (WinCC flexible), Magelis Industrial communications –Profibus, Profinet, Ethernet/IP –Modbus TCP * In preparation CERN Enrique Blanco46