CNIC Stefan Lüders IT/CO JCOP Team Meeting ― July 7th, 2005 ► CyberThreats on the Horizon ► The CNIC Mandate ► CNIC Tools for Control Systems & Networks ► The Impact on You Computing and Network Infrastructure for Controls

Incidents at CERN “New Virus / Nouveau Virus” (2005/05/30: MyDoom derivatives) “This morning the CERN network was heavily disturbed ” (2004/12/15: Network problems) “A major worm (similar to Blaster) is spreading on the Internet” (2004/5/3: Sasser Worm) “It has been confirmed that the network problems during the week-end were due to a security break-in” (2004/6/7: General network problem) Insecure computers place site at risk DAILY !

Change in Trend June 2005: systems compromised (24 Win, 1 LX, 4 VPN) 5 account compromised (all LX) 6 PCs spreading viruses/worms 61 PCs with unauthorized P2P activity (11 VPN) 4 Privacy exposures Suckit Rootkits (Linux) Code Red Worm (Webservers) Blaster Worm variants (Windows) IRC Based Hacker Networks (all platforms) 2004: 1179 incid. 2003: : 123 Non-centrally managed PCs & downloaded code Systems exposed to firewall

How do Intruders Break-in? Poorly secured systems are being targeted Weak passwords, unpatched software, insecure configurations Known security holes Unpatched systems and applications are a constant target Zero Day Exploits: security holes without patches Firewall, application and account access controls give some protection Break-ins occur before patch and/or anti-virus available People are increasingly the weakest link Attackers target users to exploit security holes Infected laptops are physically carried on site Users download malware and open tricked attachments Weak/missing/default passwords Beware of installing additional applications

Ways to Mitigate Use managed systems when possible Ensure prompt security updates: applications, patches, anti-virus, password rules, logging configured and monitored, … Ensure security protections before connecting to a network E.g. Firewall protection, automated patch and anti-virus updates Use strong passwords and sufficient logging Check that default passwords are changed on all applications Passwords must be kept secret: beware of “Google Hacking” Ensure traceability of access (who and from where) Password recommendations are at

CyberThreats on Controls ? Password Guessing Self-Replicating Code Password Cracking Exploiting Known Vulnerabilities Disabling Audits Burglaries Hijacking Sessions Sweepers Sniffers Distributed Attack Tools Denial of Service GUI Packet Spoofing Network Management Diagnostics Automated Probes/Scans WWW Attacks “Stealth”/Advanced Scanning Techniques Intruder Knowledge High Low Back Doors Zombies BOTS Morphing Malicious Code Attack Sophistication War Dialing Era of Legacy Process Control Technology (Security by Obscurity) Era of Modern Information Technology Current SCADA/PCS Zone of Defense

Control Systems are NOT safe Adoption of Open Standards: TCP/IP & Ethernet: Increasing integration of IT and Controls Windows: Control O/S can not always be patched immediately OPC / DCOM runs on port 135 Controls network is entangled with the Campus network Use of exposed infrastructure: The Internet, Wireless LAN Account passwords are know to several (many?) people Automation devices have NO security protections PLCs, SCADA, etc. Security not factored into their designs

Aware or Paranoid ? SM18 W32.Blaster.Worm 11 Aug DoS (1’10”) stops any control Exchange of network equipment Badly designed TCP/IP stack Wide use of ISO protocol

People Personal safety (safety alarms transmitted via the Ethernet) Equipment (in order of increasing costs) Controls equipment: Time-consuming to re-install, configure and test Infrastructure process equipment: Very expensive hardware Accelerator & Experiment hardware: Difficult to repair Process Many interconnected processes (e.g. electricity and ventilation) Very sensitive to disturbances A cooling process PLC failure can stop the particle beam A reactive power controller failure can stop the beam Difficult to set up Requires many people working, possibly out-of-ordinary hours CERN Assets at Risk Risks and costs ARE significant !

CNIC Working Group Created by the CERN Executive Board Delegated by the CERN Controls Board “…with a mandate to propose and enforce that the computing and network support provided for controls applications is appropriate” to deal with security issues. Members cover all CERN controls domains and activities Service providers (Network, NICE, Linux, Security) Service users (AB, AT, LHC Experiments, TS)

CNIC Members TS Uwe EPTING - TS/CSE Søren POULSEN - TS/EL AB Pierre CHARRUE - AB/CO Mike LAMONT - AB/OP Patrick LIENARD - AT/MAS IT/CO Bruce FLOCKHART - IT/CO Stefan LÜDERS - IT/CO Experiments Beat JOST - PH-LBC Guiseppe MORNACCHI - PH/ATD Martti PIMIA - PH/CMC Peter CHOCHULA - PH/AIT Network David FOSTER - IT/CS Jean-Michel JOUANIGOT - IT/CS Nils HOIMYR - IT/CS Nuno CERVAENS COSTA -IT/CS NICEFC Alberto PACE - IT/IS Ivan DELOOSE - IT/IS LINUXFC Jan IVEN - IT/ADC Matthias SCHRÖDER - IT/ADC Security Denise HEAGERTY - IT/DI Lionel CONS - IT/DI

Phase I: Specification IIIIII CNIC PolicyApprovalSpec’s NICEFC: Spec’s LinuxFC: Networking: 09/200401/ /200501/200607/2006 Define rules, policies and management structures Define tools for Controls Network Configuration, Management & Maintenance Control System Configuration, Management & Maintenance Investigate technical means and propose implementation Stimulate general security awareness Awareness campaign

Approval of Phase I 1)Security Policy 2)Network Segregation & Management “Network Domains” 3)Control System Configuration & Management “NICEFC” & “LinuxFC” 4)Services, Maintenance & Support Approval procedure launched

Security Policy Network Domains Physical network segregation & Functional Sub-Domains Hardware Devices Restricted USB; no modems, CD-ROMs, wireless access, … Operation System Central installation Strategy for security patches Controls Software Development guidelines Central installation Strategies for patching and upgrading Development & Testing Outside the Domains Logins and Passwords Traceability, restrictions of generic accounts Following IT recommendations Training Awareness Campaign User training on rules & tools Security Incidents and Reporting Reporting and follow up Disconnection if risk for others

Networking Technical Network (TN) and Experiment Networks (EN) Domain Manager with technical responsibility Only operational devices Authorization procedure Desktop Computing (GPN) Dependencies DNS, NTP, DB, DFS, DIP, … Inter-Domain Communications Application gateways Trusted services NetMon and IDS Performance and statistics Disconnection on “breakpoints”

Networking Use Cases Vulnerable Devices (e.g. PLCs) : Protected against security risks Grouped into Functional Sub-Domains Access only possible from the host system that controls them External access to the host system via application gateway Office or Wireless Connection to Control System: Connection to application gateway Open session to application (e.g. PVSS) with connection to controls machines and/or PLCs

NICEFC & LinuxFC NICEFC and LinuxFC Centrally managed and distributed Also for desktop/office PC: the current NICE will be replaced Named Set of Control Computers (NSCC) Groups of computers with identical configuration Responsible persons will be contacted in case of emergency, or if e.g. security patches need to be applied. Configuration Version management database Operating System (NICEFC or LinuxFC) User defined software packages (e.g. PVSS, …) Rollback to previous version Local firewalls, anti-virus, intrusion detection

Services Operation, Support and Maintenance (IT Support) Standard equipment Network connections (24h/d, 365d/year) Operating system installation Security patches Test Environment Vulnerability tests (e.g. TOCSSiC) Integration tests (one test bench per domain) Hardware Support Standard (“office”) PCs “Industrial” PCs

Phase II: Implementation Deployment of CNIC policy IIIIII CNIC PolicyApproval Training on policy and tools Deployment PilotDev. PilotDev. PilotDev. Spec’s NICEFC: Spec’s LinuxFC: Networking: 09/200401/ /200501/200607/2006 Install.Pilot WTS: Awareness campaign Implementation of tools for configuration, management & maintenance Installation of Windows Terminal Servers Training

Phase II: Implementation Pilot tools ready by September 1st, 2005

Phase III: Operation Review of Effectiveness of Policies and Methods: Under real operation Review Possible Changes: Incorporating User feedback Extension of the CNIC Membership IIIIII CNIC PolicyApproval Operation Training on policy and tools DeploymentOp. Operation Spec’s NICEFC: Spec’s LinuxFC: Networking: 09/200401/ /200501/200607/2006 Awareness campaign PilotDev. PilotDev. PilotDev. Install.Pilot WTS: Finally full separation of TN and GPN

Man Power Situation IIIIII CNIC PolicyApproval Operation Training on policy and tools DeploymentOp. Operation Spec’s NICEFC: Spec’s LinuxFC: Networking: 09/200401/ /200501/200607/2006 Awareness campaign PilotDev. PilotDev. PilotDev. Install.Pilot WTS: Tools (development & support) 3 FTE assigned to IT last FTE arrives 08/2005 But: No manpower for packaging WTS Support Originally not foreseen 1 FTE missing in IT CNIC Operation (administration & user support) - 1 FTE per domain needed

What Does Change for YOU ? New Access Scheme Access via application gateways (like WTS, LXPLUS, …) For all office PCs and wireless access New Connection Policy Connections must be authorized by Domain Manager Easier Installation Procedures for O/S and controls applications Configuration Transparent Procedures for Security patches und updates Installation scenarios Development & Testing Must be possible outside on GPN

As Budget Responsible Collect requirements for security cost Assure funding for security improvements What do YOU have to do ? As Hierarchical Supervisor Make security a working objective Include as formal objectives of relevant people Ensure follow up of awareness training As Technical Responsible Assume accountability in your domain Delegate implementation to system responsible

Conclusions Adoption of open standards exposes CERN assets at security risk. CNIC provides methods for mitigation. CNIC tools are ready soon. Do YOU act before or after the incident ?

Questions ? Domain Responsible Persons: GPN: IT/CS TN: Uwe Epting & Søren Poulsen (TS), Pierre Charrue,Alastair Bland & Nicolas de Metz-Noblat (AB/AT) ALICE EN: Peter Chochula ATLAS EN: Giuseppe Mornacchi CMS EN: Martti Pimia LHCb EN: Beat Jost Security Incidents: Computer Security Info: