Stefan Lüders IT/CO JCOP Team Meeting ― July 7th, 2005 CNIC Computing and Network Infrastructure for Controls CyberThreats on the Horizon The CNIC Mandate CNIC Tools for Control Systems & Networks The Impact on You Stefan Lüders IT/CO JCOP Team Meeting ― July 7th, 2005

Insecure computers place site at risk DAILY ! Incidents at CERN “New Virus / Nouveau Virus” (2005/05/30: MyDoom derivatives) “This morning the CERN network was heavily disturbed ” (2004/12/15: Network problems) “It has been confirmed that the network problems during the week-end were due to a security break-in” (2004/6/7: General network problem) Insecure computers place the site at risk daily Laptops physically entering the gate bypass site firewall protections Insecure operating systems and applications are targets Weak passwords are regular targets Locally administered accounts can bypass central checks Some (commercial) applications are installed with default passwords known to intruders Computers are blocked from the network almost every day Resource intensive for everyone – compromised computers must be re-installed from scratch with latest secure software Relies on up-to-date data in the Network Database to inform the registered contacts “A major worm (similar to Blaster) is spreading on the Internet” (2004/5/3: Sasser Worm) Insecure computers place site at risk DAILY !

Change in Trend June 2005: 101 2004: 1179 incid. 2003: 643 2002: 123 Non-centrally managed PCs & downloaded code Systems exposed to firewall Suckit Rootkits (Linux) Code Red Worm (Webservers) Blaster Worm variants (Windows) IRC Based Hacker Networks (all platforms) 2004: 1179 incid. 2003: 643 2002: 123 June 2005: 101 25 systems compromised (24 Win, 1 LX, 4 VPN) 5 account compromised (all LX) 6 PCs spreading viruses/worms 61 PCs with unauthorized P2P activity (11 VPN) 4 Privacy exposures

How do Intruders Break-in? Poorly secured systems are being targeted Weak passwords, unpatched software, insecure configurations Known security holes Unpatched systems and applications are a constant target Zero Day Exploits: security holes without patches Firewall, application and account access controls give some protection Break-ins occur before patch and/or anti-virus available People are increasingly the weakest link Attackers target users to exploit security holes Infected laptops are physically carried on site Users download malware and open tricked attachments Weak/missing/default passwords Beware of installing additional applications Hacker tools are easy to find and use Google and click – better tools appear every day Intruders are difficult to find and stop Easy to hide behind multiple hacked computers Laws are lagging and require International cooperation Financial gain – hacking pays! Extracted from a presentation by Peter Haag, SWITCH CERT: Incomes estimated to be between $US 2,000 – 40,000 per month Accounts sell for $US 50-80,000 Leases for SPAM relays: $US 100/week for 20,000 machines daily Leases for centrally (IRC) controlled computers: $500 for 500 machines DDoS attack: $US 50 -1,000 (depends on size and duration) Credit cards and banking details are also traded Brokers take a 50/50 share Payment is via compromised payment accounts (PayPal) or cash

Ways to Mitigate Use managed systems when possible Ensure prompt security updates: applications, patches, anti-virus, password rules, logging configured and monitored, … Ensure security protections before connecting to a network E.g. Firewall protection, automated patch and anti-virus updates Use strong passwords and sufficient logging Check that default passwords are changed on all applications Passwords must be kept secret: beware of “Google Hacking” Ensure traceability of access (who and from where) Password recommendations are at http://cern.ch/security/passwords

CyberThreats on Controls ? Password Guessing Self-Replicating Code Password Cracking Exploiting Known Vulnerabilities Disabling Audits Burglaries Hijacking Sessions Sweepers Sniffers Distributed Attack Tools Denial of Service GUI Packet Spoofing Network Management Diagnostics Automated Probes/Scans WWW Attacks “Stealth”/Advanced Scanning Techniques 1980 1985 1990 1995 2000 2005 2010 Intruder Knowledge High Low Back Doors Zombies BOTS Morphing Malicious Code Attack Sophistication War Dialing Era of Modern Information Technology Current SCADA/PCS Zone of Defense Era of Legacy Process Control Technology (Security by Obscurity) Controls traditionally on isolated networks using obscure software and protocols

Control Systems are NOT safe Adoption of Open Standards: TCP/IP & Ethernet: Increasing integration of IT and Controls Windows: Control O/S can not always be patched immediately OPC / DCOM runs on port 135 Controls network is entangled with the Campus network Use of exposed infrastructure: The Internet, Wireless LAN Account passwords are know to several (many?) people Automation devices have NO security protections PLCs, SCADA, etc. Security not factored into their designs

Aware or Paranoid ? SM18 Exchange of network equipment W32.Blaster.Worm 11 Aug. 2003 SM18 Exchange of network equipment Badly designed TCP/IP stack Wide use of ISO protocol Aware not paranoid ! DoS (1’10”) stops any control

Risks and costs ARE significant ! CERN Assets at Risk People Personal safety (safety alarms transmitted via the Ethernet) Equipment (in order of increasing costs) Controls equipment: Time-consuming to re-install, configure and test Infrastructure process equipment: Very expensive hardware Accelerator & Experiment hardware: Difficult to repair Process Many interconnected processes (e.g. electricity and ventilation) Very sensitive to disturbances A cooling process PLC failure can stop the particle beam A reactive power controller failure can stop the beam Difficult to set up Requires many people working, possibly out-of-ordinary hours Risks and costs ARE significant ! Deal with security before or after the incident…

CNIC Working Group Created by the CERN Executive Board Delegated by the CERN Controls Board “…with a mandate to propose and enforce that the computing and network support provided for controls applications is appropriate” to deal with security issues. Members cover all CERN controls domains and activities Service providers (Network, NICE, Linux, Security) Service users (AB, AT, LHC Experiments, TS)

Phase I: Specification II III CNIC Policy Approval Awareness campaign Spec’s NICEFC: LinuxFC: Networking: 09/2004 01/2005 07/2005 01/2006 07/2006 Define rules, policies and management structures Define tools for Controls Network Configuration, Management & Maintenance Control System Configuration, Management & Maintenance Controls Network Domains: Define domains and responsibilities as well as tools for set-up and maintenance. Set up rules, policies and authorization procedures for what can be connected to a domain. Rules, policies and mechanisms for inter-domain communications incl. with the Campus Network. Investigate technical means and propose implementation Stimulate general security awareness

Approval of Phase I https://edms.cern.ch/document/584092/1 Security Policy Network Segregation & Management “Network Domains” Control System Configuration & Management “NICEFC” & “LinuxFC” Services, Maintenance & Support Approval procedure launched

Security Policy Network Domains Development & Testing Physical network segregation & Functional Sub-Domains Hardware Devices Restricted USB; no modems, CD-ROMs, wireless access, … Operation System Central installation Strategy for security patches Controls Software Development guidelines Strategies for patching and upgrading Development & Testing Outside the Domains Logins and Passwords Traceability, restrictions of generic accounts Following IT recommendations Training Awareness Campaign User training on rules & tools Security Incidents and Reporting Reporting and follow up Disconnection if risk for others

Networking Desktop Computing (GPN) Technical Network (TN) and Experiment Networks (EN) Domain Manager with technical responsibility Only operational devices Authorization procedure Dependencies DNS, NTP, DB, DFS, DIP, … Inter-Domain Communications Application gateways Trusted services NetMon and IDS Performance and statistics Disconnection on “breakpoints” Desktop Computing, testing, access from outside, … File systems (DFS, …), databases (CERNDB, …), servers (DNS, …) What can be connected Breakpoints can be defined

Networking Use Cases Office or Wireless Connection to Control System: Connection to application gateway Open session to application (e.g. PVSS) with connection to controls machines and/or PLCs Vulnerable Devices (e.g. PLCs) : Protected against security risks Grouped into Functional Sub-Domains Access only possible from the host system that controls them External access to the host system via application gateway

NICEFC & LinuxFC NICEFC and LinuxFC Centrally managed and distributed Also for desktop/office PC: the current NICE will be replaced Named Set of Control Computers (NSCC) Groups of computers with identical configuration Responsible persons will be contacted in case of emergency, or if e.g. security patches need to be applied. Configuration Version management database Operating System (NICEFC or LinuxFC) User defined software packages (e.g. PVSS, …) Rollback to previous version Local firewalls, anti-virus, intrusion detection Today: Windows XP SP2 (NICE XP), Scientific Linux CERN 3 (SLC3)

Services Operation, Support and Maintenance (IT Support) Standard equipment Network connections (24h/d, 365d/year) Operating system installation Security patches Test Environment Vulnerability tests (e.g. TOCSSiC) Integration tests (one test bench per domain) Hardware Support Standard (“office”) PCs “Industrial” PCs

Phase II: Implementation CNIC Policy Approval Deployment Awareness campaign Training on policy and tools Spec’s NICEFC: LinuxFC: Networking: Pilot Dev. Install. Pilot WTS: 09/2004 01/2005 07/2005 01/2006 07/2006 Deployment of CNIC policy Implementation of tools for configuration, management & maintenance Installation of Windows Terminal Servers Training

Phase II: Implementation Pilot tools ready by September 1st, 2005

Training on policy and tools Phase III: Operation I II III CNIC Policy Approval Deployment Op. Awareness campaign Training on policy and tools Spec’s NICEFC: LinuxFC: Networking: Pilot Dev. Operation Install. Pilot WTS: Operation 09/2004 01/2005 07/2005 01/2006 07/2006 Review of Effectiveness of Policies and Methods: Under real operation Review Possible Changes: Incorporating User feedback Extension of the CNIC Membership Finally full separation of TN and GPN

What Does Change for YOU ? New Access Scheme Access via application gateways (like WTS, LXPLUS, …) For all office PCs and wireless access New Connection Policy Connections must be authorized by Domain Manager Easier Installation Procedures for O/S and controls applications Configuration Transparent Procedures for Security patches und updates Installation scenarios Development & Testing Must be possible outside on GPN

What do YOU have to do ? As Hierarchical Supervisor Make security a working objective Include as formal objectives of relevant people Ensure follow up of awareness training As Budget Responsible Collect requirements for security cost Assure funding for security improvements As Technical Responsible Assume accountability in your domain Delegate implementation to system responsible

Conclusions Adoption of open standards exposes CERN assets at security risk. CNIC provides methods for mitigation. CNIC tools are ready soon. Do YOU act before or after the incident ?

Questions ? http://cern.ch/wg-cnic Domain Responsible Persons: GPN: IT/CS TN: Uwe Epting & Søren Poulsen (TS), Pierre Charrue, Alastair Bland & Nicolas de Metz-Noblat (AB/AT) ALICE EN: Peter Chochula ATLAS EN: Giuseppe Mornacchi CMS EN: Martti Pimia LHCb EN: Beat Jost http://cern.ch/wg-cnic Security Incidents: Computer.Security@cern.ch Computer Security Info: http://cern.ch/security