Fermilab / FermiGrid / FermiCloud Security Update Work supported by the U.S. Department of Energy under contract No. DE-AC02-07CH11359 Keith Chadwick Grid & Cloud Computing Department Fermilab

About Keith Chadwick Fermilab employee for 25+ years, –Started as a VAX/VMS system administrator & code management for the CDF collaboration. Head of the Grid and Cloud Computing Department at Fermilab, –FermiGrid Project Leader, –FermiCloud Project Sponsor. Serve on the Fermilab Computer Security Policy Board & the Fermilab Network Design Task Force, Up until March 2012 – Served as the Deputy Head of the Fermilab Computer Incident Response Team. 17-Sep-2012EGI TF CSIRT Meeting1

About FermiGrid FermiGrid = Fermilab Campus Grid + operation of the central services used to implement the Fermilab Campus Grid, Currently have 7 Grid clusters (1 x CDF, 1 x CMS, 4 x D0, 1 x GP) with > 24K job slots, Services include GUMS (Grid User Mapping Service), SAZ (Site AuthoriZation) Service, Squid, MyProxy, etc. Services provided in a distributed highly available architecture (two sites separated by ~1.6 km). Strongly “encourage” that all logs are sent to the central security logging service. FermiGrid members are available to assist with incident response. FermiGrid operates in the Open Science Environment (formerly Enclave), with a different security baseline than the remainder of the Fermilab General Computing Environment. 17-Sep-2012EGI TF CSIRT Meeting2

17-Sep-2012EGI TF CSIRT Meeting3

17-Sep-2012EGI TF CSIRT Meeting4

17-Sep-2012EGI TF CSIRT Meeting5

FCC and GCC 25-Apr-2012Business Continuity at Fermilab6 FC C GC C The FCC and GCC buildings are separated by approximately 1 mile (1.6 km). FCC has UPS and Generator. GCC has UPS.

About FermiCloud A private IaaS cloud operated at Fermilab based on OpenNebula with x.509 credential based authentication. Used by: –Fermilab Grid and Cloud computing personnel (service development, testing, integration, hardening, etc.), –Open Science Grid for development of the OSG software distribution, –Used for “low impact” production services, –Tests have shown that it can support HPC code development, Will be used to support Fermilab 100 Gigabit/second network testing and join the Fermilab and ESnet 100 Gigabit/second network. 17-Sep-2012EGI TF CSIRT Meeting7

Fermilab Incident Response Due to the changing nature of the incident response, the incident response coordination duties are now handled by members of the Fermilab Computer Security Team. Most incidents today are “routine” infections of laptops/desktops. They are identified by well defined “triggers” based on network and anti-virus monitoring, and occasional external reports. These incidents are handled through well defined “deskside” support procedures (up to and including “wipe and reinstall from known good media”). Knowledgeable members of the Fermilab community (such as FermiGrid/FermiCloud administrators) are available to be conscripted as part of any incident response. 17-Sep-2012EGI TF CSIRT Meeting8

WLCG 2012 Security Challenge Apparently did not include CMS T1 at Fermilab… ??  / ?? 17-Sep-2012EGI TF CSIRT Meeting9

Recent Grid Incidents User “1” submitted a bunch of misbehaving jobs and left on travel… User “2” submitted a bunch of misbehaving jobs and did not check their over the weekend… VO “A” submitted a bunch of misbehaving jobs… In all of the above cases, we used our Site AuthoriZation (SAZ) service capability to “ban” the user/VO until they claimed that they had fixed their jobs and we verified that the jobs were really fixed. 17-Sep-2012EGI TF CSIRT Meeting10

OSG CA Transition DOEgrids CA will be ending operation in mid 2013, OSG has signed a contract with DigiCert to provide CA services to OSG members, Tools have been revised/upgraded/enhanced to deal with DigiCert, Will be released as part of late September/mid October OSG software package. 17-Sep-2012EGI TF CSIRT Meeting11

Thank You! Any Questions? 17-Sep-2012EGI TF CSIRT Meeting12