Trusted Virtual Machine Images the HEPiX Point of View Tony Cass October 21 st 2011

CERN.ch Where’s the Problem?  Amazon EC2 (and similar services) allow users to create random images for instantiation on Amazon owned hardware. Why can’t HEP sites allow this? –Paranoia –Insufficient isolation of virtualisation service hardware »NFS and other systems using simple uid/gid access protection are vulnerable if end-users have root access. –Traceability and Logging requirements »Grid sites are required to maintain a trace of job/system activities for 6 months. (process accounting, syslog, …). »Why doesn’t Amazon need this?  T&C say you are responsible for the VM actions—so no multi-user pilot jobs…  They have your credit card details---unlike Grid sites where there may be no direct contract between someone submitting a job and the site at which it runs.  Amazon probably can link suspicious activity to a VM image instantiation. 2

CERN.ch Is there really a Problem?  Does it matter if user supplied virtual images are run in a “De-Militarised Zone” which protects the rest of the site from these images?  Virtual machine images must have the same level of access (bandwidth…) to key Grid services—notably to the mass storage system. –This is often not the case. “Real” worker nodes have better connections to storage and other services than ring-fenced virtual images. This is not acceptable. 3

CERN.ch The HEPiX Solution Trusted Virtual Machine Images  These require –A “contract” between image providers and sites where the are instantiated. –“Hooks” in the virtual machine image enabling sites to contextualise images to meet local site policies. –Multi-hypervisor support –The ability for end-users to request instantiation of a specific image for their work. 4

CERN.ch The HEPiX Solution Trusted Virtual Machine Images  These require –A “contract” between image providers and sites where the are instantiated. »Sites decide whether or not an image provider can be trusted to respect the “contract” when generating images. If the sites do trust an image provider then there is no need to verify the individual images. There is some overhead, but this is a one-time process and will concern a small number of people. –“Hooks” in the virtual machine image enabling sites to contextualise images to meet local site policies. –Multi-hypervisor support –The ability for end-users to request instantiation of a specific image for their work. 5

CERN.ch Policy for Trusted Image Generation  You recognise that VM base images, VO environments and VM complete images, must be generated according to current best practice, the details of which may be documented elsewhere by the Grid. These include but are not limited to: –any image generation tool used must be fully patched and up to date; –all operating system security patches must be applied to all images and be up to date; –images are assumed to be world-readable and as such must not contain any confidential information; –there should be no installed accounts, host/service certificates, ssh keys or user credentials of any form in an image; –images must be configured such that they do not prevent Sites from meeting the fine-grained monitoring and control requirements defined in the Grid Security Traceability and Logging policy to allow for security incident response; –the image must not prevent Sites from implementing local authorisation and/or policy decisions, e.g. blocking the running of Grid work for a particular user.  6

CERN.ch Trusted Images 7 Image courtesy of Romain Wartel

CERN.ch The HEPiX Solution Trusted Virtual Machine Images  These require –A “contract” between image providers and sites where the are instantiated. –“Hooks” in the virtual machine image enabling sites to contextualise images to meet local site policies. –Multi-hypervisor support –The ability for end-users to request instantiation of a specific image for their work. 8

CERN.ch Image Contextualisation  Contextualisation is needed so that sites can configure images to interface to local infrastructure – e.g. for syslog, monitoring & batch scheduler.  Contextualisation is limited to these needs! Sites may not alter the image contents in any way. –Any site are concerned about security aspects of an image should refuse to instantiate it and notify the endorser.  Contextualisation mechanism –Images should attempt to mount a CDROM image provided by the sites and, if successful, invoke two scripts from the CDROM image: »prolog.sh before network initialisation »epilog.sh after network initialisation 9

CERN.ch The HEPiX Solution Trusted Virtual Machine Images  These require –A “contract” between image providers and sites where the are instantiated. –“Hooks” in the virtual machine image enabling sites to contextualise images to meet local site policies. –Multi-hypervisor support »Xen/kvm compatibility demonstrated, but this has become less important as kvm becomes more dominant as a hypervisor (at least for compute node virtualisation). –The ability for end-users to request instantiation of a specific image for their work. 10

CERN.ch The HEPiX Solution Trusted Virtual Machine Images  These require –A “contract” between image providers and sites where the are instantiated. –“Hooks” in the virtual machine image enabling sites to contextualise images to meet local site policies. –Multi-hypervisor support –The ability for end-users to request instantiation of a specific image for their work. »Still a work-in-progress; the group is currently working on how newly created images can be advertised to sites and users and how these are requested. »One option being investigated is the StratusLab Marketplace. 11

CERN.ch A step towards Cloud Computing?  Most (all?) current implementations of virtualised cpu servers integrate virtual machines with local workload management systems –requests to the local WMS lead to instantiation of virtual machines –locally provided machines have WMS client installed; contextualisation allows WMS client installation for remotely provided images.  There is an alternative! –Sites can instantiate VM images that connect directly to a VOs pilot job infrastructure. –This decouples completely »Site role to allocate resources according to funding »VO interest in managing priorities between different users and tasks. –Dynamic instantiation of VM images according to demand patterns leads to changing “virtual batch system” for the VOs. 12

CERN.ch Summary  The HEPiX Virtualisation Working Group has established a framework which should enable the free interchange of virtual machine images between HEP sites –a policy governing how images should be generated, to enable trust between sites and endorsers, and –a mechanism for contextualising images to be compliant with local policies at the instantiating site, and –a mechanism for cataloguing endorsed images and to track images approved for instantiation at a given site  VO supplied images could connect to pilot job infrastructure directly, not the local workload management system –allowing sites to instantiate images dynamically according to workload patterns, a step towards Cloud Computing for HEP. 13