HEPiX Virtualisation Working Group Status, February 10 th 2010 April 21 st 2010 May 12 th 2010

CERN.ch Objective  Enable virtual machine images created at one site to be used at other HEPiX (and WLGC) sites.  Working assumptions –images are generated by some authorised or trusted process »Some sites may accept “random” user generated images, but most won’t –images are “contextualised” to connect to local site workload management system »But at least one site (other than CERN…) is interested in seeing images connect directly to experiment workload management system. 2 No root access by end user during image generation. Recipient site controls how “payload” ends up in the image

CERN.ch Working group areas & Status  Generation  Transmission  Expiry & Revocation  Contextualisation  Support for multiple Hypervisors 3 Image endorser required to revoke images in case of security issues and the like.

CERN.ch Working group areas & Status  Generation –Led by Dave Kelsey & Keith Chadwick –Likely to produce »Policy proposal for image generation process. If sites can demonstrate they meet the requirements of the policy then their images should be trusted for execution at remote sites »Recommendations for hypervisor configuration to ensure maximum security.  Transmission  Expiry & Revocation  Contextualisation  Support for multiple Hypervisors 4 Sites anyway expected to follow best practices. Current discussion is around roles and endorsers for the different components (“base” operating system and VO software) and about who can be trusted. Draft Policy Document:

CERN.ch 5

Working group areas & Status  Generation  Transmission –Led by Owen Synge –Likely to produce »Recommendation for basic transport protocol(s) to be supported  Prescriptive for sites wishing to generate images »Proposal for optional protocols to improve transmission efficiency  E.g. transmission of only differences w.r.t. a reference image  Status of “interesting” protocols such as bitTorrent likely to be an issue. –Unlikely to comment on intra-site image transmission  Expiry & Revocation  Contextualisation  Support for multiple Hypervisors 6 Will not Current model is tagged images distributed in manner akin to mechanism used for VO software today.

CERN.ch 7

Working group areas & Status  Generation  Transmission  Expiry & Revocation –Status a little unclear »a mix of standalone area and generation policy? –“Image Revocation List” a la CRL? »Technical proposal required  Contextualisation  Support for multiple Hypervisors 8 Image endorser required to revoke images in case of security issues and the like.

CERN.ch Working group areas & Status  Generation  Transmission  Expiry & Revocation  Contextualisation –Led by Sebastien Goasguen –Likely to produce »Proposal for mechanism allowing site to configure image  File system mounted at image instantiation and automated invocation of scripts on the file system during the initialisation.  Final job/payload will not execute as root »Restrictions on aspects sites are allowed to configure  No changes to C compiler, perl, python, … to be allowed  Support for multiple Hypervisors 9 Only basic discussions so far. Contentious issue is kernel patching. Group conclusion is that this is not allowed; sites who have security concerns with an image must refuse to run this and must notify the endorser to allow wider revocation. This ensures that all sites are protected.

CERN.ch Working group areas & Status  Generation  Transmission  Expiry & Revocation  Contextualisation  Support for multiple Hypervisors –Led by Andrea Chierici –Produce, if possible, »Recommendations/recipe(s) to enable sites to generate images that can be used with a range of hypervisors  Perhaps a limited set of all possible, however,…  Poll underway to identify most popular hypervisors 10 Little discussion in the group so far. We have identified the hypervisors of interest (kvm and both Xen modes). Andrea is testing extensively at present.

CERN.ch Summary  A year ago, sites were rejecting any possibility of running remotely generated virtual machine images.  Today, we have the skeleton of a scheme that will enable sites to treat trusted VM images exactly as normal worker nodes. –This enables »VOs to be 100% sure of the worker node environment »(potentially) inclusion in the VM image of the pilot job framework enabling “cloud like” submission of work to sites.  Active involvement of VOs is now highly desirable as we move towards delivering a proof-of-concept system.  Nothing in what is being done –prevents sites that wish to do so from implementing Amazon EC2-style instantiation of user generated images, or –precludes use of CERNvm. 11