WP4 – Cloud Platform & Provisioning Technical Review Period 1 Michel van Adrichem, Mick Symonds, Josep Martrat Atos This document produced by Members of the Helix Nebula consortium is licensed under a Creative Commons Attribution 3.0 Unported License. Permissions beyond the scope of this license may be available at The Helix Nebula project is co-funded by the European Community Seventh Framework Programme (FP7/ ) under Grant Agreement no Members of the Helix Nebula consortiumCreative Commons Attribution 3.0 Unported Licensehttp://helix-nebula.eu/

WP4 Objectives Assemble the resources needed to meet the requirements provided by WP3, plus the tools necessary to configure, deploy, monitor and evaluate the flagship. Specific goals To produce, validate and make available a method of determining and describing a suitable service offering to meet user requirements for the Science Cloud (beyond tools). To produce, validate and make available a method of matching services to requirements in any specific case. To deploy that method in the three flagships, based on the specific requirement statements for each flagship (from WP3), in order to allow evaluation (by WP9) of the approach

The high level process The matching process This high level method is part of the process description to match services to requirements resulting from Work Package

Effort Contribution Lead Beneficiary: Atos Person-Months per Participant ParticipantPerson- months EGI.eu3.00 T-Systems3.00 Atos12.00 CloudSigma3.00 TOTAL21.00 WP4 – 21 pm WP4 – 1pm Extension of WP4 into P2 requested to update D4.3: ‘Cloud Provisioning Report’ with the evaluation of the pilots after they are completed

S cientific/technical achievements and their impact The main objective of Work Package 4 (service provisioning) is to produce, describe and analyse a suitable service offering. WP4 is at the centre of the project execution since it liaises: Work Package 3 where requirements for the flagships use cases are elicited and Work Package 5 where the response to those requirements is instantiated for each of these use-case. The main contribution of Work Package 4 to Helix Nebula has been in bringing knowledge and experience of how “services” are actually delivered. This has included, for instance, documentation of the options and recommendations regarding the establishment of a “broker” role. Work Package 4 has documented many of the service aspects, which have also appeared in the documents produced by the Service Architecture (ServArch) team

Requirements perspectives Requirements from Requirements perspective Technical requirements Service requirements Financial / Business requirements Current flagships Requirements available Requirements were needed Future flagships Need to allow for new requirements Potential future users Need to predict industry-standard requirements (e.g. from ODCA) [1] [1] Open Data Center Alliance, see: Requirements situation at the start of the project Example of knowledge and experience of how “services” are actually delivered contributed by Work Package 4 to Helix Nebula - 6 -

Deliverables and Milestones – Period 1 TypeDel. noNameNatureDissemination level Date Delivered Deliverable4.1Access to the Services Defined for WP5OtherRE17/01/ nebula.eu/index.php/ media- room/videos.html Deliverable4.2 Cloud Provisioning: Case histories of decisions taken ReportPU30/05/2013 Deliverable4.3 Cloud Provisioning Report ReportRE26/06/2013 Milestone9 Supplier workshop to validate inputs and service matching as part of GA1 ReportPU18/12/2012 Milestone10 Workshop to gather learnings and improvement opportunities ReportPU11/03/

Overall modifications, corrective actions, re-tuning of objectives Initially WP4 followed the Waterfall approach as described in the Seventh Framework Programme During the execution it became clear that a more iterative and innovative approach resulted in a more efficient and effective way in product delivery Learning in relative short cycles is an efficient way to deliver products with a higher quality Following an iterative approach means: extra effort is needed to schedule additional product and review cycles extra effort is needed for monitoring the development in requirements (Requirements management) release management must be in place in order to have control on the progress and development of versions of the deliverables

Exploitation and use of foreground Knowledge and experience of how Blue Box services are actually delivered, and documentation of options and recommendations regarding the establishment of a “broker” role, were the main contributions of Work Package 4 to Helix Nebula Documentation of service aspects (see also documents produced by the Service Architecture (ServArch) team) Contributed to the identification and description of the roles in the HN ‘value chain’, elaborated in Work Package 7 (business models) Above mentioned aspects are crucial for the establishment of Helix Nebula as a viable and sustainable service, incorporating both commercial and public contributions - 9 -

Cooperation models Contribution by Work Package 4 to the identification and description of the roles in the HN ‘value chain’, elaborated in Work Package 7 (business models)

Implementation without Blue Box during Proof of Concept Without a Blue Box each customer and supplier have a point-to-point connection resulting in M x N relationships

Helix Nebula Proofs of Concept Learnings from the Supplier perspective The technology works: required workloads from participating scientific organisations can be successfully deployed in cloud environments The Demand sides each had slightly different requirements, and Supply-side clouds all differ significantly in implementation, even those sharing common hypervisors so each user-supplier case was effectively a bespoke deployment Software requirements (“legacy”) from the demand side can be complex and are critical to understanding POC requirements and ensuring a successful outcome A common API is desired and would significantly streamline driving deployments across multiple clouds The ability to move VM images between clouds and potentially convert VM formats is needed for better cross cloud deployment Differences between private and public deployment models on the supply side are important and influence various aspects of PoC’s directly i.e. WAN, open Internet or VPN access Networking between demand-side institutions and supply-side providers, and between multiple supply-side providers, is an important success criteria Areas such as service structure/architecture require further exploration

Proof of Concepts were successful All PoC environments completed successfully by most suppliers Clouds can deliver the facilities required to fulfil the scientific requirements starting at the IaaS level Inter-Supplier communication (open and often) was crucial to success We now/then need(ed) to move on to improving the overall experience making the multi-supplier environment more coherent performance optimisation and price/performance not yet considered Points of learning for further addressing common interfaces: e.g. unified API server image management and conversion, especially to cope with bespoke/ legacy software usage inter-cloud workload assignment and scaling networking: bandwidth and interfaces, charging algorithms breaking the demand-supply scaling “chicken and egg” syndrome common charging and billing mechanisms not yet in sight

Blue Box functions Each customer and supplier have a single connection to the Blue Box resulting in M + N relationships

Blue Box use in pilot Implementing two Blue Boxes in the pilot improves the evaluation by adding practical experience to the study before the pilot

Interaction with other FP7 projects and stakeholders outside the consortium Taking part in the Select Industry Group (SIG) discussions, established by the EC as part of the European Cloud Partnership (ECP) programme. Active member of the Open Data Centre Alliance (ODCA which is developing common user requirements, leading to implementation within standards, for many of these subjects. Use has been made of knowledge gained from the development of other cloud management projects, such as Optimis ( project.eu), BonFire ( and Stratuslab (stratuslab.eu) for interaction with FP7 projects

Contribution to the dissemination of project results Most of the documents produced by Work Package 4 (and the associated documents produced by the ServArch team) have been published on the Helix Nebula web site. Service aspects have been discussed at a number of internal Helix Nebula events and externally, most recently at IM2013 ( WP4 has also contributed to position the HN initiative in several conferences such as EUDAT with data infrastructure providers. These have helped promulgate the services, as opposed to just technology, thinking that is necessary for such developments to succeed. Work Package 4 has also contributed to the Communications Plan and reviewed other deliverables

Summary WP4 very useful for obtaining a clear statement of the Requirements, beyond the technical configurations required Acted as a linking point between gathering requirements (WP3) and flagship deployment (WP5) Highlighted the need for service aspects, rather than technical, aspects, especially as regards federation and brokerage Necessary to act as ambassadors for the need to define a complete Service Architecture, most of which was done outside the scope of this WP Also highlighted the business questions and need for business requirements that arose from WP