Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) The Cirrus & Cumulus Project: Build a Scientific Cloud for a Data Center Lizhe Wang and Marcel Kunze Steinbuch Centre for Computing (SCC) Karlsruhe Institute of Technology (KIT) Germany

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Karlsruhe Institute of Technology (KIT) Cooperation between Research Centre Karlsruhe und Karlsruhe University Largest scientific center in Germany scientists, students Annual budget: > 500 Million Euro R&D focus: Energy research and Nanotechnology > +

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) 3 | Autor | Steinbuch Centre for Computing | Google Trends: Grid Computing vs. Cloud Computing

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) OpenCirrus Cloud Computing Research Testbed An open, internet-scale global testbed for cloud computing research a tool for collaborative research focus: data center management & cloud services Resources: Multi-continent, multi-datacenter, cloud computing system Centers of Excellence around the globe each with 100–400+ nodes and up to ~2PB storage and running a suite of cloud services Structure: a loose federation Sponsors: HP Labs, Intel Research, Yahoo! Partners: UIUC, Singapore IDA, KIT, NSF Members: System and application development Available towards end of the year Great opportunity for Cloud R&D 4April 2, 2014© Hewlett-Packard Company

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Physical and Virtual Resource Sets (PRS, VRS) experiment save/restore

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) A possible Definition of Cloud Computing Definition A computing Cloud is a set of network enabled services, providing scalable, QoS guaranteed, inexpensive computing platforms on demand, which could be accessed in a simple and pervasive way. Functionalities SaaS: Software as a Service HaaS: Hardware as a Service DaaS: Data as a Service PaaS: Platform as a Service IaaS: Infrastructure as a Service

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Anatomy of Cloud Computing Key features User-centric access On-demand service provisioning QoS guaranteed offer Autonomy Scalability & flexibility Enabling technologies Virtualization Service flow/workflow orchestration Web service & SOA Web 2.0 World-wide distributed storage & file system Parallel & distributed programming model

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Clouds vs. Grids: A Comparison Cloud ComputingGrid Computing ObjectiveProvide desired computing platform via network enabled services Resource sharing Job execution InfrastructureOne or few data centers, heterogeneous/homogeneous resource under central control, Industry and Business Geographically distributed, heterogeneous resource, no central control, VO Research and academic organization MiddlewareProprietary, several reference implementations exist (e.g. Amazon) Well developed, maintained and documented ApplicationSuited for generic applicationsSpecial application domains like High Energy Physics User interfaceEasy to use/deploy, no complex user interface required Difficult use and deployment Need new user interface, e.g., commands, APIs, SDKs, services … Business ModelCommercial: Pay-as-you-goPublicly funded: Use for free Enabling technologyVirtualization, SaaS, Web 2.0, Web service, … HPC, Grid infrastructure, middleware, … QoSPossibleLittle support On-demand provisioningYesNo

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Cumulus: A Cloud Computing Prototype

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Re-engineering the Globus Virtual Workspace Service (GVWS) GVWS Limitations Force all backends to install GVWS control agents Cloud users need to select network solutions Cloud users need to prepare VM images Re-engineering the GVWS for the Cumulus Remove GVWS control agents, GVWS frontend talks directly to LVMS, e.g., OpenNEbula Consider the similar scenario: Globus + PBS The forward network requirements to LVMS Prepare VM image via OS Farm for users

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) OpenNEbula as a Backend The Cumulus frontend communicates with OpenNEbula via: SSH XML-RPC Improvement NIS LDAP: shared user management - oneadmin NFS OCFS (Oracle Cluster File System): shared volumes for VM images Can develop other plug-in, embedded in the Cumulus frontend, for other LVMS, e.g., VMware Virtual Infrastructure

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) OS Farm as a VM Provisioning Tool Development of CERN OpenLab OS Farm as server to generate VM images Can accept HTTP requirements, or via wget: wget eate?name=&transfer=http& class=slc_old&arch=i386&filet ype=.tar&group=core&grou p=base A Java client is embedded in the Cumulus frontend to invoke OS Farm service dynamically.

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) The forward Network Solution Users do not need to specify the network configuration of a VM Users only get network access to VM, e.g., IP address or hostname + domain name Cumulus forwards the network requirements to backend (LVMS), e.g., OpenNEbula Network solutions on Cumulus backend: OpenNEbula starts a VM, listens to central DHCP server, arranges an IP address to VM, returns it to Cumulus frontend Set hostname when the virtual machine is created Lease an IP address from the IP pool Cumulus frontend returns the IP address or the host name to the client.

KIT - Die Kooperation von Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) Thank you! Question? Contact: Lizhe Wang, Marcel Kunze Steinbuch Centre for Computing (SCC) Karlsruhe Institute of Technology (KIT) Hermann-von-Helmholtz-Platz 1 D Eggenstein-Leopoldshafen, Germany