Cloud Computing Application in High Energy Physics Yaodong Cheng IHEP, CAS

2 Cloud and Grid workshop, Joint CHAIN/EPIKH School Outline  From Grid to Cloud  Some cloud projects in HEP  Cloud activities at IHEP, CAS

3 Cloud and Grid workshop, Joint CHAIN/EPIKH School Outline  From Grid to Cloud  Some cloud projects in HEP  Activities at IHEP, CAS

4 Cloud and Grid workshop, Joint CHAIN/EPIKH School Terminology  What is a grid?  A platform for scientific collaboration  A scientific tool to help with data manipulation and processing  A computational platform  And how it compares with a cloud?  A source for computing and storage capacity Flexible, easy to access resource  And a cluster?  A building block for both grids and clouds

5 Cloud and Grid workshop, Joint CHAIN/EPIKH School Grid as a collaborative platform  State before the Grid:  Scientists/teams have resources or access to them  Teams are working independently, they do not share their resources (no technology support)  Data sharing very simply, e.g. secure copy (scp) between teams  State with a Grid:  Team’s resources are connected  Sharing is easy  Scientists could focus on their science and not on the technology behind it

6 Cloud and Grid workshop, Joint CHAIN/EPIKH School Grid View

7 Cloud and Grid workshop, Joint CHAIN/EPIKH School Clouds  A rather recent commercial platform  (Large) Pool of virtualized servers  Users submits not jobs, but full virtual machines with jobs inside them  Targets real-time requirements  Fast deployment of new virtual server  Can quickly react on user’s changing requirements  Standard clouds rather simple  Easy to use web interface  No collaboration support (standard Consumer – Provider model, ideal for commercial use)

8 Cloud and Grid workshop, Joint CHAIN/EPIKH School What is cloud  The NIST definition lists five essential characteristics of cloud computing:  on-demand self-service  broad network access  resource pooling  rapid elasticity or expansion  measured service  Three "service models"  software, platform and infrastructure  Four "deployment models"  private, community, public and hybrid

9 Cloud and Grid workshop, Joint CHAIN/EPIKH School Cloud and Grid: A Comparison Grid Middleware Computing /Data Center Cloud Middleware Computing /Data Center

10 Cloud and Grid workshop, Joint CHAIN/EPIKH School From Grid to Cloud  Grid has been the necessary infrastructure for many scientific research, e.g.. HEP  But, there are still some disadvantages  How to schedule jobs efficiently to improve the resource utilization (vs. static policy)  diversified service model on demand (vs. job submission)  Compatible with legacy programs (vs. unified system environment)  …  Virtualization/Cloud is feasible solution

11 Cloud and Grid workshop, Joint CHAIN/EPIKH School Outline  From Grid to Cloud  Some cloud projects in HEP  Activities at IHEP, CAS

12 Cloud and Grid workshop, Joint CHAIN/EPIKH School CernVM  CernVM is a baseline Virtual Software Appliance for the participants of CERN LHC experiments  Motivation  large, complicated Install/update/configure, …  Multi-core with hardware support for virtualization  Using virtualization and extra cores to get extra comfort zero configuration, reduce compiler-platform combinations  CernVM: Build a “thin” Virtual Software Appliance for use by the LHC experiments  provide a complete, portable and easy to configure user environment  independent of physical software and hardware platforms cernvm.cern.ch/

13 Cloud and Grid workshop, Joint CHAIN/EPIKH School “Thin” Software Appliance JeOS (based on rPath Linux) rAArAA KERNEL fuse module FILESYSTEMFILESYSTEM rAA plugin Extra Libs & Apps Cache HTTPDHTTPD Software Repository 10 GB1 GB0.1 GB LAN/WAN (HTTP)

14 Cloud and Grid workshop, Joint CHAIN/EPIKH School CVMFS: CernVM File System App On same host: /opt/lcg -> /chirp/localhost/opt/lcg open(“/opt/lcg”) On File Server /opt/lcg -> /grow/host/opt/lcg Cache Kernel NFSLFSFUSE CernVM Fuse !Cache

15 Cloud and Grid workshop, Joint CHAIN/EPIKH School Bridging Grids & Clouds  Volunteer Computing  uses computers belonging to ordinary people  BOINC  Open-source software for Volunteer Computing and Grid computing  CernVM is extended to support BOINC client  CernVM CoPilot development  Based on BOINC, experience and CernVM image  Image size is of utmost importance to motivate volunteers  Can be easily adapted to Pilot Job frameworks (AliEn,Dirac, PanDA) BOINC PanDA Pilot

16 Cloud and Grid workshop, Joint CHAIN/EPIKH School CernVM CoPilot Architecture

17 Cloud and Grid workshop, Joint CHAIN/EPIKH School lxcloud  CERN Internal Cloud  Highly scalable, Linux (KVM) based cloud-like infrastructure  Optimized for efficiency/speed

18 Cloud and Grid workshop, Joint CHAIN/EPIKH School Resource Pool details  Quattor managed pool of resources  Hardware: (cheap) CPU server type, local disks  LANDB integration  Pre-allocation of VM “slots” in landb  Hypervisor “knows” the name of guests  Disk management  Use of LVM snapshots  All free disk space in one big LV  Pre-stage raw images on LV on the hypyerviors  Fast installation of VMs Using LV snapshots

19 Cloud and Grid workshop, Joint CHAIN/EPIKH School Image management  Central image catalogue (VMIC)  Close collaboration with HEPiX  No direct user access/user images  Images require endorsement by IT  Image distribution system  Image distribution repository of trusted images  Fast distribution using Bit-torrent (rtorrent)  Pull model: Hypervisors ask if there are updates  Transparent update of images using LV tools  Hypervisors advertise existing images

20 Cloud and Grid workshop, Joint CHAIN/EPIKH School Virtual Machine Management  OpenNebula  an open source Cloud Data Center Management Solution  provides a powerful, scalable and secure multi-tenant cloud platform for fast delivery and elasticity of virtual resources  OpenStack  The Open Source Cloud Operating System  The Main components: Compute, Object Storage, Image service  A interesting product worth to be checked

21 Cloud and Grid workshop, Joint CHAIN/EPIKH School Lxcloud ecosystem lxcloud Physical Resource OpenNebula ONE EC2 Interface ONE 3.0 Master Image creation and endorsement Enduser VO Application manager VM Provisioning Image repository (VMIC) Image repository (VMIC) CernVM Golden Nodes Quattor Image creation

22 Cloud and Grid workshop, Joint CHAIN/EPIKH School Clever: A New VIM  CLEVER: A CLoud-Enabled Virtual EnviRonment  To simplify the access management of private/hybrid clouds  To provide simple and easily accessible interfaces to interact with different “interconnected” clouds, deploy Virtual Machines and perform load balancing through migration

23 Cloud and Grid workshop, Joint CHAIN/EPIKH School Clever on Grid Host1 Host Manager (HM) Host Manager (HM) Ejabberd XMPP Server Ejabberd XMPP Server Sedna Distributed Databases Sedna Distributed Databases job Submission CLEVER.jar and X.509 Certificate User Interface Resource Broker CE HM Worker Nodes jobs Running tiny.vdi Host2 Host Manager (HM) Host Manager (HM) HostN-1 Cluster Manager (CM) Cluster Manager (CM) HostN Host Manager (HM) Host Manager (HM) … Matchmaking and Jobs Scheduling Storage Element Computing Element XQuery/XPath Administration tool XMPP

24 Cloud and Grid workshop, Joint CHAIN/EPIKH School Outline  From Grid to Cloud  Some cloud projects in HEP  Activities at IHEP, CAS

25 Cloud and Grid workshop, Joint CHAIN/EPIKH School Virtual Cluster  Motivation  Build Virtual machine pool on physical machines, elastic to expand or shrink on demand  Flexible to support more kinds of applications  Compatible with legacy programs  R&D cloud for users  Key technologies  hypervisors (KVM, XEN, …) evaluation suitable for HEP  VIM management (OpenNebula, OpenStack, …)  Monitoring and accounting  Interface to PBS, WLCG, and other services  dynamic scheduling Live migration VM resource adjustment (CPU, Memory, Network, …)

26 Cloud and Grid workshop, Joint CHAIN/EPIKH School Architecture of Virtual Server PBS Client Physical Machine VM Physical Machine VM VIM (VM create, start, pause, destroy, migration) Scheduler PBS Server Submit Job Query and Modify Queue Scheduling policy Power Management WLCG Grid Job

27 Cloud and Grid workshop, Joint CHAIN/EPIKH School PBS/Torque integration  Each batch queue has basic resources (physical nodes or Virtual machines)  If the jobs are too many in one queue, the scheduler will create some extra virtual machines according with scheduling policy and requirements, then added the new resources into the queue  The queue with higher priority needs more resources, the VM resources in queues with lower priority will be paused, even destroyed  Fair scheduling is very important here!  WLCG interface is simply via PBS/torque

28 Cloud and Grid workshop, Joint CHAIN/EPIKH School GUI

29 Cloud and Grid workshop, Joint CHAIN/EPIKH School BESIII Cloud  Integrated with Grid, volunteer computing, and virtualization  User submits jobs to BESIII portal, then these jobs will be dispatched to different computing resource  Volunteer computing (small sites and personal computers)  Local cluster (managed by LRMS)  WLCG  CNGrid  plugin framework  gLite, PBS, GOS plugins already completed!  Recently, BESIII Offline Software System (BOSS) has successfully run on CernVM-based  BONIC plugin is ready!

30 Cloud and Grid workshop, Joint CHAIN/EPIKH School  is the first volunteer computing platform in China  Use BOINC as its middleware  Launched by IHEP in January 2010  To help scientists from CAS or other research organizations in China to to run their scientific researches on volunteer computing resources  More than 9,000 user, 16,000 computer joined

31 Cloud and Grid workshop, Joint CHAIN/EPIKH School Architecture of BESIII Cloud BESIII portal BOINC Server BOINC Server PBS Server PBS Server gLite WMS GOS Local ClusterWLCG CNGrid Small sites and Personal Computer Plugins (gLite, GOS, PBS, BOINC, …)

32 Cloud and Grid workshop, Joint CHAIN/EPIKH School Future: Cloud-Grid Integration Cloud + Grid Computing Service Catalog Computing center Infrastructure Virtual Client service Web Application Service Compute Service Database service Storage service Content Classification Storage backup, archive… service Job Scheduling Service Collaboration Services Datacenter Infrastructure

33 Cloud and Grid workshop, Joint CHAIN/EPIKH School Thanks! Questions?