EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Giuseppe Andronico INFN Sez. CT & Consorzio COMETA Workshop Clouds vs Grids Beijing, Introduction

Outline General topics on computing Grid computing Cloud computing Cloud vs. Grid

General topics on computing The introduction of personal computers, the continuous increase in computing power, data storage capacity, network bandwidth and other technological improvements are making computing an ever and ever powerful instrument for scientific research, technical development, business assessments and more, till to impact directly day by day life

General topics on computing Scientific and technical fields –Numerical solutions of very complex problems –Numerical simulation to better understand specific systems –Simulation by agents in fields like social systems, complex systems and more Business fields –Time series analysis of financial data –Market data analysis to extract trends –Market simulation for the introduction of new products

General topics on computing Different range of problems Tiny size: you can solve them on a powerful PC Small size: you can solve them on a small-medium cluster (< than 15 personal computers) with some TeraBytes of storage Medium size: you can solve them with a cluster (till to hundreds of computers or a small supercomputer) with a small SAN (till to some hundreds of TeraBytes) Big size: you can solve them with a serious cluster (till to thousands of computers or a medium supercomputer) with a SAN (till to some PetaBytes) Huge size: you can solve them with thousands of computers (or a very serious supercomputer) and a serious SAN

General topics on computing How the computing complexity increase Linear scaling with the complexity –Trivial or embarrassing parallelism –Nice problems that scales almost linearly (MPI) More than linear (till exponential growth) The big problem of computing infrastructure is the sustainability; once the infrastructure is build then: Electrical power Cooling Maintenance Support are all costs to front

A different point of view “If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility... The computer utility could become the basis of a new and important industry.” John McCarthy, at the MIT Centennial in 1961

Grid computing A first answer was grid computing “ A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. It coordinates resources that are not subject to centralized control using standard, open, general-purpose protocols and interfaces to deliver nontrivial qualities of service.” Ian Foster More than building huge clusters or supercomputers with connected problems, coordinate what each player is able to maintain to front bigger problems

Grid computing Usually the bigger is the problem the bigger is the number of people working on it and of the institutions cooperating. Instead of creating a new computer center with huge computing resources (and huge problems) grid computing is about collaborating and sharing resources: Storage Sensors for experiments at particular sites Application Software Databases Network capacity, …

Grid computing An example is Large Hadron Collider, an experiment producing tens of PetaBytes of data to be analyzed every year. The project involves then of thousands people all around the globe in hundreds of institutions. One option was to create a huge data center at CERN with what this implied in terms of bureaucracy, handling and sustainability. At the end the solutions was to build the biggest grid infrastructure in the world, WLCG. But grid computing is used in industry too: google is the better example

Cloud Computing Cloud computing is a model for enabling convenient, on- demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. European Telecommunications Standards Institute (ETSI) The NIST Definition of Cloud Computing Cloud Computing is a large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet. [I. Foster et al. (2008)]

Cloud computing Characteristics of cloud computing: Virtual –software, databases, Web servers, operating systems, storage and networking as virtual servers On demand –add and subtract processors, memory, network bandwidth, storage.

Cloud computing Virtualization technology for “abstracting resources”. Services to the customers mainly at three different levels: Iaas, PaaS and SaaS. Application OS + App Server Stack Infrastructure IaaS Platform PaaS Software Application SaaS

Cloud computing Why only now? Broadband networks Fast penetration of virtualization technology for x86- based servers –Virtual appliances Adoption of Software as a Service –Salesforce.com –Web 2.0 mindset General purpose on-line virtual machines that can do almost anything

Cloud computing Public Clouds Management of Virtual Machine instances within a proprietary infrastructure. Many different customers can run and control their own applications. Access from a remote interface using a specific protocol. Private Clouds Infrastructure owned by a single organization offering its internal computing resources to local users: do not “sell“ computing capacity. Open Source tools employment, dedicated operating environment offered to local users with high trust level. Hybrid Clouds A private cloud which adds to the local infrastructure more computing capacity with resources coming from an external public clouds. External resources access allowed over the Internet, using remote interfaces.

Cloud computing Some of the typical new service requests at INFN T1 Customer-definable software environments. Setting up dynamic pools of virtual servers (to be used e.g. as front-end pools, or as personal compute nodes) Instantiating pre-packaged, ready-to-go services. Truly distributed, on-demand Cloud storage. Not everybody “speaks Grid”: providing access to distributed, traditional Grid infrastructures as if they were not Grids. This might be offered to non-traditional users as well (e.g. public administrations, or private companies)

Grid vs Cloud Massive scale resource sharing over the Internet, sounds a lot like grid computing, yet the driving force are different hence solutions are different too Grid Highly specialized resources that need to be shared by thousands [of researchers] Large data sets In many cases, providers are also consumers Driven by the need to increase performance (FLOPs) Cloud Reducing CAPEX, OPEX, time to market Millions of users that share to save not for the sake of sharing Providers want market share and customer lock-in Driven by the need to reduce cost (€£¥$)

Grid vs. Cloud Pragmatically Clouds are about provisioning resources Grids are about federating resources Grid computing is more a computing paradigm, while cloud computing is a business model Grid computing can be enabled to provision resources, acting like a cloud system.

Grid vs. Cloud Grids and clouds are not in competition: are different ways to approach similar problems. In some cases are different facades to the same thing The question is: what is better to solve my problem? The target of this workshop is to show use cases in order to clarify this point.