Clouds , Grids and Clusters Course code: 10CS845 Clouds , Grids and Clusters Engineered for Tomorrow Prepared by M .Chandana Department of CSE

Grid and Cluster Computing

What is a Grid? IBM : Early defs: Foster and Kesselman, 1998 “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational facilities” Kleinrock 1969: “We will probably see the spread of ‘computer utilities’, which, like present electric and telephone utilities, will service individual homes and offices across the country.” IBM : “A grid is a collection of distributed computing resources over a local or wide area network, that appear to an end-user or application as one large virtual computing system”

Vision of Grid: Create virtual dynamic organizations through secure, coordinated resource sharing among individuals and institutions. An approach that spans locations, organizations, machine architectures and software boundaries to provide unlimited power, collaboration and information access to everyone connected to the grid The Internet is about getting computers together (connected), grid computing is about getting computers work together. Combine the QoS of enterprise computing with ability to share the heterogeneous distributed resources – everything from applications to data storage and servers.

Grid basics: Grid computing is a middleware software that manages and executes all the activities related to: Identification of resources Allocation and deallocation of resources Consolidation of resources Organizations having under-utilized or over-utilized resources need a dynamically equitable distribution of resources.

Server Hardware

The Data Centre Before the data centre concepts came, organizations maintained own servers and specialized software. This approach was expensive and redundant Data Centres shared resources with organizations. Organizations connected to a data centre may not be able to use resources from other data centres Concept of grid computing enables multiple data centres (same or different organizations) to be networked and shared. Grid is a combination of: Distributed computing High Performance computing Disposable computing Grid provides a metacomputing environment, which can be a megacomputing facility for the users. Grid provides computational utility to its consumers

Cluster Computing and Grid computing Clusters are aggregations of processors in parallel configurations. Resource allocation is performed by a centralized resource manager and scheduling system. All nodes of a cluster work cooperatively together, as a unified resource. Grid has resource manager for each node. Grid does not provide a single system view. Some grids are collections of clusters. Example: NSF Tera Grid

Metacomputing Metacomputing is all computing and computing-oriented activity which involves computing knowledge (science and technology) utilized for the research, development and application of different types of computing. ---- Wikipedia Use of powerful computing resources, transparently available to the user via a networked environment is Metacomputing. Three essential steps to achieve goals of metacomputing are: To integrate the large number of individual hardware and software resources into a combined networked resource To deploy and implement a middleware to provide a transparent view of resources available To develop and deploy optimal applications on the distributed metacomputing environment to take advantage of the resources.

Challenges in metacomputing – -Viability of the linking speeds for realistic application execution -ability and feasibility to execute parallely the components of an application Resources and originating points of data are geographically distributed – may need to processed in a distributed manner Metacomputing is useful when a single point usage is required for large remotely located resources. Metacomputing encompasses two broad categories: - Seamless access to high performance -Linking of computing resources, instruments and other resources.

Metacomputer composition Metacomputer is a virtual computer – its components are individually not important Metacomputer consists of: -Processors and memory Single virtual view of several (large number) of processors and their associated memory units -Network and communication software Interconnected network of physically distributed processors High bandwidth and low latency is required to provide rapid and reliable communication -Remote data access and retrieval Date sizing upto petabytes. Retrieval, replication and mirroring support. Ability to manage and manipulate large quantity of remote data -Virtual environment A software like an operating system, that can configure, manage and maintain metacomputing env.

Evolution of Metacomputing projects FAFNER (1995) - (Factoring via Network-Enabled Recursion) Finding factors of large numbers parallely, over a large network of mathematicians. Started by Bellcore Labs, Syracuse University To distribute the code for factorizing and related information data I-WAY (1995) - Information Wide Area Year Experimental high-performance network, linking many servers and addressed virtualization environments

Scientific, Business and e-Governance Grids Grid computing approach helped to all computing communities – businesses, scientific research and government applications. Scientific grids – users belong to only scientist groups Business grids – users may belong any citizen groups using business services. The number of users in Businesses and e-Governance are high – hence setting up such girds are more complex The user interfaces, access speeds and data sizes will be large.

Web Services and Grid Computing Users of business and e-Governance grids will need we services over internet Users of business grids will not be interested in hardware and software locations They are not interested in resource allocation management as well. Hence the need for integrating web services with grids.

Business computing and the Grid – a Potential Win-win situation Grid was initially utilized for applications such as: weather forecasting models, molecular modelling, bioinformatics, drug design, etc By harnessing the grid approach businesses can achieve cost reduction and better QoS. Grid leverages its extensive information capabilities to support the processing and storage requirements to complete a task. Hence grid can provide the maximum resource utilization, providing fastest, cheapest and maximum satisfaction.

The grid computing for business is based on three factors: The ability of grid to ensure more cost-effective use of a given amount of computer resources A methodology to solve any difficult or large problem by using grid as a ‘large computer’ All the computing resources of a grid such as CPUs, disk storage systems and software packages can be comparatively and synergistically harnessed and managed in collaboration towards a common business objective.

E-Governance and the Grid Service oriented architecture OGSA – Open grid services architecture Globus toolkit

References: Grid and Cluster Computing – C.S.R. Prabhu, PHI , Jan 2012