Grid Computing School of Engineering Young Joo, Han

Contents Grid Computing Comparison with other tech Grid Architecture –Fabric layer –Connectivity layer –Resource layer –Collective layer –Application layer Grid Category –Computational Grid –Data Grid –Access Grid

Software Trends Time (years) scale Structured programming Object-oriented programming Component programming monolithic Client-server Classes Multi-tier Server-side Grid Computing Application complexity

Grid computing offers a model for solving massive computational problems by making use of the unused CPU cycles of large numbers of disparate, often desktop, computers treated as a virtual cluster embedded in a distributed telecommunications infrastructure.clusterembedded telecommunications The coordinated, transparent and secure sharing of IT resources across geographically distributed sites Grids are built on standards-based interoperability to deliver integrated solutions that allow virtual collaboration Grid Computing Virtual Servers, Storage and Instruments Grid Middleware Distributed Physical Servers and Storage

Grid Enable Source: IBM Sharing of Resources – Virtual Organizations and Collaboration Management of Resources - Quality of Service and Optimization Access to Resources - On Demand Computing and Utility Models

Grid Technology Natural evolution of distributed systems and the Internet. Middleware supporting network of systems to facilitate sharing, standardization and openness. Infrastructure and application model dealing with sharing of compute cycles, data, storage and other resources. Move towards delivering “computing” to masses similar to other utilities (electricity and voice communication). Currently used for high performance computing however the trend is towards Service Oriented Applications (SOA).

Comparison (Grid vs WWW) GridWWW Transmission Method Peer to PeerServer to Client Transmission Speed G bps ~ T bpsK bps ~ M bps SharingLarge scale DB Distributed super com. Hyper Text based information BrowserJACO3, TENT, etcIE, Netscape MiddlewareGlobus, LegionServed by application

Comparison (Grid vs P2P) GridP2P ConnectivityCollaborationPoint to Point CommunityLimited user groupOpen NetworkHigh speed networkInternet Open strategyStorage and computing power share File share

Grid Architecture

Open Grid Services Architecture (Integration of Grid and Web services) Applications on demand Secure and Universal access Business integration shareaccess manage Resources On demand Global accessibility Vast resource scalability Continuous Availability Web Services Grid Protocols

Layered Grid Architecture (By Analogy to Internet Architecture) Application Fabric “Controlling things locally”: Access to, & control of, resources Connectivity “Talking to things”: communication (Internet protocols) & security Resource “Sharing single resources”: negotiating access, controlling use Collective “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Internet Transport Application Link Internet Protocol Architecture Slide courtesy of C. Kessleman Cal(IT)2 Presentation

Layered Grid Architecture Application Collective Resource Connectivity Fabric Internet Link Transport Grid Protocol Architecture Internet Protocol Architecture

Fabric Layer Fabric layer: Provides the resources to which shared access is mediated by Grid protocols. Example: computational resources, storage systems, catalogs, network resources, and sensors. Fabric components implement local, resource specific operations. Richer fabric functionality enables more sophisticated sharing operations. Sample resources: computational resources, storage resources, network resources, code repositories, catalogs.

Connectivity Layer Application Collective Resource Connectivity Fabric Grid Protocol Architecture Nexsus Interface Grid Security Infrastructure GSI

Connectivity Layer Communicating easily and securely. Connectivity layer defines the core communication and authentication protocols required for grid-specific network functions. This enables the exchange of data between fabric layer resources. Support for this layer is drawn from TCP/IP’s IP, TCL and DNS layers. Authentication solutions: single sign on, etc.

Resource Layer Application Collective Resource Connectivity Fabric Grid Resource Information Protocol (GRIP) GridFTP Grid Resource Access Management (GRAM) Grid Protocol Architecture Grid Resource Registration Protocol (GRRP) Data Transfer Grid Information Services Resource Management

Resources Layer Resource layer defines protocols, APIs, and SDKs for secure negotiations, initiation, monitoring control, accounting and payment of sharing operations on individual resources. Two protocols information protocol and management protocol define this layer. Information protocols are used to obtain the information about the structure and state of the resource, ex: configuration, current load and usage policy. Management protocols are used to negotiate access to the shared resource, specifying for example QoS, advanced reservation, etc.

Collective Layer Application Collective Resource Connectivity Fabric Data Replication Services Directory Services Grid Protocol Architecture Monitoring Services Scheduling and Brokering Services

Collective Layer Coordinating multiple resources. Contains protocols and services that capture interactions among a collection of resources. It supports a variety of sharing behaviors without placing new requirements on the resources being shared. Sample services: directory services, coallocation, brokering and scheduling services, data replication service, workload management services, collaboratory services.

Applications Layer These are user applications that operate within VO environment. Applications are constructed by calling upon services defined at any layer. Each of the layers are well defined using protocols, provide access to useful services. Well defined APIs also exist to work with these services. A toolkit Globus implements all these layers and supports grid application development.

Grid Category

Categorization Ambient mic (tabletop) Presenter mic Presenter camera Audience camera High-Throughput Computing Grid System Computational Grid Data Grid Access Grid Distributed Super Computing On Demand Cooperative work Environment Multimedia

Computational Grid Computational grid is a hardware and software infrastructure that provide dependable, consistent, pervasive and inexpensive to access to high-end computational capabilities Infrastructure because computational grid is large-scale pooling of resources, whether compute cycle, data, sensors, or people Computational grid is analogous to electric power grid

Example (Butterfly.net) Massively Multiplayer Online Game Very complex, wide virtual world, many buildings, rooms, polygons, textures, characters, and so on… Millions of users invests enormous amounts of time, energy, money. $10 billion in 2001 to $18 billion in 2005 (US only) Lineage (NCSoft, South Korea). The biggest MMG game in the world, the most number of users. (1.2 millions users pay for it each month.) EverQuest(US), a half-million subscrivers.

Technical problems Inherently issuing problems –MMGs are often unavailable for hours at a time –MMGs suffer from lag –MMGs offer a minimal set of interactions. Reasons –Difficulties on development network-based game VS. standalone PC game –“Client-server protocol –Difficulties on load estimation –Server maintenance and reconfiguration

Solutions Low cost and high performance equipment –Intel processor + Linux OS Distributed game server architecture on Grid environment and multicast-mesh over User Datagram Protocol(UDP) –Security, Reliability, Scalability, Load balancing, high speed and high quality game, –No down time, No lag –Cost effective, –No risk

Data Grid A data grid is a grid computing system that deals with data - the controlled sharing and management are large amounts of distributed data. These are often, but not always, combined with computational grid computing systems.grid computingcomputationalgrid computing These applications require widely distributed access to data by many people in many places. The data grid creates virtual collaborative environments that support distributed but coordinated scientific and engineering research Example: Medical X-ray DB(telepacs system)

Access Grid Access Grid is a collection of resources and technologies that enable large format audio and video based collaborations:audiovideocollaborations The Access Grid is an ensemble of resources including multimedia large-format displays, presentation and interactive environments, and interfaces to Grid middleware and to Visualization environmentsmultimediamiddleware Visualization

Example (Share of 3D Objects & Collaborative Interaction) KIST Screen Remote Screen

Shuyukan High School, Japan

Grid Past, Present, Future

Past –Origins and broad adoption in eScience, fueled by open source Globus Toolkit Present –Rapidly growing commercial adoption –Open Grid Services Architecture (OGSA) Future –Key enabler of new applications & industries based on resource virtualization and distributed service integration

Reference Analysis of state Management in Web Services and Various Extensions (Ai Ting, Wang Caixia, Xie Young) Grid Computing (Wieley) Etc.

Q & A