Grid Computing & Tuple Space Presented by Nelson Chu

What is Grid? Computing Grid is a very large distributed computing network, Virtual Organizations (VO) Computing Grid is a very large distributed computing network, Virtual Organizations (VO) Grid is an infrastructure to collaborate many diverse computing resources in a network to accomplish a task(s). Grid is an infrastructure to collaborate many diverse computing resources in a network to accomplish a task(s).

What is Grid? Classification: Classification: Computational Grid (eg. WestGrid) Computational Grid (eg. WestGrid) Desktop Grid (eg. Desktop Grid (eg. Data Grid (eg. Earth Systems Grid) Data Grid (eg. Earth Systems Grid) Global Grid Forum (GGF), Globus, etc. Global Grid Forum (GGF), Globus, etc. Open Grid Services Architecture - OGSA Open Grid Services Architecture - OGSA Web Service Resource Framework - WSRF (OGSI) Web Service Resource Framework - WSRF (OGSI) ESG

Fabric: Physical resources Resource & Connectivity Protocols: secured access Collective Services: brokering, monitoring User Applications/Tools Grid Architecture

Fabric layer: resource management e.g. computational resources, storage resources, and network resources. Connectivity layer: Communication and authentication Collective layer: global-naming, brokering, scheduling, workload and workflow management, and data replication. Application layer: user defined tools and application

OGSA Architecture

OGSA provides a service oriented architecture to simplify grid services development and to build a transparent grid structure Physical resources (for example: storage, network, and computer) and logical resources services Web services (including WSRF) OGSA architect services (such as security management, data management, workflow management, file system management, etc.) Grid applications

Research Opportunities: Grid Security Multi-institutional users and heterogeneous resources Multi-institutional users and heterogeneous resources Authentication & authorization mechanisms Authentication & authorization mechanisms Different local name spaces, credentials, and/or accounts Different local name spaces, credentials, and/or accounts Trust mechanisms: domain-based, user-based, service- based, network-based, and scheduler-based Trust mechanisms: domain-based, user-based, service- based, network-based, and scheduler-based Fault Tolerance Detect and notify a fault occurrence Detect and notify a fault occurrence Handle any failure Handle any failure

Research Opportunities: Grid Scheduling Workflows in a service-based environment Workflows in a service-based environment Split and distribute jobs to resources Split and distribute jobs to resources Time, cost, performance, completeness, fairness, effectiveness, and security policies. Time, cost, performance, completeness, fairness, effectiveness, and security policies.  Resource discovery  System selection  Job submission: resources reservation and supporting operation  Monitoring and cleaning up

Current Research Trends Scheduling, Security, Fault Tolerance Scheduling, Security, Fault Tolerance Replica location services Replica location services Metadata catalog services Metadata catalog services Reliable file transfer services Reliable file transfer services Globus Toolkit 3 / 4: Replica Catalog, GridFTP, and (GASS) Global Access to Secondary Storage. Globus Toolkit 3 / 4: Replica Catalog, GridFTP, and (GASS) Global Access to Secondary Storage. Globus, IBM, Oracle 10g, Sun, etc. Globus, IBM, Oracle 10g, Sun, etc.

Research Opportunities: Data Data Versioning Data Versioning Global name space: data characterization, heterogeneous systems Global name space: data characterization, heterogeneous systems Data Security Data Security Fault tolerance: recovery Fault tolerance: recovery New challenges: inter-domains replication, data synchronization, replica registration, querying, and meta-catalog. New challenges: inter-domains replication, data synchronization, replica registration, querying, and meta-catalog.

Grid Environment Resources collaboration Resources collaboration Loosely coupling Loosely coupling Time, space, destination Time, space, destination Communication allows significant degree of interoperability, synchronization, and extensibility Communication allows significant degree of interoperability, synchronization, and extensibility Simple, flexible, and powerful mechanism Simple, flexible, and powerful mechanism

Grid System Open issues: data synchronization, fault tolerance, persistence, concurrency, transparency, etc. Open issues: data synchronization, fault tolerance, persistence, concurrency, transparency, etc. Tuple Space paradigm Tuple Space paradigm Simple operations provide powerful inter- process mechanism for communication and synchronization. Simple operations provide powerful inter- process mechanism for communication and synchronization. Flexible for function expansion: ordered tuple, leasing, … Flexible for function expansion: ordered tuple, leasing, …

Tuple Space Developed in a parallel and distributed programming system, Linda, at Yale U Developed in a parallel and distributed programming system, Linda, at Yale U Basic element ~ Tuple Basic element ~ Tuple Tuple: a vector of typed values / fields / variables Tuple: a vector of typed values / fields / variables Operations: Writing, Reading, Consuming & Operations: Writing, Reading, Consuming & Matching Matching

Tuple Space TS Vs DBS TS Vs DBS SpacesTwo SpacesOne A C Read Take “B” … … … wait/abort Write Processes Producers Consumers

Tuple Space Globally shared memory space (Communication buffer) Globally shared memory space (Communication buffer) Tuples are associatively addressed by matching Tuples are associatively addressed by matching Loosely coupling Loosely coupling Mechanism for inter-process communication and synchronization Mechanism for inter-process communication and synchronization Simple, flexible, powerful, … Simple, flexible, powerful, … ~> Grid Resource Collaboration

Tuple Space on the Grid Service Registration Service Registration Service Discovery Service Discovery Service Lookup Service Lookup Job Scheduler Job Scheduler Data Mgmt Data Mgmt Tuple Spaces (Services / Tasks / Data) Consumers Producers / Processes

Tuple Space on the Grid with Java Technology Grid Enabled Resource Mgmt (GRAM) Security Feature Fault Tolerant Data Mgmt Look up Service Discovery Mgmt Leasing Plug and Work Resources / Services Collaboration

Grid Job Scheduling Eg) WestGrid, … Eg) WestGrid, … Grid I Scheduler/ Resources Broker Resource Meta-Scheduler Grid II Centralized System Load balance Synchronization Fault Tolerance Policy Meta-Scheduler

Tuple Spaces Based Gird Job Scheduler Resource 1 Agent Resource N Agent Registrar Job Desc. Resources Required Job Desc. Resources Required Result SecurityCheck() PolicyCheck() TakeJob() WriteResult() Resources List Register () JobSpace JobSubmit() Registrar ResourcesList() ResourcesListExchange() JobExchange() Domain II Users Figure 1: Tuple Spaces Based Grid Scheduler GetResult() LeaseRenew () Fault Tolerance Management Fault Tolerance Management Leasing Structure for Dynamic Participation Leasing Structure for Dynamic Participation On-Demand Service On-Demand Service Automatic Load Balance Automatic Load Balance Global Name Space Global Name Space Security model ? Security model ?

View 3 View 1 Query … SearchView() GetJobTicket() QueueTicket() GridFTP() / TransferSchedule() SecCheck() PolicyCheck() TakeJob() UpdateView() EventNotify() UpdateData() … PriorityCheck() New Data Derived Data RenewL() TicketBox ViewSite OpenSpace Figure 2: Data Access Management with Tuple Spaces Registrar ResourcesListExchange() Domain II Resource Agent EventNotify() Update() EventNotify() Register() JobExchange () Replica Ticket: View(s)/Query(ies) Certificate User information Job description … Gird Data Management System with Tuple Spaces

References Ian Foster, Carl Kesselman, and Steven Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Intl J. Supercomputer Applications, 2001 Ian Foster, Carl Kesselman, and Steven Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Intl J. Supercomputer Applications, 2001 Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, “The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, “The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, Gregory T. Byrd, and Praveenkumar Cheruvu, “Tuple Space Computing on the Grid”, North Carolina State University Gregory T. Byrd, and Praveenkumar Cheruvu, “Tuple Space Computing on the Grid”, North Carolina State University K.A. Hawick, H.A. James, and L.H. Pritchard, “Tuple-Space Based Middleware for Distributed Computing”, Technical Report DHPC- 128, University of Wales K.A. Hawick, H.A. James, and L.H. Pritchard, “Tuple-Space Based Middleware for Distributed Computing”, Technical Report DHPC- 128, University of Wales Globus, Globus, IBM, IBM,

Discussions Thank you !

Grid DBS Vs Distributed DBS Grid DB System is a Distributed DB System Grid DB System is a Distributed DB System Inter-domains, Policy, Security Inter-domains, Policy, Security Large amount of data intensive users/applications access to large amounts of data (terabytes or petabytes) Large amount of data intensive users/applications access to large amounts of data (terabytes or petabytes) Distributed data population (production, replication) Distributed data population (production, replication) Query processing techniques Query processing techniques Secured and efficient data transport protocol Secured and efficient data transport protocol Dynamic environment and data: Dynamic environment and data: Grid (loosely coupled) ~ DDBS (highly coupled) Grid (loosely coupled) ~ DDBS (highly coupled)