Grid Standards & Forums, Summary Sathish Vadhiyar

Grid Standardization Efforts - OGSA Web Service - SOAP, WSDL and UDDI Open Grid Service Architecture (OGSA) – standard developed in 2002 by Globus Alliance A service oriented architecture – all entities are services and any operation is performed due to message exchange OGSA builds on web services – aligned with industry-accepted web service standards

Grid Standardization Efforts - OGSA While web services can be permanent, many of Grid components can be transient services Web service standards do not address issues related to basic service semantics: how services are created, how long they live, how to manage faults etc. These are addressed by a core set of standard interfaces – initially OGSI (infrastructure), now WSRF Initially (July 2003) OGSA capabilities implemented by OGSI

Components OGSI defines standard interfaces for Describing and discovering service attributes Describing and discovering service attributes Creating service instances Creating service instances Managing service lifetime Managing service lifetime Naming Naming fault model fault model Service groups Service groups Other issues addressed by OGSA services Authentication Authentication Monitoring service-level agreements Monitoring service-level agreements Managing memberships Managing memberships Integrating data resources into components Integrating data resources into components OGSA schema For providing interoperability between components For providing interoperability between components For describing properties of common Grid entities For describing properties of common Grid entities

Core Components of OGSA Web Services Open Grid Service Infrastructure OGSA services: registry, authorization, monitoring, data access, management etc. More specialized and domain specific services Hosting environmentProtocol Other Schemas OGSA Schemas

OGSA Services – Some More Details Core Services Security Security Policy Policy Messaging, queuing and logging Messaging, queuing and logging Events Events Metering and accounting Metering and accounting Data and information services Data naming and access Data naming and access Replication Replication Metadata and provenance Metadata and provenance Resource and Service management Administration and deployment Administration and deployment Service orchestration Service orchestration

WSRF New alternative to OGSI – May 2004 developed by OASIS group Web Service Resource Framework States of web service – data values that persist across and evolve because of web service interactions WSRF – representing relationships between services and standards in an explicit and a standard manner Stateless web service acting on stateful resources

More on WSRF Standardizing how state is represented and manipulated Answers questions How WS-Resource – (web service + stateful resource) is composed? How WS-Resource – (web service + stateful resource) is composed? How stateful resource is used in the execution of web service message exchanges? How stateful resource is used in the execution of web service message exchanges? How WS-resources can be created and destroyed? How WS-resources can be created and destroyed? How to enable querying of WS-Resource via web service message exchanges? How to enable querying of WS-Resource via web service message exchanges?

OGSI -> WSRF OGSI did not work well with existing web services OGSI models stateful resource as web service coupled with resource state. WSRF separates service and stateful entities acted upon by the service – WS Resource – better exploits existing XML standards and emerging web service standards

Illustration of Complexity – remote job execution with WSRF

Global (Open) Grid Forum Community-initiated forum of thousands of individuals from industry and research leading the global standardization effort for grid computing. Creation and documentation of "best practices" - technical specifications, user experiences, and implementation guidelines. Meets 3 times per year – started in March 2001 – 15 GGFs have been held GGF is responsible for OGSA

GGF Areas and Groups 9 different Areas 2 kinds of groups – working groups and research groups Applications and Programming Models Environment (APME) GridCPR-WG, GridRPC-WG, Astro-RG etc. GridCPR-WG, GridRPC-WG, Astro-RG etc. Architecture (ARCH) OGSA-WG, OGSI-WG, SEM-RG etc. OGSA-WG, OGSI-WG, SEM-RG etc. Data (DATA) DAIS-WG, GridFTP-WG, TM-RG DAIS-WG, GridFTP-WG, TM-RG

GGF Areas and Groups Grid Security (GRID SEC) OGSA AUTHZ WG OGSA AUTHZ WG Information Systems and Performance (ISP) GIR-WG, GB-RG GIR-WG, GB-RG Peer-to-Peer (P2P) OGSA P2P-RG OGSA P2P-RG Scheduling and Resource Management (SRM) DRMAA-WG, GESA-WG, JSDL-WG, GSA-RG, WFM- RG DRMAA-WG, GESA-WG, JSDL-WG, GSA-RG, WFM- RG GGF Market Awareness Committee (GMAC) Grid Research Oversight Committee (GROC)

So, what is left in Grid? Resource and Process Management This is the first area addressed by many projects Addressed by Condor, Globus, Legion, NetSolve and almost all Grid projects Not much left in this area One interesting area left to be addressed is how to combine batch and interactive systems for a single job

Information Services Projects like NWS and Globus MDS address it with LDAP mechanisms Number of projects on how to represent information etc. – This area is loosing steam nowadays Questions regarding how to get the “very recent” information and scalability still remain How to get forecast for a certain period of time in the future?

Scheduling – Problem mapping Has been there for many many years – Number of projects both within and outside Grid for mapping to heterogeneous resources – Condor, AppLeS, NetSolve, APST and many others Still an open problem Many takers for the research Heuristics is the best approach in Grids Rescheduling People agree on general strategies – Condor, LSF, Dome, GrADSolve and other recent ones The main area of interest is how to determine various thresholds automatically Lots of research need to be done regarding how to orchestrate communications during rescheduling

Meta scheduling Projects like Globus do deal with multiple local scheduling mechanisms for a single job Different meanings attached to the term Data Scheduling The coscheduling of data and computation has been there for a long time and many more methods will evolve Data replication, automatic choice of replicas, collating different replica segments for a single problem – much more work to be done, Globus provides only basic methods

Performance Modeling and Prediction of applications Has been there for a long time much before Grids Renewed interest due to running applications on Grid – AppLeS, GrADS etc. A very open area – any solution is not a good solution Coupling performance prediction with scheduling mechanisms and improving over time – a very interesting solution Latency tolerant algorithms Very few isolated projects Need automated techniques to translate a tightly coupled parallel application to a latency tolerant code Crying for help

Application Oriented Grids Many isolated projects – AstroGrid, ClimateGrid, GriPhyN etc. Can be many more for many other applications Thrust should be on developing middleware layers for specific application classes Current focus Simulation, Emulation Good starts by few projects – MicroGrid, SimGridm GridSim Need a standard and uniform simulation and emulation framework A very important area in the coming days

Checkpointing, fault tolerance and migration Many projects regarding checkpointing – CoCheck, MPICH-v, libckpt, SRS May be some active research in checkpointing when dealing with reconfigurable applications and algorithm adaptivity Fault tolerance is assuming more and more importance. Many things have not been addressed including the main one – modeling and predicting faults Economy models Nimrod-G, G-commerce In its nascent stages There is a tacit understanding that it is important. But not sure if we need it now and how critical it is going to be Has started materializing, finally

Schedule Project Presentation and Report Submission – December 18, Tuesday, 11:00 A.M., SERC 202. Sessional Marks soon

Areas Not Covered Grid RPC Ninf Ninf Grid Security SASL SASL Gang scheduling, Coscheduling, LSF scheduling Application Oriented Grid Frameworks Scientific data federation Scientific data federation world wide telescope world wide telescope medical data federation medical data federation bioinformatics research network bioinformatics research network Grid Tools Accessgrid Accessgrid instrumentation and monitoring instrumentation and monitoring Network infrastructure, Production deployment Grid economy Nimrod-G Nimrod-G Peer-to-Peer Grid Services Lessons can be learned in Grid regarding massive scalability, querying distributed information Lessons can be learned in Grid regarding massive scalability, querying distributed information Semantic Grid Chimera-like, NVO-like approaches at a much bigger level Chimera-like, NVO-like approaches at a much bigger level Current Working Grids, National Initiatives TeraGrid, EuroGrid, e-science etc. TeraGrid, EuroGrid, e-science etc. Focus in this class has been more on concepts rather than on information dumping !

Junk!

Manages lifetimes Defines Grid Services – interactions with clients Helped in managing long-lived states of resources Extended WSDL and XML schema Grid Service Handle and Grid Service Reference to address Grid Services

WSRF Partitioning OGSI into a family of separate specifications that allow for flexible compositions To query about the services Asynchronous notification of changes in the states of the resources