1 Multi-Grid and Multi-VO Job Submission based on a Unified Computational Model Krakow Grid Workshop, November 22nd 2005 Trinity College Dublin Mr. Oliver Lyttleton Mr. David O'Callaghan Mr. Gabriele Pierantoni Dr. Geoff Quigley

2 Introduction ● What is WebComG? ● Problem Statement ● A Key Concept: Internal and border regions ● General Architecture ● Implementation and Prototypes – Constrained Submission – Full Interoperability ● Future Work

3 WebComG ● Project comprising software to support development and execution of “Condensed Graphs” in a Grid environment ● Condensed Graph: graph representation of program structure ● Instances of WebComG running on different machines can communicate with each other ● It provides load-balancing, fault-tolerance, resource management, and security

4 Problem Statement ● Our use of heterogeneous grid middlewares within a single execution environment relies upon: ● A Computational Model capable of expressing the workflow of operations which may contain interactions with heterogeneous grid middlewares ● Bridge infrastructures able to communicate with different grid middlewares ● The Condensed Graph computational model is ideal for expressing such complex workflows and its implementation (WebComG) allows their execution. ● Web Services, such as those offered by GT4, are ideal for the implementation of the Bridge Infrastructures.

5 A Key Concept: Internal and Border Regions ● To represent the interoperation between WebComG and Grid Middleware applications a concept of internal and border regions is used: ● An internal region is a set of machines that run either WebComG or a particular Grid Middleware (LCG2 or GT4) ● A border region is a set of machines which must host Border Services. They may host WebComG technologies and Grid Middleware technologies in addition

6 The Border Services ● The Submission Service: Running on a GT4 Container ● The File Closet Service: Running on a GT4 Container ● The Security Service: Running on a GT4 Container ● The Logging Service: Running on a GT4 Container ● The Client Submission Service(s): Use the above Services and submit jobs to the Grid Middleware (LCG2 and GT4)

7 Architectural Overview: Collapsed Borders for LCG2 (More complex Subsets) Border Region Border Region LCG2 Region WebCom Region WebCom Region All three technologies overlap in the collapsed border subset Only two technologies overlap in the simple border subset

8 Border Region Border Region LCG2 Region GT4 Region Architectural Overview: Expanded Borders (Simple Subsets) WebCom Region WebCom Region Submission Service in border set At most two technologies overlap Grid Middleware Client Grid Middleware Resource Broker/ GRAM Server

9 Border Services Choreography SSM WCM GM GMC Grid Job Submission invokes: ● File Staging ● Security ● Logging GM Condensed Graph CG Submission invokes: ● File Staging ● Security ● Logging GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Machine

10 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware Middleware Submission Node

11 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware Job Description

12 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware Job Input Files

13 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware User Credential

14 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware Grid Submission

15 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware Grid Output

16 Border Services Choreography SSM WCM GM GMC GM Condensed Graph GMC: Grid Middleware Client Machine SSM: Submission Service Machine GM: Grid Middleware Grid Output

17 Future Work ● Complete implementation of full interoperability ● Develop interoperability with other Grid Middleware ● Use Grid Filesystem instead of File Closet Service