Global Grid Forum Applications, Programming Models & Environments Area Overview Mary Thomas San Diego Supercomputer Center Presented at the Asia Pacific Grid Workshop 2001 October 22-24, 2001 Tokyo, Japan

AP&M Area Contains 5 active research groups: Advanced Collaborative Environments (ACE) Advanced Programming Models (APM) – Satoshi Matsuoka Applications & Testbeds (APPS) -- Ed Seidel Grid Computing Environments (GCE) Grid User Services (GUS) No Working Groups yet, some proposed

Advanced Collaborative Environments (ACE) n Research Group Chair(s): n Rick Stevens n Jason Leigh n Mike Papka n Website: n New RG formed last summer n Statement of purpose: The GGF Advanced Collaborative Environment research group's charter is to investigate human- centered techniques and technologies for facilitating interactive, collaborative, and immersive access of Grid resources from any where and at any time.

ACE Goals Provide a venue at which researchers can share late-breaking ideas and results. Encourage information and code sharing by proposing and publishing strategies for interoperability. Formalize human factors techniques for optimal operation of interactive, collaborative, immersive and ubiquitous environments. Provide a systematic process through which researchers may contribute their work to the community. Identify future areas of research that will be important to the continuing advance of the Grid.

ACE “Domains” Collaboration Environments - Access Grid, Access Grid Augmented Virtual Environments (AGAVE) Tele-Immersion - CAVE, IDesks, Tiled Displays, auto stereoscopic displays, software frameworks Distributed Realtime Visualization - crossplatform, cross cluster scene graph libraries, parallel rendering techniques, video and display list streaming Computational Steering - software frameworks Collaborative Gaming - things we can learn from realtime online games and communities Ubiquitous Computing - paradigms for VR to desktop to laptop to PDA collaboration and other human-in-the-loop Grid applications

Grid User Services (GUS) Research Group Chair(s): John Towns, Website: Statement of purpose: The User Services Working Group fosters a common understanding of user and support staff requirements in a grid environment, acts as a venue for sharing resources, and facilitates communication for grid activities between users, support staff, and developers. Recent key activities: Grid User Services Best Practices doc (Lead: John Towns) Trouble Ticket Interchange (Lead: Hank Laughlin) Infrastructure Requirements for Grid Sites aka the Grid Constitution (Lead: George Myers) Services and Tools Requirements for Effective Grid Support Services (Lead: Don Frederick) - Awaiting first draft.

Grid Computing Environments (GCE) Research Group Chairs US: Geoffrey Fox, Mary Thomas, Dennis Gannon, EU Co-chair: Rob Allen (Univ. of Manchester) Website: Statement of purpose: The GGF Grid Computing Environment research group is aimed at contributing to the coherence and interoperability of frameworks, portals, PSE's, and other Grid-based computing environments by establishing standards that are required to integrate technology implementations and solutions. Recent key activities: Formation of new working groups Web Services WG Web Flow Meta Data UK activities added Testbed Project

GCE/Web Services/Testbed Today I will focus on defining web services Why are they useful for portals? Why are they useful for grid services? GCE Web Services Testbed plans

Grid Portals: the Problem Example: portal or applications need to perform grid tasks for any arbitrary user, on any arbitrary resource, and span all ‘layers’ of the grid portals must be ‘aware’ of resources (use GIS) What grid services are running on that resources: Globus/Legion/VegaGrid/SSH, etc GIS GSI/Kerberos, MyProxy Request syntax differs for each resource: GRAM/Legion/SSH/MAUI/PBS/Others Portal must have permission to use/access for user (GSI, MyProxy)

Grid Portals: Complexity Grows Presents a huge complexity problem that does not scale Portals interact with/integrate all layers: GIU/Client interface Uses all middleware services (Globus, SRB, GSI-FTP, etc.) Each portal in the world must store and configure same data Repeated data, open to errors, variations Multiple programmers repeating same tasks and implementations Much portal software is “hard-coded” and not dynamic The Grid is international, need for scaleable, interoperable services Too much ‘hard-coding’ needed at this time (big issue for Portals)

Grid Portals: Example “Simple” example all portals face: authentication Portals represent user, must act as a proxy Complex for single user: Each user needs an account/allocations on each machine Each user needs certificate signed by a CA Each CA must be recognized by resources as valid Each users DN must be placed into mapfile on each resource to be used: Large burden on local administrator Portals are gateways: large number of resources, always changing large numbers of users (GryPhyN ~ 2000) Some grid software must be configured by local site administrator to accept portals as proxy (e.g. MyProxy) Complexity is one represented as a fully connected, N-dimensional, graph

Web Services: a Proposed Solution Web services architecture provides mechanisms for dynamic service discovery (UDDI) Separation of implementation from function (WSDL) Know protocol (SOAP/HTTP, SOAP/RPC) Service provider encapsulates implementation details Client does not need to know details, just where to send the request Challenge will be discovery  problem with Jini/CORBA Commercial world developing web services technologies in P2P world: Implies funding/support rapid development/technology advancement Caution: this does NOT imply cohesiveness or standards Note: in some ways, Globus/GRAM is a web service Advantage: language independent, so can run on any system We are pursuing Perl, Python, Java, C++ at this time

Testbed Architecture Will include the following: XML schemas for language/description SOAP Protocol (Simple Object Access Protocol) Over HTTP as transport protocol/RPC WSDL Interface for discovery (Web Services Description Language) UDDI (Universal Description, Discovery and Integration) Go here to find who/what/where/why/how Security model – Need to support single login Security at all levels Adopt “anatomy of the Grid model” virtual organizations Portals be built as services in addition to applications

How will Portals use Web Services? Portals need to worry about when to use XML, and when to use other protocols: eg for large files, need to use gtp Use XML to find files, and then have suggested protocols for moving them “Beep” low level protocol for describing way data is encoded; can carry XML, and XMLP Block Extensible Exchange Protocol XMLP – standardization of SOAP – may subsume SOAP A W3C project – mainly commercial world, Suggested GF doc: talking about XML based interoperability protocols, and when to use/not use XML. Describe experiences using them. Should portals create and instance of UDDI, and how does this related to existing information service dicsovery such as GIS? May need a bridge between UDDI and lDAP or others. Think about bridges (or wrappers) to data for protocols – Separate protocol from services from data

NPACI GridPort Architecture

Portal Web Services Architecture

Clients Application Portals Grid Web Services Grid Services Collective and Resource Access Resources Compute (many) Storage (many) Communi- cation Instruments (various) GRAM GridFTP Replica Catalog / Management Grid Monitoring Architecture Grid Information Service Web Browser Grid Web Service Description (WSDL) & Discovery (UDDI) Grid X.509 Certification Authority SRB/MCAT Condor-G CORBA MPI Secure, Reliable Group Comm. Discipline / Application Specific Portals (e.g. SDSC TeleScience) Problem Solving Environments (AVS, SciRun, Cactus) Environment Management (LaunchPad, HotPage) Job Submission / Control File Transfer Data Management Credential Management Monitoring Events Workflow Management other services: visualization interface builders collaboration tools numerical grid generators etc. Apache Tomcat&WebSphere &Cold Fusion=JVM + servlet instantiation + routing CoG Kits implementing Web Services in servelets, servers, etc. Python, Java, Perl, etc., JSPs format to html composition frameworks XML / SOAP over Grid Security Infrastructure Grid Protocols and Grid Security Infrastructure Apache SOAP,.NET, etc. …… http, https

Proposed Set of Web Services Information Services Jobs Users Machines Jobs Job Submission (Atomic) Batch script builder Job Management Job Control Job History  uses events Job Status Applications Data Management FTP Collections (SRB) Grid Messaging (atomic) Events Monitoring GMA (Generic Monitoring Framework) Network/Performance NWS Performance Accounts/Allocations Scheduling Security Single login env CA * SC01 Demonstration

What is Required to Participate? Goal: Drive development of interoperable technologies for portals and the services needed by them Make portal middleware programming task easier -- demonstrate that applications can use the services provided Commitment at different levels: Participation: Agree to use WSDL’s and protocols adopted by testbed Development: Agreement to support personnel to develop web service $$Funding$$ Evidence of support from Organizations, Institutions, Funding Organizations Principal need: resource sharing agreements Informal, limited scope/user group Only for testbed purposes

Testbed Participants Initial List is limited US: PACI (NPACI, Alliance, PSC, DTF) NASA/IPG PNNL LBL/DOE Asia: apGrid members Europe: Daresbury (UK) EPCC Cactus Italy

What’s Next? This is an experiment, not a drive towards standards (yet) Meet at SC01 – BOF on Thursday, 5:30 pm Discuss web services and testbed Discuss formally by GCE list Testbed doc will circulate between now & GGF4 (Montreal) Formal session at GGF4: New web services working group Testbed – Agree to goals, protocols, standards, etc. Other mechanisms: Use AG nodes to meet monthly at start. GCE Research Group: Website: