Grappa Grid Access Portal for Physics Applications CHEP 2003 UCSD March 24-28,2003 Daniel Engh (UC), Shava Smallen (IU), Liang Fang (IU), Jerry Gieraltowski (ANL), Rob Gardner (UC), Dennis Gannon (IU), Randy Bramley (IU)

Grappa: A Web Portal GRAM Services -compute sites GRAM Services -compute sites GridFTP Services -Athena Libraries -packaged physics apps -data storage GridFTP Services -Athena Libraries -packaged physics apps -data storage Web Services -Magda Catalogue -Ganglia Monitor -ATLAS web services Web Services -Magda Catalogue -Ganglia Monitor -ATLAS web services Grappa Portal BrowserCommand line The “Grid” Replica Locator Service The Grappa portal is a web portal designed to provide to the user an interface that coordinates and integrates the use of a diverse set of web and grid services

Grappa History First Design proposals (2001) Implementation in XCAT Science Portal (2002) Demos for ATLAS (2002) –To U.S. ATLAS grid testbed sites –To joint iVDGL/DataTAG WorldGrid testbed IST/SC 2002 demos (Nov 02) Jetspeed implementation (Mar 03)

Grappa Component Layers user Xbook (notebook scripts) –scripts customized for the user (as simple or complex as desired) –conform to the xbooks API –Athena xbook customized for ATLAS job submission –Many scripting languages supported (Jython, Python, Perl…) Xbooks server –A Jetspeed portlet –manages requests from user xbook Jetspeed: Apache-Jakarta web portlet framework Tomcat server: provides the web server Grid Services BrowserCommand line

Athena Xbook HTML form interface –Organizes input parameter selection –Menu selection of grid services Jython scripts –Python syntax –Java toolkits –Java CoG Implementation of Globus Client tools –XCAT technology available Toolkit to componentize applications Implementation of DOE- CCA spec. as web services Currently not used (current demo has only 1 element) Athena Xbook customized: –Treat Athena (ATLAS) as an external app (don’t integrate it into portal) –Prestage xbook/ CoG/ Athena libraries, input data –Submit jobs via GRAM –Transfer results to storage site –Monitor/cancel/resubmit jobs. Submission parameters managed by xbooks manager GRAM IDs managed by Athena xbook

Xbooks Architecture The user’s xbook (notebook scripts) interacts with the xbooks server layer –Contacts xbooks server, registry, and manager –Creates an instance of the user’s xbook. See refs for xbook API

Xbooks Manager The xbook manager dispatches requests –finds the proper script engine –Launches script engine (e.g. JythonEngine) –Records submission parameters in xbook repository –Generates application instance (e.g. job submission requests)

Design Factors: Jetspeed Apache-Jakarta Jetspeed framework –Open Source –A large mainstream effort Hundreds of developers Solid support community for portlets –Portlets componentize the portal interchangeable among projects Proxy Manager could be same for LCG, Chem portals, etc. GridFTP portlet Xbooks server is a portlet, easily integrates user scripting with wide availability of portlets –World moving in direction of Grid/Web services We want to come along Don’t want to develop everything ourselves See refs for Jetspeed API

Design Factors: Java Jython chosen as the demo Athena xbook scripting language –Java toolkits provide lots of stuff –Java CoG provides a java implementation of Globus client tools within the xbook. –Globus 3.0 & OGSA moves further in the java direction –Java CoG part of Globus 3.0 Python could be used as well. –pyGlobus CoG available--provides interface to a local installation of Globus. –Swig to integrate C/C++ toolkits –Python CoG also to be part of Globus 3.0..

Design Factors: Portals Portals vs. GUIs –Not interchangeable approaches –A portal can have an integrating function Integrate vs. Interface –Functionality built into portal? –Interface with functionality found external to portal (e.g. in web services)? –Integrate the use of interfaces (integrate the service clients)? –Today, many believe a philosophy of distributing functionality as grid components and services to achieve greater: Portability Scalability Reliability –No single answer. Need to decide for your app what needs to be integrated into your portal or notebook (xbook) and what needs to be interfaced. –Portal framework not the framework for all your components.

Status and Future Grappa Status –Several demos to date –Basic job submission and monitoring –Now based on standard Jetspeed framework (XCATSP-use discontinued) Large jetspeed community Moving in direction of Grid Components Grappa Future –Parameter tracking and workflow generation Chimera Virtual Data System –Portlet development and integration –Develop/wrap/access Physics-apps as Grid Components & Services –Interaction with other portal projects (eg. Ganga)

References Grappa: grid.uchicago.edu/grappa/grid.uchicago.edu/grappa/ GriPhyN: Python CoG: Jakarta Jetspeed: jakarta.apache.org/jetspeedjakarta.apache.org/jetspeed Xportlets webpage: ATLAS webpage: atlasexperiment.orgatlasexperiment.org Java CoG: Gregor von Laszewski, Ian Foster, and Jarek Gawor. CoG Kits: A Bridge Between Commodity Distributed Computing and High-Performance Grids. In ACM 2002 Java Grande Conference, June pp3,22. Jython: XCATSP: Sriram Krishnan, Randall Bramley, Dennis Gannon, Madhusudhan Govindaraju, Rahul Indurkar, Aleksander Slominski, Benjamin Temko, Jay Alameda, Richard Alkire, Timothy Drews, and Eric Webb. The XCAT Science Portal. Proceedings of Supercomputing p4. XCAT 2.0: Design and Implementation of Component based Web Services. Technical Report 562, Indiana University Computer Science Department, June p22. Chimera-VDS: Ian Foster, Jens Voekler, Michael Wilde, and Yong Zhao. Chimera: A Virtual Data System for Representing, Querying, and Automating Data Derivation. 14 th International Conference on Scientific and Statistical Database Management, p22. www-unix.griphyn.org/chimera/ www-unix.griphyn.org/chimera/ XBOOKS: Extreme! Computing Lab

Glossary Component: A small binary object or program that performs a specific function. The component is designed to easily operate with other components and applications. DOE-CCA: DOE Common Component Architecture, a specification for designing components. Chimera-VDS: Virtual Data System developed under GriPhyN, at U.Chicago. A system for tracking file, parameter, and executable dependencies used to construct workflows. CoG: Commodity Grid, user interfaces to Globus –Java CoG, a java implementation of globus client tools –Python (pyGlobus), interface to local globus tools Grappa: Grid Access Portal for Physics Applications, an integration of scripting notebooks (or “xbooks”), an Xbooks server portlet, Jetspeed, and Tomcat. GriPhyN: Grid Physics Network

Glossary Jetspeed: Apache portlet framework for designing web services as interoperable components of a common framework Jython: Java+Python scripting language. Uses python syntax, interprets python and java code Tomcat: Web Server OGSA: Open Grid Services Architecture (GLOBUS) Portal or Web Portal: a web site that provides services. Portlet: Similar to a Portal but reduced in scope and composed as a web service component to interoperate with other portlets. RLS: Replica Locator Service (GLOBUS) SOAP: Simple Object Access Protocol, XML-based messaging protocol used to encode the information in Web service request and response messages before sending them over a network.

Glossary Xbooks: Jetspeed Portlet for managing user scripts (notebooks) XCAT: Ind.Univ. Extreme! Computing Lab’s Implementation of CCA as web services XSOAP: I.U.E!C.L. Implementation of SOAP for XCAT XCATSP: I.U.E!C.L. Portal framework for XCAT (now replaced with Jetspeed) Xportlets: I.U.E!.C.L. Jetspeed portlets. Workflow: Series of tasks needed to produce a result, this may involve tracking software component dependencies, analysis parameters, etc. See Chimera-VDS toolkit.

Appendix: Screenshots Screenshots of grappa portal showing: –Web access –Proxy management –GridFTP browsing –Xbooks pages Job submission Job monitoring –Some more Portlets –Adding portlets to portal.