EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space Cracow Grid Workshop’10 Kraków, October 11-13, 2010 Interoperation between production grids using high-level environment Bartek Palak Poznan Supercomputing and Networking Center

2 Outline  Availability and accessibility  Interoperation vs. Interoperability  Guiding principles  Main features  System architecture  Migrating Desktop functionality  Kepler workflow system  Users and developers support  Conclusions

3 Availability and accessibility  Significant improvement of a computing power availability over the last years  Accessibility of the resources, understood as the ease of use experienced by users, is still not at a satisfactory level  diversity of environments  various concepts and architectures  deep knowledge required to access the complex resources  lack of interface standards (or their implementations)  especially annoying when the nature of the problem being solved, forces the user to use computing resources of various kinds.  often discourages users who are non-experts in the technology and systems being used.  The need for high-level environments that provide:  user-friendly access to heterogenous resources  seamles co-operation of underlying middleware solutions

4 Interoperation vs. Interoperability  Interoperability is the goal of many projects: EMI, OMII-EUROPE, ARC,…, but:  Working interoperation solutions are at one's fingertips:  Examples: gEclipse, GridSphere, P-Grade, Migrating Desktop,… Interoperation is the setup of ad hoc components and methods to make two or more systems work together as a combined system with some partial functionality […]. Interoperation is usually performed when the systems having to be combined were designed before standardization (for example legacy systems), or when standard compliance is too expensive, too difficult, or immature. Interoperability theoretically permits any number of systems, compliant to a given standard, to work together […] as a combined system with the full functionality described by the standard.

5 Migrating Desktop (1) The idea: „To bring heterogenous computing resources to user’s desktop and allow users to focus on their research” Guiding principles:  To hide the complexity of the computing environment under the advanced graphical user interface  To provide a unified view of the infrastructures used at the interoperation level

6 Migrating Desktop (2) Main features:  Single sign-on (x.509 certificates, VOMS extensions)  The job’s life-cycle handling – from the job definition and submission, up to obtaining results and the visualization of the job outcome Support for batch and interactive jobs (i2glogin) Handling of sequential and parallel applications  Easy management of files  Open architecture based on idea of plug-ins  Web application accessible independently of the user location  Most of applications need no integration to be run  Single sign-on mechanism Successfully used in several projects:

7 Interoperation  The environment acts as a bridge between the middleware stacks to ensure a seamless interoperation between underlying middleware of various kind  Interoperation ensured by the development of middleware plug ‑ ins:  Plug-ins provide a unified interface to the upper layers of the middleware stacks  The implementation of those plug-ins requires, in most cases, either the integration of existing 'local' standards or re-using the libraries which abstract various infrastructure implementations (e.g. Vine Toolkit).

8 System architecture  Client-server application  Migrating Desktop Client  Network application  Rich GUI java client  Based on OSGi plug-ins  Kepler workflow orchestrator  Roaming Access Server  Based on web services  One point access to various infrastructures  Supported infrastructures gLite, Unicore, i2g (int.eu.grid)‏

9 File management  Capability of file exchange among different kinds of storage seen as abstract file systems  Intuitive file manager - GridCommander  Easy access to files resources  Transparent support for LFC, FTP, GridFTP, SFTP, etc…  Provided also as an independent web application  Built-in file viewers and editors  Open architecture (plug-ins)

10  Job submission to any of the supported infrastructures  Tool for easy job defining - Job Submission Wizard  Specifying general job parameters  Defining specific job parameters  Choosing input/output files  Support for interactive jobs Job defining

11 Job monitoring  Unified view of jobs run on different infrastructures  Intuitive tracking of submitted jobs - Job Monitor  Real time steering of interactive jobs  Processing of job outcome  Access to partial results  Visualization of output files

12 Kepler workflow system  KEPLER workflow system  a free-software system for designing and executing scientific workflows  Alternate client to Migrating Desktop Platform  Integration works performed within The EU Euforia project  Objectives of integration:  Transparent scheduling of scientific workflows on the grid and HPC infrastructure  Execution of interactive tasks on grid

13 Users & developers support  Web pages:  Product information  Succesful use cases  Manuals  Installation guides  Aplications enhanced integration procedures  Tutorials  Support for educational events (like summer schools)  HelpDesk  Mailing list:  Bugtracking system JIRA:

14 Conclusions  „Availibility” not always means „accessibility” of resources;  The need for high level tools providing seamless access to heterogenous resources  Interoperation - a temporary solution on the way to full interoperability  Migrating Desktop – advanced GUI ensuring interoperation of underlying middlewares, successfully used in several projects  Kepler scientific workflow orchestrator – alternate client to Migrating Desktop platform

15 Additional information  Product homepage   Int.eu.grid   BalticGrid   EUFORIA   PSNC   CVS Repository – source code   The OSGi Alliance 

16 Thank you for your attention!