1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal P. Kacsuk*, G. Sipos* and T. Kiss** *MTA SZTAKI ** Univ. of Westminster

2. Enabling Grids for E-sciencE INFSO-RI-508833 Properties of the P-GRADE Portal What is P-GRADE portal? –General purpose, graphical, workflow-oriented Grid portal –Supports the development and execution of workflow-based Grid applications –Components of the workflows can be  Sequential jobs  Parallel jobs (MPI, PVM)  Legacy code (GEMLCA) services –Easy porting of legacy and other applications to the Grid –Enables the exploitation of two levels of parallelism  intra-job  inter-job –Enables to control and observe the execution of the workflow applications –Enables fault-tolerant workflow execution

3. Enabling Grids for E-sciencE INFSO-RI-508833 Properties of the P-GRADE Portal Enables application-specific extensions by new portlets (E-GRID, Traffic simulation Grid, etc.) Hides the low-level Grid details Enables the access of every important feature of the underlying Grid, e.g.: –brokering service –information service, etc. Can serve as a bridge between different Grids to solve the Grid interoperability problem at the workflow level: –Enables easy porting of legacy and other applications between various Grids (e.g. from LCG to gLite, from GT2 to GT4, etc.) –Hides middleware changes for the end-user –Enables parallel execution of a workflow in several Grids

4. Enabling Grids for E-sciencE INFSO-RI-508833 Properties of the P-GRADE Portal Enables application-specific extensions by new portlets (E-GRID, Traffic simulation Grid, etc.) Hides the low-level Grid details Enables the access of every important feature of the underlying Grid, e.g.: –brokering service –information service, etc. Can serve as a bridge between different Grids to solve the Grid interoperability problem at the workflow level: –Enables easy porting of legacy and other applications between various Grids (e.g. from LCG to gLite, from GT2 to GT4, etc.) –Hides middleware changes for the end-user –Enables parallel execution of a workflow in several Grids

5. Enabling Grids for E-sciencE INFSO-RI-508833 Proxy management P-GRADE portal in a nutshell Grid resources management Workflow creation Job mapping to Grid resources Workflow management and execution visualization

6. Enabling Grids for E-sciencE INFSO-RI-508833 Two levels of parallelism by the P-GRADE workflow Semantics of the workflow enables two levels of parallelism: The job can be a parallel program – – Parallel execution inside a workflow node – – Parallel execution among workflow nodes Multiple jobs can run parallel The P-GRADE Portal workflow concept enables the efficient parallelization of complex problems

7. www.cpc.wmin.ac.uk/GEMLCA Grid Execution Management for Legacy Code Applications OGSA-DAI Integration with P-GRADE Tamas Kukla and Tamas Kiss Centre for Parallel Computing University of Westminster T.Kukla@student.westminster.ac.uk, kisst@wmin.ac.uk

8. www.cpc.wmin.ac.uk/GEMLCA OGSA-DAI portlets 4 portlets implemented 1. 1.Data Service Manager 2. 2.Database Browser 3. 3.Query Manager 4. 4.Data Manipulation

9. www.cpc.wmin.ac.uk/GEMLCA Data Service Manager portlet Add/remove OGSA- DAI services List available resources of a service List available tables Select and describe tables

10. www.cpc.wmin.ac.uk/GEMLCA Database Browser portlet Select service / resource / table Display the selected table or XML resource Order by the selected column

11. www.cpc.wmin.ac.uk/GEMLCA Query Manager portlet Display results on screen Run query on selected database See next slide Display results

12. www.cpc.wmin.ac.uk/GEMLCA Query Manager portlet Deliver results to file Data can be sliced up and compressed before delivery Define target location Define query statement

13. www.cpc.wmin.ac.uk/GEMLCA Data Manipulation portlet Manipulate data of selected database Select database Define update statement See next slide Update

14. www.cpc.wmin.ac.uk/GEMLCA Data Manipulation portlet Deliver results from file to database Define target table Define files to be delivered Extracts and merges Select target service / resource

15. www.cpc.wmin.ac.uk/GEMLCA A possible user scenario The new OGSA-DAI portlet allows to browse an OGSA-DAI database and deliver query results to a set of files. These files can be used in normal or parameter study workflows as input ports. The workflow output can be converted and transferred back to the database. 1. 1.Select the data service and resource 2. 2.Run a query and deliver results to a (set of) file(s) 3. 3.Run the (parameter study) workflow on the data-sets 4. 4.Transfer the results back into an OGSA-DAI database

16. www.cpc.wmin.ac.uk/GEMLCA Extended PG-Portal DataBrowserWorkflow Step1: Select the OGSA-DAI service and DataBase resource with the DataBrowser OGSA-DAI service Oracle DataBase resource User eXist XML DataBase resource Request List of resources Request List of resources Storage resource (local or remote)

17. www.cpc.wmin.ac.uk/GEMLCA Step1: Select the OGSA-DAI service and DataBase resource with the DataBrowser Select service and resource

18. www.cpc.wmin.ac.uk/GEMLCA Extended PG-Portal DataBrowserWorkflow Step2: Run a query and deliver results to a (set of) file(s) OGSA-DAI service Oracle DataBase resource User eXist XML DataBase resource Query Results Query Results Query Storage resource (local or remote)

19. www.cpc.wmin.ac.uk/GEMLCA Step2: Run a query and deliver results to a (set of) file(s) Define query statement Deliver results to a set of files

20. www.cpc.wmin.ac.uk/GEMLCA Extended PG-Portal DataBrowserWorkflow Step3: Run (parameter study) workflow (this step does not use the OGSA-DAI portlets!) OGSA-DAI service Oracle DataBase resource User eXist XML DataBase resource Create workflow Workflow output Storage resource (local or remote) Input data

21. www.cpc.wmin.ac.uk/GEMLCA Extended PG-Portal DataBrowserWorkflow Step4: Transfer results back into an OGSA-DAI database OGSA-DAI service Oracle DataBase resource User eXist XML DataBase resource Workflow output Storage resource (local or remote) Workflow output Confirmation Request

22. www.cpc.wmin.ac.uk/GEMLCA Step4: Transfer results back into an OGSA-DAI database Define target database Define target table Define files to be transferred and converted