1. www.cpc.wmin.ac.uk/GEMLCA Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre for Parallel Computing University of Westminster London

2. www.cpc.wmin.ac.uk/GEMLCA Presentation Outline 1. 1.GEMLCA objectives and concept 2. 2.GEMLCA architecture 3. 3.The GEMLCA P-GRADE Portal 4. 4.Deployment example 5. 5.Conclusion

3. www.cpc.wmin.ac.uk/GEMLCA Legacy Applications Code from the past, maintained because it works Often supports business critical functions Not Grid enabled Port them onto the Grid with minimum user effort What to do with legacy codes in service Grids? Bin them and reimplement them as grid services Reengineer them  who knows the source code?

4. www.cpc.wmin.ac.uk/GEMLCA GEMLCA – Grid Execution Management for Legacy Code Architecture To deploy legacy code applications as Grid services without reengineering the original code and minimal user effort To support the development and execution of “legacy code” grid service workflows To make these functions available from a Web Portal GEMLCA GEMLCA P-GRADE Portal Objectives

5. www.cpc.wmin.ac.uk/GEMLCA GEMLCA Concept Compute Servers Resource manager deploys: LC GEMLCA GEMLCA Grid service Client 1: to register legacy code as a Grid service Client 2: to invoke legacy code Grid service Send custom input Invoke legacy code Produce result Return result

6. www.cpc.wmin.ac.uk/GEMLCA The GEMLCA-view of a legacy code Any code that correspond to the following model can be exposed as Grid service by GEMLCA: Legacy code Input data (files, command line params, env. vars) Output data (files) Call shared libraries Query databases Communicate over the network Perform computation...

7. www.cpc.wmin.ac.uk/GEMLCA GEMLCA Concept Compute Servers Resource manager deploys: LC GEMLCA GEMLCA Grid service Client 1: to register legacy code as a Grid service Client 2: to invoke legacy code Grid service Custom input Return result Legacy code Input data is provided dynamically Access to the code is controlled by the GEMLCA Grid service Invoke legacy code Produce result Code is deployed statically

8. www.cpc.wmin.ac.uk/GEMLCA What ’ s behind the GEMLCA service … Client GEMLCA Service Compute Servers Local jobmanager e.g. Condor/Fork GEMLCA Process Legacy Code Job Job submission and file transfer services Grid middleware layer Grid Service Client

9. www.cpc.wmin.ac.uk/GEMLCA What ’ s behind the GEMLCA service … Client GEMLCA Service Compute Servers Local jobmanager e.g. Condor/Fork GEMLCA Process Legacy Code Job Gateway service (e.g. GRAM) Grid middleware layer Grid Service Client Consequences: 1.Not only the legacy code and the GEMLCA service, but also a local jobmanager and a Grid middleware layer must be installed on the hosting system! 2.The aim of the code registration process is to tell GEMLCA how to submit the legacy code to the grid middleware layer

10. www.cpc.wmin.ac.uk/GEMLCA Heterogeneous codes can be hidden behind the same interface (the programming interface of the GEMLCA service) –Different programs can be invoked in the same way Extend non grid-aware programs with security infrastructure (access enabled through a Grid service) –Share your codes with your colleagues or partner institutes –Expose business logic to your employers or customers Build customized GEMLCA clients (such as the GEMLCA P-GRADE Portal) –Compose complex processes by connecting multiple legacy code grid services together What ’ s the point?

11. www.cpc.wmin.ac.uk/GEMLCA The GEMLCA P-GRADE Portal A Web-based GEMLCA client environment… University of Westminster, London MTA SZTAKI, Budapest

12. www.cpc.wmin.ac.uk/GEMLCA To provide graphical clients to GEMLCA with a portal-based solution To enable the integration of legacy code grid services into workflows The aim of the GEMLCA P-GRADE Portal

13. www.cpc.wmin.ac.uk/GEMLCA 3 rd generation Grids: (OGSA: GT4, gLite) Legacy applications Grid Site 1 Grid Site 2... The GEMLCA P-GRADE Portal GEMLCA P-GRADE Portal Server Desktop 1 Desktop N Web browser LC 1 LC 2 Result Input

14. www.cpc.wmin.ac.uk/GEMLCA It contains a web page to register legacy codes as grid services It contains a GEMLCA-specific workflow editor –Workflow components can be “legacy code grid services” (not only batch jobs) It contains a GEMLCA-specific workflow manager subsystem –It can invoke GEMLCA services (not only submitting jobs) The GEMLCA-specific version of the P-GRADE Portal is different from the original P-GRADE Portal!

15. www.cpc.wmin.ac.uk/GEMLCA Legacy code registration page “GEMLCA Administration Tool” portlet

16. www.cpc.wmin.ac.uk/GEMLCA Legacy code registration page

17. www.cpc.wmin.ac.uk/GEMLCA Workflow Creation UoW Sztaki UoR GEMLCA workflow editor in a nutshell

18. www.cpc.wmin.ac.uk/GEMLCA Batch components vs. GEMLCA components in P-GRADE Portal workflows User provides the binary executable User provides input data for the executable Job produces result files Executable is already installed on a machine User provides input data for the executable Job produces result files Batch component GEMLCA component Input files represented by ports Output files represented by ports Workflow components must be defined in different ways Ports guarantee compatibility  batch and GEMLCA components can mutually produce data to each other!

19. www.cpc.wmin.ac.uk/GEMLCA Workflow Creation UoW Sztaki UoR Defining GEMLCA components

20. www.cpc.wmin.ac.uk/GEMLCA Combine legacy codes with new codes inside the same workflow! Code invocation Job submission Publish successfully executed new code as legacy code Combining legacy and non-legacy components

21. www.cpc.wmin.ac.uk/GEMLCA Sharing jobs with other GEMLCA portal users New button for sucessfully executed jobs: create a legacy code service from this jobs

22. www.cpc.wmin.ac.uk/GEMLCA GEMLCA P-GRADE Portal server Sharing jobs with other GEMLCA portal users Job Repository Put job into Portal level repository

23. www.cpc.wmin.ac.uk/GEMLCA GEMLCA – Globus-based service  proxy certificate is needed for its invocation Proxy download and usage processes are identical in both the original and the GEMLCA-specific P- GRADE Portals Workflow visualization remains the same: –Editor –Workflow portlet Status info Detailed visualization images Workflow execution

24. www.cpc.wmin.ac.uk/GEMLCA An example for a GEMLCA workflow Workflow to analyse road traffic Manhattan road network generator Traffic simulators Analyser

25. www.cpc.wmin.ac.uk/GEMLCA UK NGS Manchester LeedsOxford Rutherford GT2 GEMLCA and Production Grids The problem Sorry, no additional utilities to be deployed on core resources Service reliability Administration ??? I’m a biologist not a Grid expert

26. www.cpc.wmin.ac.uk/GEMLCA The solution UK NGS Manchester LeedsOxford Rutherford GT2 Portal server at UoW User-friendly Web interface

27. www.cpc.wmin.ac.uk/GEMLCA Centralised GEMLCA solution P-GRADE Portal Server Desktop 1 Desktop N Web browser 3 rd generation Grids: (OGSA: GT3, WSRF, gLite) Legacy applications Grid Site 1 Grid Site 2 The grid middleware and local jobmanager layers remain untouched!

28. www.cpc.wmin.ac.uk/GEMLCA The scenarios Scenario 1 – legacy code repository NGS site 1 (GT2) NGS site 2 (GT2) NGS site n (GT2) P-Grade Portal user job submission GEMLCA resource (GT4 + GEMLCA classes) Central repository legacy code 1 legacy code 2 …. legacy code n Workflow definition 3 rd party service provider (UoW) GEMLCA interface (GT4) Interface (GT2)

29. www.cpc.wmin.ac.uk/GEMLCA Advantages of Scenario 1 legacy codes can be uploaded into a central repository and made available for authorised users through a Grid portal would extend the usability of NGS as users utilise others’ legacy codes stored in the repository No support needed at the NGS sites

30. www.cpc.wmin.ac.uk/GEMLCA The scenarios Scenario 2 – Legacy code services P-Grade Portal user GEMLCA resource (GT4 + GEMLCA classes) Workflow definition NGS site 1 (GT2) NGS site 2 (GT2) NGS site n (GT2) Execution request Legacy codes 3 rd party service provider (UoW)

31. www.cpc.wmin.ac.uk/GEMLCA Advantages of Scenario 2 extends the NGS Grid towards the service- oriented Grid concept Users can access the legacy codes that are deployed on the NGS sites NGS system administrators have full control of legacy codes that they deploy on their own resources

32. www.cpc.wmin.ac.uk/GEMLCA Conclusions GEMLCA enables the deployment of legacy code applications as Grid services without any real user effort. GEMLCA is integrated with the P-GRADE portal to offer user-friendly development and execution environment. The integrated GEMLCA P-GRADE solution is available for the UK NGS as a service! www.cpc.wmin.ac.uk/ngsportal

33. www.cpc.wmin.ac.uk/GEMLCA Email: gemlca-discuss@cpc.wmin.ac.uk Website: www.cpc.wmin.ac.uk/gemlca www.cpc.wmin.ac.uk/ngsportal Thank you for your attention … Any questions?