1. Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education Data Sharing www.chain-project.eu proj-office@chain-project.eu Grant Agreement n. 306819 CHAIN-REDS tools for the use of a Cloud Infrastructure Giuseppe LA ROCCA, INFN Catania - Italy Clouds for Research and Education Workshop, 27 June 2014, Rome - Italy

2. Outline  Some introductory concepts and driving considerations  Catania Science Gateway Framework’s overview  The Cloud & Grid Engine  The architecture of the JSAGA adaptor for OCCI  The MyCloud portlet for clouds orchestration  Summary and Conclusions Clouds for Research and Education Workshop – 27.06.2014

3. Some barriers limit the e-Infrastructure adoption (*) The eResearch2020 report http://www.eresearch2020.eu / Some barriers limit the e-Infrastructure adoption Clouds for Research and Education Workshop – 27.06.2014

4. The “non-global” middleware Genesis II Interoperability and easiness of access are issues Clouds for Research and Education Workshop – 27.06.2014

5. A very «cloudy» sky… Clouds for Research and Education Workshop – 27.06.2014

6. Interoperability & Interoperation (source: Wikipedia)  According to ISO/IEC 2382-01 (Information Technology Vocabulary, Fundamental Terms), interoperability is: "The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units“  In engineering, interoperation is: “The setup of ad hoc components and methods to make two or more systems work together as a combined system”  The adoption of standards => are the key for a long-term sustainability! Clouds for Research and Education Workshop – 27.06.2014

7.  A Scientist can sign in on a Catania Science Gateway using his/her federated credentials, select an application from a menu and seamlessly execute it on HPC machines, Grids and Clouds  The fractions of executions on the three different platforms can be adjusted to simulate the need to “boost” the resources in case of temporary peaks of activity Two Scenarios …  The Cloud Tenant of a real or virtual organisation can sign in on a Catania Science Gateway using his/her federated credentials, select virtual machine(s) from a geographically shared repository and deploy/move/copy it/them across his/her personal cloud  The graphic user interface will be very intuitive including point & click and drag & drop functionalities  The virtual machine(s) will belong to the same domain name (chain- project.eu in the particular case) independently of the site where it/they will be instantiated and of the underlying Cloud middleware stack Clouds for Research and Education Workshop – 27.06.2014

8. Science Gateway Science Gateway Administrator Scientist Cloud tenant Users belonging to Identity Federations Users belonging to Identity Federations Grid/Cloud Engine (based on SAGA) Orchestrator (based on OCCI) Cloud #2 Cloud #n Cloud #1 Single domain HPC Clusters MyCloud / The Catania Science Gateway Framework’s high-level architecture Clouds for Research and Education Workshop – 27.06.2014

9. The Catania Grid & Cloud Engine Users Tracking DB Science GW Interface SAGA/JSAGA API Job Engine Data Engine Users Track & Monit. Grid/Cloud/Local MWs User Tracking DB eToken Server 9 Clouds for Research and Education Workshop – 27.06.2014

10. 10 Use case implementation: JSAGA Adaptor for OCCI Science Gateway Catania Grid/Cloud Engine Security context Job management Stage-in/out 2) kill VM SG stage-in & run app VM1) stage-out SG VM SG start VM robot certificate Clouds for Research and Education Workshop – 27.06.2014

11. CHAIN-REDS Science Gateway -CHAIN-REDS RI-306819

12. Clouds for Research and Education Workshop – 27.06.2014 12 Enable advanced settings Specify the advanced settings for the application Start VM Select the VM profile Enable e-mail notification Select the VM template Job description CHAIN-REDS Science Gateway -CHAIN-REDS RI-306819

13.  The Weather Research and Forecasting (WRF) Model is a next-generation meso-scale numerical weather prediction system designed to serve both atmospheric research and operational forecasting needs. (WRF) Running WRF on various distributed computing Infrastructures through the CHAIN-REDS SG CHAIN-REDS Science Gateway -CHAIN-REDS RI-306819 Clouds for Research and Education Workshop – 27.06.2014

14. CHAIN-REDS Science Gateway -CHAIN-REDS RI-306819 Clouds for Research and Education Workshop – 27.06.2014

15. 15 Cloud 1 M/W 1’ Cloud 2 M/W 2’ Cloud 3 M/W 3’ Cloud 4 M/W 4’ Cloud 5 M/W 5’ Cloud 6 M/W 6’ Cloud 7 M/W 7’ Cloud 8 M/W 8’ Cloud 9 M/W 9’ Cloud 10 M/W 10’ MyCloud Clouds for Research and Education Workshop – 27.06.2014 CHAIN-REDS Science Gateway -CHAIN-REDS RI-306819

16. 16 10 clouds 6 countries 3 m/w stacks 1 SME IT ES CZIT ZA EG GR FedCloud Clouds for Research and Education Workshop – 27.06.2014 CHAIN-REDS Science Gateway -CHAIN-REDS RI-306819

17. Managing VMs across different cloud providers: MyCloud @ work! Current functionalities: Federated authentication Fine-grained authorisation Single/multi-deployment of VMs on a cloud and across clouds, including the EGI FedCloud Single/multi-move of VMs across clouds Single/multi-deletion of VMs on a cloud and across clouds SSH connection to VMs Direct web access to VMs hosting web services 17 Clouds for Research and Education Workshop – 27.06.2014 VMs are Moveable across Clouds Available VMs Multi/Single instance Available VMs Multi/Single instance

18. Managing VMs across different cloud providers: MyCloud @ work! 18 Clouds for Research and Education Workshop – 27.06.2014 Turn On/Off the VM

19.  The Catania Science Gateway Framework successfully bridges e- Infrastructures built on different architectures (Grid, HPC, Cloud, local clusters, desktops, etc.) and make them interoperable at user application level thanks to standard adoption (SAGA, SAML, OCCI, etc.)  The MyCloud service allows seamless multi-cloud service operation across different OCCI-compliant middleware stacks  Next steps are:  Creation of the shared storage infrastructure to support stateful VMs  Allow deployed VMs to «find themselves» in MyCloud Summary and outlook Clouds for Research and Education Workshop – 27.06.2014

20. Email: sg-licence@ct.infn.it sg-licence@ct.infn.it cloud-interop@chain-project.eu Social Networks: Contacts Clouds for Research and Education Workshop – 27.06.2014

21. Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education Data Sharing www.chain-project.eu proj-office@chain-project.eu Grant Agreement n. 306819 www.chain-project.eu proj-office@chain-project.eu