1. Christian Delbé Kryzysztof Kurowski and Mariusz Mamonski (Poznan), Gabor Szemes and George Kampis, (Collegium Budapest) Walter De Back and Lazlo Gulyas (Aitia) Christian Delbé (ActiveEon) Extending ProActive for QosCosGrid: Support for Advance Reservation and Multi-Cluster Allocation

2. Christian Delbé QosCosGrid Project EU 6th Framework Programme STREP Project 2,5 years, ends in 03/2009 11 partners (2 private companies) from 10 countries Aim at providing Quasi-Opportunisitc Supercompting for COmplex Systems on GRIDs 1.Quasi (i.e. not really) opportunistic Reservations, predictable performances 2.Framework for Complex systems on Grids Very broad application class with widely varying requirements (no implicit restrictions on applications) 3.For the Grids…

3. Christian Delbé QosCosGrid Project Status GRMS Grid Scheduler Reservation and orchestration of resources Specific XML job description (Job Profile) Resources needs, processes affinity,… GRMS Portal OpenDSP/LSF … Job Profile Cluster 1Cluster N

4. Christian Delbé QosCosGrid Project Status GRMS Grid Scheduler Reservation and orchestration of resources Specific XML job description (Job Profile) Resources needs, processes affinity,… Programming Framework Fault-tolerant cluster-to-cluster message passing libraries based on Open MPI (FORTRAN/C/C++) and ProActive (Java) 9 Use Cases Written in C/MPI and in Java/ProActive Benchmarked on multi-clusters testbed

5. Christian Delbé QosCosGrid/ProActive challenge 1 : Deployment 1.Preserve ProActive deployment properties (Nodes, Virtual Nodes,…) 2.Provide end-users JobProfile as a single description (No explicit deployment descriptor) 3.Avoid need for direct connection on remote clusters machine (ssh,…) Provides a GRMS deployment process ? Unfortunalty … The main process must be connected to deployed processes during the execution Provides a 2 steps submission, i.e. submit the main process that will submit rest of the application ? Unfortunalty … Sub-jobs are not supported by GRMS (reservation and accounting) Submit ProActive application as a whole with a specific asynchronous deployment process

6. Christian Delbé Deployment for QosCosGrid : ProActive Node Coordinator GRMS Portal OpenDSP/LSF … ProActive Node Coordinator 1.Submit Job Profile 2.Create reservation and submit QCG-PA Wrappers 3.Start QCG-PA Wrappers 1. Main is started and registered to the PNC 2. Runtimes are started and registered to the PNC Job Profile WRAP main WRAP rt WRAP rt WRAP rt

7. Christian Delbé Deployment for QosCosGrid : ProActive Node Coordinator GRMS Portal OpenDSP/LSF … ProActive Node Coordinator 1.Submit Job Profile 2.Create reservation and submit QCG-PA Wrappers 3.Start QCG-PA Wrappers 1. Main is started and registered to the PNC 2. Runtimes are started and registered to the PNC main rt

8. Christian Delbé QCG is multi-clusters without restriction on applications Communication must be possible from anywhere to anywhere But clusters are usually behind a firewall and/or a NAT Use ProActive’s RMISSH on port 22. Unfortunatly… Does not deal with NATs Provide a new protocol that supports NATs : RMIQCG QosCosGrid/ProActive challenge 2 : Connectivity

9. Christian Delbé Extending inter-cluster communications for QosCosGrid RMIQCG uses SOCKS protocol instead of SSH SOCKS server deployed on the front-end node One port must be externally available Single proxy per cluster implies contention…

10. Christian Delbé Benchmarks Testbed Testbed portal @ node2.qoscosgrid.man.poznan.pl/gridsphere/gridesphere

11. Christian Delbé UseCase 9 Distributed MultiAgent Simulation Active Object Network Communcation Cellular Automata 1. Partition 2. Deploy 3. Iterate

12. Christian Delbé UseCase 9 Distributed MultiAgent Simulation On 8 machines (8 and 4+4) Scalability issue with RMIQCG

13. Christian Delbé Conclusion Important external contribution Dedicated QCGProActive version (based on 3.9) Ongoing integration in official ProActive 4 Provides solutions to scalability problems inherent to RMIQCG Similar solutions are studied in the OASIS team Successful partnership between QosCosGrid and ActiveEon Support for QCGProActive deployment design Support for upgrade from 3.2 to 3.9 Support for use cases application and design

14. Christian Delbé Thank you ! Questions ?

15. Christian Delbé EU 6th Framework Programme STREP Project 2,5 years, ends in 03/2009 2 800 000 Euro Strong QCG Consortium: 11 partners (2 private companies) from 10 countries QosCosGrid Project

16. Christian Delbé UseCase 8 Active Object Network Communcation