1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org Federated Network Performance Monitoring for the Grid K. Kavoussanakis, EPCC, The University of Edinburgh A. Phipps, EPCC, The University of Edinburgh C. Palansuriya, EPCC, The University of Edinburgh A. Trew, EPCC, The University of Edinburgh A. Simpson, EPCC, The University of Edinburgh R. Baxter, EPCC, The University of Edinburgh GridNets 2006, 1st Oct 2006, San Jose, CA, USA

2. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 2 Outline Network Performance Monitoring for EGEE Users and Requirements Architecture EGEE Network Performance Monitoring Diagnostic Tool Future Work & Conclusions

3. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 3 EGEE Large scale heterogeneous e-Science infrastructure ~ 200 sites in 40 countries ~ 25 000 CPUs > 10 PB storage > 35 000 jobs per day > 100 Virtual Organizations

4. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 4 EGEE Network Services Development Network Performance Monitoring (NPM) –Scale and heterogeneity of EGEE fabric is such that it necessary to use and support the existing different NPM frameworks. –Aim to standardise access to NPM across different network domains and frameworks leading to a federated approach –Developed a reference implementation of world’s first single-point, single interface access to NPM data from heterogeneous frameworks

5. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 5 Users of NPM

6. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 6 Backbone Perfmonit NM-WG Some Client Backbone PiPEs NM-WG Backbone PerfSONAR NM-WG End-site Home-grown NM-WG End-site e2emonit NM-WG Access to data GGF Network Measurements Working Group (NM-WG) the basis for standardisation –XML schemas to request and share NPM data Interoperability pursued through adoption of NM-WG –EGEE should not and cannot aim to enforce the uptake of a specific NPM framework across the diverse EGEE fabric or the associated networks –The NM-WG interface has been adopted; many networks are publishing through it. EGEE NPM pursues a federated approach to Network Monitoring

7. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 7 NPM User Requirements Middleware Programmatic interface to –Web services –Databases Info for 100 paths returned in 0.2s Map Compute/Storage Element with Network Monitoring Points Raw, historical data for 24 hrs Mainly end-to-end data Operation Centres NOCs and GOCs –Web-based GUI –Interface to define alarms –On-demand & historical data –Backbone & end-to-end data NOCs –Which tool gathered the results –Per hop data/ability to zoom in GOCs –High-level statistics, e.g., end- to-end throughput

8. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 8 Backbone Perfmonit NM-WG Some Client Backbone PiPEs NM-WG Backbone perfSONAR NM-WG End-site Home-grown NM-WG End-site e2emonit NM-WG NPM Architecture and Deployment Data from: GÉANT Abilene ESNet Soon to provide LCG data Single point of contact Standard interface Insulation from framework interface changes

9. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 9 NPM DT Scenario (1) Step 1: Access the NPM Diagnostic Tool. – Accessed using a standard web browser. individually authorised to use. Please mail us for access! – The intended user is a NOC or GOC operator

10. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 10 NPM DT Scenario (2) Step 2: Select a Time. – Problem occurred within the past two days. – Enters an appropriate time range for the period – Presses the Set button to confirm

11. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 11 NPM DT Scenario (3) Step 3: Select a Path. – Problem between UEDIN and CNRS. – Selects e2emonit sites at UEDIN and CNRS – Adds the path – Selects “Find Data For This Query”

12. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 12 NPM DT Scenario (4) Step 4: Select a Metric. – The end-user experienced throughput problems. – Several possibly relevant metrics to choose from – Decides to look at the Achievable Bandwidth

13. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 13 NPM DT Scenario (5) Step 5: Select a Statistic. – If necessary, choose a statistic to be applied. – An interval can be applied; e.g., an hourly mean over the past two days. – Just wants a general overview of measurements – Elects to retrieve raw data (Statistic check-box not checked).

14. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 14 NPM DT Scenario (6) Step 6: Select a View. – Currently Data Table and Time Plot views are available. – Wants an overview of how the Achievable Bandwidth has changed over time – Selects the Time Plot. – Query entry is complete – Selects Submit Query.

15. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 15 NPM DT Scenario (7) Step 7: Examine results. – Achievable Bandwidth Vs Time – The parameters used to gather measurements are also shown – Here, the iperf tool was used to gather the achievable bandwidth information.

16. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 16 NPM Publisher

17. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 17 Future Work Moving to NM-WG v2 –Gives us access to many more frameworks/NMPs through  perfSONAR  UK GridPP (project gridmon) Improving Diagnostic Tool –Hardening –Features Authentication and Authorisation –Collaboration with networks; what data will they restrict and is their method compatible to EGEE?

18. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 18 Conclusions Grid Operations Centres and Middleware require network performance data EGEE Network Services Development activity pursues and practices federated network performance monitoring GGF Network Measurements Working Groups schemas are the foundation for standardising access to NPM data

19. Enabling Grids for E-sciencE INFSO-RI-508833 C. Palansuriya, Federated NPM for the Grid 19 Acknowledgements EGEE is partly funded by the European Commission; contract no: INFSO- RI-508833 EPCC is jointly funded by the UK Joint Information Systems Committee (JISC). The following organisations participated in EGEE NPM: –EPCC, The University of Edinburgh –DANTE –CNRS –DFN –DL, CCLRC –GARR The GÉANT, Abilene and ESNet data are provided through our collaboration with GÉANT2 and the perfSONAR projects. http://www.egee-npm.org/