Jaime Pérez Virginia Martín-Rubio TERENA Networking Conference Prague, May 2011

1.The ability to monitor the status of the Identity and/or Service Providers of our production federation. 2.User centric: provider’s status must be seen as from the point of view of the users. 3.Deploy a complete monitoring platform that allows us to manage alerts, reports, graphs, statistics, and more. Goals

1.It must be compatible with our running monitoring infrastructure, based on Nagios: Automated tests executed on demand Follow the Nagios plugins API 2.It must be independent of the underlying technology: SIR federation is a mixture of protocols Users don’t know about technology, just use it Requisites

–We started looking for the most suitable tools to fit the requirements. Some software to allow automation of the user’s (and his/her web browser) behaviour. –We made our choice to be Apache JMeter. Mainly used as a benchmarking tool, it’s perfect to simulate web browsers. It lacks support of Javascript, but provides mechanisms to simulate it. Challenge #1: find the appropriate tools

Apache JMeter

1.First we developed a test plan that simulates a login through our federation, authenticates and returns back to a specially crafted SP. Automating JMeter

2.Then we set up a dedicated machine to run the test plan on it by means of the JMeter command line interface. Automating JMeter

3.We also considered using a farm of JMeter servers that receive the test plans and run them: better performance and scalability. Automating JMeter

–Since it is desirable to have just one test plan for all monitored IdPs, we designed it with macros and variables that we change in runtime to fit the specific details of each IdP. That is: Username Password The names of the input fields of the login form A cookie to bypass the WAYF and go straight to the IdP. Automating JMeter

–Once we were able to test individually each IdP, we needed a way to run the tests and get the results in a specific Nagios format. –We developed a shell script that receives as command line parameters the variables mentioned before, modifies the test plan in runtime, runs JMeter with it and evaluates the output to translate to a Nagios service status/performance data. Challenge #2: nagios integration

–It is flexible enough to allow us to evaluate the settings of and IdP. For instance, looking for some mandatory attributes and triggering a warning if any of them is missing: adding logic to the Fake Service Provider –It also allows us to perform security tests, like making sure a non-existent user is unable to successfully login to the IdP: testing twice with real and fake users Challenge #2: nagios integration

22 IdPs already being monitored and increasing Achievements #1 Private Nagios interface

Achievements #2 Manual testing of an IdP

Achievements #3 Public web app (the SP itself)

Achievements #4 Comprehensive data about IdP status

Achievements #5 Monthly reports

Achievements #6 reports & alerts

–User centric federation monitoring: we simulate users and browser behaviour, so if the monitor says an IdP is working, then we can guarantee it really does. –Technology independent: though it is adapted to our running infrastructure, it doesn’t know anything about the underlying technology, and in fact supports several protocols mixed altogether. –Want more info? Look for the extended abstract! Summary

Thanks for listening!