On Monitoring, Diagnostics and Measurement in eduGAIN and Beyond Miroslav Milinović University Computing Centre, University of Zagreb, Zagreb, Croatia <miro@srce.hr> Zurich, Switzerland, July 12, 2016

The problem space ... monitoring testing diagnostics continuous, automated testing to check the health of the service (or a component or a feature) simple vs. functional testing on demand to verify compliance to a standard / specification to check the functionality diagnostics to resolve incidents to enhance user experience measurement (statistics) facilitate maintenance / management compute service performance indicators

An inventory … The tools we know about: Wiki pages listing tools: https://wiki.refeds.org/display/MON/Monitoring+Home https://wiki.geant.org/display/gn41sa5/SAML+Testing+Tool+Comparison … Web pages with tools collections: http://fed-lab.org/ www.edugain.org/technical The tools we know about: eduGAIN Access Check Service (EACS) https://access-check.edugain.org/ eduGAIN Connectvity Check Service (ECCS) https://technical.edugain.org/eccs/ eduGAIN isFederated Check Service (IFCS) https://wiki.edugain.org/isFederatedCheck/ eduGAIN Attribute Release Check Service (EARCS) http://earc.eduid.hu/ CoCo monitor (+on demand check) https://monitor.edugain.org/coco SAML and OIDC testing tools (Roland Hedberg / fed-lab) MET ? F-Ticks (eduroam, SWAMID, AAI@EduHr)

Proposal for monitoring and testing Build a web page (portal/catalogue) list all recognized tools grouped according to the purpose and scope basic info/description Link to appropriate tool page/service (if in place) link to SW repository (if available) use fed-lab.org or edugain.org or refeds.org ? currently we have too many pages that list tools Identify the missing tools and build them

Measurement and statistics Some work has been done: http://www.geant.org/Resources/Documents/Issues%20and%20Solutions%20for%20SAML%20Identity%20Federation%20Statistics.pdf F-ticks based tools https://tools.ietf.org/html/draft-johansson-fticks-00 https://portal.nordu.net/display/SWAMID/SAML+f-ticks+for+Shibboleth SSP tool internally used by AAI@EduHr may be used … Other tools RAPTOR AMAAIS …

Proposal for measurement and statistics Step 1: Standardise the F-ticks (aka log entry) format Finish the work on f-ticks RFC Step 2: Build the eduGAIN F-ticks service (based on positive experience and tools of eduroam, AAI@EduHr, SWAMID, …) Ticks will be sent by the IdPs (even one per federation can sufice in pilot phase?) can be collected by more than one aggregator Aggregator Will provide public stats (popular SPs per IdP, popular IdPs per SP, …) Will provide additional stats for IdP/SP Stats of failed AA might be used for diagnostics Value for actors (IdPs in particular) ready to consume measurement dana Logs in f-ticks format might be processed with other (common) tools