Quality of Service and Experience – Metrics Survey

Purpose and Approach The QoSE DWG wanted to assess the current status of OGC members interest and usage of quality of service metrics and monitoring Survey conducted via Google docs in late 2017 Circulated on OGC mailing lists and promoted via various channels available to QoSE DWG members (such as Canadian agencies)

Summary of results Only 20 responses – but a range of organisations However the small sample size should create caution in analysing results

Summary of results There is clear recognition of the importance of measuring the ongoing quality of web services.

Summary of results Most already monitor their web services – suggesting there are sufficient tooling and infrastructure to enable this

Summary of results The focus of web service monitoring is response time and availability as key metrics Few measure response content, accuracy or conformance to standards This suggests any standardisation must support response time and availability There may be a gap in tooling, best practices or standards for measuring content or accuracy

Summary of results Most responders use internal tools to monitor these metrics – this could suggest a lack of COTS or FOSS implementations (the lack of standardisation even in the wider IT industry may play a role here)

Summary of results Most responders do not let their users know their performance Could a lack of standardisation in metric definition be playing a role here?

Summary of results Most responders do not include these metrics in an SLA Potentially suggests these are not or not seen as critical services

Summary of results Most responders do see potential value in standardised methods of communicating quality of service Are there two areas here – standard definitions and measurement of metrics and standard ways of communicating them?

Questions? Michael Gordon Michael.Gordon@os.uk @G0rd82