1. Quality of Service and Experience – Metrics Survey

2. Purpose and Approach
The QoSE DWG wanted to assess the current status of OGC members interest and usage of quality of service metrics and monitoring
Survey conducted via Google docs in late 2017
Circulated on OGC mailing lists and promoted via various channels available to QoSE DWG members (such as Canadian agencies)

3. Summary of results Only 20 responses – but a range of organisations
However the small sample size should create caution in analysing results

4. Summary of results
There is clear recognition of the importance of measuring the ongoing quality of web services.

5. Summary of results
Most already monitor their web services – suggesting there are sufficient tooling and infrastructure to enable this

6. Summary of results
The focus of web service monitoring is response time and availability as key metrics
Few measure response content, accuracy or conformance to standards
This suggests any standardisation must support response time and availability
There may be a gap in tooling, best practices or standards for measuring content or accuracy

7. Summary of results
Most responders use internal tools to monitor these metrics – this could suggest a lack of COTS or FOSS implementations (the lack of standardisation even in the wider IT industry may play a role here)

8. Summary of results
Most responders do not let their users know their performance
Could a lack of standardisation in metric definition be playing a role here?

9. Summary of results
Most responders do not include these metrics in an SLA
Potentially suggests these are not or not seen as critical services

10. Summary of results
Most responders do see potential value in standardised methods of communicating quality of service
Are there two areas here – standard definitions and measurement of metrics and standard ways of communicating them?

11. Questions?
Michael Gordon
@G0rd82