1. Quality of Service Experience DWG OGC Discussion Paper 17-049 – Status and Overview
OGC TC Orléans – March 21, 2018
Cindy Mitchell, Natural Resources Canada
Michael Gordon, Ordnance Survey
Tom Kralidis, Environment and Climate Change Canada
Amanda Chiprout, Natural Resources Canada

2. Presentation overview
Quality of Service Experience Discussion Paper
Overview and status
Linkage with OGC Testbed 14
Linkage with OGC Portrayal CDS
Proposed Next Steps in 2018
OGC Testbed 14 results integration
OGC Best Practices paper development

3. Quality of Experience for Web Mapping Services – Why?
As the QoSE DWG Charter notes “… a lack of methods and best practices on evaluating and improving the user experience and human interaction…”
So Discussion Paper is intended to address the following OGC QoSE DWG key activities and goals :
Develop and promote best practices for qualitative measurement and improvement of QoE related to web mapping services, such as rating or ranking services based on well-defined Quality of Experience criteria.
Develop best practices, create and promote guidance on evaluating and self- assessing the Quality of Experience of Spatial Data Services, that promote practical means for improving the user experience of web mapping services.

4. Quality of Service Experience Discussion Paper 17-049
This paper builds on research and experience contributed from the following organizations:
Natural Resources Canada
UK Ordnance Survey
Environment and Climate Change Canada
United States Geological Survey
Intended as a first step towards:
Identifying quality and usability issues related to web mapping services
Suggesting solutions and assessment methods
Providing a basis for discussion and development of formal guidance and best practices
Topics addressed in this paper include:
Proposed service quality indicators and assessment framework
Recommendations and discussion of proposed best practices to improve quality of experience
Discussion Paper explores means of ensuring that persons using OGC web mapping services consistently experience and achieve a positive interaction between themselves, any web services being accessed, and the data they represent.
Artifact Information
Filename: Ensuring Quality of User Experience with OGC Web Mapping Services – Discussion Paper.docx
Updated: :12:05
Size: MB
File URL:
Directory: /Standards Program/Quality of Service and Experience DWG/QoE Best Practice
Directory URL:
File requires authentication to access.
TC vote ended Feb 26, 2018 with no objections. Approved for publication by OGC.

5. Quality of Service Experience Discussion Paper 17-049
Intended Audience:
Geospatial data providers and custodians who prepare and make OGC web services openly available on the Internet.
Scope of Discussion:
Focus is how to make OGC web services, with particular emphasis on OGC WMS, more usable and more effective for new or non-expert users, through application of relatively simple good practices.
A web service usability assessment framework for performing service usability evaluation that aligns with the suggested good practices, is also provided.
Practices recommended in the paper do not:
Impact or require changes to any standard or other normative references. They are more grounded in practical common sense than technology
Address Web Content Accessibility concerns
Discussion Paper explores means of ensuring that persons using OGC web mapping services consistently experience and achieve a positive interaction between themselves, any web services being accessed, and the data they represent.

6. QoSE Web Service Quality Assessment Criteria
Service Quality of Experience Guidance and Indicators:
Title Form and Meaning
Consistency between Title and Map Content
Legend Appearance
Legend Content
Feature Attribution Scope
Feature Attribute Naming
Feature Attribute Completeness
Feature Attribute Values
Map Visualization
Map Cartography
Map Scaling Consistency
Map Scaling Visibility
Supporting Docs
Service Metadata
Assessments to date have shown that key considerations are often very simple, but
very impactful on quality of experience.
…are titles, names, values meaningful??
…is the legend readable and easy to understand??
…is the map clear, appealing and easy to view and interpret??
…is the metadata adequate, are supporting docs relevant??

7. QoSE work continues within OGC Testbed 14
Work plan for upcoming OGC Testbed 14 includes testing and revision of recommendations made in OGC Discussion Paper :
OGC Testbed Update:
Kickoff Workshop event April
Testbed execution: April – Nov 2018
ERS vote at Dec 2018 OGC TC meeting
Demonstration Event: planned for Dec 2018
Quality of Service Experience Assessment Framework
Quality of Service Experience Practices to Alleviate Usability Issues
Quality of Service Experience Implementing Best Practices: Man or Machine?
Quality of Service Experience Test Suite

8. QoSE-related Testbed 14 Anticipated Deliverables
D011 WMS QoSE ER - Engineering Report will capture all results
D115 Multiple WMS with QoSE support - At least two WMS instances to test the assessment criteria
D116 Client with WMS QoSE support – Two WMS clients dedicated to test the assessment criteria of WMS instances
D117 WMS QoSE Test Suite – Development and implementation of Test Suite

9. Linkage with OGC Portrayal Concept Development Study
OGC Portrayal Concept Development Study is exploring the future direction of digital portrayal and cartographic practices with a focus on web map symbology
Scope of study goes deeper than QoSE recommendations; with a closer investigation, based on past solutions with SLD and SE, as well as case studies for aspects of Map Cartography
Colour
Symbology
Labelling
Great to see growing awareness and attention given to improving cartography for web mapping
OGC Portrayal Concept Development Study
Exploring the future direction of digital portrayal and cartographic practices with a focus on web map symbology:
Includes an in‐depth review of work that has occurred to assess what works and what doesn’t
Study includes two case studies (Natural Resources Canada, Arctic Spatial Data Infrastructure)
Draft report summarizes results of past work summary and makes recommendations for next steps

10. To continue to address OGC QoSE key activities and goals:
Proposed Next Steps:
To continue to address OGC QoSE key activities and goals:
Develop and promote best practices for qualitative measurement and improvement of QoE, such as rating or ranking services based on well-defined Quality of Experience criteria.
Collect best practice, create and promote guidance on evaluating and self-assessing the Quality of Experience of Spatial Data Services, and practical means for improving the user experience of these services.
Proposed next steps:
Monitor and assess ongoing results of Testbed 14
Monitor and assess outputs of Portrayal CDS and similar OGC activities
Incorporate QoSE web service metadata best practices in body of recommendations
Move forward with development of draft Best Practices paper within QoSE DWG
Seek further test implementations and assess initial results of Web Service quality and usability testing at NRCan/FGP.

11. Thank you / Merci! Questions? Cindy Mitchell
Federal Geospatial Platform
Canada Centre for Mapping and Earth Observation
Natural Resources Canada