® Sponsored by GeoViQua User Feedback on Data Quality 90th OGC Technical Committee Washington, DC Joan Masó 25th March 2014 Copyright © 2014 Open Geospatial Consortium

OGC ® User feedback in GeoViQua GeoViQua provides a set of scientifically developed software components and services that facilitate the creation, search and visualization of quality information on EO data integrated and validated in the GEOSS Common Infrastructure. Pilot case studies C R O S S S B A Community building GEO S&T Label 2

OGC ® GeoViQua (265178) FP7 Collaborative Project Environment 10 partners with CREAF coordinating Duration: 36 months 1 February 2011 – 31 January 2014 Budget: ,42 € EC Contribution: ,98 € 2 reporting periods: 1.Months Months QUAlity aware VIsualisation for the Global Earth Observation system of systems 3

OGC ® Elements that are not in the GEOPortal Documentation phase –No quality focused metadata model. No pixel level quality. Discovery phase: –Query It is not possible to use a quality indicator as a way to refine the search No user feedback consideration –Response Quality is not a criteria for ranking results Quality is not emphasize in the results Metadata can not be easily compared N o visual representation of the metadata completeness Evaluation phase –No pixel level quality visualization strategies Access phase –No data access with pixel level quality Interaction phase –No immediate user feedback 4 DocumentDiscoverEvaluateAccessInteract

OGC ® Embedding quality in all phases of the geospatial data exploitation DocumentDiscoverEvaluateAccessInteract DAB-Q: Query with quality information WMS-Q extension to relate data with spatialized quality layers WMS-Q extension to relate data with spatialized quality layers GEOLabel Graphical picture that helps to evaluate data User feedback system has been introduced WCS-Q: GMLCov extension for spatialized quality layers (in project) 5 GeoViQua Producer and User Quality Model, UncertML & QualityML GeoViQua Producer and User Quality Model, UncertML & QualityML

OGC ® Assessing fitness-for purpose peer and expert review better traceability and provenance information information on citations and usage of a dataset warnings about problems identified with a dataset and potential workarounds, ‘soft knowledge’ from data producers (e.g. recommendations for use which are not easily encoded using the existing standards) Chrisman, 1988: asked for ‘user experience’ to be included in data quality specs Epstein et al 1998: disclaimers only cover ‘reasonably foreseeable use’ Comber et al., 2006: identify mismatches between producer/user ontologies Devillers et al., 2007: tools needed to help users understand quality information. Boin & Hunter, 2007: users want a simple summary that they can interrogate for more detail as appropriate. Lush et al., 2011: Interviews identified that these needs are still not being met. Document 6

OGC ® Document 7

OGC ® 8 Reference datasets Document 8

OGC ® George Mason University Global Agricultural Drought Monitoring and Forecasting System Page 9 GEO Week. AIP6. Geneva January 13, 2014

OGC ® User feedback input Page 10 GEO Week. AIP6. Geneva January 13, 2014

OGC ® We needed to define a new model... Copyright © 2014 Open Geospatial Consortium

OGC ®...that uses known concepts... Rating Comment Usage Report Quality Overwrite Copyright © 2014 Open Geospatial Consortium

OGC ® …in an aggregation model… Based in feedback items that are related to “targets” –Primary –Secondary –Supplementary –Data –Metadata with the idea of feedback collection Copyright © 2014 Open Geospatial Consortium

OGC ® …and the summary capacity. This a dynamic element calculated during a query Copyright © 2014 Open Geospatial Consortium

OGC ® –Mandatory information on user role (application domain, expertise level etc). –Subject, application domain and keyword tags –Other optional information such as rating, comments, a quality override (superseding producer quality information), a report of usage or a citation. –Focus -spatial, temporal or other subsets to which feedback pertains 15 User feedback Interact 15

OGC ® 16 Populating the user model Interact 16

OGC ® Feedback Catalog integration –Discover of user feedback items to the Feedback Catalog –Upload of user feedback items to the Feedback Catalog –Merging of discovered metadata records with related user feedback items Support to quality-related constraints –Producer Quality Model constraints (e.g.: misclassification rate, count of data quality info elements, etc…) –User Quality Model constraints (e.g.: rating score, user domain, etc…) WMS-Q integration Insertion of GEO label link to all discovered records DAB-Q capabilities Discover 17

OGC ® CSW-Q extension CSW-Q extension Discover 18

OGC ® Components diagram Discover 19

OGC ® Portal integration Copyright © 2014 Open Geospatial Consortium

OGC ® Encoding When validating the Suitability Map of Golden Apple (link) with field survey evidences, we noticed that in the area of the “Corn de Borges Blanques, UTM: xxx” the results were not the expected. Instead of having high suitability rates for this particular fruit tree model matching the field reality, the multicriteria model shows low values. This conflict is caused by the GDD layer (an input in the fruit trees suitability multicriteria model). This layer is in turn derived from the mean June temperature, mean July temperature and mean August temperature layers included in the DCAIP (MD_Identifier id). This leads us to discover that the climatic surfaces of these layers in the DCAIP do not reflect appropriately these local climate features ultimately favouring the golden apple suitability (i.e., the climatic model locally in this geographic area used as input in the fruit tree model gives lower values than the high suitability evidences found in the field). This is caused by the lower than expected values of the nearest in- situ meteorological sensors used in the climatic surfaces interpolation models, even though the values series of this sensor are compliant with the time series length specifications. This causes that the residual interpolation structure is not representative of these particular climatic features. text/plain 5 The data itself is appropriated for this application and the metadata records is beyond the expectation allowing for error traceability. Copyright © 2014 Open Geospatial Consortium

OGC ® Motion Motion to move this idea into the OGC process. The Metadata.DWG proposes to the TC the creation of the Geospatial User Feedback Standards Working Group that will adopt the charter document OGC Geospatial User Feedback Standards Working Group Charter ersion=1 Geospatial User Feedback Standards Working Group Charter Copyright © 2014 Open Geospatial Consortium

OGC ® GEO label –a quality indicator for GEOSS geospatial data and datasets –assist in searching - provide users with visual clues of dataset quality / relevance. –provide accreditation, provenance, monitoring –increase visibility of EO data –emphasize open access and easy availability Evaluate 23 Producer quality model ISO19115 metadata User Feedback model

OGC ® GEO label meaning Producer of the dataset, e.g., organisation or individual who produced the dataset, their contact information, etc. Lineage/provenance information, e.g., processing applied to data and number of process steps. Informal comments about the dataset quality as provided by the dataset producer, e.g., any identified problems, suggested use, etc. Dataset’s compliance with international standards, e.g., compliance with ISO 19115, Dublin Core, etc. Quality measures of the dataset, e.g., uncertainty measures recorded in UncertML, errors, accuracy information, etc. Feedback, comments and ratings provided by the users of the dataset, e.g., general comments on dataset quality, identified problems, suggested use for the dataset, etc. Domain experts’ comments on dataset quality, e.g., results of formal quality checks, expert suggestions on the dataset applications, etc. Citations where the dataset was used and cited, e.g., formal reports on dataset quality checks, journal articles, etc. Evaluate 24

OGC ® GEO Label : Metadata+feedback summary with drill-down 25 Evaluate

OGC ® GEO Label-based Dataset Intercomparison Tool Evaluate 26

OGC ® Initiative to have GEOLabel as an IEEE standard Dear SiriJodha, Stefano and SCC40, I think the GEO Label concept is suitable as a potential standard to be developed by IEEE and would be willing to support SCC40 in sponsoring the corresponding project. The drafting of a Project Approval Request (PAR) would be the opportunity for the GEOSS Standards and Interoperability Forum to consult stakeholders (GEOSS IDIB and IEEE GEO WG/Hans-Peter Plag) to confirm the feasibility of such an approach and hopefully lead to a PAR. Copyright © 2014 Open Geospatial Consortium

OGC ® 28

OGC ® Some Useful URL’s Schemas XSD and UML (ASTON) – User Feedback System (S&T) – – GEO label services, API and code (ASTON+52N) – – – Tutorial (ASTON) – DAB-Q CSW, GI-CAT (CNR) description and code – Copyright © 2014 Open Geospatial Consortium

® Sponsored by Thanks Copyright © 2014 Open Geospatial Consortium