DQ in the citizen science project COBWEB: extending the standards 96th OGC Technical Committee Nottingham, UK DQ in the citizen science project COBWEB: extending the standards Didier Leibovici, Sam Meek, Julian Rosser & Mike Jackson University of Nottingham, UK Data Quality DWG 16th September 2015

Copyright © 2015 Open Geospatial Consortium On Monday the 14th VGI workshop: standards / data exchange / quality assessment Challenges for VGI data quality assessment - role of standards Outline: VGI data quality model (data captured and citizen capturing) Provenance: citizens & data curation process (including the QAQC) Metaquality Single data quality assessment vs dataset quality ‘aggregation’ Implications for data fusion and propagation of uncertainty Copyright © 2015 Open Geospatial Consortium

mobile data capture & Quality Assurance / Conflation COBWEB: mobile data capture & Quality Assurance / Conflation QA a priori , a posteriori and hic et nunc

FP7 COBWEB project: pilot case studies EO enhancement: content (linked data), accuracy (bidirectional) Contribute to: land cover, habitat, biodiversity Data existing: optical, radar ,LiDAR, … plant ontologies? Citizen Observations: vegetation (species, communities, etc.) biophysical (moisture, greenness, phenological state) Biological monitoring: content (linked data), accuracy (bidirectional), timely and updated information Contribute to: environmental policy, habitat RSPB, linked to EO derived data, Species and habitat related information (fauna, flora) Flooding: timely information, finer scale, Calibrating flood models (hydraulic and erosion), Validation of flood extents and water pathways Contribute to: environmental policy, warning systems (hazards and risks) Historical data, floodplains, flood risk maps, weather data, Network of sensors Time tagged photo (flood limits, colour [sediment transport])

Challenges for VGI data quality assessment - role of standards facts: Variation in the capturing device i.e., (User(s)) impacting data collected accuracy vs completeness, e.g., OSM, biodiversity precision but lack of consistency e.g., different ontologies Different types of crowdsourcing .citizen science, VGI, passive crowdsourcing) VGI used to ‘validate’ authoritative data vs authoritative data helps to quality assure VGI Other facts? Copyright © 2015 Open Geospatial Consortium

Challenges for VGI data quality assessment - role of standards needs: Qualifying ‘users’ along with ‘single/series’ data Flexible ways of understanding the QAQC Metaquality & provenance (also at single data level!) Machine readable provenance (to be used for data fusion) Methods for aggregating / fusing single data qualities to datasets Organising data & metadata at multiple levels Copyright © 2015 Open Geospatial Consortium

Data life cycle Generic SDI-QA-Fusion-Decision  QAQC service: -enriches the data collected with quality metrics, update them as new data comes in -feedbacks on existing data with quality metrics -qualifies users with quality metrics by direct assessments or profiling conflation service: -retrieves relevant information -compares and re-use informed quality of data -combines the information to achieve better quality meta-quality decision service: -compares policy requirement and achieved data quality -elaborates new data collection requirements -estimates the potential impact of current data quality in the policy decision-making -lifecycle of Life data   a. in COBWEB citizen science data is captured and qualified (metadata about quality) and eventually conflated/fused with authoritative data   b. as new data is collected the data management policy may imply redoing a. (new qualification as some qualification processes are like bootstrap, and of course the fusion makes use of the new data as well as the new quality evaluation) -the effective cross-fertilisation of people, data and processes across multiple biodiversity and environmental disciplines COBWEB aiming at Biospahere Reserve environmental policy envolving citizens via crowdsourcing into policy decision making the COBWEB platform enables cross-disciplinary policy making. he platform is suposed to allow conflation of sensor network data, authoritative data and crowdsourced data (citizen science)with data services (citizen science surveys), quality assurance service (sharing workflows of quality controls as WPS), conflation service (linking data and fusing data) - Biodiversity Data Integration (IG) and Geospatial IG ... metadata WG  COBWEB has EO validation for habitat for example as a major prototype (flooding, EO validation, biological monitoring (e.g. invasive species such as Japaneese Knotweed))

Flexible QAQC with authoring tool and WPS calls QAwAT .QA workflow Authoring Tool BPMN encoding .QA workflow Ontology Semantic support SKOS encoding .running WPS, workflow engine QAwOnt The QA workflow is composed of more than one QC into a workflow that may loop back /feedback to the user or to other users etc. to get additional information. (confirmatory / ensemble / linked data ) QAwWPS WoQC-AT also QAwAT or now COBWEB QA workflow Authoring Tool WoQC-O also QAwOnt or now COWEB QA workflow ontology WoQC-WPS also QAwWPS or now COBWEB QA WPS

QA workflow Ontology (top classes) the 7 pillars 1.LBS: mainly devoted to position accuracy 2.Cleaning: erroneous / true mistakes / intentional mistakes removals/corrections position and attribute 3.Automatic Validation: simple checks: topology relations and attribute ranges 4.Authoritative Data Comparison: more statistical orientated to attribute values credibility as well as feature position/occurrence mking the most of legacy data and metadata about quality 5.Model-Based Validation: same as 4 but using previous crowdsourcing data or physical models or behavioural models 6. Linked Data Analysis: looking for evidence in social mediia and linked data framework etc…could in fact plug into into 5. afterwards 7. Semantic Harmonisation: conformance enrichment and harmonisation in relation to existing ontologies Meek, S Jackson, M Leibovici, DG (2014) ) A flexible framework for assessing the quality of crowdsourced data .AGILE conference, 3-6 June 2014, Castellón, Spain

A flooding data capture QA workflow Qualifying the observations, the users and the authoritative data Quality elements -Obs /Auth - ISO19157 standard -Auth - GeoViQUA-feedback model -User -COBWEB-Stakeholder Quality Model O&M profile Quality elements Encoding User quality!

Copyright © 2015 Open Geospatial Consortium Quality models DQ_ xxx producer model ISO19157 GVQ_xxx consumer model User Feedback (GeoviQua) CSQ_xxx qualifying the user COBWEB StakeHolder Quality model A single QC Copyright © 2015 Open Geospatial Consortium

Copyright © 2015 Open Geospatial Consortium Quality elements Extending the ISO19157 ISO19157: DQ_Usability DQ_Completeness DQ_CompletenessCommission DQ_CompletenessOmission DQ_ThematicAccuracy DQ_ThematicClassificationCorrectness DQ_NonQuantitativeAttributeAccuracy DQ_QuantitativeAttributeAccuracy DQ_LogicalConsistency DQ_ConceptualConsistency DQ_DomainConsistency DQ_FormatConsistency DQ_TopologicalConsistency DQ_TemporalAccuracy DQ_AccuracyOfATimeMeasurement DQ_TemporalConsistency DQ_TemporalValidity DQ_PositionalAccuracy DQ_AbsoluteExternalPositionalAccuracy DQ_GriddedDataPositionalAccuracy DQ_RelativeInternalPositionalAccuracy GeoViqua (simplified) GVQ_PositiveFeedback GVQ_NegativeFeedback COBWEB Stakeholder Quality Model:  where DQ_Scope will be "user" CSQ_Ambiguity CSQ_Vagueness CSQ_Judgement CSQ_Reliability CSQ_Validity CSQ_Trust Copyright © 2015 Open Geospatial Consortium

Quality elements created concerning the Observation the User and the Authoritative data Interactivity with the user (messages)

Similar QC in different pillars for different quality elements

Copyright © 2015 Open Geospatial Consortium Modifications and accumulations of quality elements throughout the QA workflow Copyright © 2015 Open Geospatial Consortium

Copyright © 2015 Open Geospatial Consortium Potential outcomes Combining quality models for citizen sciences: can all be seen as extending the ISO19157 Flexibility of QAQC: domain dependency, stakeholder dependency, fit for purpose dependency (?) The whole QAQC workflow as «quality measure »  DQM_measure or DQ_EvaluationMethod (linked to metaquality and dataset Producer Quality elements evaluations (DQ) depending on User Quality evalutions (CSQ) and feedback (GVQ) Questions Metaquality at single data captured level? Updates of of the DQ GVQ and CSQ (do we keep the lineage of the updates?) Aggregation or not of data quality from single users (series or single captured data) at dataset level (standalone report or metadata style) ….e.g., quality map Need of a citizen data profile (see VGI workshop and Citizen observatories session) Copyright © 2015 Open Geospatial Consortium