1. DQ in the citizen science project COBWEB: extending the standards
96th OGC Technical Committee
Nottingham, UK
DQ in the citizen science project COBWEB: extending the standards
Didier Leibovici, Sam Meek, Julian Rosser & Mike Jackson University of Nottingham, UK
Data Quality DWG
16th September 2015

2. Copyright © 2015 Open Geospatial Consortium
On Monday the 14th VGI workshop: standards / data exchange / quality assessment
Challenges for VGI data quality assessment - role of standards
Outline:
VGI data quality model (data captured and citizen capturing)
Provenance: citizens & data curation process (including the QAQC)
Metaquality
Single data quality assessment vs dataset quality ‘aggregation’
Implications for data fusion and propagation of uncertainty
Copyright © 2015 Open Geospatial Consortium

3. mobile data capture & Quality Assurance / Conflation
COBWEB:
mobile data capture & Quality Assurance / Conflation
QA a priori , a posteriori and hic et nunc

4. FP7 COBWEB project: pilot case studies
EO enhancement:
content (linked data), accuracy (bidirectional)
Contribute to: land cover, habitat, biodiversity
Data existing:
optical, radar ,LiDAR, … plant ontologies?
Citizen Observations:
vegetation (species, communities, etc.) biophysical (moisture, greenness, phenological state)
Biological monitoring:
content (linked data), accuracy (bidirectional), timely and updated information
Contribute to: environmental policy, habitat
RSPB, linked to EO derived data,
Species and habitat related information (fauna, flora)
Flooding:
timely information, finer scale,
Calibrating flood models (hydraulic and erosion),
Validation of flood extents and water pathways
Contribute to: environmental policy, warning systems (hazards and risks)
Historical data, floodplains, flood risk maps, weather data,
Network of sensors
Time tagged photo (flood limits, colour [sediment transport])

5. Challenges for VGI data quality assessment - role of standards
facts:
Variation in the capturing device i.e., (User(s)) impacting data collected
accuracy vs completeness, e.g., OSM, biodiversity
precision but lack of consistency e.g., different ontologies
Different types of crowdsourcing .citizen science, VGI, passive crowdsourcing)
VGI used to ‘validate’ authoritative data
vs authoritative data helps to quality assure VGI
Other facts?
Copyright © 2015 Open Geospatial Consortium

6. Challenges for VGI data quality assessment - role of standards
needs:
Qualifying ‘users’ along with ‘single/series’ data
Flexible ways of understanding the QAQC
Metaquality & provenance (also at single data level!)
Machine readable provenance (to be used for data fusion)
Methods for aggregating / fusing single data qualities to datasets
Organising data & metadata at multiple levels
Copyright © 2015 Open Geospatial Consortium

7. Data life cycle Generic SDI-QA-Fusion-Decision
 QAQC service:
-enriches the data collected with quality metrics, update them as new data comes in
-feedbacks on existing data with quality metrics
-qualifies users with quality metrics by direct assessments or profiling
conflation service:
-retrieves relevant information
-compares and re-use informed quality of data
-combines the information to achieve better quality
meta-quality decision service:
-compares policy requirement and achieved data quality
-elaborates new data collection requirements
-estimates the potential impact of current data quality in the policy decision-making
-lifecycle of Life data   a. in COBWEB citizen science data is captured and qualified (metadata about quality) and eventually conflated/fused with authoritative data   b. as new data is collected the data management policy may imply redoing a. (new qualification as some qualification processes are like bootstrap, and of course the fusion makes use of the new data as well as the new quality evaluation) -the effective cross-fertilisation of people, data and processes across multiple biodiversity and environmental disciplines COBWEB aiming at Biospahere Reserve environmental policy envolving citizens via crowdsourcing into policy decision making the COBWEB platform enables cross-disciplinary policy making. he platform is suposed to allow conflation of sensor network data, authoritative data and crowdsourced data (citizen science)with data services (citizen science surveys), quality assurance service (sharing workflows of quality controls as WPS), conflation service (linking data and fusing data) - Biodiversity Data Integration (IG) and Geospatial IG ... metadata WG  COBWEB has EO validation for habitat for example as a major prototype (flooding, EO validation, biological monitoring (e.g. invasive species such as Japaneese Knotweed))

8. Flexible QAQC with authoring tool and WPS calls
QAwAT
.QA workflow Authoring Tool
BPMN encoding
.QA workflow Ontology
Semantic support
SKOS encoding
.running WPS, workflow engine
QAwOnt
The QA workflow
is composed of more than one QC
into a workflow
that may loop back /feedback to the user or to other users etc.
to get additional information.
(confirmatory / ensemble / linked data )
QAwWPS
WoQC-AT also QAwAT or now COBWEB QA workflow Authoring Tool
WoQC-O also QAwOnt or now COWEB QA workflow ontology
WoQC-WPS also QAwWPS or now COBWEB QA WPS

9. QA workflow Ontology (top classes) the 7 pillars
1.LBS: mainly devoted to position accuracy
2.Cleaning: erroneous / true mistakes / intentional mistakes removals/corrections position and attribute
3.Automatic Validation: simple checks: topology relations and attribute ranges
4.Authoritative Data Comparison: more statistical orientated to attribute values credibility as well as feature position/occurrence mking the most of legacy data and metadata about quality
5.Model-Based Validation: same as 4 but using previous crowdsourcing data or physical models or behavioural models
6. Linked Data Analysis: looking for evidence in social mediia and linked data framework etc…could in fact plug into into 5. afterwards
7. Semantic Harmonisation: conformance enrichment and harmonisation in relation to existing ontologies
Meek, S Jackson, M Leibovici, DG (2014) )
A flexible framework for assessing the quality of crowdsourced data
.AGILE conference, 3-6 June 2014, Castellón, Spain

11. A flooding data capture QA workflow
Qualifying the observations, the users and the authoritative data
Quality elements
-Obs /Auth - ISO19157 standard
-Auth - GeoViQUA-feedback model
-User -COBWEB-Stakeholder
Quality Model
O&M profile
Quality elements
Encoding
User quality!

12. Copyright © 2015 Open Geospatial Consortium
Quality models
DQ_ xxx producer model
ISO19157
GVQ_xxx consumer model
User Feedback (GeoviQua)
CSQ_xxx qualifying the user
COBWEB StakeHolder Quality model
A single QC
Copyright © 2015 Open Geospatial Consortium

13. Copyright © 2015 Open Geospatial Consortium
Quality elements
Extending the ISO19157
ISO19157:
DQ_Usability
DQ_Completeness
DQ_CompletenessCommission
DQ_CompletenessOmission
DQ_ThematicAccuracy
DQ_ThematicClassificationCorrectness
DQ_NonQuantitativeAttributeAccuracy
DQ_QuantitativeAttributeAccuracy
DQ_LogicalConsistency
DQ_ConceptualConsistency
DQ_DomainConsistency
DQ_FormatConsistency
DQ_TopologicalConsistency
DQ_TemporalAccuracy
DQ_AccuracyOfATimeMeasurement
DQ_TemporalConsistency
DQ_TemporalValidity
DQ_PositionalAccuracy
DQ_AbsoluteExternalPositionalAccuracy
DQ_GriddedDataPositionalAccuracy
DQ_RelativeInternalPositionalAccuracy
GeoViqua (simplified)
GVQ_PositiveFeedback
GVQ_NegativeFeedback
COBWEB Stakeholder Quality Model:
 where DQ_Scope will be "user"
CSQ_Ambiguity
CSQ_Vagueness
CSQ_Judgement
CSQ_Reliability
CSQ_Validity
CSQ_Trust
Copyright © 2015 Open Geospatial Consortium

14. Quality elements created concerning the Observation the User and the Authoritative data
Interactivity with the user (messages)

15. Similar QC in different pillars for different quality elements

16. Copyright © 2015 Open Geospatial Consortium
Modifications and accumulations of quality elements throughout the QA workflow
Copyright © 2015 Open Geospatial Consortium

17. Copyright © 2015 Open Geospatial Consortium
Potential outcomes
Combining quality models for citizen sciences: can all be seen as extending the ISO19157
Flexibility of QAQC: domain dependency, stakeholder dependency, fit for purpose dependency (?)
The whole QAQC workflow as «quality measure »  DQM_measure or DQ_EvaluationMethod (linked to metaquality and dataset
Producer Quality elements evaluations (DQ) depending on User Quality evalutions (CSQ) and feedback (GVQ)
Questions
Metaquality at single data captured level?
Updates of of the DQ GVQ and CSQ (do we keep the lineage of the updates?)
Aggregation or not of data quality from single users (series or single captured data) at dataset level (standalone report or metadata style) ….e.g., quality map
Need of a citizen data profile (see VGI workshop and Citizen observatories session)
Copyright © 2015 Open Geospatial Consortium