Workshop on Quality Assurance in Geographical Data Production 1 Results of the Survey on Quality Asurance Routines Anders Östman University of Gävle SWEDEN

Workshop on Quality Assurance in Geographical Data Production 2 Background Changing requirements –Increased focus on data maintenance –Trends towards subcontracting –New data products and services –User requirements (INSPIRE etc) Survey of software and procedures being used for data quality evaluation –Aim: Identification of needs not currently met –Questionnaire to European NMCA’s –In-depth interviews carried out

Workshop on Quality Assurance in Geographical Data Production 3 The questionnaire Not intended for quantification !!! 12 replies received during Q Concerns topographic data ~ 1:10k Questions on –Data quality specifications –Datasets, methods and software used for quality assurance and evaluation

Workshop on Quality Assurance in Geographical Data Production 4 Missing specifications Quality elementMissing Positional accuracyNone (0 %) Thematic accuracyMany (50 %) CompletenessMany (58 %) TimelinessFew (8 %) Logical consistencyFew (17 %)

Workshop on Quality Assurance in Geographical Data Production 5 How satisfied are you with the current software Very dissatisfied DissatisfiedNeitherSatisfiedVery satisfied Positional accuracy 45 Thematic accuracy 322 Completeness331 Needs for updating 31 Logical consistency 342

Workshop on Quality Assurance in Geographical Data Production 6 How confident are you on your QA routines Very unconfiden t UnconfidentNeitherConfidentVery confident Positional accuracy 2341 Thematic accuracy 3321 Completeness3321 Needs for updating 143 Logical consistency 1342

Workshop on Quality Assurance in Geographical Data Production 7 Additional findings Some organisations use quality assurance routines, but estimates not specified. Problems in proper estimation procedures? Field checks are costly, change detection problematic Need for tools, guidelines and procedures

Workshop on Quality Assurance in Geographical Data Production 8 Interviews 8 telephone interviews performed Not suitable for quantification Performed during Q3 2005

Workshop on Quality Assurance in Geographical Data Production 9 Why are some QE not specified? Difficult to estimate No procedures implemented No measures specified It’s a matter of time No user requirements In general there is a need for work procedures

Workshop on Quality Assurance in Geographical Data Production 10 QM as a tool for more effective data production? QM not a driving force for change, at the moment. Not discussed by management In conjunction with improving the products, not in conjunction with cost reduction At the moment, QM is seen as a cost Yes, and it is implemented In general, QM is seen as a cost, not a driving force for change. QM is often considered considered to be important by mgmt, but its strategical potential is less discussed.

Workshop on Quality Assurance in Geographical Data Production 11 QA of subcontractors work Same routines (most answers) We fully trust our subcontractor (long relationship) Our subcontractors must also perform an accreditation test. In general, no specific routines. Subcontracting often relying on old stable cooperation / accreditation

Workshop on Quality Assurance in Geographical Data Production 12 QA for new products The same (many replies) Other QA routines, because other types of data. I don’t know, but I assume they are sufficient. QA for services quite different. Difficult to answer for many respondents

Workshop on Quality Assurance in Geographical Data Production 13 User impact on product design Feedback mechanisms through –A named person in metadata –service / product management department –web service –user satisfaction surveys Both formal and informal routines used User feedback mainly for correcting errors. Complaints mainly concerns pricing (?) and less for product development

Workshop on Quality Assurance in Geographical Data Production 14 Importance of tools 1(3) Error propagation tool –Input: Statistical estimates + processing model –Output: Statistical estimates Example: Completeness of buildings –Existing databas was 92 % complete 5 years ago –It is updated using a method giving 80 % completeness –Detected changes are checked manually –Completeness of updated database? Answers ranging between 1 and 5. Median = 3.75, Average = 3.25 –Perhaps for completeness –For photogrammetric work and cadastral –Not for data production, perhaps for land valuation

Workshop on Quality Assurance in Geographical Data Production 15 Importance of tools 2(3) Cost optimising sampling tool –Input: Cost model + population of feature –Output: List of selected features Example: Estimating attribute accuracy –The correctness of an attribute is checked by field checks –Some kind of random sampling requiered –The cost of checking depends on distance to roads –Which features to check in a statistical correct way and with minimal cost? All answered 3 –Maybe in the future –Perhaps for attributes –Experienced based selection used at the moment

Workshop on Quality Assurance in Geographical Data Production 16 Importance of tools 3(3) Statistical estimation tool –Input: Redundant data + condition models –Output: Statistical estimates Example: Completeness of road data –We hava a road database, based on 1:10k data –We also have a road database from the city, based on 1:400 data –The major roads should coincide –How complete is the 1:10k road database? Answers ranging between 3 and 4.5. Median = 3.83, Average = 3.73 –Maybe in the future –At the moment, we don’t have any redundant data sets to use. Problems in access rights etc. –Other data sets are very useful

Workshop on Quality Assurance in Geographical Data Production 17 Why are completeness and thematic accuracy less commonly specified? –Quality evaluation does not seem to be the main problem. The tools being proposed received moderate enthusiasm –Of those who specify completeness and thematic accuracy, many are less confident about their current QA routines –QA routines for completeness and thematic accuracy seems to be difficult to implement, so therefore priority is given to other development tasks. –Specification is not the main issue. Good QA routines perhaps more important. –The ISO quality elements may be good for production control, but less for data quality specifications!?!

Workshop on Quality Assurance in Geographical Data Production 18 Data quality and product development Data quality usually a criterion, not in the center of the product development process. QA routines for subcontractors does not seems to be a hot issue, at the moment. Needs for accreditation? User feedback mainly for correcting errors. Informal feedback mechanisms indicates lack of customer focus?

Workshop on Quality Assurance in Geographical Data Production 19 Conclusions The partial lack of quality specifications doesn’t seem to be the main concern among the NMCA’s There is a need for improved QA routines It seems to be less need for additional tools for quality evaluation Data quality is not a strategic issue at managerial level Quality of services is an unknown area