Quality Assurance in Population and Housing Censuses

Slides:



Advertisements
Similar presentations
Data Quality Assurance and Dissemination International Workshop on Energy Statistics Aguascalientes, Mexico.
Advertisements

Enhancing Data Quality of Distributive Trade Statistics Workshop for African countries on the Implementation of International Recommendations for Distributive.
The United Nations Demographic Yearbook: In Need of Improvement Expert Group Meeting to Review the United Nations Demographic Yearbook System November.
1 Quality Assurance In moving information from statistical programs into the hands of users we have to guard against the introduction of error. Quality.
Slide 1 D2.TCS.CL5.04. Subject Elements This unit comprises five Elements: 1.Define the need for tourism product research 2.Develop the research to be.
Quality assurance activities at EUROSTAT CCSA Conference Helsinki, 6-7 May 2010 Martina Hahn, Eurostat.
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys Bangkok,
Chapter 9: Data quality and metadata Ilaria DiMatteo United Nations Statistics Division The 4 th meeting of the Oslo Group on energy statistics Ottawa,
Overall Quality Assurance, Selecting and managing external consultants and outsourcing Baku Training Module.
European Conference on Quality in Official Statistics Session 26: Quality Issues in Census « Rome, 10 July 2008 « Quality Assurance and Control Programme.
1 Improving Data Quality. COURSE DESCRIPTION Introduction to Data Quality- Course Outline.
Use of Administrative Data Seminar on Developing a Programme on Integrated Statistics in support of the Implementation of the SNA for CARICOM countries.
United Nations Regional Seminar on Census Data Dissemination and Spatial Analysis for Arabic Speaking Countries, Amman, Jordan May 2011 Identification.
The Role of Metadata in Census Data Dissemination Presented By Mrs. Shirley Christian-Maharaj Assistant Director of Statistics CSO Trinidad &Tobago.
UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region: Contemporary technologies for data capture, methodology and practice of data.
European Conference on Quality in Official Statistics 8-11 July 2008 Mr. Hing-Wang Fung Census and Statistics Department Hong Kong, China (
Statistical Expertise for Sound Decision Making Quality Assurance for Census Data Processing Jean-Michel Durr 28/1/20111Fourth meeting of the TCG - Lubjana.
Copyright 2010, The World Bank Group. All Rights Reserved. Principles, criteria and methods Part 2 Quality management Produced in Collaboration between.
Regional Seminar on Promotion and Utilization of Census Results and on the Revision on the United Nations Principles and Recommendations for Population.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
United Nations Oslo City Group on Energy Statistics OG7, Helsinki, Finland October 2012 ESCM Chapter 8: Data Quality and Meta Data 1.
United Nations Regional Seminar on Census Data Dissemination and Spatial Analysis for Arabic Speaking Countries, Amman, Jordan May 2011 Identification.
Census Processing Baku Training Module.  Discuss:  Processing Strategies  Processing operations  Quality Assurance for processing  Technology Issues.
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Asunción,
Copyright 2010, The World Bank Group. All Rights Reserved. Recommended Tabulations and Dissemination Section B.
Expert Group Meeting on the Revision of the Handbook on the Management of Population and Housing Censuses New York, 14 – 17 December 2015 Overview of the.
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Bangkok,
Regional Workshop on the 2010 World Programme on Population and Housing Censuses: International standards, contemporary technologies for census mapping.
Census quality evaluation: Considerations from an international perspective Bernard Baffour and Paolo Valente UNECE Statistical Division Joint UNECE/Eurostat.
10 August Inter-Regional Workshop on the Production of Gender Statistics New Delhi, India, 6-10 August 2007 Strengthening National Gender Statistics.
A Training Course for the Analysis and Reporting of Data from Education Management Information Systems (EMIS)
United Nations Statistics Division
Metadata models to support the statistical cycle: IMDB
Census Planning and Management for next Nigerian Census
Quality assurance framework for the census of agriculture
National Population Commission (NPopC)
Proposed Outline of Volume 2
30TH May 2017 Dar-es-Salaam, Tanzania
Quality assurance in population and housing censuses
Quality assurance in population and housing census SUDAN’s EXPERIANCE in QUALITY assurance of Censuses By salah El din. A . Magid OUR EXPERIANCE IN 5.
Census Planning and Management United Nations Statistics Division
Quality assurance in population and housing censuses
Quality assurance in official statistics
Census Planning and Management United Nations Statistics Division
Dissemination Workshop for African countries on the Implementation of International Recommendations for Distributive Trade Statistics May 2008,
Session 7.1 Data Quality 1.
4.1. Data Quality 1.
LIVESTOCK PRODUCTION AND PRODUCTIVITY
Linking Population and Housing Censuses with Agricultural Censuses
Survey phases, survey errors and quality control system
Measuring Data Quality and Compilation of Metadata
Survey phases, survey errors and quality control system
Metadata in the modernization of statistical production at Statistics Canada Carmen Greenough June 2, 2014.
Project Management Process Groups
Assessment of Quality in Statistics GLOBAL ASSESSMENTS, PEER REVIEWS AND SECTOR REVIEWS IN THE ENLARGEMENT AND ENP COUNTRIES Mirela Kadic, Project Manager.
Albania 2021 Population and Housing Census - Plans
Sub-Regional Workshop on International Merchandise Trade Statistics Compilation and Export and Import Unit Value Indices 21 – 25 November Guam.
Tool for Assessing Statistical Capacity (TASC)
Quality assurance in population and housing censuses
Planning and Implementation of Post Enumeration Surveys Pres. 4
United Nations Statistics Division
The role of metadata in census data dissemination
GSBPM AND ISO AS QUALITY MANAGEMENT SYSTEM TOOLS: AZERBAIJAN EXPERIENCE Yusif Yusifov, Deputy Chairman of the State Statistical Committee of the Republic.
Quality assurance in population and housing censuses
Metadata on quality of statistical information
Multi-Mode Data Collection
United Nations Economic Commission for Europe
STEPS Site Report.
Guy Van Gyes CAWIE-meeting 23-24/01/2012
International Standards and Contemporary Technologies,
Presentation transcript:

Quality Assurance in Population and Housing Censuses United Nations Regional Workshop on the 2020 World Programme on Population and Housing Censuses: International Standards and Contemporary Technologies, Colombo, Sri Lanka, 8-11 May 2018 Session 6 Quality Assurance in Population and Housing Censuses Meryem Demirci United Nations Statistics Division

Overview Importance of a quality assurance programme The quality assurance circle The role of managers Quality assurance by census phase Dimensions of quality Evaluation of quality Process quality Data quality

Importance of a quality assurance programme P&R recommends that “Each country must have a quality assurance and improvement programme to measure the quality of each stage of the census” (P&R 2.169) A major goal of quality assurance programme is to detect errors during the process and take remedial actions- a quality improvement programme The system should be managed in an integrated way and established at all phases of census operations including planning, questionnaire design, mapping, enumeration, flow of census materials, data capture, coding, editing/imputation, dissemination Quality of census processes will determine the quality of census results

Importance of a quality assurance programme Deficiencies in quality are usually the results of deficiencies in process rather than the actions of individuals working in that process The focus of quality assurance is: to monitor and measure the process quality, to detect errors early to improve procedures for achieving the targets to prevent errors from reoccurring –if possible-, The key to quality assurance and improvement is to be able to regularly measure the cost, timeliness and quality of a given process

The role of managers Managers have a vital role in establishing quality – their main roles include: Establishing a culture within the census agency that has a focus on quality issues –having a programme for all statistical operation, training all staff Ensuring the commitment of staff to strive to achieve high‐quality goals Creating an environment in which everyone has the opportunity to contribute to quality improvement Ensuring that processes for implementing quality assurance programmes are documented and such documentation provide information on: how quality is going to be measured who is involved in identifying root causes of problems with quality, how the process improvements are going to be implemented

Quality assurance circle Many census tasks are highly repetitive and a quality assurance programme can be established Careful design should identify: The types of errors that may occur What information is required to identify such errors How this information will be collected in a timely manner during live operations Who will be responsible for monitoring What action will be taken Implement corrective action Identify most important problems Identify causes of problems Measure quality

Quality assurance by census phase Quality assurance circle should be applied to the entire census cycle with: Performance in the previous phase being evaluated at any given level of detail Impacts of quality problems occurred in previous phase (s) should be evaluated Comparing with positive experience of the previous census Root  causes  identified  and  corrective  action  implemented

Quality assurance by census phase Topic selection The first step in managing the quality of census statistics to ensure that the product will be relevant to users and meets with the requirements outlined in census legislations The key process is extensive consultation with actual and potential users- user consultation programme Consultation with users in different formats: consultations with key government departments and agencies; advice from professional advisory committees in major subject matter areas; user feedback; ad hoc consultations with interested groups; Methods: Meetings, surveys Assessment of quality –preparing a report on user consultation programme, achievements, failure in inclusion of some topics and reasons

Quality assurance by census phase Form Design and Testing Second quality management task: testing of each census question and testing of the design The results of each test being analyzed and necessary changes made using quality circle approach Evaluation of improvements in census questionnaire and satisfaction of key internal stakeholders The dissemination team- ensuring to meet with the needs of users The subject matter specialist team; The team of the data capture or processing system The field operations team The respondents -- to ensure that the forms are easy to complete- self enumeration mail‐out/mail‐back and Internet Assessment report on satisfaction and improvements

Quality assurance by census phase Printing and distribution of census questionnaires/materials Establishment of a system for monitoring the quality of the work done by the printing company Regular check of the quality of the printed documents Monitoring the progress in printing Monitoring distribution of census materials from printing house to final destination – by timing and number of questionnaire and other census materials Assessment on the quality of printing and timely distribution of enough materials and collecting all materials from the field

Quality assurance by census phase Enumeration Checking the quality of the work of enumerators - coverage/content Collecting periodical information from enumerators/ supervisors to assess the progress in enumeration- through management and information system Collecting information on number of enumerated population/housing units, refusal, housing units with no contact Monitoring non-response and follow-up rates and implementing special procedures Monitoring the duration needed for completing enumeration- if not as scheduled, implementing remedial action Quality assessment based on information collected during enumeration-especially on census coverage and the quality of the work of field staff

Quality assurance by census phase Data capture, coding and editing/imputation Procedures should be developed with a view to minimizing the risk of erroneously cancelling, losing or artificially creating households/individuals during data processing-validation of enumerated population/households in different phases Procedures for monitoring the quality of each phase Repeating certain procedures based on the sample of batches/records and comparing two datasets Identifying systematic errors Assessment of the quality of data capture, coding and editing/imputation to ensure these procedures do not create significant errors in census data – Data capture error, Coding error, Imputation rate/Dissimilarity index

Quality assurance by census phase Dissemination The dissemination team is responsible for the timely delivery of products and services to the census data users Management of the quality in census dissemination is driven by concerns to: deliver relevant products and services while maintaining accuracy of the data and timeliness Assessment report on timely dissemination of census products and relevant tools for ensuring that the results are accessible to users

Dimensions of data quality Quality is a multidimensional concept Quality of outputs of any statistical exercise should be assessed with some or all of the following six main attributes: Relevance Accuracy Timeliness Accessibility Interpretability Comparability and coherence Some of these dimensions are inter-dependent and involve trade-off (eg. timeliness and accuracy)

Dimensions of quality: Relevance The relevance of statistical information is the degree to which it meets the needs of users -- and suggests the need to avoid the collection and production of data for which there is no significant use This dimension is important in census content development and dissemination Relevance is a qualitative assessment of the value of the census data produced, including in terms of meeting the mandate of the agency, legislated requirements and users` needs

Dimensions of quality: Accuracy The accuracy of statistical information is the degree to which those data correctly estimate or describe the quantities or characteristics that the statistical activity was designed to measure It is usually characterized in terms of errors in statistical estimates introduced by major sources of errors such as coverage error, content error, item non-response, imputation rate, etc. PES, information collected in field enumeration and data processing

Dimensions of quality: Timeliness Timeliness refers the length of time between the census reference day and the date on which the information becomes available There is a trade-off against accuracy Often for a census there are several release dates to be considered in dissemination schedule-temporary results, final results, thematic reports, online dissemination database, etc. Major information releases should have publication dates announced well in advance

Dimensions of quality: Accessibility The accessibility of statistical information refers to the ease with which it can be obtained The suitability of the form in which the information is available to users, the media of dissemination Where data products are provided at cost, the affordability of the information to users also affects accessibility

Dimensions of quality: Interpretability The interpretability of statistical information reflects the availability of supplementary information and metadata necessary to interpret and use it Usually it covers the underlying concepts, definitions, variables, classifications used, the methodology of data collection and processing and indications of the accuracy of the information

Dimensions of quality: Comparability The comparability of statistical information reflects the degree to which statistical information is comparable across countries and time Usually underlying concepts, definitions, classifications used, the methodology of data collection and processing provide information on comparability Using international recommendations on concepts, definitions, classifications and census methods increases comparability

Dimensions of data quality Coherence Coherence reflects the degree to which the census information can be successfully brought together with other statistical information within an integrated framework over time The use of standard concepts, definitions and classifications – possibly agreed at the international level - promotes coherence

Evaluation P&R recommends that “a complete evaluation takes place and is documented at the end of each phase of the census” Evaluation of the overall census operation is vital for identifying strengths and weaknesses of census phases, including planning, enumeration, data processing and dissemination, and also for the purpose of analysing the quality of census statistics, which are the major output of these processes. A comprehensive evaluation programme should include: Evaluation of census processes – operational assessments Evaluation of the quality of products- assessment of coverage and content errors and dimension of the quality The results of evaluations of census operation for both operational aspects and the quality of data should be made available to the stakeholders

Evaluation – Operational aspects/processes P&R recommends that “the census evaluation programme should be undertaken by subject specialists according to the agreed goals and methodologies covering all possible dimensions of quality.” Some areas for evaluation include: Identification of the deficiencies and achievements in data capture, coding and editing; Relevance of census data to user needs and satisfaction of users with dissemination tools and products (based on information collected through user consultation); Achievements and difficulties in use of new technologies and methodologies and identification of improvements for the next census; Realization of the census calendar, including the calendar of releasing census results, and, in the case of changes to the calendar, the reasons and consequences.

Evaluation – Data quality P&R recommends that “Evaluation of the accuracy of the census data should be undertaken, to the extent possible, by: conducting a post‐enumeration survey for measuring coverage and content errors, comparing the census results with similar data from other sources (surveys and administrative records in a similar time frame & previous census results) and applying demographic analysis The purposes of evaluating the accuracy of the data are to inform users on the quality of the current census data and to assist in future improvements.

Conclusions Quality assurance and improvement systems should be developed as part of the overall census programme, and integrated with other census plans, schedules and procedures The systems should be established at all phases of census operations, including planning, pre‐enumeration, enumeration, document flow, coding, data capture, editing, tabulation and data dissemination. Quality management procedures for each phase of the census should be assessed with appropriate methods