Enhancing Data Quality of Distributive Trade Statistics Workshop for African countries on the Implementation of International Recommendations for Distributive Trade Statistics May 2008, Addis Ababa, Ethiopia UNITED NATIONS STATISTICS DIVISION Trade Statistics Branch Distributive Trade Statistics Section

Outline of the presentation Quality measurement of Distributive Trade Statistics (DTS) Quality indicators versus direct quality measures Metadata on DTS Recommendations

Quality measurement of DTS Goal of quality measurement To provide the user with sufficient information to judge whether or not the data are of adequate quality for their intended use “Fitness for use” of the data The users must be able to: Verify that the conceptual framework and definitions that would satisfy their particular data needs are the same as, or sufficiently close to those employed in collecting and processing the data Asses the degree to which the accuracy of the data is consistent with their intended use or interpretation Quality management - all the measures that NSO takes to assure quality of statistical information

Data quality assessment frameworks QAFs – integrate various dimensions (aspects) of quality, their definitions and quality measurement Overall aim of QAFs Standardize and systematize statistical quality measurement and reporting across countries Allow an assessment of national practices to be made against internationally accepted statistical approaches for quality measurement Use of QAFs Guide countries ’ efforts for strengthening their statistical systems by providing a self-assessment tool and for identifying areas of improvement Technical assistance purposes Reviews of particular statistical domains performed by international organization Assessment by other groups of data users

Dimensions of quality (1) Prerequisites of quality All institutional and organizational conditions that have an impact on the quality of DTS data Elements – legal basis; adequacy of data sharing and coordination; assurance of confidentiality; adequacy of human, financial, and technical resources; quality awareness Relevance Degree to which DTS data meet the real needs of users Measuring relevance requires identification of user groups and their needs Credibility Confidence that users place in the data based on the image of the statistical agency that produces the data Trust in objectivity of the data Data are perceived to be produced professionally in accordance with appropriate statistical standards Policies and practices are transparent

Dimensions of quality (2) Accuracy Degree to which the data correctly estimate or describe the characteristics they are designed to measure Defined in terms of errors in statistical estimates Systematic errors Random errors Timeliness Delay between the end of the reference period to which the data pertain and the date on which the data are released Closely related to the existence of a publication schedule Involved in a trade-off against accuracy Accessibility Ease with which data can be obtained from the statistical office Suitability of the form or the media of dissemination through which the information can be accessed

Dimensions of quality (3) Methodological soundness Application of international standards, guidelines and good practices in production of DTS Elements - adequacy of the definitions and concepts, target population of units, variables and terminology underlying the data; information describing the limitations of the data Closely related to the interpretability of data Interpretability reflects the ease with which the user may understand and properly use/analyze the data Coherence Degree to which the data are logically connected and mutually consistent Coherence within datasets Coherence across datasets Coherence over time Coherence across countries

Quality indicators versus direct quality measures Quality measures Items that measure directly a particular aspect of quality - time lag from the reference date to the release date Most of them are difficult or costly to calculate in practice Quality indicators Summarize quantitative information to provide evidence about the quality or standard of data Do not measure quality directly but provide enough information for the assessment of quality - response rate is a proxy quality indicator for measurement of non-response bias Quality measures and quality indicators can either supplement or act as substitutes for the desired quality measurement

Quality Indicators Criteria for defining quality indicators Cover part or all of the dimensions of quality Methodology for their compilation is well established Indicators are easy to interpret Types of quality indicators Key indicators – coefficient of variations (accuracy), time lag (timeliness) Supportive indicators – average size of revisions (accuracy) Indicators for further analysis – user satisfaction survey (relevance)

Key quality indicators for DTS

Content of statistical data Microdata - data on the characteristics of units of the population Macrodata - derived from the microdata by grouping or aggregation Metadata - “data about data”, describes the microdata, macrodata or other metadata

Statistical metadata Fundamental purposes of metadata Describe or document statistical data Facilitate sharing, querying, and understanding of statistical data over the lifetime of the data Help users understand, interpret and analyze the data Help the producers of statistics to enhance the production and the dissemination of the data A bi-directional relationship between metadata and quality Metadata describe the quality of statistics Metadata are a quality component Provide a mechanism for comparing national practices in the compilation of DTS

Metadata on DTS Levels of metadata Structural metadata – integral part of DTS data tables Reference metadata - provide details on the content and quality of data, may accompany the tables or may be presented separately Components of DTS metadata Data coverage, periodicity, and timeliness Access by the public Integrity of disseminated data Data quality Summary methodology Dissemination formats

Recommendations (1) Quality dimensions are overlapping and interrelated and form a complex relationship. NSOs can decide to: Implement directly one of the existing QAFs Develop national QAFs that fit best their countries practices and circumstances Not all quality dimensions should be addressed for all data Countries are encouraged to select those quality measures/indicators that together provide an assessment of the overall strengths, limitations and appropriate uses of a given dataset Quality review of DTS should be undertaken every 4 to 5 years or more frequently if significant methodological changes or changes in the data sources occur

Recommendations (2) Countries are encouraged to: Accord a high priority to development of metadata Consider their dissemination an integral part of dissemination of DTS Adopt a coherent system and a structured approach to metadata across all areas of economic statistics, focusing on improving their quantity and coverage Identify user needs of metadata and arrange users into groups so a layered approach to metadata presentation can be applied Issue regularly, quality reports as part of their metadata

Thank You