Administrative Data and Official Statistics Administrative Data and Official Statistics Principles and good practices Quality in Statistics: Administrative Data and Official Statistics High-level Seminar for EECCA countries, 4-6 September 2013, Vilnius, Lithuania United Nations Economic Commission for Europe Statistical Division

UNECE Statistical Division Slide 2 Primary vs. Secondary Data Sources /1 Primary data sources (statistical surveys):  Primary data are sources containing information which is exclusively collected for official statistics’ purposes  Primary data sources (statistical surveys) are collected exclusively by statistical authorities Secondary data sources (administrative data):  Administrative data are primarily collected for non-statistical purpose, in general by a public agent implementing an administrative regulation  The agent that supplies the administrative data to statistical authorities and the unit to which the data relates (observation unit) are different

UNECE Statistical Division Slide 3 Primary vs. Secondary Data Sources /2 Traditional concept Statistical sources Statistical surveys Administrative sources Broader concept Statistical sources Statistical surveys Secondary sources Other sources Private Administrative sources

UNECE Statistical Division Slide 4 The Principles (CoP and FPs) /1 Choice of data sources:  Data for the production of official statistics may be drawn from all type of sources, be they statistical surveys (primary sources) or administrative data (secondary sources)  Statistical authorities are to choose the data sources based on professional considerations and in particular with regard to quality, costs and the burden on respondent Professional independence:  Statistical authorities have the sole responsibility for deciding on the methods and procedures for the collection, processing and storage of data sources

UNECE Statistical Division Slide 5 The Principles (CoP and FPs) /2 Access to administrative data:  Statistical authorities have a clear legal mandate to collect information for the production of official statistics  Administration (holders of administrative data) may be compelled by law to allow access to administrative records or deliver administrative data at the request of statistical authorities; including confidential micro-data  Statistical authorities co-operate with owners of administrative data in assuring data quality

UNECE Statistical Division Slide 6 The Principles (CoP and FPs) /3 Confidentiality:  Individual data/micro-data from natural and legal persons held by statistical authorities are to be strictly confidential and used exclusively for statistical purposes  The principle of confidentiality applies for all data sources, be they statistical surveys (primary sources) or administrative data (secondary sources).  Statistical confidentiality is guaranteed in law

UNECE Statistical Division Slide 7 Defining the National Statistical System  The National Statistical System consists of all producers of official statistics (statistical authorities)  Statistical authorities strictly comply with the national statistical law and the Principles; and in particular:  P rofessional independence and impartiality  Statistical confidentiality  Equal (and simultaneous) access to statistical information  Sound methodology and international comparability  Output of statistical authorities are listed in the statistical programmes submitted to the Statistical Council  Other holders or providers of data sources (micro or aggregated) are not to be considered as producers of official statistics

UNECE Statistical Division Slide 8 National Statistical Institute (NSI) Other statistical authorities Statistical surveys primary sources Statistical sources/registers primary and secondary sources National Statistical System Statistical Law and Principles Administrative data/registers Administration Ministries - Agencies Specific regulations

UNECE Statistical Division Slide 9 Benefits of Using Administrative Data  Costs: statistical surveys and censuses are expensive to conduct and process (human and financial resources)  Response burden: using administrative sources reduces the burden on respondents and allow more frequent data collection  Coverage: in some cases administrative sources offer a better coverage of the target population and allow the production of reliable information for small-area and small population group  Timeliness: administrative data might be faster to process than survey data

UNECE Statistical Division Slide 10 Quality Issues of Administrative Data /1  Administrative data are collected according to administrative concepts and definitions  The main purpose of collecting data for administrative purposes and statistical data collection are often different  Administrative units and definition of variables may be different to statistical concepts: household, enterprise, market/non-market, public/private, employment …  In some cases nomenclatures and classifications may differ

UNECE Statistical Division Slide 11 Quality Issues of Administrative Data /2  Administrative reference period (e.g. financial year) may not coincide with the statistical one  Missing data and other quality issues are often not documented  The holder of administrative data refuses to give access to or transfer the requested information to statistical authorities  IT systems and software are different and do not allow data transfer or data access

UNECE Statistical Division Slide 12 Some solutions /1  Providers of administrative data should be clearly distinguished from statistical authorities (e.g. statistical law and statistical programme)  Revise the law, if necessary, to allow statistical authorities to get access to administrative data (and not the other way round!)  Establish good communication with the providers of administrative data and explain the advantages for all (and not for the statistical authorities) to grant access to administrative data for statistical purposes

UNECE Statistical Division Slide 13 Some solutions /2  If needed, conclude an agreement on data transfer/data access with the relevant holders of administrative data covering:  Legal basis (statistical law and in particular mandate for data collection and confidentiality provisions)  Description of the purpose of the data transfer/data access  Detail description of data covered including frequency and timeliness  Quality standards and quality reports  Technical standards  Cooperation mechanisms for improving the adequacy of the administrative data to statistical requirements (consultation mechanisms)  Contact persons Confidentiality rules imply no feed back on micro data to holders of administrative data: single flow principle. This is possible only with “certified” statistical authorities

UNECE Statistical Division Slide 14 Some solutions /2  But quality of statistical products can be improved by:  Matching and linking different data sources: statistical surveys and administrative data  Data editing, outlier detection and treatment  Imputation  Documenting quality issues and solutions  Improving coherence and consistency of different data sources  A statistical registers is an excellent tool for integrating data from different sources  A statistical register can (should) contain additional variables than the related administrative register or data sources A statistical register is owned and controlled by statisticians (single flow principle)