ESSnet on Small Area Estimation Stefano Falorsi Istat ESSnet Workshop, Köln 2011

Partners Istituto Nazionale di Statistica (ISTAT) Institut National de la Statistique France et des Etudes Economiques (INSEE) Statistisches Bundesamt (DESTATIS) Centraal Bureau voor de Statistiek (CBS) Statistisk Sentralbyrå (SSB) Główny Urząd Statystyczny (GUS) Instituto Nacional de Estadística de España (INE) Office for National Statistics (ONS) Swiss Federal Statistical Office (FSO) contacts : Stefano Falorsi : coordinator of the project stfalors@istat.it saessnet@istat.it Denisa Florescu : Eurostat xxx denisa.florescu@ec.europa.eu webpage : http://www.essnet-portal.eu/sae-2 ESSnet Workshop, Köln 2011

Needs for Small Area Estimation Basic diagram: U a population Ui a portion of the population, let us call it ith small area s a random sample from U The part of the sample s that falls in the small area Ui is si = Ui ∩ s The size ni of si is usually random and often small ESSnet Workshop, Köln 2011

When and How SAE? When to use SAE methods: Whenever direct estimator which is based only on sampling units observed for each small area are not reliable (small sample size or sometimes even no observed units) – i.e. variance or other measure of sampling variability considered to be too high for the indicator and level (area) under study How to use SAE methods: Exploiting relationship among variables by means of explicit or implicit modeling ESSnet Workshop, Köln 2011

SAE: Borrowing Strength from? How does SAE work? Borrowing Strength: Cross - sectional from Auxiliary data from Spatial relationship Over time ESSnet Workshop, Köln 2011

SAE methods SAE is a collection of different methods: Synthetic methods (often implicit assumptions on the nature of relationship; a simple case: rates at NUTS2 level = rates at NUTS4 level) Composite methods (linear combination of synthetic methods and direct estimators to balance bias and variance) Estimator based on linear mixed models (EBLUP, EB, HB) Non-linear models (e.g. logistic models for binary responses) with spatial and/or temporal correlation structures among random effects Semi- parametric models … ESSnet Workshop, Köln 2011

nevertheless Motivation for the project There is an increasing need for estimates at local and detailed geographic and socio-demographic level where direct estimation (based on sample data only) cannot ensure enough precision nevertheless policies in Official Statistics with regard to the implementation and quality of SAE need to be defined. Applications and experiences of SAE in Statistical Institutes overlap in terms of similar needs and sampling schemes, while may differ for external information Collaboration action creates synergies and allows everyone to benefit from the exchange of practices and know-how, representing the basis for identifying best practises in similar contexts ESSnet Workshop, Köln 2011

Motivation for the project The project takes advantage from the results (in terms of methodology and routines for the implementation of estimators) already available from previous projects on small area estimation (e.g. EURAREA, SAMPLE, BIAS, AMELI,...) The experiences of the ESS in small area estimation represent for all aspects of the project an important starting points for sharing knowledge The project aims at building a common reference framework for the development of SAE in the NSIs ESSnet Workshop, Köln 2011

Description of the Project The project is planned in phases which are a series of theoretical and application activities in order to facilitate and promote the use of small area techniques in the production of statistical information The results of the ESSnet project will be helpful to detect the best practises to define the guidelines to be followed for the applications of SAE to give some advices for software tools users. Dissemination of results is another important step of the ESSnet project, therefore a web-site will be available in order to share information and the results among the partners be a forum platform in order to boost the communication among NSIs willing to apply SAE methods. ESSnet Workshop, Köln 2011

Description of the Project The project is composed by 7 work-packages: WP1 - Project management WP2 (GUS,Destatis, INSEE, INE, ONS) State of the art (completed) The WP2 aims to provide a comprehensive overview of small area estimation in the ESS social surveys with respect to implementation, needs and expectations, the starting point will be represented by documents available on small area estimation, BUT the perspective of NSI have been highligthed; then the current application in UE NSIs and non-UE NSIs is a central issue of the review ESSnet Workshop, Köln 2011

Description of the Project WP3 (INE, Istat, CBS, INE, GUS, ONS) Quality assessment (completed) In order to support the choice and the use of SAE methods, this work-package aims to review and develop suitable criteria to assess the quality of SAE methods, in particular the study on methodologies for top-down assessment (from larger to smaller domains) and determination of accuracy thresholds, the review of methods for comparative assessment for the choice among different set of auxiliary information, the review on relevant methods for quality assessment (external validation using available information or MSE) and model diagnostic (model fitting, error analysis, etc.) As for WP2 the sharing of actual experiences of countries on the use of quality assessment has been the core of the WP ESSnet Workshop, Köln 2011

Description of the Project WP4 – (Istat, CBS, GUS) Software tools (still running) This WP aims at assisting real application of SAE methods, examining first the available routines and software useful for SAE. Special attention is devoted to examine the software in use in the NSIs. The main spotlight is on open source software for wider dissemination. Objective of this work is to provide recommendations for standardization and certification of ESS tools for SAE, with a list of desirable aspects of the software tools. For this reason, a study will be devoted to analyze the real capacity of routines to be applied to large scale surveys which are characterized by a large number of records and domains. ESSnet Workshop, Köln 2011

Description of the Project WP4 – (Istat, CBS, GUS) Software tools (still running) The main deliverable of the work-package is a collection of R functions to perform SAE estimation, model selection and model diagnostic. The R functions will include : Estimation : unit level : Synthetic, EBLUP (uncorrelated random effects), EBLUP (correlated random effects) area level : Synthetic, EBLUP (uncorrelated random effects) unit level (EBLUP type) logistic mixed model Model choice : conditional AIC and cross validation (unit and area level linear mixed model) Model diagnostic: - Bias diagnostic, Goodness of fit diagnostic, Coverage diagnostic, Calibration diagnostic ESSnet Workshop, Köln 2011

Description of the Project WP5 (CBS – FSO, GUS. INSEE, INE, Istat, SSB) Case studies (still running) The activities of this WP are focused on applying significant SAE methods acknowledged in WP2 and relevant tools for model diagnostic, model selection, quality assessment identified in WP3. The case studies will also be the ground for training the software or routines developed in WP4. The NSIs’ case studies focus on: CBS: Crime survey FSO: New Census strategy: simulation study from previous census GUS: Labour Force survey INE: New Census strategy: simulation study from previous census INSEE: Labour Force survey ISTAT: Health survey - SSB: Register of deaths: modelling mortality rates ESSnet Workshop, Köln 2011

Description of the Project WP6 (Istat – CBS, FSO, GUS, INE, ONS,SSB) Guidelines (still running) This work-package aims to summarise the activities and the results produced in the previous WPs in order to provide practical guidelines in ESS social surveys context. The guidelines will contain: - a description of the process that should be fullfilled when SAE methods are applied; - the standard SAE methods and their main extensions; - the main tools to assess the quality of the estimates; raccomandations about the use software and routines to be used to produce small area estimates; The guidelines will focus on issues related to real surveys. ESSnet Workshop, Köln 2011

Description of the Project WP7 - (Istat - CBS - Destatis, FSO, GUS, Insee, INE, SSB, ONS) Transfer of knowledge and knowhow (still running) The WP7 aims to transfer knowledge and know-how to non participating NSIs and to disseminate the results through a course and three on the job trainings. Three NSIs have already been selected for on the job training on LFS, and poverty indicators. A course for NSIs will be held in Rome, focus on case studies of the participants will be given as well. Aweb-site to update the information on the progress of the ESSnet to participating and non participating countries was designed within the ESSnet. The final report soon available at the ESSnet portal http://www.essnetportal.eu/sae-2. ESSnet Workshop, Köln 2011

Benefit of the ESSnet approach As stated above it allows synergies, exchange of practices and know-how, representing the basis for identifying best practises in similar contexts It is particularly motivated by needs of NSIs, guided by the discussion carried out in DIME Communication among similar projects Borrowing strength from other projects on the same area, avoiding useless overlapping and exploiting common topics – exchange of results Issues on SAE not exploited in the project but of great interest (to be developed in a future project?) this ESSnet covers cross-sectional methods; the temporal correlation in SAE methods will not be investigated here. This issue is important for all repeated surveys multivariate responses strategies for SAE at design and estimation level SAE for business surveys (now research activity in WP6 of BLUE-ETS, later the ESSnet structure after the end of the project may help the use in NSIs similarly as the current project) ESSnet Workshop, Köln 2011