LAMAS Working Group 7-8 December 2015 Agenda Item 4.1 Variance estimation in the LFS Hannah.KIIVER@ec.europa.eu Melina.ANTUOFERMO@ec.europa.eu

Contents Estimation of variance for point estimates Estimation of variance of annual net changes Quality report: additional information on variance estimates

Use of variance estimates on request Rare occurrence – about once a year; Eurostat will forward request if only one or two MS are affected; Eurostat will not give out confidence intervals, and explain limitations of methodology used.

POINT ESTIMATES Previous results and issues Eight MS participated in this pilot project: EL, ES, IT, LV, PT, PL, SI and FI. Thank you ! Data for 2013Q1-2014Q4 transmitted, with two additional variables: STRATA and PSU Variance estimation method: analytical approach using Taylor Linearization -> point estimates for some quaterly indicators Methodology for annual average also tested

Previous results and issues First results match quite well for some indicators but non negligible difference were observed for others -> Overestimation Agreement to test the impact of the calibration

Results of tests taking into account the calibration Test with Portugal and Latvia STRATA, PSU, sampling coefficients, variables used for calibration and calibration margins CALJACK SAS macro used These two countries sent, in addition to STRATA and PSU, the sampling coefficients (weights before calibration), the variables used for calibration and, in separate files, the calibration margins.

Issues Collinearity Some calibration variables only concern a subsample Margins in case of breakdowns Weights affected by calibration CALJACK cannot be applied to all calibration methods and countries design During the tests a number of issues arose, among which: - Collinearity (Collinearity arises when two or more variables are highly correlated.): In some cases, variables used for calibration cannot be used at the same time because of the existence of collinearity between the variables. This is particularly the case when these variables have been computed using several attributes. To correct this, some calibration margins and therefore some modalities have to be dropped from the calibration design. - Variables known only for subsamples: In theory the sum of the calibration margins for each variable should be equal to the same number, i.e. the total population. However some variables used for calibration concerned only a subsample. For example, Latvia created a variable based on the information from the State Employment Agency by sex and five age groups. This variable is only defined for people registered in this agency and aged 15-62. So, the sum of the associated calibration margins cannot be equal to the total population as unregistered persons have missing values. - Margins for indicators breakdowns: Each time Eurostat wants an estimate for an indicator breakdown (for example the employment rate of women or the unemployment rate in a NUTS region), the calibration margins are affected as the total population differs (it becomes the total female population or the regional population in the examples). Consequently, Eurostat has to know the calibration margins for each breakdown of each indicator. In the case of indicators at national level, the variance is the sum of the variance for each NUTS region (e.g. Portugal case). - Weights affected by calibration: Working with subsamples, the obtained calibrated weights will be different from NSIs weights (corresponding to weights transmitted after calibration). Calibrated weights obtained from tests do not reflect the total population dispersion as it concerns subgroups. It is very important for Eurostat to obtain the same calibrated weights as Member States, in order to obtain the same indicators. - Tool to be used: The CALJACK macro may probably not be applied to all calibration methods. NSIs use different sampling designs, different calibration methods and different methods (Jackknife, Taylor linearisation) for estimating standards errors. The CALJACK macro cannot be applied to countries using a simple design excluding clusters, such as Finland and Slovenia.

POINT ESTIMATES: Conclusion and Way forward Too complex to apply calibration on a large scale Eurostat proposes to use the same simple solution (taylor) for all MS, to provide information on statistical significance NSI are invited to provide their own estimates Annual average -> tool developed conjointly with the net change project Taking account of the calibration design in the variance estimation is a complex issue, which Eurostat is currently not able to apply on a large scale. Will be time consuming for MS too. Main objectif: to find a quick and easy way for ponctual estimates when need arises. Eurostat therefore proposes to use the same simple method for all countries to estimate the variance. The first results, prvious LAMAS were acceptable and drawn on the same conclusion Eurostat will provide users with information on the statistical significance without disseminating the exact values, i.e. Eurostat will tell users if the value is in the confidence interval (CI) and/or if the CI is wide or narrow without giving the exact CI boundaries.

Estimation of variance of annual net changes for LFS based indicators Variance of annual net changes and annual levels requested by DG Employment Replace simple rule of thumb used now by DG EMPL; NOT used for compliance exercise! List of 23 variables: 15 in group 1 and 8 in group 2 Group 1: start delivery of variances in June 2017, covering levels of 2015 and 2016, and net change 2015-2016 Group 2: start delivery of variances in June 2018

Further details on estimation Estimation preferred by Member States, otherwise Eurostat will take over Methodology: take quarterly and annual overlap into account SAS code available (annex II), will be expanded to include all indicators in the list during 2016

Planning Member States should indicate until end of March 2016 plans for estimations (full sets only); exceptionally, June 2016 (EoV) Otherwise, MS should be prepared to send PSU and STRATA from 2016Q1 onwards, with backdata 2015Q1-Q4 Test transmission: 2016Q4 Final transmission: June 2017

Variance estimates in quality report Differences in assumptions underlying the denominator for employment rates Addition of open text box in annual quality report to explain approach

LAMAS is invited to: agree to use the simple analytical method take note of the proposed approach for variance estimates of annual net changes agree with the proposed timeline for variance estimates of annual net changes confirm the national approach used in the calculation of CVs in quality reports