1. LAMAS Working Group 7-8 December 2015
Agenda Item 4.1 Variance estimation in the LFS

2. Contents Estimation of variance for point estimates
Estimation of variance of annual net changes
Quality report: additional information on variance estimates

3. Use of variance estimates on request
Rare occurrence – about once a year;
Eurostat will forward request if only one or two MS are affected;
Eurostat will not give out confidence intervals, and explain limitations of methodology used.

4. POINT ESTIMATES Previous results and issues
Eight MS participated in this pilot project: EL, ES, IT, LV, PT, PL, SI and FI. Thank you !
Data for 2013Q1-2014Q4 transmitted, with two additional variables: STRATA and PSU
Variance estimation method: analytical approach using Taylor Linearization -> point estimates for some quaterly indicators
Methodology for annual average also tested

5. Previous results and issues
First results match quite well for some indicators but non negligible difference were observed for others -> Overestimation
Agreement to test the impact of the calibration

6. Results of tests taking into account the calibration
Test with Portugal and Latvia
STRATA, PSU, sampling coefficients, variables used for calibration and calibration margins
CALJACK SAS macro used
These two countries sent, in addition to STRATA and PSU, the sampling coefficients
(weights before calibration), the variables used for calibration and, in separate files, the
calibration margins.

7. Issues
Collinearity
Some calibration variables only concern a subsample
Margins in case of breakdowns
Weights affected by calibration
CALJACK cannot be applied to all calibration methods and countries design
During the tests a number of issues arose, among which:
- Collinearity (Collinearity arises when two or more variables are highly correlated.): In some cases, variables used for calibration cannot be used at the same time because of the existence of collinearity between the variables. This is
particularly the case when these variables have been computed using several
attributes. To correct this, some calibration margins and therefore some
modalities have to be dropped from the calibration design.
- Variables known only for subsamples: In theory the sum of the calibration
margins for each variable should be equal to the same number, i.e. the total
population. However some variables used for calibration concerned only a
subsample. For example, Latvia created a variable based on the information
from the State Employment Agency by sex and five age groups. This variable is
only defined for people registered in this agency and aged So, the sum of
the associated calibration margins cannot be equal to the total population as
unregistered persons have missing values.
- Margins for indicators breakdowns: Each time Eurostat wants an estimate for an
indicator breakdown (for example the employment rate of women or the
unemployment rate in a NUTS region), the calibration margins are affected as
the total population differs (it becomes the total female population or the
regional population in the examples). Consequently, Eurostat has to know the
calibration margins for each breakdown of each indicator. In the case of
indicators at national level, the variance is the sum of the variance for each
NUTS region (e.g. Portugal case).
- Weights affected by calibration: Working with subsamples, the obtained
calibrated weights will be different from NSIs weights (corresponding to
weights transmitted after calibration). Calibrated weights obtained from tests do
not reflect the total population dispersion as it concerns subgroups. It is very
important for Eurostat to obtain the same calibrated weights as Member States,
in order to obtain the same indicators.
- Tool to be used: The CALJACK macro may probably not be applied to all
calibration methods. NSIs use different sampling designs, different calibration
methods and different methods (Jackknife, Taylor linearisation) for estimating
standards errors. The CALJACK macro cannot be applied to countries using a
simple design excluding clusters, such as Finland and Slovenia.

8. POINT ESTIMATES: Conclusion and Way forward
Too complex to apply calibration on a large scale
Eurostat proposes to use the same simple solution (taylor) for all MS, to provide information on statistical significance
NSI are invited to provide their own estimates
Annual average -> tool developed conjointly with the net change project
Taking account of the calibration design in the variance estimation is a complex issue, which Eurostat is currently not able to apply on a large scale. Will be time consuming for MS too.
Main objectif: to find a quick and easy way for ponctual estimates when need arises.
Eurostat therefore proposes to use the same simple method for all countries to estimate the variance. The first results, prvious LAMAS were acceptable and drawn on the same conclusion
Eurostat will provide users with information on the statistical significance without disseminating the exact values, i.e. Eurostat will tell users if the value is in the confidence interval (CI) and/or if the CI is wide or narrow without giving the exact CI boundaries.

9. Estimation of variance of annual net changes for LFS based indicators
Variance of annual net changes and annual levels requested by DG Employment
Replace simple rule of thumb used now by DG EMPL; NOT used for compliance exercise!
List of 23 variables: 15 in group 1 and 8 in group 2
Group 1: start delivery of variances in June 2017, covering levels of 2015 and 2016, and net change
Group 2: start delivery of variances in June 2018

10. Further details on estimation
Estimation preferred by Member States, otherwise Eurostat will take over
Methodology: take quarterly and annual overlap into account
SAS code available (annex II), will be expanded to include all indicators in the list during 2016

11. Planning
Member States should indicate until end of March plans for estimations (full sets only); exceptionally, June 2016 (EoV)
Otherwise, MS should be prepared to send PSU and STRATA from 2016Q1 onwards, with backdata 2015Q1-Q4
Test transmission: 2016Q4
Final transmission: June 2017

12. Variance estimates in quality report
Differences in assumptions underlying the denominator for employment rates
Addition of open text box in annual quality report to explain approach

13. LAMAS is invited to: agree to use the simple analytical method
take note of the proposed approach for variance estimates of annual net changes
agree with the proposed timeline for variance estimates of annual net changes
confirm the national approach used in the calculation of CVs in quality reports