1. Fredrik Olsson, Statistics Sweden, fredrik.olsson@scb.se
Analysis on nonresponse bias for the Swedish Labour Force Surveys (LFS)
Fredrik Olsson, Statistics Sweden,
28 June 2018
Session 18

2. Nonresponse rate in the Swedish LFS

3. Analysis on nonresponse bias for the Swedish Labour Force Surveys (LFS)
Joint work by
Martin Axelson
Vanja Hultkrantz
Pär Sandberg
Fredrik Olsson
Frida Videll
Report published at

4. Methods for analyzing effects of nonresponse
The Hansen-Hurwitz method
Subsample of nonrespondents
+ Enable analysis on survey variables
- Expensive
- Hard to collect data from the entire subsample
Study by using relevant and available data
Register variables are collected for the entire sample
+ Relatively cheap
+ Easy to repeat
- Not survey variables
There exists different methods for analyzing the effects of nonresponse.
In our work we discussed two methods, Hansen-Hurwitz and a register based analysis.
In the Hansen-Hurwitz method, a subsample of nonrespondents is sampled and asked about some or all of the survey variables.
On the plus side, this method enables analysis on the survey variables.
On the down side it is expensive and it’s hard to collect data from the entire subsample.
In a register based analysis, relevant and available data are used as an approximation for survey variables.
To enable this analysis, register variables are collected for the entire sample.
The advantages with this method is that it’s relatively cheap, especially compared to the Hansen-Hurwitz method, and that it’s easy to repeat.
One negative thing is that the analysis isn’t done on the survey variables.

5. Chosen method Analysis using register variables
𝜃 𝑧𝑟 is the estimator for variable 𝑧 under nonresponse
𝜃 𝑧𝑠 is the estimator for variable 𝑧 under full response
Bias : 𝐵 𝜃 𝑧𝑟 = 𝜃 𝑧𝑟 − 𝜃 𝑧𝑠
Relative bias: 𝑅𝐵 𝜃 𝑧𝑟 = ( 𝜃 𝑧𝑟 − 𝜃 𝑧𝑠 )/ 𝜃 𝑧𝑠
We chose to do an analysis using register variables.
We computed two different estimates for each register variable. These estimates are computed using the same estimation currently in use in the Swedish LFS.
𝜃 𝑧𝑟 is the estimator for variable z that is obtained under nonresponse.
𝜃 𝑧𝑠 is the estimator for variable z that is obtained under full response.
The estimated nonresponse bias is computed by subtracting the estimate for the sample from the estimate for the respondents.
The estimated relative bias is computed by dividing the estimated bias with the estimate for the sample.

6. Example of registers used
The Employment Register
Employed
Swedish Public Employment Service´s register of job- seekers
Unemployed
The Employment Register and Swedish Public Employment Service´s register of job-seekers
Not in the labour force
Register on Participation in Education
Students
Register over young people not in employment, education or training
NEET
We have used variables from different registers to approximate some of the sampling variables in the LFS.
From the Employment Register we use information to approximate the LFS-variable employed.
From the Swedish Public Employment Service’s register of job-seekers we use information to approximate unemployed.
We also use information from these two registers to approximate not in the labour force.
To approximate employees we use information from register on activity.
From Register on Participation in Education we use information to approximate students.
From Register on income and taxes we analyze three different groups of income.
To approximate NEET, we use Register over young people not in employment, education or training.

7. Summary. Age 16-74, December 2015 Employed Unemployed
Estimate respondents
Estimate sample
Bias
Relative bias
Employed
4 726 000 (±31 000)
4 673 000 (±23 000)
53 000 (±21 000) *
1,1 (±0,4)
Unemployed
276 000 (±18 000)
268 000 (±13 000)
8 000 (±13 000)
2,9 (±4,9)
Not in the labour force
2 180 000 (±32 000)
2 241 000 (±24 000)
-61 000 (±21 000)
-2,7 (±0,9)
Students
1 123 000 (±32 000)
1 019 000 (±23 000)
104 000 (±23 000)
10,2 (±2,2)
NEET (16-24, year 2014)
(±4 700)
(±4 000)
(±2 500)
-29,1 (±3,2)
Here’s a table that summarize the results for December 2015 for those aged
There are two different estimates of each variable. One estimate is based on the respondents and one estimate is based on the sample.
The estimated bias is obtained by subtracting the estimate based on the sample from the estimate based on the respondents.
The estimated relative bias is obtained by dividing the bias with the estimate based on the sample.
Each of these estimates are presented with margin of error.
If the estimated bias is higher than the margin of error, the bias is significantly different from zero and is marked with a star. The same goes for the estimate of the relative bias.
We can see that the relative bias is significantly different from zero for all variables except for unemployed.
For employed, the relative bias is 1,1…
We can see that NEET and students obtain the highest levels of relative bias. What they have in common is that they are focused on young people.

8. ”Employed”
Relative bias for employed. Age December 2011 – December Per cent.
Looking at the relative bias for employed for December 2011 to December 2015 we can see that it has been relatively stable over time.

9. ”Unemployed”
Relative bias for unemployed. Age December 2011 – December Per cent.
Looking at the relative bias for unemployed for December 2011 to December 2015 we can see that there has been some changes but there hasn’t been an increase in the relative bias.

10. ”Not in the labour force”
Relative bias for not in the labour force. Age December 2011 – December Per cent.
The relative bias for not in the labour force has been relatively stable looking at estimates for December 2011 to December 2015.
For the same time period the nonresponse rate has increased from 25 to 40 per cent.
Looking at these results we can’t see a clear indication of an increasing nonresponse bias as the nonresponse rate increases.

11. Estimates of change
Employed or unemployed according to register variables
Monthly basis
Comparison of corresponding months from one year to the next
Let’s continue with estimates of change.
In order to conduct this analysis we have created a variable for employed and unemployed using register variables.
This variable is created on a monthly basis and is used to compare corresponding months from one year to the next.

12. ”Employed” and ”unemployed”
Estimates of change regarding employed and unemployed. Age January 2013 – December 2015.
In this graph we can se the estimates of change regarding employed and unemployed for January 2013 to December 2015.
The blue lines represent estimates of employed and the purple line represent estimates of unemployed.
The solid lines represent estimates based on respondents and the dotted lines represent estimates based on the sample.
For unemployed, the estimates based on the respondents are similar to those based on the sample.
The same pattern is also seen for employed with some exceptions.

13. ”Employed”
Estimated bias and 95 per cent confidence interval. Estimate of change regarding employed. Age January 2013 – December 2015.
If we take a closer look at employed we can see that only a few of the observed differences are significantly different from zero.
In this graph we can see the estimated bias and the 95 per cent confidence interval.
If all of the marks belonging to a given period are over or under zero, we have a difference that is significantly different from zero.
Here we can see two observations that don’t cover zero.

14. ”Employed” and ”unemployed”, upper secondary or non-tertiary education
Estimates of change regarding employed and unemployed. Upper secondary or non-tertiary education. Age January 2013 – December 2015.
The pattern of higher levels of nonresponse bias for subgroups after education that we saw for estimates of level is also seen for estimates of change.
Here we can see the estimates for employed and unemployed for upper secondary or non-tertiary education.
For unemployed, the estimates based on the respondents are similar to those based on the sample.
For employed, the estimates based on the respondents tend to be lower than those based on the sample.
Let’s take a closer look at the estimates for employed…

15. ”Employed”, upper secondary or non-tertiary education
Estimated bias with 95 per cent confidence interval. Upper secondary or non-tertiary education. Age January 2013 – December 2015.
Here we can see that the estimates based on the respondents are systematically lower than the estimates based on the sample.

16. Summary – Estimates of level
The estimated bias is significantly different from zero for many estimates
Estimates with a focus on young people have the highest levels of estimated bias
NEET
Students
The relative bias has been fairly constant over time
We’ll start with estimates of level.
For many estimates, the estimated bias is significantly different from zero.
The highest levels of estimated bias is obtained for estimates with a focus on young people, that is for estimates of NEET and students.
We have also seen that the estimated bias varies over subgroup and that the highest levels are obtained for education.
Estimates by primary or lower secondary education tend to be underestimated and estimates by tertiary education tend to be overestimated.
When we looked at the relative bias over time we could see that it’s been fairly constant over time for employed, unemployed and not in the labour force.

17. Summary - Estimates of change
Unemployed
Estimates based on respondents are similar to those based on the sample
Employed
In general the same pattern as for unemployed
A difference is seen for education
Upper secondary or non-tertiary education
Estimates based on respondents are systematically lower than estimates based on the sample
Tertiary
Estimates based on the respondents are systematically higher than estimates based on the sample
For estimates of change, we looked at unemployed and employed.
For unemployed, the estimates based on respondents are similar to those based on the sample.
For employed, we can, in general, see the same pattern as for unemployed.
For upper secondary or non-tertiary education, estimates based on respondents are systematically lower than those based on the sample.
For tertiary education, estimates based on the respondents are systematically higher than those based on the sample.

18. Fredrik Olsson, Statistics Sweden, fredrik.olsson@scb.se
Analysis on nonresponse bias for the Swedish Labour Force Surveys (LFS) (
Fredrik Olsson, Statistics Sweden,