1. Weighting on National Statistics Household Surveys Jeremy Barton Office for National Statistics

2. Outline of talk The ONS surveys Why should we weight?
The weighting process
When should we use weights?

3. ONS social surveys Labour Force Survey (LFS)
General Household Survey (GHS)
Expenditure & Food Survey (EFS)
Family Resources Survey (FRS)
Omnibus Survey (OMN)

4. Labour Force Survey Quarterly panel survey (c. 56K hh per qtr)
HHs stay in survey for 5 qtrs
require estimates of:
totals (e.g. employment)
rates (e.g. unemployment)
interview: all hh members
Local boosts for annual estimates

5. General Household Survey
9,000 hhs per annum
housing, consumer durables, employment, health, family structure, pensions, education
also ad hoc trailers, e.g. drinking
interview: all hh members

6. Expenditure & Food Survey
Merger of Food and Expenditure surveys
7,000 HHs in UK
14 day expenditure diary
Expenditure and income
Food consumption and nutrient intake
Interview: all hh members

7. Family Resources Survey
For DWP
25,000 HHs per year
Income, benefits, pensions, savings
Fieldwork shared by ONS and NatCen
HHs, individuals, benefit units
Interview: all hh members

8. Omnibus Survey Interview: 1 adult per hh 1,800 adults per month
Core questionnaire and modules
Covers a great range of different topics

9. Why should we weight? Adjust for unequal selection probabilities
Adjust for nonresponse
Adjust our sample to match known population totals

10. Probability weights Weight µ 1/(prob of selection) Boost samples
EFS in NI, weight = GB weight /4
more common in ad hocs

11. Probability weights Subsampling of units Omnibus (1 adult per hh)
weight = # adults in hh
FRS (Multi-household addresses)

12. Nonresponse weights

13. Nonresponse weights Sample-based nonresponse methods
Split set sample into weighting classes
Estimate weighted response rates in each class
New weight is 1/RR

14. Nonresponse weights Response rates different in each weighting class
Means for major survey variables must differ between each class
Means for major survey variables must be same for R and NR within each class

15. Nonresponse weights GHS and EFS Based on Census-link studies 1991
Target nonresponse in specific demographic groups
Sampling frame information
Interviewer observations

16. EFS NR weighting classes

17. Population weights
LFS Mar-May 2003

18. Population weights Produce population totals of estimates
Reduce nonresponse bias further
Improve precision (reduce SEs)
Comparability across surveys
a.k.a. calibration, post-stratification

19. LFS Population weights
LFS - Individual level weights
raking to 3 controls:
5 yr age group by sex within region
Local Authority
Single years by sex
population projections

20. LFS Population weights
LFS HH level weights
Same weight each hh member (Lemaitre/Dufour)
software: Calmar
bounded weights
Age group 5 yrs and single years by sex and region

21. GHS/EFS Pop. weights HH-level weights Pre-weighted by NR/prob weights
Calibrate to 5-year age groups by sex and to region
Pop estimates excl. communal establishments

22. FRS Population weights
Calibration to:
Age group, sex, marital status
Lone Parents
Families
Tenure Type
Council Tax band
Region

23. The weighting process
EFS 2001/02

24. When to use weights Always (whenever you can)
Problems with presentation /interpretation
estimates / sample sizes / SEs
NR & probability weights tend to increase variances

25. When to use weights
Stat packages (e.g. SPSS) don’t always deal with weighting correctly
Scale weights to average 1
Stata/SAS survey estimation procedures
Calibration tends to reduce variances

26. Conclusions Weights combination of probability, NR, calibration
Required for unbiased estimation
May require specialist software for correct hypothesis testing

27. Current Issues Use of 2001 Census data Census-linked NR studies
Change in Pop. Controls (back-weighting)
Integrated survey (CPS)
LFS:
Attrition weighting
Local LFS
Number of controls

28. References Weighting for non-response, Elliot, D. NM17
Grossing Up - when and how, Butcher, B. SMB 14
The presentation of weighted data in survey report tables, Elliot, D. SMB 38
Using weights in regression analysis: A comparison between SPSS and STATA packages, Insalaco, F. SMB 45
Developing a weighting and grossing system for the GHS, Barton, J. SMB 49
Evaluation nonresponse on household surveys, Foster, K. GSS Methodology Series 8.
Report of the Task Force on Weighting and Estimation, Elliot, D. GSS Methodology Series 16.