Weighting on National Statistics Household Surveys Jeremy Barton Office for National Statistics

Outline of talk The ONS surveys Why should we weight? The weighting process When should we use weights?

ONS social surveys Labour Force Survey (LFS) General Household Survey (GHS) Expenditure & Food Survey (EFS) Family Resources Survey (FRS) Omnibus Survey (OMN)

Labour Force Survey Quarterly panel survey (c. 56K hh per qtr) HHs stay in survey for 5 qtrs require estimates of: totals (e.g. employment) rates (e.g. unemployment) interview: all hh members Local boosts for annual estimates

General Household Survey 9,000 hhs per annum housing, consumer durables, employment, health, family structure, pensions, education also ad hoc trailers, e.g. drinking interview: all hh members

Expenditure & Food Survey Merger of Food and Expenditure surveys 7,000 HHs in UK 14 day expenditure diary Expenditure and income Food consumption and nutrient intake Interview: all hh members

Family Resources Survey For DWP 25,000 HHs per year Income, benefits, pensions, savings Fieldwork shared by ONS and NatCen HHs, individuals, benefit units Interview: all hh members

Omnibus Survey Interview: 1 adult per hh 1,800 adults per month Core questionnaire and modules Covers a great range of different topics

Why should we weight? Adjust for unequal selection probabilities Adjust for nonresponse Adjust our sample to match known population totals

Probability weights Weight µ 1/(prob of selection) Boost samples EFS in NI, weight = GB weight /4 more common in ad hocs

Probability weights Subsampling of units Omnibus (1 adult per hh) weight = # adults in hh FRS (Multi-household addresses)

Nonresponse weights

Nonresponse weights Sample-based nonresponse methods Split set sample into weighting classes Estimate weighted response rates in each class New weight is 1/RR

Nonresponse weights Response rates different in each weighting class Means for major survey variables must differ between each class Means for major survey variables must be same for R and NR within each class

Nonresponse weights GHS and EFS Based on Census-link studies 1991 Target nonresponse in specific demographic groups Sampling frame information Interviewer observations

EFS NR weighting classes

Population weights LFS Mar-May 2003

Population weights Produce population totals of estimates Reduce nonresponse bias further Improve precision (reduce SEs) Comparability across surveys a.k.a. calibration, post-stratification

LFS Population weights LFS - Individual level weights raking to 3 controls: 5 yr age group by sex within region Local Authority Single years 16-24 by sex population projections

LFS Population weights LFS HH level weights Same weight each hh member (Lemaitre/Dufour) software: Calmar bounded weights Age group 5 yrs and single years 16-24 by sex and region

GHS/EFS Pop. weights HH-level weights Pre-weighted by NR/prob weights Calibrate to 5-year age groups by sex and to region Pop estimates excl. communal establishments

FRS Population weights Calibration to: Age group, sex, marital status Lone Parents Families Tenure Type Council Tax band Region

The weighting process EFS 2001/02

When to use weights Always (whenever you can) Problems with presentation /interpretation estimates / sample sizes / SEs NR & probability weights tend to increase variances

When to use weights Stat packages (e.g. SPSS) don’t always deal with weighting correctly Scale weights to average 1 Stata/SAS survey estimation procedures Calibration tends to reduce variances

Conclusions Weights combination of probability, NR, calibration Required for unbiased estimation May require specialist software for correct hypothesis testing

Current Issues Use of 2001 Census data Census-linked NR studies Change in Pop. Controls (back-weighting) Integrated survey (CPS) LFS: Attrition weighting Local LFS Number of controls

References Weighting for non-response, Elliot, D. NM17 Grossing Up - when and how, Butcher, B. SMB 14 The presentation of weighted data in survey report tables, Elliot, D. SMB 38 Using weights in regression analysis: A comparison between SPSS and STATA packages, Insalaco, F. SMB 45 Developing a weighting and grossing system for the GHS, Barton, J. SMB 49 Evaluation nonresponse on household surveys, Foster, K. GSS Methodology Series 8. Report of the Task Force on Weighting and Estimation, Elliot, D. GSS Methodology Series 16.