1. Non-response weighting at NatCen/ScotCen - a brief (and biased) history Susan Purdon Survey Methods Unit, NatCen

2. The case against 10 years ago non-response weighting not the norm Reasons: Response rates pretty high on govt sponsored surveys Beyond age and sex, not many control totals around

3. The case against Many surveys had long histories: weighting would introduce discontinuities Non-response is subjective and cosmetic! No two statisticians would create the same set of non-response weights. Unscientific. Weighting makes analysis more complex and error prone.

4. The counter-arguments Non-response rates have now grown (c. 1%/year) The under-representation of certain groups is a constant and clearly biasing Trends in response rates undermine the argument against introducing discontinuities ONS have concluded that all national statistics should be calculated on a consistent basis (same age-sex-region distn)

5. The implications Most surveys now come with non-response weights New industry of calculating weights for old surveys Not too comfortable a position (what if new method around the corner?)

6. How it’s done on gen pop surveys Approach kept as simple as possible. Adjust (standardise) for age-sex; no major attempt to eliminate other biases. Usual approach = calibration weighting where (a) adjust to national age-sex totals; but (b) give all household members same weight.

7. Calibration weighting Biases observed for individuals, but non- response is a household process (yes/no determined by whoever answers the door); Implies that probability of response depends upon who you live with (e.g. young men living with parents v. other young men) So only under-represent young men in certain types of households

8. Calibration weighting Calibration weighting attaches a probability of selection per household that ‘explains’ difference between survey and population age/sex. Assumes that non-response is not an independent process

9. More elaborate approaches Non-response hierarchies internal to the survey Panel attrition Studies with informative sampling frames

10. Suggestions for analysts Non-response weights are subjective. You don’t have to trust them. Check that observed differences are not attributable to weighting If you think the survey organisation has missed a trick then tell them!