1. Small Area Estimation Programme
Dr Alison Whitworth

2. Outline History of SAE at ONS SAE and Census Transformation
Small Area Statistics Problem
Successful ONS experiences on Small Area Estimation
SAE and Census Transformation
Best use of administrative sources
Lessons learned and challenges

3. Small Area Statistics Problem
Requirement for:
Comprehensive, timely and reliable information
Data for detailed geographical areas or subcategories
But
Financial and operational constraints
Pressure to reduce survey sample sizes and respondent burden
Solution!
Borrow information from other related datasets
from similar areas, previous occasions

4. Borrowing strength Small area estimation methods: Modelling procedure
Use statistical models that relate the survey data with auxiliary information (borrow strength)
Auxiliary information: administrative data or census data available for all areas/domains
Modelling procedure
Variable of interest -> survey data
(Dependent variable)
Independent variables -> auxiliary data (covariates)
Model based estimates:
predicted values used for obtaining area/domain estimates

5. Successful ONS experiences
Small Area Estimation at ONS begun as a research project in 1990s
Small Area Estimation Project (SAEP), 1998
EURAREA,
Applications:
Unemployment (GB)– annual LAD and PCA estimates updated quarterly
Mean Income (E&W) – ward/MSOA estimates 98/99, 01/02, 04/05 and 07/08
Census – 2001 One Number Census, 2011

6. Mean household weekly income
SAEP method (Heady et al. 2003)
Estimates for Middle layer Super Output Area (MSOA)
Survey data: Family Resources Survey
(clustered household sample survey)
Auxiliary data: Social benefit claimants, Income tax, data, council tax banding, Census variables ….
Linear regression model for log income
Uses unit level response and area level covariates

7. 2011-2012 Official model-based MSOA estimates (England and Wales)
95% Confidence Intervals for net income, equivalised, after housing costs

8. Limitations of the ONS current approach (SAEP) for household income
Only allows for estimation of mean household income in each MSOA
User requirement for mean and percentiles estimates at lower level geographies

9. Research in progress in the SAE Unit
Examples :
Estimation of Household Income Distribution for 2001 and 2011 Using the Empirical Best Predictor (EBP) Method
Population Estimates by Local Authority and Ethnic Group Using Generalised Structure Preserving Estimators (GSPREE)

10. Example 1: Household Income using EBP approach
Empirical Best Predictor (Molina and Rao, 2010)
Estimation of mean and percentiles of income and poverty measures under one framework
Unit level model (household & area level covariates)
Involves prediction of income for each household in the population
Requires access to household level census data in addition to survey data (non-census years?)

11. 2001 EBP estimates North West and South East regions of England
Income Distribution across four different MSOAs

12. Coefficients of variation for income for
five percentiles of the population
2001 EBP estimates North West and South East regions of England

13. Conclusions EBP approach seems promising
MSOA estimates of income distribution and poverty measures under one methodology
Still more questions!
How to apply EBP approach in non-census years

14. Example 2: Population by Ethnic Group using GSPREE
Motivation:
National Statistician’s recommendation: make the best use of all available data in the production of population statistics.
Governments ambition: Censuses after 2021 be conducted using other sources of data…
Census Transformation programme Research the potential use of administrative data and surveys to produce population, household and characteristic information currently provided in a Census.
Generalise Structure Preserving Estimation (GSPREE) 14

15. Census Table Population by Local Authority and Ethnic Group
England (March 2011)
Local Authority
White
Mixed
Asian
Chinese
Black
Other
Total
Fareham
107959
1359
1200
467
357
239
111581
Southampton
203528
5678
16443
3449
5067
2717
236882
Portsmouth
181182
5467
9863
2611
3777
2156
205056 …. …
Tower Hamlets
114819
10360
96392
8109
18629
5787
254096
Slough
64053
4758
54900
797
12115
3582
140205
…..

16. Data for Ethnic Group 2011 Census estimates (Mar 2011)
Proxy: Detailed cross tabulation but outdated
School Census (Jan 2014)
Proxy: Detailed cross tabulation but age 5-15 only
Annual Population Survey (2014)
Total population by ethnic group
Mid Year Population Estimates (2014)
Total population by local authority

17. Solution…
Combine administrative and census data with survey data to borrow strength and produce reliable estimate for each cell (domain) using GSPREE (Zhang and Chambers, 2004 and Luna-Hernandez, A, 2014).

18. Data for Ethnic Group MYE 2014 Census 2011
MYE
White
Mixed
Asian
Chinese
Black
Other
Total
Fareham
107959
1359
1200
467
357
239
……..
Southampton
203528
5678
16443
3449
5067
2717
Portsmouth
181182
5467
9863
2611
3777
2156
…..... …. …
Tower Hamlets
114819
10360
96392
8109
18629
5787
Slough
64053
4758
54900
797
12115
3582
………
…..
APS July June 2014 (weighted estimates)
National total
……….
………..
School Census Dec 2014
White
Mixed
Asian
Chinese
Black
Other
Fareham …
Southampton
Portsmouth ….
Tower Hamlets
Slough
…..

19. Distribution of LA estimates by ethnic group, 2014
(England)

20. Proportion of White – GSPREE & Census

21. RMSE. LA by ethnic group, 2014 Fixed Effects GSPREE estimator
(England)
Overall, GSPREE is successful in providing reliable estimates for most LAs.
However, non-negligible RMSEs (and CVs) are observed in some areas

22. Conclusions GSPREE shows good performance Work in progress
Small bias/RMSE in most LAs
Work in progress
Validation study (1991/2001 Census)
GSPREE: 2001 Census x 2011 data (APS, MYE, ESC)
Validation: 2011 Census
Still more questions…
Modelling strategy for more detailed categories
Discuss alternatives to generate the synthetic population (bootstrap)
Consider different attributes

23. Academic support
Collaborative projects Structure Preserving Estimation (SPREE),
Expert advisory group Small area estimation
Funded research/ Bids e.g. NCRM
Conferences NCRM Bath (July)
SAE Maastrict (Aug)

24. Lessons learned Good small area estimates depend on: Challenges:
adequacy of the modelling procedures + covariates with good prediction power
model validation
Challenges:
ability to master the complexities of the required statistical theory
availability of relevant administrative/auxiliary data
capacity to overcome barriers for the acceptance of model based estimates as official statistics outputs

25. References
Molina, I. and Rao, J. N. K. (2010). Small area estimation of poverty indicators. The Canadian Journal of Statistics 38, Purcell, N. J. and Kish, L. (1980). Postcensal Estimates for Local Areas (or Domains). International Statistical Review, 48, Zhang, L.C. and Chambers, R. (2004). Small area estimates for cross-classifications. Journal of the Royal Statistical Society, B, 66, 479–496. Luna-Hernandez, A. (2014). On Small Area Estimation for Compositions Using Structure Preserving Models. Unpublished PhD upgrade document, Department of Social Statistics and Demography, University of Southampton.