General Register Office for S C O T L A N D information about Scotland's people General Register Office for Scotland 2006 Census Test – Evaluation Methodology Paul Fensom, GROS Bernard Baffour, UoS Royal Statistical Society 29 April 2008

General Register Office for S C O T L A N D information about Scotland's people Presentation Outline Brief overview of the Scottish Census Test Preliminary Results and Critique Further analysis Concluding remarks

General Register Office for S C O T L A N D information about Scotland's people Objectives Trial different enumeration methods on areas known to specifically contain ‘special’ population groups Complete census of households in selected areas Two Design Variables: Postal Delivery and Income

General Register Office for S C O T L A N D information about Scotland's people Block design of the 2006 Census Test Design Variables Block 1Block 2Block 3 PostoutDeliveryPostoutDeliveryPostoutDelivery Income PostoutDeliveryPostoutDeliveryPostoutDelivery No Income Glasgow NorthGlasgow SouthWest Dunbartonshire Block 4Block 5 PostoutDeliveryPostoutDelivery Income PostoutDeliveryPostoutDelivery No Income LochaberBreadalbane

General Register Office for S C O T L A N D information about Scotland's people Initial Results – Response Rates Postout 46%; Hand Delivery 53% households that had their forms hand delivered by enumerators tended to have a higher response rate Income 48%; No Income 44% income question did not detrimentally affect household response rate

General Register Office for S C O T L A N D information about Scotland's people Critique Lack of inference due to the non- randomness in the selection of areas –too much focus on ‘extreme’ areas –design not equipped to make national conclusions Introduction of intra-ED correlations –too much similarity in sub-areas –lack of randomisation of households

General Register Office for S C O T L A N D information about Scotland's people Further Evaluation Logistic Regression Analysis –‘Event’ (response) v ‘Non-Event’ (non- response) –Makes use of non-responding households Scottish Index of Multiple Deprivation –analyse effects of clustering

General Register Office for S C O T L A N D information about Scotland's people Methodology The normal logistic procedure assumes data sampled randomly; But this is not entirely valid for complex sample designs; Evidence of cluster effects in Scottish Census Test; Use of PROC SURVEYLOGISTIC.

General Register Office for S C O T L A N D information about Scotland's people Scottish Index of Multiple Deprivation (SIMD) There is a link between undercount and deprivation; Deprivation was used in the Design stage, so it follow that it needed to be included in analysis; Deprivation is a defining characteristic in logistic modeling of the results.

General Register Office for S C O T L A N D information about Scotland's people Clustering The design applied same treatments to neighbouring households. –Dependence due to clustering needs to be accounted for A ‘cluster’ variable created based on SIMD quintile and treatment effect to group ‘similar’ households.

General Register Office for S C O T L A N D information about Scotland's people Results Distribution of households by Deprivation Quintile

General Register Office for S C O T L A N D information about Scotland's people Results The odds ratios under both methods are the same BUT different standard errors.

General Register Office for S C O T L A N D information about Scotland's people Results – ‘best’ selected logistic model

General Register Office for S C O T L A N D information about Scotland's people Conclusion Results remain pretty much unchanged: –evidence that the income question did not have a significant effect on the household response. –the enumeration methodology had a significant effect on whether or not a household responded.

General Register Office for S C O T L A N D information about Scotland's people Remarks The Test has shown difficult challenges ahead for achieving high response in 2011 Census; Non-response schemes should be dynamic to take account of these difficulties.