Crime survey Neuchatel, 7-8 July 2011 CBS case study Crime survey Neuchatel, 7-8 July 2011

Crime and victimization survey Planned domains: police districts Introduction Crime and victimization survey Planned domains: police districts Sample size approx 750 / district 2005-2008: NSM 2008 onwards: ISM SAE of crime statistics

Data collection: sequential mixed-mode Different questionnaire From NSM to ISM Local oversampling Data collection: sequential mixed-mode Different questionnaire Discontinuities expected SAE of crime statistics

Quantifying discontinuities Survey transition from NSM to ISM Small scale NSM in parallel to new ISM (full scale: approx 18,000; small: 1/3rd) Discontinuities at national level Now: police district level discontinuities required But: NSM sample too small => SAE SAE of crime statistics

Example of discontinuity Bicycle thefts NSM and ISM 2009 Total: 541,000 (NSM) ; 897,000 (ISM) SAE of crime statistics

Coeff of variation, bicycle theft, 2009 NSM: 0.41 ; ISM: 0.24 SAE of crime statistics

SAE to increase precision of NSM Fay-Herriot model: linear mixed, area level EBLUP and HB estimators Bayesian estimation of model variance also in EBLUP (avoiding zero-estimates of model variance) SAE of crime statistics

Bayesian estimation of model variance SAE of crime statistics

Covariates from registers Police: Reported offences: property crimes, violence, assaults, threats, illicit drugs, weapons, vandalism, traffic offences Administration: age, ethnicity, urbanisation, house prices, welfare claimants SAE of crime statistics

Covariates from ISM survey Design based GREG estimates as auxiliary information (Ybarra & Lohr 2008) Consequences for small area estimates Model estimate weighted lower in BLUP due to error in covariate Achieved through higher estimate of model variance in EBLUP (not Y & L adjustment) Variance of GREGs approx. equal for all areas (Other idea: multivariate FH model) SAE of crime statistics

Simulating errors in covariates Bicycle thefts: NSM survey ~ police-reported No error post. mean model var = 1.22 Adding error, mean 0, sd 2, iterate 1,000 x post. mean model var = 1.32 To add detail, e.g. estimated beta SAE of crime statistics

Dimension reduction: PCA Rather than using a small subset of covariates, use small dimension of PC subspace Not guaranteed to work as correlation with survey variables not used in PCA Use as a separate set of potential covariates in model selection pc 1 2 3 4 5 6 … 12 var. expl. .39 .55 .67 .75 .81 .87 .99 SAE of crime statistics

PC space of dim 2 SAE of crime statistics

Cross validation (CV) LOO: leave-one-out, predictive accuracy Model selection Conditional AIC (Vaida & Blanchart 2005) cAIC = - 2 cond_llh + 2 eff_d ( AIC = - 2 llh + 2 d ) Cross validation (CV) LOO: leave-one-out, predictive accuracy Start from minimal model, and add terms, maximizing improvement wrt cAIC or CV, until no further improvement Llh = log likelihood evaluated at ML estimates of fixed effects coeffs beta and model var Cond_llh = conditional llh: conditional on random effects D = model complexity = number of fixed effects Eff_d = effective degrees of freedom, between no random effect and random effect as fixed (between 1 and the number of areas), is trace of hat matrix SAE of crime statistics

Model selection results For each NSM survey variable: 2 years, 2 criteria CV-models are larger cAIC are nested within CV models Hence: Use cAIC models Models differ between years Alternative: choose single model for both years SAE of crime statistics

Selected models 2008 2009 violent crimes satisf. police victimization ISM-bicycle-theft, REG-property, REG-weapons, ISM-property pc21, pc10, pc4 satisf. police ISM-satisf age,ISM-satisf,urbanisation victimization ISM-property, REG-property pc1, pc21,pc5,pc6 property crimes ISM-victim, elderly pc1, pc21,pc2,pc5,pc6 nuisance ISM-nuisance, elderly ISM-victim, REG-traffic, ISM-property feeling unsafe ISM-nuisance, house val, ISM-satisf ISM-unsfae, ISM-satisf degradation pc1,pc4,pc10,pc22 ISM-degrad bicycle theft ISM-bicycle ISM-bicycle, ISM-satisf SAE of crime statistics

Selected models excl. ISM 2008 2009 violent crimes PC satisf. police victimization REG-property, elderly property crimes REG-property, age REG-property, REG-traffic, REG-weapons nuisance feeling unsafe degradation urban, house val, REG-vandalism bicycle theft SAE of crime statistics

SAE results (hybrid EBLUP), reduction in coeff. of variation incl. ISM excl. ISM violent crimes -40 % satisf. police -47 % -46 % victimization -43 % -41 % property crimes -44 % -42 % nuisance -33 % feeling unsafe -25 % degradation -35 % -16 % bicycle theft -39 % SAE of crime statistics

Bicycle theft, cv, 2009 NSM: 0.41, EBLUP: 0.23, ISM:0.24 SAE of crime statistics

SAE results, weight of direct est. in BLUP incl. ISM excl. ISM violent crimes 0.21 0.27 satisf. police 0.24 0.35 victimization 0.20 0.22 property crimes 0.19 nuisance feeling unsafe 0.39 0.41 degradation 0.31 0.64 bicycle theft 0.32 SAE of crime statistics

EBLUP vs. Hierarchical Bayes Diff. point est. Diff. var est. violent crimes -0.1 % -4.7 % satisf. police +0.0 % -3.8 % victimization -0.0 % property crimes -4.5 % nuisance -4.6 % feeling unsafe -3.1 % degradation -4.0 % bicycle theft -0.2 % -2.6 % HB accounts for uncertainty in estimating the model variance SAE of crime statistics

Considerable increase in precision with SAE Conclusions Considerable increase in precision with SAE Gain in precision depends on variable PCA is important for some variables Using ISM outcomes important for some variables MSE estimates HB higher (preferable) SAE of crime statistics

Sort out errors in input data! And re-run everything. To do (maybe) Sort out errors in input data! And re-run everything. Calibration to direct estimate of totals (is model diagnostic) Study residuals Elaborate on errors in covariates Use past survey outcomes as covariates More detailed comparison of HB-NSM estimates with ISM SAE of crime statistics

Future work (post-ESSnet) Multivariate modelling of NSM and ISM variables Consider model averaging Using more detailed areas, with smaller sample sizes: beneficial? SAE of crime statistics