The second wave of the new design of the Dutch EU-SILC: Possibilities and challenges Judit Arends

Overview The Dutch EU-SILC (from 2005) Reasons of redesign - new design Redesign 2016 and 2017 Datacollection: introducing CAWI and incentives Sampling design Questionnaire design Field-work design Results of P1 2016 and 2017: response rate, data quality (missings), incentives, some challenges Design and results of P2 in 2017: response rate, quality Plans and discussion points

EU-SILC: sources of data Statistics Netherlands Register country: most information from registers Selected respondent: only one person is interviewed Income component Mostly registers Material deprivation variables Survey Work intensity: employment status Register / Survey

Design EU-SILC: Statistics Netherlands From registers: Construct nearly all income target variables based on census income from the tax authorities in t-1 Some other target variables: country of birth, citizenship, NUTS, ethnic origin Calender of activities: main income source in a specific month Employee status From 2016: child care costs, rent, housing cost (HH070) From survey/interview: Child-support and students, income from other sources All material deprivation items Working hours and current (self-defined) status

Design EU-SILC from 2005 until 2016 Integrated in the LFS: approaching EU-SILC respondents one month after the 5th wave of the LFS CATI only, one person is chosen randomly as the respondent of EU-SILC Respondents will be re-approached 3 times (1 year) Advantage: Response rate is high (80%) BUT: Response is selective and design is complex Changes in LFS: panel attrition in 2nd wave

Redesign – new design Aim: Working with a new stand-alone design from 2016: ‘’uncoupled’’ from the LFS New design for data-collection / fieldwork New sampling design Adapted questionnaire New / adapted data processing Longitudinal part: current design until 2019

Cross-sectional and longitudinal design Cross-sectional 2016 Longitudinal 2016 Cross-sectional and longitudinal 2017 Cross-sectional and longitudinal 2019 P1 2016 P2 2015 P3 2014 P4 2013 P2 2015 P3 2014 P4 2013 P1 2017 P2 2016 P3 2015 P4 2014 P2 2016 P3 2015 P4 2014 P1 2019 P2 2018 P3 2017 P4 2016

Main features of the redesign Subject Design from 2005 Redesign from 2016 Sampling design Persons from households LFS Persons Oversampling low income groups Duration fieldwork Four months June - September March - June Data collection mode CATI CAWI and CATI Questionnaire Designed for CATI Uni-mode design Incentives No Yes (split half in 2016)

Challenges (framework, development) Data-collection: Redesign: using mostly CAWI: 60-70% of the responses Questionnaire design: uni-mode design: CAWI and CATI Respondents of 16/17 years old: difficulties with household questionnaire in CAWI-mode Non-respons / panel attrition (incentives)

Main reasons: introducing web surveys SN has issued a data collection strategy: combining administrative data and data from surveys Data collection budgets: cheaper alternatives Clients’ request Many potential respondents cannot be reached: CATI: 52% known telephone number, high noncontact Respondents are not willing to participate in surveys in which an interviewer is involved Increased access to internet and internet users in NL “Experiments’’ from 2005 with web-surveys Steps to enhance response rate CAWI

Sampling design 2016 Sample persons were drawn form the sampling frame of persons from the Population Register Stratified sampling design Strata: income, household size, and 16 years 30 strata: 10 decile income groups (t-2), 16 years household size 17+

Sampling size Screening: - 7% 1.07*2.84*16.268=49.435 units strata age Hh size Income decil total population over-sampling 1 17+ 1301 441262 2.45 2   1402 409657 2.84 3 1229 389943 2.62 4 911 352467 2.15 5 613 311516 1.63 6 496 275288 1.50 7 402 245862 1.36 8 346 219692 1.31 9 287 190783 1.25 10 289 173555 1.38 11 2 or more 643 539456 0.99 12 788 575937 1.14 13 930 733345 1.05 14 896 849475 0.88 15 815 990995 0.68 16 851 1120995 0.63 17 859 1230166 0.58 18 898 1331365 0.56 19 931 1435404 0.54 20 1022 1442278 0.59 21 1 or 2 39 15017 2.18 22 52 18777 2.27 23 45 17759 2.10 24 35 16760 1.75 25 34 20404 1.37 26 22900 1.26 27 32 23481 1.16 28 31 22678 1.12 29 20753 1.08 30 20641 1.18 Tot 16268 13506529 1.00 Screening: - 7% 1.07*2.84*16.268=49.435 units Thinning out: each strata

Questionnaire design: Uni-mode design Uni-mode design: CAWI and CATI Household composition (gender, birth year) Personal questionnaire for selected respondent Work and educational attainment Personal material deprivation variables and health Households questionnaire Head of the household (CAWI?)

Design field work

Design field work PARTS START RAPPEL END CAWI 1 3-mrt 17-mrt 24-mrt   17-mrt 24-mrt 4-apr CATI 1 29 -apr CAWI2 31-mrt 14-apr 21-apr 2-mei CATI 2 31-mei CAWI 3 28-apr 12-mei 19-mei 30-mei CATI 3 30-jun

Invitation letters 16.268 persons received a letter of invitation to complete an online questionnaire URL Username Password

Questionnaire

Call centre in Heerlen

Response rate P1 2016 Total response: 42%: 6.794 (incentive: 46%; no: 37%) CAWI: Response: 29%: (incentive: 34%; no: 24%) Non response: 3% Goes to CATI: 37% No tel.nr: 31% CATI: Response: 34% (35%-33%) Sampling frame error (wrong tel.nr.): 16% Refusal: 21% Non-contact 14% Others 15%

Response distribution income group Strata Age Hhsize Incomedecil Response 1 17+ 26% 2 29% 3 33% 4 37% 5 42% 6 41% 7 8 46% 9 47% 10 39% 11 2 or more 35% 12 31% 13 14 45% 15 49% 16 17 51% 18 54% 19 56% 20 57% 21 1 or 2 1 to 5 22 6 to 10 Total

Response distribution: income group

Response rate CAWI: 70% (4.763); CATI:30% (2.031) Mean age: CAWI: 48 years; CATI 58 years Participating next year: Total: 78% CAWI: 74% CATI: 85% With incentives: 81% Without incentives: 73%

Results: General and Households No differences between incentive and non-incentive groups regarding Questionnaire/interview duration CAWI: 28 -27 minutes (5% outliers) Item non-respons/missings/refusals More item non-response in CAWI than in CATI (3% loss) Some problems with household composition in CAWI: students and mixed families Missing birth year, especially students (3% loss)

Results: Missings Variabeles Totaal CAWI CATI CAWI incentive no incentive One-week annual holiday 5,2 7 1 6,2 8,1 Keeping home warm 3,7 5,1 0,3 4,6 5,9 Replacing furniture 6,7 9,2 0,8 8,5 10,2 Replacing cloths 4,9 0,9 6,1 7,6 Meeting family 6,9 9,5 8,9 10,3 Warm meal every second day 3,3 4,7 0,2 4,3 Unexpected expenses 7,7 9,8 2,5 10 9,6

Lessons learnt for 2017: P1 Sampling: 22 strata Challenges with CAWI: Household composition: definition Item non-response: no refusal Instruction for interviewers regarding birth date Incentives: less sample units Half: iPad lottery Half: 10 euros

Response rate P1 2017 (total: 42%) CAWI (14.000): Response: 32%: (10 euro: 33%; iPad: 30%) (29%) Non response: 2% ( 3%) Goes to CATI: 33% (51%  thin out: 36%) (37%) No tel.nr, no CATI: 33% (15% + 18%) (31%) CATI (4.597) 2017 2016 Response: 32% (34%) Sampling frame error (wrong tel.nr.): 16% (16%) Refusal: 19% (21%) Non-contact 20% (14%) Others 13% (15%)

Response distribution income group Stratum age hh size income decil oversampling response 1 17+ 2,75 2 2,90 3 2,42 4 2,02 5 1,75 6 1,72 7 1,81 8 1,66 9 1,48 10   1,64 11 2 or more 0,94 12 0,95 13 0,81 14 0,71 15 0,63 16 17 0,64 18 0,61 19 0,56 20 21 1 or 2 1-5 1,40 22 6-10 1,09 total

P2 in 2017 Field work: CAWI and CATI Approaching in the same months in 2017 (M-M) Working with incentives: 5 euros in advance for everyone Taking over information about household, work and education: check (are there any changes?)

Response rate P2 2017 (total: 80%) CAWI (n = 4.173): Response: 65% (no differences between the P1 2016 groups) Non response: 2% ( 79) Goes to CATI: 28% (1.188) No tel.nr: 5% ( 206) CATI (n = 1.188 ; n = 922) CAWI-CATI CATI Response: 56% 79% Sampling frame error (wrong tel.nr.): 13% 5% Refusal: 5% 2% Non-contact 18% 7% Others 8% 7%

Response rate P1 and P2 2017 Income groups: P1 P2 Low High Low High CAWI 25% 39% CAWI 59% 69% CATI 31% 33% CATI 67% 65% Incentive in P1 (CAWI P2) Yes 60% 69% No 59% 70% Participating next year in P1: CAWI: 78% (78% - 77%) 2016:74% CATI: 87% (88% - 87%) 2016: 85% Participating next year in P2: CAWI: 94% CAWI/CATI: 92% CATI: 95%

Results: Missings Variables P1 2016-2017 Total 2016 2017 CAWI CATI One-week annual holiday 5,2 4,8 7,0 6.2 1,0 0,8 Keeping home warm 3,7 3,2 5,1 4,1 0,3 Replacing furniture 6,7 6,3 9,2 8,0 1,2 Replacing cloths 4,9 4,3 5,5 0,9 0,6 Meeting family 6,9 9,5 0,7 Warm meal every second day 3,3 2,9 4,7 3,8 0,2 0,4 Unexpected expenses 7,7 5,8 9,8 7,2 2,5 1,7 P2 2017 Total CAWI CATI One-week annual holiday 2,0 2,9 0,3 Keeping home warm 1,0 1,4 0,1 Replacing furniture 2,8 3,9 0,6 Replacing cloths 1,9 2,6 0,4 Meeting family 4,2 0,2 Warm meal every second day 0,8 1,1 Unexpected expenses 3,6

Weighting models Cross sectional 2016 P1 Longitudinal: P2, P3 and P4 Two years longitudinal sample Three years longitudinal sample

Weighting model: cross sectional The revised cross-sectional weighting procedure includes the following population totals: Personal level: Gender (2 classes) x Age (15 classes) NUTS2 (12 classes) x Age (2 classes) Ethnicity (3 classes) Household size (5 classes) NUTS1 (4 classes) x Low income category (3 classes) Degree of urbanisation (5 classes) x At-risk-of-poverty rate (2 classes) NUTS2 (12 classes) x At-risk-of-poverty rate (2 classes) NUTS2 (12 classes) x Activity status (5 classes) Household level NUTS2 (12 classes) x Household Income (deciles) NUTS1 (4 classes) x Tenure status (3 classes) Tenure status (3 classes)

Future plans and discussion points Using incentives: Design and field work P1: which incentives? Design P2 and P3: incentives Design for the future: CAWI and selected CATI/CAPI? Using more register variables: child care, education Reducing missing values in CAWI