Nonresponse and Measurement Error in Employment Research Gerrit Müller (IAB) joint with: Frauke Kreuter (JPSM – U Maryland), Mark Trappmann (IAB) PASS

Research Questions Do survey respondents recruited with extra effort, provide answers of lower quality? Are cooperators more motivated to provide accurate data? Or, are late respondents hampered by recall deficits? How does extra effort affect total bias? PASS

Survey Data Panel Study “Labor Market and Social Security” (PASS) Dual frame survey (benefit recipients / residential population) Wave1: 12,000 HH 20,000 P RR1: 30.5% (within HH: 85%) Mixed mode survey (sequential CATI -> CAPI) PASS

Record Linkage I Individual survey data linked with individual administrative data (80% of all Rs agreed; 72% successfully matched) Administrative records on: employment, earnings, unemployment, labor market programs , Contact data on HH-level only

Record Linkage II Administrative data linked with paradata for the gross sample of recipients (from unemployment register) Contact data on HH-level only Indicator for Respondents / Nonrespondents on HH-level only ,

Hypotheses about Measurement Error (response process model, Tourangeau ’84) Unemployment benefit (UBII) July 2006 Nov 2006 at time of interview Income in month prior to interview Occupation Educational degree Relationship between ME and response propensity (number of contact attempts) PASS

Contact Quintiles and Follow-Up Efforts # of contacts: _min _max Q1 (high contactability) 1 2 Q2 3 4 Q3 5 7 Q4 8 14 Q5 (low contactability) ≥15 Transfer CATI to CAPI CATI NR follow-up of “soft refusals”

Measurement Error (in percent) by Contact Quintiles and Follow-up Efforts UB II July 2006 UB II Interview Date Q1 (high contactability) 12 11 Q2 13 Q3 14 Q4 16 17 Q5 20 15 To CAPI 18 CATI NR follow-up 21 PASS

Measurement Error by Contact Quintiles and Follow-up Efforts Income (abs. dev.) Years of Edu. (% mismatch) Q1 (high contactability) 311 25 Q2 343 28 Q3 370 27 Q4 335 26 Q5 380 24 To CAPI 451 CATI NR follow-up 407 PASS

5. NR-ME Bias Decomposition for Recipient sample only HH-level variables only (!) UBII in Jul06 not feasible for bias decomposition UBII in Nov UBII at date of interview PASS

“Pick your brains” ME-Model for UBII in July06 (handout) Puzzle: high ME for the young? HH-interview by target head? Administrative data not always „Gold Standard“ (error-free) Assumption: ME in register data unrelated to ME in survey reports and response propensity

“Pick your brains” Decomposition findings statistic-specific Extend analyses to P-level variables (e.g. employment, income) Problem: unknown on individual level How to go ahead?