1. The different roles of interviewers: How does interviewer personality affect respondents’ survey participation and response behavior? Michael Weinhardt (DIW Berlin) International Total Survey Error Workshop Tällberg 2009 The German Socio-Economic Panel Study is a service unit of the

2. 2 Multiple Error Sources Measurement Error Interviewer Characteristics: Adherence to int. protocol appropriate probing accurate keying Respondent Characteristics Acquiescence Extreme respsonse style Item Characteristics: Social Desirability Privacy Unit Nonresponse No contact, no opportunity Lack of time and/or motivation Fear or maybe even shame to participate Different Interviewer characteristics interact with different error sources differently One characteristic may reduce one type of error but increase another → conflicting priorities when recruiting interviewers

3. 3 Survey Error and Interviewer Personality Conscientiousness neat, orderly and meticulous Extraversion Sociability, enthusiasm, outgoing character Agreeableness Gentle, kind, seeking consultation, not confrontation Openness to Experience Open for intellectual stimulation; new and unconventional ideas Neuroticism Emotional instability, nervousness Conscien- tiousness Extraversion Openness Neuroticism Int’s performance Social Desira- bility Effects Privacy Effects Satisficing + + - - - - Agreeable- ness Unit Nonresponse

4. 4 Data 1: German Socio-Economic Panel Study “German PSID”: Nationally representative longitudinal panel study of private households and individuals Annual since 1984: 12,000 households; 22,000 persons Two-step sampling design: 1.Register sampling of communities (polling districts) 2.Random route procedure Mixed Modes: –Personal Interviewing whenever possible (25% PAPI, 25% CAPI) –36% self-administered (Interviewer present) –14% postal

5. 5 Data 2: GSOEP Interviewer Survey Mail survey of all interviewers of the 2006 SOEP wave Response rate: 94%; N=552 interviewers Update on interviewers’ demographics and other characteristics Systematic feedback on interviewers’ work experience: motivation, workload, opinion on incentives etc Self-rated measures of attitudes, values, beliefs etc. Same question format as in SOEP questionnaires regular for respondents Data linkage to individual level respondent data of the 2006 wave Measure of personality: Shortend Big Five Inventory (BFI) – Also in SOEP 2005 → for both respondents and interviewers available

6. 6 Interviewers’ Big Five Mean St.d. 52.71 7.90 50.89 7.72 53.31 8.21 54.16 7.46 44.57 9.16

7. 7 Analysis: Model Specification Two level hierarchical regression models, face-to-face interview cases only Dependent variables: Measurement Error indicators Independent variables: Big Five variables Personality congruence between interviewer and respondent Interviewer level covariates: Key demographics, experience as an interviewer, workload Respondent level covariates: Key demographics (age, gender, region, education, whether self-employed), experience with SOEP

8. 8 Looking at Measurement Error: Results and Challenges Findings: Interviewer conscientiousness reduces overall item nonresponse Personality congruence influential across a range of measures Interviewers’ personality dimensions are significantly related to (some) indicators of measurement error But: Difficult to hypothesize links between personality dimensions and measurement error – lack in theory Single dimensions of interviewers’ personality show no consistent links to measurement error across a range of measures Unexpected directions of effects

9. 9 Significance: * = 10% level; ** = 5% level ; *** 1% level Interviewer performance and overall data quality Logistic RegressionNegative Binominal Regression Interviewer Level Covariates Overall Item NR (binary)Overall Item NR (Count) Odds RatioStd. Err.CoefficientStd. Err. Experience as Interviewer 1.0020.0100.0000.003 Gender0.767**0.103-0.249***0.042 Region0.7910.257-0.1490.119 Age1.015**0.0070.013***0.002 Workload0.9970.002-0.002***0.001 Openness1.0060.008-0.001***0.001 Conscientiousness0.984**0.0080.010***0.002 Extraversion1.0120.008-0.0150.003 Agreeableness0.9970.0090.002***0.003 Neuroticism1.0030.007-0.0070.003 Pers. Congruence0.990***0.003-0.001***0.002 # of Items 0.000 Rho0.2380.021 Number of Obs 8975 Number of Int's 417

10. 10 Significance: * = 10% level; ** = 5% level ; ***= 1% level Privacy effects: NR income items Logistic Regression Interviewer Level Covariates Gross Income NR (binary) Net Income NR (Count) Salary NR (Count) Odds RatioStd. Err.Odds RatioStd. Err.Odds RatioStd. Err. Experience as Interviewer 1.0050.0230.9880.0201.0040.020 Gender0.9660.2800.8710.2361.1720.302 Region2.8231.8030.9410.5761.4540.909 Age0.9810.0140.9780.0140.9850.013 Workload0.9950.0050.9970.0040.991*0.005 Openness1.0130.0180.9860.0160.9990.016 Conscientiousness0.9840.0180.9850.0170.9870.017 Extraversion1.0090.0171.0230.0171.030*0.016 Agreeableness1.0150.0201.0190.0191.0140.018 Neuroticism1.028*0.0151.0190.0141.0090.013 Pers. Congruence0.9920.0070.980**0.0080.982**0.009 Rho0.5070.1460.3880.1420.3330.049 Number of Obs 4540 4604 4224 Number of Int's 369 372 366

11. 11 Significance: * = 10% level; ** = 5% level ; ***= 1% level Social Desirability: Alcohol, cigarettes and weight Logistic RegressionOLS RegressionLogistic Regression Interviewer Level Covariates Regular alcohol consumption (binary) Cigarettes smoked per day (continuous) Round values weight variable (binary) Odds RatioStd. Err. Coefficient Std. Err.Odds RatioStd. Err. Experience as Interviewer 1.0010.0090.017 1.0010.005 Gender0.8970.099-1.032**-1.0321.0660.064 Region1.1130.340-0.711 0.8950.158 Age1.012**0.006-0.011 0.9980.003 Workload1.0010.002-0.005 0.9990.001 Openness0.987**0.0060.047*0.0471.011***0.004 Conscientiousness0.9890.0070.028 1.0060.004 Extraversion0.9930.0060.024 1.0050.004 Agreeableness0.9920.007-0.049*-0.0491.0030.004 Neuroticism1.0010.0060.009 0.9960.003 Pers. Congruence0.9980.003-0.030 0.991***0.002 Rho0.1090.0160.013 0.0180.005 Number of Obs 8927 2463 8907 Number of Int's 416 342 417

12. 12 Significance: * = 10% level; ** = 5% level ; ***= 1% level Acquiescence and Extreme Response style OLS RegressionNegative Binomial Regression Interviewer Level Covariates Indicator AcquiescenceIndicator Extreme Response Coefficient Std. Err. Coefficient Std. Err. Experience as Interviewer-0.0100.0100.0010.003 Gender-0.1660.1360.0380.041 Region0.4370.3510.0260.097 Age0.013**0.007-0.004**0.002 Workload0.0030.002-0.002***0.001 Openness0.015*0.008-0.004*0.002 Conscientiousness-0.0050.0090.010***0.003 Extraversion0.0030.0080.0000.002 Agreeableness0.0060.0090.0000.003 Neuroticism0.019***0.007-0.011***0.002 Pers. Congruence-0.032***0.0030.004***0.001 Rho0.107 Number of Obs 8927 8975 Number of Int's 416 417

13. 13 Attrition and unit Non-response Preliminary analysis: same model set-up as before Linked 2006 data on interviewers and respondents to 2007 outcomes - binary variable: attriter or not No significant results: neither for big five nor for personality congruence However: overall item missingness is significantly related to attrition Further thoughts on attrition analysis: What kind of effect would we expect? Original distribution of Big 5 among interviewers not known – self selection of interviewers possible Attrition due to personality effects may have happened before 2006 already

14. 14 Many thanks ! mweinhardt@diw.de

15. 15 Similarity in Personality Literature: similarity in personality is linked to affection and friendship, e.g. personality congruence in married couples (Rammsttedt and Schupp 2009)  Personality congruence might also play an important role in the interviewer- respondent relationship Measurement: Euclidean distance between interviewers‘ and respondents‘ big five scores The sqareroot of the sum of squared differences in the big five factor scores Mean 50 ; Standard deviation: 10 Premultiplied by -1 for ease of interpretation