The different roles of interviewers: How does interviewer personality affect respondents’ survey participation and response behavior? Michael Weinhardt (DIW Berlin) International Total Survey Error Workshop Tällberg 2009 The German Socio-Economic Panel Study is a service unit of the

2 Multiple Error Sources Measurement Error Interviewer Characteristics: Adherence to int. protocol appropriate probing accurate keying Respondent Characteristics Acquiescence Extreme respsonse style Item Characteristics: Social Desirability Privacy Unit Nonresponse No contact, no opportunity Lack of time and/or motivation Fear or maybe even shame to participate Different Interviewer characteristics interact with different error sources differently One characteristic may reduce one type of error but increase another → conflicting priorities when recruiting interviewers

3 Survey Error and Interviewer Personality Conscientiousness neat, orderly and meticulous Extraversion Sociability, enthusiasm, outgoing character Agreeableness Gentle, kind, seeking consultation, not confrontation Openness to Experience Open for intellectual stimulation; new and unconventional ideas Neuroticism Emotional instability, nervousness Conscien- tiousness Extraversion Openness Neuroticism Int’s performance Social Desira- bility Effects Privacy Effects Satisficing Agreeable- ness Unit Nonresponse

4 Data 1: German Socio-Economic Panel Study “German PSID”: Nationally representative longitudinal panel study of private households and individuals Annual since 1984: 12,000 households; 22,000 persons Two-step sampling design: 1.Register sampling of communities (polling districts) 2.Random route procedure Mixed Modes: –Personal Interviewing whenever possible (25% PAPI, 25% CAPI) –36% self-administered (Interviewer present) –14% postal

5 Data 2: GSOEP Interviewer Survey Mail survey of all interviewers of the 2006 SOEP wave Response rate: 94%; N=552 interviewers Update on interviewers’ demographics and other characteristics Systematic feedback on interviewers’ work experience: motivation, workload, opinion on incentives etc Self-rated measures of attitudes, values, beliefs etc. Same question format as in SOEP questionnaires regular for respondents Data linkage to individual level respondent data of the 2006 wave Measure of personality: Shortend Big Five Inventory (BFI) – Also in SOEP 2005 → for both respondents and interviewers available

6 Interviewers’ Big Five Mean St.d

7 Analysis: Model Specification Two level hierarchical regression models, face-to-face interview cases only Dependent variables: Measurement Error indicators Independent variables: Big Five variables Personality congruence between interviewer and respondent Interviewer level covariates: Key demographics, experience as an interviewer, workload Respondent level covariates: Key demographics (age, gender, region, education, whether self-employed), experience with SOEP

8 Looking at Measurement Error: Results and Challenges Findings: Interviewer conscientiousness reduces overall item nonresponse Personality congruence influential across a range of measures Interviewers’ personality dimensions are significantly related to (some) indicators of measurement error But: Difficult to hypothesize links between personality dimensions and measurement error – lack in theory Single dimensions of interviewers’ personality show no consistent links to measurement error across a range of measures Unexpected directions of effects

9 Significance: * = 10% level; ** = 5% level ; *** 1% level Interviewer performance and overall data quality Logistic RegressionNegative Binominal Regression Interviewer Level Covariates Overall Item NR (binary)Overall Item NR (Count) Odds RatioStd. Err.CoefficientStd. Err. Experience as Interviewer Gender0.767** ***0.042 Region Age1.015** ***0.002 Workload ***0.001 Openness ***0.001 Conscientiousness0.984** ***0.002 Extraversion Agreeableness ***0.003 Neuroticism Pers. Congruence0.990*** ***0.002 # of Items Rho Number of Obs 8975 Number of Int's 417

10 Significance: * = 10% level; ** = 5% level ; ***= 1% level Privacy effects: NR income items Logistic Regression Interviewer Level Covariates Gross Income NR (binary) Net Income NR (Count) Salary NR (Count) Odds RatioStd. Err.Odds RatioStd. Err.Odds RatioStd. Err. Experience as Interviewer Gender Region Age Workload *0.005 Openness Conscientiousness Extraversion *0.016 Agreeableness Neuroticism1.028* Pers. Congruence ** **0.009 Rho Number of Obs Number of Int's

11 Significance: * = 10% level; ** = 5% level ; ***= 1% level Social Desirability: Alcohol, cigarettes and weight Logistic RegressionOLS RegressionLogistic Regression Interviewer Level Covariates Regular alcohol consumption (binary) Cigarettes smoked per day (continuous) Round values weight variable (binary) Odds RatioStd. Err. Coefficient Std. Err.Odds RatioStd. Err. Experience as Interviewer Gender ** Region Age1.012** Workload Openness0.987** * ***0.004 Conscientiousness Extraversion Agreeableness * Neuroticism Pers. Congruence ***0.002 Rho Number of Obs Number of Int's

12 Significance: * = 10% level; ** = 5% level ; ***= 1% level Acquiescence and Extreme Response style OLS RegressionNegative Binomial Regression Interviewer Level Covariates Indicator AcquiescenceIndicator Extreme Response Coefficient Std. Err. Coefficient Std. Err. Experience as Interviewer Gender Region Age0.013** **0.002 Workload ***0.001 Openness0.015* *0.002 Conscientiousness ***0.003 Extraversion Agreeableness Neuroticism0.019*** ***0.002 Pers. Congruence-0.032*** ***0.001 Rho0.107 Number of Obs Number of Int's

13 Attrition and unit Non-response Preliminary analysis: same model set-up as before Linked 2006 data on interviewers and respondents to 2007 outcomes - binary variable: attriter or not No significant results: neither for big five nor for personality congruence However: overall item missingness is significantly related to attrition Further thoughts on attrition analysis: What kind of effect would we expect? Original distribution of Big 5 among interviewers not known – self selection of interviewers possible Attrition due to personality effects may have happened before 2006 already

14 Many thanks !

15 Similarity in Personality Literature: similarity in personality is linked to affection and friendship, e.g. personality congruence in married couples (Rammsttedt and Schupp 2009)  Personality congruence might also play an important role in the interviewer- respondent relationship Measurement: Euclidean distance between interviewers‘ and respondents‘ big five scores The sqareroot of the sum of squared differences in the big five factor scores Mean 50 ; Standard deviation: 10 Premultiplied by -1 for ease of interpretation