Messages for Gaining Respondent Cooperation in Multiple Languages Patricia Goerman, Mikelyn Meyers, and Yazmin Garcia Trejo, U.S. Census Bureau Mandy Sha RTI International Alisu Schoua Glusberg, RSS Presented at the International Conference on Questionnaire Design, Development, Evaluation, and Testing (QDET2) Miami, FL: November 9-13, 2016

Introductory Survey Messages for use across Languages Little literature about development of doorstep messages for use across languages Decade-long qualitative research program at the U.S. Census Bureau using different methods Goals of this paper: State of the art findings on tailoring introductory survey messages for use in 7 languages: English, Spanish, Chinese, Korean, Vietnamese, Russian, Arabic Discussion of qualitative methods used Lay out next steps for further research

Outline of Talk Review of the literature Field interview observation study Expert review study Focus group study Summary and conclusions Next steps

Literature Review: Interviewer Characteristics and Behavior Studies of interviewer characteristics and behavior (Couper 1997; Groves and Couper 1998; Groves et al 2004; Durrant and D’Arriago 2014) Interviewer race and gender Interviewers’ expertise and conversational skills Socioeconomic disparities Cross-cultural differences in survey administration

Literature Review: Introductory Messages Communication style Scripted v. unscripted introductory messages Announced v. unannounced visits Particular messages: eg. non-solicitation Paradata on doorstep interaction to explain non-response Little work on messages across languages/ cultural groups

Study 1: Field observation 2010 U.S. Census observation of 586 CAPI interviews Observation of: Interviewer messages and behaviors Linguistic and cultural factors affecting survey administration and response Debriefing of interviewers and respondents Language Interviews observed Arabic 61 Chinese 23 Korean Russian 67 Spanish 63 Vietnamese 75 English 278

Scripted doorstep introduction: 2010 S1. Hello, I’m (Name) from the U.S. Census Bureau (Show ID). Is this (Address)? Yes – Continue with question S2 No – Ask: Can you tell me where to find (Address)? END INTERVIEWER S2. I’m here to complete a Census questionnaire for this address. It should take about 10 minutes. (Hand respondent an information Sheet.) The first part explains that your answers are confidential. I’ll refer to the other parts later. Did you or anyone in this household live or stay here on April 1, 2010? Yes – Continue with question S3 No – Skip to question S4

Observation Findings Example strategies for gaining respondent cooperation in various groups Observation findings Language De-emphasize mandatory message Arabic & Spanish Emphasize mandatory message Russian Use of honorific or deferential terms Arabic, Korean & Vietnamese Appropriate attire Emphasize that survey is short Spanish Initiating contact in target language Various Discuss benefits to respondents’ communities Introduction from trusted community partners

Expert Review 2015 U.S. Census Bureau study Review of draft doorstep messages in Chinese, Korean, Vietnamese, Russian and Arabic Draft messages translated from English Teams of 3 or more reviewers with language expertise, formal education in target language and cultural knowledge and experience with target population Recommendations on how to best tailor the messages

Expert review recommendations Language Use interactive approach, Engage in conversation, Order of messages Chinese Order of messages, Seniority and politeness, Honorific markers Korean Abruptness, Use of appropriate pronouns based on age and sex of Resp. and Interviewer Vietnamese Interactive approach, Gender, Attire, Non-verbal behavior Arabic Politeness, Apologize in advance, Official ID needed, Emphasis on interviewer assisting respondent Russian

2 various national origins, 1 PR Focus groups Language Number of focus groups Arabic 2 Chinese English Korean Russian Vietnamese Spanish 2 various national origins, 1 PR Total 15 Videos of doorstep interactions Respondent aware of the Census going on in area Respondent did not know about the Census

Census messages tested in focus groups Interviewer introducing self Showing ID card Explain Census test Census takes place every 10 years Confidentiality Use of honorific markers, titles, pronouns

Focus Group Results across Languages Importance of verifying interviewer identity Some wanted more proof of identity, uniforms Purpose and importance of Census Desire for advance notification of visit Confidentiality Politeness Mandatory message (good or bad, variation)

Focus Group Results: Language Specific Focus groups findings Language Shared ethnicity interviewer and R, written materials, scheduling, emphasize no negative consequences Spanish Need for different order in messages: background before purpose of visit, politeness Chinese Forms of address, honorific markers, emphasize no negative consequences Korean

Focus Group Results: Language Specific (cont.) Focus groups findings Language Need planned intro statement, survey is short, how it will help Vietnamese community Vietnamese De-emphasize mandatory nature, offer to help, be careful about body language, looking in bag can be threatening Russian Gender matching of Interviewer/ respondent, be careful about causing embarrassment, regional dialect v. Modern Standard Arabic (MSA) Arabic

Summary and Conclusions Multi-method approach: Field observation, Expert review, Focus groups Three levels of feedback: Interviewers, language experts, respondents Messaging & increasing response: Help with tailoring, ordering and specific messages that might be most effective to increase response to the Census and other surveys among hard-to-reach populations

Next Steps New project Focus groups with new doorstep videos incorporating all required messages but tailored to different groups Tailoring of order and specific messages, culturally appropriate customs, phrases, attire Ultimate goal: to develop tailored training materials to support field interviewers

Messages for Gaining Respondent Cooperation in Multiple Languages Patricia Goerman, Mikelyn Meyers, and Yazmin Garcia Trejo, U.S. Census Bureau Mandy Sha RTI International Alisu Schoua Glusberg, RSS For more information: E-mail: Patricia.L.Goerman@census.gov Disclaimer: This presentation is intended to inform people about research and to encourage discussion. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.