Evgeny Terentev Natalia Maloshonok Aigul Mavletova Increasing response rates by changing Invitations in web surveys

Previous studies Usage of personal salutation increases response rate [Cook C., Heath F., Thompson R. L. 2000; Heerwegh D., Loosveldt G. 2006; Joinson A.N., Woodly A., Reips U-D. 2007; Muñoz-Leiva F., et al. 2010]. Offering personalized feedback compensates for the negative effects of low topic salience on response rates [Marcus B. et al. 2007]. Usage of authoritative or high status sender increases response rate [Guéguen N., Jacob C. 2002; Joinson A. N., Reips U. D. 2003]. Subject line in a form of request for help increases response rate [Henderson V. 2011; Trouteaud A. R. 2004]. Changing in the text of reminders increases response rate [Sauermann H., Roach M. 2013].

Empirical base Applicants to HSE undergraduate programs: N=6027, RR=37%, BR= 6%. Applicants to HSE master programs: N=1873, RR=45%, BR=7%. Graduates of HSE bachelor programs: N=2437, RR=11%, BR=6%. HSE Students (bachelors and masters): N=13350, RR=11%, BR=4%. Students of pedagogical university: N=186, RR=33%, BR=10%. Students of technical university: N=223, RR=31%, BR=5%.

Experimental factors FactorsFactor levelsSurveys 1) Personal salutation  First name  First name and middle name  Dear applicant/student/graduate! Applicants to HSE undergraduate programs, HSE students and graduates, students of technical university 2) Information about importance of participation for particular groups  This year the answers of 3 rd and 4 th year students are especially important  - HSE 3 rd and 4 th years students 3) Selectivity  You are one of a small number of graduates who have been selected to answer specific questions about the knowledge acquired in the HSE. Your participation in the survey will be especially important for us!  - Applicants to HSE undergraduate programs, HSE graduates

Experimental factors FactorsFactor levelsSurveys 4) Feedback  We are pleased to announce that we will share with you the main results of this survey. We will send the small report to your .  - Applicants to HSE undergraduate and master programs, HSE students 5) Sender  Dean of university department  Institutional research office Students of Russian pedagogical university, HSE graduates 6) Subject line  Survey of students of leading universities in the field of economics and management  Express your opinion, help to make university better! Students of technical university 7) Changing in reminders  Different reminders  Same reminders Applicants to HSE undergraduate and master programs

Results 1. Sender HSE graduates Students of pedagogical university Dean of university department 18.3%*** (671) 44.6%*** (92) Institutional research office 9.1%*** (1766) 22.3%*** (94) Chi square(graduates)=40.098, df=1, p<0.001 Chi square(students of pedagogical university)=10.707, df=1, p<0.001

Results 2. Personal salutation Applicants to undergraduate programs HSE graduates HSE 1 st, 2 nd, 5 th years students and master programs students Students of technical university First name 38.4% (747) % (990) 29.6%* (115) First name and middle name 39.7% (765) 12.0% (1215) %* (44) Dear applicant/student/ graduate 36.4% (4514) 11.3% (1222) 11.8% (8921) 23.5%* (68) Chi square=7.595, df=2, p<0,05

Results 3. Selectivity Applicants to undergraduate programs HSE graduates Yes 41.5%** (911) 13.4% (748) No 36.2%** (5115) 10.9% (1689) Chi square=9.127, df=1, p<0.01

Results 4. Interaction of factors Survey: HSE 3 rd and 4 th years students Feedback YesNo Information about importance of participation for particular groups Yes11,2%7% No7,2%7,9% OR=1,592,p<0,01 Survey: Students of technical university Personal Salutation First name and middle name First nameDear student Subject line Dean of university department 72,7%30,2%21,2% Institutional research office 22,7%29,0%25,7% OR=7,273,p<0,001

Results 5 Applicants to HSE undergradua te programs Applicants to HSE master programs Graduates of HSE bachelor programs HSE 1 st, 2 nd, 5 th years students and master programs students HSE 3 rd and 4 th years students Students of technical university Students of pedagogical university Personal salutation Information about importance of participation for particular groups - Selectivity + - Feedback Sender + + Subject line - Changing in reminders --

Conclusions and discussion Not all factors have the same effect on all categories of respondents. If main effects were not always significant, the interaction effects can significantly increase response rates.

References Cook C., Heath F., Thompson R. L. A Meta-Analysis of Response Rates in Web-or Internet-Based Surveys // Educational and Psychological Measurement Vol. 60. №. 6. P Heerwegh D., Loosveldt G. An Experimental Study on the Effects of Personalization, Survey Length Statements, Progress Indicators, and Survey Sponsor Logos in Web Surveys // Journal of Official Statistics Vol. 22. №. 2. P. 191–210. Joinson A.N., Woodly A., Reips U-D. Personalization, Authentication and Self-Disclosure in Self-Administered Internet Surveys // Computers in Human Behavior Vol. 23. №. 1. P Muñoz-Leiva F., Sánchez-Fernández J., Montoro-Ríos F., Ibáñez-Zapata J. Á. Improving the Response Rate and Quality in Web-Based Surveys through the Personalization and Frequency of Reminder Mailings // Quality & Quantity Vol. 44. №. 5. P Marcus B., Bosnjak M., Lindner S., Pilischenko S., Schütz A. Compensating for Low Topic Interest and Long Surveys. A Field Experiment on Nonresponse in Web Surveys // Social Science Computer Review Vol. 25. №. 3. P Guéguen N., Jacob C. Solicitation by and Solicitor's Status: A Field Study of Social Influence on the Web // CyberPsychology & Behavior Vol. 5. №. 4. P Joinson A. N., Reips U. D. Personalized Salutation, Power of Sender and Response Rates to Web-Based Surveys // Computers in Human Behavior Vol. 23. №. 3. P Sauermann H., Roach M. Increasing Web Survey Response Rates in Innovation Research: An Experimental Study of Static and Dynamic Contact Design Features // Research Policy Vol. 42. №. 1. P

Thank you for attention!