HTW Berlin University of Applied Sciences General Online Research Conference (GOR 17) 15-17 March 2017 HTW Berlin University of Applied Sciences Berlin, Germany Maria Andreasson Elias Markstedt Johan Martinsson Laboratory of Opinion Research, University of Gothenburg Day of the week and time of the day for survey dispatch Contact: info@lore.gu.se Andreasson, M, Markstedt, E, Martinsson, J. 2017. “Day of the week and time of the day for survey dispatch.” General Online Research (GOR) Conference, Berlin. This work is licensed under a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/)

Day of the week and time of the day for survey dispatch Two large-scale randomized experiments Maria Andreasson Elias Markstedt Johan Martinsson Laboratory of Opinion Research (LORE) University of Gothenburg

Laboratory of Opinion Research (LORE) The Swedish Citizen Panel (SCP): Probability (9,000) and Nonprobability-based panels (ca 50,000) Web survey experiments Panel data Survey methodology

Study 1 Q: Are click rates affected by the day-of- week respondents receive an invitation email? Expectation: Week-end days are worse

Survey design Study 1: Day-of-week 11,300 pre-recruited members of the panel Non-probability sample Randomly assigned day-of-week for survey invitation (n=1,600/day) Survey invitation 15–21 October 2014 Reminder after one week

Study 2 Q: Are click rates affected by when during the day respondents receive an invitation email? Expectation: Mornings are better

Survey design Study 2: Time-of-day 47,000 pre-recruited members of the Citizen Panel Nonprobability sample Randomly assigned to receive a survey invitation on one of six time-of-day treatments n=7,900 per group Field period 27th November–3rd December 2014

Time-of-day Late evening 21.00 Early evening 18.00 Early morning: 07.00 Afternoon 14.20 Late morning 10.00 Lunch time 12.20

DV: Click rates Excluding email bounces Clicking on survey link in email invitation Click rate after: 24 hours 6 days (1 day before reminder) 8 days (1 day after reminder) 14 days

RESULTS STUDY 1 DAY-OF-WEEK

Click rates (%) Anova: 24 h p=.00, 6 days p=.43, 8 days p=.27, 14 days p=.16

ANOVAs by labor market status

ANOVAs by age

Click rate for individuals of different labor market status, by day-of-week Workers are less likely to respond quickly (within the first 24 hours) to invitations they receive on a Saturday than to invitations they receive on a Monday (diff= -10%, t= -4.61, p= .000), Tuesday (diff= -7%, t= -3.27, p= .019), Wednesday (diff= -9%, t= -4.18, p= .001), Thursday (diff= -8%, t= -3.69, p= .004), and, at a 90% confidence level, on Friday (diff= -6%, t= -2.71 , p= .095). Workers are also less likely to respond quickly when they receive the invitation on a Sunday than on a Monday (diff= -6%, t= -3.07, p= .035). Pensioners are less likely to respond quickly (within the first 24 hours) to invitations they receive on a Saturday than to invitations they receive on a Monday (90% confidence level) (diff= -12%, t= -2.90, p= .057), Wednesday (diff= -15%, t= -3.66, p= .005), and a Thursday (diff= -13%, t= -3.21, p= .023). There are no significant differences in participation rate for unemployed/students.

Summary Study 1 Invitation day has significant effects on click rates after 24 hours Saturday worst Early days of the week best All differences disappear within 6 days When comparing different subgroups, effects are only seen for gainfully employed and pensioners, and for individuals between the age of 30-69

RESULTS STUDY 2 TIME-OF-DAY

Click rate by time-of-day Table 4 shows the net participation rates split by the time respondents received the invitation email. The first column shows the participation rate after 24 hours and the second shows the participation rate after 6 days. Only 1 of 15 group comparisons differ significantly from zero (99% confidence level). The 21:00 group had a 2 percentage points lower participation rate than the 12:20 group. No differences are found after 6 days.

Click rate by labor market status and time-of-day An analysis of participation rate in different labor market status groups shows that pensioners are less “flexible” than other groups (see Table 5). Pairwise comparison tests reveal that, on 90% confidence levels, these respondents are less likely to respond quickly, that is within the first 24 hours, to emails they receive at 18:00 than to emails they receive at 12:20 (diff= -4%, t= -2.67, p= .081), and to emails they receive at 21:00 than to emails they receive at 12:20 (diff= -5%, t= -2.83, p= .054). There are no significant differences between treatment groups for workers or for unemployed/students.

Summary study 2 Click rate can vary within first 24 hours by time-of-day for invitation e-mail. Looking at subgroups, more pronounced effects are seen for pensioners Differences disappear after 6 days

Further studies Preference for receiving inviations Matching and mismatching preference with day and hour Small gains in the short run Complicated

Conclusions Minor differences in click rates when time-of-day and day-of-week for invitation email is varied Effects can only be seen 24 hours after survey invitation, not later For pensioners and gainfully employed, click rates can be improved somewhat within the first 24 hours by sending invitations during weekday evenings All differences disappear in the long run Implications: there are small gains in varying the timing of invitation emails to surveys overall

Thank you! You can read more at lore.gu.se