Factors associated with frequency of responding to electronic surveys among students attending a large minority-serving university: The Student Behavioral Health Survey (SBHS-Web) Meredith L. Wilcox, MPH Department of Epidemiology William W. Darrow, PhD Department of Health Promotion Florida International University, Miami, Florida, US

Presenter Disclosures Meredith L Presenter Disclosures Meredith L. Wilcox (1) The following personal financial relationships with commercial interests relevant to this presentation existed during the past 12 months: No relationships to disclose

BACKGROUND Electronic surveys have numerous advantages over traditional survey modes: Fewer physical and financial resources Reach a larger number of potential participants Favorable for collecting data on sensitive topics Overall, response to electronic surveys are estimated to be 11-20% lower than traditional survey modes. Response higher among student-, employee-, military-populations Response higher among non-Hispanic White and Asian students compared with their Hispanic and African American counterparts. Participation in electronic surveys are influenced by society-, respondent-, and design-related factors. AIM Identify factors associated with history of responding to electronic surveys among student respondents of the Student Behavioral Health Survey (SBHS- Web) conducted in Spring 2013 - Primarily favorable for collecting data on sensitive topics owing to the confidentiality and anonymity offered to participants. Favorable for surveying college students, a population that is readily available for sampling and that tends to have greater knowledge and access to technology than the general public Society-related: population-type, with greater response among employee, student, and military populations Respondent-related: age, gender, income, health status, computer resources, computer literacy Design-related: Survey content, Length of survey and email invitations, personalization of the emails, use of pre-notification, number and frequency of emails This finding unfortunately is not consistent for college students of all demographics; African American and Hispanic students are notably less likely to complete electronic surveys than their White or Asian American counterparts.(Sutherland 2013; Fan 2010; Krebs 2010; Cranford, 2008; Krebs 2007) This disparity may be attributed to decreased availability of reliable technology at the universities serving minority students, or to historically low participation by minorities in health research.(Fan 2010; UyBico 2007)

Method of data collection SBHS-Web AIMS Measure sensitive behaviors (sexual activity, use of drugs and alcohol) of the student population Increase student response using 2-factor experimental design MOTIVATION The National College Health Assessment II (NCHA-II), a similar survey on sensitive topics, has been administered at the same university for the past few years as an online survey. Year Method of data collection Response Rate 2004 Paper-based questionnaire (NCHA) 40% of selected classes participated, 80% of students participated 2010 Online survey 5.7% 2011 7.0% 2012 5.0% 2013 3.5%

SBHS-Web METHODOLOGY Large, minority-serving, 4-year public university - Student population ≈54,000 - 57% female; 61% Hispanic 10-day period in March-April 2013 8,000 student email addresses randomly selected Randomized into 4 groups of equal size Invited to participate through email Pre-notification email (select students), invitation email, 3 reminder emails at 48-hour intervals No incentives were used

Pre-notification email SBHS-Web TREATMENT GROUPS Pre-notification email No Yes Improved email design Standard (S) pre-notification (SP) Innovative (I) (IP) Pre-notification email: Sent 48 hours prior to the survey invitation email Innovative group: Increased personalization of email notifications Sender name Subject lines of the email Logo of the sender Email header Closing of the email

SBHS-Web IMPROVEMENTS IN EMAIL DESIGN Design element Standard Innovative Sender name Survey Research Nine (SR-9) Individual names of the SR-9 team members Email subject line “Introducing the SBHS-Web” “Complete the SBHS-Web” “Why don’t students respond?” “What do you think?” “Spam it’s not” “Time is running out” “Your last chance” Email headers “Adult online consent to participate in a research study” “This is your opportunity!” “We really want to know!” “We still really want to know!” Logo University/college logo Logo of SR-9 research team Closing of emails “Thank you for taking time to complete this survey” Name, signature, website of principle investigator; Individual names of SR-9

SBHS-Web Invitation email

SBHS-Web Response rate of the SBHS-Web by treatment group– 2013 SBHS-Web, March-April 2013, Miami, Florida *Response rate significantly lower than that of the IP group at p<0.001 Treatment group Response n (%) Standard (S) 129 (6.5)* Standard pre-notification (SP) 125 (6.3)* Innovative (I) 98 (4.9)* Innovative pre-notification (IP) 280 (14.0) TOTAL 632 (7.9)

Frequency of responding to previous electronic surveys SBHS-Web Frequency of responding to previous electronic surveys sent by the university email system, by treatment group– 2013 SBHS-Web, March-April 2013, Miami, Florida Chi-squared, p<0.001 Treatment group Frequency of responding to previous electronic surveys Never/ rarely (row %) Sometimes Usually/always Standard (S) 25% 31% 44% Standard pre-notification (SP) 30% 46% 23% Innovative (I) 40% Innovative pre-notification (IP) 38% 39% TOTAL 33% 28%

AIM Identify factors associated with history of responding to electronic surveys among student respondents of the Student Behavioral Health Survey (SBHS-Web) conducted in Spring 2013

Methodology STUDY OUTCOME Frequency of responding to previous electronic surveys sent through university email system Never/rarely Sometimes Usually/always STATISTICAL ANALYSIS Association between outcome and factors: Chi-squared tests, logistic regression models Two-tailed significance level of α≤0.05 SPSS version 19 (SPSS Inc., Chicago, IL)

RESULTS Demographics of student respondents (n=632)– 2013 SBHS-Web, March-April 2013, Miami, Florida Characteristic % Age (years), Median (range) 23 (18-60) Gender, % female 60.1 Hispanic ethnicity 58.9 Marital status, Single 76.9 Employment, Employed full-time 22.3 Foreign-borne 42.6

RESULTS School-related characteristics of student respondents (n=632)– 2013 SBHS-Web, March-April 2013, Miami, Florida Characteristic % Class level Undergraduate, 1st-2nd year 25.5 Undergraduate, 3rd year or higher 51.1 Graduate/Professional/other 23.4 Place of residence On campus 7.8 Off campus alone or w/parents/family 61.6 Off campus w/partner or roommate, or other 30.7 Majority of classes taken at main campus 82.0 How often do you respond to electronic surveys sent through the university email system? Never/rarely 32.6 Sometimes 39.1 Usually/always 28.3

RESULTS Independent variables selected a prior NOT associated with outcome (chi-squared, p>0.20) Eliminated due to collinearity Age in years X Hispanic ethnicity Gender Marital status Relationship status Employment Place of residence Foreign-borne Class level Time of day of classes Campus where majority of classes are taken Member of fraternity/sorority/social club Member of pre-professional/service/honor society Frequency of opening survey invitations sent by university email system Treatment group in SBHS-Web S

*Compared with usually/always responding RESULTS Odds of students never/rarely* responding to electronic surveys sent through the university email system– 2013 SBHS-Web, March-April 2013, Miami, Florida *Response rate significantly lower than that of the IP group at p<0.001 Unadjusted Adjusted OR (95% CI) p Hispanic ethnicity 1.58 (1.05-2.37) 0.029 1.29 (0.83-2.01) 0.265 Male gender 1.81 (1.19-2.75) 0.006 2.03 (1.31-3.16) 0.002 Class level Undergraduate, 1st-2nd year 1.49 (0.84-2.63) 0.172 1.51 (0.79-2.91) 0.215 Undergraduate, 3rd year+ 2.40 (1.45-4.00) 0.001 2.42 (1.38-4.23) Graduate/Professional/other Ref. --- Treatment group S SP 2.33 (1.22-4.47) 0.100 2.28 (1.17-4.45) 0.015 I 1.84 (0.94-3.60) 0.073 1.87 (0.95-3.71) 0.072 IP 2.95 (1.73-5.03) <0.001 3.26 (1.88-5.66) *Compared with usually/always responding Adjusted for “place of residence” and “campus where majority of classes are taken” OR=odds ratio; CI=confidence interval; S=standard; SP=standard pre-notification; I=innovative; IP=innovative pre-notification

*Compared with usually/always responding RESULTS Odds of students sometimes* responding to electronic surveys sent through the university email system– 2013 SBHS-Web, March-April 2013, Miami, Florida *Response rate significantly lower than that of the IP group at p<0.001 Unadjusted Adjusted OR (95% CI) p Hispanic ethnicity 1.26 (0.85-1.85) 0.248 1.00 (0.66-1.52) 0.999 Male gender 1.57 (1.05-2.36) 0.029 1.61 (1.06-2.46) 0.027 Class level Undergraduate, 1st-2nd year 1.14 (0.67-1.93) 0.636 1.31 (0.72-2.41) 0.379 Undergraduate, 3rd year+ 1.78 (1.11-2.84) 0.016 1.78 (1.06-2.99) Graduate/Professional/other Ref. --- Treatment group S SP 2.85 (1.56-5.20) 0.001 2.96 (1.60-5.49) I 1.92 (1.02-3.59) 0.042 1.95 (1.03-3.70) 0.040 IP 2.45 (1.47-4.07) 2.61 (1.55-4.39) <0.001 *Compared with usually/always responding Adjusted for “place of residence” and “campus where majority of classes are taken” OR=odds ratio; CI=confidence interval; S=standard; SP=standard pre-notification; I=innovative; IP=innovative pre-notification

Strengths & Limitations SBHS-Web was anonymous and confidential Students were randomized into treatment groups Large sample overall and within each treatment group Many demographic and school-related variables available LIMITATIONS All data self-reported Data not available for non-responders Difficulties in implementing survey may have affected response within certain treatment groups Timing of the survey may have affected response Some factors associated with frequency of responding were not available

Summary Low response to previous electronic surveys sent by the university was more prevalent within the treatment groups that received the pre- notifications and/or improved email design. After adjusting for other covariates, the odds of infrequently responding to previous electronic surveys sent by the university were significantly higher among students in the SP, I, and IP groups compared with those in the S group. This suggests that improved design of email notifications and use of pre-notification may be promising methods of increasing response among students who infrequently respond to electronic surveys. Public health implications: Reduce bias in student response to electronic surveys collected using electronic surveys.

Thank You