1 Kenneth E. Wallen a,b,c, Adam C. Landon a,b, Gerard T. Kyle a,b,c, Michael A. Schuett a,c, Jeremy Leitz d, & Ken Kurzawski d a Department of Recreation, Park and Tourism Sciences, Texas A&M University b Human Dimensions of Natural Resources Lab, Texas A&M University c Applied Biodiversity Science NSF-IGERT Program, Texas A&M University d Texas Parks and Wildlife Department Sampling Efficacy & Bias in Mode of Response for Survey-Based Research

2 2 Background ‘’89 ‘90 ‘93 ‘97 ‘04 ‘08 ‘01 ‘12

3 3 Background

4 4

5 5

6 6

7 7

8 8

9 9

10 Background

11 Background

12 Response Rates Why do we care about response rates? Need (desire) to reduce survey error… the difference between the estimate using the collected data and the true value of the variables in the population Non-response error… emerges when those who do not respond differ from those who respond Mail questionnaires are costly… printing, postage & labor

13 Trends in Response Rates All modes of collection area are on the decline Mixed-mode designs (mail/web) have demonstrated success for reducing non-response error Respondents to mail and web-based collection modes are different demographically and with their use of technology Mixed-mode benefit is twofold… opportunity in preferred manner & remind respondent of the opportunity to respond

14 Texas Recreational Anglers Heterogeneous Demographics Motivations Preferred resources Target species Avidity Expenditures Imperative to reduce non- response error to obtain data that is reflective of the angling population to make informed decisions that will impact this population

15 Study Purpose Compare three modes of questionnaire administration in terms of: 1.Response rate 2.Respondent characteristics

16 Survey Methodology Data Collection Sample drawn randomly from TPWD database of licensed fresh/saltwater anglers Three modes of contact commencing the last week of September, 2012 Mixed-mode (n=4,000) -Only (n=4,000) Combination (n=1,000) Web-based questionnaire was designed/administered thru Qualtrix with the URL:

17 Mixed-Mode 1.Initial contact: Invitation letter with web push 2.One week follow-up: Thank you/reminder postcard with web push 3.Two week follow-up: Second contact letter to non- respondents with web push 4.Three week follow-up: Survey packet to non-respondents… cover letter (with web push), questionnaire, and postage- paid reply envelope

18 -Only Four invitations, one week apart Similar to the cover letter sent to the Mixed-Mode group, the invitations outlined the study purpose and invited respondents to complete the questionnaire online Both a URL to be entered into respondents’ browser and hyperlinked icon (“Take Survey”) were provided for respondents to access the questionnaire

19 Combination A combination of contacts replicating the “Mixed Mode” and “ Only” methods Eight invitations (4 hardcopy, 4 invitations) were sent to respondents, one week apart The sending of and postal invitations were synchronized to arrive simultaneously; i.e., invitations were sent approximately two days following the mailing of hard copies

20 Survey Response Effective Response Rates -only 29.9% (784/2685) Mixed-mode 20.0% (697/3486) Combination 63.4% (407/640)

21 Survey Response 53.8% of Mixed-mode respondents completed hard copies of the questionnaire 29.5% of Combination respondents completed hard copies

22 Survey Analysis Multinomial Logistic Regression Model (logit) Dependent variable: survey response mode Null-model Approach Independent Variables Socio-demographics, motivations, & avidity Reference group: Mixed-mode Ø = Mixed-mode 1 = Combination 2 = only Age + gender + income + race + ethnicity + motivation + avidity

23 Table 1. Multinomial logistic regression results and item descriptives Mixed-modeCombination -only Mean% Coeff.s.e%StdXMean%Coeff.s.e.%StdX Socio-demographics Age Income ** ** Female12.99%4.60%-1.012** %-1.015** Hispanic6.86%5.44% % Black1.72%.42% %-1.676* N=1148, Likelihood-ratio X 2 = , p > X 2 =.000, Pseudo R 2 =.06, **p<.005, *p<.05 Note: Reference group is Mixed-mode Socio-demographics

24 Table 1. Chi-square comparison of observed and expected income by survey mode Under $20,000 $20,000- $59,999 $60,000- $99,999 $100,000- above Mixed-mode Observed Expected Percentage only Observed Expected Percentage Combination Observed Expected Percentage Notes. χ 2 = 49.90, p < 0.001, Cramér's V = 0.15 Income

25 Table 2. Chi-square comparison of observed and expected income by survey mode Under $20,000 $20,000- $59,999 $60,000- $99,999 $100,000- above Mixed-mode Observed Expected Percentage only Observed Expected Percentage Combination Observed Expected Percentage Notes. χ 2 = 49.90, p < 0.001, Cramér's V = 0.15 Income

26 Table 3. Chi-square comparison of male and female respondents by survey mode FemaleMale Mixed-mode Observed11228 Expected Percentage only Observed24513 Expected Percentage Combination Observed53355 Expected Percentage Notes. χ 2 = 27.95, p < 0.001, Cramér's V = 0.15 Gender

27 Within Groups – Socio-demographics Table 4. T-test results between respondents who completed survey by paper or online within mixed-mode and combination groups Mixed-modeCombination PaperOnlinep-valuePaperOnlinep-value Socio-demographics Age

28 Table 1 (continued). Multinomial logistic regression results and item descriptives Mixed-modeCombination -only Mean% Coeff.s.e%StdXMean%Coeff.s.e.%StdX Motivations/Avidity To be outdoors * For family recreation * To experience new things * To be with friends * ** (Not) most important activity35.29%28.87% %-.512** N=1148, Likelihood-ratio X 2 = , p > X 2 =.001, Pseudo R 2 =.06, **p<.001, *p<.05 Note: Reference group is Mixed-mode Motivation & Avidity

29 Table 5. Results of ANOVA for Motives EffectMeanSDSSF-valuedf To be outdoors 1 Mixed-Mode Only4.56 a 0.64 Combination4.40 a 0.72 Between-groups ***2 To be with friends 1 Mixed-Mode Only3.95 b 1.01 Combination3.75 b 1.08 Between-groups **2 Notes. 1 Mean score value is on a scale ranging from 1 (not important) to 5 (extremely important). Like superscripts indicate significant differences at p ≤ 0.01 (Scheffe post-hoc analysis). ** p-value < 0.01 ***p-value < Motives

30 Table 6. Chi-square comparison of observed and expected importance/non-importance of angling as primary recreational activity by survey mode Most importantNot most important Mixed-mode Observed69170 Expected Percentage only Observed Expected Percentage Combination Observed Expected Percentage Notes. χ 2 = 9.19, p < 0.01, Cramér's V = 0.09 Avidity

31 Within Groups - Motivation Table 7. T-test results between respondents who completed survey by paper or online within mixed- mode and combination groups Mixed-modeCombination PaperOnlinep-valuePaperOnlinep-value Motivations To be outdoors To experience new things To get away from demands For the experience of the catch To experience natural surroundings To get away from the regular routine To obtain a trophy fish For the challenge or sport Mixed-mode: N=408, LR X 2 = 66.63, p =.000, Pseudo R 2 =.12 Combination: N=239, LR X 2 = 51.93, p =.000, Pseudo R 2 =.21

32 Within Groups - Motivation Table 7. T-test results between respondents who completed survey by paper or online within mixed-mode and combination groups Mixed-modeCombination PaperOnlinep-valuePaperOnlinep-value Motivations To be outdoors To get away from the regular routine For the challenge or sport Most important/Not most important 35.7%35.0% %28.6%.843 Mixed-mode: N=408, LR X 2 = 66.63, p =.000, Pseudo R 2 =.12 Combination: N=239, LR X 2 = 51.93, p =.000, Pseudo R 2 =.21

33 Discussion – Response Rates Combination (surface mail & ) yielded strongest response rate Lowest non-response error? Respondents (or non-respondents) are reluctant to go from the paper invitation to their computer or device

34 Discussion - Socio-demographics Socio-demographic variation Variations in age Web-based respondents slightly higher household incomes Men more inclined than women to complete online

35 Discussion – Motivation & Avidity Motivation On items where there was significant variation web-based respondents considered these facets more important Avidity Some indication that hard copy respondents most avid

36 Next Steps 2015 Survey of licensed Texas Anglers Mixed-Mode – Mail survey packet with a web push Incentive – “lifetime license”

37 Acknowledgements Texas Parks and Wildlife Department Robin Riechers John Taylor