Effectiveness of Monetary Incentives and Other Stimuli Across Establishment Survey Populations ICES III 2007 Montreal, Quebec Canada Danna Moore and Mike Ollinger

(1) Why is this study important? (2) Gaps in Establishment Survey Literature (3) Provide suggestions for implementing establishment surveys Objective

Establishment Respondents: What type of respondent are they? Household Respondent e.g. small farmer, small business owner Large business or Org Respondent Multiple locations Gate keepers Record system Can vary along the continuum

Why is this important?  Questionnaire variation  Converting  Mandatory reporting  R’s Vary over time  Risk/Difficulty -- cash incentives  Population based sample sizes  Experimental design  Cost/Benefit  Generalize

Stimuli Tested Cash Incentives Cash plus special postage/package Multiple modes Mode sequencing Mode preferences Visual Design— color background vs. none Answer boxes stand out

Some Answers Towards the Big Question Do incentives help or hinder in obtaining survey responses from businesses? Crosses types of establishments and industries. Experiment based Population based Random assignment

Understanding Why People Participate Several theories (Dillman, 1978; Gouldner, 1960; Biner and Kidd, 1994; Groves et al., 1999). Social exchange Leverage Saliency Theory—Groves et al. (1999) Decision to participate is a series of interactive additive factors. Some are survey specific and others are person specific. Incentives are viewed as an inducement used to compensate for absence of some factors (i.e., saliency or sense of duty).

1999 POQ vol 64 Leverage Saliency Theory Of Survey Participation

Features of Establishment Surveys That Often Lead to Survey Errors 1. One person selected to represent establishment. 2. Burden increases as they answer as a representative. 3. There is a respondent questionnaire interaction. 4.Respondent’s characteristics in relation to the establishment influences their ability to respond. 5. intermediary between the questionnaire and the characteristics of the record system. 6. Organizational environment 7. Extenuating survey situation

Businesses differ across industries by size, structure, and organizational environment. Each survey may have a situation or circumstance that impedes contact. These differences and circumstances often influence how well a survey request can penetrate an establishment Factors Influencing Response

Gaps Lack of monetary incentive studies:  Few experimental treatments for definitive comparisons and outcomes.  Few comparisons of cash versus “cash like” ( e.g. checks, ATM cards) studies—could be especially important for government.  No empirical demonstrations of effectiveness of incentives across types of industries and firm size.  Few comparing effects of various size incentives.  Not much on effectiveness of survey modes and mode sequencing  Response attributable to survey mode sequencing.

2001 HMO Survey Physicians N=1474 *** **

2003 USDA Nationwide Meat Manufacturers N=1,705 Response rates achieved by experimental treatment group ***

2006 Snake River Grain Warehouse and Shipper Survey Response by Experimental Group (n=424 elevators) Chi SQ 6.7 P <.01 *** N.S.

2007 Evaluation of WA Plastic Pesticide Container Use/Recycling Control versus Treatment, N=1,986 Chi Sq ***P<.001 Chi Sq ***P<.0001

2006 Oregon Business Environmental Management Survey Type and Experimental Treatment Group Initial sample n=1964 *** ** *** ** ***

2006 Trade Adjustment Assistance Survey, All Qualifying Industries N=6,429 Exp. Incentive Treatment vs. Control ---Completion Rates Sig. Chi Sq P<.001 *** ***

Trade Adjustment Assistance (TAA) 2006 Survey of Program Nationwide – Fisheries N=5,592 ** ***

Trade Adjustment Assistance (TAA) 2006 Survey of Program Nationwide – Commodities N=837 NS

TAA Survey—Percentage of Completes Associated with Incentive Experimental Assignment and Survey Mode Mail Completes Telephone Completes Web Completes

Suggestions for implementing establishment surveys Effective Practice: Contact respondents multiple times Contact respondents in multiple modes. Allow for survey mode preference. Design surveys that reduce burden short, conditional branching, ease Use leverages

Visual Design Effect – FARW Commercial Dealers N=1600 Background Shading w/ Visible Answer Boxes Does it make a difference? Is there an interaction effect w/ Incentives? ***

Visual Design Test FARW Growers With Pesticide License ** ***

All 3 survey modes generated completes Exper. treatment (Cash incentive & priority mail) stimulated more responses in all 3 survey modes—large interaction effect. Offering web as an alternative option garnered 15% Telephone last 3% – still effective

Influential Circumstances Saliency: Topic interest  area of business emphasis for entity.  High personal interest for respondent.  High level of public or political concern Role of survey sponsor.  Regulator or source of certification.  Mandatory reporting.  Source of program $$ or sponsorship Response Burden  Complexity, length, multiple reports

What was learned from experimental trials Token cash incentives were effective across types of establishment and industry populations. 2-day Priority mail was more effective than first class mail. Priority mail alone just slightly better than First class Cash incentives combined with priority mail was synergistic. Mixed mode strategies are very helpful and work. Respondents may have mode preferences. Establishment population characteristics and the selected respondent characteristics need to be considered jointly in explaining response. The survey circumstances and situation impact establishment response.