Do Monetary Incentives Increase Business Survey Response Rates? Results from a Large Scale Experiment Paul Biemer, RTI International and University of North Carolina Christopher Ellis, Angela Pitts and Kimberly Robbins RTI International

Outline of the Presentation  Essential information about the establishment survey  Design of the incentive experiment  Results  Conclusions

The O*NET Survey  Occupation Information Network Survey (O*NET)  Provides descriptive ratings on 800+ U.S. occupations  Target population is all U.S. employees in these 800+ occupations  Continuing survey since 1997  Mode of interview  Telephone contact of establishments  Point of contact (POC) is usually Human Resources Manager  PAPI or Internet questionnaires completed by selected employees

The O*NET Survey (cont’d)  Two-stage sample design  Establishments and employees within establishments in selected occupations  ~24,000 establishments/year (79% RR)  ~36,000 employee respondents/year (69% RR)

O*NET Incentive System  All incentives are prepaid (prior to receipt of questionnaires)  For the establishment:  O*NET toolkit (personnel job description aid)  For POC:  Desk clock  Framed certificate of appreciation  $20 money order (to be tested)  For employees  $10 cash

POC Incentive Experiment: Motivation  Concern that POC incentives not commensurate with POC tasks  Some evidence of diminished POC effort during employee nonresponse follow-up phase  Establishment and employee nonresponse rates were low for some occupations  POC does not receive monetary incentive although employees do ($10)

Overview of Data Collection Protocol

Incentives sent here Brochure sent here

Cover of the POC Incentive Brochure FRONT BACK

Inside of the POC Incentive Brochure

Experimental Design  Split sample design (75/25 split)  Incentive group: POCs offered $20 money order in addition to clock and framed certificate  Control group: POCs offered only clock and framed certificate  Money orders could be made out to POC, the business or a favorite charity  Interviewers (called business liaisons or BLs) handled both incentive and non-incentive cases

Sample Allocations to the Control and Incentive Groups ControlIncentive Establishments2,6247,874 Employees7,69422,309

Results

Unweighted Response Rates (%) ControlTrtDiffP-value Establishment Employee

Other Results  No effect on response rates by  size of establishment  industry grouping  urban or rural regions  occupation of employee  Some evidence of a negative effect for some sectors

Cumulative Employee Response Rates for by Week of Data Collection

Summary of Results  No evidence of an effect on establishment cooperation rates  Weak evidence of an effect on employee cooperation rates  Both negative and positive effects detected for small subgroups  Overall, no significant difference  No cost advantage  Follow-up costs were the same  Speed of response was the same  Incentive costs were not offset by efficiency gains

Interpreting the Findings  No establishment-level effect  $20 incentive viewed as trivial by POCs  Framed certificate and clock could have much larger perceived value  Decision to participate shared by POC and company  POC cannot accept money in many establishments  No effect at the employee-level  POC has limited ability to affect employee response  $20 ineffective for increasing follow-up intensity