1. Center for Survey Research University of Virginia Center for Survey Research University of Virginia The Variable Costs of Cell Phone Interviewing Understanding Cost and Productivity Ratios in Dual-Frame Telephone Surveys Presented at AAPOR 2010 Chicago, IL May 14, 2010

2. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Authors Thomas M.Guterbock University of Virginia TomG@virginia.edu Paul J. Lavrakas Stamford, CT pjlavrak@optonline.net Trevor N. Tompson Associated Press TTompson@ap.org Randal ZuWallack Macro International Randal.zuwallack@ macrointernational.com

3. Center for Survey Research University of Virginia Center for Survey Research University of Virginia What we already know: Many telephone studies in the U.S. are using dual- frame designs that combine cell phone RDD with traditional (list-assisted) landline RDD. Cell phone RDD interviewing in the U.S. is expensive. Cell phone RDD interviewing costs more than landline RDD interviewing.

4. Center for Survey Research University of Virginia Center for Survey Research University of Virginia This paper explores: Why might cell phone cost more? –Focus: production interviewing costs. –Front-end and back-end project work not considered What are the key cost components? How much more? –What are the cost ratios [CP/LL]? How much variation is there in the cost ratios? What factors drive the cost ratios up or down?

5. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Overview Review of cost components Closer look at production rate differentials –HPC ratio, SRHPC ratio Notation and formulas An empirical example Results of a survey of 38 dual-frame studies –Mean costs ratios and variation in cost ratios Regression model of some design features The future of cell phone interviewing costs

6. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Cost per interview [CPI]: three components CPI is sum of three types of cost: –Incentives (plus any mailing costs) –Cost of purchased sample phone numbers –Interviewer hours Including interview time and non-interview time

7. All 3 yield higher cost for CP Cell phone RDD Landline RDDHigher cost for: Incentives$5 or $10 standardNone usually offered CP Sampled numbersMore numbers needed per complete Fewer numbers needed CP Hours per completion Much higherLowerCP

8. Center for Survey Research University of Virginia Center for Survey Research University of Virginia A closer look at interviewer hours The “interviewer hour” or “CATI lab hour” is the basic unit of CATI survey budgets –for fees and for costs Interviewer hours per completion = HPC = hours/completes = 1/CPH screening and recruiting hours per completion = SRHPC HPC = interview length + SRHPC

9. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Interview length Interview length is usually similar for CP and LL –CP interview may include: extra phone service questions time at end of interview to offer incentive & get contact info –CP interview usually slightly longer –But some studies have used a shortened interview on the CP side

10. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Factors that affect SRHPC Dialing method –CP must be manually dialed –LL might use predictive dialer, or 1-by-1 autodialing Four factors that affect number of dialings needed per complete –Working number rate –Contact rate –Eligibility rate –Cooperation rate Of interest: differentials in these rates between CP and LL on the same study

11. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Working number rate Lower for CP –Landline RDD is based on working number banks –CP cannot be pre-screened for non-working numbers –Working status for cell phones may be unclear –Working number density for CP usually lower But working-number density for LL and CP can vary from area to area –Not clear that CP density is always much lower

12. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Contact rate Lower for CP Some people use cell phone as an auxiliary device –Many phones are turned off Every cell phone has call screening built in CP may get less effort (attempts per number) by design

13. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Eligibility rate Lower for CP: –Many more minors reached in CP They can’t as easily hand off to adults as in LL –Geographic eligibility may be lower in CP LL RDD can only approximate specific geographies CP approximates less well, based on rate centers –Screening design affects eligibility differential “CPO-only” designs will lower eligibility for CP, compared to “all-cell” designs Study requirements (age, ethnicity) can affect eligibility differentials

14. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Cooperation rate Probably no great difference between CP and LL –Cash incentive on CP side helps reduce the differential –Refusal conversion more difficult on CP side –But advance letters, token cash incentives, or refusal conversion letters on the LL side could increase the differential

15. Center for Survey Research University of Virginia Center for Survey Research University of Virginia A look back at 2010 Cell phone cost survey was part of 2010 AAPOR Cell Phone Task Force report –Cost section by Guterbock, Tompson, zuWallack, Lavrakas Convenience sample of 8 SRO’s –4 academic, 4 private sector 26 usable surveys reported 15

16. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Notation and formulas LLab cost per hour Includes interviewer labor, allocated supervisor time, long distance costs, overhead. In most studies, this is the same for cell and RDD. CPHCompletions/hour=1/HPC HPCHours per completion=1/CPH C number of complete interviews N Number of sampled (attempted) phone numbers Unused numbers are not included in calculated cost YYield = C/N How many completes per sampled number Ssample cost per numbere.g., 13¢ per number for RDD Eoverall cost per interview E for 'expense'; sometimes called CPI for Cost Per Interview

17. Three equations eq. 1: E = I + S*N/C + L*(HPC) Expense of a complete case is the incentive cost plus the cost of sample plus the cost of required interviewing hours. [r]subscript for LL RDD [c]subscript for cell phone eq. 2 e = E [c] /E [r] e is the ratio of cell phone cost per interview to landline cost per interview SRHPC Screening and recruiting hours per completion All of the time in the HPC outside the interview itself. MMinutes of interview length M/60Interview length in hours eq 3: HPC = M/60 + SRHPC SRHPC = HPC – M/60

18. An example: Prince William County 2009 Cell Phone RDDLandline RDD Basic data: Completions [C] 269446 Interview length (minutes) [M] 2523.6 Completions per hour [CPH].394.853 Numbers attempted 66475212 Cost per hour of interviewing [L] $32 Cost per sampled number [S] $.10 Cash incentive [I] $10.00--

19. PWC 2009 Cost Ratios Derived data: Cost ratios: Hours per completion [HPC] 2.5371.172 hpc (ratio): 2.16 Interview length (hours) [M/60].416.393 Screening and recruitment hours per completion [SRHPC = HPC - interview length] 2.121.779 srhpc (ratio): 2.72 Numbers sampled per completion [N/C = 1/Y] 24.711.7 n/c (ratio): 2.11 Sample cost per completion [S*N/C] $2.47$1.17 Interviewing cost per completion (HPC × cost per hour) [L*HPC] $81.19$37.51 Overall cost per completion [E = Interviewing + sample + incentive] $93.66$38.68 e (cost ratio): 2.42

20. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Our data Cell Phone Task Force Cost Committee members contacted sources at eight firms –4 academic, 4 non-academic –Identity of firms and surveys is confidential Obtained info on 38 separate dual-frame RDD surveys Generated cost ratio for each survey –Data incomplete for some surveys –About 26 usable in most analyses

21. Productivity Statistics and Cost Ratios for Dual Frame RDD Surveys Screening and Recruiting Hours per Completion Hours per Completion Overall Cost per Interview Ratio (cell/landline)SRHPC RatioHPC RatioCost Ratio Mean 2.532.002.05 Minimum 1.211.171.35 Maximum 5.373.473.97 N 26 20 Std. deviation 1.02.63.77

22. Means for All-Cell (No Screening) versus CPO-Only Surveys Type of Dual- Frame Design: Screening and recruiting hours per completion Hours per completion Overall Cost per Interview SRHPC ratioHPC ratioCost ratio All-cell designsMean 2.381.871.96 N20 17 CPO-only designsMean 3.032.452.60 N663

23. Means for Local Surveys versus National/State Surveys, by Design Type Dual frame design: Geography : Screening and recruiting hours per completion Hours per completion Overall Cost per Interview SRHPC ratio (srhpc)HPC ratio (hpc)Cost ratio All-cell designs National & State wide Mean 2.361.841.86 N 14 12 LocalMean 2.421.922.19 N 665 CPO-only designs National & State wide Mean 3.032.451.92 N 662 LocalMean -- 3.97 N 001

24. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Modeling srhpc (the SRHPC ratio) Distribution of srhpc is somewhat skewed Log of srhpc serves better as a dependent variable Predictors: –Geography statewide or local (compared to national) –CPO-only screening (compared to all-cell) –Predictive dialing (compared to 1-by-1 dialing)

25. Distribution of raw shrpc

26. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Distribution of logged srhpc

27. Regression of logged srhpc on study features (n = 26, r 2 =.171)

28. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Interpreting the regression result Based on effect size, screening design and predictive dialing have strong effects –CPO-only screening raises srhpc by 10.187 = 1.22 = 22% –Predictive dialing raises srhpc by 10.179 = 1.20 = 20% Geography has no effect on srhpc net of the other factors Since sample size is small, no coefficient is statistically significant.

29. Center for Survey Research University of Virginia Center for Survey Research University of Virginia This is but a first step... We need data on more studies –In more closely comparable format –From a probability sample of dual-frame studies We need to explore effects of more design features Explore: explicit modeling of component rates (working, contact, eligible, cooperation)? –But these are not easily observed in real studies

30. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Practical use of this approach Most dual-frame surveys grow out of landline RDD studies –So cost factors for LL are known from prior fielding Estimate srhpc (the ratio) using a model of this kind –Apply this ratio to LL SRHPC to estimate CP SRHPC Then use equations 1 and 3 to add in (known) interview length, hourly cost of CATI interviewing, cost of sampled numbers, incentives RESULT: Cell phone cost per interview [CPI]

31. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Summary of cost findings Cell phone calling DOES cost more –But cost differentials vary considerably across studies Differentials in SRHPC are the main source of the cost differences In a typical dual-frame survey with “all-cell” design, CP will be half as productive as LL If CPO-only screening design is used, CP will cost two- and-a-half times LL –By making CP less efficient Predictive dialing increases cost difference –By making LL more efficient Geography of study area has little effect on cost ratios

32. Center for Survey Research University of Virginia Center for Survey Research University of Virginia Cost differentials may change in the future If LL working-number densities decrease and CP densities increase, CP cost will be more like LL cost If more cell users become CPO’s, they may be easier to contact and more likely to respond, decreasing cost differential If public directories of CP numbers emerge, then list- assisted CP RDD might become possible Some researchers may substitute directory-listed LL [EWP] samples for RDD, thus saving costs on the LL side Researchers may succeed in improving CP productivity by better timing of calls and better-tailored protocols

33. Center for Survey Research University of Virginia Center for Survey Research University of Virginia The Variable Costs of Cell Phone Interviewing Understanding Cost and Productivity Ratios in Dual-Frame Telephone Surveys Presented at AAPOR 2010 Chicago, IL May 14, 2010 For a copy of our paper, e-mail: Thomas M. Guterbock TomG@virginia.edu The Variable Costs of Cell Phone Interviewing For a copy of our paper, e-mail: Thomas M. Guterbock TomG@virginia.edu