RTI International is a trade name of Research Triangle Institute th Street, NW ■ Suite 750 ■ Washington, DC, USA Phone A Revolution in Survey Research Karol Krotki Washington Statistical Society and DC-AAPOR Washington, DC 20 May, 2009

2 Challenges – Cell Phones  Close to 20% household are cell-phone-only (CPO)  Mapping no longer 1-1  One individual can own several numbers  Several individuals can share the same number  More difficult to link number to geographical location  Data collection  Safety  Compensation  Separate frame

3 Challenges – Response Rates  OMB used to insist on 80%  Now 60% is difficult to achieve  Decrease in response rates results in increase in survey costs  Decrease in response rates leads in increased risk of nonresponse bias

4 Challenges – New Technologies  Traditional approaches  Landline phone endangered species  In-person too expensive  Internet/Web  Smart phones  Other

5 Future Directions – Multiple Frames  Cell phone and landline phone  Address lists (see below) and phone lists  Internet data bases  Commercial lists  Administrative data/large data bases  External sources of information  Disclosure control/confidentiality  Data linkage

6 Future Directions – Address-Based Sample  Complete list of addresses  Process  Select sample of addresses  Match as many addresses with telephone numbers as possible  Use multiple modes (mail, telephone, in-person, internet) to collect data  Nielsen  Moved from telephone to ABS last November  “RDD is dead”

7 Future Directions – Internet Surveys  Coverage and selection bias  Harris – large base  Frame bias  Membership bias  “Professional respondents”  Is it possible to model to overcome shortcomings?  Knowledge Networks

8 Future Directions – Multi-Mode  Offer respondents a choice  Increase cooperation  Select optimal mode as a function of:  survey topic  target population(s)  However, beware of mode effects  Adjust mode during field period

9 Future Directions – GIS  Frame building  Sample selection  Screening  Data collection

10 Future Directions - GIS

11 Future Directions – Rolling Samples  Increase sample size by combining samples over time  E.g., American Community Survey (ACS)  Sampling and sampling errors more complex

12 Future Directions – Nonresponse Bias  Gauge survey quality using nonresponse bias rather than response rates  Correlation need not be high  Challenges  How to measure bias?  How to standardize measurement process?

13 Future Directions – Responsive Design  Adaptive/responsive surveys  Requires multi-mode, flexible approach  Requires monitoring during data collection  Adjust strategies “mid-stream”  Cumulating knowledge  Sampling  Interviewing

14 Conclusions  Surveys of the future will be more complex  Need to be flexible  Need to be creative  Future looks very interesting, albeit not very clear