Does mixing F2F and web lead to cost savings? Mari 18 July 2013 ESRA2013.

Slides:



Advertisements
Similar presentations
Welsh Health Survey Anne Kingdon Welsh Assembly Government Health Promotion Division.
Advertisements

Statistical Directorate Public Health Improvement Division Cath Roberts & Rachel Dolman Welsh Assembly Government.
Survey expenses and budget allocation Ed Rege ILRI 26 February 2003.
Summer Internship Program Outline
1 FY10 ACS Methods Panel Update Jennifer Guarino Tancreto Chief, ACS Data Collection Methods Staff Decennial Statistical Studies Division Presentation.
 Survey Skills Programme - 1 seminar day field days  Working at Survey Organization - 10 (+0-3) days of placement on a survey related.
Analytical Evidence on Research & Innovation in the Danube Region Progress of WP4 Vienna, Béla Kardon, PhD; RCISD
Innovative Approaches to Mapping Local Public Health Systems Serving Aboriginal Peoples Martin Cooke, University of Waterloo Storm Russell, Métis Nation.
Mixed-Mode Approaches in the Generations and Gender Survey Past Experience and Future Expectations Aat Liefbroer Netherlands Interdisciplinary Demographic.
The English Longitudinal Study of Ageing (ELSA) Sample design & response Shaun Scholes NatCen.
Chapter 11: Collecting Data by Communication. Key Issues for Collecting Information by Communication.
ESS Mixed mode experiment ESRA Conference, Ljubljana, July 2013 Alun 18 July 2013.
Building Web Collection Capability for ONS Social Surveys Fiona Dawe and Laura Wilson.
Response rate in Switzerland Dominique Joye
Knowledge is Power Marketing Information System (MIS) determines what information managers need and then gathers, sorts, analyzes, stores, and distributes.
Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Increasing response rates Ineke Stoop SCP.
Introduction of Internet Survey Methodology to the Emirate of Abu Dhabi Andrew Ward, Maitha Al Junaibi and Dragica Sarich.
Two Better Than One? Exploring Couples’ Home Energy Use Attitudes Shu Yang Visiting doctoral student UCL Energy Institute Current
Understanding Society: the UK Household Longitudinal Study Professor Vernon Gayle ISER, Essex University
Fourth Joint EU-OECD Workshop on Business and Consumer Opinion Surveys EU-wide business survey in the financial service sector Brussels, 12th October 2009.
An initiative by the Economic and Social Research Council, with scientific leadership by the Institute for Social and Economic Research, University of.
Timing Isn’t Everything, But Money Talks How to encourage a face-to-face household panel to go online? University of Essex, July 2013 Gerry Nicolaas Carl.
Data Collection, Non-response Error, & Questionnaire Screening.
Adding a Mode as a Final Step in the Follow-up of a Panel Survey Seminar on New Frontiers for Statistical Data Collection Geneva, Switzerland, 31 October.
Making Sense of the Social World 4th Edition
Nonresponse issues in ICT surveys Vasja Vehovar, Univerza v Ljubljani, FDV Bled, June 5, 2006.
Fieldwork efforts  Monitoring fieldwork efforts  Monitoring fieldwork efforts: Did interviewers /survey organisations implement fieldwork guidelines.
1 Metagora: Current Progress and the Way Forward PARIS21 Steering Committee Paris, 13 November 2007.
Version 1 | Public (DELETE CLASSIFICATION) Version 1 | Internal Use Only Version 1 | Confidential Version 1 | Strictly Confidential© Ipsos MORI ESRC Research.
Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.
Old.libqual.org What will ARL do for you? What will you do? January 2005 Shrivenham, UK.
2020 Census: Program Overview, Testing, and Technological Innovations
Understanding the Decision to Participate in a Survey and the Choice of the Response Mode Anders Holmberg and Boris Lorenc European Conference on Quality.
Lesli Scott Ashley Bowers Sue Ellen Hansen Robin Tepper Jacob Survey Research Center, University of Michigan Third International Conference on Establishment.
WHO ARE KENYANS PREPARED TO VOTE FOR AS OF NOW? SPEC BAROMETER RESULTS November 9th 2007.
Internet Surveys & Future Works In Official Statistics 1 The 4th International Workshop on Internet Survey Methods, Sept. Statistical Center, Statistics.
Professor Charles Crothers New Zealand Internet Rights Survey.
Census 2011 in Portugal: Development of e-Census system Statistics Portugal - Census Unit and IT Department Geneva, October 2009.
1 Renee M. Gindi NCHS Federal Conference on Statistical Methodology Statistical Policy Seminar December 4, 2012 Responsive Design on the National Health.
Eurostat Data collection. Presented by Johan Erikson Statistics Sweden.
Setting Your Survey in Motion & Getting It Done Prepared by: Ali Al-Baggal.
The British Household Panel Survey Began in September 1991 National sample of England, Scotland and Wales 5,000 households/10,000 interviewed adults 16+
Six Bells Event Jane and Neil WERU. Valleys Regional Park Six Bells One Day Event in June Six Bells Event - background The Survey – practical details.
Understanding the Respondent – a Key to Improve our Data Collection Strategies? Seminar on Statistical Data Collection 26 th September 2013 Anton Johansson,
The Challenge of Non- Response in Surveys. The Overall Response Rate The number of complete interviews divided by the number of eligible units in the.
Copyright 2010, The World Bank Group. All Rights Reserved. Reducing Non-Response Section A 1.
30 September – 1 October 2010 European Health Interview Survey (EHIS): update EHIS Workshop in Berlin.
AAPOR 69 th Annual Conference Anaheim, CAMay 17, 2014 Casey Langer Tesfaye and Susan White Can a survey of U.S. High Schools be Replaced or Reduced through.
Cuckmere Estuary Follow-on Survey Public Evidence Meeting Presentation Alfriston Memorial Hall, 5 April 2011.
Kick-off Meeting Granada, 16-17/04/2009 Agri-Multifunctionality II: Phases and Objectives.
Patterns of community involvement: what are they and can they be changed? Kirby Swales, Survey Research Centre.
The measurement effect in PC smartphone and tablet surveys Valerija Kolbas University of Essex ISER Ipsos-MORI The European Survey Research Association.
Analysis of the characteristics of internet respondents to the 2011 Census to inform 2021 Census questionnaire design Orlaith Fraser & Cal Ghee.
15-April-10 Johan van der Valk Sub sample of persons in Labour Force household Survey Just an idea.
BHPS User Group Meeting, 2 nd May 2008 BHPS: Wave 18 Jon Burton ISER, University of Essex.
Margaret Blake RESEARCH METHODS FESTIVAL: Evaluating questionnaires with cognitive testing.
Day 1 Quality control methods and practice Kees van Berkel Mariëtte Vosmer Jerusalem, July 2013.
Component D: Data Collection in Field Surveys Activity D.1: Management and monitoring of field interviewers Surveys Department EU Twinning Project.
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning Alessandra Gaia Institute for Social and Economic Research (ISER)
Context for the experiment?
Planning, Monitoring and Performing Surveys
Mixed mode data collection in official surveys
Survey Scheduling And Monitoring in eRCTs (SESAMe): a digital tool to improve data collection in clinical trials. Trygve Skonnord, Finn Steen, Arne.
Using the web to survey the UK general population
Pilot Testing of the VAW Survey Module in Georgia: Preliminary Results
The second wave of the new design of the Dutch EU-SILC: Possibilities and challenges Judit Arends.
The effects of rotational design and attrition
Panel care in the Austrian EU-SILC
In-field resolution of fieldworker falsification
Workshop on best practices for EU-SILC revision, −
Presentation transcript:

Does mixing F2F and web lead to cost savings? Mari 18 July 2013 ESRA2013

1 Contents Background About the study Response Potential for cost savings Actual cost savings Cost model Conclusions

Background 1.

3 Context Increasing attention on mixed modes Potential for cost savings Potential for increasing response Unclear to what extent this potential realised in reality Especially if face-to-face involved

4 Survey I NatCen ran for ISER at University of Essex Main survey largest household panel in the world IPs test innovations and carry experiments IP5 – first test of F2F/Web in a large F2F panel survey

5 Survey II Household survey: Household questionnaire Individual questionnaire Adult self-completion Youth self-completion Split off households Sample: original W1, W4 refresh, W4 non-respondents 15 experiments Interview length 60 minutes per household Incentives: £5, £10, £20, £30

6 Design Experimental group F2F phase, web open Web only phase Control groupF2F only phaseNo web

Response 2.

8 Web take-up 23% Households completed fully online Another 13% completed partially

9 Household response by sample type, original responding sample Base: All issued households in original sample (responding) Source: Jäckle, Lynn, Burton (2013)

10 Household response by sample type, refreshment sample Base: All issued households in refreshment sample Source: Jäckle, Lynn, Burton (2013)

11 Web response by incentive amount Base: All households in mixed mode sample Source: Jäckle, Lynn, Burton (2013)

Cost 3.

13 Expected effect on cost Decrease No interviewing fees for web completes No trips required to web completes Increase More programming: web and F2F questionnaires Set up and manage s Man a helpline for the web group

14 One-off costs Research time to develop new procedures and support the mixed modes Develop sample management system Update response monitoring tools Field management cost to support interviewers

15 Effect on costs Total costsup by third Fielwork costsdown by tenth Programming costup three times Research costup by two-thirds

16 Fieldwork costs Main source of savings: Fees for productive cases Marginal savings on: Travel time and mileage – 3% Why travel less affected: Main source of cost – travel to and from PSUs Smaller points – more travel Web cases in F2F more difficult – require more calls

17 Cost of achieved interview by incentive amount Base: All issued households

18 Simulated costs per achieved sample size

Conclusion 4.

20 Conclusions Mixed modes take up is higher than expected No evidence that response in mixed modes is higher Risk of more refusals and partial households Costs increase in the first year Fieldwork costs do not decrease as much as might be expected Increase in web response with higher incentives is not enough to offset the incentive value Mixed modes design becomes cost effective at higher achieved sample sizes

If you want further information or would like to contact the author, Mari Toomse-Smith Senior Research Director T E. Visit us online, natcen.ac.uk Thank you