Mixed-Mode Approaches in the Generations and Gender Survey Past Experience and Future Expectations Aat Liefbroer Netherlands Interdisciplinary Demographic Institute

Overview of the presentation  GGP: what is it?  Past experience  3 rd wave of Dutch GGP  Future expectations  Pilot 4 th wave in Slovenia 2011  Blueprint for GGP 2015  Conclusions Overview

Aim of the GGP  To advance our knowledge on intergenerational and gender relationships, with a focus on understanding gender inequalities and generational differences  Internationally comparable data on relationships between partners and between parents and adult children across Western societies  Main vehicles: Generations and Gender Survey (GGS) and Contextual Database Overview

Characteristics of the GGS  Broad age-range (18-79), because understanding demographic challenges such as population ageing asks for a focus on both young and old people  Large-scale surveys on population challenges (N≈10,000) to allow in- depth analysis  Panel design to allow for better causal analysis and studying processes of adaptation to change  Cross-national to allow for examining the influence of the social context (including the policy context)  Theory-driven questionnaire design  Contextual macro-level database to allow for multi-level analyses Overview

GGP Countries  Wave 1  18 countries (data for 12 countries currently available)  Wave 2  Bulgaria, Netherlands, Australia, Germany, France, Hungary, Italy, Russian Federation, Georgia (Bulgaria and Germany to be released this year) Overview

Challenges for the future of the GGP  Increasing the number of participating countries  Extending the data-collection beyond three waves  Reducing fieldwork costs  Application of mixed mode design? Overview

Dutch GGP  1 st wave 2004 ≈ 8,200 respondents, CAPI  2 nd wave 2007 ≈ 6,200 respondents, CAPI  3 rd wave 2010 ≈ 4,300 respondents, mixed mode  Response rate in 3rd wave: 72% of wave 2 participants  Costs per respondent in 2nd wave: ≈ €195  Costs per respondent in 3rd wave: ≈ €115 3 rd wave Dutch GGP

Distribution by mode 3 rd wave Dutch GGP

Mode distribution by age 3 rd wave Dutch GGP

Mode distribution by educational attainment 3 rd wave Dutch GGP

Missing income info by mode 3 rd wave Dutch GGP  Percentage that does not mention earnings from labour

Disclosure 3 rd wave Dutch GGP  Mean score on loneliness scale by mode (with and without controls)

Attrition information (I) 3 rd wave Dutch GGP  Percentage that gives permission to contact their partner to fill out a questionnaire

Attrition information (II) 3 rd wave Dutch GGP  Percentage that gives permission to link data to register data from Statistics Netherlands

Attrition information (III) 3 rd wave Dutch GGP  Percentage that gives permission to be recontacted for the fourth wave

Conclusions 3 rd wave Dutch GGP  WEB more popular than expected  Missing values on WEB limited  Small, but systematic mode effects in disclosure  WEB respondents are less willing to give permission for additional or future involvement

Pilot 4 th wave Slovenia 2011 Pilot Slovenia  Participating institutions  University of Ljubljana (Lozar-Manfreda, Petrič)  University Utrecht (Hox, De Leeuw)  NIDI (Kveder, Liefbroer)  Study 1: comparison of mode effects  Study 2: comparison of mixed mode systems

Comparison of mode effects  Aim: testing new pilot and examination of potential mode effects  Sample: regular participants in commercial web panel, randomly assigned to different modes  Examination of  Item non-response  Scalability of item sets  Distributional characteristics  Length of interview  Evaluation of interview Pilot Slovenia

Comparison of mode effects  Item non-response only slightly higher in web, with one clear exception…. Pilot Slovenia

Income  Percentage of refusals and ‘do not know’ on question on household income Pilot Slovenia

Comparison of mode effects  Item non-response only slightly higher in web, with one clear exception….  Scalability of item sets still has to be analysed  First analyses of distributional characteristics suggest  More disclosure on social desirable and difficult questions  Slightly higher variance in responses Pilot Slovenia

Example  Percentage answering ‘yes’ to the question ‘Over the past 12 months, have you thought about breaking up your relationship?’ Pilot Slovenia

Comparison of mode effects  Item non-response only slightly higher in web, with one clear exception….  Scalability of item sets still has to be analysed  First analyses of distributional characteristics suggest  More disclosure on social desirable and difficult questions  Slightly higher variance in responses  Interview took somewhat longer in CATI mode  Evaluation of interview most negative in CATI mode Pilot Slovenia

Duration of survey Pilot Slovenia

Subjective length of survey  Percentage stating ‘yes’ or ‘definitely yes’ to question whether the interview was judged too long Pilot Slovenia

Comparison of mixed-mode systems (I)  Aim: testing of alternative mixed-mode systems  Sample: two-stage random sample from the population register  Examination of  Response rate of different mixed-mode systems  Costs of different mixed mode systems  Evaluation of modes by a random sample Pilot Slovenia

Comparison of mixed-mode systems (II)  Comparison of four ‘systems’  CAPI → WEB → CATI (no incentive offered)  CATI → WEB → CAPI (no incentive offered)  WEB → CATI → CAPI (no incentive offered)  WEB → CATI → CAPI (incentive of € 5 offered)  Difference in response rate across systems  Difference in costs across systems Pilot Slovenia

Blueprint GGP 2015  Blueprint for GGP 2015 should be ready by the end of 2012  Questionnaire GGP 2015  Sampling design  Set of fieldwork rules  Whether to opt for mixed-modes, and if so when?  Is it an option in the first wave (or in the first wave with a refreshment sample), or in later waves only?  What kind of mixed-mode system to prescribe?

Conclusions Conclusion  Overall, WEB seems to perform quite well in terms of answering patterns (item nonresponse, scalability and distribution of responses)  WEB leads to very considerable cost reductions  The big issue that is not yet clear is whether attrition is negatively affected  Use of WEB in a first wave seems questionable (but pilot will give partial answers)  Use in later waves could lead to higher dropout in future waves as a result of the legal requirement to ask for permission for re- contacting

Thank you for your attention!

Topics included  Provision of care to older adults  Successful ageing  Economic participation of women and pre-pensioners  Realisation of fertility intentions  Balancing work and family life  Gender equality in household and childcare labour Overview

Members of Consortium Board  Netherlands Interdisciplinary Demographic Institute (NL)  Institut National d’Études Démographiques (FR)  Bocconi University (IT)  Statistics Norway (NO)  Demographic Research Institute (HU)  Norwegian Social Research (NO)  University of Ljubljana (SL)  Utrecht University (NL)  University of York (GB)  Max-Planck Institute for Demographic Research (DE)  Erasmus University Rotterdam (NL)  Population Unit of the UN Economic Commission for Europe