HETUS Pilot Group 7 Designing experiments for calibration: cross-mode, cross-country, and cross-time comparability Kimberly Fisher, Centre for Time Use Research – co-ordinator Jonathan Gershuny, Ignace Glorieux, Joeri Minnen, Hannu Paäkkönen, Vesna Pantelic, Annemarie Wennekers, and external consultant Kai Ludwigs 3 January 2016

Comparability in time use surveys 2 of many key policy values from time use surveys: Tracking changes in behaviour across time Comparing time use across countries Apparent differences in time use can arise from: Varying data processing across time & countries Improvements have comparability consequences Variations in survey directions & introductions New modes of data collection

Comparability in time use surveys Key value and flexibility of the HETUS design arises from own-words reporting of activities and context information The way people think about and report their time use subtly changes over time Cultural and social change Response to new technologies generating new methods of accomplishing tasks

Survey Opportunities Range of comparability concerns increasing Most recent research underlying resilience and robustness of the time diary method – GPS maps, accellerometers, and camera images enhancing accounts, adding details and improving accuracy of reporting, but also revealing the same overall picture captured by the traditional paper diary

Gold standard aspiration Distinguish changes in time use that reflect variations in behaviour between countries or across time from: Changes in the way people think about and report their time use Variations in the way national statistical institutes collect and process time use survey data Different modes of time use survey collection

Topics covered by Group 7 Knowledge on which we can build New mode tests and mixed-mode surveys Measuring comparability Work to be done with recently collected data Experiments need to reflect decisions to be made by other groups

Knowledge on which to build HETUS experience Tables for Eurostat Statistics Sweden table generator; Statistics Finland next generation analytic tool Multinational Time Use Study TUS-X element of IPUMS Time Use National survey sequences (Belgium (Flanders region), the Netherlands, Norway)

Lessons from harmonisation Survey metadata matters Survey introductions, instructions, layout produce subtle variations in data Documentation & data require harmonisation Diaries collect narrative accounts; can create harmonised file columns from more than one original diary column Harmonisation requires checking of Internal consistency across diary & survey elements Consistency of reporting across time

New mode surveys Many regional / small GPS, web, app surveys Netherlands smartphone (HETUS in mind) 2011 UK GPS monitor (HETUS diary used) Capture 24 Happiness.org time use surveys MOTUS (in parallel to Belgian HETUS)

Mixed-mode national surveys Danish 2008-2009 (HETUS) – included paper & web options to complete some instruments Canada 2015 time diary element of General Social Survey 2015-16 UK Millennium Cohort Survey – 19,000 young people aged 14, smartphone app, web and paper light diaries

Millennium Cohort Survey Pilot Testing

Millennium Cohort Survey Pilot Testing

Mixed-mode national surveys Danish 2008-2009 (HETUS) – included paper & web options to complete some instruments Canada 2015 time diary element of General Social Survey 2015-16 UK Millennium Cohort Survey – 19,000 young people aged 14, smartphone app, web and paper light diaries

Measuring comparability Engagement with each of the diary fields Proportion with complete 24 hour account Proportion empty, missing 91+ & 1-90 minutes Episodes generated solely by that field Episodes generated by that field (and others) Total time in aggregated categories in each field Participation in aggregated categories in each field

Measuring comparability Check most frequent multi-tasked activity combinations (consistency of which is the main and which is the secondary activity) Profile of respondents providing partial returns Profile of respondents returning unusable diaries & refusing to participate

Pilot Group 7 Priorities Experiments need to reflect decisions reached from other pilot groups A significant amount of data presently available or which will become available later in 2016 minimally analysed or not yet analysed Fund testing of existing data now to Learn lessons from existing work Learn what gaps require further testing