HETUS Pilot Group 7 Designing experiments for calibration: cross-mode, cross-country, and cross-time comparability Kimberly Fisher, Centre for Time Use Research – co-ordinator Jonathan Gershuny, Ignace Glorieux, Joeri Minnen, Hannu Paäkkönen, Vesna Pantelic, Annemarie Wennekers 12 January 2016

Comparability in time use surveys 2 of many key policy values from time use surveys: Tracking changes in behaviour across time Comparing time use across countries Apparent differences in time use can arise from: Varying data processing across time & countries Improvements have comparability consequences Variations in survey directions & introductions New modes of data collection

Comparability in time use surveys Key value and flexibility of the HETUS design arises from own-words reporting of activities and context information The way people think about and report their time use subtly changes over time Cultural and social change Response to new technologies generating new methods of accomplishing tasks

Survey Opportunities Range of comparability concerns increasing Most recent research underlying resilience and robustness of the time diary method – GPS maps, accelerometers, and camera images enhancing accounts, adding details and improving accuracy of reporting, but also revealing the same overall picture captured by the traditional paper diary

Gold standard aspiration Distinguish changes in time use that reflect variations in behaviour between countries or across time from: Changes in the way people think about and report their time use Variations in the way national statistical institutes collect and process time use survey data Different modes of time use survey collection

Topics covered by Group 7 Knowledge on which we can build New mode tests and mixed-mode surveys Measuring comparability Work to be done with recently collected data Experiments need to reflect decisions to be made by other groups

Knowledge on which to build HETUS experience Tables for Eurostat Statistics Sweden table generator; Statistics Finland next generation analytic tool Multinational Time Use Study TUS-X element of IPUMS Time Use National survey sequences (Belgium (Flanders region), the Netherlands, Norway)

Lessons from harmonisation Survey metadata matters Survey introductions, instructions, layout produce subtle variations in data Documentation & data require harmonisation Diaries collect narrative accounts; can create harmonised file columns from more than one original diary column Harmonisation requires checking of Internal consistency across diary & survey elements Consistency of reporting across time

New mode surveys Many regional / small GPS, web, app surveys Netherlands smartphone (HETUS in mind) 2011 UK GPS monitor (HETUS diary used) Danish 2008-2009 (HETUS) – included paper & web options to complete some instruments MO-TUS 2014 (in parallel to 2013 Belgian HETUS)

Mixed-mode national surveys 2015-16 UK Millennium Cohort Survey – 19,000 young people aged 14, smartphone app, web and paper light diaries Happiness Research Organisation Canada 2015 time diary element of General Social Survey Capture 24

Millennium Cohort Survey Pilot Testing

Millennium Cohort Survey Pilot Testing

Mixed-mode early lessons Comparable levels of support to diarists completing each mode of diaries Best to allow participants multiple paths to record new episodes App modes collect most passive detail, but diary narratives sufficiently flexible for multiple modes to produce similar pictures of behaviour Methodological work to promote comparability key

Measuring comparability Any spike or pronounced drop not directly associated with externally verifiable information suspicious Need to consider whole survey, as well as men separately from women for all measures during testing at a minimum

Engagement with survey overall Time required to complete each instrument Response rate by instrument / section Basic demographic profile of respondents completing each instrument by each mode / survey year

Engagement with diary How often entries made in diaries When diaries completed relative to observed day Proportion completing Two good quality diaries One good quality diary Only insufficient diaries No diary

Engagement with diary Engagement with each of the diary fields Proportion with complete 24 hour account Proportion empty, missing 1-90 & 91+ minutes Total missing time Episodes generated solely by that field Episodes generated by that field (only + with others) Main activity reporting Number of distinct main activities Presence of basic domains Mean time in 2-digit activity codes

Engagement with diary Secondary activity reporting Location reporting Number of distinct secondary activities Proportion of diaries with some secondary activity Mean secondary activity time Location reporting Number of distinct locations reported Mean time at home

Engagement with diary Who else was present reporting Proportion with some time alone & some with others Proportion with time alone only; time with others only Proportion with time alone with others reporting Enjoyment / happiness reporting Proportion with the lowest and highest ratings Median range of emotions reported Proportion with a single emotion report for whole day

Promoting comparability Retain capacity to recode to current HETUS standards Produce guidance on constructing calibration tables – consider creating some calibration estimates for use of smart technologies? Detailed documentation essential

Pilot Group 7 Priorities Experiments need to reflect decisions reached from other pilot groups A significant amount of data presently available or which will become available later in 2016 minimally analysed or not yet analysed Fund testing of existing data now to Learn lessons from existing work Learn what gaps require further testing