1. HETUS Pilot Group 7
Designing experiments for calibration: cross-mode, cross-country, and cross-time comparability
Kimberly Fisher, Centre for Time Use Research – co-ordinator
Jonathan Gershuny, Ignace Glorieux, Joeri Minnen, Hannu Paäkkönen, Vesna Pantelic, Annemarie Wennekers, and external consultant Kai Ludwigs
3 January 2016

2. Comparability in time use surveys
2 of many key policy values from time use surveys:
Tracking changes in behaviour across time
Comparing time use across countries
Apparent differences in time use can arise from:
Varying data processing across time & countries
Improvements have comparability consequences
Variations in survey directions & introductions
New modes of data collection

3. Comparability in time use surveys
Key value and flexibility of the HETUS design arises from own-words reporting of activities and context information
The way people think about and report their time use subtly changes over time
Cultural and social change
Response to new technologies generating new methods of accomplishing tasks

4. Survey Opportunities Range of comparability concerns increasing
Most recent research underlying resilience and robustness of the time diary method – GPS maps, accellerometers, and camera images enhancing accounts, adding details and improving accuracy of reporting, but also revealing the same overall picture captured by the traditional paper diary

7. Gold standard aspiration
Distinguish changes in time use that reflect variations in behaviour between countries or across time from:
Changes in the way people think about and report their time use
Variations in the way national statistical institutes collect and process time use survey data
Different modes of time use survey collection

8. Topics covered by Group 7
Knowledge on which we can build
New mode tests and mixed-mode surveys
Measuring comparability
Work to be done with recently collected data
Experiments need to reflect decisions to be made by other groups

9. Knowledge on which to build
HETUS experience
Tables for Eurostat
Statistics Sweden table generator; Statistics Finland next generation analytic tool
Multinational Time Use Study
TUS-X element of IPUMS Time Use
National survey sequences (Belgium (Flanders region), the Netherlands, Norway)

10. Lessons from harmonisation
Survey metadata matters
Survey introductions, instructions, layout produce subtle variations in data
Documentation & data require harmonisation
Diaries collect narrative accounts; can create harmonised file columns from more than one original diary column
Harmonisation requires checking of
Internal consistency across diary & survey elements
Consistency of reporting across time

11. New mode surveys Many regional / small GPS, web, app surveys
Netherlands smartphone (HETUS in mind)
2011 UK GPS monitor (HETUS diary used)
Capture 24
Happiness.org time use surveys
MOTUS (in parallel to Belgian HETUS)

12. Mixed-mode national surveys
Danish (HETUS) – included paper & web options to complete some instruments
Canada 2015 time diary element of General Social Survey
UK Millennium Cohort Survey – 19,000 young people aged 14, smartphone app, web and paper light diaries

13. Millennium Cohort Survey Pilot Testing

14. Millennium Cohort Survey Pilot Testing

15. Mixed-mode national surveys
Danish (HETUS) – included paper & web options to complete some instruments
Canada 2015 time diary element of General Social Survey
UK Millennium Cohort Survey – 19,000 young people aged 14, smartphone app, web and paper light diaries

16. Measuring comparability
Engagement with each of the diary fields
Proportion with complete 24 hour account
Proportion empty, missing 91+ & 1-90 minutes
Episodes generated solely by that field
Episodes generated by that field (and others)
Total time in aggregated categories in each field
Participation in aggregated categories in each field

17. Measuring comparability
Check most frequent multi-tasked activity combinations (consistency of which is the main and which is the secondary activity)
Profile of respondents providing partial returns
Profile of respondents returning unusable diaries & refusing to participate

18. Pilot Group 7 Priorities
Experiments need to reflect decisions reached from other pilot groups
A significant amount of data presently available or which will become available later in minimally analysed or not yet analysed
Fund testing of existing data now to
Learn lessons from existing work
Learn what gaps require further testing