1. HETUS Pilot Group 7
Designing experiments for calibration: cross-mode, cross-country, and cross-time comparability
Kimberly Fisher, Centre for Time Use Research – co-ordinator
Jonathan Gershuny, Ignace Glorieux, Joeri Minnen, Hannu Paäkkönen, Vesna Pantelic, Annemarie Wennekers
12 January 2016

2. Comparability in time use surveys
2 of many key policy values from time use surveys:
Tracking changes in behaviour across time
Comparing time use across countries
Apparent differences in time use can arise from:
Varying data processing across time & countries
Improvements have comparability consequences
Variations in survey directions & introductions
New modes of data collection

3. Comparability in time use surveys
Key value and flexibility of the HETUS design arises from own-words reporting of activities and context information
The way people think about and report their time use subtly changes over time
Cultural and social change
Response to new technologies generating new methods of accomplishing tasks

4. Survey Opportunities Range of comparability concerns increasing
Most recent research underlying resilience and robustness of the time diary method – GPS maps, accelerometers, and camera images enhancing accounts, adding details and improving accuracy of reporting, but also revealing the same overall picture captured by the traditional paper diary

7. Gold standard aspiration
Distinguish changes in time use that reflect variations in behaviour between countries or across time from:
Changes in the way people think about and report their time use
Variations in the way national statistical institutes collect and process time use survey data
Different modes of time use survey collection

8. Topics covered by Group 7
Knowledge on which we can build
New mode tests and mixed-mode surveys
Measuring comparability
Work to be done with recently collected data
Experiments need to reflect decisions to be made by other groups

9. Knowledge on which to build
HETUS experience
Tables for Eurostat
Statistics Sweden table generator; Statistics Finland next generation analytic tool
Multinational Time Use Study
TUS-X element of IPUMS Time Use
National survey sequences (Belgium (Flanders region), the Netherlands, Norway)

10. Lessons from harmonisation
Survey metadata matters
Survey introductions, instructions, layout produce subtle variations in data
Documentation & data require harmonisation
Diaries collect narrative accounts; can create harmonised file columns from more than one original diary column
Harmonisation requires checking of
Internal consistency across diary & survey elements
Consistency of reporting across time

11. New mode surveys Many regional / small GPS, web, app surveys
Netherlands smartphone (HETUS in mind)
2011 UK GPS monitor (HETUS diary used)
Danish (HETUS) – included paper & web options to complete some instruments
MO-TUS 2014 (in parallel to 2013 Belgian HETUS)

12. Mixed-mode national surveys
UK Millennium Cohort Survey – 19,000 young people aged 14, smartphone app, web and paper light diaries
Happiness Research Organisation
Canada 2015 time diary element of General Social Survey
Capture 24

13. Millennium Cohort Survey Pilot Testing

14. Millennium Cohort Survey Pilot Testing

15. Mixed-mode early lessons
Comparable levels of support to diarists completing each mode of diaries
Best to allow participants multiple paths to record new episodes
App modes collect most passive detail, but diary narratives sufficiently flexible for multiple modes to produce similar pictures of behaviour
Methodological work to promote comparability key

16. Measuring comparability
Any spike or pronounced drop not directly associated with externally verifiable information suspicious
Need to consider whole survey, as well as men separately from women for all measures during testing at a minimum

17. Engagement with survey overall
Time required to complete each instrument
Response rate by instrument / section
Basic demographic profile of respondents completing each instrument by each mode / survey year

18. Engagement with diary How often entries made in diaries
When diaries completed relative to observed day
Proportion completing
Two good quality diaries
One good quality diary
Only insufficient diaries
No diary

19. Engagement with diary Engagement with each of the diary fields
Proportion with complete 24 hour account
Proportion empty, missing 1-90 & 91+ minutes
Total missing time
Episodes generated solely by that field
Episodes generated by that field (only + with others)
Main activity reporting
Number of distinct main activities
Presence of basic domains
Mean time in 2-digit activity codes

20. Engagement with diary Secondary activity reporting Location reporting
Number of distinct secondary activities
Proportion of diaries with some secondary activity
Mean secondary activity time
Location reporting
Number of distinct locations reported
Mean time at home

21. Engagement with diary Who else was present reporting
Proportion with some time alone & some with others
Proportion with time alone only; time with others only
Proportion with time alone with others reporting
Enjoyment / happiness reporting
Proportion with the lowest and highest ratings
Median range of emotions reported
Proportion with a single emotion report for whole day

22. Promoting comparability
Retain capacity to recode to current HETUS standards
Produce guidance on constructing calibration tables – consider creating some calibration estimates for use of smart technologies?
Detailed documentation essential

23. Pilot Group 7 Priorities
Experiments need to reflect decisions reached from other pilot groups
A significant amount of data presently available or which will become available later in minimally analysed or not yet analysed
Fund testing of existing data now to
Learn lessons from existing work
Learn what gaps require further testing