Design and Assessment of the Toronto Area Computerized Household Activity Scheduling Survey Sean T. Doherty, Erika Nemeth, Matthew Roorda, Eric J. Miller MCRI Caucus Meeting University of Toronto September 13-14, 2003

INTRODUCTION Traditional activity/travel diary surveys have for some time served as the primary source of data for understanding and modeling travel behavior Traditional activity/travel diary surveys have for some time served as the primary source of data for understanding and modeling travel behavior There are no generally accepted standards for transport surveys There are no generally accepted standards for transport surveys Statistics Canada “Quality Guidelines” Statistics Canada “Quality Guidelines” The following specific “checks” of survey data are suggested: The following specific “checks” of survey data are suggested: consistency with external sources of data; consistency with external sources of data; internal consistency checks, e.g., calculation of ratios that are known to lie within certain bounds; internal consistency checks, e.g., calculation of ratios that are known to lie within certain bounds; unit-by-unit reviews of the largest contributors to aggregate estimates; unit-by-unit reviews of the largest contributors to aggregate estimates; calculation of data quality indicators such as non-response rates and imputation rates; calculation of data quality indicators such as non-response rates and imputation rates; debriefings with staff involved in the collection and processing of the data; debriefings with staff involved in the collection and processing of the data; “reasonableness” checks by knowledgeable subject matter experts; “reasonableness” checks by knowledgeable subject matter experts; Further suggestions to assess potential sources of error include detection of non-response, coverage, measurement, processing and sampling errors. Further suggestions to assess potential sources of error include detection of non-response, coverage, measurement, processing and sampling errors.

Assessing the Quality of Travel Surveys Unit non-response rates Unit non-response rates Activity and Trip rates per day Activity and Trip rates per day Less commonly reported assessors Less commonly reported assessors - number of missing values - number of cleaning steps - inconsistent unrealistic values

Characteristics of the Sample 271 households, total of 452 individuals (an additional 30 individuals filtered out due to high incompletion rates). 271 households, total of 452 individuals (an additional 30 individuals filtered out due to high incompletion rates). Males represented 53% of the sample, females 47%. Males represented 53% of the sample, females 47%. The Average income was $46,000. The Average income was $46,000. The proportion of individuals: The proportion of individuals: involved in partnership 65.3%, involved in partnership 65.3%, single adults 21.2%, single adults 21.2%, other adult living in a household 8.4%, other adult living in a household 8.4%, teens 3.3%, and teens 3.3%, and unknown 1.8%. unknown 1.8%.

Schedule completion rates and login durations 453 individual respondents provided complete 7-day schedules accounting for every hour of every day 453 individual respondents provided complete 7-day schedules accounting for every hour of every day Login durations calculated based on examination of the time-stamp of every entry in the program Login durations calculated based on examination of the time-stamp of every entry in the program The average login duration was 139 minutes per week, with a standard deviation of The average login duration was 139 minutes per week, with a standard deviation of 70.0.

Total Login Duration

Activity, trip and scheduling step rates Overall, respondents entered an average of Overall, respondents entered an average of 12.0 additions (s.d. 4.57), 12.0 additions (s.d. 4.57), 1.47 modifications (s.d. 1.97) and 1.47 modifications (s.d. 1.97) and 0.25 deletion (s.d. 0.80) scheduling decisions per day, 0.25 deletion (s.d. 0.80) scheduling decisions per day, average of 11.8 observed activities (s.d. 4.4) and 3.74 one-way trips (s.d. 2.5) per day. average of 11.8 observed activities (s.d. 4.4) and 3.74 one-way trips (s.d. 2.5) per day.

Scheduling decisions rates (additions, modifications and deletions) per person per entry day

Planning Time Horizons Valuable to examine the timing of when people entered their decisions into the program Valuable to examine the timing of when people entered their decisions into the program Overall, 31.2 % of the 35,137 addition decisions were entered before the activity occurred, 24.6% on the same day, and 44.2% after the activity took place. Overall, 31.2 % of the 35,137 addition decisions were entered before the activity occurred, 24.6% on the same day, and 44.2% after the activity took place. What is of most concern is the proportion of activities entered after-the-fact that were reported to have been planned ahead by a day or more. What is of most concern is the proportion of activities entered after-the-fact that were reported to have been planned ahead by a day or more.

Activity addition decisions entered the “same day” as the activity, before- the-fact, and after-the-fact, by when they reported to have been planned

Conclusions The overriding objectives were to attempt to propose and quantitatively analyze potential data quality measures that could be used as a basis to assess an emerging class of activity scheduling process surveys. This is in part due to the lack of such guidelines in the literature The overriding objectives were to attempt to propose and quantitatively analyze potential data quality measures that could be used as a basis to assess an emerging class of activity scheduling process surveys. This is in part due to the lack of such guidelines in the literature Thus the results presented in this papers support the following data quality guidelines and suggestions for activity scheduling decision process surveys: Thus the results presented in this papers support the following data quality guidelines and suggestions for activity scheduling decision process surveys:

Key Guidelines Activity/trip rates: Arguably, activity rates can be artificially inflated simply by asking for more detailed activities to be reported. Thus, using activity/trip rates to assess activity scheduling surveys is not encouraged, at least not in isolation. Activity/trip rates: Arguably, activity rates can be artificially inflated simply by asking for more detailed activities to be reported. Thus, using activity/trip rates to assess activity scheduling surveys is not encouraged, at least not in isolation. Scheduling steps: It would appear more appropriate to place more weight on the rates of decisions captured in assessing data quality. Our suggestions at present would be that a good quality scheduling survey be able to capture a minimum of 2-3 modifications/deletions per day. Scheduling steps: It would appear more appropriate to place more weight on the rates of decisions captured in assessing data quality. Our suggestions at present would be that a good quality scheduling survey be able to capture a minimum of 2-3 modifications/deletions per day. Planning time horizons: A good quality scheduling survey should provide ample opportunity and encourage entry of scheduling decisions as they occur – in operational terms, Results of this survey suggest that over 80% of preplanned decisions should indeed be entered before the associated event occurs. Planning time horizons: A good quality scheduling survey should provide ample opportunity and encourage entry of scheduling decisions as they occur – in operational terms, Results of this survey suggest that over 80% of preplanned decisions should indeed be entered before the associated event occurs.

Login Durations: On average, people spent about 20 minutes per scheduling day completing the survey. Our suggested guideline concerning login durations is simple – reduce it as much as possible. Login Durations: On average, people spent about 20 minutes per scheduling day completing the survey. Our suggested guideline concerning login durations is simple – reduce it as much as possible. The goal should be to tighten the distribution of login durations, and shift them to the left of 20 minutes per day. Several suggestions for doing so, and reducing burden generally, include: Several suggestions for doing so, and reducing burden generally, include: choose “target” days well enough in the future for which people schedule, leaving intervening days blank choose “target” days well enough in the future for which people schedule, leaving intervening days blank provide a means for subjects to login to the survey more frequently, thereby spreading the burden out over the course of a day (e.g. using hand-held computers rather than laptops). provide a means for subjects to login to the survey more frequently, thereby spreading the burden out over the course of a day (e.g. using hand-held computers rather than laptops). make the survey questions less repetitive and time consuming by choosing only certain days to ask them, or implementing more random or periodic prompting make the survey questions less repetitive and time consuming by choosing only certain days to ask them, or implementing more random or periodic prompting utilize other technologies to passively trace certain aspects of people schedules, especially the more repetitive ones (e.g. through using Global Position Systems to trace activity locations, travel routes, and activity start/end times). utilize other technologies to passively trace certain aspects of people schedules, especially the more repetitive ones (e.g. through using Global Position Systems to trace activity locations, travel routes, and activity start/end times). higher incentives would not appear to improve matters higher incentives would not appear to improve matters

Despite the shortcomings and biases found in the survey, those that did start the survey, tended to provide complete 7-days worth of scheduling information, involving a substantial time commitment. Despite the shortcomings and biases found in the survey, those that did start the survey, tended to provide complete 7-days worth of scheduling information, involving a substantial time commitment. It is no small success that the survey was able to capture over 35,000 scheduling decisions from a fairly representative sample of 422 individuals – information that has been repeatedly called for in the literature, but has simply not been available from any other source in such a quantitative fashion. It is no small success that the survey was able to capture over 35,000 scheduling decisions from a fairly representative sample of 422 individuals – information that has been repeatedly called for in the literature, but has simply not been available from any other source in such a quantitative fashion.

THANK YOU