1. presented to presented by Cambridge Systematics, Inc. Comprehensive Validation of Activity-Based Models Experiences from Houston-Galveston Area Council’s Activity-Based Model Development TRB Applications Conference May 21, 2015 Martin Milkovits Arun Kuppam David Kurth Thomas Rossi

2. Tour-Level Choices Long-Term Choices Stop/Trip-Level Choices H-GAC ABM Structure 2 All Tour Stop Generation and Mode Choice Tour Generation Mandatory Tour Destination and Time of Day Auto Ownership, Work Location, School Location Daily Activity Pattern (Including Work/School Travel) Fully Joint Travel Stop (Trip)-Level Destination, Time of Day, and Mode Choice Individual Nonmandatory Travel School Escorting Model Joint Tour Destination and Time of Day Individual Nonmandatory Tour Destination and Time of Day

3. Validation Approach 3 Identify key areas of future analysis Compile validation data Assure correct model operation Prevalidation Population synthesis validation Single-pass validation (TourCast) Full-feedback validation Sensitivity test Elasticity test Backcast test Validation Tests

4. Lessons Learned Don’t have eyes bigger than your stomach (for cross classifications) » The greater the stratification The less available/statistically significant observed data The more difficult to effectively calibrate » Avoid information overload Threshold screens Identify trends using heat-maps GIS representations » Know when to trust the model Particularly with ordinal data (age, income, etc.) 4

5. Lessons Learned ( continued) Need consistent expansion factors between synthesized population and validation survey summaries 5 Synthesized Population Expanded Survey (Prenormalized)

6. Step-wise/iterative process is most robust » Single-pass before full-feedback » Logsums create circular dependencies within single-pass » Iterative adjustments to accommodate multiple stratifications » Time-intensive process, particularly for models with many segments and/or alternatives Sensitivity Test to check and demonstrate ABM advantages » Peak period tolling » Changing demographic 6 Lessons Learned ( continued)

7. Acknowledgements 7 Chris van Slyke, Chi-Ping Lam, Sharon Ju, Heng Wang, David Gao, Michael Onuogu H-GAC Vincent Sanders Houston Metro Andy Mullins Texas Transportation Institute Pat Coleman, Srikanth Neelisetty, Nagaruju Kashayi AECOM Vijay Mahal HDR

8. Vehicle Availability Stratified by County HH size HH income HH size/income Workers Workers/income Children Workers/children Accessibility 8 Expanded Household Survey Model Percentage Difference (model – survey)/survey HHIncome HHSize Total<$20000 20,000- $39,999 $40,000- $69,999 $70,000- $99,999 >$100,000 Total2.051.351.882.372.522.64 11.100.881.191.291.431.48 22.051.591.932.162.332.31 32.571.862.292.823.003.01 4+2.541.792.372.80 2.92 HHSize HHIncome HHSize Total<$20000 20,000- $39,999 $40,000- $69,999 $70,000- $99,999 >$100,000 Total2.051.201.732.152.402.59 11.130.851.131.331.391.47 22.121.511.882.152.272.39 32.231.431.882.302.472.65 4+2.591.612.152.662.853.00 HHSize HHIncome HHSize Total<$20000 20,000- $39,999 $40,000- $69,999 $70,000- $99,999 >$100,000 Total0%-15% -22%-12%-5% 13%-3%-6%4%-3%-1% 27%-8%-5%-1%-6%8% 3-34%-43%-41%-52%-53%-35% 4+5%-18%-22%-15%5%8% HHSize

9. Aggregate-Level Results Initial assignment results comparable (or better!) than current trip-based model after substantial calibration demonstrates value of comprehensive single-pass validation 9 TDMABM Percent DifferenceRMSEPercent DifferenceRMSE Freeway-10%28%-9%27% Toll-14%31%-11%30% Principal Arterial 3%65%3%64% Other Arterial 6%79%6%80% Collector11%121%11%121% All1%71%2%71%