1. Source: NHI course on Travel Demand Forecasting (152054A) Session 11: Model Calibration, Validation, and Reasonableness Checks

2. Objectives: Identify and interpret trends affecting travel demand Explain difference between calibration and validation Identify critical reasonableness checks – socioeconomic – travel survey – network – trip generation – mode split – trip assignment

3. Terminology Model Calibration Model Validation – Reasonableness checks – Sensitivity checks Special generators Screen lines (some modelers do not think this is important) Is the model sensitive to policy options?

4. Not enough attention on model evaluation and reasonableness checks Checks performed after each step – reduces error propagation Errors can also “cancel” Key Concepts

6. Planner responsibilities Actively involve all participants – Modelers – Planners – Decision makers – Public Fairly present all alternatives – Timely – Unbiased Identify (clearly) the decision making process – Who, when, and how – Allows input from all interested groups You must rely on the TDM – Therefore, must be validated – Accurate and easy to understand (documented)

7. Planners should monitor the following trends: – Demographics – Composition of the labor force – Immigration and emigration – Regional economic development – Modal shares – Vehicle occupancy – Average trip length – Freight transport Are trends consistent with assumptions made in the modeling process? Must be aware of trends to ensure reasonable forecasts Trends Affecting Travel Demand

8. Information Requirements for Validation and Reasonableness Demographics and employment Highway and transit networks Model specification Base year survey Base year traffic counts

9. Sources of Error Coding Sampling Computation (if done by hand) Specification Data Transfer Data aggregation Improper structure of model, e.g., wrong variables

10. Scrutinize these characteristics: Data requirements Hardware requirements Logic of structure and conceptual appeal Ease of calibration Effectiveness of the model (accuracy, sensitivity) Flexibility in application Types of available outputs Operational costs Experience and successes to date Public or private domain availability Compatibility with other models and model types How do you judge a model/recommend improvement?

11. Reasonable? Methodology? Source? Current? Reasonable? Complete? Level of Detail? Sensitive? Documentation of calibration? Valid for base year? Evaluation and Reasonableness Checks Overview Transportation system (supply) ► Network Data Number and location of households and employment (demand) ► Socioeconomic Data TDF ► Model Specification ► Model validation and calibration Travel survey data Transportation system performance

12. Model Calibration and Validation Model Calibration Model Validation Model Application Feedback Loop

13. “Transportation Conformity Guidelines” (Air Quality) require model validated < 10 years ago Model Validation Validation of new model – Model applied to complete model chain – Base year model compared to observed travel – Judgment as to model suitability, return to calibration if not Validation of a previously calibrated model – Compare to a new base year, with new … SE data Special gen. Network Counts

14. Socioeconomic Data: Check Reasonableness reviewSource for estimates and forecasts Population and household size trends (graph 1950 to present and check trend) Household income trends (graph as far back as this goes … 1990?) Check dollar values used in forecast (use constant dollars) If used, check trend of automotive availability (S curve?) Check distribution of employment by type (basic, retail, service) over time Plot and check trend of employees per household and per capita … rate of increase is decreasing Check future household and employment changes by zone

15. Travel not sensitive to fuel price? http://www.eia.doe.gov/oiaf/aeo/pdf/trend_4.pdf

16. Travel Survey Data Reasonableness Checks Determine source of travel survey data – Types of survey conducted – Year of survey If no survey (borrowed) – Check source of trip rates, lengths, TLFD – Is area similar Geographic area? pop/HH/empl. characteristics? Urban density and trans system? Compare to similar regions and to same region in earlier times: – Person trip rates by trip purpose – Mean trip lengths by trip purpose HBW longest? HBO shortest? – TLFDs by trip purpose

17. Network Data Reasonableness Checks Check Trees for 2-3 major attractions Check coded facility types – how used (BPR?)? Verify speed and capacity look-up table (what LOS used for capacity?) Significant transportation projects – narrative included? Still viable? Consistency with MTP Plot (facility types, # lanes, speeds, area types) to detect coding errors

18. Trip Generation Reasonableness Checks Examine trip production and attraction models – Form? – sensitivity? Examine trip purposes used External-through and external-local trips – how modeled? Truck trips – how modeled? Person trip or vehicle trip rates used? P&A balance (0.9-1.1 ok) Special generators (check, and be consistent in future model)

19. Trip Generation Calibration Typical Values Person trips per household: 8.5 to 10.5 HBW person trips per household: 1.7 to 2.3 HBO person trips per household: 3.5 to 4.8 NHB person trips per household: 1.7 to 2.9 HBW trips: 18% to 27% of all trips HBO trips: 47% to 54% of all trips NHB trips: 22% to 31% of all trips

20. Scale survey for participation (relative participation) Note: each income class is a purpose! TRIP PURPOSES Scaling Factor HBW low income0.795 HBW low-middle income0.823 HBW middle income0.861 HBW upper middle income0.908 HBW high income0.936 HB elementary school0.733 HB high school1.991 HB university0.895 HB shopping0.698 HB social-recreation0.945 HB other0.875 NHB work-related0.858 NHB other0.820 Truck0.985 Internal-external0.591 Trip Generation Calibration Colorado Springs 1996 Travel Demand Model Calibration

21. Trip Generation Calibration Reasonableness checks – compare to other cities, check future trends Population503,345 Households201,116 Average Household Size2.50 Basic employment76,795 (33%) Retail employment50,465 (24%) Service employment101,697 (43%) Military employment42,800 Population per employee1.81 Person trips per person4.26 Person trips per household10.65 HBW attractions per employee1.44 HBW productions per household 1.74 HB shopping attractions per retail employee 5.99 Colorado Springs 1996 Travel Demand Model Calibration

22. Trip Distribution Reasonableness Checks Examine … Mean trip length (increasing or decreasing?) TLFDs Treatment of friction factors (same?) Treatment of terminal times (logic?) Treatment of K factors Comparison with JTW trip length Comparison with JTW sector interchange volumes or percentages.

23. Calibrate friction factors 1 st iteration

24. Travel Times Ranges from Skims Observed Trip Expanded from Surveys Input Friction Factors Gravity Model Trips Adjustment Factor Observed Gravity Model New Friction Factors Friction Adjustment Factor x Friction Factor 2.57,10030.08,2000.8725.98 5.014,9502.5016,3000.922.29 7.517,8501.8019,2500.931.67 10.016,0001.5019,1000.841.26 12.515,5001.2017,1000.911.09 15.015,9001.0012,3001.291.29 17.516,4000.9518,0000.910.87 20.015,1500.9014,3001.060.95 22.513,5000.8511,9001.130.96 25.011,0000.809,2501.190.95 27.59,5000.758,1001.170.88 30.09,1000.706,1001.491.04 32.55,7000.654,9001.160.76 ……………… Calibrating a Gravity Model Adjusting Friction Factors

25. 2 nd iteration

26. Commute Length in Minutes PercentJourney-to-WorkFlowsPercent < 15 27.87Central-CentralCounty31.49 15-2941.63Central-SuburbanCounty7.48 30-3917.04 Suburban-Central County 15.13 40-597.70 Within Suburban County 32.98 >603.00 To Other Suburban County 10.81 Mean 21.44 Work out of area 2.11 Trip Distribution Calibration and Validation Check modeled vs. household survey TLFD and mean trip lengths Get HBW area-to-area flows from JTW HBW 1990 JTW TLFD and Area-to-Area Flows for Kansas City

28. Mode Split Reasonableness Checks Automobile occupancy factors by trip purpose used? Basis? Constant? Mode split model? Form? Variables included in the utility functions? Coefficients logical? Value of time assumptions Parking cost assumptions How do mode shares change over time? Mode share comparisons with other cities

29. Experienced planning consultant required … – Form of LOGIT model – Variables included in utility functions – Calibration of coefficients for utility function variables – Testing for IIA properties – Analysis of household survey data – Analysis of on-board transit survey data Calibration tasks we can do: Compare highway and transit trips Total By purpose Compare Ridership by route CBD cordon line survey (if bus service is downtown only) Mode Split Calibration and Validation

30. All-or-nothing assignment study effect of increasing capacity Compare to Equilibrium assignment Check volume delay equation (BPR parameters) Compare screen line volumes Cut line volumes Time-of-day assignments? Source of factors Peak spreading used for future? If not, conversion factors source? (peak hour to 24-hour) Local VMT (% assigned to intrazonals and centroid connectors All or Nothing Equil ibrium Trip Assignment Reasonableness Checks

31. Assignment calibration performed last Trip Assignment Calibration and Validation Overall VMT or VHT check 40 to 60 miles per day per HH in large metro areas 30 to 40 miles per day per HH in medium metro +/- 10% OK on screen lines Sign is important

32. Compute by … - volume group - facility type - transit assignments - time of day

34. Other Factors Impacting Forecasted Travel Demand (use your noodle) Can be implied in travel surveys (but not explicit) – Telecommuting – Flexible work hours – HB business How to account for … – Aging population – Internet shopping – Roadway congestion (will it affect generation in the future) – New modes