Session 11: Model Calibration, Validation, and Reasonableness Checks Source: NHI course on Travel Demand Forecasting (152054A)

Terminology Model Calibration Model Validation Special generators Reasonableness checks Sensitivity checks Special generators Screen lines (some modelers do not think this is important) Is the model sensitive to policy options?

Key Concepts Not enough attention on model evaluation and reasonableness checks Checks performed after each step reduces error propagation

Information Requirements for Validation and Reasonableness Demographics and employment Highway and transit networks Base year survey Base year traffic counts

Sources of Error Network coding Sampling Computation Specification Data transfer Data aggregation

Documentation of calibration? Inputs and Outputs Number and location of households and employment (demand) Socioeconomic Data Complete? Level of Detail? Transportation system (supply) Network Data Reasonable? Methodology? Source? TDF Model Specification Model validation and calibration Travel survey data Sensitive? Documentation of calibration? Valid for base year? Transportation system performance Current? Reasonable?

Model Calibration and Validation Independent Data Sets Feedback Loop Model Validation Model Application

Socioeconomic Data: Check Reasonableness of … Source for estimates and forecasts Population and household size trends (graph 1950 to present and check trend) Household income trends (graph as far back as this goes … 1990?) Dollar values used in forecast (use constant dollars) If used, trend of automotive availability (S curve?) Distribution of employment by type (basic, retail, service) over time Employees per household and per capita … rate of increase is decreasing Household and employment changes by zone Area type (density) changes – are the growth areas reasonable?

Travel Survey Data Reasonableness Checks Types of survey conducted Year of survey If no survey (borrowed) Source of trip rates, lengths, TLFD Is area similar Geographic area? pop/HH/emp characteristics? Urban density and trans system? Trip chaining convention Compare to similar regions and to same region in earlier times: Person trip rates by trip purpose Mean trip lengths by trip purpose HBW longest? HBO shortest? TLFDs by trip purpose

Network Data Reasonableness Checks Facility types, level of detail Speed and capacity look-up table (what LOS used for capacity?) Significant transportation projects – narrative? Plot (facility types, #lanes, speeds, area types)

Trip Generation Reasonableness Checks Trip production and attraction models Form? sensitivity? Trip purposes used Person trip or vehicle trip rates used? P&A balance (0.9-1.1 ok) Special generators (be consistent in future model)

Trip Generation Calibration Typical Values Person trips per household: 8.5 to 10.5 HBW person trips per household: 1.7 to 2.3 HBO person trips per household: 3.5 to 4.8 NHB person trips per household: 1.7 to 2.9 HBW trips: 18% to 27% of all trips HBO trips: 47% to 54% of all trips NHB trips: 22% to 31% of all trips

Trip Distribution Reasonableness Checks Mean trip length (increasing or decreasing?) TLFDs

1st iteration Calibrate friction factors

Mode Split Reasonableness Checks Automobile occupancy factors by trip purpose Basis? Constant? Form of mode split model? Variables included in the utility functions? Coefficients logical? IIA properties of LOGIT models Mode share changes over time Mode share comparisons with other cities

Mode Split Calibration and Validation Advanced concepts needed to formulate and calibrate mode split models Experienced planning consultant Calibration tasks: Form of LOGIT model Variables included in utility functions Calibration of coefficients for utility function variables Testing for IIA properties Analysis of household survey data Analysis of on-board transit survey data Total highway and transit trips Ridership by route

Trip Assignment Reasonableness Checks Compare to see where adding capacity may help All-or-nothing assignment Equilibrium assignment Volume delay equation (BPR) Screen line volumes Time-of-day assignments Local VMT Equil ibrium All or Nothing

Trip Assignment Calibration and Validation Assignment calibration performed last Overall VMT or VHT check 40 to 60 miles per day per HH in large metro areas 30 to 40 miles per day per HH in medium metro

Other Factors Impacting Forecasted Travel Demand Telecommuting Flexible work hours HB business Aging population Internet shopping Roadway congestion New modes