Transportation leadership you can trust. presented to Florida Model Task Force Model Advancement Committee presented by Robert G. Schiffer, AICP Thomas F. Rossi Cambridge Systematics, Inc. Yongqiang Wu, P.E. Florida Department of Transportation November 28, 2007 FSUTMS-Cube Calibration Standards Draft Guidelines and Standards

1 Presentation Overview Literature Review Default Guidelines and Standards (LRTP transit models) Checking Input Data Trip Generation Trip Distribution Mode Choice Trip Assignment Validation for Other Model Applications Discussion on Volume-over-Count Ratios Next Steps

2 Project Overview Follow-up to Phase I Study on model parameters Prepared summary of NHTS statistics for potential model use Identified adjustable parameters and potential sources for defaults Final report is available at the FSUTMSOnline web site − Cube_Parameters.pdf Phase II Study on calibration standards includes 4 subtasks Literature Review (complete) Model Calibration/Validation Guidelines and Standards (today) Best Practices for Model Calibration/Validation (next) Documentation – 1) Calibration and Validation Standards; 2) Best Practices for Model Validation; 3) Guidelines for Model Application

3 Project Overview (continued) Calibration vs. validation Calibration – process where models are adjusted to simulate or match observed travel behavior in the study area Validation – procedure used to adjust models to simulate base-year traffic counts and transit ridership figures Standards vs. guidelines/benchmarks Standards – desirable accuracy levels for comparing estimated versus observed metrics Benchmarks – documented statistical ranges from literature review, model outputs, NHTS, etc. Literature review 60+ documents reviewed – specific models and reference reports

4 Default Guidelines and Standards Checking Input Data Socioeconomic data Visual comparisons Statistical comparisons Regionwide comparisons (below) −Persons per DU (or HH) −Employment/ population ratio −Autos/DU (or HH)County Census Data NERPM Data Percent Difference Duval779,618817,4805%762,674810,4936% Clay141,671157,50211%139,036167,02020% Nassau57,90361,6256%56,89764,69514% St Johns 124,458142,86915%120,738150,08424% Total1,103,6501,179,4767%1,079,3451,192,29210% StatisticBenchmarks/SettingsLowHigh Regionwide Persons/DU (or HH) Regionwide Employment/Population Ratio Regionwide Autos/DU (or HH) Approximate Population/TAZ N/A3,000

5 Default Guidelines and Standards Checking Input Data (continued) Highway network data Transit network data Highway and transit speed data Hierarchy Balance highway and transit Terminal times Hierarchy Phase I Report

6 Default Guidelines and Standards Trip Generation Aggregate trip rates Person trips/TAZ Person trips/person Person trips/DU (or HH) HBW person trips/employee Total unbalanced attractions versus productions by purpose Preferred +/-10%; acceptable in some instances +/-50% Percent external-external trips by zone/station Great variation expected (4-21 percent range documented) Statistic Benchmarks a LowHigh Person Trips/TAZ N/A15,000 Person Trips/Person Person Trips/DU (or HH) HBW Person Trips/Employee a Generally excludes nonmotorized trips; including motorized trips could increase person trips per DU up to 11.5.

7 Default Guidelines and Standards Trip Generation (continued) Percent trips by purpose Statistic Benchmarks Low (Percent)High (Percent) Percent Trips by Purpose – HBW 1224 Percent Trips by Purpose – HBSH 1020 Percent Trips by Purpose – HBSR 912 Percent Trips by Purpose – HBSC 58 Percent Trips by Purpose – HBO a 1428 Percent Trips by Purpose – HBNW b 4560 Percent Trips by Purpose – NHB c 2033 a HBO includes a variety of special trip purposes depending on the model (e.g., airport, college, and shop). b HBNW accounts for all home-based trip purposes except HBW. c NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate.

8 Default Guidelines and Standards Trip Distribution Average trip length by purpose Trip length frequency distributions by purpose Coincidence ratios by purpose – measures the percent of area that coincides for two trip length frequencies Statistic Benchmarks LowHigh Average Trip Length – HBW (minutes)1235 Average Trip Length – HBSH (minutes)919 Average Trip Length – HBSR (minutes)1119 Average Trip Length – HBSC (minutes)716 Average Trip Length – HBO a (minutes)820 Average Trip Length – NHB b (minutes)619 Average Trip Length – IE (minutes)2658 StatisticStandards Mean Trip Length, Observed Total Trips+/-3% Trip Length Frequency Distribution versus Observed +/-5% Coincidence Ratios by Purpose c 70% a HBO includes a variety of special trip purposes, depending on the model (e.g., airport, college, and school). b NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate. c Some lower coincidence ratios have been deemed acceptable for trip purposes that had relatively few trips and therefore higher error rates. 8% 6% 4% 2% 0% Percent of Total Trips Travel Time (in Minutes) Coincidence Ratio = 0.82 Estimated (ATL = 18.2 Min) Observed (ATL = 18.9 Min)

9 Default Guidelines and Standards Trip Distribution (continued) Percent intrazonal trips by purpose Node-point charts Zone-based Number of trips Trip productions/attractions by purposeStatisticBenchmarksLowHigh Percent Intrazonal – HBW 1%4% Percent Intrazonal – HBSH 3%9% Percent Intrazonal – HBSR 4%10% Percent Intrazonal – HBSC 10%12% Percent Intrazonal – HBO a 3%7% Percent Intrazonal – NHB b 5%9% Percent Intrazonal – Total Trips 3%5% Standards StatisticAcceptablePreferable Percent Intrazonal, Observed Total Trips +/-3%+/-5% a HBO includes a variety of special trip purposes, depending on the model (e.g., airport, college, and school). b NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate.

10 Default Guidelines and Standards Mode Choice Mode split targets (ideal) Trip purpose Mode Auto ownership level Geographic subareaMode Zero-Vehicle Households One-Vehicle Households Two-Vehicle Households Three-Vehicle Households Walk5,0006,0004,0003,000 Bike2,0001, Drive Alone -130,000350,000200,000 Shared Ride 2 Persons 6,00015,00020,00010,000 Shared Ride 3 Persons 1,0002,0004,0002,000 Local Bus, Walk 6,0007,0004,0001,000 Local Bus, PNR -5002, Local Bus, KNR -200 Express Bus, Walk 1,0001,0001, Express Bus, PNR -2,0004,0002,000 Express Bus, KNR LRT, Walk 5001, LRT, PNR LRT, KNR

11 Default Guidelines and Standards Mode Choice (continued) Mode splits by observed calibration targets Total area transit trips, estimated versus observed Transit trips between districts Tabular comparisons (CTPP) Desire lines Mean trip length, estimated transit trips versus observed Statistic Standards LowHigh Total Area Transit Trips versus Observed+/-1%+/- 2% Transit Trips between DistrictsCompare model trip table against CTPP or HH survey Mean Trip Length Transit Trips versus Observed+/-5%+/-15% Mode Splits by Observed Calibration Targets+/- 2% Elasticity of Demand with Respect to LOS Variables

12 Default Guidelines and Standards Trip Assignment Volume-over-count ratios +/-1 lane percent error (recalculated per FDOT LOS Handbook) Aggregate VMT VMT/HH (60-75) VMT/person (24-32) VMT/commercial vehicle (3-25%) Statistic Standards AcceptablePreferable Freeway Volume-over-Count +/- 7%+/- 6% Arterial Volume-over-Count +/- 15%+/- 10% Collector Volume-over-Count +/- 25%+/- 20% Frontage Road Volume-over-Count +/- 25% Freeway Peak Volume-over-Count 75% of +/-20%; 50% of +/-10% Major Arterial Peak Volume-over-Count 75% of +/-30%; 50% of +/-15% Assigned VMT-over-Count Areawide +/-5%+/-2% Assigned VHT-over-Count Areawide +/-5%+/-2% Assigned VMT-over-Count by FT/AT/NL +/- 25%+/- 15% Assigned VHT-over-Count by FT/AT/NL +/- 25%+/- 15% Statistic Standards AcceptablePreferable Percent Error – LT 10,000 volume (2L road)50%25% Percent Error – 10,000-30,000 (4L road)30%20% Percent Error – 30,000-50,000 (6L road)25%15% Percent Error – 50,000-65,000 (4-6L freeway)20%10% Percent Error – 65,000-75,000 (6L freeway)15%5% Percent Error – GT 75,000 (8+L freeway)10%5%

13 Default Guidelines and Standards Trip Assignment (continued) Screenline volume-over-count RMSE by volume group Transit assignment validation Statistic Standards AcceptablePreferable RMSELT 5,000 AADT RMSE – LT 5,000 AADT150%45% RMSE5,000-9,999 AADT RMSE – 5,000-9,999 AADT45%35% RMSE10,000-14,999 AADT RMSE – 10,000-14,999 AADT35%27% RMSE15,000-19,999 AADT RMSE – 15,000-19,999 AADT35%25% RMSE20,000-29,999 AADT RMSE – 20,000-29,999 AADT27%15% RMSE30,000-49,999 AADT RMSE – 30,000-49,999 AADT25%15% RMSE50,000-59,999 AADT RMSE – 50,000-59,999 AADT20%10% RMSE60,000+ AADT RMSE – 60,000+ AADT19%10% RMSE Areawide 45%35% Statistic Benchmarks LowHigh Estimated-over-Observed Transit Trips+/- 9%+/- 3% Standards StatisticAcceptablePreferable Acceptable Error – Transit Screenlines+/-20%+/-10% Transit Ridership – <1,000 Passengers/Day+/-150%+/- 100% Transit Ridership – 1k-2k Passengers/Day+/- 100%+/- 65% Transit Ridership – 2k-5k Passengers/Day+/- 65%+/- 35% Transit Ridership – 5k-10k Passengers/Day+/- 35%+/- 25% Transit Ridership – 10k-20k Passengers/Day+/- 25%+/- 20% Transit Ridership – >20,000 Passengers/Day+/- 20%+/- 15%

14 Other Model Applications LRTP Highway Only Models Same default guidelines and standards except Replace mode choice checks with auto occupancy comparisons against NHTS and other surveys Commercial vehicle VMT checks not likely relevant No transit assignment validation Auto Occupancy Rates PurposeCurrentModel1988FLSWM2001 NHTS FL 2001 NHTS US HBW HBShop HBSR HBO NHB Statistic Benchmarks/Settings LowHigh Auto Occupancy Rates – HBW Auto Occupancy Rates – HBSH Auto Occupancy Rates – HBSR Auto Occupancy Rates – HBO a Auto Occupancy Rates – NHB b a HBO includes a variety of special trip purposes, depending on the model (e.g., airport, college, and school). b NHB includes combined purposes for NHB Work and NHB Non-Work, where appropriate.

15 Other Model Applications FTA New Starts Models Transit networks and pathbuilding checks Compare skim settings to on-board surveys Trip distribution checks Mode choice calibration Highway assignment checks Transit assignment checks Assign on-board survey trip table and compare ridership SUMMIT diagnostics Statistic Acceptable Range of Values LowHigh Elasticity of demand with respect to LOS variables IVT parameter – HBW* IVT parameter – HBNW* 0.1 to 0.5*CIVT HBW trips IVT parameter – NHB* ~CIVT HBW trips RatioOVT/IVT parameters – HBW* Ratio – OVT/IVT parameters – HBW* RatioOVT/IVT parameters – HBNW* Ratio – OVT/IVT parameters – HBNW* RatioOVT/IVT parameters – NHB* Ratio – OVT/IVT parameters – NHB* Implied value of time – Percent of income 25% 33% Implied value of time – HBW $2.00 $7.00 Implied value of time – HBNW $0.50 $5.00 Implied value of time – NHB $0.20 $5.00 * FTA published guideline.

16 Other Model Applications Subarea Models Prerequisite – approved regional model validation Input data – focus on socioeconomic and network data Trip generation – review and compare subarea versus regional model aggregate trip rates Trip distribution – compare subarea versus regional average trip length and percent intrazonal trips by purpose Mode choice – check subarea mode shares versus regional Trip assignment – volume-over-count (v-o-c), percent error, VMT and VHT v-o-c, v-o-c by screenline/cutline, and RMSE

17 Other Model Applications Corridor Models Same subarea model validation checks Input data – focus on network details surrounding corridor Trip generation – review corridor productions and attractions by zone Trip distribution – desire line analyses Mode choice – review of mode shares within study area Trip assignment – more stringent standards for v-o-c, v-o-c by screenline/cutlineStatisticStandardsAcceptablePreferable Freeway Volume-over-Count +/- 6% +/- 5% Arterial Volume-over-Count +/- 10% +/- 7% Collector Volume-over-Count +/- 15% +/- 10% Frontage Rd Volume-over-Count +/- 20% +/- 15%

18 Other Model Applications Models for DRIs and Other Impact Studies Input data SE data – site, nearby zone assumptions, pop/TAZ Networks – verify coding, path traces from site Transit – access coding, headways, stop locations near site Trip generation – document trip rate assumptions Trip distribution – district summaries Mode choice – check ITE trips versus model trips Trip assignment – select zone and select link, turn volumes

19 An Issue for the MTF to Discuss… Discussion on volume-over-count ratios Ratio of summed modeled volumes for group of links and the sum of count volumes on the links (should be near 1.0) This check does appear in the draft report at this time In the opinion of some, this is mathematically erroneous because of double counting It is somewhat duplicative of VMT checks

20 An Example 1 mile V = 7,500 C = 5,000 V = 5,000 C = 5,000 v/c = (7,500+7,500+5,000)/ (5,000+5,000+5,000) = 1.33

21 An Example (continued) 1 mile V = 5,000 C = 5,000 V = 7,500 C = 5,000 v/c = (5,000+5,000+7,500)/ (5,000+5,000+5,000) = 1.17

22 Proposed Solution Use VMT check Example has same solution for both cases VMT(m) / VMT(c) = 1.25 VMT is not double counted Screenline/cutline checks should be retained since double counting should not be an issue

23 Next Steps Take comments from MTF committee today Revise draft guidelines and standards based on FDOT and MTF committee comments Prepare technical report on best practices Develop guidelines for model application work