presented to GIS Committee, Florida Model Task Force presented by Albert Gan Associate Professor Lehman Center for Transportation Research Florida International University May 14, 2010 FSUTMS Data Framework

Problems with FSUTMS Input Data Input data are stored in individual files of different file formats DBF, CSV, PEN, DAT, LIN, NET, MAT, etc. Text files do not have variable names Variable names are inconsistent among different models Files are not saved in standard file folders Consequence: Data are more difficult to maintain and understand Consequence: Tools cannot be developed without customizing for individual models

Project Objective Design a data framework for FSUTMS that facilitates the maintenance of model input data allows tools to be developed and applied across all models with minimum or no customization

Three Basic Design Approaches Use of geodatabase Use of “master” geodatabase Include all variables used by all FSUTMS models

Use of Geodatabase Store all files as a set of tables in a personal geodatabase file GIS-TM uses this approach more than 10 years ago 2 GB size limit of Personal Geodatabase is not a problem Geodatabase for all base-year data for SERPM is only 65 MB, or about 3% of the 2 GB limit

Use of Master Database Approach Traditional Approach 2000 Master Database Scenario Extraction Scenario Merging 2010 Forward or Backward Propagation

Include all Variables for all FSUTMS Models Each model has all of its variables included in the data framework There is NOT a standard set of data variables in the data framework Local models will not be affected as a result of transitioning to the data framework However, each model will have to do the following: modify to standard table (was file) names, variable names, and file folders modify input scripts to read from a standard geodatabase instead of individual files

FSUTMS Models Considered Florida Statewide Model (FLSWM) Northeast Regional Planning Model (NERPM) Tampa Bay Regional Planning Model (TBRPM) Southeast Regional Planning Model (SERPM) Treasure Coast Regional Planning Model (TCRPM) Greater Treasure Coast Regional Planning Model (GTCRPM) Central Florida Regional Planning Model (CFRPM) Northwest Florida Regional Planning Model (NWFRPM) Gainesville Urbanized Area Transportation Model (GUATS) Polk TPO Model (POLK)

Project Tasks Developing standards for folder names and structure, database names, table names, field names, and field types for all input data as well as setting standards for metadata files. Designing a master database that is inclusive of all standard input data tables and fields of all modes from all scenarios of all local models. Developing a master database system that implements the master database approach. Developing a procedure for converting existing input files into the proposed master database.

Inconsistent Variable Names: An Example FLSWMNERPMTBRPMSERPMTCRPMGTCRPMCFRPMNWFRPMGUATSPOLK ZZZONETAZZZ ZZZ HBWPHBWKPHBWPP1 PHBWP1HBWP HBWAHBWKAHBWAA1 AHBWA1HBWA HBSHP P2 PHBSHP2HBSHP HBSHA A2 AHBSHA2HBSHA HBSRP P3 PHBSRP3HBSRP HBSRA A3 AHBSRA3HBSRA HBOPHBOTPHBOPP6P5 PHBOP4HBOP HBOAHBOTAHBOAA6A5 AHBOA4HBOA NHBPNHBSPPNHBP5NHBP NHBANHBSAANHBA5NHBA NHBWPP7P6 NHBWAA7A6 NHBOPP8P7 NHBOAA8A7 HBUP

Variable Naming Conventions 10 characters maximum Use meaningful names If a variable name contains multiple words, use underscores only if it helps to avoid confusion If a variable is used by multiple models: choose one that is most commonly used subject to the above criteria If a variable is used by only a specific model: use the original name unless they do not meet the general criteria above Reference document: FSUTMS Powered by Cube/Voyager Data Dictionary, December 2005FSUTMS Powered by Cube/Voyager Data Dictionary

Applying the Naming Conventions FLSWMNERPMTBRPMSERPMTCRPMGTCRPMCFRPMNWFRPMGUATSPOLKStandard ZZZONETAZZZ ZZZ HBWPHBWKPHBWPP1 PHBWP1HBWP HBWAHBWKAHBWAA1 AHBWA1HBWA HBSHP P2 PHBSHP2HBSHP HBSHA A2 AHBSHA2HBSHA HBSRP P3 PHBSRP3HBSRP HBSRA A3 AHBSRA3HBSRA HBOPHBOTPHBOPP6P5 PHBOP4HBOP HBOAHBOTAHBOAA6A5 AHBOA4HBOA NHBPNHBSP PNHBP5NHBP NHBANHBSA ANHBA5NHBA NHBWPP7P6 NHBWP NHBWAA7A6 NHBWA NHBOPP8P7 NHBOP NHBOAA8A7 NHBOA HBUP

Table Name Conventions Maintain most traditional names SPDCAP, VFACTORS, FF, PANDA, etc. No need to attach model year and model alternative; they are part of the scenario database file name Transportation-network related tables HwyNodes, HwyLinks, PtNodes, PtLinks, PtLines Non-transportation network related tables See next slide

Non-Transportation Network Related Tables ZONEDATA ZONEDATA_LIFESTYLE SPECGEN EETRIPS INT_EXT PRODRATES ATTRRATES DUWEIGHTS PANDA SPDCAP TOLLLINKS TURN_PENALTIES VFACTORS FF TRANSIT_MODES TRANSFER_FARES TRANSIT_OPERATORS WAIT_CURVES TRANSIT_SPEED

Master Geodatabase File Contains all tables for all models, all networks, all scenarios

Scenario Geodatabase Contains only applicable tables for specific model, specific model year and model alternative

Metadata Standards Not metadata files currently used by any models FGDC Plus Stylesheet (Federal Geographic Data Committee) Template: Edit in Word: C:\FSUTMS\D4\SERPM\MASTER\SERPM_MASTER.DOC C:\FSUTMS\D4\SERPM\MASTER\SERPM_MASTER.DOC

Source: FSUTMS New Standards and Enhancements – A User Oriented Approach, January 2006FSUTMS New Standards and Enhancements – A User Oriented Approach File Folder Structure and Naming Conventions

Master Database System

Scenario Extraction

Scenario Merging: saving as a new scenario

Scenario Merging: update original scenario

Data Conversion Procedure (from existing files to master geodatabase) Step 1: Create a geodatabase for the base-year data Create an empty Cube geodatabase in Cube Import highway network Import public transit network Import TAZ feature class (optional) Step 2: Create a master geodatabase Create an empty geodatabase in ArcCatalog Import feature classes from Cube geodatabase into master geodatabase Import other non-transportation network related data

Data Conversion Procedure (continued) Step 3: Update master geodatabase in Microsoft Access Rename variables to standard names Add master database system tables (BaseYearNumber, ProjectInfo, ImpExpInfo) Add LineID field to PTLines feature class Add master geodatabase key fields (MNet_Year, MNet_Status, and MNet_ProjectID) to feature class and data tables Fill key fields with appropriate values

Master Database System Install Package Download: (beta version, 14 MB) Include the SERPM master database file as an example (installed to C:\FSUTMS\D4\SEPRM\MASTER\) Require ArcGIS Engine Runtime or ArcGIS Desktop installed on the computer running this application

Project Contributors Vidya Mysore, FDOT Project Manager Albert Gan, FIU Project Manager Kaiyu Liu and Tao Wang, Programmers, FIU Lina Kulikowski, Broward MPO Sarah McKinley, Capton Siluvairajan, and Robert Schiffer, CSI Minhua Wang, Citilabs, Inc. David Schmitt, AECOM USA, Inc. Kenneth Kaltenbach, Corradino Group, Inc. FDOT Research Center and Staff Members

Questions? Albert Gan Lehman Center for Transportation Research Florida International University