1. Leveraging Tools to Better Grok Model Calibration
TRB Applications Conference 2017
Bhargava Sana [ SFCTA ]
Jayne Chang [ SFCTA ]
Lisa Zorn [ MTC ]
Elizabeth Sall [ UrbanLabs ]
Brice Nichols [ PSRC ]
Good Afternoon! I am here to share our experience with building visualization tools for calibrating Fast-Trips Dynamic Transit Assignment model. There is a Fast-Trips tutorial in progress in a parallel session!

2. Outline Motivation Desired Features
Python + Tableau (“Py-bleau”) Approach
Fast-Trips Calibration and Validation
Lessons Learned
Tableau – a business intelligence tool

3. Motivation Aggregate More Disaggregate
Fast-Trips (FT) Dynamic Transit Assignment O D O D
Aggregate
More Disaggregate
Better Calibration and Validation Tools ?
Recently, there has been a continuous shift towards more disaggregate travel models
Visualization tools for calibration and validation are lagging behind - reliance on conventional tools (aka Excel)
Worksheets of pivot tables and charts
ArcGIS/proprietary TDM software for mapping
Fast-Trips (FT) is one such model that simulates a transit path choiceset for each traveler
FT accounts for rider heterogeneity and quality of transit path
Fast-Trips (FT) Calibration runs using observed transit paths in California HTS
Need for both - System level metrics and Individual path level comparisons

4. Ideal Tool Desired Features Simple development and extension
Interactive visualization
Integrated mapping
Easy sharing and dissemination

5. - $ Picking a Tool Data Operation Integrated Mapping
Interactive Visualization
Ease of Overall Development
Ease of Sharing
Cost
Overall
Excel +
ArcGIS
Slow for big datasets - $ ★★
iPython Notebook
Performance impacted by large data
Add-on packages
Free
★★★ 󠅞
Tableau
Powerful for large datasets
Basic-inbuilt
★★★★
Easy dashboard building tool
Fits multiple criteria for desired enhancements
More powerful and flexible than Excel
Publish and share using web
Easy sharing ability is important for dispersed team members

6. Fast-Trips Calibration and Validation
Model Path
Observed Path =? O D O D
Compare modeled transit paths against survey paths
Primary mode/Routes/Stops
Access/Egress distances and times
Transfer rates
Python useful for Tableau setup
Relate observed and model output data
Organize/re-structure data (table joins, metrics, etc.)

7. Tableau in Action
Video

8. FT Calibration Dashboards
Validation metrics
Transit path visualizer
1. Show validation metric dashboard
Primary mode match pie chart (overall performance)
Acc/egr distance gap, suggests issues in generating access links
Transfer rate, suggesting low transfer penalty
Confusion matrix (and display full data for some cells)
2. Show a few records on map

9. Lessons Learned Interactivity allows for quick problem identification
Maps of disaggregate records more informative
Easy and intuitive usage
Using separate tools for data cleaning and visualization is manageable
Maps of disaggregate records can be more informative and direct than aggregate statistics and numbers
Lower entry barrier for the tool encourages more people to use it
There may certainly be other off the shelf tools that might work just as well as Tableau. Similarly, could use R instead of Python.
Effort to build Tableau dashboards not insignificant but much lower than any custom solution

10. Thank You!