1. Critical Needs ChapterS webinar
Presentation for TSSM Stakeholders and FHWA
4th in the series of 6
06 December 2017
George List, Shannon Warchol, NC State
David Hale, Leidos
Brandon Nevers, Kittelson
Rob Bertini, USF

2. Deliverables Schedule
Task
Deliverable
Due Date 4
Final Annotated Outline
04/21/2017 5
Draft Critical Need Chapters
11/21/2017
Stakeholder Webinar #4
12/21/2017
Stakeholder Webinar Meeting Minutes
01/06/2018
Final Critical Need Chapters
01/21/2018 6
Draft Intermediate Need Chapters
03/21/2018
Stakeholder Webinar #5
04/21/2018
05/06/2018
Final Intermediate Need Chapters
05/21/2018 7
TRB Annual Meeting Materials
December 2017 and December 2018
Meeting Minutes from TRB Annual Meeting
4 weeks after TRB Annual Meeting
Stakeholder Webinar #6 -- PFS/Task Force Review Webinar of TRB Annual Materials
07/21/2018
08/06/2018
Draft Volume V
08/21/2018
Final Volume V
09/21/2018 8
Problem Statements
10/21/2018
Government Review Period
Four months added to ensure document clears 508 compliance

3. Summary of Feedback http://tssm.kaiproject.com/
Chapter 5: 115 comments
Chapter 6: 120 comments
Chapter 7: 106 comments
Chapter 8: 191 comments
Chapter 9: 66 comments

4. Overarching Feedback (all chapters)
Too technical, too academic
Need example problems, case studies
Too product-specific in some cases…
…too product-agnostic in other cases
Prefer step-by-step guidance
Too focused on microsimulation

5. Chapter 5 (Defining the Problem)
Chapter Themes:
Bounding the problem wisely is key to successful simulations
Performance measures are needed to:
Evaluate design and operation alternatives
Determine if you have a good simulation model
Simulation is often a key part of the solution to many analysis problems, but it is not the solution to all problems

6. Chapter 5 (Defining the Problem)
Selecting performance measures
Alternatives analysis
Calibration
Setting study bounds
1. Initial ID based on existing conditions
2. Review for future conditions
3. Continuing check of simulation results
Scoping the role of simulation

7. Chapter 5 (Defining the Problem)
Major Comments Received
Differences in terminology and content between chapters
Refer to overall technical editor to resolve conflicts/redundancies in content.
FHWA reliability calibration method is not clearly described in Chapter 5.
Refer reader to Chapter 8 for more info on calibration using FHWA method.
FHWA reliability calibration method is not cost-effective for most users.
Highlight method is to be used only if simulating reliability.
Need more material on meso and macro simulation
Avoided use of “micro” where guidance applies to all levels.
Add bullets on meso simulation
DTA Discussion not clear
Clarified distinctions between DTA and mesoscopic simulation modeling

8. Chapter 6 (Data) INTRODUCTION DATA TYPES AND SOURCES
Geometric Data | Traffic Count Data | Traffic Signal Ops Data | Calibration Data | Safety Data
First-Gen Probe Data | NPMRDS Data | Big Data Sources
DATA TYPES AND SOURCES
Signalized Intersections/Arterials | Transit Priority Lanes | Roundabouts | Non-traditional intersections | Ped-Bike Lanes | Freeways/Interchanges | Two-lane highways | Multilane highways | Work Zones | Toll Plazas | Managed Lanes/Ramp Metering | Safety Analyses
DATA REQUIREMENTS (ANALYSIS TYPES)
Macroscopic | Mesoscopic | Microscopic
DATA REQUIREMENTS (MODEL TYPES)
Data Comprehensiveness and Reliability | Data Accuracy | Data Fusion and Aggregation | Data Quality Control
DATA CHALLENGES
Freeway Corridor | Freeway Interchange | Highway Corridor | Reliability | Sampling | Simulation Runs | Data Measurements with NGSIM
SAMPLE DATA CASE STUDIES/ APPENDICES

9. Some Excerpts… Connected Vehicle Data

10. Reviewer Comments Summary
No of reviewers: 8
Total comments received: 120
Responses to comments up on the Kittelson project website
Chapter updates based on comments and discussions on-going

11. Chapter 7 (Creating the Model)
Fill time, equilibrium, spatial-temporal limits
Network development
Geometric data
Control data
Traffic data
Operating condition data
Typical calibration data

12. Chapter 7 Material One-page tech briefs with visuals
Fundamentals (e.g., link, node, equilibrium)
Approaches to data entry (e.g., link-connector, demands vs. volumes, O-D flows, conditional turns)
Concepts that overlap with other chapters

13. Chapter 7 Feedback Quality of diagrams and images
Tool specific vs. tool agnostic
Step-by-step instructions
Overlapping material
Add new material:
Volume balancing, global vs. local parameters
Optimize discussion of:
Volumes vs. demands, equilibrium, editing “raw” data, Monte-Carlo analysis, diagonal movements, results-oriented analysis

14. Chapter 8 (VCV) Introduction Performance metrics Verification
Basic tenets of the VCV process
Performance metrics
Options for micro, meso, and macroscopic models
Verification
Important considerations in checking the simulation software’s capabilities and the completeness of the input data
Calibration
Perspectives to consider, sequence options, assessment metrics
Validation
Checking robustness, assessing limitations
Step-by-Step Process
Recap previous material in a “do this” fashion
Summary

15. Chapter 8 Focus (VCV) Verification Calibration Validation
Software logic reflects system behavior (software developers)
Input data correctly describe the system (users)
At the end: logic is okay and the input data are nominally clean.
Calibration
Software parameters are tuned to predict observed behavior
Input and output for site-specific and global system performance metrics
At the end: model parameters allow the observed performance to be matched
Validation
Check the robustness of the model’s predictive capability
Use data for scenarios not employed in calibration
At the end: model robustness and limitations have been assessed.

16. Chapter Highlights

17. Chapter Highlights (2)

18. Chapter 8 Comments More examples
Be more mindful of the audience – writing style
Move as much of the dense technical material as possible to appendices
Break down the introduction into smaller sections
Place more stress on macro- and meso-scale modeling
Clarify why certain practices are deemed important
Ex-situ tests, for example, and hypothetical facility tests
OD calibration/adjustment
Why the ideas of warm-up, operating, and cool-down periods may be problematic

19. Chapter 9 (Processing, Analysis, and Presentation)
Running simulation models
Run times, random seeds, keeping track of model runs
Data processing
Output methods, data types, aggregation issues, data synthesis
Analysis
Definitions, statistically understanding the results, interpretation
Presentation
Multiple metrics, tables, charts, figures, visualization options, visualization tools, written documentation

20. Chapter 9 Guiding Principles
Short paragraphs
Heavy use of figures/tables/examples
Discrepancy in comments between usefulness of seeing “staged” figures versus screenshots
Internal & external links
Statistics from the ground up

21. Chapter 9: Comments Formatting / Links to Tables & Figures
October review was mid-process; formatting will be standardized for the draft review
Adjustments & Additions to Examples
Will be incorporated as able; figures cleaned up by editor
Addition of other procedure options
Will incorporate as possible while maintaining impartial to software
Further explanation of statistical assumptions
Incorporate while maintaining usability

22. Timeline Review Today July 10th ~1/3rd checkpoint Traffic analysis PFS
TRB Task Force
August 15th
ITE SimCap
Today
December 6th
Webinar #4
Stakeholder webinar #3
(annotated outline)
April 21st
Webinar #5
Start date
Jan
2017
Jan
2018
Jan
2019
January 7-11
TRB
Task force meeting
Today
Interviewed 16
PFS states
TRB Mid-Year
HCQSC (Sturrock)
March 21st
Draft intermediate need chapters
November 21st
Draft critical need chapters
January 21st
Final critical need chapters

23. Activities at TRB Sunday Monday Tuesday Workshop 130:
Data-Driven Simulation
9:00 am – 12:00 pm
Report from TSSM team
Chapter Liaisons
1:00 pm – 6:00 pm
One hour per chapter
- With TSSM team
SimSub
7:30 pm – 9:30 pm
Report from TSSM team
Task Force
8:00 am – 12:00 pm
Brief report from liaisons
Report from TSSM team
Also, members of the TSSM project team will be attending:
Simulation subcommittee meeting, HCQS
Simulation subcommittee meeting, Freeway Ops
Simulation subcommittee meeting, Traffic Signals

24. Questions and Answers Thank You