1 based on Federal Highway Administration Capability Maturity Model Workshops Transportation Systems Management and Operations (TSM&O) Performance Measurement Dimension

2 Challenges Congestion and delay are increasing as economic and population growth continue but existing roadways have capacity constraints. A high value is placed on the reliability of the system. Reviewing the existing TSM&O versus state of practice: −Unsystematic; and −Pockets of excellence. Opportunities Unexploited potential of aggressive, integrated, collaborative TSM&O applied to existing roadways. Agencies can develop the key capacities needed to improve TSM&O effectiveness.

3 Objective: “Mainstreaming” continuous improvement in TSM&O Process: Major program of SHRP2 research and over 40 FHWA-sponsored State DOT workshops Workshop Process: Agency staff self-evaluation and improvement plans, using capability maturity model (CMM) White Papers are available covering: State of the practice regarding agency capabilities Current agency approaches for improvement nizing_for_op.htm

4 Business Processes Organization and Staffing Culture Systems and Technology Performance Measurement “Capabilities” Business and technical processes support strategies Organization and relationships support processes Effective TSM&O Strategies Collaboration

5 Business Processes, including planning, programming and budgeting (resources) and project development and procurement. Systems and Technology, including use of systems engineering, concepts of operations, systems architecture standards, interoperability, and standardization. Performance Measurement, including measures definition, data acquisition, analytics, communication and utilization. Culture, including technical understanding and business case, leadership, outreach, and program legal authority. Organization and Staffing, including programmatic status, organizational structure and accountability, staff capabilities, training/development, and recruitment and retention. Collaboration, including relationships with public safety agencies, local governments, MPOs, and the private sector.

6

7

8 Capability Level 1 Some output-level performance is measured and reported by some jurisdictions Capability Level 2 Output-level performance measures are used directly for after-action debriefings and improvements; data easily available and “dashboarded” Capability Level 3 Outcome-level measures identified (networks, modes, impacts) and routinely utilized for objective-based program improvements Capability Level 4 Output and outcome performance measures reported internally for utilization and externally for accountability and program justification

9

10 Workshop findings will be presented. Followed by a brief case example. Along the way – think of your questions or comments: Type these into the webinar comment box.

11 The summary in slides that follow will provide:  Typical state-of-play;  Self-improvement actions suggested by workshop participants (in Implementation Plans);  Not a “how to” – but a descriptions of what TSM&O staff identify as key actions for improvement in Performance Measurement Dimension; and  May help you think through applicable actions in your agency. State DOTs differ widely in size, number of regions/districts, urban vs rural, etc.  No standard solutions (but there are peer agencies).

12 Accountability to elected officials and the public: “What did we get for our money?” Proactive public relations for the agency: Shows we know what’s going on without relying solely on anecdotes or reacting to events. Evens the playing field for Operations in promoting a data-driven program. …but most importantly…

13 It is a sound business practice. We measure performance because it helps us get better at what we do:  Detects and corrects problems in an era of shrinking budgets and leads to better projects being implemented.  Helps to manage, describe, and improve processes: operations programs are better structured.  Allows for ongoing evaluations: demonstrating the value of our activities.  Becomes the foundation for performance management.

14 The key elements of Performance Measurement:  Measures definition;  Data acquisition; and  Measures utilization (taking action based on trends and evaluations.)

15 Policy visibility of performance.  Impending requirements of Moving Ahead for Progress in the 21st Century Act (MAP-21.) Developing performance measures.  Lack of performance measure definitions for weather, work zones, and signalized arterials was frequently mentioned as a problem.  Transportation Incident Management (TIM) measures often not coordinated between DOTs and safety agencies.

16 Input, output, and outcome measures:  Output measures most commonly measured (operations activities); and,  Need for standardization and guidance mentioned. Resources for Performance Measurement:  Obtaining funding for Performance Measurement is a challenge for some agencies; and  In some cases, upper management is not convinced of the need for it, and the funding must come from existing budgets.

17 MAP-21 performance management requirements will give push to programs. FHWA projects on performance measures for TIM, work zones, weather, as well as conducting evaluations.

18 Coordinate TSM&O Performance Measurement with TSM&O Strategic Plan. Develop Performance Measurement Plan and/or Data Business Plan that includes:  Measures and targets;  Data availability and acquisition;  Use of measures for project selection/prioritization and resource allocation;  Internal and external reporting – audiences/needs/methods; and  Evaluations of completed projects.

19 Standardize TSM&O Performance Measurement related to both recurring and nonrecurring congestion, especially TIM. Develop a TSM&O Performance Measurement pilot program or staged approach to implementation. Review MAP-21 requirements being implemented at the agency level:  Targets and timeframes; and  Ensure common terminology and objectives between TSM&O and other agency activities.

20 Existing data availability:  Some TSM&O units collect and “own” TIM data; others are dependent on emergency responder and computer aided dispatch (CAD) systems.  Freeway detector data also are widely available but not all agencies use them to develop congestion statistics (outcome measures).  Work zone data are difficult to obtain – usually overseen by other units within the agency.  Signalized arterial data is difficult to obtain. Outsourcing:  Private vendor vehicle probe data are becoming more widely available.

21 Travel time data becoming widely available from private vendors; can serve multiple units within DOTs. Several agencies mentioned MAP-21 as a driving force behind travel-time/speed data acquisition:  Many suggested that they were looking into probe data not only to meet MAP ‑ 21 requirements but also to fill in gaps where detectors do not exist. Several agencies producing operations-oriented performance reports.

22 Identify ongoing collaboration with external partners:  Measure definition;  Shared data; and  Coordinated/integrated planning and programming (e.g., with MPOs.) Identify responsibility for maintaining data system in Business Plan.

23 Internal utilization:  Incident management and snow and ice control are the two areas where performance data are used for operational management.  TIM data widely collected and reported, but infrequently used to modify TIM activities and programs.  After-action review of incidents becoming common.  Traveler information program performance (e.g., web site hits and VMS messages) monitored and used to make upgrades.  The development of outcome measures is impeded by limitations on the availability and integration of multisource data.  Overall, agencies struggling with how to integrate performance measurement into decision-making.

24 External reporting:  Production of periodic performance reports was the most common use of performance measures, although not all agencies produced them.  A few states included TSM&O-related activity measures – largely output data on external dashboards, usually incident trends.  Travel time (congestion) based reports based on measured data were far more rare.  Conducting customer surveys related to the delivery of operations services is rare.

25 Management accountability:  Accountability for TSM&O program performance is in the early stages; and  Incident clearance targets are sometimes set; reviews conducted only when the target (often 90 minutes) is exceeded. Comprehensive performance management program:  No agency has achieved a fully integrated Performance Measurement system that links inputs, outputs, outcomes, and targets into a formal TSM&O performance management process.  Outcomes also are affected by other programs, such as capacity expansion, demand management, alternative mode use, and safety countermeasures.

26 Again, MAP-21 seen as the driver for agency performance management activities. Agencies are venturing out into components of performance management that are most useful to their own situations, but these examples are scattered around the country.

27 Develop a strategy for how performance data will be integrated into agency processes, including project planning and arterial operations. Establish a format and overall approach for how measures will be displayed and reported for both internal and external use. Review experience and activities of peer agencies. Formalize process for post-incident management assessment, including definitions, performance-related criteria follow-up, and data assembled.

28 Use of performance measures in business case materials:  Guidance on conducting before/after evaluations of operations projects needed; and  Demonstrating benefits of operations needs to be a routine activity in order to build support for TSM&O programs: –A communication strategy for describing the benefits of TSM&O to upper management and the public is the basis for this.

29 One Executive Summary (covers all dimensions) Six White Papers – one on each capability dimension

30 Activities Possible sponsorship Checklist for developing a TSM&O Performance Management PlanFHWA Standardized definitions for a wide range of TSM&O performance measures covering TIM, work zones, weather, and signalized arterials FHWA Best practices for communicating TSM&O performanceFHWA OPMM #13 Guide to TSM&O data acquisition, management, and analysis methods NCHRP or FHWA Peer-to-Peer Exchange on TSM&O Performance MeasurementFHWA Best practices in using TSM&O performance measures in investment decision-making NCHRP or Pooled Fund FHWA Federal Highway Administration OPMM Operations Performance Measures and Management NCHRP National Cooperative Highway Research Program

31 Read the CMM reports and white papers. Participate in FHWA state DOT Self-Evaluation Workshops and Regional Operations Forums. Utilize NOCoE website to track developments and access resources. Contact and communicate with corresponding staff in peer states.

32 FHWA HQ office through your local FHWA Division offices or NOCoE