Performance Measurement and Benchmarking Module 5, Lesson 2 Performance Measurement and Benchmarking

Measures and Indicators Efficiency Effectiveness Reliability Safety Quality Cost per Passenger System based Passenger based Measures are general areas or aspects of performance which transit system managers and governing authorities monitor in order to identify potential problem areas, determine whether performance goals and objectives are being met, and to allocate resources. Indicators are specific statistics that inform our understanding of how the transit system is performing for a given measure Efficiency measures what the military calls “bang for the buck” – how much output is achieved per unit of input? Indicators of efficiency include cost per mile, cost per passenger, operator payroll/platform ratio, maintenance person-hours per 1,000 miles, etc. Effectiveness measures how well transit is serving demand. Indicators of effectiveness include ridership, % population within ½ mile of a transit route, mode split, peak and off-peak headways, span of service, Title VI compliance, etc. Reliability measures service availability at the time and place scheduled. Indicators of reliability include systemwide and route-level on-time performance [typically a bus is considered “on-time” if it arrives at a timepoint no more than 5 minutes late and departs zero minutes early], missed trips by cause [e.g., no driver, no vehicle, detour, accident, breakdown], chargeable roadcalls, etc. Safety indicators include collisions per 100,000 miles, passenger injuries per 100,000 passengers, OJI [on-the-job injury, or Worker’s Compensation] claims, accidents graded “preventable,” accident liability claims (both in terms for frequency and cost), etc. Quality indicators include customer satisfaction survey results, complaints (by category) and commendations, percent of buses in service with functioning a/c in summer and heat in winter, percent of buses cleaned and washed each night, etc. Cost per passenger is a composite indicator that reflects cost of operating service, ridership, and revenue. This is often the indicator most watched by governing boards and other authorities. It should be emphasized that a critical function of management is to find balance among these measures. For example, a transit system that is super-effective in meeting virtually all potential ridership demand may be very inefficient. A system can be super-efficient in ways that compromise reliability, safety and reliability. A good class discussion would be to talk about how increasing emphasis on one measure can affect another.

Sources of Information and Pitfalls National Transit Data Base Inconsistent definitions/data collection methods among transit systems Supplement discussion of what the NTD is and how it came into being with handouts of actual transit system data from the NTD. Thanks to NTD, most financial data are collected in a consistent manner among all transit systems. However, many non-financial data are defined and collected differently among transit operators. Examples of data that are highly varied in terms of local definition and data collection methods include accidents, preventable accidents, roadcalls, complaints, missed trips, on-time performance, etc.

Benchmarking Uses and misuses of performance data to compare one transit system with others How the same statistic can connote “good” or “bad” performance Local characteristics that affect comparability and performance Benchmarking involves comparisons among “peer” entities. Just like the highly controversial “ranking” of colleges each year by US News and World Report, attempts to rank-order “best” and “less best” transit systems are always problematic. Nevertheless many transit boards and governing authorities rely on benchmarking in order to gain a relative degree of confidence in the transit system management and an assessment of transit system performance. It is risky to make normative judgments based on a single indicator. For example, System A had more collisions last year than System B? Does that mean A is “safer?” What if you learned that A operates in a severe cold weather climate where black ice on the roads causing minor fender bender accidents is very common? What if you learned none of A’s accidents was serious, but some of B’s accidents involved serious injuries and a fatality? A good class discussion would be to list the various local characteristics that could affect the performance of two transit systems, assuming they operate the same number of buses. Examples: extreme cold weather climate (Minneapolis) vs. extreme hot weather climate (Phoenix); high population density packed into a relatively small service area (Santa Monica) vs. regional service that includes lower density suburbs and long-distance express routes (Hartford); CBD-oriented economy vs. sprawl; local service only vs. local plus express; tight labor market in which the transit system has difficulty competing and suffers high employee turnover vs. labor market where the transit system is a preferred employer; well-funded transit system with new facilities and vehicles vs. cash-strapped operation struggling to maintain old infrastructure; booming local economy vs. stagnant or declining; high degree of environmental consciousness among the populace (Portland, many college towns) vs. areas where everybody drives an SUV; etc.

Using Performance Data Using local system data to allocate funding Indicators of primary interest to boards and governing authorities Indicators used by management for monitoring day-to- day operations Reference TCRP S-06 regarding how performance measures are used to allocate funding (Florida?) See Session 2 assignments Board focus on ridership, subsidy per passenger, revenue recovery ratio, complaints, etc. Management focus on payroll/platform ratio, missed trips by cause, absenteeism and overtime, etc.