1. Systemic Methods for Pedestrian and Bicycle Safety

2. Topics Systemic Introduction Methodologies and tools
Potential countermeasures
Examples

3. Systemic Introduction
There are two general approaches to safety management: 1) selecting and treating sites based on site-specific crashes (referred to as the crash-based approach for this guide), and 2) selecting and treating sites based on site-specific geometric and operational attributes known to increase crash risk (referred to as the systemic approach for this guide). These two approaches are complementary and support a comprehensive approach to safety management. The primary
difference is the way in which analysts identify issues and develop projects in the planning stage.
What is the purpose or intended outcome?
The selection of the systemic methodology influences data needs, selection of performance measures, and screening methods

4. Choosing an approach to improve safety
Systemic
Traditional versus Systemic?
When identifying locations for treatments to improve safety, there are two approaches: a traditional crash-based approach and a systemic approach. The two approaches provide a complimentary way to address safety. The following slides will describe benefits and limitations of both.
Traditional

5. Traditional Crash-Based Approach
Identify/treat sites and with the highest potential for site-specific safety improvement.
The underlying safety issue typically varies at each site.
Projects can range from relatively low-cost to larger capital improvement projects.
Characteristics of a traditional crash based approach:
Identify/treat sites and with the highest potential for site-specific safety improvement.
Identified by dense clusters of crashes.
The underlying safety issue typically varies at each site (i.e., agencies are not focused on addressing a specific issue unless the screening is carried out for a specific crash type).
Projects can range from relatively simple and low-cost improvements (e.g., enhancing signing or striping, trimming vegetation, or modifying signal phasing) to substantial capital improvement projects (e.g., constructing a roundabout, modifying the skew angle of an intersection, or realigning a horizontal curve).
There is an opportunity to achieve reductions in average crash frequency and severity at treated locations given the focus on site-specific issues and targeted treatments.
Also known as hotspots, blackspots, or crash clusters.
Image Source: VHB

6. Limitations of a Traditional Crash-Based Approach
The need for site-specific data.
The inability to efficiently address highly-dispersed crashes.
The potential for high-cost improvements at spot locations.
Limitations:
The need for site-specific data,
The inability to efficiently address highly-dispersed crashes,
The potential for high-cost improvements at spot locations.
Image source: FHWA – EDC 3 Work Zones (

7. Systemic Approach
Targets specific crash types or severity for formulation of system-wide policy as identified by risk.
Identify sites based on site-specific geometric and operational attributes rather than observed crashes.
Addresses many sites throughout the network through lower cost improvements.
Characteristics of a systemic approach:
Targets specific crash types or severity for formulation of system-wide policy.
Targets common underlying safety issues across sites, agencies can implement similar projects across a network to address high priority crash types and risk factors.
Agencies use the systemic approach as an effective means to identify sites and implement treatments at those sites with the highest risk across a road network. Agencies can use this approach to target specific areas from their Strategic Highway Safety Plan (SHSP). While treatment sites aren’t identified based on specific crash locations, crash data is used to identify the targeted geometric or operational features.
Agencies typically aim to make modest site-specific safety improvements with proven countermeasures on relatively high-risk sites identified by the presence of risk factors rather than site-specific crash history.
Agencies can apply the systemic approach without site-level crash and exposure data or when the average crash frequency at individual sites is relatively low (i.e., highly-dispersed crashes).
Given the typical extent of improvements (i.e., many improved sites), systemic projects are generally low-cost improvements (e.g., enhancing signing or striping, installing rumble strips, or upgrading signal heads). Higher-cost improvements are also candidates for the systemic approach, but the improvement should be highly effective to justify the increased cost.
Provides an opportunity to achieve reductions in crash frequency and severity across a large portion of the system given the focus on priority crash types and risk factors rather than site-specific crash history.

8. Limitations of the Systemic Approach
Unknown countermeasure level of benefit as compared to the crash-based application.
Range of project and maintenance costs.
Limitations of a systemic approach:
It is generally unknown if the systemic application will result in the same level of benefit as the crash-based application for the same countermeasure. As such, it can be difficult to analyze the expected benefit and cost-effectiveness of some systemic projects. Also, when doing an evaluation of projects, it’s possible to show an increase in crashes as some of the countermeasures may have been installed in locations where there were no documented crashes.
Project and maintenance costs can also range from negligible to relatively high depending on the treatment and level of implementation. Although the unit cost per site is often relatively low, the service life for low-cost countermeasures is typically less than the service life for higher-cost countermeasures. As such, it is important to consider the life-cycle costs prior to implementation.
Image source:

9. Factors Influencing Approach Selection
Data availability
Resources
Established priorities
State/local agency relationship
Just as specific processes for conducting site analysis vary widely, the systemic approach used by individual agencies will vary.
Data availability dictates the level of detail in the analysis. While a systemic analysis can be completed with nearly any amount of data, using more data will allow for more refinement of potential risk factors.
The availability of resources determines the extent of improvements that can be made. Resources may also impact the level of analysis that can be completed.
The established priorities of an agency may define the direction of the analysis.
The relationship between the State and local agencies may impact the funding available for systemic improvements on non-state routes as well as the extent systemic improvements are applied to non-state routes.
Systemic
Traditional

10. Systemic Approach Return on Investment Example
Option 1 (Traditional): Install 3 roundabouts
Cost = $1M/site
Ave. crash history = 20 crashes/yr.
CMF = 0.6 (40% reduction in crashes)
Benefit = reduction of 24 crashes/yr.
Option 2 (Systemic): Install intersection improvements at 500 sites
Cost = $6,000/site
Ave. crash history = 3 crashes/yr.
CMF = 0.95 (5% reduction in crashes)
Benefit = reduction of 75 crashes/yr.
This example highlights the potential opportunity to achieve reductions in crash frequency and severity across a large portion of the system given the focus on priority crash types and risk factors rather than site-specific crash history.
For example, consider a $3M safety program and the opportunity to implement one of two options. The first option is to install three roundabouts at an average cost of $1M per site with an average crash history of 20 crashes per year and assuming a 40 percent reduction in crashes as the average treatment effect. The second option is to install intersection improvement packages at 500 sites at an average cost of $6000 per site with an average crash history of 3 crashes per year and assuming a 5 percent reduction in crashes as the average treatment effect. The system benefit for the first option is a reduction of 24 crashes per year and the system benefit for the second option is a reduction of 75 crashes per year. Even with a modest crash reduction per site, targeted systemic improvements can have a large impact on the system as a whole. Chapter 3 provides examples of such comparisons, including a method to allocate funding between crash-based and systemic approaches.

11. Why Use Systemic to Address Bicycle and Pedestrian Safety?
Is not crash dependent.
Addresses many sites throughout the network through lower cost improvements.
Bicycle and pedestrian crashes are often under-reported so it can be difficult to determine “hot spots”. Systemic allows agencies to focus on bicycle and pedestrian risks.
Agencies are able to treat many locations throughout their network.
Image source:

12. Methodologies and Tools

13. Systemic “Big Picture”
An improvement that is widely implemented
based on roadway characteristics correlated with
particular severe crash types.
So what is the “big picture” of systemic analysis? It is an improvement that is widely implemented based on roadway characteristics correlated with particular severe crash types.

14. Systemic “Big Picture”
What is “risk”? The potential for a specific type of severe crash to occur at a specific location because of the location’s characteristics or features.
So what is the “big picture” of systemic analysis? It is an improvement that is widely implemented based on high-risk roadway features that are correlated with particular severe crash types.

15. Let me show you what I mean using data from Missouri DOT as an example
Let me show you what I mean using data from Missouri DOT as an example. This shows you the fatal crash location in MO in Pick a county and watch the fatal crash location as we progress over a couple of years. (NOTE: CLICK THROUGH 4 VERSIONS OF THE MAP) What do you see? That’s right…the location of fatal crashes varies from year to year. I would encourage you to complete this exercise using data from your agency to confirm.
The reality is, the occurrence of crashes is random in nature, and this is even truer of fatal and serious injury crashes. Just look at this map!
While many of the fatal crashes occur on these roadways, there are very few high crash locations. For example, a study of the entire rural secondary system in Minnesota (22,000 miles, 13,000 intersections and 19,000 curves) failed to find a single location that averaged one severe crash per year. Yet the crash data proves severe crashes do occur on the secondary system.
As a result, agencies will have trouble meeting their safety planning goals by only investing in high-crash locations; some system-based deployment will be needed. Thus, in an effort to overcome this challenge, some states and local agencies started implementing systemic safety improvements in addition to spot improvements.

16. 2005
2006
2007
2008
Now let’s look at fatal crash types and see what happens. What you see here is data by collision type. Focus in on one collision type as I move through the years. (NOTE: CLICK THROUGH 4 VERSIONS OF THE CHART )What do you see? That’s right…fatal crash types are pretty consistent from year to year. So what does that tell us…fatal crash types are predictable. With this knowledge, states began to implement a systemic approach to safety.
A defined approach for systemic safety planning was developed to provide guidance to agencies so they can incorporate systemic improvements into their safety programs to increase the potential to reduce severe crashes. The systemic planning approach encompasses the successes and lessons learned from initial agency efforts along with more recent knowledge gains.
Let’s discuss the difference between systematic and systemic. A systematic improvement is one that is deployed across an agency’s entire roadway system. There are two concerns with this approach….first, severe crashes tend to be scattered and not spread evenly across a roadway network; and second, funding is not typically adequate to support systematic deployment. Therefore, systemic planning provides the means to prioritize locations within the roadway system that are most at risk for a severe crash, and invest the limited funding toward proactively improving these locations.

17. Systemic “Big Picture”
Essential components of analysis include:
Identifying crash trends and common geometric features common to those crashes.
Screening and treating those geometric features throughout roadway network.
There are a variety of methods to conduct a systemic analysis but the primary components include:
Identifying crash trends and common geometric features common to those crashes.
Screening and treating those geometric features throughout roadway network.
We are going to review a couple of these methodologies, including those from FHWA Systemic Project Selection Tool and the Highway Safety Manual.

18. FHWA Systemic Project Selection Tool
FHWA developed the Systemic Safety Project Selection Tool to provide a clear process for States and local agencies to follow.
The tool is intended to provide:
A step-by-step process to conduct systemic safety analysis and planning
Analytical techniques for determining a reasonable distribution between the implementation of spot safety improvements and systemic safety improvements
A mechanism for quantifying the benefits of safety improvements implemented through a systemic approach
The Systemic Safety Project Selection Tool builds upon current practices for identifying roadway safety problems and developing improvement projects for them. It fills the current void of analytical techniques and models useful for conducting the systemic analysis approach to roadway safety (current techniques and models focus on site-specific analysis).
The key to the Systemic Tool is the concept of evaluating an entire system using a defined set of criteria that will vary depending on the available data to produce results that are an inferred prioritization indicating some elements of the system are better candidates than others for safety investment.
The Tool is organized into three elements:
Element 1: The Systemic Safety Project Selection Process
Element 2: Balancing Traditional and Systemic Safety Investments
Element 3: Performing Systemic Program Evaluation

19. FHWA Systemic Project Selection Tool Element 1: Systemic Safety Planning Process
Transition to discussion about the systemic safety project selection process

20. FHWA Systemic Project Selection Tool Element 1: Systemic Safety Planning Process
The Tool presents a four-step process for selecting systemic safety projects which is referred to as the Systemic Safety Planning Process. The process can be scaled based on the availability of technical resources and the quality or quantity of data to support different analytical approaches. The Tool is intended to be flexible and adaptable to various agencies to account for differences in data availability, agency practices and regulations, and available resources.
The Systemic Tool begins by looking at system-wide data to analyze and identify systemic safety problems.
In the first step, system crashes are reviewed to identify and document the particular crash types that will become the focus of the crash reduction efforts, including the characteristics of the locations where the focus crash types tend to occur.
In the second step, locations are then screened in the second step based on a defined threshold value for characteristics identified at locations where the crashes tend to occur. The screening process identifies high priority locations for safety investment.
One or more low-cost safety countermeasures are then identified in Step 3 to address the underlying contributing circumstances on a majority of roads where the focus crash types occur. The solutions need to be low cost so they can be widely deployed across the system in order to address a large number of locations.
The fourth and final step is to develop a process for selecting recommended safety improvements at site specific locations and then select the recommended project for each prioritized location.

21. What we mean by “focus crash type”
The crash type that represents the greatest number of severe crashes across the roadway system being analyzed and provides the greatest potential to reduce fatalities and severe injuries.
Road Departure
Intersection
Pedestrian
Speeding
An analysis of the agency’s data can help to determine this focus crash type. The state Strategic Highway Safety Plan is another source for identifying focus crash types.
Some agencies focus on the highest proportion of certain crash types while others focus on which crash types are over represented based on exposure/volumes or severity.

22. What we mean by “focus facility”
The facility type on which the focus crash type most frequently occurs.
Rural, Two-Lane Highways
Urban, Signalized Intersections
Horizontal Curves
Rural, Thru-STOP Intersections
This chart shows the locations of bicycle-motor vehicle crashes in North Carolina from From this the agency may want to further identify roadway characteristics associated with rural, non-intersection crashes to identify risk factors (discussed on next slide).
Source: The University of North Carolina Highway Safety Research Center. North Carolina Bicycle Crash Types, August 2011. 

23. What we mean by “risk factor”
A representation of risk in terms of the observed characteristics associated with the locations where the targeted crash types occurred
Volume
Alignment
Intersection Control
Presence of Shoulders
Risk can include elements such as traffic volume, roadway alignment, type/conditions of intersection control, presence of shoulder, among others. Over the next two slides there are examples of bicycle and pedestrian-specific risk factors.
What are some of the potential risk factors in this photo?
Lack of shoulder
Pavement condition
Horizontal curve/limited sight distance
Photo source: FHWA (

24. Examples of Bicycle Risk Factors
Signalized intersection
Inadequate signal phasing
Inadequate sight distance
Turning movement conflicts
Inadequate lighting
Failure to stop/yield
Unsignalized intersection
Segment
Lack of dedicated space
Narrow lanes
Poor pavement quality
Large speed differential
High percentage of heavy vehicles
Inadequate shoulder width
Poor nighttime visibility or lighting
Inadequate buffer/barrier from vehicles
Inadequate pavement markings
These are a sample of bicycle risk factors. This list is not comprehensive and each agency should review potential risk factors and their crash data to determine which factors are appropriate.

25. Examples of Pedestrian Risk Factors
Signalized intersection
Inadequate signal phasing
Inadequate sight distance
Turning movement conflicts
Inadequate lighting
Failure to stop/yield
Unsignalized intersection
Mid-block
Inadequate warning of mid-block crossing
High traffic volume
High approach speed
Failure to stop/yield
Multi-lane roadway (multiple threat)
Inadequate delineation/warning (signs, pavement markings, delineators)
Poor nighttime visibility or lighting
These are a sample of pedestrian risk factors. This list is not comprehensive and each agency should review potential risk factors and their crash data to determine which factors are appropriate.

26. Helpful Hints
Local focus crash types can differ from statewide focus crash types
Focus crash types can include causal factors from the 4 E’s

27. Helpful Hints
Crash tree can include all severe crashes or just severe crashes for one focus crash type
Experience suggests that 100 severe crashes or more is best for identifying patterns.
Fewer facility types streamlines the process of identifying candidates for investment
After selecting the focus crash types in Task 1, Task 2 answers the question “Where are the crashes occurring?” Focus facilities could be two-lane rural roads, or local unsignalized intersections.
The crash tree diagram is a recommended approach and tool for this analysis. The crash tree can have a number of different formats, depending on the local agency capabilities and data availability. One such example is to begin the crash tree with the total number of severe crashes at the highest level. Each subsequent level separates the severe crashes by facility type.
Discuss the example crash tree diagram by mentioning the levels and stepping through the diagram to show how the focus facility type was selected. Possible discussion: After selecting lane departure as their focus crash type based on an analysis of crash data for the years 2007 through 2011, NYSDOT created a crash-tree diagram for the state roadway system to identify the focus facility type for their systemic planning effort. The first level separates the severe roadway departure crashes into rural, urban, and New York City areas. Separating the data into three more levels (divided versus undivided, number of lanes, and speed limit) identifies the appropriate focus facility as those that are rural, undivided, and two lanes with a posted speed limit of 55 miles per hour (represented by the highlighted boxes in the diagram).
The number of levels in crash tree depends upon the available data, most of which will likely come from the crash report. Caution should be exercised to use data that are considered reliable and relevant for the focus crash type. Care should be taken not to create too many levels in the crash tree or the number of crashes will become small and the patterns difficult to identify.

28. Helpful Hints
A minimum of 2 to 3 risk factors is suggested to differentiate between sites
Occasions may occur in which combining risk factors can indicate if a particular crash type is overrepresented
There is no rule on how many risk factors must be selected. At a minimum, you would need two to three risk factors to differentiate between sites. Of these, the presence of one or more fatal or serious injury crashes can be viewed as a risk factor. Selecting more—up to seven to ten—requires more time to perform the screening but also helps to determine the likelihood of future crashes.
There might be occasions in which individual risk factors do not appear to be overrepresented in the crash dataset. In these cases, you can perform the descriptive statistics analysis for combinations of risk factors to determine whether the related crashes are overrepresented. For example, one pilot agency’s crash data indicated that neither shoulder width nor shoulder surface type were risk factors for road departure crashes in horizontal curves. Engineering judgment suggested further analysis was required. Combining the data associated with these two risk factors revealed that severe crashes were found to be overrepresented in curves that had either narrow gravel shoulders or wide paved shoulders (which were also high volume corridors).
Image source:
Pedestrian:
Right-hook:
Bicyclist:
Speed limit:

29. Highway Safety Manual (HSM)
Network Screening
Diagnosis
Countermeasure Selection
Economic Appraisal
Project Prioritization
Safety Effectiveness Evaluation
The Highway Safety Manual (HSM) also incorporates the systemic methodology into it’s 6-step roadway safety management process.

30. Highway Safety Manual Network Screening
Diagnosis
Countermeasure Selection
Economic Appraisal
Project Prioritization
Safety Effectiveness Evaluation
Network Screening
Site-specific: identify specific sites for further analysis.
Systemic: identify common risk factors of crashes.
The first step of this 6-step Roadway Safety Management Process is to conduct network screening. Network Screening is when you analyze the entire network to identify potential sites or issues for further investigation.
There are two methods of network screening:
Site Specific identify specific sites for further analysis (typically those with high crashes or over-represented crashes).
Systemic: identify specific sites for further analysis (typically those with high crashes or over-represented crashes).

31. HSM Systemic Analysis Methodology
Establish Focus
Identify Network
Select Performance Measures
Select Screening Method
Screen/ Evaluate Results
The HSM methodology for network screening, and therefore systemic analysis, includes the following 5-steps:
Establish the focus – either a traditional site-specific, or crash-based, approach or a systemic approach.
Identify the Network –Once the focus of the network screening process has been established (site specific or systemic), the next step is to identifying the network elements to be screened and organizing these elements into reference populations. Example roadway elements can include: intersections, roadway segments, facilities, ramps, ramp terminal intersections, and at-grade rail crossings. A reference population is a grouping of sites with similar characteristics (e.g. four legged signalized intersections, two lane rural highways). The ultimate prioritization of individual sites is made within that reference population.
Select Performance Measures – select Intersection safety can be quantitatively measured with performance measures. One or more performance measures can be used – using multiple performance measures may improve the level of confidence in the results. Examples of performance measures include average crash frequency, crash rate, Relative Severity Index, and excess proportion of specific crash types, among others.
Select Screening Method - In a network screening process, the selected performance measure would be applied to all sites under consideration using a screening method. These methods for screening the network can include using a simple ranking method for specific locations, such as intersections, or sliding window or peak searching methods for segments (or some combination of both node and segment screening methods).
Screen/Evaluate Results – The performance measure and screening method are applied to one or more of the segments, nodes, or facilities according to the methods outlined in Steps 3 & 4. For each segment or node under consideration, the selected performance measure is calculated and recorded. Results can be recorded in a table or on maps as appropriate or feasible. The sites higher on the list are considered most likely to benefit from countermeasures intended to reduce crash frequency. Further study of these sites will indicate what kinds of improvements are likely to be most effective
Screen/Evaluate Results
The first step, to establish the focus, is to select a traditional site-specific, or crash-based, approach or a systemic approach.

32. Activity 1
Select Focus Crash Type Select Focus Facility Type Identify Potential Risk Factors
Using the method presented in the FHWA Systemic Project Selection Tool the group will conduct an activity.
There is no rule on how many risk factors must be selected. At a minimum, you would need two to three risk factors to differentiate between sites. Of these, the presence of one or more fatal or serious injury crashes can be viewed as a risk factor. Selecting more—up to seven to ten—requires more time to perform the screening but also helps to determine the likelihood of future crashes.
There might be occasions in which individual risk factors do not appear to be overrepresented in the crash dataset. In these cases, you can perform the descriptive statistics analysis for combinations of risk factors to determine whether the related crashes are overrepresented. For example, one pilot agency’s crash data indicated that neither shoulder width nor shoulder surface type were risk factors for road departure crashes in horizontal curves. Engineering judgment suggested further analysis was required. Combining the data associated with these two risk factors revealed that severe crashes were found to be overrepresented in curves that had either narrow gravel shoulders or wide paved shoulders (which were also high volume corridors).

33. Potential Countermeasures
In this section we’ll discuss data needs for selecting countermeasures, resources, systemic considerations, and ped/bike systemic countermeasures.

34. Data Needs to Select Countermeasures
Crash, roadway, traffic data
Area type characteristics
Countermeasures
Effectiveness
Implementation & maintenance costs
Agency polices, practices, and experiences
Image: Non-motorized traffic volumes (Town of Duck, NC)
source: FHWA Non-Motorized User Safety (

35. Resources for Assembling the Comprehensive List of Countermeasures
FHWA PEDSAFE
FHWA BIKESAFE
FHWA Non-Motorized User Safety: A Manual for Local Rural Road Owners
Crash Modification Factors Clearinghouse
Highway Safety Manual
Agency experience / engineering judgment
There are also other sources available for all roadway users:
NCHRP Report 500 Series
Crash Modification Factors Clearinghouse
Highway Safety Manual
Strategic Highway Safety Plan
Intersection Safety Plans
Roadway Departure Improvement Plans
FHWA’s illustrated guide sheets and proven countermeasures
NHTSA’s Countermeasures That Work
Agency experience / engineering judgment

36. FHWA Proven Countermeasures

37. Potential Countermeasures
Why are certain countermeasures better suited for systemic analysis/application?
Examples of systemic countermeasures?
Why are certain countermeasures better suited for systemic analysis/application?
Two images will appear on “click”: on left is the traditional crash-based analysis with 3 hotspots identified and on the right are many locations identified through systemic analysis. As the systemic method is intended to address risk at many locations throughout a roadway network, countermeasures tend to be “lower-cost” in nature.
On click a question (Examples of systemic countermeasures?) will appear. Ask the group to provide examples of “lower-cost” countermeasures. (The next slide will provide an example of one State’s pre-selected list of ped/bike systemic countermeasures). Pedestrian Hybrid Beacons are one countermeasure that many agencies have debated if it is suited to systemic projects due to it’s cost.

38. Bicycle Systemic Countermeasures
Bike signal
On-street bike facilities (bike lanes or separated bike lanes)
Bicycle pavement markings (shared lane markings)
Advanced bicycle warning signs
Flashing beacons on advance warning signs
Lighting
This listing includes examples of bicycle systemic countermeasures used by one State (from Virginia HSIP program). The next slide includes example pedestrian countermeasures.
Image Source: PBIC/WABA (

39. Pedestrian Systemic Countermeasures
Enhanced crosswalks
Remove parking near intersection (daylighting)
Advanced pedestrian warning signs
Flashing beacons on advance warning signs
Median refuge
Curb extensions
Pedestrian countdown signals
Lighting
This listing includes examples of bicycle systemic countermeasures used by one State (from Virginia HSIP program). The next slide includes example pedestrian countermeasures.
Image Source: FHWA (

40. Helpful Hints
Remove initial countermeasures that are not feasible from consideration prior to workshops or meetings
Seek input from stakeholders during screening process
There is no optimum number of countermeasures
There is no rule on how many risk factors must be selected. At a minimum, you would need two to three risk factors to differentiate between sites. Of these, the presence of one or more fatal or serious injury crashes can be viewed as a risk factor. Selecting more—up to seven to ten—requires more time to perform the screening but also helps to determine the likelihood of future crashes.
There might be occasions in which individual risk factors do not appear to be overrepresented in the crash dataset. In these cases, you can perform the descriptive statistics analysis for combinations of risk factors to determine whether the related crashes are overrepresented. For example, one pilot agency’s crash data indicated that neither shoulder width nor shoulder surface type were risk factors for road departure crashes in horizontal curves. Engineering judgment suggested further analysis was required. Combining the data associated with these two risk factors revealed that severe crashes were found to be overrepresented in curves that had either narrow gravel shoulders or wide paved shoulders (which were also high volume corridors).
Image source:
Pedestrian:
Right-hook:
Bicyclist:
Speed limit:

41. Evaluating Effectiveness

42. Why Evaluate Effectiveness
Input into systemic planning process
Proof of effectiveness generates support
Addresses agency responsibility to invest resources effectively
Focus on before vs. after crash statistics
Guidance for interpreting results
For both the FHWA Systemic Safety Project Selection Process and the HSM Roadway Safety Management Process, evaluating program performance is the last step in the overall process. It provides useful feedback into the safety management process, for both the site analysis and systemic approaches. Quantifying effectiveness is critical to generate support for the systemic approach and to build institutional and cultural support to invest funding for this type of analysis and implementation. Evaluating effectiveness also addresses an owning agency’s responsibility to invest resources in a way that best serves the traveling public and builds confidence that a systemic program is a worthwhile investment.
Incorporating systemic safety countermeasures on focus facility types can be accomplished during planning, design, operations, and maintenance projects, not just through dedicated, standalone safety projects. Through these additional implementation channels for safety improvements, the systemic approach reaches more locations in less time than safety funding alone accomplishes. Sharing the benefits of the approach helps all agency offices understand the justification for adopting changes and the results expected (such as fewer severe crashes) if traditional practices are modified to incorporate safety planning.
Performance evaluation results, especially lives saved and injuries prevented, can be compelling information to bring about changes in business practices. A data-driven analysis of the effectiveness of systemic safety improvements can provide evidence for use in presentations to elected officials and citizens that promote an overall understanding for why it is beneficial to incorporate systemic planning into a safety program.
Tracking changes in crashes can provide information to generate crash modification factors, which can then be shared with other agencies to inform safety planning efforts and further the ability to reduce crashes nationally. Developing systemic program-specific CMFs is preferable to using site-specific project CMF’s for systemic program analysis and can be especially useful by offices that are considering incorporating systemic countermeasures into planning, design, or operations/maintenance level projects.
Evaluation effectiveness also addresses an owning agency’s responsibility to invest resources in a way that best serves the traveling public and will help these agencies have confidence that a systemic program is a worthwhile investment. If results indicate that a particular countermeasure or set of countermeasures is not lowering the potential for crashes or reducing crashes, then they can be modified or eliminated from future projects.

43. Evaluating Effectiveness
There are two primary levels for evaluating effectiveness:
Project-level: impacts of each individual countermeasure or the average impact of combined countermeasures (new CMFs).
Program-level: number and rate of crashes, injuries, and fatalities on the network; specific programs such as intersection, roadway departure, and pedestrian safety.
There are two methods to evaluate project effectiveness. A few agencies are in the early-stages of evaluating effectiveness of systemic projects but, due to the recent adoption of this type of analysis methodology, not many have evaluated the effectiveness of systemic projects.
As systemic projects are intended to reduce the total amount of crashes or shift proportion of crash types and severities, a program-level analysis

44. Systemic Program Evaluation is System-wide
Addresses locations with no crash history
Maximizes data sample size
Countermeasure-based
The systemic program evaluation element of the Tool adapts processes traditionally applied for evaluating effectiveness of individual safety improvement projects. However, a systemic program evaluation does not evaluate changes in crash frequency for specific locations. Because of the focus on high risk locations that may or may not have a crash history, systemic improvement evaluation is performed collectively for all sites where improvements were implemented.
This “roll up” to the system level recognizes that, in the systemic approach, some deployment might take place at locations with a history of no or few crashes, which means that change in crash frequency at specific locations does not sufficiently tell a story about effectiveness. Also, a system-level evaluation maximizes the data sample size, which will provide the greatest chances for statistical reliability, which is highly influenced by sample size.
Evaluation is countermeasure-based to determine effectiveness of specific improvements, to assist with determining if they should continue to be implemented.
Image source:

45. Data Needs for Systemic Program Evaluation
Before and After Data
Macro level = statewide, regionwide, and system- wide crash and roadway characteristics
Micro level = project-specific data about type, location, implementation date
Data required to evaluate the systemic portion of an agency’s safety program depends upon the level of analysis – system-wide versus improved locations. Evaluating improved locations requires the crash, roadway, and traffic data assembled during the Systemic Safety Planning Process (Element 1) and for a minimum of three years after implementation, and details about the implementation of specific systemic safety countermeasures. Data required to perform system-wide evaluations include statewide, regionwide, and system-wide crash and roadway data within the study area.
The countermeasure-specific data focus on the actual sites where projects were implemented and include key, descriptive information about the project type, a detailed definition of the location, and documentation of when specific projects were implemented.
Crash data would likely be documented on a site-by-site basis. These data should ultimately be “rolled up” to represent the entire system along which a particular countermeasure was deployed to provide the greatest chances for statistical reliability.
A minimum of three years of crash data should be accumulated after implementation to attain a sufficient sample of crashes to attain a sufficient sample of crashes. Research regarding the statistical reliability of computed CMFs indicates that using at least three years of “Before versus After” study data accounts for regression to the mean as well as more complex statistical analyses.
Image: The Regional Transportation Commission (RTC) of Washoe County has implemented many Road Diets within the City of Reno to allow for the addition of bicycle lanes. These projects were created as part of the Complete Street initiative to stimulate economic development and improve citizens' quality of life. The RTC has been proactive in educating the public during the entire process of implementing Road Diets. Once projects are complete, the RTC also publicizes the annualized crash rates1 for the road segments which have undergone the Road Diet treatment. This increases the public's understanding of the safety benefits.
Source: FHWA (

46. Recap
The systemic method allows practitioners to proactively address safety risks throughout transportation network.
This method is not dependent on crash locations so it’s appropriate for bicycle/pedestrian analysis.
Systemic countermeasures are lower cost and can be widely deployed.
Tools are available, such as the FHWA Systemic Project Selection Tool and the AASHTO Highway Safety Manual.
Key takeaways:
The systemic method allows practitioners to proactively address safety risks throughout transportation network.
This method is not dependent on crash locations so it’s appropriate for bicycle/pedestrian analysis.
Systemic countermeasures are lower cost and can be widely deployed.
Tools are available, such as the FHWA Systemic Project Selection Tool and the AASHTO Highway Safety Manual.