1. Scientific Principles for Effective Road Safety Analysis
Apply scientific principles to decision-making regarding road safety problem identification, countermeasure selection, and evaluation.
In this module, we will begin to explore scientific methods for analyzing crash factors, such as location, severity, etc. This information will enable you to ask better questions and understand how investment decisions are, or should be, made.
NCHRP 17-40, June 2010

2. Major Topics Rational vs. Pragmatic Style of Road Safety Research
Regression-to-the-Mean
Exposure Data
Safety Performance Functions
With-Without Rather than Before-After
This module seeks to have students understand the importance of using rigorous scientific procedures to identify and solve road safety problems. Use of unscientific ad-hoc methods can identify the wrong problems, involve the wrong entities (i.e. locations, driver groups, vehicle groups), and result in ineffective use of scarce safety resources. Road safety problems are defined as entities (e.g. road segments, intersections, vehicles, and drivers) identified as having an unacceptable level of crash risk. Scientific methods are based on analysis of safety metrics using methods accepted by the safety research profession. This may seem problematic at first; a plethora of methods exist and many appear to be contradictory. Fortunately, major safety initiatives are underway to define scientific methods (e.g. Highway Safety Manual or HSM, Human Factors Guide to Safety, NCHRP 17-33: Effectiveness of Behavioral Highway Safety Countermeasures). Fundamental to the use of these methods is dependence on measures of crashes, injuries, or fatalities rather than surrogates or substitutes. While other metrics may be useful in analyzing some aspect of the problem, it is strongly preferred to use “safety methods” which use crashes, injuries, and fatalities.
This module introduces students to the importance of using suitable scientific methods in the analysis of road safety problems. The discussions include: rational vs. pragmatic style of road safety research; regression-to-the-mean; applying exposure data; the use of safety performance functions for comparison, and with-without rather than before-after.
Major topics include:
Rational vs. Pragmatic Style of Road Safety Research
Regression-to-the-Mean
Exposure Data
Safety Performance Functions
With-Without Rather than Before-After
NCHRP 17-40, June 2010 1

3. Rational vs. Pragmatic Styles
Data and analysis
Factual experience
Estimates based on the effect on safety
Pragmatic
Lay beliefs
Personal anecdotal experience
Meeting standards means “safe”
In an important paper describing the evolving style of road safety research, Ezra Hauer (2002) points out the movement from a pragmatic style to a rational style of road safety management. The management practices of the past are contrasted with those currently evolving and likely to continue to evolve in the future. The past style is viewed as pragmatic because it is based on lay beliefs and personal anecdotal experience. The future is described as rational because it is founded on understanding the expected consequences of actions based on factual experience and involves learning from experience. The pragmatic style is exemplified by portions of the Manual of Uniform Traffic Control Devices (MUTCD) and the AASHTO Green Book. The assumption is that meeting the standard is necessarily safe. As an example, Hauer cites the determination of the radius of a horizontal curve, the choice of which is based at least in part on local geographic conditions. Designers know that a longer radius is safer, but do not know the nature of the relationship between radius and expected number of crashes. The designer is thus unable to quantify benefits of alternative radii compared to cost. This situation is changing as we speak with the development of the Interactive Highway Safety Design Manual (IHSDM), but strict use of the Green Book does not support tradeoffs.
Another example in the behavioral area is the continued use in some behavior change efforts of scare tactics based on their appeal to human emotions, even though studies have since raised questions about the effectiveness of such tactics.
The rational style emphasizes estimates of the effect on safety (data and analysis), not adherence to standards based on personal experience, beliefs and intuition.
NCHRP 17-40, June 2010 2

4. Regression-to-the-Mean (RTM)
It is tempting to characterize the safety of an entity as the number of crashes for one time period only (overwhelmingly a year). This temptation is to be avoided given random crash variation and the concept of “regression-to-the-mean” (RTM) as commonly discussed in safety literature. RTM results in entities having a relatively high number of crashes one year, and then a drop in the next year.
This phenomenon was observed as early as 1877 in a study by Galton in which, “. . . the offspring of tall parents are generally found to be shorter than their progenitors (and vice versa)”. In this sense the heights tend to move or “regress” back toward the underlying mean, e.g. average height. The specific concern in road safety is that one should not select sites for treatment based on a high count in one year because the count will tend to “regress” back toward the mean in subsequent years.
This fluctuation must be considered at two critical points in safety analyses: 1) identifying the best entities for investment; and 2) evaluating effectiveness of the action. If the investment is initiated when the crash count is randomly high, the determination of the expected number of crashes prior to implementing a countermeasure is artificially inflated. Any effectiveness measure derived from the difference between this baseline and the mean number of crashes after implementation will be artificially and wrongly inflated. As a result the accident modification factor or other such improvement estimate derived after the investment is wrong; perhaps even opposite in direction as well as incorrect in magnitude.
Let’s explore this idea using a specific example demonstrated in a classic paper by Hauer in 1986.
NCHRP 17-40, June 2010 3

5. Table 1.1 2014-2015 San Francisco Data
Number of
Intersections
Number of Accidents Per Intersection In 2014
Average Number of Accidents Per Intersection in 2015
553
0.54
296 1
0.97
144 2
1.53 65 3
1.97 31 4
2.10 21 5
3.24 9 6
5.67 13 7
4.69 8
3.80
6.50
Using data from 1,142 San Francisco intersections, counts of crashes in each of several years are compared. For each year, the sites are grouped and ranked based on the number of crashes experienced in a given year.
The first column shows the number of intersections in a given year with the number of crashes experienced at each group of intersections shown in column two. The last column indicates the mean number of crashes per intersection in the following year (in this case, 2015). The mean is calculated as the sum of the number of crashes at a site divided by the number of years of observation (in this case, divided by 1.0). The mean is used as a measure of central tendency or as an indication of an underlying value in multiple observations of an event (such as crashes at a location or the number of fatalities per year involving unbelted drivers).
The use of a one-year count as an estimate of the underlying expected number of crashes implies, for example, that the group of intersections that had 4 crashes in 2014 is expected to have generally the same number in 2015, but those same 31 intersections had only an average of 2.10 crashes in the next year.
NCHRP 17-40, June 2010 4

6. Table 1.1 2014-2015 San Francisco Data
Number of
Intersections
Number of Accidents Per Intersection In 2014
Average Number of Accidents Per Intersection in 2015
553
0.54
296 1
0.97
144 2
1.53 65 3
1.97 31 4
2.10 21 5
3.24 9 6
5.67 13 7
4.69 8
3.80
6.50
Discrepancies are found for all other accident counts except for the 296 intersections that experienced 1 crash in 2014 and experienced an average of 0.97 crashes in 2015.
Also note how the average number of crashes at the 553 intersections experiencing 0 crashes in 2014 went up in 2015, and all the average number of crashes at intersections with more than 1 crash in 2014 went down in all “regressing” back toward 1, which turns out to be the average number of crashes for all 1,142 intersections over a three year period.
NCHRP 17-40, June 2010 5

7. Table 1.2 1 2015-2016 San Francisco Data
Number of
Intersections
Number of Accidents Per Intersection In 2015
Average Number of Accidents Per Intersection in 2016
559
0.55
286 1
0.98
144 2
1.41 73 3
1.82 35 4
1.97 18 5
2.50 11 6
3.91 9 7
4.22 8
2.00
3.00 10
5.00
This slide shows the same 1,142 intersections and the new breakdown of crashes in 2015 along with the averages for those same groups in 2016.
Do you see the same trend toward an average of 1 crash per year?
NCHRP 17-40, June 2010 6

8. Table 1.3 2016-2017 San Francisco Data
Number of
Intersections
Number of Accidents Per Intersection in 2016
Average Number of Accidents Per Intersection in 2017
562
0.53
287 1
0.94
155 2
1.37 74 3
1.72 33 4
2.61 13 5
3.00 11 6
2.64 7
2.25 8
1.00 9
3.50
Let’s continue to 2017.
The trend is clear; the count in any one year is a poor indicator of the count expected in subsequent years, with the exception of the count close to the underlying mean number of crashes for the entire data set.
The data indicate intersections that experience a given number of crashes in one year are very likely to experience a change in the next year. The random variation in crash counts is also common in injury and fatality statistics. The point of the exercise is that the count in one year is a poor indicator of the count to be expected in the second year. This variation, or “regression-to-the-mean”, argues for computation of the mean for each site over several years rather than just the number of observations for a year.
Even if one accepts this guidance, question remains about how many years? Rule of thumb says at least three, but this is not based on science, it is more a convention of practice. It is sometimes difficult to wait if data are not available for multiple years while crashes continue to occur, so we need a way to enhance the estimate for sites with few years of data. We will discuss later some statistical methods for supplementing information to obtain a more useful mean estimation.
The important point here is that we do not want to invest resources into fixing a road segment based on one year of relatively high crash data given that it may be a random spike that would naturally come down through RTM. We also do not want to evaluate a high visibility speed enforcement campaign based on one year of crash data as it may inflate the measure of effectiveness given the potential for the number of speed related crashes to drop without the campaign but from random variation. On the other hand the potential exists to miss a particularly dangerous site usually experiencing a relatively high number of crashes, but which experiences a random drop in the one year observed due to some unknown factors.
NCHRP 17-40, June 2010 7

9. Applying Exposure Data
Exposure – the amount of travel that underlies any observed number of crashes
Crash rate
Crashes/VMT
Fatalities/100 million VMT
Injuries/capita
Exposure should be related to crashes by a function
Safety analysts have long recognized the need to represent the amount of travel that underlies any observed number of crashes. This is commonly referred to as “exposure”. Exposure has also been referred to as reflecting intensity of use. The most common practice is to divide the number of crashes (or injuries/fatalities) by an exposure measure (e.g., vehicle miles traveled or VMT) to produce a crash rate. In public health fields it is common to use population or number of licensed drivers as exposure measures.
However, using crash rates has proven to be a questionable practice for roadway applications. The crash rate should not be used to compare countermeasure effectiveness but can be used to assess the risk imposed on individual users. The road is safer per road user as the risk of a crash for an individual user has dropped because there are more users.
Exposure is still a critical metric for safety analysis. We cannot ignore the relationship between vehicle miles traveled or the number of drivers on the road and the number of crashes. However, Hauer argues that, rather than simply dividing one number by another, exposure should be related to crashes by a statistical function.
A “Safety Performance Function” (SPF) is a relatively new statistical model that allows us to relate crashes and exposure to estimate the expected number of crashes (or fatalities/injuries) for any specific site.
NCHRP 17-40, June 2010 8

10. SPF for Countermeasure Evaluation
1.2 X
X X
X X X
X X
1.0
X X
X X
Crashes per
unit time X
0.6
XXX XX
Let’s take a general look at what goes into a SPF and what one actually looks like.
This slide shows a hypothetical plot for 4 similar entities (i.e., two-lane, four-way, signalized, rural intersections) in State X. As discussed earlier, it is important to have multiple years of data and in this case we are fortunate to have 6 years of data for each entity (which is what makes hypotheticals so nice to work with). X
0.4 X
XX XX
0.2
Annual Driving Miles
1000
2000
3000
4000
5000
Safety 101
NCHRP 17-40, June 2010 9

11. SPF for Countermeasure Evaluation
1.2 X
X X
X X X
X X
1.0
SPF
X X
X X
Crashes per
unit time X
0.6
XXX XX
A curve is fitted through these 4 entities using a negative binomial regression formula.
This line is the Safety Performance Function (SPF) and it represents the change in the mean expected number of crashes at all similar entities as ADT (or other exposure measure) increases, while all other factors affecting crash occurrence are held constant. SPFs are typically based on a far greater number of sites for greater accuracy, but we are using just four to illustrate the basic concepts. Keep in mind that it is almost never possible to control for all other factors.
Again, State X has developed this Safety Performance Function for all two-lane, four-way, signalized, rural intersections. Note, other than some roadway features, we have not yet identified any contributing crash factors. We will see in subsequent modules this basic SPF, once established, will be useful throughout problem identification, countermeasure selection, and evaluation. X
0.4 X
XX XX
0.2
Annual Driving Miles
1,000
2,000
3,000
4,000
5,000
Safety 101
NCHRP 17-40, June 2010 10

12. With-Without Rather than Before-After
SPFs show how changes in exposure alone can affect crash frequency or rates at any given entity
Before-after does not account for changes in exposure and can lead to inaccurate measures of effectiveness
With-without considers changes in exposure for more accurate problem identification, countermeasure selection, and evaluation
The SPF shows how RTM or changes in exposure alone can significantly impact crash frequency or rates, depending on what has been plotted on the y axis.
This raises serious questions about the traditional before and after studies for measuring effectiveness of safety projects.
These studies looking only at crash frequency/rate changes before and after countermeasure implementation do not account for changes in safety that may occur even in the absence of any intervention due to RTM or changes in exposure. While such information may anecdotally be added to a discussion on safety impacts, it still limits the ability to fully understand whether or not the countermeasure itself was responsible for any safety improvements and to what degree.
What we need is a method to compare the expected number crashes in the after period, with the number expected in that time period if the intervention had not been undertaken. In the literature, this is referred to as a comparison of with-without rather than before-after.
The answer lies in the construction of the SPF. Remember the SPF is defined as the change in the expected number of crashes as ADT (or other exposure measure) increases, while all other factors affecting crash occurrence are held constant. Thus if we know the ADT in the after period, we can use the SPF to produce an estimate of the expected number of crashes with the change in ADT, without the intervention being implemented. This would give us an estimate of the number of crashes expected in the after period if nothing were done at all.
This provides a better basis for comparison to the number or rate with the countermeasure and leads to more accurate safety analysis.
NCHRP 17-40, June 2010 11

13. Review Rational vs. Pragmatic Style of Road Safety Research
Regression-to-the-Mean
Exposure Data
Safety Performance Functions
With-Without Rather than Before-After
Several important and new concepts need to be understood concerning the use of scientific methods in road safety analysis. One is we have been gradually evolving from a pragmatic safety management style stressing anecdotal knowledge and occasional observation to a rational style founded on understanding the expected consequences of actions on safety based on analysis of data and learning from experience.
We may be tempted to characterize the safety of an entity as the observation of the crashes for one time period only (overwhelmingly a year). This temptation is to be avoided to accommodate year-to-year variability of crash frequencies. Year-to-year variability is an example of regression-to-the-mean. Because of RTM, we need to characterize crashes with a mean and some notion of variability.
Crash rate should not be used to compare countermeasure effectiveness but can be used to assess the risk imposed on individual users. The road is safer per road user as the risk of a crash for an individual user has dropped because there are more users. It is important to relate exposure to crashes by a function, not by simply dividing one number by another.
The Safety Performance Function is the change in the expected number of crashes as ADT (or other exposure measure) increases, while all other factors affecting crash occurrence are held constant.
The SPF can be used to estimate the number of crashes expected without a countermeasure. This should be compared to the number with the countermeasure. The concept of with-without comparison should replace the concept of “before-after”.
NCHRP 17-40, June 2010 12