1. NC Local Safety Partnership Evaluation Methods

2. Workshop Roadmap Program Background and Overview Crash Data Identifying Potential Treatment Locations Preparing Collision Diagrams Selecting Interventions Evaluation Methods Program Implementation and Discussion 2

3. Module Objective ￭ Understand why evaluation is important ￭ Learn about the two basic kinds of evaluation methods ￭ Learn how to perform a simple before- after analysis 3

4. How well has a treatment, project, or group of projects reduced crash frequency or severity? Safety Effectiveness Evaluation Basics Evaluation 4

5. Safety Effectiveness Evaluation Basics Why do safety effectiveness evaluations? ￭ Prove investment effectiveness ￭ Demonstrate program value to decision makers (accountability) ￭ Contribute new scientific knowledge ￭ Improve decisions ￭ Optimize future safety investments 11-5 8-5

6. Results in Winston-Salem MetricBeforeAfterChange Targeted Crashes8,5024,078-51% Total Crashes18,65915,129-18% Injuries11,0587,578-31% Property Damage$52.3 mil.$41.9 mil.-19% ￭ Result of 858 Low Cost Treatments in Winston-Salem over past 25 years 6

7. ￭ Goal – Measure true effect of a countermeasure ￭ We want to be sure that the observed change is due to the countermeasure alone ￭ What other factors could cause the change? Safety Effectiveness Evaluation Basics 7

8. ￭ What other factors could cause the change? ￭ Other “treatments” at the same time (e.g., primary seat-belt law at the same time as adding a protected left-turn phase to intersections) ￭ Changes in AADT ￭ Regression to the mean ￭ Underlying trends in crashes (e.g., economy- related changes) ￭ Others ￭ So how do we control for/discount these other “causes”? 8

9. Two Basic Evaluation Study Designs 1. Before-after studies 2. Cross-sectional studies Choice of method is affected by: Nature of treatment Site type Available data 9

10. Before-After Study ￭ Examines crash data before and after the treatment is installed ￭ Types of before-after studies ￭ Simple before-after study ￭ Does not account for certain biases ￭ Before-after study with reference/comparison groups ￭ Accounts for changes in volumes and other factors 10

11. Simple Before-After Study Estimated average expected crash frequency without treatment Measured Is this assumption realistic? TREATMENT ASSUME these are the crashes WITHOUT TREATMENT 11

12. Before-After Study with Reference/Comparison Group Expected average crash frequency without treatment Measured Expected average crash count without treatment TREATMENT 12

13. Before-After Study with Reference/Comparison Group Estimated average expected crash frequency without treatment Measured 13

14. Cross-Sectional Study ￭ Compare crash data for sites with and without treatment over same time period 14

15. Cross-Sectional Study ￭ Why do a cross-sectional study? ￭ Treatment installation dates unknown ￭ Volumes and crash counts in before period unknown 15

16. Evaluation Study Type Selection Guide Evaluation Method Treatment SitesNontreatment Sites Before Data After Data Before Data After Data Simple Before-AfterXX Before-After Using Reference/Comparison Group XXXX Cross-Sectional StudyXX 16

17. EXAMPLE SIMPLE BEFORE- AFTER EVALUATION Module 6 – Evaluation Methods 17

18. Example Simple Before-After Evaluation ￭ Traffic Signal Installation in Wake Co. ￭ NCDOT Safety Evaluation Unit conducted before-after evaluation ￭ Intersection of SR 1004 (East Garner Road) and SR 2555 (Auburn - Knightdale Road) ￭ Before: Two-way STOP-controlled ￭ After: Traffic signal (actuated) 18

19. Example Simple Before-After Evaluation 19

20. Example Simple Before-After Evaluation ￭ Signal installed in October 2003 ￭ Before period: Nov 1998 – Sept 2003 ￭ After period: Dec 2003 – Sept 2008 20

21. Example Simple Before-After Evaluation BeforeAfter Percent Reduction (-) Percent Increase (+) Total crashes4010-75% Target Crashes345-85% Fatal injury Crashes00N/A Class A injury Crashes20-100% Class B injury Crashes70-100% Class C Injury Crashes94-56% Total Injury Crashes184-78% Total Severity Index7.753.96-49% Target Crash Severity Index8.723.96-55% Volume8,8009,3006% 21

22. Example Simple Before-After Evaluation 22

23. Example Simple Before-After Evaluation 23

24. EXAMPLES FROM WINSTON- SALEM EVALUATIONS Module 6 – Evaluation Methods 24

25. Before and After Collision Diagrams 25

26. 26

27. Summary 27

28. Graph of Poisson Distribution 28

29. 29

30. 30

31. 31

32. 32

33. Take Away Messages ￭ Evaluation helps justify investments and improves decision making ￭ Pros and cons of different types of evaluation ￭ Steps for conducting simple before/after evaluation 33