1. Second U.S. Roadway Safety Data Capabilities Assessment2
Second U.S. Roadway Safety Data Capabilities Assessment
Bob Scopatz, VHB

2. First Capabilities Assessment
2012
First assessment conducted in
Specific data quality improvements
Improvements to data governance
Data integration to support data driving decision making
The first Capabilities Assessment was conducted in 2012 and showed where States shared common needs. Some of these needs included specific data quality improvements, data governance in general, and data integration to support data driving decision making. In 2012, many of the tools for data integration and analysis were new or not well established, and data governance was also a new concept for many safety practitioners.

3. Objectives
Develop and carry out consistent, repeatable, systematic process Understand what State’s capability goals are, identify gaps, solutions, and available funding to achieve data goals Identify the most effective ways FHWA can support the State’s efforts to improve their data, analysis, and decision making
The purpose of this second-round assessment was to conduct a consistent and thorough roadway safety data capabilities assessment and record goals for each State. The project had three primary objectives: (1) to develop and carry out a consistent, repeatable, and systematic process for working with the States to assess their roadway safety data capabilities; (2) to understand what State’s capability goals are, and to help them identify critical gaps, potential solutions, and available funding sources to achieve their data goals; (3) to identify the most effective ways FHWA can support the State’s efforts to improve their data, analysis, and decision making.

4. 2017 – 2018 Assessment 2017-18 Second assessment conducted in
Designed to place less burden on States
Gathered info and pre-filled questionnaires
Reviewed results with State
Scored questionnaire using a Capability Maturity Model (CMM)
Created an Action Plan for each State
FHWA designed the second-round capability assessment to place less of a burden on States that the first-round assessment. The assessment teams gathered information from the States from January to December of 2018 using existing sources and pre-filled most of the questionnaires before meeting with a State. States reviewed the questionnaires and made edits before reviewing for a final time with the consultant teams. The assessors scored the final questionnaire answers using a Capability Maturity Model (CMM) process, giving each State an objective review of their capabilities. States met with their assessment team and decided on goals for the are of the CMM and discussed the best ways to meet those goals. The final meeting, conducted in 2018 and in January/February 2019 for a few States, resulted in an Action Plan for each State that shows status, goals, and methods of reaching those goals.

5. Focus Areas
Area 1 Roadway Safety Data Collection/ Technical Standards Area 2 Data Analysis Tools and Uses Area 3 Data Management and Governance Area 4 Data Integration Area 5 Performance Management
The assessors reviewed the completed questionnaire and determined the CMM level for each of the elements in areas 1 through 5. The team used a scoring method for consistency which included five areas:
Area  1: Roadway Safety Data Collection/Technical Standards, which was developed by two types of capability scores and addressed inventory, traffic volume, and crash data. States provided information on the completeness, timeliness, accuracy, and uniformity of their safety data, and received scores based on these quality attributes and management of the data.
Area 2: Data Analysis Tools and Uses focused on how well States are implementing advanced analytic methods and using the various tools that support those methods. This included gathering information from States on their network screening, diagnosis, countermeasure selection, evaluation, and accessibility.
Area 3: Data Management and Data Governance focused on States’ data management and data governance roles and responsibilities, policies, and processes.
Area 4: Data Integration focused on integration among the “six pack” of crash, roadway, driver, vehicle, citation and adjudication, and injury surveillance.
Area 5: Performance Management, which was a new to the second-round assessment, focused on States’ safety performance management. States received scores in the sub-areas of performance measurement data, process, and analysis capabilities.

6. After the Assessment After Assessment
Action Plan development Peer Exchanges
The pre-assessment process included a kickoff call and sending the pre-populated questionnaire to the State. Four to six weeks in advance of assessment, the lead assessor identified a State contact list and ed the State’s point of contact to begin the assessment scheduling by conducting a kickoff call. The kickoff conference call was held with the State’s primary contact and invited additional key data managers to attend. Following the meeting, the lead assessor sent the pre-populated questionnaire to the State for their review and to add in any missing information two weeks prior to the assessment day.

7. Key Findings: Data Collection
For each of the four data quality attributes (completeness, timeliness, accuracy, and uniformity) each State received two sets of scores that reflect the perceived level of data quality achieved. For data collection, on average, States set goals to improve both data quality and their data management practices.
Nationally, the assessment team also found that perceived data quality is consistently higher than the States’ capabilities for managing data quality. States are implementing automates methods for collecting roadway inventory data. Field data collection systems for crash reporting are common as well.
Intersection inventory systems lag the segment-oriented data; however, this is improving due to GIS-based automated methods for creating an intersection inventory.
Finally, data quality measurements are more robust for traffic count and crash data than for roadway inventory data.

8. Key Findings: Data Analysis
The national average is high for each facet of safety analysis and States are using advanced methods for network screening and diagnosis. Nationally, countermeasure selection is lagging.
Some highlights from the results included; States are using data analysis tools; however, the tools vary from State to State. Some States use AASHTOWare Safety Analyst while others are creating their own custom tools. Another highlight is States are using the CMF Clearinghouse, but they are not adopting the methodology in all relevant cases, particularly for local roads, which is why the scores lag for countermeasure selection. States are using GIS to integrate spatial data, as you can see in Area 4, which supports more efficient use of advanced analytic methods and tools.

9. Data Management & Governance
Key Findings:
Data Management & Governance
Compared to the 2012 assessment, scores in data governance and management slipped. This indicates that States have an understanding of what data governance is and how far their State is from optimal performance. In addition, States are engaged in formal data governance, but few are at the highest level on the CMM. Enforcing data standards is also difficult. States are recognizing the need but have are challenged when deciding how best to work with local agencies. Another confounding issue is States often do not own the systems they rely on to do their jobs and the data system documentation is incomplete, which makes it difficult to establish and enforce data standards.

10. Key Findings: Data Integration
GIS is being used as a key tool for data integration
Business units are more concerned with expanding data integration beyond crash data
States are interested in expanding all forms of data integration
Looking at the results for data integration, the scores for spatial data integration are the highest, which is a result of expanded GIS implementation for safety data integration and analysis. While States are using GIS as a key tool for data integration of roadway inventory, traffic, and crash data, they are lagging in integrating driver, vehicle, citation and adjudication, and injury surveillance. While States are lagging in some areas, business units outside the State DOT safety engineering group are interested in expanding data integration beyond crash, roadway, and traffic data. States are also interested in expanding data integration, which is a major change from the first-round capabilities assessment.

11. Performance Management
Key Findings:
Performance Management
The lowest score in this focus area was performance management analytic capabilities and the highest was for data coordination. This reflects the benefits of States’ efforts to improve data for decision making. The assessment team was unable to make comparisons from the first-round capabilities assessment because this was a new focus area.

12. National Gaps Summary
This figure provides a side-by-side comparison of the national average capability scores for all sections of the assessment. The lowest scores are in data quality management, even though States scored relatively high on data quality. Other low scores include countermeasure selection, data management and governance policies roles and responsibilities, and in performance management analysis capabilities. The inverted triangles show the corresponding results reported in the first-round safety data capabilities assessment. There are no triangles for the performance management scores because they were not collected in the first round.

13. Peer Exchanges
June 3-5, 2019, Phoenix, AZ Data Collection and Quality Management June 17-19, 2019, Montgomery, AL Data Governance and Data Integration September (TBD), 2019, via webinar Data Analysis

14. Conclusions Assessment identifies key areas States need assistance
Focused approach category for data improvements
Value in coordination with local agencies
This assessment will be useful to FHWA in refining and expanding the resources available through the RSDP website, promote States’ use of the information, guidance, technical assistance, and tools linked there. Ultimately, FHWA should tailor the approach to each State, through consideration of a “focused approach” category for data improvements or other methods, to implement their Roadway Safety Data Action Plans in alignment with their safety goals outlined in their Strategic Highway Safety Plan. Parallel work has also uncovered that States are finding value in coordinating with local agencies on data collection and performance management; however, States will need help and assistance involving local agencies in data governance.

15. Bob Scopatz Esther Strawder?
Questions?
Bob Scopatz
Esther Strawder