Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Improving Accuracy of KABCO Injury Severity Assessment Using Classification Trees Beau Burdett, Research Assistant Zhixia (Richard) Li, Researcher Andrea R. Bill, Researcher and David A. Noyce, PhD, PE, Professor and Chair October, 2015

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Audience  Engineer?  Law Enforcement?  Behavior?  IT?  Other?

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Background: At the scene of a crash law enforcement officers assess the injury severity of each crash victim using the KABCO scale o Used for safety analyses of projects, allocation of funds MAP-21 requires serious injuries to be used for analyzing highway safety Crash Outcome Data Evaluation System (CODES) contains MAIS (actual health outcomes) and KABCO (law enforcement officer injury severity assessment) o Can be used to compare KABCO and MAIS injury severity scales Introduction & Background Research Objectives MethodologyResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Law Enforcement Officer: KABCO o “K” - Fatality o “A” – Incapacitating Injury o “B” – Non-Incapacitating Injury o “C” – Possible Injury o “O” – Property-Damage-Only Crash “KA” considered serious Introduction & Background Research Objectives MethodologyResultsConclusions  Medical Practitioners: MAIS (Maximum Abbreviated Injury Score) o 1: Minor o 2: Moderate o 3: Serious o 4: Severe o 5: Critical o 6: Maximum (fatality) 3+ considered serious

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Comparing KABCO and MAIS Popkin et al. (1991) o 51% of crash reports had a discrepancy between KABCO ratings – 12% of 2 or more levels o Police over report severity when: – Pregnant or intoxicated crash victim o Overestimated when “superficial” wound o Underestimated when “occult” injuries Farmer (2003) o 46% coded as “Incapacitating” only had minor injuries – 3% had no injuries at all o Females had higher overestimation than males o Drivers had higher overestimation than 65+ Introduction & Background Research Objectives MethodologyResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Comparing KABCO and MAIS Compton (2005) o Officers did good job determining injury severity – 65% of incapacitating labeled as AIS 2+ – Overestimation came from superficial injuries – Underestimation from occult injuries Flannagan et al. (2012) o KABCO was strongly associated with serious injury – Higher KABCO rating more likely serious injury – Correlation between “KA” and MAIS 3+ » “KA” overestimated serious injury 3x » Using “KA” + hospitalization improves overestimation to 21% Introduction & Background Research Objectives MethodologyResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Comparing KABCO and MAIS TOPS Lab Research on Injury Severity Discrepancies (2014) o High levels of discrepancy between assessments – 67% of “A” crashes overestimated (MAIS 1 or 2) – 12% of “B”,”C”, and “O” crashes underestimated (MAIS 3+) o Most common injuries noted by officers of crash reports with injury severity discrepancies – Bone Injuries (Specifically broken bones) – Lacerations – Abrasions – Contusions Introduction & Background Research Objectives MethodologyResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Introduction & Background Research Objectives MethodologyResultsConclusions  Comparing KABCO and MAIS TOPS Lab Research on Injury Severity Discrepancies (2014) o Injured body regions identified by officers on crash reports – Areas with obvious injuries more frequently overestimated – Abdomen/pelvic region frequently underestimated o Body regions with internal injuries are missed by officers in nearly 50% of crash victims o Alcohol, gender, and vehicle type contribute to overestimation and underestimation – Lighting conditions also significant for underestimated crashes

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Objectives: Analyze crashes where assessments by law enforcement officials (KABCO) and medical practitioners (MAIS) agree Improve law enforcement officer assessment using a classification tree developed using accurately assessed injury severities Research Objectives Introduction & Background MethodologyResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Crash Database - Objectives  Implement the Revised Crash Form by January 1, 2017  Streamline Crash Data Processing  Modernize the Data Management System  Improve Crash Data Quality – MMUCC / MIRE  Take Full Advantage of the TraCS Incident Locator Tool (ILT)  Enhance Crash Data Access and Interoperability  Support More Frequent Updates to the Crash Form

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison National Perspective: Crash Data Improvement Program (CDIP)  The Performance “Six Pack” Timeliness Accuracy Completeness Consistency Accessibility Integration Traffic Safety Information Systems

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison MAIS Score KABCO SCORE OCBAK 1 (Minor)13,914 (92.2%)40,893 (88.7%)32,554 (76.1%)5,993 (38.2%)0 (0%) 2 (Moderate)1,104 (7.3%)4,405 (9.6%)7,654 (17.9%)4,459 (28.4%)0 (0%) 3 (Serious)41 (<1%)507 (1.1%)1,770 (4.1%)3,044 (19.4%)0 (0%) 4 (Severe)25 (<1%)263 (<1%)807 (1.9%)1,905 (12.1%)0 (0%) 5 (Critical)4 (<1%)12 (<1%)40 (<1%)296 (1.9%)0 (0%) 6 (Maximum/Fatal)1 (<1%)3 (<1%)2 (<1%)14 (<1%)824 (100%) Total Accurate “A” Severity Crashes Accurate “B” Severity Crashes Accurate “C” Severity Crashes  Analysis of Crashes with Accurate Law Enforcement Assessment KABCO “A” with MAIS 3+ o 33% of “A” Crashes KABCO “B” with MAIS 2 o 18% of “B” Crashes KABCO “C” with MAIS 1 o 89% of “C” Crashes Methodology Introduction & Background Research Objectives ResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Initial System Deployed in 2006 WisTransPortal Crash Data System

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Crash Reports (2008) and Statewide GIS Crash Map (2012) WisTransPortal Crash Data System

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Identified Injuries in Crash Report Narratives Methodology Introduction & Background Research Objectives ResultsConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Analysis of Accurate Crash Reports Nearly 14,000 accurate crash reports analyzed o Injuries referenced in approximately 10% of accurate “A” and “B” crash reports o Injuries referenced in 5% of accurate “C” crashes Injuries are not required to be reported in crash report narrative o Only noted on small portion of crash reports Methodology Introduction & Background Research Objectives ResultsConclusions Injury Severity Crash Reports Examined Symptom/Injury Description Reference KABCOMAIS A32, (9.8%) A41, (9.8%) A (9.8%) A6131 (7.7%) Ground Truth “A” Incapacitating Injury Total 4, (9.8%) Ground Truth “B” Non-Incapacitating Injury Total 3, (9.8%) Ground Truth “C” Possible Injury Total examined5, (5.3%)

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Methodology Introduction & Background Research Objectives ResultsConclusions  Development of Classification Tree Classification trees are a machine learning algorithm used for prediction o Created by recursively partitioning data to yield least variability in response (injury severity) Graphical nature makes them easily interpretable by officers at the scene of a crash Used injuries on crash reports to estimate injury severity o Injuries were categorical predictor variables o Injury severities “A”, “B”, and “C” were response variables

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Results Introduction & Background Research Objectives MethodologyConclusions  Development of Classification Trees Two Trees were created o Without misclassification costs o With misclassification costs – Used NSC crash costs Injury SeverityUnit Cost (2011 $) Incapacitating Injury (A)$221, Non-Incapacitating Injury (B)$56, Possible Injury (C)$26, Misclassification Costs Predicted Observed ABC A B C

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  Without Misclassification Costs Results Introduction & Background Research Objectives MethodologyConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison  With Misclassification Costs Results Introduction & Background Research Objectives MethodologyConclusions

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Results Introduction & Background Research Objectives MethodologyConclusions  Without Misclassification Costs  With Misclassification Costs Predicted Injury Severity Actual Injury Severity ABCAccuracy A % B % C % Total % Predicted Injury Severity Actual Injury Severity ABC Accuracy A % B % C % Total %

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison KABCO Injury Severity Injury Severity Assessment Accuracy Existing Officer Accuracy Classification Tree without Misclassification Costs Classification Tree with Misclassification Costs A 33%67%70% B 18%62%80% C 89%83%53% Overall (Total)51%70%69% Conclusions Introduction & Background Research Objectives MethodologyResults  Conclusions Both classification trees can be used to assess injury severities to a higher level than currently in the field Misclassification costs raise “A” and “B” estimation o “C” estimation is slightly lower when misclassification costs are used Classification trees developed using injuries noted by officers in crash reports can be an easy to use tool that, along with an officer’s experience and judgment can improve injury severity estimation at the scene of a crash

Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison ACKNOWLEDGEMENT The work presented in this paper is funded by the National Highway Traffic Safety Administration (NHTSA) via the Wisconsin Department of Transportation (DOT). A special thanks to the Wisconsin Traffic Records Coordinating Committee. Thank You ! QUESTIONS?