PERFORMANCE MEASURES BASELINE LOCATION DATA Stacey Gierisch Wyoming Department of Transportation Highway Safety Hi – my name is Stacey Gierisch and I am from WYDOT (the Wyoming Department of Transportation). I would like to talk with you about our experience with performance measures – and specifically about finding our baseline for location errors.

Main Points Location is a CRITICAL Element Perception vs Hard Data Hard & Scary – But Worth It Made Significant Improvements We are going to cover 4 main points this afternoon. Location is a very very critical element Perception of what I thought vs the hard data of what we actually found It’s not an easy task and it was scary, but it was so worth it The improvements we made I’m going to ask you all a question and I’d like a show of hands – how many of you have been asked by your boss, your boss’s boss, your consultant, or anyone that doesn’t live under that mushroom with you “How many crashes come in straight from the officer with a correct location?

I have been asked this question quite often I have been asked this question quite often. In the beginning I was able to just pull the answer out of the bag magically. I would tell them the errors were low and we were doing a great job with quality control fixing what little errors we had!! However, after a time it was no spoon full of sugar I wanted to make sure my answers were based on solid ground So I decided I needed to get some facts and measures on the locations coming into our office.

“Without Data, You're Just Another Person With an Opinion.” W. Edward Denning I found this quote by W. Edward Denning to fit exactly in what we were doing. If you don’t have the data, you are just someone with an opinion and everyone usually has a different one.

Our August Experiment to Measure Baseline We liked to refer to it in our office….(twilight sound) – Our August Experiment to measure our baseline You are now entering into the world of Wyoming, crashes, law enforcement and location accuracy. We picked the month of August because it’s an average month for crashes – not too heavy with winter crashes or too light for summer crashes. Also because it gave us enough time to get it done before our year-end QC.

How many location errors were in the reports coming from the field? The main question we asked was how many location errors were in the reports coming in from the field?

Step 1: *From Aug 1 to Aug 31 Process crashes through our System without correcting locations Step 1 for this experiment was that from August 1 to August 31 we processed all crashes through our system without correcting locations. My employees would print out the crashes that had OTHER variable corrections on them. We kept all August crashes separate from any other month crashes that came in. Then mid-September we ran a query to capture all the August crashes and we sorted them by Law Enforcement Agency. We waited until mid September to ensure that we had the majority of the August crashes.

Valid vs Correct In pass I just wanted to clarify between valid vs correct. Valid meaning the location exists. Correct meaning the crash happened at the right location.

Step 2: Checked every single crash: *Is the location given? *Is the location valid? *Is the location data consistent? We then checked every crash: What we were looking for was: Did it have latitude/longitude? Was the lat/long valid? Was it correct? Was it a valid route and milepost? Was it a correct route & milepost? Did it have an intersection? Or did it have an intersection distance? Was it a valid intersection or intersection distance? We recorded all of this in an Excel Spreadsheet and made corrections to the locations if necessary. We also multi-tasked during this and put in the correct information and fixed other corrections as necessary – killed two birds with one stone

SURPRISE!!!!! Well..the findings of our experiment were a huge surprise. And this is exactly how I looked after putting all the numbers together! I was really thinking that most of the crashes that come in had correct locations, especially if there was a lat/long and a route and milepost. WRONG!! Not a good one

Crash Data Flow Collect Consolidate Analyze Decide Feedback Loop Basically when we started it was the Officer collecting the location data Highway Safety consolidating it and correcting it Then it went on to be analyzed and off to the decision makers Collect Consolidate Analyze Decide Feedback Loop Data Flow

Crash Data Flow Collect Consolidate Analyze Decide We corrected the location errors here assuming that all the errors were made by the officers Collect Consolidate Analyze Decide Correct Location Errors Feedback Loop Data Flow

Crash Data Flow Collect Consolidate Analyze Decide But we checked the location errors again after we had run our QC process and found we still had mistakes that were made by Highway Safety. Collect Consolidate Analyze Decide Correct Location Errors Check after QC process Feedback Loop Data Flow

That was a very hard pill to swallow!!

Inconsistent Location Fully Valid Location 3.9% 60.9% 35.2% RESULTS No Location Given Inconsistent Location Fully Valid Location The other results we found was that the crashes with no valid location was 3.9%, the crashes with a inconsistent locations was 60.9% and the crashes that had a full valid location was 35.2% 3.9% 60.9% 35.2%

Minutes to Process by Staff (per crash) No Valid Location 8.5 – 11 Inconsistent Location 6.5 – 9 Valid Location 3.5 – 4 We also recorded how long it took each staff member to fix each location error. On crashes that had no valid location it took anywhere from 8 ½ minutes to 11 minutes depending on the crash being urban or rural. For partial valid location errors it took 6 ½ minutes to 9 minutes and for a full valid location crash it took 3 ½ to 4 minutes. This did not include correcting any other variables or the regular processing of the crash by Highway Safety. This was only to fix the location errors.

Out Come *Validation for map based location (WIGLS) *Changed our process *Locations are corrected sooner *Using GIS and other tools The result of doing this experiment and finding out the baseline We now have real objective data in support of spending a large amount of money for map based location project which is now underway We have changed the way Highway Safety processes the crashes, which saves us time and less touching of the crash The locations are being corrected in a better way and much faster for analysis. We are using our GIS and other research tools to a better advantage to match the lat/long with the route/milepost with the intersection node. We want all 3 of the specifics to match.

Informing Law Enforcement When we reported our findings to Law Enforcement, it was a very scary thing – these guys carry guns!! My staff was a bit nervous as well, so being the brave supervisor I am, I told them to forward all calls to me to handle. I have big shoulders I can do this!! But when they did call, we had nothing but positive feedback!! Not one yelling episode, or telling us how horrible we are. It was very refreshing to see that they wanted their data to be as good as we did! Informing Law Enforcement

Real Quotes from Law Enforcement: “Can we have some training on locations as we want to make our error rate go down” Jackson PD “We used this to compare ourselves with other like agencies – and we are way ahead of them!!” Anonymous “We are a small office, when are you coming to train so we can join it” Shoshoni PD “How can you help us to locate county roads better? That seems to be hard for my deputies” Laramie County SO “Our Lieutent’s look at this and want improvement! This was great for our performance Evaluations for our Troopers” Highway Patrol Very positive..no negative at all

“To Improve Something, You Have to Measure It.” -W. Edward Deming To quote W. Edward Deming – Without data you’re just another person with an opinion. That is exactly what I was – just a person with an opinion. To Improve something, you have to measure it – I wanted to improve our location accuracy.

Main Points Location is a CRITICAL Element Perception vs Hard Data Hard & Scary – But Worth It Made Significant Improvements Let me summarize the main points again. Location is a very very critical element in the crash report. What I thought was going on with location errors was totally different than what the hard cold facts told us. It was a hard project and it was very scary as it did put us behind about a month or so, but the findings were well worth it!!! Because of finding our baseline measure of location errors, we have made significant improvements of our daily processes.

Thank you Stacey Gierisch Highway Safety So now life is just a spoonful of sugar with our location data!! Thank you very much!! Stacey Gierisch Highway Safety Wyoming Department of Transportation Stacey.gierisch@wyo.gov (307) 777-4194