ViDA Data Management/Quality Assurance

Outline – ViDA Data Management/Quality Assurance QA review before the coding process QA review during the coding process QA review after the coding process Pre-processing of data prior to analysis

Road rating training usRAP inspection manual How to use software Practice road length Peer reviews Supervisor reviews

By the end of the training, the coders must: understand what usRAP is and aims to achieve, including the purpose of Star Ratings and Safer Roads Investment Plans be familiar with the road attributes and categories that must be recorded know how to use the software to code roads be able to consistently code roads with a high level of accuracy 7-14 hours (intro, manual, software)

Target Level of Accuracy

Before the coding … All coders rate same section of road Up to 8 hours Practice Comparison (peer, trainer) Up to 8 hours

QA review during the coding process Fatigue Quality review Interruptions Data management

Road rating/coding… human factors Perception (volume, expectation, contrast) Interpretation (workload, motivation, mistakes) Response (coding, saving) 4-6 coders per 2000 miles 2-7 mph range

Example: slope

Recommend Reviews Peer to Peer Progress Reviews External Reviews (opt) Encourage discussion Cross-check each other Progress Reviews Supervisor checks Target 10% review External Reviews (opt) Independent Peer to peer reviews One of the best ways for new raters to learn is to ask questions, of both their supervisor and other raters. Often, this will be the main way to learn some of the more difficult issues to grasp. To ensure that all raters share exactly the same views on rating, peer to peer reviews are used. Whenever a difficult question is raised, the answer needs to be shared with all rating staff in the team. For example what guardrail end conditions are considered safe and where are they considered a hazard (see photos on slide). It is also helpful to have a cross check of rated roads. To do this, one rater will review the ratings made by his or her peer and then they have a discussion about each item where the reviewer identifies each attributes rated with which there is not agreement and they review the inspection manual and come to a conclusion as to how each such attribute should be rated. Peer to peer review should be undertaken and documented regularly throughout the rating process. As an example, a peer to peer review can be undertaken on a short section of each road or major section that is completed. It is recommended that the rating supervisor sit in on these peer to peer reviews, initially taking a proactive role and with time simply listening and intervening only in the case of an improper conclusion. Progress reviews Data should be checked by the rating supervisor as soon as possible after rating to identify any issues of concern quickly. Initially, detailed checks should be undertaken at the end of each day’s rating until it is clear that the rater is providing outputs in accordance with the rating guidelines. A general rule is that 10% of the roads rated should be reviewed by the rating supervisor. Rather than wait until an entire network of roads have been rated to find an error, it is best that constant reviews be done, which would eventually add up to the final 10% check of all roads. In addition to checking the ratings against the specifications, the following issues can be checked. Typical scenarios that can cause errors include: duplicate records missing 100m sections not rated (gaps between rated sections) missing data in some attributes only. Some general checks of the data may include: has the correct length of road been rated? has the correct segments been rated? has horizontal alignment been rated or is it based on inertial measurement data (eg. Gipsi-trac)? where there is dual carriageway is there data for the reverse direction? If major errors are found in the sample, complete re-rating of the entire work by that rater may be required. Minor and isolated errors can be ratified and corrected by the rating supervisor. It is important to discuss any issues with raters straight away to minimise the need for re-rating. If all questions as to how to rate a particular item or situation are resolved by the rating supervisor it will be possible to reduce both discrepancies and errors. The rating supervisor should prepare a weekly report of completed surveys and the peer to peer evaluation and discrepancies and errors found on each survey. These should be shared with the usRAP team to promote uniformity. External reviews External review of the survey should occur after completion of 25%, 50% and 100% of the rating. The external review should cover approximately 10% of the network, and be based on the specifications for accuracy. These external reviews should be undertaken by someone qualified to be a rating supervisor who was not directly involved in the rating task.

Typical errors duplicate records missing sections not coded (gaps between coded sections) missing data in some attributes only

Documentation Errors, by coder Results of peer reviews Weekly progress report Share

General Checks has the correct length of road been coded? have the correct segments been coded? has horizontal alignment been coded? where there is divided highway, are there data for the reverse direction? Were the correct photos used?

Not always easy to see speed limit signs in streetview

Notes Non-standard sign

Notes Location Attribute Recorded item Correct item Notes 0.1 to 0.7 Speed limit unknown 60mph Speed sign at 0.7m is a repeater sign (small diameter), therefore previous section has same speed limit. 0.1 Roadside Left 0-3m 3 -7m Only hazardous item present is telegraph pole, around 7m. See image 1 below Intersection type 3-leg (unsignalised) no turn lane 4-leg (unsignalised) no turn lane This is a staggered 4 leg intersection. See image 2 0.1 to Lane width Medium Wide Compare the lane width to the car width 0.3 Curvature Moderate Straight or gentle There is no curve in this 100m (further on there is) Median type Central hatching Centre line The hatching stopped in the previous 100m Image 1. A9, distance 0.1m Image 2. A9, distance 0.1m (looking back from end of the section)

QA review after the coding process Debrief with raters Consistency checks Produce and review maps Spreadsheet computations Summaries Pivot tables

Simple Consistency Checks See Chapter 4, p. 17 of the Producer Manual

Results Review Check extreme risk values – are the coded attributes correct. Check a random sample – are the coded attributes correct. Distribution of items for each category using detailed condition report (road specific) Do risk scores follow logical road structure/ classification? Are countermeasures in the correct places – cross reference with road inspection data. Are the maps accurately plotted on the road network?

Spreadsheet summary statistics

Pivot Tables – comparing coders

Pivot Tables – summary by codes

Mapping Checks

Example 1 – Roadside Severity Review Using ArcGIS and Google Maps After rating, plot points and attributes in ArcGIS. Filter on attributes that you want to review. In this example, roadside severities: Deep drainage ditch (9) Downwards slope (> - 15) Based on some preliminary descriptive statistics on roadside severity, I felt that the above had been reported too frequently by one rater. The spatial distribution of locations coded in this manner are presented. Convert records of interest to KMZ and display in Google Earth. In Google Earth, review the Streetview images and re-assess the attributes. Simultaneously, but independently, update the appropriate attributes in ArcGIS by selecting the corresponding records

GIS on base map

Street View review

Example 2 – Intersection Attribute Review Using ArcGIS (thematic maps) After rating, plot points and attributes in ArcGIS. Filter on attributes that you want to review In this example, intersection locations and recorded volumes. Compare intersection locations to an underlying roadway network, intersection database or aerial image. Where available, compare traffic volumes to the recorded/assumed traffic volumes. Thematically map the intersection and traffic (roadway) data in the same manner. Where available, compare provided traffic signal locations to recorded traffic signal locations. Identify locations where an intersection was recorded but traffic volume was not (this could be done in Excel as well). Update any pertinent attributes in 3-6 in ArcGIS by selecting the corresponding records.

Example 3 –Attribute Review Using Street View and preprocessor After rating, plot points and attributes in ArcGIS. Filter on attributes that you want to review In this example, intersection locations and recorded volumes. Compare intersection locations to an underlying roadway network, intersection database or aerial image. Where available, compare traffic volumes to the recorded/assumed traffic volumes. Thematically map the intersection and traffic (roadway) data in the same manner. Where available, compare provided traffic signal locations to recorded traffic signal locations. Identify locations where an intersection was recorded but traffic volume was not (this could be done in Excel as well). Update any pertinent attributes in 3-6 in ArcGIS by selecting the corresponding records.

Pre-processing of data prior to analysis Although data for the majority of these fields is collected during the road inspection and coding phases, some fields are difficult to rate or require further investigation, or pre-calculation data processing (pre-processing).

Fields covered road name section traffic volume motorcycle volume Updated: see page 7 of producer manual Fields covered road name section traffic volume motorcycle volume pedestrian and bicycle flows speed overtaking demand horizontal alignment vertical alignment intersecting road volume

Use “local knowledge” Document this to ensure there is a high level of transparency and quality

Motorcycle flow – usually default in the US

Ped/Bike Flow estimation (general approach, being revised for ViDA) Method Area type Sidewalk Ped crossing facility Land use 3 Stages of preprocessing: global, local, smoothing Engineer/planner w/local knowledge may check/adjust Plotting useful

Pedestrian Crossing – Global Stage Direct observation (coded flow)

Ped Crossing – Global Stage Estimate flow

Ped Crossing – Global Stage

Ped Crossing – Local Stage Example Adjustments If no flow observed, but crossing facility, adjust to low If no flow observed, but intersection, adjust to low Global Stage may produce excessive flows for US conditions – consider reducing by one unit Do not smooth

Ped Flow Along Road – Global Stage

Ped Flow Along Road - Global

Ped Flow Along Road - Global May produce excessive flows for US conditions – consider reducing by one unit

Ped Flow Along Road – Local Stage/Smoothing Example Adjustments If no flow observed, but sidewalk, adjust to low If no flow observed, but semi-urban/urban, adjust to low Global Stage may produce excessive flows for US conditions – consider reducing by one category/unit Smoothing +/- 1.5 mi. (don’t carry into rural) Observed rural carry over to nearest semi- urban/urban

Bike Flow – Global Stage Similar to ped flow along road

Bike Flow – Local Stage/Smoothing Example Adjustments If no flow observed, but bike facility, adjust to low If rural and isolated med/high flow observed, reduce by one Global Stage may produce excessive flows for US conditions – consider reducing by one category/unit Smoothing +/- 1.5 mi. (don’t carry into rural) Observed rural carry over to nearest semi-urban/urban

Speed Three categories: Posted speed (coded by the coders) 85th percentile speed (from surveys, similar sites, or set to 5 over posted) – post processing Mean speed (from surveys, similar sites, or set to posted) – post processing

Horizontal Curvature Add it if you’ve got it (e.g., from an inventory)

Vertical Alignment Variation Add it if you’ve got it

Passing Demand (inside the program) Physical median – none 1 lane w/median rumble, central hatching, <2000 ADT, non-physical median – low 1 lane w/centerline median and <4000 ADT, – medium 1 lane w/centerline median and >=4000 ADT, – high Document method

Intersecting Road Volume Detailed volume data is best Post process if you have some If have all, don’t code Better to estimate than to use default (high)

Questions? Thank you