Case Study: The Evaluation Process Tanya Fosdick

The Issue years represent 13% of local population but 33% of all casualties Also comprise 25% of KSI casualties (largest age group) More male casualties 68% are driver casualties Group E represented highest numbers and above average risk

Lifestyle Summary

Persona

The Result an interactive presentation which contains several short video clips the presentation also contains interviews with the mother of a local victim and offender a crash car simulator which also includes a storyline using young local actors creation of a microsite and ongoing development of a social media presence

Evaluation Process Consultation in January 2014 between RSA and Leicestershire FRS LFRS provided details of the intervention so we could understand Delivery Management Structure Setting of Aims and Objectives

Aim To increase the knowledge of presentation attendees in relation to the risks of the ‘Fatal 4’ and to encourage attendees to agree to behave in a responsible way when they drive

Objectives 1.25% of attendees who before attending did not exhibit knowledge of road safety risks in each of the following areas, did exhibit this knowledge afterwards (mobile phone use, speeding, failing to wear seatbelt, drink & drug driving) 2.15% of attendees who stated before attending that they would make calls or texts on their mobile phone while driving, agreed afterwards that they would not now do so 3.10% of attendees who stated before attending that driving at 40mph in a 30mph limit was not unsafe, agreed afterwards that it was unsafe 4.15% of attendees who stated before attending that they would not always wear their seatbelt, agreed afterwards that they would now do so 5.25% of attendees who stated before attending they thought they could handle a drink or two while still able to drive safely, agreed afterwards that they did not now think so 6.25% of attendees who stated before attending they thought they could use cannabis and still be able to drive safety, agreed afterwards that they did not now think they could.

How to measure against the objectives Example involving 1,000 respondents in pre & post Question asked if they would switch off phone before driving Before, 149 agreed and 851 disagreed After, 261 agreed – 112 more than before 112 reported a positive change, representing 13.2% of 851 (not 1,000)

Logic Model

Evaluation Design Pre- and post-intervention design with comparison group Paper questionnaires, inputted by LFRS into Prometheus online tool Pre-questionnaires identical for intervention and comparison groups Baseline questions to establish attitudes and knowledge Demographic questions Post-questionnaires – same as pre- for comparison group with additional questions about No More Lives Wasted Roadshow for intervention group URNs could be created from DOB, gender, postcode and school questions

Evaluation Process Data collection started in April 2014 and closed January 2015 Questionnaires delivered before intervention & collected, on average, 6 weeks later (Average 4 weeks for comparison group) Approximately 10,000 young people attended No More Lives Wasted Roadshow in that time Needed a sample size of 370 (based on 5% margin of error, 95% confidence level & 50% response distribution)

Results? At the report writing stage Sample sizes: 1,096 completed pre-surveys for intervention group 1,071 completed post-surveys for intervention group 757 matched pre and post (exceeding target sample) 204 completed pre-surveys for comparison group 214 completed post-surveys for comparison group 171 matched pre and post

Process Takes time! 15 months….. Consultation Evaluation design Aims and Objectives setting Logic model Questionnaire design and testing Data collection and intervention delivery Analysis and report writing