Local Evaluator Meeting June 19, 2012 University of Maryland School of Pharmacy

Agenda 10:30Welcome and Introductions 10:45Local Evaluators Role 11:00Implementation & Evaluation Plans Process Evaluation Outcome Evaluation IRB 12:00Lunch 12:30Activities on Implementation & Evaluation Plans 2:00SAMHSA Requirements NOMS CLI part II PLI 2:30Wrap-up

LOCAL EVALUATOR ROLES

IMPLEMENTATION AND EVALUATION PLANS

The Strategic Plan identifies the Short-term and Long-term objectives. The Implementation Plan identifies the measureable outcomes. The Evaluation Plan identifies the steps that need to be taken when conducting an outcome evaluation.

Overview of Evaluation – Process Evaluation documents all aspects of the implementation of an intervention. It monitors the outputs of the strategy. – Outcome Evaluation documents whether the strategies made a difference, and if so, what changed.

Priority Decrease Underage Drinking (12-20 year olds) and decrease Young Adult Binge Drinking (18-25 year olds) in the Frostburg Community. Intervening Variable Social Availability Contributing Factor (CF) 1.Students host drinking parties and charge $5 for a refillable cup of “jungle juice” or beer 2.Mt. Ridge High School students are attending college house parties and consuming alcohol 3.Students lack awareness of the consequences for providing alcohol to those under 21 years old. Strategy Key Action Steps Implementa tion Timeframe Who’s Responsible - Implementing Measurable Outcome (s)? How will you measure? Who will collect the data? Advertise TIP hotline already established at FSU to city residents (CF #1) 1.Design a logo 2.Order magnets 3.Disburse magnets 05/12 to 6/121.Logo-Ashton Graham 2. Ordering- Lyndsey Baker 3.Disbursement- Jesse Ketterman Social Availability Core Team The number of calls to the hotline Increase the number of calls over the past year for the same two month period Lyndsey Baker

PROCESS EVALUATION

How will the implementation of the strategic plan be assessed? A Process Evaluation assesses: What activities were implemented Were activities implemented as planned Who participated and for how long? Were the resources sufficient? What obstacles were encountered? Strengths and weaknesses of the implementation What adaptations were made?

The Process Evaluation will assess whether key action steps from the implementation plan occurred as expected. Two examples of process measures to track: 1. Activities – number of meetings with legislators, – PSAs aired 2. Participants

If the strategy is…A possible measure might be… Merchant training in RBS Number of merchants attending the training (people served) Percentage of merchants who complete the training (people served) Compliance checks Number of attempts to buy alcohol by youth (services delivered) A public awareness campaign explaining social host liability laws to adults Number of PSAs aired (services delivered) Number of parents attending an information session (people served)

Priority Decrease Underage Drinking (12-20 year olds) and decrease Young Adult Binge Drinking (18-25 year olds) in the Frostburg Community. Intervening Variable Enforcement and Adjudication Contributing Factor (CF) 1.Frostburg City Police is currently understaffed by three officers, thereby making it difficult to patrol effectively. Strategy Key Action Steps Implementa tion Timeframe Who’s Responsible - Implementing Measurable Outcome (s)? How will you measure? Who will collect the data? Enhance and support law enforcement to address college parties using saturation patrols and party patrols Work with police to map out a plan for saturation patrols one weekend prior to the end of the semester. Frostburg PD= May , 2012 for 48 hours FSU PD=May , 2012 for 32 hours 1.Frostburg PD-- Chief Douty 1.FSU PD--Lt. Ralston 1.Enforcement and Adjudication Core Team The number of alcohol citation issued in the student neighborhoods Increase the number of citations over last year for the same weekend according to police reports Lyndsey Baker

Sample of Process Evaluation Planning Tool Process Evaluation Questions Process Evaluation Data Collection Methods Schedule of Completion Person Responsible Did the selected strategy follow the plan of implementation? What are the demographic characteristics of the participants? What are the participants’ levels of satisfaction with the selected strategy ? Are resources to implement the strategy available? What is the staff’s perception of the selected strategy ?

OUTCOME EVALUATION

How well did the strategies affect the contributing factors/intervening variables/priorities? Outcomes are changes that occur as a result of implementing a strategy. Short-term Outcomes are changes of the focus population at the end of the strategy. Examples include changes in knowledge, attitudes, and skills. Long-term Outcomes are the ultimate effects of the strategies. For example, the reduction in past 30 day alcohol use in youth age

Examples of Outcomes Measures Reduced access to alcohol by underage youth More responsible beverage training Decline in number of alcohol licenses issued Decline in the number of conditional use permits utilized when granting alcohol licenses Increase in the number of alcohol outlets in compliance Increase in the number of public events that restrict access to alcohol

Examples of Outcomes Measures Decrease in availability of alcohol to youth at special events and in public places (e.g., number of patrons under 21 being able to purchase, get, or consume alcohol) Rates of DUI arrests and convictions Number of calls to law enforcement complaining of incidents related to alcohol outlets Number of ER admissions that involve alcohol

Outcome Evaluation Plan What will be measured Design of the evaluation ( Who to assess, when to conduct the assessment, who will collect the data) Methods to be used in the evaluation and develop a plan to put those methods in place. Analyze the data and interpret the findings