1. Strategic Planning 201: Revisiting IV Prioritization, Selecting Strategies, and Developing Your Logic Model February 24, 2016 – 10:00am to 12:00pm Webinar Presentation Scott Formica Social Science Research & Evaluation, Inc.

2. Moderator and Presenter Moderator: Ben Spooner, Technical Assistance Provider, MassTAPP Presenter: Scott Formica Social Science Research & Evaluation, Inc. 2

3. Upcoming dates and deadlines to be aware of: – April 8, 2016: Draft of strategic plan, including logic model to MassTAPP [44 days] – April 11, 2016: Red Sox Home Opener at Fenway [47 days]* – April 29, 2016: Completed strategic plan due to BSAS [65 days] – April 30, 2016: Third Quarterly Report Due [66 days] – June 30, 2016: End of first full year of the grant [127 days] – July 1, 2016: Begin Strategy Implementation Upon BSAS Approval of Plan [128 days] – July 30, 2016: Fourth Quarterly Report Due [157 days] *Not a reimbursable expense. 3 Welcome – Important Dates

4. Reflect on the first 8 months of the grant – review some early data. Examine the Components of Steps 3-5 of the Strategic Plan Deliverable. Emphasis on IV prioritization, strategy selection, and logic model. Provide an opportunity for peer sharing and discussion. Provide an opportunity to ask questions directed to MassTAPP TA providers, the state evaluators, and/or BSAS. 4 Webinar Objectives

5. What has been accomplished during the first 8 months? Funded Clusters Sector Representation Across Clusters Sector Involvement Across Clusters SAPC Cluster Meetings SAPC Pilot Strategies Submission of Section 1 and 2 of the Strategic Plan 5 Pause for Some Reflection

6. 6 SAPC Municipalities 26 Leads and 104 Partners (N=130)

7. Sector Representation Across SAPC Cluster Municipalities (N=130) Source: SAPC Quarterly Reports (Q2: October to December, 2015) 7

8. Sector Involvement Across SAPC Cluster Municipalities (N=130) Source: SAPC Quarterly Reports (Q2); Scale: [1=Not Represented; 4=Medium Involvement; 6=Very High Involvement] 8

9. Average Number of Cluster Meetings Per Month Across SAPC Clusters [July – December 2015] (N=26) Source: SAPC Quarterly Reports (Q1-Q2); [Mean=20; Median=14; Mode=10] 9

10. SAPC Cluster Pilot Strategies (N=26) Source: SAPC Quarterly Reports (Q2: October to December, 2015) 10

11. Does anyone want to share anything about the first 8 months of the grant? Successes? Challenges? Lessons Learned? Observations? Comments? 11 Sharing Is Caring

12. Overview/Abstract [save this for last] Step 1: Assessment (sections 1.1 – 1.3) –  Step 2: Capacity Building (sections 2.1 – 2.5) –  Step 3: Strategic Planning (sections 3.1 – 3.5) Section 3.4 (Logic Model) Step 4: Implementation (sections 4.1 – 4.2) Step 5: Evaluation (sections 5.1 – 5.2) 12 Finishing Up Your Strategic Plans

13. 3.1: Describe the process for developing/writing the strategic plan Review Criteria: Did the site describe the process used to create the plan? 3.2: Planning to Address Underage Drinking and Other Drug Use We’ll spend a lot of time on this one 3.3: Problem Statement Related to Underage Drinking and Other Drug Use Review Criteria: 2-4 sentence statement on why UAD is an issue and the basis for making this claim. Separate statement for Other Drugs if the site is also proposing to address something other than UAD. 3.4: Logic Model We’ll spend some time on this one 3.5. Technical Assistance Needs Related to Strategic Planning/Logic Models 13 SAPC Strategic Plan Requirements (3.1-3.5)

14. 3.2: Planning to Address Underage Drinking and Other Drug Use A) The final set of IVs from section 1.2 that you selected, including how this list was selected (prioritized) from among the larger list of variables considered B) The specific target population(s) for underage drinking and other drug use C) The list of strategies you propose to implement to address underage drinking and other drug use, and the area(s) in your cluster in which they will be implemented D) Rationale for each selected strategy (conceptual fit, practical fit, link to research) E) The cultural competence of the selected strategy or strategies F) The sustainability of the selected strategy or strategies 14 SAPC Strategic Plan Requirements (3.2)

15. Intervening Variables: [what is contributing to the consumption pattern?] Factors that have been identified through research as being strongly related to and influencing the occurrence and magnitude of substance use and related risk behaviors and their subsequent consequences. These variables are the focus of prevention interventions, changes in which are then expected to affect consumption and consequences. (SAMHSA, 2009) Includes risk and protective factors, but not limited to individual-level factors. Emphasis on population-level change across multiple domains and systems. 15 Revisiting Intervening Variables

16. Intervening Variables: [what is contributing to the consumption pattern?] 16 Prioritization, Prioritization, Prioritization

17. Requirement: The final set of intervening variable(s) from section 1.2 that you selected, including how this list was selected (prioritized) from among the larger list of variables considered We advised you to consider around 6 IVs for UAD consumption (and an equivalent number for any other drugs) and plan to narrow this down to 3 for each consumption pattern. This is where you are asked to prioritize and narrow down the list from section 1.2 into a more manageable subset. We want to know what you looked at (1.2), what you have decided to focus on (3.2a) and how/why you arrived at this decision. 17 SAPC Strategic Plan (3.2a) – Tips

18. What are some consumption IVs? – Low levels of law enforcement – Community norms favorable to use – Perception of parental disapproval or attitude – Family communication – Parental monitoring or family management practices – Availability/Ease of Access – economic, retail, social – Perception of harm/risk – Perception of peer disapproval or attitude – Close friends who use/abuse – Attachment to school and family Representation across individual, peer, family, and community domains. 18 IVs – Sample Consumption IVs

19. Importance Changeability Feasibility Capacity Readiness Resource availability Supported by data – qualitative and quantitative Data quality 19 Corroborated by different sources Cultural Competence Evaluability – data available Fit – relevant to issue/community Potential for unintended consequences Sustainability Wisdom of Practice Present in multiple communities It is helpful to generate a list of criteria on which each IV can be examined and compared. Potentially relevant criteria include: IVs – Sample Prioritization Criteria

20. This is an example from a former grantee on how you might frame a set of questions on multiple criteria: – Is the IV identified independently by multiple sources? – How reliable and valid are the data supporting it? – Have local changes in the variable produced changes in use? – How actionable is the variable? – Is it feasible to address the variable within the time frame of this grant? – Are other efforts in place to change this variable? – Are there capacity and resources available or that could be developed to address the intervening variable? 20 IVs – Sample Prioritization Criteria Questions

21. Step 3 in the SAPC Guidance Document (pg 31-32) discusses prioritizing IVs.Guidance Document Emphasis is on Importance and Changeability – the latter includes many of the sub-categories we have discussed (e.g., readiness, resources, suitable interventions, time required for change to occur) Other potential criteria include: – Directionality – can you interpret the change? – Resource/Need gap – are resources already going to this? – Political will – is there political/public will to address this factor? – Distance – is the IV proximal enough to affect change/measure? 21 High Importance Low Importance IVs – Guidance on Prioritization

22. 22 IVs – Be Systematic

23. Efficiency and Synergy – Linkages between intervening variables – Multiple IVs that can be impacted by a single intervention – Is the IV related to other health or behavioral health issues? Appropriateness – Has the IV been shown to be linked to consumption with the group that you are targeting? Age? Gender? Race/Ethnicity? SES? Urban, Suburban, Rural? Etc. Acceptability – Did the process and final decision-making process provide adequate opportunity for community input? Are there missing segments? Was the assessment comprehensive and inclusive? 23 IVs – Additional Things to Consider

24. Which groups/bodies should make the final decision? Small group of cluster/steering committee representatives attempt to reach consensus based on available evidence? Develop an IV Workgroup that reports to the steering committee? Each cluster solicits input from key stakeholders and brings this to the SC? How, if at all, are community members included? How, if at all, are principles of cultural competence woven in? Be a Conductor Prioritization and selection of IVs and strategies should involve key stakeholders not just the Coordinator. Invest in future buy-in 24 IVs – Sample Decision-Making Processes

25. Does anyone want to share anything about IV Prioritization? Successes? Challenges? Lessons Learned? Observations? Comments? Questions? 25 Sharing Is Still Caring

26. 3.2: Planning to Address Underage Drinking and Other Drug Use A) The final set of IVs from section 1.2 that you selected, including how this list was selected (prioritized) from among the larger list of variables considered B) The specific target population(s) for underage drinking and other drug use C) The list of strategies you propose to implement to address underage drinking and other drug use, and the area(s) in your cluster in which they will be implemented D) Rationale for each selected strategy (conceptual fit, practical fit, link to research) E) The cultural competence of the selected strategy or strategies F) The sustainability of the selected strategy or strategies 26 SAPC Strategic Plan Requirements (3.2)

27. It is important to identify and select strategies that have been shown through research to be effective, are a good fit for your community, and are likely to promote sustained change. There is a great body of research demonstrating the effectiveness of various strategies to reduce underage alcohol consumption, other drug use, and their consequences. Literature reviews, published studies, unpublished evaluation findings, and other resources may help you identify the strategies with the greatest potential to affect the intervening variables you identified as a priority. For each strategy you consider: – Review the research evidence that describes how the strategy is related to your selected intervening variable(s) – Based on this evidence, present a rationale describing how the strategy addresses the intervening variable(s) – Note: Be sure to discuss potential strategies with your TA provider. 27 Strategy Selection

28. Consumption Strategies (selected list) – SAMHSA’s National Registry of Evidence-Based Programs and Practices (NREPP)NREPP – Wyoming Survey and Analysis Center’s Catalog of Environmental Prevention StrategiesCatalog of Environmental Prevention Strategies – U.S. Department of Health and Human Services Stop Alcohol Abuse websitewebsite – CAPT Decision Support Tool: Strategies to Prevent Binge or Heavy Episodic Drinking Among Adolescents and Young AdultsStrategies to Prevent Binge or Heavy Episodic Drinking Among Adolescents and Young Adults – CAPT Decision Support Tool: Prevention Programs that Address Youth Marijuana UsePrevention Programs that Address Youth Marijuana Use – SAMHSA’s Principles of Substance Abuse Prevention GuideGuide 28 Strategies – Consumption Strategies

29. You are not limited to the resources listed on the previous slides – provided that you can supply evidence of the effectiveness of any strategies that do not appear in any of these resources. Start with the intervening variable – not with the strategy. If you can’t find an appropriate strategy – consider looking at strategies that have been linked to the IV for other behavioral health issues. 29 Strategies – Identification

30. Step 3 in the SAPC Guidance Document (pages 31-38) discuss strategy selection.Guidance Document Emphasis is on: – Evidence of linkages between the targeted intervening variable(s) and the proposed strategy – Conceptual fit – Practical fit 30 Strategies – Selection Criteria

31. Conceptual Fit – How has the strategy been tested with the identified target population or if it has not how can it be generalized to the target population? – How will implementing this strategy in your local community help you achieve your anticipated outcomes? Practical Fit – Resources (cost, staffing, access to target population, etc.) – Collaborative/Coalition Climate (fit with existing prevention/reduction efforts, willingness to accept new programs, buy-in of key leaders, etc.) – Community Climate (community attitude toward the strategy, buy-in of key leaders, etc.) – Sustainability of the strategy (community ownership, renewable financial support, community champions, etc.) 31 Strategies – Fit

32. Do a small number of things really well versus trying to do too much and doing it poorly. You probably do not want to propose more than 3-5 UAD strategies in isolation or more than 3 UAD strategies and 3 other drug strategies if you are focusing on multiple substances. Remember that not every community or location in your cluster needs to implement every strategy in your logic model – nor do they all have to do the same thing at the same time. You may want to focus on low-hanging fruit at the outset. Strategy-specific workgroups are a good idea. 32 Strategies – A Word of Advice

33. REMEMBER – Be a Conductor! Prioritization and selection of IVs and strategies should involve key stakeholders not just the Coordinator. Invest in future buy-in 33 Strategy Questions?

34. Perhaps the most important piece of the entire strategic plan. This has the potential to become your elevator speech for the project. The logic model either validates your strategic plan or exposes and amplifies issues with the planning process. 34 Logic Model (3.4)

35. Intervening Variables: factors that have been identified through research as being strongly related to and influencing the occurrence and magnitude of substance use and related risk behaviors and their subsequent consequences. These variables are the focus of prevention interventions, changes in which are then expected to affect consumption and consequences. If you cannot demonstrate how B in the logic model DIRECTLY addresses A – your plan will be returned to you. 35 Problem identified by BSAS: Intervening Variable (A) Short- Term Target Group Strategy (B) Outputs Outcomes IntermediateLong-Term Local manifestation of the problem: Connection Between IVs/Strategies This is your problem statement Too many college freshmen in our community (22%) report binge drinking on a regular basis (weekly). Too many eighth-graders (15%) in our town report trying alcohol for the first time.

36. 36 Problem identified by BSAS: Underage drinking. Local manifestation of the problem: In 2014, past-30-day use of alcohol among high school students in the cluster was higher than the state average of 36% (Smithtown: 42%; Jackson: 38%; Redmond: 39%)). Intervening VariablesStrategyTarget GroupOutputs Outcomes Short-TermIntermediateLong-Term High perceived ease of access to alcohol from commercial sources among 9th– 12th-graders in the cluster Responsible beverage service training All alcohol retail establishments in the cluster (both on- and off-premise) Number of establishments targeted Number of establishments trained Number of individuals Trained Increase in awareness, knowledge, attitudes, and responsible serving/selling practices among those trained Decrease in perceived ease of access to alcohol from commercial sources among 9th–12th-graders in the cluster Decrease in the % of 9th–12th grade students in the cluster who report past- 30-day use of alcohol Complete a logic model sheet for each problem identified. Include additional rows for each intervening variable being targeted. SAPC Logic Model Example

37. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: The Problem Identified by BSAS This is taken from the RFR for each BSAS initiative. It describes why BSAS has made these grant dollars available. Example Underage Drinking. Part 1: BSAS Identified Problem

38. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Local Manifestation of the Problem Defines the extent of the problem in the local community (quantitative or qualitative). This is your problem statement. Example In 2014, past-30-day use of alcohol among high school students in the cluster was higher than the state average of 36% (Smithtown: 42%; Jackson: 38%; Redmond: 39%). Part 2: Local Manifestation of the Problem

39. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Intervening Variable Biological, social, environmental, and economic factors research has shown to be related to substance use and consequences of use. Subsumes, but not limited to R&P factors. Example High perceived ease of access to alcohol from commercial sources among 9th–12th-graders in the cluster. Part 3: Intervening Variable

40. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Strategy (or intervention) Programs, policies, and/or practices to reduce use and/or consequences of use. Expected to affect intervening variable – which effects outcomes. It is likely than multiple strategies will be used to address each IV. Example Responsible beverage service training. Part 4: Strategy

41. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Target Group The immediate audience of each strategy. Also identify any secondary groups. Example All alcohol retail establishments in the cluster (both on- and off-premise). Part 5: Target Group

42. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Outputs Measures of the extent to which strategies are being implemented as planned (e.g., head counts of individuals participating in a program, estimated views of a prevention billboard). Examples Number of establishments targeted; Number of establishments trained; Number of individuals trained. Part 6: Outputs

43. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Short-Term Outcomes The immediate effects of a program – often focus on the knowledge, attitudes, and skills gained by a target audience. Example Increase in awareness, knowledge, attitudes, and responsible serving/selling practices among those trained. Part 7: Short-Term Outcomes

44. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Intermediate Outcomes Changes in behaviors, norms, and/or policies. Often expressed as changes in the intervening variable. Example Decrease in perceived ease of access to alcohol from commercial sources among 9th–12th- graders in the cluster. Part 8: Intermediate Outcomes

45. Problem identified by BSAS: Intervening Variable Short- Term Target Group StrategyOutputs Outcomes Intermediate Long-Term Local manifestation of the problem: Long-Term Outcomes The ultimate goals of the program – often take time to achieve. Example Decrease in the % of 9th–12th grade students in the cluster who report past-30-day use of alcohol. Part 9: Long-Term Outcomes

46. 46 Logic Model Questions?

47. 4.1: Implementation of UAD and Other Drug Use Strategies Review Criteria: Did the site provide an implementation action plan (including action steps, person(s) responsible, timeline, and measure(s) of success) for each strategy identified in section 3.2 and the logic model? 4.2: Technical Assistance Needs Related to Implementation We’ll spend a lot of time on this one 5.1: Existing and Planned Youth Surveys Review Criteria: Did the site provide a description of any existing (or planned) student surveys within the lead community? 5.2: Technical Assistance Needs Related to Evaluation DON’T FORGET TO INCLUDE OVERVIEW/ABSTRACT AND ADHERE TO PAGE LIMITS! 47 SAPC Strategic Plan Requirements (4.1-5.2)

48. 48 When in Doubt Call Them – Not Me