Selecting and Prioritizing Intervening Variables for Your SAPC Grant December 14, 2015 – 10:30am to 12:00pm Webinar Presentation Scott Formica Social Science Research & Evaluation, Inc.

Moderator and Presenter Moderator: Ben Spooner, Technical Assistance Provider, MassTAPP Presenter: Scott Formica Social Science Research & Evaluation, Inc. 2

Pause to celebrate the first 6 months of the grant………. Okay. Moving on…. There are several important dates and deadlines approaching: –D–Dec. 18, 2015: Touch base with MassTAPP on progress on Step 1 and Step 2. –J–Jan. 11, 2016: MIS and Narrative Report Webinar Part II. –J–Jan. 29, 2016: Draft of Section 1 and 2 of Strategic Plan due to BSAS & MassTAPP. –J–Jan. 31, 2016: Narrative Report and MIS for Quarter 2 due. –A–Apr. 8, 2016: Full Strategic Plan and logic model due to MassTAPP. –A–Apr. 29, 2016: Full Strategic Plan and logic model due to BSAS. 3 Welcome – Important Dates

Examine requirements and expectations for the 1/29/16 deliverable. Assessment of UAD and Other Drug Use and Intervening Variables. Capacity Building. Explore the concept of intervening variables (IV) – including the prioritization process that will occur in Step 3 following the 1/29/16 deliverable. Touch upon linkages between problem statements, IVs, and strategies. Provide an opportunity to ask questions directed to MassTAPP TA providers, the state evaluators, and/or BSAS. 4 Webinar Objectives

Overview/Abstract - NO Step 1: Assessment (sections 1.1 – 1.3) – YES – Focus of today Step 2: Capacity Building (sections 2.1 – 2.5) – YES Step 3: Strategic Planning (sections 3.1 – 3.5) – NO Section 3.4 (Logic Model) - NO Step 4: Implementation (sections 4.1 – 4.2) – NO Step 5: Evaluation (sections 5.1 – 5.2) - NO 5 What Is Due on 1/29/16?

6 ASSESSMENT: Sections

The SAPC strategic plan template asks you to write about your process and findings related to underage drinking and other drug use across the cluster. How did you conduct the assessment? (write about all the steps you took) What data sources and techniques for data collection did you use? Include numbers/rates/percentages based on your best data sources. Identify where any data you use came from. Are any subpopulations disproportionately affected? Evidence? Data gaps and how you plan to address these. Additional information relevant to the issue or the assessment. Sustainability and cultural competence of the process. 7 SAPC Strategic Plan Requirements (1.1)

Was a comprehensive process used to systematically review data on underage drinking and, as needed, other drug use? Does the plan clearly describe the data sources that were used? Does the plan include data on underage drinking and, as needed, other drugs? Were different subgroups considered? Were data gaps identified along with plans to address them? Were sustainability and cultural competence discussed? 8 SAPC Strategic Plan (1.1) – Expectations

Convince the readers that a comprehensive assessment was conducted. Honesty is the best policy – we KNOW that there are going to be data gaps across the cluster and that not every cluster is going to have the same data. Section 1.1 should be limited to describing consumption data. Do not get into the suspected reasons for use or the potential ways to address use. This is a long-term grant. Be prepared to discuss how you will revisit assessment over the life of the grant and enhance the assessment over time. Whatever appears here as a plan to address data gaps should most likely make its way into your capacity building plans in Section 2. 9 SAPC Strategic Plan (1.1) – Tips

Step 1.1 (Assessment Data on Underage Drinking and Other Drug Use) needs to occur before you move on to Step 1.2 (Assessing Intervening Variables). You don’t want to look for reasons why until you have a thorough understanding of what the issue is and which groups, if any, are disproportionately affected. Per Page 3 of the Guidance Document, decisions to focus on issues in addition to underage drinking require substantial data to support this decision. It is important to consider the use pattern of interest (e.g., initiation, experimentation, current use, moderate use, high-risk use). This will influence both your IVs and your strategy selection. 10 SAPC Strategic Plan (1.1) – Tips

Any questions specific to Section 1.1 – Assessment Data on Underage Drinking and Other Drug Use? Does anyone want to discuss any successes they have had during the assessment that could benefit others? Any interesting or novel data sources or findings that others may want to explore? Any challenges that others may encounter? 11 Questions About Section 1.1

Intervening Variables: [what is contributing to the consumption pattern?] Factors that have been identified through research as being strongly related to and influencing the occurrence and magnitude of substance use and related risk behaviors and their subsequent consequences. These variables are the focus of prevention interventions, changes in which are then expected to affect consumption and consequences. (SAMHSA, 2009) Includes risk and protective factors, but not limited to individual-level factors. Emphasis on population-level change across multiple domains and systems. 12 Intervening Variables

The SAPC strategic plan template asks you to write about your process and findings related to intervening variables (IVs) for underage drinking and other drugs across the cluster. How did you conduct the IV assessment? (write about all the steps you took) What data sources and techniques for data collection did you use? List all IV investigated, including data on each variable and the source(s). Data gaps and how you plan to address these. Additional information relevant to the issue or the assessment. 13 SAPC Strategic Plan Requirements (1.2)

Was a comprehensive process used to systematically review data on IVs related to underage drinking and, as needed, other drugs? Does the plan clearly describe the data sources/sources of evidence used? Does the plan list ALL of the intervening variables that were considered? Were data gaps identified along with plans to address them? 14 SAPC Strategic Plan (1.2) – Expectations

Convince the readers that a comprehensive assessment was conducted. This should not be based on convenience/availability alone (i.e., our student survey asks about perception of risk). Again. Honesty is the best policy. Consider that IVs may vary across cluster communities more than consumption patterns – it is okay to say that. Again. This is a long-term grant. Be prepared to discuss how you will revisit IV assessment over the life of the grant and enhance the assessment over time. IVs are really where cultural competence plays a key role. 15 SAPC Strategic Plan (1.2) – Tips

Again, whatever appears here as a plan to address data gaps should most likely make its way into your capacity building plans in Section 2. You probably want to consider around 6 IVs for UAD consumption (and an equivalent number for any other drugs) and plan to narrow this down to 3 for each consumption pattern – later in Section 3 of the plan. Do not prioritize or choose IVs in section 1.2. This is where you list everything – even those IVs that aren’t ultimately retained. If the evidence for the IV is based on key stakeholder interviews or focus groups, we need to know how, who, when, how many, etc. Are these based on the responses of 1 or 2 individuals? Were there corroborating data? 16 SAPC Strategic Plan (1.2) – Tips

Intervening Variables: [what is contributing to the consumption pattern?] 17 Intervening Variables: In-Depth

What can we learn from looking back at MassCALL2, PFS-II, and MOAPC sites about identifying and collecting IVs? I came up with three big questions in this area… 1.How did sites approach IV data collection? 2.What questions guided these efforts? 3.What consumption IVs did sites identify and pursue? 18 IVs – Identification and Collection

Where did former grant sites turn to identify/generate IV data? – Key stakeholder interviews – Focus groups – Surveys – Youth – Parents – Community – Key stakeholder – Geographical mapping (e.g., outlet density) – School administrative data – Hospital data – Police data – Tx provider data 19 IVs – Identification/Collection

Treatment Providers Social service providers/outreach workers Education and High Education – administrators, guidance, wellness staff Hospital/ED personnel – including SBIRT Law Enforcement Mental Health Service Providers Local Government Parents and Guardians Recovery Community Cultural Spokespersons Faith Community Justice/Corrections 20 IVs – Sample Key Stakeholders

Community A: Describe the situation in terms of how severe the problem is; how the problem has changed over time; the priority groups – including age, gender, race/ethnicity who are engaging in the behavior; when and where the behavior occurs; and risk factors that drive the behavior. Community B: Is there something about this area that makes people more likely to engage in the behavior? Is there a community attitude or characteristic that you think contributes and, (b) Have you noticed patterns in availability of the substance? Community C: Assess the problem in community; what is currently occurring to prevent the behavior? Community D: What is the nature and extent of problem; who is the priority population(s); what factors are driving the problem? 21 IVs – Sample Key Stakeholder Questions

What are some consumption IVs? – Low levels of law enforcement – Community norms favorable to use – Perception of parental disapproval or attitude – Family communication – Parental monitoring or family management practices – Availability/Ease of Access – economic, retail, social – Perception of harm/risk – Perception of peer disapproval or attitude – Close friends who use/abuse – Attachment to school and family Representation across individual, peer, family, and community domains. 22 IVs – Sample Consumption IVs

Any questions specific to Section 1.2 – Assessment Data on Underage Drinking and Other Drug IVs? Does anyone want to discuss any successes they have had during the IV assessment that could benefit others? Any interesting or novel data sources or findings that others may want to explore? Any challenges that others may encounter? 23 Questions About Section 1.2 or IVs

24 CAPACITY BUILDING: Sections

2.1: Community and Key Stakeholder Involvement Key sectors currently collaborating; colleges and universities; general community; sectors not yet represented. 2.2: Structure and Functioning Org chart; team functioning; decision-making; challenges and solutions. 2.3: Core Planning Committee Membership; challenges and solutions 2.4. Capacity Building Strengths; areas of growth; capacity-building action plan 2.5. TA Needs 25 SAPC Strategic Plan Requirements ( )

2.1: Are there actual concrete plans to engage the community and sectors not yet involved? Is there good representation? Which sectors are missing? 2.2: What structures have been put into place and how are they functioning? 2.3: Who is on the core planning committee? How are they supporting the development of the strategic plan? 2.4: Have strengths and areas of growth been identified? Is there a concrete capacity building action plan? Were gaps/capacity needs identified in Section 1 carried forward here as needed areas of growth? 26 SAPC Strategic Plan ( ) – Expectations and Tips

27 Prioritizing Intervening Variables

This step should not occur until after the 1/29/16 deliverable when you write section 3.2 of the strategic plan. The first bullet in Section 3.2 asks for the final set of intervening variables from section 1.2 that you selected, including how this list was selected (prioritized) from among the larger list of variables considered. This assumes that you conducted a comprehensive assessment of IVs and that you need to pare down this list into a more manageable/practical subset. Critical step that ultimately helps point to the strategies you will put in place. 28 IVs – Prioritization

Importance Changeability Feasibility Capacity Readiness Resource availability Supported by data – qualitative and quantitative Data quality 29 Corroborated by different sources Cultural Competence Evaluability – data available Fit – relevant to issue/community Potential for unintended consequences Sustainability Wisdom of Practice Present in multiple communities It is helpful to generate a list of criteria on which each IV can be examined and compared. Potentially relevant criteria include: IVs – Sample Prioritization Criteria

This is an example from a former grantee on how you might frame a set of questions on multiple criteria: – Is the IV identified independently by multiple sources? – How reliable and valid are the data supporting it? – Have local changes in the variable produced changes in use? – How actionable is the variable? – Is it feasible to address the variable within the time frame of this grant? – Are other efforts in place to change this variable? – Are there capacity and resources available or that could be developed to address the intervening variable? 30 IVs – Sample Prioritization Criteria Questions

Which groups/bodies should make the final decision? Small group of cluster/steering committee representatives attempt to reach consensus based on available evidence? Develop an IV Workgroup that reports to the steering committee? Each cluster solicits input from key stakeholders and brings this to the SC? How, if at all, are community members included? How, if at all, are principles of cultural competence woven in? 31 IVs – Sample Decision-Making Processes

Work from the assessment to the literature. There is a danger that starting from a “list” will be self-fulfilling – you will find what you are looking for rather than finding the major factors that operate in your community. Once you have identified the major IVs in your site then turn to reviews of the literature to determine whether or not it has been linked to consumption. 32 IVs – Ideas, Tips, and Resources

You probably want to consider around 6 IVs for each consumption pattern and narrow this down to 3. IV’s may differ from community to community and gaps are OK. If an IV does not appear in a list or a review, it is worth going into the literature yourself – or asking for help from BSAS/MassTAPP. Terminology will sometimes differ across articles and reviews. 33 IVs – Ideas, Tips, and Resources

SAMHSA’s CAPT websiteCAPT – Risk and Protective Factors Risk and Protective Factors – Sample Risk Factors for Underage Drinking Sample Risk Factors for Underage Drinking – R&P Factors for Drinking Among Adolescents and Young Adults R&P Factors for Drinking Among Adolescents and Young Adults – Risk and Protective Factors for Marijuana Use Risk and Protective Factors for Marijuana Use NIDA website NIDA – Preventing Drug Abuse Among Children and Adolescents Preventing Drug Abuse Among Children and Adolescents BSAS website BSAS – Risk and Protective Factors Risk and Protective Factors Literature – General Causal Model for Alcohol, Tobacco, and Illicit Drug Prevention General Causal Model for Alcohol, Tobacco, and Illicit Drug Prevention 34 IVs – What Lists or Resources Are Out There?

Step 3 in the SAPC Guidance Document (pg 31-32) discusses prioritizing IVs.Guidance Document Emphasis is on Importance and Changeability – the latter includes many of the sub-categories we have discussed (e.g., readiness, resources, suitable interventions, time required for change to occur) Other potential criteria include: – Directionality – can you interpret the change? – Resource/Need gap – are resources already going to this? – Political will – is there political/public will to address this factor? – Distance – is the IV proximal enough to affect change/measure? 35 High Importance Low Importance IVs – Revisiting Prioritization

36 Importance CHANGEABILITY HighLow High Low Intervening Variables Criteria Feasibility Data AvailabilityPreventability Low peer disapproval Lack of parental knowledge High Social Availability IVs – Revisiting Prioritization

37 IVs – Revisiting Prioritization

Efficiency and Synergy – Linkages between intervening variables – Multiple IVs that can be impacted by a single intervention – Is the IV related to other health or behavioral health issues? Appropriateness – Has the IV been shown to be linked to consumption with the group that you are targeting? Age? Gender? Race/Ethnicity? SES? Urban, Suburban, Rural? Etc. Acceptability – Did the process and final decision-making process provide adequate opportunity for community input? Are there missing segments? Was the assessment comprehensive and inclusive? 38 IVs – Additional Things to Consider

Any questions specific to the IV prioritization process? 39 Questions About IV Prioritization

40 When in Doubt Call Them – Not Me Melissa Ward Picture to Come!