1. SPF SIG State-Level Evaluation PARTICIPANT LEVEL INSTRUMENT (PLI)

2.  SAMHSA/CSAP cross-site evaluation overview  Purpose of the PLI  Data collection required for the PLI  Roles and responsibilities  Timing and submission deadlines  Resources  Q & A WEBINAR AGENDA

3.  All communities funded by SPF SIG are required to submit cross-site evaluation data.  Quick Cross-site evaluation terms in cross-site evaluation tools:  “Grantee” (State of AK who received the SPF SIG)  “Subrecipients” (6 funded communities in Alaska) SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

4.  CLI: Part 1 –progress through the Strategic Prevention Framework  CLI: Part 2 –information about the specific interventions being implemented (adaptations, # reached, cost, demographics)  PLI – participant change in alcohol consumption resulting from individual level interventions  Completed annually throughout the SPF SIG (Steps 1-5)  Completed quarterly for each intervention once implementation begins (Step 4)  Completed by participants of individual level interventions at pre, post, & follow-up (Step 4) SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

5. 5 PURPOSE OF THE PLI  Why does SAMHSA require this?  Determine the specific types of interventions and strategies that work, for which populations, and under what circumstances.  Determine strategies that are most effective in producing outcomes related to alcohol consumption One version for adults and another for youth

6. Environmental strategies:  All middle/high school youth in your community  All adults in the community (examples: media campaigns, policy change in schools, limiting hours of operation in bars/liquor stores) Individual-level strategies:  “Captive Audience” – Face to face contact with individuals is a key component  target small sub-groups or individuals in the community (examples: Education program for adults who get a DUI, Lifeskills camp for youth) WHAT ARE INDIVIDUAL-LEVEL STRATEGIES?

7. Interventions/ Strategies are planned activities intended to change alcohol related behavior (not building general capacity) REMINDER ABOUT STRATEGIES

8. The minimum data collection requirement includes:  A Baseline Survey. Baseline data collection must be conducted before the participant has been exposed to any intervention services and can be conducted any time during the 30 days preceding program entry.  An Exit Survey. The exit survey must be administered after the participant has completed the entire program but no later than 10 days after program completion.  A Followup Survey. The follow-up survey must be administered to program participants at least three months after program exit. Paper and pencil format WHEN TO ADMINISTER THE PLI

9.  You can add additional evaluation measures specific to the goals of your strategy  Also fulfills state requirement of tracking change in alcohol consumption for each strategy  Help you determine the reach of your strategy which is essential for process evaluation  Help to complete the demographic information required for each strategy on the CLI: Part II. USEFULNESS OF PLI

10.  Every person who participates in an individual-level strategy must have a unique ID  This same ID is used on each PLI survey participants complete (baseline, exit, and followup)  Following up with participants more than 3 months after the program is difficult  Issues around parental consent for surveys done in schools (for youth)  SAMHSA may decide to change some of the survey items in the future CHALLENGES OF THE PLI

11. We can provide generic examples of the following to use if helpful or needed:  Parental consent forms  Youth assent forms  Consent to Contact forms for follow-up  Letter describing the SPF SIG and the purpose of the PLI survey to inform others as needed Your local evaluator likely has forms they have used before HOW CAN WE HELP YOU?

12. Required PLI Questions  Gender (Q1)  Ethnicity (Q2-3)  Age (Q4)  Military service (Q5-6)  Past 30 day alcohol use (Q7C)  Age at first use of alcohol (Q8C)  Approve of someone your age having 1-2 drinks a day (Q9E)  Perceived risk of having 5+ alcoholic drinks 1-2x/week (Q10C)  Questions about attitudes/experiences (Q11-14)  Days (in past 30 days) with 5+ drinks in couple of hours (Q17)  Average # drinks on days you drank in past 30 days (Q18) CONTENT OF THE PLI -YOUTH

13. Optional Alcohol-Related PLI Questions  How many days consumed alcohol during your life? (Q15)  How easy is it for you to get alcohol? (Q19)  How many students in your grade drink alcohol? (Q20)  How many closest friends been drunk in last 30 days (Q21)  How do you usually get alcohol? -Multiple choice (Q22)  Thoughts and feelings about alcohol use (Q23a, b, c)  Where and when have you used alcohol in past 30 days? (Q24) CONTENT OF THE PLI -YOUTH

14. Required PLI Questions  Gender (Q1)  Ethnicity (Q2-3)  Age (Q4)  Military service (Q5-7)  Past 30 day alcohol use (Q8C)  Age at first use of alcohol (Q9C)  Perceived risk of having 5+ alcoholic drinks 1-2x/week (Q10C)  Questions about attitudes/experiences (Q11-13)  Days (in past 30 days) with 5+ drinks in couple of hours (Q16)  Average # drinks on days you drank in past 30 days (Q17) CONTENT OF THE PLI - ADULTS

15. Optional Alcohol-Related PLI Questions  How many days consumed alcohol during your life? (Q14)  In past 30 days, how many times did you ride with someone who had been drinking? (Q18)  In past 30 days, how many times did you drive a car or other vehicle when you had been drinking? (Q19) CONTENT OF THE PLI- ADULTS

16.  Surveys submitted manually online https://www.pmrts.samhsa.gov/pmrts/  Project Directors and Evaluators are responsible for ensuring data is entered  An Excel file can be downloaded at any time which will include all PLI data you have entered to-date.  Excel file can be used for local data analyses and to enter any added survey variables.  Reminders will be sent by state evaluators of upcoming deadlines ENTERING PLI DATA

17. Quarterly Survey collection period Data Entry Due Dates 1.1.13 - 3.30.134.15.13 4.1.13 - 6.30.137.15.13 7.1.13 - 9.30.1310.15.13 10.1.13 - 12.31.131.15.14 SUBMISSION DEADLINES FOR PLI …..Continue with same schedule throughout life of grant  Surveys should be entered as soon as possible upon completion  No later than by the end of the quarter

18. 18  One-on-one assistance by state evaluators  Submission deadlines schedule  All surveys, guides, codebooks, webinars etc. will be put out on the SPG SIG website by Oct 31 http://alaskaspfsig.org/ Click: Tools + Resources tab Under: Evaluation  Frequently Asked Questions  Federal Cross-site evaluation team collects grantee questions and disseminates FAQ document to grantees regularly. WHAT OTHER ASSISTANCE IS AVAILABLE?

19. 19 WHAT ASSISTANCE IS AVAILABLE TO COMMUNITIES? (CONT’D)  On-line Instrument Training Video  Interactive videos demonstrating navigation through the web-based MRT system. http://kitusers.kithost.net/support/diticsupport/MRT/SPFSIGInstrument sGLICLIPLITrainingVideos/tabid/1114/Default.aspx http://kitusers.kithost.net/support/diticsupport/MRT/SPFSIGInstrument sGLICLIPLITrainingVideos/tabid/1114/Default.aspx  Help Line  If you experience technical difficulties in using the web-based system, contact the DITIC Help Desk by email or phone 1-888-DITIC-4-U (1-888-348-4248) Monday - Friday, 9:00 am - 8:00 pm EST