2016 HYS Regional Workshop Using your HYS Results April/May 2017   2016 HYS Regional Workshop Using your HYS Results April/May 2017 Krissy Johnson, Office of Superintendent of Public Instruction Rebecca Grady, DSHS Division of Behavioral Health and Recovery Susan Richardson, Looking Glass Analytics

Welcome! General housekeeping Powerpoint and workbook Find out who is here Does anyone need AskHYS access?

Workshop purpose and objectives Help people understand and use their HYS results: Accessing your data HYS background & administration Statistical issues Results available on AskHYS Talking about your results

Introduce yourself and share one thing you want to get out of today Introductions Introduce yourself and share one thing you want to get out of today

The Importance of HYS “The Healthy Youth Survey is important because it is a way for us to hear directly from students, without adults filtering their answers. It is up to us to provide programs and services that will help students succeed, and what better way to know what’s on their hearts and minds than by asking them?” - Supt. Chris Reykdal

Getting Access to AskHYS

Levels of access County, ESD and State results are available to everyone. School district and school building HYS results are available to school and district employees through EDS, with permission from District Data Security Manager. Others only with the permission from district superintendent.

Log on for school district and building access Use your EDS login and password Contact your District Data Security Manager for appropriate permissions to your building or district data

Open AskHYS and log in www.AskHYS.net Schools and districts go to the Log On page

HYS Background & Administration

Past youth surveys U W Y W H 88 90 92 95 98 88 90 92 95 98 99 00 02 04 06 08 10 12 14 16 H H U Student Alcohol and Drug Use Survey W Washington State Adolescent Health Behaviors Y Youth Risk Behavior Survey H Healthy Youth Survey

Statewide support for HYS + Schools, ESDs, Local Health & Community Partners!

Requirements for surveying Students are considered a vulnerable population. The Washington State Institutional Review Board (IRB) requirements protect students. Requirements include: Approval of all HYS materials and methods Parent notification Minimum reporting requirements Same day administration Anonymous and voluntary Students must be provided with an alternate activity

2016 HYS participation Over 231,000 students… In 236 school districts… In all 39 counties… In 236 school districts… In 1,034 schools…

Small School District options Can survey extra grades 7, 9 and 11 Small district = 150 or less in a grade In 2016, 108 districts participated (150 or less in a grade) Can get combined Middle (6/7/8) & High School (9/10/11/12) results Goal = increase the number of respondents, smaller confidence intervals, more stable results

How to register Online registration opens in late January and closes in June. Check with your district to see if someone will register all schools The next administration is October 2018. Registration will be open from January - June of 2018.

Interleaved survey Forms A & B for 8th, 10th and 12th graders Also for 9th & 11th graders

Survey Form C for 6th graders Also for 7th graders

Survey form content Core All students Form C All 6th graders Form A Form B ½ students

Perforated “tear-off” pages Optional questions Form A = 1 Form B = 5 Perforated “tear-off” pages

New questions in 2016 More categories of Asian/Pacific Islander Hours worked Free/reduced lunch E-cigs – type of vape liquid, usual source, use on school property, perceived risk Marijuana – how long usually high Swimming competency Insulted at home Person to talk to when depressed Received sexual photo (“sext”)

Feedback on 2016 Process What went well with administration this year? What challenges did you face?

Statistical Issues: Fundamentals & Understanding Your Results

Concerns about the quality of HYS Results Do students tell the truth? Can I trust my HYS results?

Reliability and validity Image from Wikipedia Reliability is consistency. Standard survey administration procedures Validity is accuracy. Questions from established surveys Data cleaning procedures

Reliability = Students do tell the truth Administration Procedures: Standardized and consistent since 2002 Importance of survey explained No names, only students see their answers Goal: Create an environment where students feel the survey is important and they safe to answer honestly  

Validity = We can trust HYS results HYS uses questions from established surveys. Data Cleaning Procedures, that remove respondents with: Inconsistent answers (3) High level of substance use (all substances, every day) Dishonest Fake drug Wrong grade Goal: Clean quality results

Do my HYS results represent my whole school or district? What was your response rate? Did specific groups of students not participate? Need to consider how these challenges affect interpretation of the results

What if my results are not representative? Use caution in interpreting the results Use caution when comparing your results over time When talking about your results, explain any limitations: Students at our school who took the survey said….

Open your Frequency Report What was your participation rate? Who was missing during the survey?

Understanding N’s N for a core question (all/most students) N for question on Form A only (half the students) N for optional question only on Form B (half the students, and some schools requested surveys without option questions) N for near the end of the survey

What are confidence intervals and why should I care? A confidence interval accounts for the fact that the reported value is probably a little different than the true value for all of the students With a 95% confidence interval, we are 95% confident that the true value is within the ± range Confidence intervals are a tool to help you interpret your results

What do CIs look like? Your survey results say 18.0% (± 2.0%) of Grade 10 students used marijuana. 18.0 – 2.0 = 16.0 18.0 + 2.0 = 20.0 Interpret as between 16.0% and 20.0% used marijuana and looks like:

What do CIs look like? Your survey results say 18.0% (± 5.0%) of Grade 10 students used marijuana. 18.0 – 5.0 = 13.0 18.0 + 5.0 = 23.0 Interpret as between 11.0% and 23.0% used marijuana and looks like:

Why are CIs different sizes? The size of a confidence interval is affected by: Number of students. In general, the more students surveyed, the smaller the confidence interval. Inherent variability. If most students answer a survey question in the same way, then there is less variability. The more variable the answers, the wider the CIs.

Why do we need CIs if data are valid? CIs account for variability, NOT the validity of the data. CIs allow for the comparison of results to others and to ourselves over time. To protect you---say, from exaggerated claims.

What is “statistical significance” and when does it matter? The probability that differences in results are not due to chance alone. When using 95% confidence intervals, a difference between two groups is considered statistically significant if chance could explain it only 5% of the time or less.

Comparing CI’s Local students 25% ±5% (so range is 20% to 30%) State students 36% ±3% (so range is 33% to 39%) CI’s don’t overlap, so difference is significant 20% 30% 33% 39%

Tool for testing significance If you need to know for sure, there is a “Tool” to test for significance at: www.AskHYS.net/Training The spreadsheet tests the difference between two point estimates and their 95% CI to compute a p-value. If your p-value is less than 0.05, then your difference is significant Only use this test if you have at least 30 students Don’t use this test if you have 0% or 100%

Using results with large confidence intervals If you are a small school, district or county your CIs will be large... The estimate may be good, but only of the students who took the survey this year. CIs don’t take participation rates into account: Low participation – there may be additional bias High participation – you can feel more confident in your point estimates.

Deadly sins of statistical misinterpretation Assuming differences are significant Equating significance with real-world importance Implying causation Forgetting to consider outside causes Creating deceptive graphs Trusting coincidence Source: theconversation.com

Source: tylervigen.com

Quick AskHYS Walkthrough

What is available on AskHYS? Frequency Reports Fact Sheets Other resources

Turn, Talk & Play Spend some time looking at your results.

Comparisons over time A few things to think about Did the survey question change? Did you expect a change? Were challenges to generalizability similar? What is a trend and do you have one? Fact sheet show “trends” from 2006, but do not test trend, they only test year-to-year change. To measure trends you need statistical software: Free software from the National Cancer Institute – uses your point estimates and confidence intervals: http://surveillance.cancer.gov/joinpoint

Q x Q = Question by Question What can you do with it? Run frequencies: responses to individual questions. Just like the results in your Frequency Reports. Run crosstabs: looking at the relationship between two questions - crossing one question by the second question.

Q x Q limitations Before you start, remember there are limitations You many not have results to analyze Check Past Participation The questions may not be on the same Forms and can’t be crossed Check the Data Dictionary/Crosswalk Cell size State level minimum 5 per cell. Sub-state level minimum 10 per cell.

HYS Results & Talking About Them

School and district use of HYS results Common Uses HYS data works as a great backbone in grant applications Counselors around the state use HYS data learn about student views “It informs administrators and counselors about their students and the needs they have for support. We develop plans to systematically address student issues and reflect on annual data for improvement.” “We look at the data and make decisions on areas to focus on with our Intervention/Prevention person and with help from the domestic violence shelter personnel in our area. We also use info for decision in health classes.” - Based on an 2016 survey of all HYS coordinators

State use of HYS results OSPI Graduation: A Team Effort (GATE) Initiative Project AWARE: Mental Health Grant Division of Behavioral Health and Recovery: It is a primary source of data for Washington State’s Substance Use Prevention and Mental Health Promotion Strategic Plan. Liquor and Cannabis Board Assessing policy and programmatic implications and informing change. Providing educational information on our website. Department of Health Legislative purposes – most recently T-21 bill and E-cig bill Needs assessment data used as part of the Comprehensive Needs Assessment for the Maternal and Child Health Block Grant. Governor’s Office Results Washington & Healthiest Next Generation

Why do you want to talk about your results? Share information about our kids Try to change or open people’s minds Help plan or make decisions Help evaluate effectiveness Ask for help, a change, or money

Talking about survey results Survey results are estimates Round to a whole number Say “about” Carefully include your confidence interval If 75.3% (5), between 70 and 80 percent Plus or minus 5 percent Acknowledge low participation – “among those who took the survey”

Use visuals There are charts in your PowerPoint slides and fact sheets Create other types of charts, visuals or info- graphics: Examples of software – Tableau, Piktochart, Venngage, Canva 7 ½ out of 10 students…

Be able to back up your talk Know your facts about HYS How, when and to who it was administered Details about the questions Do the numbers make sense? Are you using them in an appropriate context? What else is needed to tell the whole story? Additional data sources Information from informed people

Talking tips Keep it short and simple Double check your numbers Run it past a few people, especially a critic Don’t speculate. Remember your limitations It’s ok to say “I don’t know, I’ll get back to you” Be compassionate - these numbers are actually kids Tell your audience what you want them to learn or do

Where do you start? What information do you have? HYS reports, slides, fact sheets. What/who else can help you: Are other data available? (e.g. school discipline data) Are there people who have more information? Who are your allies? What do your 2016 HYS results say? What HYS questions address school safety? Are results different, have they changed, or are they just important?

Developing your talking points If you could only make one point – what would it be? Are their specific opinions you need to influence? What do you ultimately want to happen?

What’s your Communication Objective? This is the “so what” of your message. Ask yourself: “If I could only make one point, what would it be?” Tells your audience what you want them to learn or do

Dig into your HYS results and practice creating a message Choose a topic area of concern or celebration, using your data as a guide Identify who you want to share this data with in your organization and broader community Who do you need to connect with to develop your message and action plan? Who are your allies? Who is your audience for the message? The decision makers? Who do you want to do something? Identify other sources of data or information that can help you contextualized Use the worksheet to practice honing in on your message

Questions?

Additional resources Life Skills curriculum www.StartTalkingNow.org www.theathenaforum.org Community coalition: Community Prevention and Wellness Initiative (CPWI) or Drug Free Community Coalition (DFC) Your Educational Service District Your local health jurisdiction Opiate Awareness Social Media Campaign – April 19 – May 31

Thank you! From the Healthy Youth Survey Planning Committee: Division of Behavioral Health and Recovery (DBHR): Rebecca Grady, Sarah Mariani, Can Du Office of Superintendent Public Instruction (OSPI): Krissy Johnson, Dixie Grunenfelder Department of Health (DOH): Kevin Beck, Anar Shah, Jennifer Sabel, Cathy Wasserman, Riley Peters Washington Liquor and Cannabis Board (WSLCB): Mary Segawa Looking Glass Analytics (LGAN): Joe Kabel and Susan Richardson

Questions? About this training: Krissy Johnson: Krissy.Johnson@k12.wa.us Rebecca Grady: Rebecca.Grady@dshs.wa.gov Susan Richardson: susan.richardson@lgan.com Other HYS questions: healthy.youth@doh.wa.gov www.AskHYS.net

Overview of 2016 Results

Marijuana Use: Past 30 Days Decreasing trends for 6th and 8th graders, but an increasing trend for 12th graders Decrease in the past 2 years for 6th graders Source: Washington Healthy Youth Survey - 2002 through 2016

10th Grade: Easy* Perception of Access Decreasing trends for alcohol and cigarettes Decreases in the in the past 2 years for all substance *Sort of easy + Very easy Source: Washington State Healthy Youth Survey - 2002 though 2016.

10th Grade: Perception of Great Risk from Substance Use Increasing trends for alcohol and cigarettes, but an decreasing trend for marijuana No changes in the past 2 years for any substance Source: Washington State Healthy Youth Survey - 2002 though 2016.

10th Grade: Past 30 Day Marijuana Use by Race/Ethnicity No difference by gender Higher among Hispanics, American Indian/Alaska Native, Black/African American or Hawaiian/Pacific Islander compared to white 10th graders. Asian lower than white 10th graders. Source: Washington State Healthy Youth Survey – 2016.

E-cigarette/Vape Use: Past 30 Days Decreasing trends for 6th and 8th graders, but an increasing trend for 12th graders Decrease in the past 2 years for 6th graders Source: Washington Healthy Youth Survey – 2014 and 2016

Mental Health Status 1 out of 3 10th graders had depressive feelings 1 out of 5 10th graders had serious thoughts about suicide 1 out of 10 t0th graders attempted suicide Note: Depressive feeling: felt so sad or hopeless for two weeks in a row that they stopped doing usual activities. Source: Washington State Healthy Youth Survey – 2016.

Mental Health Status 1 out of 3 10th graders had depressive feelings 1 out of 5 10th graders had serious thoughts about suicide 1 out of 10 t0th graders attempted suicide Note: Depressive feeling: felt so sad or hopeless for two weeks in a row that they stopped doing usual activities. Source: Washington State Healthy Youth Survey – 2016.

10th Grade: Substance Use and Mental Health Status 10th graders experiencing depressive feelings were also more likely to report substance use. Use Alcohol Use Marijuana Use Alcohol Use Marijuana Smoke Cigarettes Use Pain Killers Smoke Cigarettes Use Pain Killers Source: Washington State Healthy Youth Survey – 2016.

Risk and Protective Factors Scales Protective - School Opportunities for Prosocial Involvement: Decisions about class activities Talk to teacher one-on-one Work on special projects Get involved in extracurricular activities Class discussion involvement Risk - Academic Failure: Usual grades in school Grades better than other students

Protected by “School Opportunities” ? 10th graders who are protected by “School Opportunities” have lower substance use rates. * Other drug use not including alcohol, tobacco , or marijuana. Source: Washington State Healthy Youth Survey – 2016.

At Risk for “Academic Failure”? 10th graders with the risk factor “Academic Failure” have higher rates of substance use than students without that risk factor. * Other drug use not including alcohol, tobacco , or marijuana. Source: Washington State Healthy Youth Survey – 2016.