1. 2016 HYS Regional Workshop Using your HYS Results April/May 2017
2016 HYS Regional Workshop
Using your HYS Results
April/May 2017
Krissy Johnson, Office of Superintendent of Public Instruction
Rebecca Grady, DSHS Division of Behavioral Health and Recovery
Susan Richardson, Looking Glass Analytics

2. Welcome! General housekeeping Powerpoint and workbook
Find out who is here
Does anyone need AskHYS access?

3. Workshop purpose and objectives
Help people understand and use their HYS results:
Accessing your data
HYS background & administration
Statistical issues
Results available on AskHYS
Talking about your results

4. Introduce yourself and share one thing you want to get out of today
Introductions
Introduce yourself and share one thing you want to get out of today

5. The Importance of HYS
“The Healthy Youth Survey is important because it is a way for us to hear directly from students, without adults filtering their answers. It is up to us to provide programs and services that will help students succeed, and what better way to know what’s on their hearts and minds than by asking them?”
- Supt. Chris Reykdal

6. Getting Access to AskHYS

7. Levels of access
County, ESD and State results are available to everyone.
School district and school building HYS results are available to school and district employees through EDS, with permission from District Data Security Manager.
Others only with the permission from district superintendent.

8. Log on for school district and building access
Use your EDS login and password
Contact your District Data Security Manager for appropriate permissions to your building or district data

9. Open AskHYS and log in www.AskHYS.net
Schools and districts go to the Log On page

10. HYS Background & Administration

11. Past youth surveys U W Y W H 88 90 92 95 98 98
H H
U Student Alcohol and Drug Use Survey
W Washington State Adolescent Health Behaviors
Y Youth Risk Behavior Survey
H Healthy Youth Survey

12. Statewide support for HYS
+ Schools, ESDs, Local Health
& Community Partners!

13. Requirements for surveying
Students are considered a vulnerable population. The Washington State Institutional Review Board (IRB) requirements protect students.
Requirements include:
Approval of all HYS materials and methods
Parent notification
Minimum reporting requirements
Same day administration
Anonymous and voluntary
Students must be provided with an alternate activity

14. 2016 HYS participation Over 231,000 students… In 236 school districts…
In all 39 counties…
In 236 school districts…
In 1,034 schools…

15. Small School District options
Can survey extra grades 7, 9 and 11
Small district = 150 or less in a grade
In 2016, 108 districts participated (150 or less in a grade)
Can get combined Middle (6/7/8) & High School (9/10/11/12) results
Goal = increase the number of respondents, smaller confidence intervals, more stable results

16. How to register
Online registration opens in late January and closes in June.
Check with your district to see if someone will register all schools
The next administration is October Registration will be open from January - June of 2018.

17. Interleaved survey Forms A & B for 8th, 10th and 12th graders
Also for 9th & 11th graders

18. Survey Form C for 6th graders Also for 7th graders

19. Survey form content Core All students Form C All 6th graders Form A
Form B
½ students

20. Perforated “tear-off” pages
Optional questions
Form A = Form B = 5
Perforated “tear-off” pages

21. New questions in 2016 More categories of Asian/Pacific Islander
Hours worked
Free/reduced lunch
E-cigs – type of vape liquid, usual source, use on school property, perceived risk
Marijuana – how long usually high
Swimming competency
Insulted at home
Person to talk to when depressed
Received sexual photo (“sext”)

22. Feedback on 2016 Process What went well with administration this year?
What challenges did you face?

23. Statistical Issues: Fundamentals & Understanding Your Results

24. Concerns about the quality of HYS Results
Do students tell the truth?
Can I trust my HYS results?

25. Reliability and validity
Image from Wikipedia
Reliability is consistency.
Standard survey administration procedures
Validity is accuracy.
Questions from established surveys
Data cleaning procedures

26. Reliability = Students do tell the truth
Administration Procedures:
Standardized and consistent since 2002
Importance of survey explained
No names, only students see their answers
Goal: Create an environment where students feel the survey is important and they safe to answer honestly

27. Validity = We can trust HYS results
HYS uses questions from established surveys.
Data Cleaning Procedures, that remove respondents with:
Inconsistent answers (3)
High level of substance use (all substances, every day)
Dishonest
Fake drug
Wrong grade
Goal: Clean quality results

28. Do my HYS results represent my whole school or district?
What was your response rate?
Did specific groups of students not participate?
Need to consider how these challenges affect interpretation of the results

29. What if my results are not representative?
Use caution in interpreting the results
Use caution when comparing your results over time
When talking about your results, explain any limitations:
Students at our school who took the survey said….

30. Open your Frequency Report
What was your participation rate?
Who was missing during the survey?

31. Understanding N’s N for a core question (all/most students)
N for question on Form A only (half the students)
N for optional question only on Form B (half the students, and some schools requested surveys without option questions)
N for near the end of the survey

32. What are confidence intervals and why should I care?
A confidence interval accounts for the fact that the reported value is probably a little different than the true value for all of the students
With a 95% confidence interval, we are 95% confident that the true value is within the ± range
Confidence intervals are a tool to help you interpret your results

33. What do CIs look like?
Your survey results say 18.0% (± 2.0%) of Grade 10 students used marijuana.
18.0 – 2.0 = 16.0
= 20.0
Interpret as between 16.0% and 20.0% used marijuana and looks like:

34. What do CIs look like?
Your survey results say 18.0% (± 5.0%) of Grade 10 students used marijuana.
18.0 – 5.0 = 13.0
= 23.0
Interpret as between 11.0% and 23.0% used marijuana and looks like:

35. Why are CIs different sizes?
The size of a confidence interval is affected by:
Number of students.
In general, the more students surveyed, the smaller the confidence interval.
Inherent variability.
If most students answer a survey question in the same way, then there is less variability. The more variable the answers, the wider the CIs.

36. Why do we need CIs if data are valid?
CIs account for variability, NOT the validity of the data.
CIs allow for the comparison of results to others and to ourselves over time.
To protect you---say, from exaggerated claims.

37. What is “statistical significance” and when does it matter?
The probability that differences in results are not due to chance alone.
When using 95% confidence intervals, a difference between two groups is considered statistically significant if chance could explain it only 5% of the time or less.

38. Comparing CI’s Local students 25% ±5% (so range is 20% to 30%)
State students 36% ±3% (so range is 33% to 39%)
CI’s don’t overlap, so difference is significant
20% %
33% 39%

39. Tool for testing significance
If you need to know for sure, there is a “Tool” to test for significance at:
The spreadsheet tests the difference between two point estimates and their 95% CI to compute a p-value.
If your p-value is less than 0.05, then your difference is significant
Only use this test if you have at least 30 students
Don’t use this test if you have 0% or 100%

40. Using results with large confidence intervals
If you are a small school, district or county your CIs will be large...
The estimate may be good, but only of the students who took the survey this year.
CIs don’t take participation rates into account:
Low participation – there may be additional bias
High participation – you can feel more confident in your point estimates.

41. Deadly sins of statistical misinterpretation
Assuming differences are significant
Equating significance with real-world importance
Implying causation
Forgetting to consider outside causes
Creating deceptive graphs
Trusting coincidence
Source: theconversation.com

42. Source: tylervigen.com

43. Quick AskHYS Walkthrough

44. What is available on AskHYS?
Frequency Reports
Fact Sheets
Other resources

45. Turn, Talk & Play
Spend some time looking at your results.

46. Comparisons over time A few things to think about
Did the survey question change?
Did you expect a change?
Were challenges to generalizability similar?
What is a trend and do you have one?
Fact sheet show “trends” from 2006, but do not test trend, they only test year-to-year change. To measure trends you need statistical software:
Free software from the National Cancer Institute – uses your point estimates and confidence intervals:

47. Q x Q = Question by Question
What can you do with it?
Run frequencies: responses to individual questions. Just like the results in your Frequency Reports.
Run crosstabs: looking at the relationship between two questions - crossing one question by the second question.

48. Q x Q limitations Before you start, remember there are limitations
You many not have results to analyze
Check Past Participation
The questions may not be on the same Forms and can’t be crossed
Check the Data Dictionary/Crosswalk
Cell size
State level minimum 5 per cell.
Sub-state level minimum 10 per cell.

49. HYS Results & Talking About Them

50. School and district use of HYS results
Common Uses
HYS data works as a great backbone in grant applications
Counselors around the state use HYS data learn about student views
“It informs administrators and counselors about their students and the needs they have for support. We develop plans to systematically address student issues and reflect on annual data for improvement.”
“We look at the data and make decisions on areas to focus on with our Intervention/Prevention person and with help from the domestic violence shelter personnel in our area. We also use info for decision in health classes.”
- Based on an 2016 survey of all HYS coordinators

51. State use of HYS results
OSPI
Graduation: A Team Effort (GATE) Initiative
Project AWARE: Mental Health Grant
Division of Behavioral Health and Recovery:
It is a primary source of data for Washington State’s Substance Use Prevention and Mental Health Promotion Strategic Plan.
Liquor and Cannabis Board
Assessing policy and programmatic implications and informing change.
Providing educational information on our website.
Department of Health
Legislative purposes – most recently T-21 bill and E-cig bill
Needs assessment data used as part of the Comprehensive Needs Assessment for the Maternal and Child Health Block Grant.
Governor’s Office
Results Washington & Healthiest Next Generation

52. Why do you want to talk about your results?
Share information about our kids
Try to change or open people’s minds
Help plan or make decisions
Help evaluate effectiveness
Ask for help, a change, or money

53. Talking about survey results
Survey results are estimates
Round to a whole number
Say “about”
Carefully include your confidence interval
If 75.3% (5), between 70 and 80 percent
Plus or minus 5 percent
Acknowledge low participation – “among those who took the survey”

54. Use visuals There are charts in your PowerPoint slides and fact sheets
Create other types of charts, visuals or info- graphics:
Examples of software – Tableau, Piktochart, Venngage, Canva
7 ½ out of 10 students…

55. Be able to back up your talk
Know your facts about HYS
How, when and to who it was administered
Details about the questions
Do the numbers make sense?
Are you using them in an appropriate context?
What else is needed to tell the whole story?
Additional data sources
Information from informed people

56. Talking tips Keep it short and simple Double check your numbers
Run it past a few people, especially a critic
Don’t speculate. Remember your limitations
It’s ok to say “I don’t know, I’ll get back to you”
Be compassionate - these numbers are actually kids
Tell your audience what you want them to learn or do

57. Where do you start? What information do you have?
HYS reports, slides, fact sheets.
What/who else can help you:
Are other data available? (e.g. school discipline data)
Are there people who have more information?
Who are your allies?
What do your 2016 HYS results say?
What HYS questions address school safety?
Are results different, have they changed, or are they just important?

58. Developing your talking points
If you could only make one point – what would it be?
Are their specific opinions you need to influence?
What do you ultimately want to happen?

59. What’s your Communication Objective?
This is the “so what” of your message. Ask yourself:
“If I could only make one point, what would it be?”
Tells your audience what you want them to learn or do

60. Dig into your HYS results and practice creating a message
Choose a topic area of concern or celebration, using your data as a guide
Identify who you want to share this data with in your organization and broader community
Who do you need to connect with to develop your message and action plan? Who are your allies?
Who is your audience for the message? The decision makers? Who do you want to do something?
Identify other sources of data or information that can help you contextualized
Use the worksheet to practice honing in on your message

61. Questions?

62. Additional resources Life Skills curriculum www.StartTalkingNow.org
Community coalition: Community Prevention and Wellness Initiative (CPWI) or Drug Free Community Coalition (DFC)
Your Educational Service District
Your local health jurisdiction
Opiate Awareness Social Media Campaign – April 19 – May 31

63. Thank you! From the Healthy Youth Survey Planning Committee:
Division of Behavioral Health and Recovery (DBHR): Rebecca Grady, Sarah Mariani, Can Du
Office of Superintendent Public Instruction (OSPI): Krissy Johnson, Dixie Grunenfelder
Department of Health (DOH): Kevin Beck, Anar Shah, Jennifer Sabel, Cathy Wasserman, Riley Peters
Washington Liquor and Cannabis Board (WSLCB): Mary Segawa
Looking Glass Analytics (LGAN): Joe Kabel and Susan Richardson

64. Questions? About this training:
Krissy Johnson:
Rebecca Grady:
Susan Richardson:
Other HYS questions:

65. Overview of 2016 Results

66. Marijuana Use: Past 30 Days
Decreasing trends for 6th and 8th graders, but an increasing trend for 12th graders
Decrease in the past 2 years for 6th graders
Source: Washington Healthy Youth Survey through 2016

67. 10th Grade: Easy* Perception of Access
Decreasing trends for alcohol and cigarettes
Decreases in the in the past 2 years for all substance
*Sort of easy + Very easy
Source: Washington State Healthy Youth Survey though 2016.

68. 10th Grade: Perception of Great Risk from Substance Use
Increasing trends for alcohol and cigarettes, but an decreasing trend for marijuana
No changes in the past 2 years for any substance
Source: Washington State Healthy Youth Survey though 2016.

69. 10th Grade: Past 30 Day Marijuana Use by Race/Ethnicity
No difference by gender
Higher among Hispanics, American Indian/Alaska Native, Black/African American or Hawaiian/Pacific Islander compared to white 10th graders. Asian lower than white 10th graders.
Source: Washington State Healthy Youth Survey – 2016.

70. E-cigarette/Vape Use: Past 30 Days
Decreasing trends for 6th and 8th graders, but an increasing trend for 12th graders
Decrease in the past 2 years for 6th graders
Source: Washington Healthy Youth Survey – 2014 and 2016

71. Mental Health Status 1 out of 3 10th graders had depressive feelings
1 out of 5 10th graders had serious thoughts about suicide
1 out of 10 t0th graders attempted suicide
Note: Depressive feeling: felt so sad or hopeless for two weeks in a row that they stopped doing usual activities.
Source: Washington State Healthy Youth Survey – 2016.

72. Mental Health Status 1 out of 3 10th graders had depressive feelings
1 out of 5 10th graders had serious thoughts about suicide
1 out of 10 t0th graders attempted suicide
Note: Depressive feeling: felt so sad or hopeless for two weeks in a row that they stopped doing usual activities.
Source: Washington State Healthy Youth Survey – 2016.

73. 10th Grade: Substance Use and Mental Health Status
10th graders experiencing depressive feelings were also more likely to report substance use.
Use Alcohol
Use Marijuana
Use Alcohol
Use Marijuana
Smoke Cigarettes
Use Pain Killers
Smoke Cigarettes
Use Pain Killers
Source: Washington State Healthy Youth Survey – 2016.

74. Risk and Protective Factors Scales
Protective - School Opportunities for Prosocial Involvement:
Decisions about class activities
Talk to teacher one-on-one
Work on special projects
Get involved in extracurricular activities
Class discussion involvement
Risk - Academic Failure:
Usual grades in school
Grades better than other students

75. Protected by “School Opportunities” ?
10th graders who are protected by “School Opportunities” have lower substance use rates.
* Other drug use not including alcohol, tobacco , or marijuana.
Source: Washington State Healthy Youth Survey – 2016.

76. At Risk for “Academic Failure”?
10th graders with the risk factor “Academic Failure” have higher rates of substance use than students without that risk factor.
* Other drug use not including alcohol, tobacco , or marijuana.
Source: Washington State Healthy Youth Survey – 2016.