Lessons Learned: The Minnesota Young Adult Alcohol Survey American Evaluation Association Annual Conference October 26, 2012

Young Adult Alcohol Survey (YAAS) Background In 2009 the Minnesota Department of Human Services received a Strategic Prevention Framework State Incentive Grant (SPF SIG) from the federal Substance Abuse and Mental Health Services Administration (SAMHSA). 2

YAAS Background (continued) Data on young adult alcohol use is needed to assess and evaluate the interventions performed locally. Invitation Health Institute was contracted to conduct the Minnesota YAAS. Young adult populations are extremely difficult to survey. 3

YAAS Administration Round One was conducted in the spring and summer of 2012 in eight SPF SIG communities and three comparison sites. Round Two will be conducted in 2014 to evaluate the SPF SIG program. Mail-based mode of administration was chosen. All mailings included a URL for an online responses. Attempts were made to contact up to 1,200 young adults aged in each community. 4

YAAS Administration The survey design Employed a version of Dillman’s Tailored Design Methodology rooted in social exchange theory Respondent participation motivated by the return on their actions (not necessarily monetary) Takes into account changes in technology Recommends specific sequences of survey mailings 5

YAAS Administration Four mailings were sent: 6 Pre- notification postcard Initial survey packet including incentive information Follow-up survey packet Reminder postcard

YAAS Administration Small populations of some of the SPF SIG communities combined with narrow sample frame (for example,18-25 year olds) resulted in lower than normal list accuracy. – Information appended onto address-based samples more likely to be incorrect or missing – Not an issue for phone surveys, or surveys of whole populations Low list accuracy makes calculating precise response rates impossible. 7

YAAS Administration A total of 1,733 responses were received from all 11 communities. Non-response bias was associated with several demographic characteristics. Weighting was used to correct for non- response bias associated with gender and age. – Adding weights that include additional variables, such as race and ethnicity, would have required the use of U.S. Census microdata, a more intensive process. 8

Respondent Gender 9

10

Respondent Age 11

Respondent Age 12

Respondent Race and Ethnicity 13

Respondent Sexual Orientation 14

Mode Effects Key Findings 15

Mode Effects The mode of survey administration (mail vs. online) did not have a significant effect on reported alcohol use. Respondents who completed the survey online were more likely to report the following: Past 30-day marijuana use Employment Being a student Rural residence 16

Mode Effects 17

Mode Effects 18

Mode Effects 19

Mode Effects 20

Other Initial Findings Males were significantly more likely to report past 30-day binge drinking. No significant difference in any past 30-day alcohol use between genders Whites were more likely to report past 30-day use and past 30-day binge drinking compared to all non-whites 21

Results by Gender 22

Results by Race and Ethnicity 23

Results by Race and Ethnicity 24

Results by Sexual Orientation Respondents who identified as lesbian, gay, bisexual, transgender or queer (LGBTQ) were not more likely to report past 30-day use or binge drinking. LGBTQ respondents were more likely to report: Past 30-day tobacco use Past 30-day marijuana use Past 12-month marijuana use Past 12-month other illicit drug use 25

Results by Sexual Orientation 26

Implications and Next Steps SPF SIG grantees are currently using their YAAS data to perform assessment activities in their communities. This, coupled with other assessment activities, will affect which interventions grantees choose. When the SPF SIG enters the evaluation phase in 2014, the YAAS will be performed again. Analysis will measure any change in: Past 30-day binge drinking Past 30-day alcohol use Changes in “intervening variables” 27

Considerations for Implementation Surveying small populations is very difficult. Narrow sample frames can result in low list accuracy. Organizations need sufficient capacity. Mail-based survey cost considerations: Printing Postage Mailing service Staffing 28

Questions? 29

Contact Information Bill Woywod, M.P.H., biostatistician (763) Becky Sechrist, M.Ed., associate executive director (763)