1 MSU Model Grant Overview Dennis Martell Sandi Smith Michigan State University

MSU MODEL GRANT What Makes MSU a Model Program? 1. Unique make up of team –Service, Survey, and Academic Experts 2. Effectiveness of global campaign to reduce extreme drinking. 3. Unique application of social norms messaging on celebratory events. 4. Unique focus on use of protective behaviors to reduce harm.

3 Outline of the Presentation Overview of evaluations Intervention design and evaluation Social norming models MSU’s social norming program and evaluation design Formative research on celebratory drinking Dissemination and enhancement strategies

4 Evaluations in the Life Course of Programs Adapted from: Rossi, P., and H. Freeman, Evaluation: A Systematic Approach Output Outcome/Impact Cost/Benefit Cost Effectiveness Decision to Act Needs Assessment Formative Evaluation Process Conformity to Plan Target Reach/Coverage Program Life Course Implementation Institutionalization Assessment of Utility Outcome Planning Monitoring Problem Evaluations Conceptualization & Design

5 To Evaluate a Program's Outcome, We Need to Know...  What was the program supposed to accomplish? (Problem to change)  How was it to accomplish this?  For whom?  Why was the intervention supposed to have the intended effect? (Conceptual Model)  Did change occur? (Outcome/Impact)  How much?  Is the change associated with the program elements?  Can alternative explanations for the change be ruled out?  How much did it cost?

6 The Social Norming Conceptual Model of High-Risk Drinking Harm Consumption Perception of Norms: Descriptive Perception of Norms: Injunctive Familiarity With the Culture Desire To be Normal Alcohol poisoning Physical injury Auto accidents Violence Sexual assault Sexual risk behavior Academic performance Type Quantity Absorption Duration Pace Protective Behaviors Setting limits1 drink per hour Designated driverDrink look-a-like Watch out for friendsStay with same group Stay with same alcoholEating food

7 The MSU Social Norming Intervention Strategy for High-Risk Drinking Harm Consumption Perception of Norms: Descriptive Perception of Norms: Injunctive Familiarity With the Culture Desire To be Normal Protective Behaviors Intervention: Posters Table Tents Intervention: Flyers Newspaper Ads AOP

8 The Initial Question: Was there a discrepancy between the perceived norm around drinking and the actual drinking norm? i.e., Was a Social Norming approach an appropriate intervention strategy?

9 Difference Between Perceived Norm and Actual Behavior Before Starting Social Norming Campaign Perceived Norm (Mean) Actual Norm (Mean) Number of Drinks Last Time Partied/ Socialized 2000 Source: MSU NCHA 2000 The average perceived number of drinks consumed by the typical student the last time s/he partied is 13% greater than the actual average }

Difference Between Perceived Norm and Actual Behavior Before Starting Social Norming Campaign 10 Overall Actual % 0-4 = (46.6%) % Who Believe Typical Student Drank 0-4 Drinks Last Time Partied/Socialized by Gender and Status, 2000 Grad % 0-4 = 57.8% Undergrad % 0-4 = 43.7% Source: MSU NCHA 2000

2007 National Social Norms Conference11 Examples of Ads, etc.

12 Data Collection Strategy for Evaluation Formative –Focus groups for ads Monitoring/Process –Web survey near end of each semester –Undergrads only –N = 1,000 – –Perceptions of others, celebration event drinking, protective behaviors, familiarity with ads Monitoring/Outcome –NCHA (web) every 2 years (Feb.-March) –Grad and Undergrad –N = 900 – 1,400 –Overall drinking, harm, protective behaviors

13 The Evaluation Questions Did the interventions reach the targeted audience? Did change occur in normative perceptions? Is change associated with the intervention? Did change occur in protective behaviors? Did change occur in drinking behavior? Did change occur in amount of harm? Is there reason to believe this isn’t just coincidence?

14 Did the Interventions Reach the Targeted Audience? Males as likely to report seeing ads as females Freshmen more than Sophomores more than Juniors more than Seniors Average number times reported seeing ads varied from 2.8 (Fall, 06) to 7.5 (Spring, 04) No difference between males and females Generally, Freshmen saw more times than Sophomores more times than Juniors more times than Seniors Source: Celebration Surveys 2-9, n=1,211; n=1,040; n=1,277; n=1,073; n=1,334; n=1,110; n=891; n=1,405 % of Undergrads Reporting Seeing at Least 1 of the Norming Ads During the Semester, by Semester and Year

15 Are Respondent Reports of Seeing Ads Reliable? PERCENT Source: Spring Celebration Survey, 2005 % of Respondents Who Claimed to Have Seen Each Ad/Poster

2007 National Social Norms Conference16 Percent Students’ Latitude of Acceptance/Rejection of Ad Claims Ave. Number Drinks R’s Believe Typical Student Drank “For each percentage, please indicate if you find the statement to be very believable, somewhat believable, or not believable at all.” "The percentage of MSU students that typically drink 4 or fewer drinks when they party is ___% " Source: Spring Celebration Survey, 2005, n=1,073 Actual % of Respondents who drank 0-4 Last Time

17 Did Change Occur in Normative Perceptions? Average Number of Drinks the Typical Student Believed to Have Consumed the Last Time Partied/Socialized Source: MSU NCHA } A 19% Decrease } A 6.8% Decrease Undergrads Grads

Did Change Occur in Normative Perceptions? 18 Percent % of Respondents Who Believe the Typical Student Drank 0-4 Last Time Partied, by Gender and Student Status Source: MSU NCHA

2007 National Social Norms Conference19 Did Change Occur in Use of Protective Behaviors? % of Undergrads Who Always or Usually Use Various Protective Behaviors, by Year

20 Change in Use of Protective Behaviors Among those who drank 5+ drinks last time partied

2007 National Social Norms Conference21 Did Change Occur in Drinking Behavior? Percent Of All Students Number of Days in Previous Month Drank Alcohol 5.9% decrease from 2000 in percentage of students drinking 3 or more days per month

22 Did Change Occur in Drinking Behavior? Mean Number of Drinks Last Time Partied/Socialized, by Year

23 Is Drinking Related to the Likelihood of Harm? Physical/Psychological –Injury to self as result of alcohol –Injury to other as result of alcohol –Involved in fights –Did something later regretted –Forgot where you were/what did –Had unprotected sex –Had forced sex Academic –Lower grade on exam or project –Lower grade in course –Incomplete or dropped course Types of Harm Asked About in NCHA

24 Is Drinking Related to the Likelihood of Physical/Psychological Harm?

25 % of Students Experiencing Harm by Drinking Levels Odds Heavy vs. Mod Odds Mod vs. Light Odds Heavy vs. Light Heavy = Drink 10 or more days per month and 5+ drinks last time partied Light = Drink 2 or fewer days per month and 2 or fewer drinks last time partied Moderate = Everything else Source: MSU NCHA 2000, 2002, 2004, 2006; n = 4,155

2007 National Social Norms Conference26 Did Change Occur in Amount of Harm? Source: MSU NCHA Among those who drink This school year, as a consequence of your drinking, have you experienced % reduction from 2000 to % reduction from 2000 to % reduction from 2000 to % reduction from 2000 to 2006

27 Change in Academic Harm, by Year

28 Summary Evidence of initial misperception of drinking norm Evidence norming campaign reached targeted audience Evidence perception of norm changed Evidence use of protective behaviors changed Evidence drinking behaviors changed Evidence for reduction of drinking related harm Program is working

29 Is There Reason to Believe This Isn’t Just Coincidence? Dejong et al., “A multisite randomized trial of social norms marketing campaigns to reduce college student drinking” Journal of Alcohol Studies 67: From 2000 to 2003, found experimental group of colleges implementing social norming campaign had no or very small increases in perceptions of drinking norms, actual consumption and harm, while control group of colleges and universities not implementing social norming campaigns had substantial increases in all three. The magnitude of the effect at experimental sites correlated positively with the intensity of the norming campaign. The secular trend in the time period was for drinking and harm to increase. Evidence MSU’s results are not just coincidence because of a secular trend.

Respondents’ Assessments of Social Norming Ads Distributed in Ad% Who Saw Mean # Times Saw % Saw as Believable % Saw as New Information Tailgating ad40.7% %60.5% Designated driver ad36.6% %41.8% Basketball playing ad27.3% %68.7% Pool playing ad (Fall)36.9% %56.0% Moderation ad (Fall)41.2% %58.1% Pool playing ad (Spring)61.8% %54.4% Moderation ad (Spring)52.3% %54.3% Spring Break: snow18.8% %41.1% Spring Break: luggage14.8% %41.0% MSU basketball26.3% %44.2%

Respondents’ Assessments of Social Norming Ads Distributed in Ad/Poster% Who Saw Mean # Times Saw % Saw as Believable % Saw as New Information Sidelines ad (football) 31.8% %50.0% Spartans Think (fall)48.0% %63.3% Spartans Do (fall)50.5% %68.4% Spartans Do (spring)53.8% %61.4% Spartans Think (spring)52.0% %56.7% St. Patrick’s ad (spring)46.4% %55.2% Courtside ad (spring)32.6% %43.4% Courtside ad (Izzone)44.9% %40.3% Halloween ad25.4% %39.9% Halloween Poster29.3% %52.4%

Respondents’ Assessments of Social Norming Ads Distributed in Ad/Poster% Who Saw Mean # Times Saw % Saw as Believable % Saw as New Information Global ad (Spring) 25.5% %29.0% Courtside ad18.0% %37.8% Izzo ad38.5% %51.8% Izzo Poster38.1% %47.o% Spring Break Ad42.6% %52.0% Spring Break Table tent48.2%**73.4%56.9% Global Scrambled (Fall)43.1% %44.3% Spartan Do TT59.6%**60.7%70.8% Halloween TT44.4%**72.3%47.9% Halloween ad48.3% %50.5%

*Ad Believability and Over and Under Estimation of Drinking Norms This study looked at estimation and accuracy of normative perceptions for students during both everyday drinking and celebration drinking.

Believability and Estimation It found that students who drank less than four drinks underestimated the norm, and those who drank more than five drinks overestimated the norm. Ad believability played a crucial role in this process. Those who believed the ad more closely estimated alcohol consumption by their peers.

Effectiveness The reduction of specific primary harms (NCHA and Celebratory Survey data) The increased adoption of protective behaviors across the target population The level of acceptance the campaign has received from students, including levels of message believability and message usefulness

Ethnographic and Formative Research Findings “Celebratory Occasions” U.S. Department of Education Grant

DRINKING AND DRUNKENNESS ON CELEBRATORY OCCASIONS Holiday DRINK  DRUNK Occasion: PREVAL RATE Halloween 32% (57%) St. Patrick’s 26% (58%) Typical Thursday 19% (48%) DRINK PREVAL = Percent of all students that report drinking DRUNK RATE= Proportion of drinkers who report getting drunk

DRINKING AND DRUNKENNESS ON CELEBRATORY OCCASIONS Football DRINK  DRUNK Occasion: PREVAL RATE Big Gameday 38% (56%) Other Games 37% (50%) Typical Saturday 23% (39%) DRINK PREVAL = Percent of all students that report drinking DRUNK RATE = Proportion of drinkers who report getting drunk

DRINKING AND DRUNKENNESS ON CELEBRATORY OCCASIONS Weeklong DRINK  DRUNK Occasion: PREVAL RATE Spring Break 48% (62%) Welcome Week 37% (72%) End of Semester 23% (55%) Typical Week 47% (48%)

Non-Drinking Norm Prevalence: 76% End of Semester 74% St. Patrick’s 68% Halloween 62% Big Game 60% Welcome Week 52% Spring Break

Non-Drunkenness Norm Prevalence: (Moderately or not at all) 86% End of Semester 85% St. Patrick’s 82% Halloween 79% Big Game 71% Welcome Week 70% Spring Break

Types of Drinkers: “Anytime” Drinkers -- 54% Drink on typical days and special days “Celebration” Drinkers -- 35% Drink on special days but not typical days

Types of Drinkers: “Non-Celebration” Drinkers -- 2% Drink on typical days but not special days Seldom Drinkers -- 9% Drinkers who didn’t drink on the special or typical days measured in survey

Table 5. Percentage Distribution of Drinking Types, by Demographic Characteristics n Anytime Celebration Seldom Non-Celebration …… Gender Female Male % 58.9% 35.0% 34.1% 12.5% 4.9% 2.7% 2.1% p<.001 Status Freshman Sophomore Junior Senior % 54.7% 55.0% 56.5% 39.4% 35.8% 33.1% 31.9% 11.7% 6.6% 8.7% 9.6% 1.7% 2.8% 3.3% 1.9% 8.66 N.S. Race White African American Other % 32.8% 45.1% 33.7% 37.7% 40.7% 7.9% 26.2% 6.6% 1.8% 3.3% 7.7% p<.001 Drank in Yes H.S. No % 44.6% 34.1% 36.1% 6.0% 16.4% 2.1% 2.9% 30.6 p<.001

“Focused” Findings Identified Protective Behaviors showed a reduction in harm independent of consumption Normative climate seemed to approve of drinking and higher than normal consumption on celebratory occasions Individuals identified as ‘ celebratory ’ /event specific drinkers!

Protective Behaviors: Promising protective behaviors not necessarily related to consumption: (Celebration ‘02) –Staying with same group of friends- –Remaining in one location- –Consuming only one type of alcohol.

Findings from the Social Norming Study of Student Alcohol Use

Student approval of activities Respondents were asked about their approval of various activities, and also their estimates as to whether ‘most MSU students’ approve the same behavior. The following slides show the results as a function of level of student drinking

Perception of various celebrations: Celebration Survey data

Approval & estimates of celebration drinking

Having more than 5 drinks

Having more than one drink/hour

Getting wasted

Celebration drinking

Results of Note: Injunctive Norms 77% of MSU students approve of drinking more than usual during celebration events. 74% of MSU students approve of getting drunk during welcome week. 73% of MSU students report they approve of getting drunk during Halloween. Celebrations Surveys

Dissemination Strategies for the Model Grant Develop and evaluate a website that will provide interactive instructions for researchers and practitioners wanting to replicate our program Submit content to Wikipedia.org on social norms Create and evaluate a handbook on the development and operationalization of a social norms marketing campaign Provide service as consultants to other institutions of higher education via hosting several teleconferences/webinars Present our model at workshops and conferences

Enhancements Enhancements to the program include: –the development of messages that shift toward subjective norm use distributed through social-interactive community sites (Facebook) –the use of the Audience Response System—(ARS) or "clicker technology" to enhance live presentations