Evaluating the effectiveness of social media in promoting audience engagement Jennifer Berktold Senior Study Director Westat National Conference on Health Communication, Marketing, and Media, Atlanta, Georgia, August 11, 2009 The Boss of Me case study

I dont want no drama in my life, even though we have a little bit, but no more letting people control you. Thats drama, because then you become something that youre not. -- Mary J. Blige

INTIMATE PARTNER VIOLENCE Teens dont achieve true Intimacy Partners are constantly being redefined making the word less relevant Reducing violence is our ultimate goal but strategy should be prevention EARLY RELATIONSHIP CONFLICTS Formative years should be emphasized Relationship is a more open term reflective of shifting expectations Conflicts can be a frequent precursor to violence Reframing the Issue

Five key strategies from research for reaching teens (ages 14 – 18, both genders, all races and ethnicities) on this issue: 2) Target the drama with humor and authentic teen language 3) Promote self-empowerment 4) Offer tools and resources to dial down the drama 5) Leverage teens organic media culture 1) Take an empathic and non-judgmental stance The Strategic Approach

Being the boss of you means youre in complete control of you at all times in your relationships. It means you refuse to submit to your own feelings of anger, fear, jealousy or insecurity. When you allow your emotions to negatively influence your actions, you lose control. And when you lose control, youre not the boss of anything. BOM (Boss of Me) is dedicated to helping young people stay in charge of their own relationship drama. BOM provides tools to help teens maintain and manage their cool, instead of losing it, during a relationship conflict. BOM or Boss of Me

Organizational Partners Funder Managing the Campaign Brand Management Evaluation

Evaluating BOM

Research goals Identify delivery mechanisms that engage young people with BOM Understand how young people engage with BOM –Gender differences Understand how the campaign impacts its target audiences

Intended uses of BOM evaluation results Learning, best practices Communicate with the teen dating violence prevention community –Effective delivery mechanisms to reach teens –Engaging messages / content –Appeal to new audiences?

Basic model of an outcome evaluation Products from activities Web traffic Number of radio shows developed Number of ads Number of events Reach Inputs Outcomes Activities Outputs Resources Money Staff and volunteer time Facilities What is done Website Radio show Ads / PSAs Community events Changes for participants Increased knowledge Attitudinal change Behavior change

Outputs, Outtakes, Outcomes Outtakes Awareness Buy-in Engagement Participation Outcomes Knowledge change Attitudinal change Behavior change Outputs Products Activities Reach

New Logic Model, Simplified Products from activities Web traffic Number of videos developed Number of ads/ ad buy Number of events Reach Inputs Outcomes Activities Outputs Resources Money Staff and volunteer time Facilities What is done Tools development Website development Ads / PSAs Community events Contests Changes for participants Increased knowledge Attitudinal change Behavior change Outtakes Participation Creative content Conversation content Tone of conversation Friending Texting opt-in

Approach

Heard of BOM but hasnt engaged Radio ads / PSAs Outreach at school Our challenge… Engaged with BOM BOM411 site visits Social media friends Text messaging Community events User-generated content Hasnt heard of BOM

5-Point Evaluation Plan Monitoring Content analysis Web survey of teens Pop-up survey at BOM411 Visits to community events

Web Monitoring BOM Sites BOM411 Facebook MySpace (2 sites) Twitter YouTube Outside BOM RSS (e.g., Google blogs, Google news) Data Google analytics Comment text Comment analysis Articles/posts with BOM mentions

17

Content analysis Subjects Posted content on BOM411 Posted content on social networking sites User-generated content –Skateboard designs –Rap lyrics Outside buzz Measures (some examples) Type of conversation Affect Attributes Type of information shared

Other Monitoring Subjects Radio Cell phones Voic line Measures (some examples) GRPs Number of subscribers Number of texts Number of phone messages

Web survey of teens Sample Teens in SF Bay Area and Los Angeles Recruited from panel that agreed to take surveys online Parental consent obtained Knowledge Networks Measures Self-reported exposure to campaign Awareness of relationship drama among peers Attitudes toward / relationship with BOM Self-reported attitude/behavior change

Pop-up survey at BOM411 Sample Random selection of visitors to BOM411 Measures How found out about website Engagement with website content (e.g., rating different parts of website, would you recommend to others)

Visits to community events Sample Random intercept interviews Interviewer observation Measures How found out about event Why attending the event Activities performed at the event Description of event Estimated attendance

Timeline TaskDate Site LaunchAugust 24 MonitoringLaunch thru February 2010 Content AnalysisLaunch thru February 2010 Other monitoringLaunch thru February 2010 Pop-up surveySeptember 2009, January 2010 Community eventsOctober 2009 Web survey of teensOctober /November 2009

Questions

Thank you! Jennifer Berktold Senior Study Director Westat