Evaluation of North Carolina TRU Media Campaign Sponsor: The North Carolina Health & Wellness Trust Fund (HWTF) Media Vendor: CapStrat Evaluators: University of North Carolina School of Medicine Tobacco Prevention & Evaluation Program (UNC-TPEP) Presenters: Kelly Kandra, Shelley Summerlin-Long

Presentation Overview  Background  Media Campaign  Evaluation Methodology  Focus Group Results  Telephone Survey Results  Conclusions

Youth and Tobacco Use in NC  25% of NC population under 18  2 million youth (1.4 million in school)  750,000 ages (target of TRU campaign)  Tobacco use:  14.3% middle school students  33.7% high school students Sources: 2003 NC Census, 2003 NC Youth Tobacco Survey

NC’s Teen Tobacco Prevention Program Health and Wellness Trust Fund  Created 2001  Receives 25% of NC’s MSA funds  Current annual budget of $15 million makes NC 21 st in country in funding for tobacco prevention

NC’s Teen Tobacco Prevention Program Teen Tobacco Use Prevention and Cessation Initiative  68 grants: 47 community; 7 statewide – minority youth; 14 TFS mini-grants  Statewide media campaign – funded at ~ $1.5 million in 2004  $2 per year old in NC

Campaign Background  Brand: Tobacco.Reality.Unfiltered. or TRU  Radio campaign: launched 2003  Decision to switch to TV based on early evaluation

Campaign Background  UNC-TPEP produced report on best practices for youth anti-tobacco media campaigns to guide formation of TV ads.*  Literature review  Expert Interviews  Stakeholder interviews * The report is available at:

Campaign Background  Made recommendations for TRU ads based on 4 point model:  Content/Themes  Serious health consequences, SHS, Industry, Addiction  Format  Testimonials, graphic images  Emotional Tone  Negative emotional tone – sadness, anger, fear, shock  People in Ads  Multicultural, real people

Campaign Background  TRU campaign launched on TV in April 2004  Ads ran April-early Nov  Ads played more frequently in Charlotte media market to test effect of different levels of exposure to the campaign

TRU Ads Anna Teen talks about grandmother smoking since age 13, getting mouth cancer Jacobi Young man gives facts about smoking; says smoking killing his 34-year-old uncle Brad Teen gives facts about smoking; says friend smokes and cannot run across the street Travelogue “TRU Road Trip” with stories from a woman with tracheotomy, addicted teen who has smoked since age 8

Evaluation: Methodology  Adolescent focus groups  Longitudinal telephone survey

Focus Group Goals  to understand how NC youth seeing/interpreting TRU ads  to gain insight on how ads might be improved The full focus group report is available at:

Focus Groups  14 groups conducted statewide after campaign, Nov/Dec 2004  Groups of varying age, gender, ethnicity, and smoking status  ~ 140 youth ages  Tested TRU and non-TRU ads

Focus Group Results  Youth response to most frequently shown ads (Anna, Jacobi) mixed  Youth felt these ads could have been more effective if:  they had grabbed attention earlier;  people were more emotional when telling their stories  they had featured the actual people suffering health consequences  they had shown graphic images of the health consequences

Focus Group Results The last ad produced for the campaign, Travelogue, was chosen as the best/most effective ad (of TRU and non-TRU ads shown) by the majority of youth in 13 of 14 groups.

Focus Group Results  The reasons youth gave for choosing Travelogue included three major elements emphasized by groups as effective or memorable.  graphic display of serious health consequences of smoking, which elicited strong reactions/emotions;  the featured individuals telling stories about their own experiences, rather than those of a friend or family member; and  a diversity (ethnic, gender, and age) of people.

Telephone Survey  Time 1: 637 NC youth (ages 11 to 17); Time 2: 604 NC youth  Longitudinal design (will continue to follow)  Spring and Fall of 2004  Assessed campaign awareness, comprehension, and reaction  Short-term outcomes of smoking behaviors and intentions, tobacco-related knowledge and attitudes  Not expected that behaviors or attitudes would change in just 6 months

Sample Design  Dual frame approach  Random digit dialing (probability of selection  )  Age-targeted (probability of selection  )  Specified 10:90 split for final sample  Stratified into two domain StratumRDDTargetedTotal Charlotte Remainder of State Total Used

Final Response Rates Time 1  2624 calls placed into one of four outcomes  Completed or partial interviews by eligible respondents (n = 637)  Non-response or refusal to participate by eligible respondents (n = 378)  Ineligibility (n = 1176)  Unknown eligibility (n = 433)  48 RDD completed calls  589 targeted completed calls

Unweighted Response Rates  Response rate = number of completed interviews / number of eligible households in sample  Response Rate 4 (RR4)  American Association for Public Opinion Research Standard Definitions (2000)

Unweighted Response Rate  Response Rate 4 (RR4)  Must determine e, the estimated proportion of cases of unknown eligibility that may be eligible

Weighted Response Rate  Computed by multiplying unweighted counts for frames by adjusted weights

Telephone Survey  Campaign designed to measure a “dose effect”  Minimum dose needed to show effect not well established  Data weighted according to NC demographics

Procedures for Weighting Data  Raw weight calculation  Trimming of excessive or large weights  Post-stratification adjustment

Telephone Survey Results Campaign Awareness  45% of youth reported seeing at least one ad  Charlotte youth reported higher campaign awareness (52.9%) than rest of state (41.5%)  Younger youth reported greater awareness (47.4%) than older youth (41%).  Non-white youth reported greater awareness (54.8%) than white youth (40.5%).*  No overall awareness differences between males and females. * Statistically significant

Individual Ad Awareness The Anna ad was the most commonly seen of all the TRU ads with 29.9% of the youth reporting confirmed awareness of that ad in comparison to 19.1% for Jacobi, 5.2% for Brad, and 5.5% for Travelogue.

Ad Reactions - Anna

Ad Reactions - Jacobi

Ad Reactions - Brad

Ad Reactions - Travelogue

Brand Recognition % of youth that recognized brand/logo *The TRU brand was not tested in Time 1.

Attitudes Toward Smoking % of youth who Strongly Agree/Agree with statement

Attitudes Toward Smoking % of youth who Strongly Agree/Agree with statement

Susceptibility to Smoking  Susceptibility to smoking is a measure of “likelihood to smoke” based on youth’s responses to several hypothetical questions:  Do you think you will smoke a cigarette in the next year?  Do you think that you will ever smoke a cigarette in the future?  If one of your best friends offered you a cigarette, would you smoke it?

Susceptibility to Smoking  Initial results show that youth who were non-susceptible, non-smokers at Time 1 and aware of the TRU campaign were less likely to have experimented with cigarettes at Time 2 than their non-TRU aware counterparts.  Other data on smoking behavior are inconclusive, as expected at such an early time in the campaign.

Limitations  Had to design and begin pre-test before ads created  Validity concerns with self-report data  Higher-level analyses still in progress

I. Conclusions: Campaign Design A. Continuation of an evidence-based mass media campaign to reduce youth consumption of tobacco products must be a high priority in NC. B. The media campaign should maintain a continuous (year-round) presence on television and in community activities with NC youth.

II. Conclusions: Ad Design A.Ads should include the following features where best practices overlap with focus group suggestions:  Serious health consequences with graphic images, personal story  Negative emotional tone – strong response  Smokers/former smokers telling own stories  Diversity of people  Content of ad must appear real, genuine

III. Conclusions: Campaign Awareness A.The 2004 TRU/Tobacco.Reality.Unfiltered. campaign reached approximately 4.5 out of every 10 youth. B. The campaign reached diverse youth (males and females, whites and non-whites, younger and older youth). C. Awareness of campaign slogan and brand increased substantially over the last two years. The brand Tobacco. Reality. Unfiltered. is now recognized by an estimated 55% of NC youth.

UNC Tobacco Prevention & Evaluation Program CB 7595, Manning Drive, UNC School of Medicine Department of Family Medicine Chapel Hill, NC T: ; F: Web: