Youth Tobacco Evaluation Project Virginia Commonwealth University

The YTEP Team Elizabeth Fries, PhD, Principal Investigator Ilene Speizer, PhD Diane Baer Wilson, EdD Brian Smith, PhD Donna McClish, PhD Samy Uguy, PhD Patricia Obando, PhD Karen Mitchell Melanie Bean Joyce Phillips Panumas Assavarakpreecha

Youth Tobacco Evaluation Project Virginia Commonwealth University Regional Training Part 1: Progress & Report on Data from Round 1

Youth Tobacco Evaluation Project: Our Mission Funded by VTSF, our mission has been to design and implement a comprehensive evaluation of state-wide tobacco prevention programming Determine impact of VTSF funding on youth participants Compare various compendium programs by setting and delivery Our mission from the beginning Specifically, we are responsible for Evaluating overall funding of VTSF programs, that is, determining the utility of this statewide funding. Have the youth participating in the programs been affected. Evaluate which program are more or less successful in varying settings at affecting youth knowledge, attitudes, skills, and behaviors. When possible, we will compare the same programs in different settings Examining the impacts of programs by program fidelity.

Progress to Date For the first round of data collection Designed outcome and process evaluations Developed all necessary tools for evaluation Designed, organized, and trained all grantees funded by VTSF to implement the evaluation Worked with grantees to ensure that all necessary tools for the evaluation are provided Receiving completed tools Data are logged, entered, and analyzed First round of data collection (year 1 of evaluation tools) Second round of data collection, beginning July 1, 2003

What We Have Received From You… Instructor surveys: 500 Logs (session and program): 634 Parent surveys: 596 Parent & Family Participant Survey: 76 Brief Participant Survey (4th-5th): 2981 Outcome surveys (6th-12th) Baseline: 14787 Follow-up: 9232

Presentation Today … Presentation of process information Instructor surveys Presentation of outcome information Baseline surveys Comments from participants

Instructor Surveys Obtains information from instructors on: Fidelity Ease of delivery Perceptions of curriculum by instructor Post-implementation survey completed by participating instructors Using 287 surveys entered and cleaned Presentation focuses on compendium programs that have at least 14 instructor surveys returned

Instructor Surveys by Compendium Program (n=287)

Locations where Programs are Delivered (Instructor Surveys, N=287) Added scriptural texts to each lesson as appropriate (faith-based organization)

Delivery Setting by Compendium Program Program is made for single-grade, classroom setting rather than a multi-grade after-school setting (Here’s Looking at You)

Fidelity of Program Delivery – Number of Sessions

Fidelity of Program Delivery – Minutes per Session I deleted some of the movement activities, not enough space in the classroom (Positive Action)

Target Grade of Participating Youth (reported by instructor)

Program was Age-Appropriate

Program was Well Received I loved it!! And I think the kids really enjoyed it too (TATU)

Youth Understood Program

Participants Were Enthusiastic

Youth were Attentive Stories are too long. Some words and stories students can’t relate with. Some are not entertaining and don’t keep students’ attention (Positive Action)

Adequacy of Materials LST is not “active” enough. Writing in a workbook is not “active” for a 6th grader

A Lot of Teaching Effort was Required

Presented as Prescribed by Curriculum

Program was Effective It’s (the program) is exactly what my students need at this age. I think that by building up their self-concept, they make better life choices. (Positive Action)

Instructor Would Recommend Program

Preliminary Data from Baseline Surveys Based on 14,557 baseline surveys entered and cleaned Caveats: Most data from 6-8th graders – emphasis of presentation Bigger programs contribute more information Not representative of VA, Regions, or sites Data presented here are based on the first 12,800 surveys that were entered and cleaned. We currently have another 2000 pre-test surveys being entered and cleaned. No post-test data are being presented yet as only a small proportion of programs have sent this in thus far.

Thoughts from Participating Youth 40% of youth gave a comment on the survey The majority of the comments were positive or neutral (55%) A third (35%) were negative Some (9%) were both positive and negative Collected thoughts from sample of participants, includes all thoughts from selected folders (all regions, all grades)

Thoughts from Participating Youth It was Kool I felt good about it I felt this survey might help to figure out a quick way to not have people smoke It was very interesting and it asks a lot of questions. But it prevents you from smoking I felt that I needed to do this where you get answers to help kids quit smoking I think it was very good to get opinions from the student It helped me get some stuff out I really felt confident taking it

I think it is a very good program to help kids to stop smoking and there should be other programs like it in school It’s an okay idea I think it is a good way to help people prevent smoking I think this was a nice survey it makes you think a lot I feel the questions are very appropriate for all ages I think it was alright. I mean you all need to know how we feel about drugs I think it was a little too much information but it could help certain things It will show how people do stuff so we can improve I don’t understand what hair/eye color have to do with smoking but I feel good about the fact that things are being done

I thought the survey was boring, some of the questions do not apply to kids my age It was personal and I didn’t like it It was hard and a lot of questions I felt it was a little embarrassing answering the questions Didn’t like it!! Dumb, no interests to my knowledge. You don’t need to know our eye color It’s too long It was long and boring Is too long This was stupid, if kids want to smoke or dip let them I thought the smoking questions were crazy too much It was stupid and too personal for an anonymous survey. I hate this survey it was very dumb. It took up too much of my time and it asked the same questions I answer every single time. It wasted too many trees. Trees are cool.

I thought it was way too long and too many personal questions about our parents and ourselves You repeat the same things too many times I didn’t feel very good because I just want my family to know and not people I don’t know It was kind of personal to me I like it in a way but I do not like private part about it I think you shouldn’t ask dumb questions like this most people I know don’t do drugs and I stay away from the ones that do Was okay (but boring) It was alright but it was long. It seemed repetitive I feel that a company that sells a legal product shouldn’t have to pay money to trash its own product It was ok. I don’t smoke so I really didn’t care

Baseline Data from the YTEP Evaluation: Ethnicity (N=10,950, 6th – 8th grades only)

Baseline Data from the YTEP Evaluation: Gender (N=10,950, 6th – 8th grades only) I felt at times it was a little confusing, but it really made me think. I don’t think I’ve ever worked harder

Participation in Tobacco Prevention Programs in Last Year

Number of Grantees Implementing Compendium Programs (not complete list)

Smoking Experience (all grades, N’s small for 10th - 12th grades)

Snuff Use by Grade (male youth only, all grades, N’s small for 10th - 12th grades)

Try Smoking in Next 6 Months (N=10,950, 6th – 8th grades only)

I Could Avoid Smoking (N=10,950, 6th – 8th grades only)

Cigarettes are Addictive (N=10,950, 6th – 8th grades only) “I felt very good because knowing that sometimes smoking was on my mind that I knew [what] the consequences of smoking was”

Report Cigarettes Easy to Get (N=10,950, 6th – 8th grades only) … its my body and business what I do. If tobacco is out there I can get it if I want to no matter what age

People Smoke Because Family Members Smoke (N=10,950, 6th – 8th grades only)

People Smoke Because Friends Smoke (N=10,950, 6th – 8th grades only)

If I Stay Tobacco Free, I Will be Healthier (N=10,950, 6th – 8th grades only)

Thoughts about Your Data Provided baseline data to programs participating with 20 or more youth in baseline Representative of your program, assuming reasonable participation The data are not meant to be representative of youth in the region, community, or school surveyed

What Will We Do With Data Entering follow-up survey Match baseline and follow-up samples Provide back to programs – tables on follow-up (with some impact vars.) Compare responses from baseline and follow-up Link instructor information on program delivery to youth responses Determine impact of programs on outcomes Examine process results for K-5th grade programs . . .