Enhanced Implementation of CDC Guidelines for School Tobacco Programs: Results of a Statewide Evaluation Shelly A. Greller, MS Wisconsin Department of Public Instruction D. Paul Moberg, Ph.D University of Wisconsin – Madison

Acknowledgements Wisconsin Department of Public Instruction –John Hisgen –Douglas White University of WI – Monitoring & Evaluation Program ( –David Ahrens –Ann Christiansen –Amy Anderson –Patrick Remington, P.I. Funded by the Wisconsin Tobacco Control Board

Participants should be able to: Describe WI statewide school tobacco grant program Describe CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction Identify effective program resources and strategies Understand & apply results of WI school tobacco program to enhance implementation of CDC Guidelines Apply assessment & evaluation tool to operationalize CDC Guidelines

Background Master Settlement Agreement led to creation of WI Tobacco Control Board (WTCB) WTCB developed comprehensive strategic plan for tobacco control –One focal area was youth tobacco prevalence –Youth goal: By 2005, tobacco use among middle & high school age youth will decline by 20% School-based programs funded for $1,250,000 by competitive process through DPI

Background DPI stated purpose for school-based tobacco control programs –“Create or expand upon strategies identified as most promising or effective in reducing or eliminating youth tobacco use” Operationally, discussion focused on increased implementation of CDC Guidelines

Monitoring and Evaluation Monitoring and Evaluation Program (MEP) contracted to perform monitoring/evaluation functions for WTCB MEP comprised of –UW – Comprehensive Cancer Center –UW – Extension –Center for Health Policy and Program Evaluation MEP collaborated with DPI on evaluation of school-based tobacco control programs

CDC Guidelines Guidelines for School Health Programs to Prevent Tobacco Use and Addiction (MMWR, 1994) –Summarizes school based strategies “most likely to be effective in preventing tobacco use by youth” –Developed by CDC in collaboration with experts –In-depth review of research, theory & current practice

Impact of Guidelines Rohde et al. (2001 in MMWR) –High or medium implementation of Guidelines in Oregon associated with significantly greater decline in 8 th graders’ 30-day smoking prevalence

Impact of Guidelines Rohrbach et al. (2002 Presentation at Society for Prevention Research-SPR) –2 waves of student data from sample of schools –Indexed Guideline Implementation based on teacher surveys –Most schools did not fully implement comprehensive program of evidence-based strategies –Guideline implementation significantly related to smoking prevalence - quit attempts - negative expectations/attitudes regarding tobacco

Impact of Guidelines Hallfors & Godette (2002, HER) –Dept. of Education has recent statement of “Principles of Effectiveness” for prevention programs –Surveyed 104 School Districts in 12 states –Only 19% of school districts are implementing research-based prevention curricula with “fidelity”

Goal of Our Evaluation Focus on institutional change in implementation of the guidelines in schools with infusion of targeted funding Not on impact of implementation of Guidelines on student outcomes

Methods of Our Evaluation 2001-’ ’03 0$$$$0$$0 00[0?] 0 = Self Assessment Form –School Building $$$$ = Funded District Design Sample: Applicant School Buildings/Districts

Measures – Self Assessment Each building in applicant district completed checklist operationalizing CDC Guidelines –Operationalizes CDC Guidelines with 58 questions –Original purpose was self assessment and planning –Program evaluation secondary Repeated at end of year 1 and year 2 –Funded districts submitted as part of year end report –Unfunded districts mailed as survey at end of year 1

Scaling/Data Reduction Exploratory Principal Components factor analysis indicated most items within areas of the guidelines were correlated We created additive indices for each of the 7 areas Scaling 0 = No 1 = Somewhat 2 = Yes

Scale Validation School Health Education Profile (SHEP) oversampled to include all funded schools Reports from SHEP will be correlated with reports from schools Compared baseline data of funded schools to full SHEP sample –How representative are they?

Scale Construction Scale# ItemsMeanS.D.Alpha School Policies Curriculum Instruction Training Family & community involvement Cessation Services Evaluation Note: Reported for baseline data, n=320. Means in terms of original metric: 0=No, 1=Somewhat, 2=Yes

Sample Size & Response Rate AssessmentNot funded (Comparison) FundedTotal Baseline Follow-Up*61 (40%)155 (86%)216 (65%) *27 additional buildings submitted follow-ups without baselines Buildings represent 98 Districts (42 funded) and 11 Consortia (4 funded)

Mean Change During Year 1 VariableBaselineOne YearDifferenceT-Statistic Policy Funded ** Unfunded * Curriculum Funded ** Unfunded * Instruction Funded ** Unfunded * *P <.05**P <.001

Mean Change During Year 1 VariableBaselineOne YearDifferenceT-Statistic Training Funded ** Unfunded * Family / Community Involvement Funded ** Unfunded * Cessation Funded ** Unfunded * *P <.05**P <.001

Mean Change During Year 1 VariableBaselineOne YearDifferenceT-Statistic Evaluation Funded ** Unfunded * Overall Funded ** Unfunded ** *P <.05**P <.001

Regression Results Regression analysis covarying baseline value found: Baseline covariate always a sig. predictor of one year implementation measure Significant effects of funding on –Training: B =.155 (.084) –Family / community involvement: B =.162 (.072) –Cessation: B =.25 (.09)

Percentage of Funded Schools Reporting Positive Change Any Positive Change +.25 SD Pos. Change # Schools Reporting Policy66.9%55.0%151 Instruction Curriculum Training Parent/community Cessation Evaluation Changed on any

Funded Schools Prohibit Tobacco Use…

Funded School Policies…

Funded Schools Tobacco Instruction…

Rather Than Punitive – Funded Schools…

Funded Schools Teach Developmentally Appropriate

Does Funded School..

Variation by Level – Funded Schools Elementary schools (57 schools) –High on baseline family & community involvement –Lowest on evaluation Middle schools (52 schools) –Highest overall score at baseline, especially in training and curriculum High schools (30 schools) –More change (than elementary & middle schools) on policy, training, parent & community involvement & cessation –Highest initially on cessation Combinations (16 schools) –Highest level of change overall: high change on policy, curriculum, and instruction

Limitations and Issues Tool originally intended as self assessment –Not a research tool –Validity/reliability to be established SHEP link will help – in progress Self Report from school staff (who want funding to continue) without validation Low response rate from comparison group No student outcomes

Conclusions Baseline implementation of Guidelines highest in areas of –Policy –Curriculum Implementation of Guidelines lowest in –Cessation –Evaluation –Training –Family / community involvement

Conclusions Most gain with funding in –Cessation –Evaluation –Training –Family / community involvement Unfunded schools also report gain in all areas-- but significantly less in several areas The tool is promising to assess school tobacco control programs both for planning & evaluation