1. Why Indicators Matter Using Outcome Indicators to Plan and Evaluate Comprehensive Tobacco Control Programs Todd Rogers, PhD California Tobacco Control Program, CX Webinar July 9, 2009

2. Plan for this Presentation Demonstrate the importance of indicators for: – Planning and evaluating tobacco control programs – Monitoring the national tobacco control effort – Advocating for policy support and additional resources

3. Specific, observable, and measurable characteristics that shows progress a program is making toward achieving a specified outcome What is an indicator? Measuring Program Outcomes: A Practical Approach. Alexandria, VA: United Way of America; 1996

4. Characteristics of a “Good” Indicator Relevant – important/useful information Affordable – requires reasonable resources Valid – Appropriate – meaningful for intended use – Understandable – makes sense Reliable – minimal measurement error

5. Inputs Activities Outputs Outcomes Context Simplified Program Logic Model

6. Focus on Program Outcomes Outcomes Completed efforts to disseminate anti-tobacco/pro-health messages Completed efforts to reduce and counteract pro-tobacco messages Inputs Activities Completed efforts to increase tobacco curricula and tobacco free policies Outputs Reduced initiation among youth Reduced youth smoking prevalence School- based prevention Policy and Regulatory Action 2 1 13 14 Targeted to Disparate Populations 3 Decreased tobacco- related disparity Reduced tobacco- related morbidity and mortality 16 15 Completed efforts to increase restriction and enforcement of tobacco sales to minors Completed efforts to increase cigarette excise tax 5 4 Community mobilization Counter marketing Long-term State Health Department and Partners Reduced susceptibility to experimentation Decreased access Increased price on cigarettes through tax Increased anti-tobacco policies and programs in schools 8 9 10 12 11 Increased in anti- tobacco attitudes, support for policies to reduce youth initiation 7 6 IntermediateShort-term Reduced tobacco industry influences Increased restriction, enforcement on restrictions on tobacco sales to minors

7. Outcome Component 9 Reduced tobacco industry influences Indicator 1.9.3 Extent of tobacco advertising outside stores

8. Indicators Drive Program Planning Develop program outcome objectives in the context of logical model “pathways” and associated outcome indicators Integrate program and evaluation planning

10. Indicators Shape Program Objectives From: Local Program Evaluation Planning Guide (December 2004) California Department of Health Services, Tobacco Control Section

11. Local Evaluation of a “Pathway” Does passage of SHS policies lead to: – Increased enforcement of policies? – Increased compliance with policies? – Reductions in reported exposure to SHS? Weber MD, Bagwell DAS, Fielding JE, Glantz SA. Long term compliance with California’s Smoke-Free Workplace Law among bars and restaurants in Los Angeles County. Tobacco Control 2003;12:269-273.

12. Although CX was an Early Innovation in State Tobacco Control Planning and Evaluation …

13. … There are Other National Efforts to Track Key Outcome Indicators for Tobacco Control …

14. … and Indicators are Being Developed for Many Other Public Health Issues

15. Indicators Are Widely Used to Track State and National Progress in Tobacco Control

16. CDC OSH Indicator Utilization by Funded State Tobacco Control Programs (2007-08) InitiationSHSCessationTotal AvailUsedAvailUsedAvailUsedAvailUsed Short3825191824238166 Inter121155442120 Long44886618 Total544032313433120104 TM

17. CDC OSH Indicators Not Being Utilized by Funded State Programs (2007-08) IndicatorType 1.6.6 Level of awareness among parents about the importance of discussing tobacco use with their children Init ST 1.8.3 Proportion of jurisdictions with policies that control the location, number and density of retail outlets Init ST 1.9.4 Proportion of jurisdictions with policies that regulate the extent of tobacco advertising outside of stores Init ST 1.9.8 Extent of tobacco advertising in print mediaInit ST 1.9.10 Number and type of MSA violations by tobacco companiesInit ST 1.9.12 Amount of tobacco industry campaign contributions to local and state politicians Init ST 1.11.5 Proportion of young people reporting that they purchased cigarettes from a vending machine Init IT TMTM

18. Of 120 KOIs named by CDC, CTCP lacks data on only 22 indicators (18%): – Youth initiation – 6/54 missing (11%) – Secondhand smoke – 4/32 missing (13%) – Cessation – 12/34 missing (35%), especially on health care provider and insurance-related indicators CTCP Evaluation Indicators Overlap Substantially with CDC OSH Indicators

19. State Indicator Tracking Motivates Policy and Media Advocacy Actions

21. Indicator Data Presented by CTFK in State-Specific Press Releases

22. Summary Indicators help your planning and evaluation efforts Indicators are used to monitor state-specific and national progress Indicator data are used to advocate for poicy change and program support Indicators matter