1. NWHF Community-Based Participatory Research Conference Portland, OR Friday, July 20, 2007 This project was funded by the Oregon DHS Public Health Division Evaluation Planning with Tobacco Prevention & Education Networks: A Community-Based Participatory Approach www.npcresearch.com

2. NPC Research2 July 2007 Panel Members R.E. Szego, OR LGBTQ Coalition Against Tobacco Jennifer Kue, Asian & Pacific Islander TPEP Coordinator Luci Longoria, Disparities Liaison for the Oregon Tobacco Prevention & Education Program Carrie Furrer & Scott Burrus, NPC Research

3. NPC Research3 July 2007 NPC Project Team Beth Green, Ph.D., Principal Investigator Carrie Furrer, Ph.D., Project Director Scott Burrus, Ph.D., Project Coordinator Jelani Greenidge, BA Research Assistant

4. NPC Research4 July 2007 Acknowledgements TPEN Program Coordinators Evaluation Teams at each of the 5 TPENs Oregon DHS Public Health Division

5. NPC Research5 July 2007 Overview of Oregon TPEP Tobacco Prevention & Education Program Comprehensive, statewide public health effort aimed at reducing the toll of tobacco-related death and disease TPEP-funded programs work to: –Create smoke-free and tobacco-free environments through public policy –Counter pro-tobacco influences –Promote quitting

6. NPC Research6 July 2007 What is a TPEN? Tobacco Prevention & Education Network Focus on groups facing tobacco disparities: –African American –Asian and Pacific Islander –Hispanic/Latino –Urban American Indian –Lesbian/Gay/Bisexual/Transgender/Questioning (LGBTQ) Develop and implement culturally-reflective best practice strategies

7. NPC Research7 July 2007 Purpose of Project Assist TPENs in the development of community-driven evaluation plans and tools Build capacity for evaluation planning and eventual evaluation activities

8. NPC Research8 July 2007 What is CBPR? Community-Based Participatory Research Engages program stakeholders Shared learning and power Evaluators as “coaches” Values community expertise, traditions, priorities

9. NPC Research9 July 2007 CBPR Evaluation Planning Community “owns” the evaluation process –Articulates priorities –Crafts questions –Involvement in tool development –Defines success Culturally mindful planning process Culturally appropriate products

10. NPC Research10 July 2007 CBPR Framework Step 1: Information Gathering Step 2: Information Synthesis Step 3: Information Sharing & Feedback Step 4: Develop Goals Step 5: Complete Evaluation Plan Step 6: Review Progress

11. NPC Research11 July 2007 Evaluation Planning Activities Create community buy-in for evaluation Identify community needs Identify TPEN practices & whether they address community needs Develop logic model for TPEN practices Craft evaluation questions & data sources for each step of the logic model Design tools to answer evaluation questions

12. NPC Research12 July 2007 Steps 1 & 2: Information Gathering and Synthesis Interview DHS key stakeholders Interview each TPEN Program Coordinator Review each TPEN’s work plan Review CDC Best Practices & related literature Review other state Tobacco Control Plans

13. NPC Research13 July 2007 Steps 1 & 2: TPEN Program Coordinator Questions What was your initial reaction to the idea of evaluation planning? Did you have any concerns? Was there anything about evaluation that you were particularly interested in?

14. NPC Research14 July 2007 Steps 1 & 2: DHS Project Officer Questions Why was CBPR important to this process? What did you hope that TPEN Program Coordinators would get out of the evaluation planning process?

15. NPC Research15 July 2007 Step 3: Information Sharing & Feedback Pre-planning meeting –Statement of purpose –Logistics –Review preliminary logic model –Recruitment

16. NPC Research16 July 2007 Step 3: Information Sharing & Feedback Initial Evaluation Team Meeting –Form collaborative partnership –Introduce basic evaluation principles –Establish structure to guide collaboration –Identify tobacco-related community needs –Select core TPEN activity –Develop logic model for selected activity

17. NPC Research17 July 2007 Step 3: TPEN Program Coordinator Questions How prepared were you for what was covered in your initial Evaluation Team Meeting? What was the most important thing that you learned from your initial Evaluation Team Meeting? What do you wish you would have learned but did not?

18. NPC Research18 July 2007 Step 4: Develop Goals Initial & Second Evaluation Team Meetings –Thinking through short-term and long-term goals –Identifying what is important for the community to know about the work –Linking evaluation questions to logic model –Authenticating different ways of knowing

19. NPC Research19 July 2007 Step 4: TPEN Program Coordinator Questions What challenges did you face going into your second Evaluation Team Meeting? What was the most important thing that you learned from your second Evaluation Team Meeting? What do you wish you would have learned but did not?

20. NPC Research20 July 2007 Step 5: Complete Evaluation Plan Second & Third Evaluation Team Meetings –Identify key evaluation questions –Revise preliminary logic model –Identify data sources, measures and data collection methods –Revise current data collection plans (if appropriate) –Develop evaluation tools

21. NPC Research21 July 2007 Step 5: TPEN Program Coordinator Questions Do you believe that the tools developed (logic models, evaluation questions, tracking forms, assessments, etc.) reflect the needs, values, traditions, and priorities of your community? Do you now have the capacity to: –revise your current evaluation plan –execute evaluation activities, and –develop an evaluation plan for other activities?

22. NPC Research22 July 2007 Step 5: DHS Project Officer Questions What was the most beneficial technical assistance that you provided during this process? What other kinds of technical assistance would have been useful? What were the top 2 accomplishments of the evaluation planning process? What do you wish would have been accomplished but was not?

23. NPC Research23 July 2007 Step 6: Review Progress Iterative process Opportunities for feedback Review at each community meeting One-on-one meetings with Program Coordinator Readiness for change

24. NPC Research24 July 2007 Step 6: TPEN Program Coordinator Questions Do you think that you were provided enough opportunities to provide feedback? What kind of technical assistance was most helpful to you? What else in the way of technical assistance would you have liked to receive? How responsive was the process to your/your community’s needs?

25. NPC Research25 July 2007 Accomplishments Program advocacy Build evaluation capacity Design culturally-appropriate tools Program documentation Define community-specific processes Develop a community knowledge base

26. NPC Research26 July 2007 Lessons Learned TPEN readiness for evaluation planning Community readiness for evaluation Community stakeholder recruitment Versatile evaluation planning team Flexible evaluation planning process Need for clear next steps

27. NPC Research27 July 2007 More Information Carrie Furrer, NPC Research furrer@npcresearch.com Full report online at: www.npcresearch.com Oregon DHS Public Health Division: www.oregon.gov/DHS/ph/tobacco/index.shtml R.E. Szego, OR LGBTQ Coalition Against Tobacco reszego@cascadiabhc.org Jennifer Kue, API TPEN Coordinator jenniferk@mail.irco.org