1. Cultural Competence in Tobacco Control Evaluation: Lessons from the Cultural Competence in Tobacco Control Evaluation: Lessons from the Field Jeanette Treiber, Ph.D, UC Davis, Tobacco Control Evaluation Center, jtreiber@ucdavis.edu 5. Conducting the Workshop PROCESS 1. Establishing the Need 2. Investigating Available Resources 3. Planning the Resource Development Process 4. Including Expertise 6. Developing Resources 7. Disseminating Resources 1.Establishing the Need California Department of Public Health funds tobacco control projects that are required to conduct program evaluations. The California Tobacco Control Evaluation Center (TCEC) provides training and assistance in developing evaluation plans, tools, and analyses for more than 100 California Tobacco Control projects. Because of California’s great population diversity, many of these projects specifically target Priority Populations (African Americans, American Indians, Asian and Pacific Islanders, Hispanic/Latinos, and other groups that are particularly targeted by the tobacco industry and have a high smoking prevalence). The need for strategies and tools to work with these populations became more apparent in 2007 as a result of a needs assessment of priority population grantees. The grantees expressed a need for evaluation resources to work with their priority populations. We found that our own center lacked sufficient cultural competence to meet these needs. Consequently, we devised a plan to build our own capacity and that of our clients. 2. Investigating Available Resources While searching for existing resources and tools on culturally competent evaluation with our target populations, we found some relevant literature in the social sciences, anthropology, and health sciences. Some useful titles are: Dixon, Mim and Pamela E. Iron. Strategies for Cultural Competency in Indian Healthcare. Washington DC: American Public Health Association, 2006. Harkness, Janet A., Fons J.R. Van de Vijever, and Peter Ph. Mohler. Cross-cultural Survey methods. Hoboken: Wiley-Interscience, 2003. Hood, Stafford, Rodney Hopson and Henry Frierson (eds.). The Role of Culture and Cultural Context in Evaluation: A Mandate for Inclusion. The Discovery of Truth and Understanding (Evaluation & Society). Charlotte: Information Age Publishing, 2005. Hopson, Rodney (Ed.). How and Why Language Matters in Evaluation. New Directions for Evaluation, Number 86. San Francisco: Jossey Bass, Summer 2000. Madison, Anna-Marie (Ed.). Minority Issues in Program Evaluation in New Directions for Program Evaluation, Number 53. San Francisco: Jossey Bass, Spring 1992. Marin, Gerardo and Barbara Van Oss Marin. Research with Hispanic Populations. Applied Social Research Methods Series, Volume 23. Newbury Park: Sage Publications, 1991. Weaver, Hilary N. Explorations in Cultural Competence. Belmont: Thomson, 2005. 3. Planning the Resource Development Process TCEC designed a five-step plan for increasing the culturally competent evaluation capacity of our center and our grantees: a. Hosting roundtable discussions at a workshop for Priority Population grantees to gather needs and experiences of working with priority populations. b. Drafting Tips and Tools sheets with strategy recommendations for evaluations with various priority populations based on initial discussions. c. Enhancing the Tips and Tools Sheets with best practices information garnered from literature reviews. d. Disseminating resources through postings on our website and through trainings, conferences, and workshops. e. Establishing channels for continuous feedback and revisions. 4. Including Expertise In a review of the literature on culturally competent evaluation we identified Professor Rodney Hopson of Duquesne University as a leading figure in the field. : Rodney K. Hopson is Hillman Distinguished Professor, Department of Educational Foundationsand Leadership, Duquesne University. Hopson is the co-editor of a recent book on educational language policies in postcolonial Africa: Languages of Instruction for African Emancipation: Focus on Postcolonial Context and Considerations (with Birgit Brock-Utne, Mkuki na Nyota/The Centre for Advanced Studies of African Society: Dar es Salaam/Cape Town, 2005) and one on culturally responsive evaluation, The Role of Culture and Cultural Context in Evaluation: A Mandate for Inclusion, the Discovery of Truth and Understanding (with Stafford Hood & Henry Frierson, Information Age: Greenwich, CT, 2005). An additional collaborative writing project editing the 3rd edition of the Program Evaluation Standards of the Joint Committee on Standards for Educational Evaluation is underway. Our Center invited Dr. Hopson to help us develop our plan and deliver a workshop to grantees. For several months Rodney and the TCEC team communicated via teleconference to draw out each others’ thoughts and expectations for the workshop. 5. Conducting the Workshop On August 23, 2007, TCEC held a workshop in Sacramento, CA, entitled “Culturally Competent Evaluation in Tobacco Control Programs.” All Tobacco Control Section grantees serving priority populations were invited to attend. Dr. Rodney Hopson presented an introduction to culturally competent evaluation, and then led the group through a discussion of a case study. Later, small groups discussed different challenges and solutions to culturally competent evaluation with the following groups: a. Low Socio-economic status b. Lesbian, Gay, Bisexual and Transgender c. African American d. Asian and Pacific Islander e. Native American f. Rural Fifty-four people attended the day-long workshop. This included project directors and/or evaluators of 12 Local Lead Agencies, 3 Competitive Grantees and 19 Priority Population Grantees. In addition, 12 staff from the California Department of Public Health, Tobacco Control Section, and 8 staff from the Tobacco Control Evaluation Center (TCEC) also attended. In a satisfaction survey workshop participants rated the interactive exercise in which participants volunteered to work on specific priority group challenges and solutions as the most useful. The workshop was an important step in developing Tips and Tools Sheets for evaluation with priority populations that will be made available on the TCEC website. 6. Developing Resources TCEC staff enhanced the results from the workshop with findings from literature reviews on the various priority populations and began creating Tips and Tools Sheets. A tool sheet for each of the identified priority groups has been developed or is currently under development. The tools focus on access to the community and recommendations for developing data collection instruments. 7. Disseminating Resources We see the Tips and Tools Sheets on evaluation work with priority Populations as living documents. They will be made available on the TCEC website: www.tobaccoeval.ucdavis.eduwww.tobaccoeval.ucdavis.edu. Any user of the TCEC website may send in suggestions for revisions and additions, and we plan periodic updates of the documents. In addition, we will disseminate the evaluation strategies through workshops, teleconferences, and other training events. In the future, our center will conduct site visits and collect “stories from the field” to help demonstrate challenges and solutions in evaluation work with priority populations. www.tobaccoeval.ucdavis.edu Content by Jeanette Treiber and the TCEC Evaluation Team Original art by Gina Mitsdarfer. Poster design and layout by Maria Tuccori ABOUT US The mission of the California Tobacco Control Evaluation Center (TC Evaluation Center) is to empower tobacco control projects funded by the California Department of Health Services, Tobacco Control Section (TCS) to understand and solve challenges associated with evaluating their programs. The TC Evaluation Center works toward this mission by providing individual technical assistance, training, and other evaluation-related resources, such as the Tips & Tools, and Lessons From the Field series and the repository of data collection instruments. The TC Evaluation Center is based at the University of California, Davis and staffed by evaluation associates with experience and expertise in evaluation. If you are a director, evaluator, or other staff of one of TCS-funded projects, please feel free to contact one of our evaluation associates for more information about our services.