E MPOWERMENT E VALUATION December 9, 2011 EPS 654: Program Evaluation Katherine Coder

“Empowerment evaluation helps transform the potential energy of a community into kinetic energy.” ⌘ Fetterman & Wandersman, 2007, p. 182

Wandersman et al. (2005) define empowerment evaluation as “an evaluation approach that aims to increase the probability of achieving program success by (1) providing program stakeholders with tools for assessing the planning, implementation, and self-evaluation of their program, and (2) mainstreaming evaluation as part of the planning and management of the program/organization” (p. 28).

Key Concepts (Fetterman, 2007) collect evidence have a “critical friend” develop a culture of evidence establish cycles of reflection and action cultivate a community of learners contribute to the development of reflective practitioners

10 Key Principles (Fetterman & Wandersman, 2007) improvement community ownership inclusion democratic participation social justice (removing inequities) community based knowledge evidence-based strategies capacity building organizational learning accountability

EE: 3-Step Approach (Fetterman & Wandersman, 2007) 1. Develop the Mission: determine mission statement and group values through a democratic process whereby meaning can be made and voice is given fairly 2. Taking Stock: list the current activities and use a cooperative process to determine the organizational priorities. First take a baseline account of how the organization is doing on priority issues. After the organization has implemented interventions, another iterative process of Taking Stock is completing to measure success. 3. Plan for the Future: establish the goals, strategies, and identify evidence/indicators. This step represents the intervention phase of the process. After the intervention, return to the next round of Taking Stock to evaluate the success of the program. The EE process is geared toward institutionalization where iterative rounds of Taking Stock and Planning for the Future occur thus developing a learning culture.

10-Step Getting to Outcomes (GTO) (Fetterman & Wandersman, 2007) This approach asks 10 questions and helps those using this system find the relevant literature, methods, and tools. 1. What are the needs and resources in your organization, school, community, or state? (needs assessment: resource assessment) 2. What are the goals, target population, and desired outcomes (objectives) for your school/community/state? (goal setting) 3. How does the intervention incorporate knowledge of science and best practices in this area? (science and best practices) 4. How does the intervention fit with other programs already being offered? (collaboration; cultural competence) 5. What capacities do you need to put this intervention into place with quality? (capacity building) 6. How will this intervention be carried out? (planning) 7. How will the quality of implementation be assessed? (process evaluation) 8. How well did the intervention work? (outcome and impact evaluation) 9. How will continuous quality improvement strategies be incorporated? (total quality management; continuous quality improvement) 10. If the intervention is (or components are) successful, how will the intervention be sustained? (sustainability and institutionalization)

EE Tools (Fetterman, 2007; Fettermen & Wandersman, 2007) EE encourages the use of technologies that align with the principles of EE including online surveys digital photos blogs picture sharing collaborative web sites You Tube videoconferencing spreadsheets

Critiques of Empowerment Evaluation (Donaldson, Patton, Fetterman, & Scriven, 2010; Fetterman & Wandersman, 2007) A number of critiques of EE have been voiced including the ideas of conceptual ambiguity, methodological specificity, and outcomes empowering others (creates an empowering setting) advocacy (evaluators are not necessarily advocates/no neutral eval) consumers (focuses too little on participants) compatibility (internal and external) and trad’l and EE practical or transformative forms EE as evaluation (not a movement) bias (self-serving) social agenda (yes) ideology (not a methodology) differences between collaborative, participatory, and EE

EE as a discipline is said (as of 2005) to be growing in 4 key areas: Defining the field Clarifying EE concepts and principles Methodological specificity (3-step & 10-step, ++) Documenting outcomes As of 2007, Fetterman and Wandersman see EE as strengthening in 4 areas: Combining quantitative & qualitative data Capturing the critical “ah-hah” moments more systematically Translating EE into policy language Learning how to build more refined EE tools and systems

EE Example: Bridging the Digital Divide/Tribal Digital Village HP awarded $15 million to 3 digital villages in the US one of which was a village consisting of 18 Native American tribes.

Tribal Digital Village Outcomes: Creation of the largest unlicensed wireless network in the country (as part of their own sovereign nation); Efforts were recognized and lauded by the head of the FCC Training their young people how to maintain the network (building capacity) Secured an E-rate grant, providing $1 million/yr from the telephone companies toward building and maintaining the network Creation of a high-end digital printing press—a small business enterprise representing a contribution to economic sustainability Implementation of a parent involvement and education center Video recording workshops have enabled the recording of native history through personal family stories

References Donaldson, S. I., Patton, M. Q., Fetterman, D. M., & Scriven, M. (2010). The 2009 Claremont debates: The promise and pitfalls of utilization-focused and empowerment evaluation. Journal of Multidisciplinary Evaluation, 6 (13), Fetterman, D. M. (2002). Empowerment evaluation: Building communities of practice and a culture of learning. American Journal of Community Psychology, 30 (1), Fetterman, D. M. (2005). Empowerment and ethnographic evaluation: Hewlett-Packard’s $15 million digital divide project (a case example). NAPA Bulletin, 24, Fetterman, D. M. (2007, July). Empowerment Evaluation. Australasian Evaluation Society. [PowerPoint Presentation]. Retrieved from homepage.mac.com/ profdavidf/ documents/Canberra.pdf Fetterman, D., & Wandersman, A (2007). Empowerment evaluation: Yesterday, today, and tomorrow. American Journal of Evaluation, 28 (2),

T HANK Y OU ! Questions???