1. Including Grantee Voices: The Good, the Bad and the Ugly Laura C. Leviton, Ph.D. Senior Adviser for Evaluation March 7, 2014

2. Guiding Principles for Evaluators D. Respect for People #5: Where feasible… foster social equity in evaluation E. Responsibilities for the General and Public Welfare #1: When planning and reporting evaluations, include relevant perspectives and interests of the full range of stakeholders #3: Actively disseminate information to stakeholders as resources allow #4: Maintain a balance between client needs and other needs. #5: Go beyond analysis of particular stakeholder interests and consider the welfare of society as a whole www.eval.org/

3. The Ugly… An Example Power differential + Expediency = Unfair evaluation

4. Other Stakeholders May Deserve a Voice Available at RWJF.org / Publications / Evaluation tools

5. Does the Grantee Call the Shots? Depends on the purpose of the evaluation: –Accountability –Learning –Program Improvement Capacity of the grantee

6. Planning for Input Who speaks for the grantee? Especially community grantees! Don’t expect all sweetness and light… This is evaluation, after all!

7. An Ugly Duckling Anger is to be expected, When communities are neglected. But that’s not the end of the story…

8. Ten Years Later: A long-standing community collaborative and a helpful evaluation

9. “Infrastructure of Trust” http://depts.washington.edu/ccph/commbas.html#Principles “Community-based participatory research is a "collaborative approach to research that equitably involves all partners in the research process and recognizes the unique strengths that each brings. CBPR begins with a research topic of importance to the community, has the aim of combining knowledge with action and achieving social change to improve health outcomes and eliminate health disparities." WK Kellogg Foundation Community Health Scholars Program

10. “Infrastructure of Trust” RWJF national program Linking community interventions to asthma care Evaluation engaged these collaboratives Quality of collaborative was associated with fewer asthma episodes

11. But Can It Be Done at Scale? Yes. Example: Salud America! The RWJF Research Network to Prevent Latino Childhood Obesity Developed priorities for study Delphi process of 318 community leaders, researchers and health groups Network is now > 2,000 people.

12. Summary The Good:Knowing how to engage stakeholders The Bad: Philanthropy just does not do it much The Ugly: Not caring.

13. Getting to Good: Expect messy process Stick with it Build infrastructure of trust Balance quality and engagement