1. Evaluating Community Change
It is difficult work!

2. Community Change Initiatives
Arose as an ambitious strategy in the 1990s to address the needs of residents of poor communities
Adopted a comprehensive approach to neighborhood change and worked according to community building principles that value resident engagement and community capacity building
Community Change Initiatives

3. “Most community change initiatives and community change efforts have not produced the degree of community transformation envisioned by their designers.

4. “One area of progress in evaluating community change is in “more realistic expectations for measuring impact,” with a growing understanding that establishing causality is difficult.”
Anne Kubisch Director of the Roundtable on Community Change The Aspen Institute

5. The Harlem Children’s Zone brought third-grade test scores up to the New York state average
Population of West Philadelphia enjoys safety, decent housing, a good school, and commercial space
16 neighborhoods in Chicago have high-quality comprehensive plans
Miner Country, S.D., has put into place renewable energy facilities that are revitalizing the local economy
Success Stories

6. A CASE STUDY: The Neighborhood Improvement Initiative
From 1996 to 2006, The William and Flora Hewlett Foundation gave $20 million to initiative designed to improve lives of residents in three San Francisco Bay area communities.
Helped some people, but its impact did not reflect the investments of time and the $17.5 million — which leveraged more than $27.4 million in public and private resources — that were expended.

7. Hewlett did not develop a detailed theory of change that specified all the connections between each of NII’s six goals, the strategies to achieve the goals, and the assumptions underlying the connections.

8. "It reminds us what happens when a foundation and its partners fail to achieve clarity about goals, strategies, and how to measure progress. …

9.  “When this happens, we should seize the opportunity to understand the causes in order to improve our own performance and benefit others working in the field.”
Paul Brest, President The William and Flora Hewlett Foundation

10. Evaluations not intended to justify the existence of a project, i. e
Evaluations not intended to justify the existence of a project, i.e., to prove that it works ✕

11. Evaluations serve as a source of important information to promote project development and growth

12. Useful CCI Evaluations:
Inform decisions
Clarify options
Identify strengths and weaknesses
Provide information on program improvements, policies, and key contextual factors affecting the program

13. COMMUNITY CHANGE INITIATIVES (CCIs)
5 RULES FOR EVALUATING
COMMUNITY CHANGE INITIATIVES (CCIs)

14. Evaluations are not experiments but part of the community change process
Controlling all the factors at play in a community is impossible - Too many contextual variables
Must include constant assessment, adaptation of strategy, and accommodation of interventions to new and changing conditions
Rigorous measurement tools and methods implemented throughout the life the program
Not an afterthought to satisfy donor

15. Evaluations need a strong focus on the processes of community change
Focusing solely on establishing causal relationships will not contribute meaningful and useful information
Document the factors that influence change, not just what the change results in on the other side
Causality
relationship between an event (the cause) and a second event (the effect), where the second event is understood as a consequence of the first

16. Evaluations need to measure ongoing progress towards achieving outcomes and results
Most successful when they have been able to define explicit results and outcomes that multiple agents and community constituencies can share
Measure program or single agency-level outcomes and neighborhood-wide or population-level outcomes
Not simply the regular measurement of performance but an instrument of accountability and feedback capable of motivating, informing, and reinforcing positive change

17. Need to understand, document, and explain the multiple theories of change at work over time
Explicitly define upfront the theories of achieving broad community scale and reaching an improved and sustainable state of transformation
Single theory insufficient. Use multiple theories from a wide range of disciplines

18. Prioritize real-time learning and the community’s capacity to understand and use data from evaluations
Building self-evaluation, learning, and accountability capacity and infrastructure among and inside organizations, systems, and the community is a necessary task of the evaluation
Community and evaluators need to build relationships and capacity to work together to make sure that effective community transformation is informed and reinforced by strong and relevant data collection, analysis, and reporting on process and outcomes

19. Deciding what to measure, how to measure it, how to collect the data, and how to interpret the results is a challenging process that involves many trade-offs of rigor, practicality, and resources.