What Influences the Influence of an Evaluation? Environmental Evaluators Network Helen Davis Picher, William Penn Foundation Andrew Johnson, William Penn Foundation Peter Szabo, Bloomingdale Management Advisors June 24, 2011

Overview Hypothesis – Evaluation breakdown Case – Possible way to address Discussion – Your views

Hypothesis Strategy ProgramEvaluation The Action Cycle

Hypothesis Strategy ProgramEvaluation The Action Cycle

Hypothesis Sponsor (Eval Staff) Client (Program Staff) Consultant Subject (Grantees) Key Actors and Dynamics

Hypothesis SponsorClient ConsultantSubject Key Actors and Dynamics

Case – William Penn Foundation Created in 1945 Private, family foundation Managed by a president and staff of 30 $80 million annual grant budget Focus on greater Philadelphia 2001 strategic plan – new goals, three program areas

Case - Before PA NJ

Case - Environment & Communities Program Four Funding Priorities: 1. Protect and Conserve Significant Regional Landscapes 2. Protect, Conserve, and Restore Water Resources 3. Improve Regional Prosperity and Competitiveness 4. Revitalize Greater Philadelphia's Urban Core

Case - Before PA NJ

Case - Before PA NJ

Case – Land Protection Work : 22 grants to 14 organizations (over $7.5 million) Types of Support: Capital acquisition funds Technical assistance Advocacy and coalition building Innovative approaches to land protection Cluster Review: Assess land preservation work Understand best practices in the field Get recommendations on how to move forward

Case – Evaluation and Follow-On Major findings Capital too diffused for long-term impact Administration – cumbersome Other funders: Forest v. trees

Case – Evaluation and Follow-On Major findings Capital too diffused for long-term impact Administration – cumbersome Other funders: Forest v. trees Key responses Narrowed geographic focus Engaged intermediary for capital

Case - Before PA NJ

Case - After PA NJ

Case - After PA NJ

Case – Why Evaluation Was Influential SponsorClient ConsultantSubject Key Actors and Dynamics

Case – Why Evaluation Was Influential Culture Clarity Communication

SponsorClient ConsultantSubject Key Actors and Dynamics Case – Why Evaluation Was Influential Culture Strategic plan Evaluation system Collaboration Senior mgmt interest

SponsorClient ConsultantSubject Key Actors and Dynamics Case – Why Evaluation Was Influential Clarity Clear charge to consultant Clear message to grantees

SponsorClient ConsultantSubject Key Actors and Dynamics Case – Why Evaluation Was Influential Communication Regular, formal, informal In-person, on-site, respectful

Discussion Task Begin with introductions, then Using your own experience as a basis, assess the thinking we have presented on: a)The issues that inhibit the influence of an evaluation b)The actions/conditions that may mitigate these issues How would you build upon these ideas?