1. Developmental Evaluation as Harvesting
Practitioner’s Gathering: Harvesting, Evaluation and Research
March

2. Action—Reflection Tension
“We do not learn from experience…we learn from reflecting on experience.” ~John Dewey

3. Traditional Evaluation
Suited for: Linear, simple and complicated
Formative: Tweaking an established program or model
Summative: Evaluating if something failed or succeeded (test, prove, and validate)
Measurement: Measures performance and success against predetermined goals (e.g. logic model)
Goal: Improve proposals, improve reporting, proving effectiveness, generalizable findings

4. Development Evaluation (DE)
Created for: Complexity, innovation, changing goals, changing context, probing, prototyping (vs. machine worldview, linear, predictable)
Measurement: Develop new measures and monitoring mechanisms (or change them) as goals emerge and evolve. Rapid and real-time.
Goal: deepen reflective culture of data-driven decisions, produce context-specific insights

6. DE can provide evidence and justification for course change
Source: Mintzberg, Quinn-Patton

7. Diverge/Converge

8. DE Can Harvest For… Intangibles
e.g., language change, slight change in practice, referenced a document
Contribution vs. attribution
e.g., I did X, which contributed to this larger change alongside others and their actions vs. I did X, which caused Y
Policy-influence
e.g., report written, report discussed, report cited, practice influenced, policy changed, make case for emergent, participatory methods approach

9. DE as Harvesting
Both about reflection to inform wise-action and next steps
DE can enhance the depth and rigor around data collection
DE research methods capture data outside of live events (i.e. working with availability and bandwidth of folks)
Data can inform and deepen sense-making at participatory gatherings

10. When to Use DE… and when to Not… it’s up for debate!
If you want to make good use of an evaluation required for a funder
If you need to make the case for participatory methods within a conventional framework
If you can fund a robust harvesting strategy through developmental evaluation
In other cases, draw on or be inspired by DE…add rigor to harvesting strategies

11. Methods for Harvesting Data
Deep Dialogue Interviews
Surveys
Small, participatory focus groups
Photovoice
World Café with robust harvest
Sensemaker
Many more...

12. Tactic: 1 hour reflection session
Easy DE Tactics
Tactic: 1 hour reflection session
Take stock, identify tangible and more intangible changes
What are the key activities we’ve done?
What impacts/changes/outcomes have I/we experienced?
Share together and discuss
motivation, understanding,  rationale

13. Tactic: Collective Sense-Making Session
Easy DE Tactics
Tactic: Collective Sense-Making Session
Provide collected data in easily digestible ways (infographic, one-pager, document for review)
Guiding Questions for Conversation:
What does this data tell you?
What surprises you about this data?
What factors may explain some of the trends we’re seeing?
Do these findings lead to any new questions?
What key insights can inform us moving forward?

14. Tactic: Partner/relationship tracking
Easy DE Tactics
Tactic: Partner/relationship tracking
Excel sheet listing partner, meeting, content, next steps, barriers, strengths, comments
Institutional memory to pass on
Look back after a year and understand how things shifted, what worked and what didn’t
Improve tactics
motivation, understanding,  rationale

15. Tactic: Add evaluative/reflective thinking in small ways
Easy DE Tactics
Tactic: Add evaluative/reflective thinking in small ways
Weekly team meetings, or quarterly reflection retreats
What’s working? What’s not working? What needs to change?
What’s up? So what? Now what?
Goal: identify, understand, pivot, impact

16. Reflecting on Data-Driven Action
Questions
What do I/we need to know in order to make good decisions or have greater impact?
Where do we and can I/we easily collect data? What obstacles, if any, do we/I experience collecting data?

17. Resources
Gamble, A. A. J. (2008) A Developmental evaluation primer. The J.W. McConnell Family Foundation.
Patton, M. Q. (2011) Development evaluation: Applying complexity concepts to enhance innovation and use. Guilford Press.
Patton, M. Q. (2014) Evaluation flash cards: embedding evaluative thinking in organizational culture. Otto Bremer Foundation.
Cobb, M., & Donnelly, G. (2015) Community-based, participatory and developmental evaluation approaches: an introductory toolkit. Ecology Action Centre. Find at: bit.ly/2nsxcRM

18. Contact
Gabrielle Donnelly, Ph.D. Miranda Cobb Research and Evaluation Coordinator Ecology Action Centre