1. Jean A. King University of Minnesota
Cultivating Participatory Evaluation in a World of Centralized Accountability
Jean A. King
University of Minnesota

2. The Land of 10,000 Lakes
J. A. King

3. The Mississippi Headwaters
J. A. King

4. Birthplace of Judy Garland. . .
J. A. King

5. . . . and of Bob Dylan
J. A. King

6. . . . and Charlie Brown and Snoopy
J. A. King

7. Home to Garrison Keillor’s Prairie Home Companion
J. A. King

8. . . . including Lake Wobegon, where. . .
J. A. King

9. . . . the women are strong,
J. A. King

10. . . . the women are strong, the men are good looking,
J. A. King

11. . . . the women are strong, the men are good looking, and all the children are above average
J. A. King

12. My Background Junior high school English teacher in upstate New York
Teacher educator in New Orleans
Head of a collaborative research center at University of Minnesota
Professor of evaluation studies
J. A. King

13. “Cultivating Participatory Evaluation (PE) in a World of Centralized Accountability”
What is participatory evaluation (PE)?
How does centralized accountability relate to PE?
How can one cultivate PE in such an environment?
What cause is there for hope?
J. A. King

14. J. A. King
My PE principles
Involving people effectively in evaluations is essential.
Participation in evaluations can and should be a learning experience.
Building people’s capacity to think evaluatively matters.
J. A. King
September, 2011

15. What is participatory evaluation (PE)?
Do our definitions matter in practice?
J. A. King

16. Interactive Participation Quotient
LOW
HIGH
Evalu-ator
Program
leaders, staff, community members
Involvement in decision making and implementation
Participant-directed
Collabor-ative
Evaluator-directed
ZONES
J. A. King

17. What I have learned about PE
The farther to the right on the IPQ, the longer the evaluation process will take
The benefit of the time spent comes from people’s learning about evaluation
J. A. King

18. What are the principles of PE?
Participants OWN the evaluation
The evaluator facilitates; participants plan and conduct the study
People learn evaluation logic and skills as part of the process
ALL aspects of the evaluation are understandable and meaningful
Internal self-accountability is valued
(Adapted from Patton, 2008)
J. A. King

19. What are characteristics of PE?
Control of the evaluation process [ranges from evaluator to practitioners]
Stakeholder selection for participation [ranges from primary users to “all legitimate groups”]
Depth of participation [ranges from consultation to deep participation]
(From Cousins & Whitmore, 1998)
J. A. King

20. Cousins & Whitmore’s framework
J. A. King
Cousins & Whitmore’s framework
To the audience:
This is one way to conceptualize and categorize participatory evaluation. You can take any participatory evaluation and plot it on this three-dimensional diagram with the characteristics we just outlined: control; depth of participation; and stakeholder selection.
J. A. King
September, 2011

21. Examples Graduate Review Improvement Process in my department
Evaluation of the Special Education Program in Minnesota’s largest school district
Community Listening Project at Neighborhood House, a social service agency in West St. Paul, MN
J. A. King

22. One unfortunate confusion
Too many labels for different types of PE
J. A. King

23. Competing participatory approaches
Practical participatory evaluation
Transformative participatory evaluation
Collaborative evaluation
Empowerment evaluation
Democratic deliberative evaluation
Inclusive evaluation
Values-driven evaluation
J. A. King

24. Related terms adding to the confusion
Utilization-focused evaluation
All kinds of action research
Responsive evaluation
Evaluation capacity building
Organizational learning
Others?
J. A. King

25. What’s in a name?
Why does the field have so many different names for similar processes?
J. A. King

26. What’s in a name?
Why does the field have so many different names for similar processes?
What difference does this make to the practice of PE?
J. A. King

27. What’s in a name?
Why does the field have so many different names for similar processes?
What difference does this make to the practice of PE?
What are the most important distinctions?
J. A. King

28. The most important distinctions
Who is invited to the table?
Who is responsible for the process(for decision making, for implementation)?
The extent to which the study is framed in broader societal terms
J. A. King

29. How does centralized accountability relate to PE?
Is it helpful to have accountability demands?
J. A. King

30. On the one hand. . .
Accountability mandates typically come from outside organizations
They can exert incredible pressure on front-line staff (feelings of helplessness)
They may not reflect the reality of meaningful requirements for achieving better or different outcomes
They often rely on a quantitative epistemology (“Prove that your program delivers these results”)
J. A. King

31. On the other hand Programs should be held accountable
Programs should engage in ongoing evaluation to improve their practice
PE engages people in collecting data and reflecting on their practice
“The best defense is a good offense”- PE can help programs both understand and defend their practice
J. A. King

32. Addressing the claim that PE is “biased”
What is bias? Systematic distortion or an inclination that prevents “unprejudiced consideration”
Why can’t PE processes minimize this?
Solution: documenting potential concerns and reflecting on them publicly
Still have the benefit of participation (possible trade-off)
J. A. King

33. How can one cultivate PE in an environment of accountability?
Standing one’s ground makes a difference
J. A. King

34. Three possibilities Shoe string interactive evaluation practice (IEP)
Evaluation capacity building (ECB)
Strategic instruction of policy makers
J. A. King

35. Three possibilities
“Shoe string” interactive evaluation practice (IEP)
J. A. King

36. An evaluator’s rule of thumb:
Good
Cheap
Quick
Choose two.
J. A. King

37. Interactive evaluation practice
“. . . The intentional act of engaging people in making decisions, taking action, and reflecting while conducting an evaluation study”
J. A. King

38. Shoe string techniques for engaging people/collecting data
Cooperative (three-step) interviews
Data dialogues
Carousel and concept formation tasks
J. A. King

39. Three possibilities Shoe string interactive evaluation practice (IEP)
Evaluation capacity building (ECB)
J. A. King

40. What is evaluation capacity building (ECB)?
Using an evaluation for
(1) its results
AND
(2) the explicit purpose of building people’s capacity to evaluate again
J. A. King

41. Evaluation capacity building
“Intentional work to constantly co-create and co-sustain an overall process that makes quality evaluation and its uses routine in organizations and systems”
Stockdill, Baizerman, & Compton (2002)
J. A. King

42. For the ECB practitioner, the focus
“. . .[is] on responding to requests for evaluation services while simultaneously considering how today’s work will contribute to sustaining the unit in the longer term”
(Compton, Glover-Kudon, Smith, & Avery, 2002, p. 55)
J. A. King

43. Where did ECB come from?
Free range evaluation
J. A. King

44. Evaluation Capacity Building
Goals of ECB
Increase an organization’s capacity to design, implement, and manage effective evaluation projects
Access, build, and use evaluative knowledge and skills
Create support for program evaluation as a performance improvement strategy
Cultivate a spirit of continuous organizational learning, improvement, and accountability
J. A. King
Jean A. King, 2011 MESI

45. Why now? Rising accountability demands The cost of evaluation
The availability of technology
The value (and fun) of the ECB process
J. A. King

46. Evaluation capacity-building (ECB) continuum
Formative / Summative
evaluation study
Evaluation specifically for building capacity to evaluate
Evaluation for
organization development
 
Use of single study
process / results
ECB = creating capacity to conduct evaluations
Capacity to sustain change
J. A. King

47. Three possibilities Shoe string interactive evaluation practice (IEP)
Evaluation capacity building (ECB)
Strategic instruction of policy makers
J. A. King

48. The PE “curriculum”
Proof of causation in the social sciences is a measurement challenge
PE is a valid and viable form of collaborative inquiry
Research documents that involvement leads to increased use
PE leads to added benefits as organizations build their capacity to evaluate over time
J. A. King

49. Three possibilities Shoe string interactive evaluation practice (IEP)
Evaluation capacity building (ECB)
Strategic instruction of policy makers
J. A. King

50. What cause is there for hope?
While there’s life, there’s hope. . .
J. A. King

51. To travel hopefully is a better thing than to arrive.
Thank you!
To travel hopefully is a better thing than to arrive.
-Robert Louis Stevenson
J. A. King

52. Jean A. King (kingx004@umn.edu)
Thank you!
Jean A. King