1. Future Trends in Evaluation
Live Webinar
1st July 2010
DevInfo 1

2. This series of webinars are based on the book published by UNICEF in partnership with key international institutions
Authors:
40 global evaluation leaders
Partnership:
UNICEF, WB, UNDP, WFP, UNIFEM, IDEAS, IOCE, DevInfo

3. Available for free download at www.mymande.org

4. 2010 Monthly webinars: 4 Saraswathi Menon Finbar O’Brien
The Role of the United Nations in Fostering National Ownership and Capacities in Evaluation
22nd June 2010
Michael Quinn Patton
Marco Segone
Future trends in evaluation.
Moving from policies to results by developing national capacities for country-led monitoring and evaluation systems
1st July 2010
Caroline Heider
Craig Russon
Evaluating policies and their results
The role of policy analysis in over-coming the implementation challenge
September 2010
Belen Sanz
Human Rights and gender in evaluation
October 2010
Jean Quesnel
Linda G. Morra Imas
The professionalization of
evaluation
Professionalizing development
evaluators. Travelling the road to
results
November 2010
Oscar A. Garcia
Angela Bester
Joint evaluation of the role and contribution of the UN system in South Africa. Lessons learned
December 2010 4

5. 2011 Monthly webinars: Michael Bamberger
Institutionalizing impact evaluation. A key element in strengthening country-led monitoring and evaluation systems
January 2011
Hallie Preskill
Alexey Kuzmin
Exploring effective strategies for facilitating evaluation capacity development
Use of evaluation training in evaluation capacity building
February 2011
Michael Quinn Patton
Utilization-focused evaluations
March 2011
David Fetterman
Rita O’Sullivan (TBC)
Empowerment evaluation
Collaborative evaluation. Creating environments to engage stakeholders in evaluation
April 2011
Indran Naidoo
The monitoring and evaluation in South Africa. Many purposes, multiple system
May 2011
Manuel Fernando Castro
Diego Dorado (TBC)
Building a results-based management and evaluation system in Colombia
June 2011
Velayuthan Sivagnanasothy
National monitoring and evaluation system in Sri Lanka. Experiences, good practices, challenges and the way forward
July 2011
David Rider Smith
Policies, institutions and personalities. Lessons from Uganda’s experience in monitoring and evaluation
September 2011

6. The recording of the previous webinars are available at www. mymande
The recording of the previous webinars are available at 1 2

7. The recording of the previous webinars are available at www. mymande
The recording of the previous webinars are available at

8. The recording of the previous webinars are available at www. mymande
The recording of the previous webinars are available at

9. Agenda
9h30 – 9h35 Welcome and introduction 9h35 – 9h50 Michael Quinn Patton, Founder and Director, Utilization- focused evaluation, and former President of the American Evaluation Association 9h50 – 10h05 Marco Segone, Systemic management, UNICEF Evaluation Office 10h05 – 10h25 Questions and Answers Moderator: Abigail Taylor, Knowledge Management Specialist, UNICEF Evaluation Office 10h25 – 10h30 Wrap-up

10. Questions and Answers 1 3 2 10

11. Keynote speakers
QUINN PATTON, Michael former President of the American Evaluation Association and author of six evaluation books including Utilization-Focused Evaluation; Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use; and Qualitative Research and Evaluation Methods. 11

12. Trends in EVALUATION UNICEF Webinar Michael Quinn Patton
Michael Q. Patton, UNICEF, July 1, 2010

13. Evolving Understandings
I keep changing what I said. Any person who is intellectually alive changes his ideas. If anyone at a university is teaching the same thing they were teaching five years ago, either the field is dead, or they haven’t been thinking.
Noam Chomsky
“The Professor Provaocateur,” The New York Times Magazine, Nov. 2, 2003: 13.
Michael Q. Patton, UNICEF, July 1, 2010

14. TRENDS 1. Globalization of the Profession IOCE IDEAS
Challenge: Non-professionals doing evaluation
Diversity of evaluation
Michael Q. Patton, UNICEF, July 1, 2010

15. Evaluation Standards and Culture
Utility
Feasibility
Propriety
Accuracy
NDE: International Perspectives on
Evaluation Standards, No. 104, 2004.
Michael Q. Patton, UNICEF, July 1, 2010

16. 2. Evaluation as a Transdiscipline Evaluative Thinking IDRC
1. Globalization of the Profession
2. Evaluation as a Transdiscipline
Evaluative Thinking
IDRC
Michael Q. Patton, UNICEF, July 1, 2010

17. 1. Globalization of the Profession
2. Evaluation as a Transdiscipline
3. Increased political interest in accountability, performance indicators, and transparency
Michael Q. Patton, UNICEF, July 1, 2010

19. SPEAKING TRUTH TO POWER

21. This is a preview. The total pages displayed will be limited. L
Loading...
Loading...
About this book
Preview this book
Shake Hands with the Devil
 By Roméo Dallaire
This is a preview. The total pages displayed will be limited. L

22. Trends Essential skills – beyond just methods Opportunities: IPDET
4. Growing emphasis on evaluation capacity-building and skill development.
Essential skills – beyond just methods
Opportunities: IPDET
The Evaluators’ Institute
Professional associations
Michael Q. Patton, UNICEF, July 1, 2010

23. Trends
5. Continuing debate about what constitutes methodological rigour
Michael Q. Patton, UNICEF, July 1, 2010

24. GOLD STANDARD DEBATE: METHODOLOGICAL APPROPRIATENESS not Methodological orthodoxy or rigidity

25. Mixed Methods
Matching methods to evaluation questions, purposes, and resources; Contingency-based evaluation
Michael Q. Patton, UNICEF, July 1, 2010

26. Trends 6. Systems thinking and complexity concepts
as frameworks for evaluation
Michael Q. Patton, UNICEF, July 1, 2010

27. “Using Systems Concepts in Evaluation” edited by
AEA monograph:
“Using Systems Concepts
in Evaluation”
edited by
Bob Williams and Iraq Iman
Michael Q. Patton, UNICEF, July 1, 2010

28. Seeing Through A Complexity Lens
Michael Q. Patton, UNICEF, July 1, 2010

29. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use
Michael Quinn Patton
2010, Guilford Press

30. Complexity Concepts Emergence Openness Dynamic High uncertainty
Nonlinear: Small actions, large impacts
Co-evolution
Adaptation
Unanticipated consequences
Michael Q. Patton, UNICEF, July 1, 2010

31. Unanticipated developments as an expected aspect of innovation and
a focus and challenge for evaluation.
Michael Q. Patton, UNICEF, July 1, 2010

32. Beyond just formative and summative to Developmental Evaluation
Michael Q. Patton, UNICEF, July 1, 2010

33. What is being developed?
Development
not just
Improvement.
Core questions:
What is developing?
What is being developed?
Michael Q. Patton, UNICEF, July 1, 2010

34. Challenge
Matching the evaluation process and design to the nature of the situation:
Contingency-based
Evaluation
Michael Q. Patton, UNICEF, July 1, 2010

35. Contingency-based Evaluation
Situational analysis & responsiveness
Context sensitivity
Clarify and focus on intended users: stakeholder analysis
Clarify and focus on intended uses
Methodological appropriateness
Criteria for evaluating the evaluation: credibility, meaningfulness, utilty
Michael Q. Patton, UNICEF, July 1, 2010

36. 1. Globalization of the profession
Summary: Six Trends
1. Globalization of the profession
2. Evaluation as a transdiscipline.
3. Increased political importance of M & E
4. Growing emphasis on capacity-building and essential skills
5. Debate about methods: Gold standard
debate
6. Using systems thinking and
complexity concepts
Michael Q. Patton, UNICEF, July 1, 2010

38. January 2009
References
Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use.
Michael Quinn Patton, 2010, Guilford Press.
Utilization-Focused Evaluation, 4th ed.,
Michael Quinn Patton, Sage Publications,
Michael Quinn Patton
UNICEF WebinarJuly 1, 2010

39. References
“Future Trends in Evaluation.” Pp in From Policies to Results: Developing capacities for country M & E systems, Marco Segone (ed.) UNICEF, 2010.
Michael Q. Patton, UNICEF, July 1, 2010

40. Keynote speakers
SEGONE, Marco is responsible for the decentralized evaluation portfolio at the UNICEF Evaluation Office, serves as a Senior Advisor to the IOCE Board and was IOCE Vice-President. 40

41. National evaluation capacity development
Key elements for a conceptual framework
Marco Segone, Systemic Strengthening,
UNICEF Evaluation Office

42. WHY

43. In line with new international development paradigm…
National ownership and capacity development: the key ingredients to development
Managing for results
Mutual accounta
bility
Harmoni
zation
Alignment
Ownership
Paris Declaration on Aid Effectiveness

44. … the UN General Assembly …
national Governments have the primary responsibility for coordinating external assistance and evaluating its contribution to national priorities
mandated the United Nations system to promote national ownership and capacity development, and to make system-wide progress in harmonizing evaluation practices
UNICEF

45. … and UN Agencies Boards request.
2009 UNICEF ExBoard:
emphasizes that programme countries should have greater ownership and leadership in the evaluation of all forms of assistance
underlines the importance of increasing the participation of national counterparts and strengthening national capacity in evaluation
encourages UNICEF to use national evaluation systems, where available, to continue to incorporate capacity-building mechanisms into programme design and implementation and ensure that evaluations are responsive to national demand
UNICEF

46. HOW

47. A Systemic approach to Capacity Development.
Individual Level
(skills, knowledge, experience)
UNICEF

48. Individual Level Demand side
Capacity to strategically plan evaluations, and to identify the key evaluation questions
Capacity to manage evaluation for independence and credibility
Capacity to use evaluation
Supply side (capacity to conduct an evaluation):
Behavioural independence
Independence of mind & integrity
Knowledge and respect of evaluation standards
Agreed evaluation processes & products
Professional competences
Formal education (Masters)
Specialized training
Professional Conferences and meeting
On the job training (country-led evaluations)
Community of Practices and networking

49. A Systemic approach to Capacity Development.
Institutional Level
(policies, procedures, frameworks)
Individual Level
(skills, knowledge, experience)
UNICEF

50. Institutional Level Evaluation culture
Set of values and attitudes supporting evaluative (critical) thinking within an organization
Institutional commitment to learning from evaluation, support evidence-based policy debate and demand for accountability.
Individual more self-directed learners and use information to act; take higher risks but also develop a greater sense of personal accountability and responsibility; consult, coach, and support each other more.
Protective culture (Remove repercussions on careers)
Understanding of the foundations and principles of Monitoring and Evaluation
Institutionalizing independence, credibility and utility
Evaluation policies
Work programme and budget
Independence & adequacy of budget
Conduct of evaluations
Institutional endorsement of standards
In-built Quality Assurance systems

51. A Systemic approach to Capacity Development.
Institutional Level
(policies, procedures, frameworks)
Enabling Environment
(policies, legislation, power relations, social norms)
Individual Level
(skills, knowledge, experience)
UNICEF

52. Enabling Environment
Public administration committed to manage for results and accountability
Transparency
Results-based public budgeting
Evidence-based policy making
Strong civil society
rights holders able to demand for and monitor quality of public services
Strong national evaluation association
foster indigenous demand for M&E, and strengthen indigenous supply

53. We need a mind shift to do things differently...
We should always aim at strengthening National ownership and leadership, and not undermining it:
Selecting topics of mutual interest
Implement evaluations jointly with Governments, as a first step towards country-led evaluation
Hiring local experts to the maximum extent possible
Not assuming there are weak evaluation capacities, even if none is immediately apparent
Co-ordinating with other UN agencies and key international stakeholders
Invest in the country-led process, even if it may require additional efforts and less control

54. Questions and Answers 1 3 2 54

55. The discussion goes on at www.mymande.org1 2

56. The discussion goes on at www.mymande.org

57. The discussion goes on at www.mymande.org

58. Next webinar, September, same time
Caroline Heider Director of the Office of Evaluation of the World Food Programme (WFP) and Vice Chair of the UN Evaluation Group
Craig Russon Senior Evaluation Officer, International Labour Organization (ILO) 58