1. Use of Training in Evaluation Capacity Building
Evaluation’2010 “Quality and Evaluation”
American Evaluation Association
San Antonio, Texas
Use of Training in Evaluation Capacity Building
Alexey Kuzmin, Process Consulting Company
Moscow, Russia

2. A.Kuzmin, Use of training in ECB
Important
Not identical
Not enough
A.Kuzmin, Use of training in ECB

3. A.Kuzmin, Use of training in ECB
What factors affect the use of evaluation training in evaluation capacity building?
Factors which cause organizations to become involved in evaluation training
Factors related to the utilization focus of the training
Factors related to the training itself
Factors related to the complementary learning strategies that reinforce use of the training
Factors related to the sustainability of training outcomes
A.Kuzmin, Use of training in ECB

4. A.Kuzmin, Use of training in ECB
Factors which cause organizations to become involved in evaluation training include:
External pressure that creates a demand for conducting evaluation, as in a request from a client or donor;
Internal pressure, as in the need to better control field operations, monitor programmes, or inform strategic decisions;
Needs related to the current state of the organization’s development, as in “start-up", “renewal", or “formalization";
The motivation of the top managers caused by dis-confirmation, lack of confirmation; feelings of anxiety, personal interest, curiosity, and so forth.
A.Kuzmin, Use of training in ECB

5. Factors related to the utilization focus of the training include:
Clarity prior to the training, about the intended use of the training and the primary intended users;
Involvement of the primary intended users in the training design and/or the training
Utilization-focused evaluation training (as opposed to evaluation training in general) is conducted for and with specific, intended primary users for specific, intended uses
A.Kuzmin, Use of training in ECB

6. Factors related to the training itself include:
Consideration of the actual needs of organizations, especially their existing and required evaluation capacities, in the training design
Duration of the program
Incorporation of practice (doing a real evaluation) in the training design;
Use of a variety of training methods, including a combination of lectures and interactive methods.
The personal involvement, competence and enthusiasm of the trainer or trainers
A.Kuzmin, Use of training in ECB

7. A.Kuzmin, Use of training in ECB
Factors related to the complementary learning strategies that reinforce use of the training include:
Learning by doing and mentoring
Peer learning
Self-education
Learning by sharing
A.Kuzmin, Use of training in ECB

8. Factors related to the sustainability of training outcomes include:
Support from the top managers
The regularity and intensity of evaluation practice
Research background of both employees involved in evaluation and the leaders of organizations
The degree of localization of evaluation capacity in organization
The inclusion of evaluation functions in the responsibilities of a person or a group
The availability of financial resources is an important condition for all five groups of factors
The availability of professional resources is an important condition for training and complementary learning
A.Kuzmin, Use of training in ECB

9. Evaluation Capacity Formula
“If you don’t use it, you don’t have it”
EC=C*P
EC is evaluation capacity;
C is evaluation competence; and
P is ongoing evaluation practice.
A.Kuzmin, Use of training in ECB

10. A.Kuzmin, Use of training in ECB
Toward a theoretical framework for an ECB intervention that includes a training component
Stages of an ECB intervention
Major contributions of activities undertaken
Make thoughtful decisions about evaluation training
Focus training on the intended use
Design and conduct a quality training
Provide complementary learning opportunities
Sustain training outcomes
Outcomes
Contributes to the development of evaluation competence
Increased evaluation competence
Contributes to the use of evaluation competence in the ongoing practice .
Effective use of the evaluation competence in the ongoing practice
A.Kuzmin, Use of training in ECB

11. Checklist: Understand the needs
Does the organization experience any external or internal pressures to develop evaluation capacity? If so, what specifically are the pressures?
What kind of evaluation capacity does the organization need to respond adequately, and in a timely manner, to these external and/or internal pressures? How will evaluation knowledge and skill help the organization to respond?
Where is the organization in its lifecycle and what are its developmental needs in consequence of being at that stage? How can evaluation training meet those needs?
What are the motivations of the top manager(s)? Why are they interested in using evaluation training in their organization? How can training meet their individual interests?
A.Kuzmin, Use of training in ECB

12. Checklist: Make sure that the training is utilization-focused
What is the specific intended use of the training?
Who are the primary intended users?
How can these users be involved in the training design?
How can they be involved in the training?
A.Kuzmin, Use of training in ECB

13. Checklist: Design and conduct a high-quality training
Are the organization’s needs reflected in the training design?
Does the training design fit the existing and required levels of the organization's evaluation capacity? Is it neither too basic nor too advanced?
Is the training long enough to achieve its goals?
Does the training include practicing evaluation in real life situations? Does it include reflection on the participants' experience?
Are various training methods used in the training? Is the balance between lectures and interactive methods appropriate?
Are the trainers competent, committed and enthusiastic?
A.Kuzmin, Use of training in ECB

14. Checklist: Provide complementary learning opportunities
Will the training participants have complementary learning opportunities in the course of the training? After the training?
Will these opportunities include learning by doing? Peer learning? Mentoring? Self-education? Learning by sharing?
Is there a quality control system in place to assure the adequacy of the participants' performance when they are learning by doing and/or by sharing?
A.Kuzmin, Use of training in ECB

15. Checklist: Make the training outcomes sustainable
Do the top managers support use of the training results? Did they make specific decisions regarding this use? Did they involve the evaluation training participants in these decisions?
Does the organization practice evaluation on a regular basis? Does it provide opportunities for trainees to do evaluation work on a regular basis?
How is evaluation capacity localized in the organization? Is this capacity distributed widely enough to prevent its loss if people competent in evaluation leave the organization?
Are the evaluation functions included in the formal job descriptions or informal responsibilities of a person or a group?
A.Kuzmin, Use of training in ECB

16. Checklist: Use financial resources effectively
Are sufficient funds available for a useful evaluation training?
What is the most appropriate allocation of available funds between the training and related activities, i.e., follow up training, complementary learning, evaluation practice, information dissemination, etc.)?
A.Kuzmin, Use of training in ECB

17. A.Kuzmin, Use of training in ECB
Short version:
Kuzmin, A. (2009). Use of evaluation training in evaluation capacity building. In M. Segone (Ed.), From policies to results: Developing capacities for country monitoring and evaluation systems. New York: UNICEF.
Full version:
Kuzmin, A. (2004). Exploration of factors that affect the use of evaluation training in evaluation capacity development (Doctorate Dissertation). Cincinnati, Ohio Union Institute and University.
A.Kuzmin, Use of training in ECB