1. Alfredo Gonzalez Cambero FAO Salomón Salcedo Baca FAO Renato Olvera Nevarez Mexican Ministry of Agriculture November 11, 2010 San Antonio, TX

2.  Program evaluation exists because of the expectation its results will be used for:  Program improvement (by means of feedback):  Valuation of program adequacy  Measurement of impacts and performance  Detection of areas of improvement  Program termination  Accountability  If evaluations are not used, their funding should be used for other purposes.

3.  Conceptual: To better understand a program or related issues.  Strategic/symbolic: To persuade others or to use evaluation findings to gain a particular outcome.

4.  Process: Engagement in the process of evaluation (deliberated use during the evaluation process).  Creating and enhancing shared understandings  Increasing participants´ engagement, sense of ownership  Learning to learn.  Instrumental: Change in decision-makers´ understanding of a situation to provide information for instrumental reasons. To make overt decision making.

5.  Bottom-line: The use of the evaluation is measured by the impact of the evaluation itself: ◦ What did the evaluation help to change? ◦ To what extent changes are due to the evaluation findings and recommendations?  This subsumes into: Influence of the evaluation. Achilles' heel of program evaluation: The use of evaluation. Achilles' heel of program evaluation: The use of evaluation.

6.  Process based influence of evaluation:  Cognitive: Changes in understandings stimulated by discussion, reflection and problem analysis.  Affective: Individual and collective feelings of worth resulting from involvement.  Political: Creation of new dialogs to draw attention to social problems or to influence power relationships.

8. Methodological quality Usefulness of findings Usefulness of recommendations: what and how Quality is a necessary condition for evaluation to be used.

9. It should be: Participatory Impartial and independent, preserving the freedom and criteria of the evaluator. Impartiality gives credibility to evaluation findings. Independence gives legitimacy to the evaluation. Objective: Avoid unsolicited flattering and unsubstantiated criticism. Proposal oriented: Concrete and viable proposals related to the substance. For the evaluation to be useful …

10. 10 The Mexican Government instruments public policies that look for the wellbeing of the rural population. UNDAF PRIORITY AREAS Strengthened public policies and capacities in order to: a) Promote new employment opportunities and quality of labor. b) Bolster equal opportunity, sustained and competitive economic growth. Principles of sustainable development incorporated into national and regional programs. COINCIDENCE IN DEVELOPMENT OBJECTIVES EVALUATION AND POLICY ANALYSIS To contribute to the sustainable development of the rural and fishing sector through greater efficacy and efficiency of the rural public policy. Judgment elements for program decision making, which are pertinent and timely.

11. To identify areas for improvement and to propose changes in the design and operation of Alianza para el Campo to increase its operating effectiveness and results. To contribute to capacity building for the evaluation and analysis of rural public policy. Evaluation Objectives Continuous analysis Prospective vision Practical usefulness Feedback for decision-making Approach

12. Verification of norms Verification of financial and physical goals Based on beneficiaries’ perception. 1998-2000 Advancement towards impact and process evaluation. 2001 Advancement towards measurement of realized impacts Incorporation of capacity building activities Technical support for program implementation in trial states (Jalisco y Tabasco) 2002 Attention to critical aspects identified by programs and by previous evaluations – continuous analysis Better feedback and participation of program officials Impact evaluation sets in Technical support for program implementation extends to more states (Tab, N.L, Jal, Gto, Sin, Son, Zac, Chih, Nay, Tam and Tlax) 2003 – 2006 Mandated by the Ministry of Agriculture Mandated by Budget Law

13. TopicfromtoAchievements Role of SAGARPA Little ownership of and feedback on evaluation reports by program officials. Greater interest by program officials to participate in the drafting of evaluation terms of reference and to discuss evaluation reports. Evaluation ownership and a better match between expectations and evaluation products. Technical support and capacity building Technical supports kicks in at a basic level (there is insufficient capacity for agricultural program evaluation) Workshops to share conceptual, methodological, and organizational guidelines. Formal distance education to train evaluators. Minimum quality standards and homogeneity in evaluations. Network of trained evaluators available. Follow up on recommendations No follow up on evaluations´ recommendation. Implementation of recommendations´ follow up. Use of a system for recommendations as a tool to support the improvement of program operations.

14. Timely and reliable information for program decision-making in order to improve public policy application and accountability:  To promote results based operation  To evaluate results  To promote the systematic use of evaluation findings  To improve accountability  Monitoring and evaluation of useful programs and government entities. Objective

15. Current situation Challenges Program operating areas are not prepared. To foster a culture of evaluation in government entities which contributes to decision-making. There is no information or monitoring systems for program operation. To implement a Monitoring and Evaluation System. Lack of coherence in programs and alignment with superior policy objectives. To adequately direct resources and to make programs more efficient and effective. Lack of coordination between agency suppliers and users of information. To adequately direct activities on planning, budgeting, execution, and public expenditure control.

16. Design ImplementationExecutionEvaluation Planning Monitoring Intermediate evaluations Continuous feedback RESULTS BASED PROGRAMS

17. Alfredo Gonzalez Cambero agonzalez@fao-evaluacion.org.mx Salomón Salcedo Baca Salomon.Salcedo@fao.org Renato Olvera Nevarez rolvera@sagarpa.gob.mx