BUILDING IMPLEMENTATION CAPACITY FOR STRUCTURAL FUNDS CSF & JROP CZ 02 IB SPP III Training seminar on evaluation – Prague 26/27 february 2004 The organization of evaluation Iolanda Anselmo Evaluation Unit Ministry of the Economy and Finance

 Objectives and use of evaluation  Evaluation and related activities  Ways of organising evaluations  Planning evaluations  Developing an evaluation project STRUCTURE OF PRESENTATION

 Democracy: being accountable to funders, politicians, citizens, & the public at large  Learning: satisfying information needs of planners & implementers WHY EVALUATE?

 Securing information for: designing programs, anticipating problems, providing directions for project (& targeting) choices, & adapting programs to nat’l & local needs, negotiating with institutional & social partners  Involve social partners in negotiations, planning, & implementation  Providing information on programs to politicians POSSIBLE USES OF EVALUATION IN STRUCTURAL FUND CONTEXT

 Program evaluation: Under which conditions/for whom/where does this intervention work?  Meta-evaluation/Synthesis of evaluations: Under which conditions has this type of intervention proven to work?  Policy analysis: What type of action(s) can solve this problem? TYPES OF ACTIVITIES PRODUCING INFORMATION ON AND FOR PROGRAMS

 Project appraisal and selection  Control  Monitoring  Research Program evaluation: systematic observation of public intervention aimed at making judgements & providing information for action OTHER RELATED ACTIVITIES

 Self-evaluation  Internal  External In any case independence and participatory approach is possible at various degrees WAYS OF ORGANIZING EVALUATIONS

 promoted by the CSF Monitoring Committee as an additional source for the mid-term review  proved useful to gather information not included in the monitoring system and which is often tacit and not easily transferred SELF-EVALUATION OF THE ITALIAN OBJECTIVE CSF (1)

 all OP measures classified according to their performance (+/-/=) against three criteria:  appropriate implementation procedures  Fulfilled expenditure forecasts  selection of strategy-consistent projects  measures scores decided by OP MAs and discussed in workshops with representatives of CSF MA, central and regional evaluation units, independent evaluators SELF-EVALUATION OF THE ITALIAN OBJECTIVE CSF (2)

Self-evaluation:  Reliable info, difficult truths, inexpensive, but info on results missing & good for internal use In-house:  quality, utility, timing, but costly External:  demand must be well designed  interactions needed between evaluator, planning administration and partners WAYS OF ORGANISING EVALUATIONS: HOW TO CHOOSE

Obtaining & using evaluations requires:  Identifying evaluation needs  Combining types of evaluation activities/ways of organizing (evaluation plan)  Good management of each evaluation (evaluation project) PLANNING EVALUATIONS

1.Develop the evaluation project 2.Express the evaluation demand 3.Manage the evaluation 4.Disseminate (& accept) results 5.Develop new evaluation questions AN EVALUATION PROJECT: TASKS

 what to evaluate and what for  who conducts the eval. (self/internal/external)  how to select external evaluators  who checks for quality  which products, when, for whom  how to protect implementers DEVELOPING AN EVALUATION PROJECT

 Collect stakeholders’ needs and questions  Write ToRs  Refine the questions over time EXPRESSING THE EVALUATION DEMAND

 Ensure that the question is answered  Ensure quality: o Steering Group (partners, experts, Evaluation Unit, Managing authority?) o Evaluation of evaluations  providing information flows to & from evaluator  protect evaluator’s independence (Steering Group, Evaluation Unit, selection of evaluator, contract) MANAGING THE EVALUATION

 A product for each product: non only eval. report, but also executive summaries, workshops, TV & radio, press releases  Dissemination occurs over time  Internal and external dissemination  Formal and informal dissemination DISSEMINATE AND ACCEPT

 Develop the evaluation project: ev. unit/responsible + M.A.  Express the evaluation demand: stakeholders with the help of experts (evaluation unit/responsible)  Manage the evaluation: steering group + ev. unit  Disseminate (& accept) results: ev. unit/responsible + M.A.+ promotion unit + Monitoring Committee AN EVALUATION PROJECT: WHO DOES WHAT