1. Evaluation plans for 2014-2020 programming period in Poland
Experience and new arrangements
Ministry of Infrastructure and Development, Poland
Athens, 15th May, 2014
Joint Evaluation Network Meeting

2. Institutional context for evaluation planning
European Commission
National Evaluation Unit
Operational Programmes Evaluation Units
Evaluation Units – Intermediate Bodies
16 Regional Operational Programmes Evaluation Units (ESF + ERDF)
ESF Evaluation Unit
7 persons
20 persons
60 persons
35 persons
43 persons
Closer coordination as a main challenge for period

3. Polish experience on evaluation planning 2007-2013
period: evaluation plans established for NSRF and all operational programmes
Strategic evaluation plans for the whole period
strategic goals for evaluation process
organisational arrangements (thematic steering groups)
main thematic areas
main evaluation capacity building activities
Annual evaluation plans
list of evaluation studies + description
evaluation capacity building activities

4. Polish experiences on evaluation planning 2007-2013
All plans examined by the National Evaluation Unit
Shortcomings:
difficult to coordinate
no strategic integration of an evaluation process with implementation and intervention logic → annually planned evaluations mainly for current informational needs
no coherent evaluation agenda for the whole Cohesion Policy
Advantages:
high degree of flexibility

5. Evaluation planning 2014-2020 in Poland
Process of evaluation planning in Poland to be regulated by the Guidelines on evaluation activities in the period (draft)
In the field of the evaluation plans the Guidelines contains:
general rules
obligations for institutions, and role of the main stakeholders
structure and content of evaluation plans
deadlines and logistic/organisational matters
list of obligatory evaluation studies (e.g. ex-ante, mid-term, ex-post, each priority evaluation, annual systematic review of evaluation studies, gender equality evaluation)

6. Evaluation planning – general principles
Evaluation process closely related to the intervention logic
evaluation plans (esp. scope and timing) reflect the intervention logic of the programme
intervention logic as a starting point for identification of the informational needs (what kind of and when evaluation evidence will be needed)
all planned evaluation activities should be justified in this context
(2) Planning process closely coordinated by NEU in order to ensure the coherence of the evaluation system
twofold aim of the planning: 1) effective evaluation of each programme 2) effective evaluation of the whole Partnership Agreement intervention
coherent evaluation agenda is needed → operational programme evaluations + evaluations in the PA level (meta-evaluations, systematic reviews, strategic and horizontal evaluations)

7. Evaluation planning – general principles
(3) Monitoring of evaluation plans implementation
whole evaluation process monitored by National Evaluation Unit
special emphasis on recommendation implementation process → Recommendations Implementation System (RIS)
annual reports on the progress of implementation of evaluation plans (esp. recommendations implementation)

8. Evaluation planning in Poland – proposed rules and procedures
Evaluation plans prepared for every operational programme by the Evaluation Units for the whole programming period
National Evaluation Unit preparing the Evaluation Plan for Partnership Agreement
Annual update as a tool for changes and supplementation
Dilemmas:
Evaluation plan for the whole period contains all evaluation studies planned for each programme vs. contains only strategic and most important studies
Annual update of one plan vs. annual plans as a supplement for the strategic plan
Evaluation Units are obligated to ensure:
linkage between the planned evaluation studies and the intervention logic of the programme
high quality of evaluation studies
identification of needed data
resources needed for evaluation process

9. Evaluation planning in Poland – proposed rules and procedures
All evaluation plans and their annual updates accepted by National Evaluation Unit
NEU’s opinion based on the criteria of: adequacy, coherence, feasibility, and efficiency in order to ensure the coherence of the whole cohesion policy evaluation process
After the NEU decision, evaluation plans are accepted by Monitoring Committees

10. Main challenges for evaluation planning
strategic planning vs. current evaluation needs → annual update or annual plans as a tool for flexibility
effective integration of evaluation process with the intervention logic (and then implementation process) within evaluation plans → high expert and evaluation potential needed → cooperation with external experts + high quality trainings
effective coordination in order to establish coherent evaluation agenda for the whole Cohesion Policy → National Evaluation Unit responsibility
identification and access to the needed data (in particular for counterfactual studies)

11. Preparation to evaluation planning in Poland
National Evaluation Unit prepared detailed Guidelines on evaluation activities in the period with section dedicated to evaluation planning.
After the discussion with the OP Evaluation Units, NEU decided to support evaluation units, and to organise the evaluation planning process in closely coordinated manner.
NEU is going to launch the project: Process of preparing evaluation plans for Cohesion Policy
In cooperation with experts on evaluation and sectoral matters
Main goals of the project:
Development of the evaluation planning methodology (taking into account intervention logic)
Preparation of evaluation plans for Operational Programmes and Partnership Agreement

12. Preparation to evaluation planning in Poland
Main stages of the project:
Pilot project – preparation of an evaluation plan for one chosen Polish region (Regional Operational Programme)
intervention logic analysis → expert panel
workshops with main Regional Operational Programme stakeholders and partners
preparation of the Regional Operational Programme Evaluation Plan
Preparation of the Partnership Agreement Evaluation Plan
presentation and discussion on a pilot project findings
workshops with main stakeholders and partners
preparation of the Partnership Agreement Evaluation Plan

13. Preparation to evaluation planning in Poland
Preparation of evaluation plans for all Operational Programmes
with the support of the NEU (help desk) and external evaluation experts
NEU opinion
Monitoring Committee approval
Sending plans to European Commission
Main products:
evaluation plans for OP’s and PA
methodology document on evaluation planning

14. EVALUATION PLAN OF KNOWLEDGE – EDUCATION – DEVELOPMENT OPERATIONAL PROGRAMME
– MAIN ASSUMPTIONS

15. A ROLE OF EVALUATION PLAN
Programme evaluation management
Predictable information needs fullfilment
Evaluation WWW - (What When Who)
Rising awarness about preparations to be done
Transparency
Stakeholder’s interest taken into account

16. MAIN PRINCIPLES OF THE EVALUATION PLAN
1 EP for 1 OP
Evaluation experts functionally independent
Evaluation capacity building
Social partners involvement
Long-term orientation = flexibility of provisions

17. FLEXIBILITY OF THE EVALUATION PLAN
EP should be a strategic oriented document.
List of evaluation studies is open and indicative.
The list covers only evaluation studies that are predictable.
Detailed information about the studies will be updated yearly.

18. ELEMENTS OF THE EVALUATION PLAN
Definition – Goals – Challenges and Principles for evaluation process
Data access and provisions
Process of coordination
Capacity building and evaluation culture
Use and dissemination of findings
Indicative list of planned evaluation studies

19. EVALUATION STUDIES CHARACTERISTICS
Subject /area
Purpouse and scope
Who
Evaluation approach planned
Tentative term – usually approximately
Budget – broadly estimated
Other information if necessary

20. TEMPLATE - EXAMPLE
LP.
SUBJECT/ AREA
GOAL/
RATIONALE
EVAL. UNIT
EVALUATION APPROACH
2015
2016
2017
2018
2019
2020
2021
2022
2023
COST
(pln)
OTHER INF. 1
ESF IMPACT ASSESMENT ON VET TRAININGS QUALITY DEVELOPMENT
……………
Ministry of Education
TBE x
Cooperation with Polish Agency for Enterpreneurial Development  

21. Metaevalution, syntheses Diagnoses, needs and barriers identification
TYPES OF EVALUATION STUDIES
Obligatory studies
Metaevalution, syntheses
Diagnoses, needs and barriers identification
Horizontal principles
Specific target groups oriented
Comparability studies
Capturing the effects
Ad hoc studies

22. Research obligatory or necessary Proposals from ex-ante evaluation
MAIN SOURCES OF IDEAS
Research obligatory or necessary
Proposals from ex-ante evaluation
Proposals made by the IB’s
and experience
Policy makers needs – if possible to identify

23. Thank you for your attention! Ministry of Infrastructure & Development
Ministry of Infrastructure & Development
POLAND