EXPERIENCES OF THE EVALUATOR RNDr. Zuzana BOUKALOVÁ CROSSCZECH, CCSS, GEO Group

Professional Experience Charles University, Faculty of Science, Dept. of Hydrogeology, Civil Engineering Charles University, Faculty of Science, Dept. of Hydrogeology, Civil Engineering Geological Institute of Czechoslovak Academy of Science - hydrogeologist Geological Institute of Czechoslovak Academy of Science - hydrogeologist University Politecnica de Valencia, Spain – groundwater modelling University Politecnica de Valencia, Spain – groundwater modelling HR Wallingford Ltd. – water management HR Wallingford Ltd. – water management

Back in Prague from 1996 GEO Group, a.s.: GEO Group, a.s.: participation in the management, design and realization of groundwater and non-saturated zone pollution survey and remediation projects participation in the management, design and realization of groundwater and non-saturated zone pollution survey and remediation projects supervising expert in the team for government remediation programmes in groundwater and soil pollution. supervising expert in the team for government remediation programmes in groundwater and soil pollution. PHARE, PHARE-CREDO, ISPA and SAPARD project coordinator PHARE, PHARE-CREDO, ISPA and SAPARD project coordinator 5- th and 6-th FP projects managing director (CEGRMOMA, IRON CURTAIN, IMPACT, TRANSCAT, FLOODsite) 5- th and 6-th FP projects managing director (CEGRMOMA, IRON CURTAIN, IMPACT, TRANSCAT, FLOODsite)

From the year 2000 VZ GLS, a.s. as the 5.FP projects LOWRGREP and ENERGY FOREST managing director VZ GLS, a.s. as the 5.FP projects LOWRGREP and ENERGY FOREST managing director CROSSCZECH a.s. : Head of the Department, Hydrogeologist, Consultant CROSSCZECH a.s. : Head of the Department, Hydrogeologist, Consultant Czech Center for Strategic Studies (CCSS), the member of the Managing Board, ARMONIA project WP leader Czech Center for Strategic Studies (CCSS), the member of the Managing Board, ARMONIA project WP leader EDUCEUM – the pool of 6 experts having a long lasting experience in different areas of European research and EU funding. EDUCEUM – the pool of 6 experts having a long lasting experience in different areas of European research and EU funding.

Evaluator, Evaluator, Rapporteur Marie Curie Actions (2003, 2004) Marie Curie Actions (2003, 2004) eContent (2003, 2005) eContent (2003, 2005) Member of the „ Sounding Board “ of the Commissioner Janez Potočnik, responsible for Science and Research within the European Commission (simplification of Framework Programme procedures, FP7), 2005 Member of the „ Sounding Board “ of the Commissioner Janez Potočnik, responsible for Science and Research within the European Commission (simplification of Framework Programme procedures, FP7), 2005

Marie Curie Actions Marie Curie Excellence Grants: Marie Curie Excellence Grants: Excellence Grants, Awards, Chairs (remote evaluation) Marie Curie Conferences and Training Courses (remote evaluation)

Marie Curie Actions For the evaluation, the proposals are divided to 8 areas (disciplines): For the evaluation, the proposals are divided to 8 areas (disciplines): CHE: chemistry CHE: chemistry ECO: economics ECO: economics ENG: engineering ENG: engineering ENV: environment ENV: environment LIFE: life sciences LIFE: life sciences MAT: mathematics and information society MAT: mathematics and information society PHY: physics PHY: physics SOC: social sciences and humanities SOC: social sciences and humanities Section Z: multidisciplinar projects Section Z: multidisciplinar projects

Marie Curie Actions Year 2004: Year 2004: GenderMFTotal Heiger Education RES IND437 OTH325 Total544296

MC Actions – SCORE (evaluations by experts) 0 – the proposal fails or cannot be judget against the criteria due to missing or uncomplete information 0 – the proposal fails or cannot be judget against the criteria due to missing or uncomplete information 1 – poor 1 – poor 2 – fair 2 – fair 3 – good 3 – good 4 – very good 4 – very good 5 – excellent 5 – excellent

MC actions EVALUTION Remote evaluation (3 evalators -independent experts- per 1 proposal); confidentiality and non-conflict of interest: Remote evaluation (3 evalators -independent experts- per 1 proposal); confidentiality and non-conflict of interest: individual report forms (IRF) A rapporteur will be nominated to prepare the Consensus Report (CR) for a given proposal and obtain approval from the other evaluators Panel meeting in Brussel; f Panel meeting in Brussel; for each proposal a consensus should be reached and a CR will be prepared and signed by the triplet. One CR form per proposal. The experts will make a list of the proposals ranked The experts will make a list of the proposals ranked The experts’ conclusions are examined by the EC The experts’ conclusions are examined by the EC Evaluation Summary Report (ESR) giving the opinion of the evaluators to the Coordinator of the proposal, on the basis of the Consensus Report Evaluation Summary Report (ESR) giving the opinion of the evaluators to the Coordinator of the proposal, on the basis of the Consensus Report

eContent program A multiannual Community programme to simulate the development and use of European digital content on the global networks and to promote the linguistic diversity in the Information Society A multiannual Community programme to simulate the development and use of European digital content on the global networks and to promote the linguistic diversity in the Information Society eContenplus programme (May 2005) eContenplus programme (May 2005) Call for experts!!! Call for experts!!!

eContent evaluation LUXEMBOURG LUXEMBOURG 1 week evaluation 1 week evaluation Score 1 – 5 Score 1 – 5 Triplet per 6 – 10 proposals Triplet per 6 – 10 proposals Panel meetings Panel meetings Evaluation and reports, all finished in Luxembourg Evaluation and reports, all finished in Luxembourg

Review Procedure of the eContent proposals 1=Unsatisfactory, 1=Unsatisfactory, 2=Poor, 2=Poor, 3=Satisfactory, 3=Satisfactory, 4=Good, 4=Good, 5=Excellent 5=Excellent

Agenda of the review Introduction, Objectives, context and purpose of a final review by PO Introduction, Objectives, context and purpose of a final review by PO Presentation: Aim and progress of work (coordinator) Presentation: Aim and progress of work (coordinator) Individual Work-packages Presentation and discussion (WP leaders) Individual Work-packages Presentation and discussion (WP leaders) Demonstration of the Web/CD-ROM product (prototype) – consortium Demonstration of the Web/CD-ROM product (prototype) – consortium Questions and answer session Questions and answer session PO + reviewers – evaluation PO + reviewers – evaluation PO close, recommendations PO close, recommendations Successful completion – Modify – Rejected Successful completion – Modify – Rejected Review reports finished by experts from home till 1 month Review reports finished by experts from home till 1 month

„ Sounding Board “ smaller actors in the context of simplification of Framework Programme procedures and implementation smaller actors in the context of simplification of Framework Programme procedures and implementation to incorporate the views of experienced stakeholders into the development of the Seventh Framework Programme (FP7) to incorporate the views of experienced stakeholders into the development of the Seventh Framework Programme (FP7) 3 – 4 meetings per year 3 – 4 meetings per year 1-st meeting: st meeting:

SB meeting Commission first ideas to achieve substantial simplification of the FP7 Commission first ideas to achieve substantial simplification of the FP7 FP Action Plan on Rationalisation and Acceleration FP Action Plan on Rationalisation and Acceleration I. Actions to simplify and accelerate I. Actions to simplify and accelerate II. Actions to improve quality and effectiveness II. Actions to improve quality and effectiveness

Actions to improve quality and effectiveness The established sets of uniform beings will be the basis for briefing all external evaluators to ensure consistency of approach The established sets of uniform beings will be the basis for briefing all external evaluators to ensure consistency of approach DONE: standard briefing available on internal website DONE: standard briefing available on internal website 1

Actions to improve quality and effectiveness Evaluation Summary Reports sent to proposers MUST always be of high quality Evaluation Summary Reports sent to proposers MUST always be of high quality EC will closely monitor the output from consensus groups and panels EC will closely monitor the output from consensus groups and panels DONE: introduced in evaluation workshops and briefings DONE: introduced in evaluation workshops and briefings 2

Actions to improve quality and effectiveness Quality of evaluators!!! Quality of evaluators!!! may include contacts by senior officials with industrial umbrella groups, requesting them to nominate highly qualified individuals – experts may include contacts by senior officials with industrial umbrella groups, requesting them to nominate highly qualified individuals – experts ONGOING ONGOING 3

Actions to improve quality and effectiveness Further use of two stage proposal submission and evaluation (for IP, NoE,…? STREP ?) more experts for evaluations Further use of two stage proposal submission and evaluation (for IP, NoE,…? STREP ?) more experts for evaluations Guidelines - revised: first stage is to be as light as possible, based on limited number of criteria with limited administrative data Guidelines - revised: first stage is to be as light as possible, based on limited number of criteria with limited administrative data The second stage will be based on FULL SET of evaluation criteria The second stage will be based on FULL SET of evaluation criteria DONE DONE 4

Actions to improve quality and effectiveness Rules on annual reviews of the projects will be developed Rules on annual reviews of the projects will be developed ONGOING ONGOING Review guidelines to be finalised and put on CORDIS Review guidelines to be finalised and put on CORDIS 5

THANKS for your attention ! Zuzana Boukalova Contact: