Experiences, perspectives and outcomes of trans-national research: EUPHRESCO Analysis Sylvia Blümel & Alois Egartner AGES, Institute for Plant Health (Partner 4) EUPHRESCO Final Dissemination Conference FERA, Sand Hutton, York (UK), 30 th June 2010 EUPHRESCO EUropean PHytosanitary REsearch COordination

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 2 “REAL” Exercise for transnational activities √ To trial √ EVALUATE and √ REFINE the instruments (produced through specific pilot activities)

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 3 PILOT ACTIVITIES  CALLS for PROPOSALS,  COMMISSIONING of PILOT PROJECTS  and subsequent MONITORING /Evaluation  EVALUATE INSTRUMENTS and PROCESSES used in the PILOT PROJECTS = CALL EVALUATION  => Recommendations for improvement

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 4 AIMS:  FEEDBACK from „actors“ involved in the call (overall impression)  BASIS FOR THE REFINEMENT of the draft tool book CALL EVALUATION Main Evaluation (VP + RP) Final Evaluation (VP, with approval of final report) NC Evaluation EVALUATION PARTS:

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 5 ACTORS & Questionnaires VP – CALL funders (CSC & NMG) partners NCCP Peer Reviewers Applicants RP – CALL funders partners Peer Reviewers Applicants NC-Mechanism partners participants

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 6 EVALUATION APPROACHES  Overall satisfaction and expectations  Topic identification and selection processes  Call, commissioning, monitoring and evaluation procedures (including the tools, documents and processes used)  Transparency in communication  The degree of involvement of different actors  The timeline and duration of various call phases  The added value of the pilot projects as joint activities  Recommendations for future calls and trans-national projects

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 7 Questionnaires – Rate of Return 54.7 % overall Rate of Return (from 247 Qs)

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 8 Results Questonnaire Evaluation  GENERAL CONCLUSIONS from the Evaluation  BENEFITS and ADDED VALUE  LESSONS LEARNT and RECOMMENDATIONS

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 9 MOTIVATION and EXPECTATIONS for PARTICIPATION in pilots Funders: testing of transnational funding mechanisms (learning-by-doing), experience with network building collaboration and optimizing policy-driven plant health research funds Researchers & scientific peer reviewers: scientific interest in the call topics, scientific interest in transnational co-operation scientific interest in the support of an important research field

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 10 Satisfaction with the overall call management and execution

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 11 BENEFITS and ADDED VALUE

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 12 BENEFITS and ADDED VALUE

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 13 BENEFITS and ADDED VALUE

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 14 The MUTUAL OPENING of NATIONAL PROGRAMMES Faster funding & commissioning of work & lower administrative burdens Building NETWORKS of FUNDERS and RESEARCHERS Building MUTUAL TRUST and CONFIDENCE in TRANS- NATIONAL COOPERATION and collaboration. OPTIMISING FUNDS & RESEARCH OUTPUTS MORE EFFICIENT USE of existing expertise and resources, contributing to multi-disciplinary work and higher quality research BENEFITS and ADDED VALUE  exchanges of information  funding collaborations  potential for trans-national flow of funds)  better address plant health policy needs & enhancing European phytosanitary science capability  division of labour  lower costs  higher quality research  research co-operation and the sharing of ideas, knowledge, expertise & facilities  avoiding duplication

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 15 BENEFITS and ADDED VALUE of ERA-NET transnational programmes

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 16 Results Questonnaire Evaluation LESSONS LEARNT & RECOMMENDATIONS

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 17 adaptations and improvements to the topic selection and development process call budget & pilot project budgets TOO LOW LESSONS LEARNT & RECOMMENDATIONS

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 18 call budget & pilot project budgets TOO LOW

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 19 more money should be allocated to future calls LESSONS LEARNT & RECOMMENDATIONS

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 20 adaptations and improvements to the topic selection and development process call budget & pilot project budgets TOO LOW call information sources to be adapted low access to and availability of scientific peer reviewers Construction of decision making body advisable (e.g. a Call Steering Committee) supported by a suitable administrative body (e.g. a Call Secretariat) LESSONS LEARNT & RECOMMENDATIONS

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 21 LESSONS LEARNT & RECOMMENDATIONS

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 22 Future participation of actors groups involved in EUPHRESCO

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 23 EVALUATION SUMMARY pilot projects CRITERIONFUNDING MECHANISM Main recommendations for future transnational activities Virtual common pot Real common potNon-competitive Number of topics524should be handled flexibly No. of peer reviewers used per topic 2–43–4n.a.should be handled flexibly Competition between topics in these EUPHRESCO pilot calls Noyesnoshould be handled flexibly Competition between research bidders/ consortiums possible yes noshould be handled flexibly Competition achieved no, since no competition between topics and only 1 bid received per topic yes (but limited), since topics competed and only 3 bids received to the call) No, since no competition between topics and only 1 bid per topic Select funding mechanism according to the need and potential for competition, i.e. topic dependant

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 24 EVALUATION SUMMARY pilot projects CRITERIONFUNDING MECHANISM Main recommendations for future transnational activities Virtual common pot Real common potNon-competitive Number of topics524should be handled flexibly No. of peer reviewers used per topic 2–43–4n.a.should be handled flexibly Competition between topics in these EUPHRESCO pilot calls Noyesnoshould be handled flexibly Competition between research bidders/ consortiums possible yes noshould be handled flexibly Competition achieved no, since no competition between topics and only 1 bid received per topic yes (but limited), since topics competed and only 3 bids received to the call) No, since no competition between topics and only 1 bid per topic Select funding mechanism according to the need and potential for competition, i.e. topic dependant VPRPNCRecommendation Competition achieved no, since no competition between topics and only 1 bid received per topic yes (but limited), since topics competed and only 3 bids received to the call) No, since no competition between topics and only 1 bid per topic Select funding mechanism according to the need and potential for competition, i.e. topic dependant

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 25 VPRPNC Restrictions on bidders and potential for trans-national flow of funds yes funders only fund their own national researchers yes Potential trans-national flow, but bidders had to have a legal base in the funding countries; others involved via sub-contracts. yes funders fund their own national researchers Ideally, reduce restrictions on bidders and encourage more mutual opening Transfers of funds between funders noyesno Transnational transfer flow funds should be increased Total budget per mechanism and per project (€) € (€80.000– ) € ( – ) n.a. project budgets for competitive calls should be increased for ‘full’ calls (compared with these pilots) Number of funders per topic 4–925–9should be handled flexibly Bidding process 1-step (full proposals) 2-step (expressions of interest, then invited full proposals) n.a. (no call; research consortium formed and invited to produce a workplan against the topic scope) should be handled flexibly A 2-step process may minimise burdens on bidders and encourage more proposals EVALUATION SUMMARY pilot projects (cont.)

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 26 VPRPNC Restrictions on bidders and potential for trans-national flow of funds yes funders only fund their own national researchers yes Potential trans-national flow, but bidders had to have a legal base in the funding countries; others involved via sub-contracts. yes funders fund their own national researchers Ideally, reduce restrictions on bidders and encourage more mutual opening Transfers of funds between funders noyesno Transnational transfer flow funds should be increased Total budget per mechanism and per project (€) € (€80.000– ) € ( – ) n.a. project budgets for competitive calls should be increased for ‘full’ calls (compared with these pilots) Number of funders per topic 4–925–9should be handled flexibly Bidding process 1-step (full proposals) 2-step (expressions of interest, then invited full proposals) n.a. (no call; research consortium formed and invited to produce a workplan against the topic scope) should be handled flexibly A 2-step process may minimise burdens on bidders and encourage more proposals EVALUATION SUMMARY pilot projects (cont.) VPRPNCRecommendation Restrictions on bidders and potential for trans- national flow of funds yes funders only fund their own national researchers yes Potential trans- national flow, but bidders had to have a legal base in the funding countries; others involved via sub- contracts. yes funders fund their own national researchers Ideally, reduce restrictions on bidders and encourage more mutual opening VPRPNCRecommendation Total budget per mechanism and per project (€) € (€80.000– ) € ( – ) n.a. project budgets for competitive calls should be increased for ‘full’ calls (compared with these pilots)

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 27 Research consortia neededyesnoyes should be handled flexibly, but consortia typically add value No. of researcher partners per topic 4–93–45-12 should be handled flexibly, but is dependant of the topic and funding mechanism Length of time from call or project initiation to start of project ~8 month (Mar 08 – Nov 08) 7–9 months<1–7 months Topic development and commissioning should be as quick as possible for emergency policy/operational needs Pilot project length12 months18 months 1st round: months; 2 nd round months should be handled flexibly, but 2 to 3-year projects are considered ideal Interim report used/testedYes, partlyYesnon.a Peer review of final reports tested? YesNonoshould be handled flexibly EVALUATION SUMMARY pilot projects (cont.) Research consortia neededyesnoyes should be handled flexibly, but consortia typically add value No. of researcher partners per topic 4–93–45-12 should be handled flexibly, but is dependant of the topic and funding mechanism Length of time from call or project initiation to start of project ~8 month (Mar 08 – Nov 08) 7–9 months<1–7 months Topic development and commissioning should be as quick as possible for emergency policy/operational needs Pilot project length12 months18 months 1st round: months; 2 nd round months should be handled flexibly, but 2 to 3-year projects are considered ideal Interim report used/testedYes, partlyYesnon.a Peer review of final reports tested? YesNonoshould be handled flexibly VPRPNCRecommendation Length of time from call or project initiation to start of project ~8 month (Mar 08 – Nov 08) 7–9 months <1–7 months Topic development and commissioning should be as quick as possible for emergency policy/operational needs

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 28  CALL EVALUATION => Recommendations  Refine instruments & processes => Final Operational Handbook

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 29 CONTENT Operational Toolbook Introduction (Background; Aim and addressees; Information sources; How to use the Handbook; Terms & References) Overview - Phases of research commissioning Research initiation (General sequence) Research initiation (Rapid sequence for NC) Research implementation – competitive mechanisms (RP, VP) Call preparation; Call execution; Evaluation of proposals; Commissioning of projects; Project monitoring and dissemination; Dissemination activities) Research implementation - non-competitive (NC) mechanism (Project commissioning and establishment; Project monitoring and dissemination; dissemination activities) Annex (Tools, documents)

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 30 Survey - Phases of research initiation & implementation Start of research implementation phase I--non–competitive--I competitive I Applying steps

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 31 Refined time chart for competitive funding mechanisms* * (based on pilot call experience)

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 32 Activity & responsibility matrix - competitive funding mechanisms

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 33 ANNEX Operational handbook: 54 separate documents C - Research initiation (General sequence) Supporting document: Checklist for principle decisions (comp. & NC) Tool: Mandate to Call Secretariat (comp. & NC) Tool: Questionnaire to identify funders and funds (comp. & NC) Tool: Questionnaire on suggestions for research topics (comp. & NC) Supporting document: Topic selection criteria (comp. & NC) Supporting document: Topic coordinator tasks (comp. & NC) Tool: Short topic description (comp. & NC) Supporting document: Table for topic prioritization (comp. & NC) Supporting document: Checklist for meeting M1 (comp. & NC) Tool: Call principles (Roadmap) (comp.) Tool: Instruction for elaboration of NC-project (NC) Tool: Letter of Commitment (LoC) - funder (comp.) Tool: Letter of Intent – funder (NC) Tool: Terms of References (TOR) (comp. & NC) D - Research initiation (Rapid sequence for NC) E - Research implementation – competitive mechanisms (RP, VP) Tool: Call announcement (comp.) Tool: Applicants Guide (full proposals) (VP) Tool: Application Form (full proposals) (VP) Tool: Applicants Guide for Expression of interest (RP) Tool: Application Form for Expression of interest (RP) Tool: Applicants Guide (full proposals) (RP) Tool: Application Form (full proposals) (RP) Tool: Letter of Declaration (RC) (comp.) Supporting document: Tasks of project coordinator (PC) (comp.) Tool: Funder evaluation form – Expression of Interest (comp.) Tool: Network eligibility check compilation (Proposal) (comp.) Tool: Confirmation of national eligibility check (comp.) Tool: Eligibility check – results per proposal (comp.) Supporting document: Scientific Peer Reviewer Check (comp. & NC) Tool: Confidentiality agreement for SPR (comp. & NC) Tool: Scientific Peer Review Guidelines (Proposal) (VP) (comp.) Tool: Scientific Peer Review Form (Proposal) (RP) (comp.) Tool: Scientific Peer Review Guidelines (Proposal) (RP) (comp.) Tool: Scientific Peer Review compilation (Proposal) (comp.) Tool: Funder Evaluation Form (Proposal) (comp.) Tool: Funder Evaluation Guidelines (Proposal) (VP) (comp.) Tool: Funder Evaluation Guidelines (Proposal) (RP) (comp.) Tool: Funder Evaluation compilation (Proposal) (comp.) Supporting document: Decision Matrix for Funding Recommendation on Research Proposals (VP) Tool: Letter of Confirmation (LoC) (VP) (comp.) Tool: Memorandum of Understanding (MoU) (RP) (comp.) Supporting document: Notes for production of a Research consortium agreement (VP) (comp.) Tool: Interim report (comp.) Tool: Final report (comp.) Supporting document: Network reporting requirements (comp. & NC) Tool: Scientific Peer Review Form (Report) (comp. & NC) Tool: Scientific Peer Review Guidelines (Report) (comp. & NC) Tool: Scientific Peer Review compilation (Report) (comp. & NC) Tool: Funder Evaluation Guidelines (Report) (comp.) Tool: Funder Evaluation Form (Report) (comp. & NC) Tool: Funder Evaluation compilation (Report) (comp.& NC) Tool: Request for amendments (Report) (comp. & NC) Tool: Final Letter of Discharge (comp. & NC) F - Research implementation - non-competitive (NC) mechanism Tool: Letter of Intent research providers (NC) Tool: Final report (NC) Many documents will be used in more than one chapter

EUPHRESCO Final Diss. Meeting, June 2010; Partner 4 (AGES, Austria) 34 THANK YOU FOR YOUR ATTENTION EUPHRESCO EUropean PHytosanitary REsearch COordination Special acknowledgement to Manuela Kienegger