Experiences of a Marie Curie Expert Evaluator Dr Sara Benetti Environmental Sciences Research Institute, University of Ulster

Vice-chair for FP7 Marie Curie IEF/IIF/IOF fellowship evaluation (2013) Expert evaluator for FP7 Marie Curie IEF/IIF/IOF fellowship evaluation (2011) Since 2011: expert evaluator for European Union FP7 programme “New and renewable sources of energy, Energy efficiency & innovation” IEF: Intra-European Fellowships; IIF: International Incoming Fellowships; IOF: International Outgoing Fellowships; CAR: Career Restart.

Panel: Project officer/s (EU person/s); Chair; Vice-chairs (normally 1 to 8-15 evaluators); Expert evaluators (lots and from all fields within the panel topic!).

Marie Curie fellowships: evaluation process (1) Candidates submit proposals to specific panel CHE, ECO, ENG, ENV, LIF, MAT, PHY, SOC Evaluators to declare any conflict of interest and select proposals (at least 100) to evaluate based on the abstract "1" denotes that the proposal is exactly in the expert's field of expertise; "2" denotes that the proposal is in the expert's broader field of expertise; "3" denotes that the proposal is not exactly in the expert's field of expertise s/he could evaluate it if necessary; "No" "No expertise" (by default this option is selected in all proposals).

Marie Curie fellowships: evaluation process (2) Depending on range of expertise, evaluators are selected and assigned up to 20 proposals to evaluate in a relatively short period (3-4 weeks) Each proposal has 3 evaluators; Evaluators will not all be expert in the specific field; They do not have contact with each others; They are not allowed to contact the applicant/s. Proposals are evaluated strictly in relation to the evaluation criteria; If proposal longer than allowed word count, anything over the word count will NOT be read or evaluated. REMOTE EVALUATION IAR

Marie Curie fellowships: evaluation process (2) REMOTE EVALUATION: scoring

Marie Curie fellowships: evaluation process (2) REMOTE EVALUATION: criteria IEF/IOF Overall threshold of 70%

Marie Curie fellowships: evaluation process (2) REMOTE EVALUATION: criteria

Marie Curie fellowships: evaluation process (2) REMOTE EVALUATION: criteria

Marie Curie fellowships: evaluation process (2) OUTREACH ACTIVITIES

Marie Curie fellowships: evaluation process (3) Rapporteur to write CONSENSUS REPORT Draft CR The rapporteur is one of the 3 evaluators of a proposal. He/she has the responsibility to: Collate a draft CR taking into account all of the 3 evaluators’ comments. Lead the consensus meeting (normally in Brussels). Write up the final CR and get it approved by other evaluators and proof-read by his/her vice- chair before final submission.

Marie Curie fellowships: evaluation process (4) Remotely or in Brussels, consensus meeting among 3 evaluators takes place. The 3 evaluators discuss the contents of the 3 IARs and draft CR; Comments in agreement are normally included in CR; Conflicting evaluations are discussed in more detail until (if) agreement among evaluators is reached; Marks for each criterion are then decided on the basis of the evaluations. CONSENSUS MEETING 2 nd draft of CR

Marie Curie fellowships: evaluation process (5) 2 nd draft of CR is checked and approved by all evaluators; Rapporteur’s vice-chair proof reads, checks for inconsistencies and match of comments to mark; If other evaluators do not approve the report, this process is reiterated until an agreement is reached on the final CR. If agreement evaluators is reached… Final CR submitted to EC

Marie Curie fellowships: evaluation process (5b) Second (third/fourth!) meeting may be scheduled to allow more time for discussion; Fourth evaluator may be selected, he/she will draft an independent IAR and join the following consensus meeting/s; A moderator may be brought in (vice-chair; chair; project officer. A consensus must be reached! If agreement among evaluators is not reached… Several iterations of CR until consensus is reached

Marie Curie fellowships: evaluation process (6) Those with an overall mark below threshold are out. Those with any mark below the threshold for specific criteria are out. Funding is allocated from highest scoring proposal going down, depending on funds (this changes from one year to the next). All proposals in panel are ranked Not a rule of thumb but any proposal below 85% is unlikely to get funded!

EU Marie Curie proposals 1.Good science; 2.Well explained and at the right level (evaluators may not be overly familiar with the science but they are scientists and they will have very limited time to read your proposal); 3.Do your homework: check criteria/requirements/etc. and stick to them; 4.Make sure you address every single evaluation criterion; 5.You must ‘prove’ everything! It is not enough that the host institution is ‘well-known’ for…how can you tell? 6.Outreach ‘points’ are easy to get if you make a little effort. Tips for success:

EU Marie Curie funding: the way forward 1.Name change (politics?); 2.Criteria will be reduced in number and simplified (to avoid repetition as much as possible); 3.It is possible more work will be done remotely (even harder for evaluators to discuss pros and cons of each proposal!); 4.Stick to the tips for success; 5.Level of funding may increase from one year to the next. Call for Marie Skłodowska-Curie Individual Fellowships (IF) H2020-MSCA-IF-2014 out in March 2014