APRE Agency for the Promotion of European Research Evaluation and submission procedures in Seventh Framework Programme Sabrina Bozzoli

Overview Information for proposers Writing your proposal Getting help Submission Selection Evaluation Eligible? Information for proposers Writing your proposal Getting help Submission of proposal Eligibility checks Evaluation (experts) Selection Poznan, 23rd January 2007

Information for proposers Call fiche Workprogramme 2007-2008 Guide for Applicants (now including the Guidance notes for evaluators, the evaluation criteria and the Background note on the funding scheme) Model grant agreement Rules for participation Rules on submission of proposals, and the related evaluation, selection and award procedures EPSS manual Poznan, 23rd January 2007

When writing your proposal…(1) Divide your efforts over the evaluation criteria [Many proposers concentrate on the scientific element, but loose marks on project implementation or impact description] a strong title, acronym an interesting project summary (objectives, results, R&D-approach, partnership, utility of results, exploitation) convincing technology background and state of the art clear objectives, methods, results and deliverables well designed work plan appropriate management structures & procedures detailed implementation & exploitation of results realistic costs that lie within the budget of the Call convincing consortium (roles, qualifications) keep it clear and simple without loosing quality The proposal is not a scientigic paper!!! Poznan, 23rd January 2007

When writing your proposal…(2) Think of the finishing touches which contribute to the quality of work Use coincise standard English Make text clear, well structured, easy to read Add a table of contents, use short paragraphs, highlight key points in italics, use bullet points to break up lists Include only relevant information Make your proposal virtually attractive and inviting using graphic devices Poznan, 23rd January 2007

Getting help with your proposal www.cordis.europa.eu/fp7/ + National Contact Points practical information & advice on modalities and rules National delegates in Programme Committees background information on EU policies Innovation Relay Centre (IRC) EC Information desks (reachable by e-mail or phone. eg. IPR Help-desk) Get early advice from NCP and EC Information Desk Show your proposal to critical (experienced) collegues Inform NCP and National delegates in Programme Committees Submit complete proposal before deadline Poznan, 23rd January 2007

Submission Fixed deadline calls Electronic proposal submission system (EPSS) only Proposal template given in the ‘Guide for Applicants’ Closely aligned to the evaluation criteria Proposals are normally submitted and evaluated in a single stage Two-stage submission of proposals May be used for large, ‘bottom up’ calls First stage short proposal (about 10-20 pages), dealing with main scientific concepts and ideas use of limited set of criteria successful proposers invited to submit complete proposals Poznan, 23rd January 2007

Electronic submission EPSS [Electronic Proposal Submission System] Online preparation only! Improved validation checks before submission is accepted FP6 Failure rate = + 1% Main reason for failure - waiting till the last minute Submit early, submit often! Proposals must be submitted electronically, using the Commission's Electronic ProposalSubmission Service (EPSS) Proposals arriving at the Commission by any other means are regarded as ‘not submitted’, and will not be evaluated. All the data that you upload is securely stored on a server to which only you and the other participants in the proposal have access until the deadline. This data is encrypted until the close of the call. You can access the EPSS from the call page on CORDIS. Full instructions will be found in the “EPSS preparation and submission guide”. This will be available from the CORDIS site early in 2007. Poznan, 23rd January 2007

Proposal Part A A1 A2 A3 Title, acronym, objective etc. Free keywords 2000 character proposal abstract Previous/current submission (in FP7) A2 Legal address/administrator address/R&D address Clear identification as SME/Public body/Research centre/ Higher and Secondary education establishment A3 More cost details (direct/indirect costs distinguished) Section A1 gives a snapshot of your proposal, section A2 concerns you and your organisation, while section A3 deals with money matters. Please note: • The coordinator fills in the section A1and section A3. • The participants already identified at the time of proposal submission (including the coordinator) each fill in section A2. • Subcontractors are not required to fill in section A2 and should not be listed separately in section A3. • The estimated budget planned for any future participants (not yet identified at the time of the proposal) is not shown separately in form A3 but should be added to the coordinator’s budget. Their role, profile and tasks are described in Part B of the proposal. Poznan, 23rd January 2007

Proposal Part B (pdf format only) Part B format directly linked to evaluation criteria Summary S&T quality (bullet points = sections) Implementation (idem) Impact (idem) Ethics Section lengths recommended Details of the work you intend to carry out will be described in part B Poznan, 23rd January 2007

Elegibility checks Date and time of receipt of proposal on or before deadline Firm deadlines - except for Continuously Open Calls Minimum number of eligible, independent partners As set out in work programme/call Completeness of proposal Presence of all requested administrative forms (Part A) and the content description (Part B) “Out of scope” Others (eg. budget limits) It is possible that a proposal is found to be completely out of scope of the call during the course of the individual evaluation, and therefore not relevant. If an expert suspects that this may be the case, a Commission staff member will be informed immediately, and the views of the other experts will be sought. If the consensus view is that the main part of the proposal is not relevant to the topics of the call, the proposal will be withdrawn from the evaluation, and the proposal will be deemed ineligible. Poznan, 23rd January 2007

Evaluation - FP7 No major change for FP7 But improved and streamlined, based on experience Adapted to the new features of FP7 where necessary What’s new? Clearer page limits Elegibility criteria (includes “scope”) Evaluation criteria (3 instead of 5 or 6) More clarity on conflicts of interest (Indipendent experts) WHAT’S NEW? Any page going behind the maximum indicated by the EC will be disregarded. An extra eligibility criterion regarding the “scope of the project” The clarity of conflict of interest will be assured through the appointment of indipendent experts. Conflicts of interest: Under the terms of the appointment letter, experts must declare beforehand any known conflicts of interest, and must immediately inform a Commission staff member if one becomes apparent during the course of the evaluation. The Commission will take whatever action is necessary to remove any conflict. Poznan, 23rd January 2007

Evaluation – The criteria Adapted to each funding scheme and each thematic area Specified in the specific programme/work programme (Annex 2) Divided into three main criteria: S&T Quality (relevant to the topics addressed by the call) quality of the objectives progress beyond the state of the art work plan Implementation individual participants and consortium as a whole allocation of resources (budget, staff, equipment) Impact Contribution to expected impacts listed in work programme Plans for dissemination/exploitation Criteria generally marked out of 5 Criterion threshold 3/5 Overall threshold 10/15 S/T QUALITY= “Scientific and/or technological excellence (relevant to the topics addressed by the call)” Soundness of concept, and quality of objectives • Progress beyond the state-of the-art • Quality and effectiveness of the S/T methodology and associated work plan IMPLEMENTATION “Quality and efficiency of the implementation and the management” Appropriateness of the management structure and procedures • Quality and relevant experience of the individual participants • Quality of the consortium as a whole (including complementarity, balance) • Appropriateness of the allocation and justification of the resources to be committed (budget, staff, equipment) IMPACT=“Potential impact through the development, dissemination and use of project results” Contribution, at the European [and/or international] level, to the expected impacts listed in the work programme under the relevant topic/activity • Appropriateness of measures for the dissemination and/or exploitation of project results, and management of intellectual property. The relevance of a proposal will be considered in relation to the topic(s) of the work programme open in a given call, and to the objectives of a call. These aspects will be integrated in the application of the criterion "S/T quality", and the first sub-criterion under "Impact" respectively. When a proposal is partially relevant because it only marginally addresses the topic(s) of the call, or if only part of the proposal addresses the topic(s), this condition will be reflected in the scoring of the first criterion. Proposals that are clearly not relevant to a call ("out of scope") will be rejected on eligibility grounds. SCORES= for each criterion the maximum is 5, so in total it’s 15 Poznan, 23rd January 2007

Evaluation - The experts (1) The Commission draws on a wide pool of evaluators about 50.000 in FP6 Call for candidates published on December 2006 Call for applications are addressed to individuals/organisations Applications via CORDIS FP6 experts have been invited to transfer to FP7 (with a request to update their information) Commission invites individuals on a call-by-call basis Not self-selection! Expertise, and experience are paramount Geography, gender and “rotation” is also considered Poznan, 23rd January 2007

Evaluation - The experts (2) Experts agree to terms and conditions of an “appointment letter” Typically an individual will review 6-8 proposals “remotely”… …then spend a couple of days in Brussels Some will participate in “hearings” with the consortia Experts sign confidentiality and conflict of interest declaration Names published after the evaluations Poznan, 23rd January 2007

Selection procedure Poznan, 23rd January 2007 PROPOSAL Eligibility Individual evaluation Consensus Security Scrutiny (if needed) Thresholds Applicants informed of results of expert evaluation Panel review with hearings (optional) Ethical Review (if needed) Commission ranking Negotiation Commission rejection decision Applicants informed of Commission decision Consultation of programme committee (if required) Commission funding and/or rejection decision Poznan, 23rd January 2007

1. From Individual assessment to Consensus May be “remote” Proposal X Copy 1 IAR* Expert 1 CONSENSUS REPORT 3 experts Proposal X Copy 2 IAR Expert 2 Consensus meeting Each proposal will first be assessed independently by at least three experts, chosen by the Commission from the pool of experts taking part in this evaluation (Of course, there may be more than three evaluators for each proposal). At this first step the experts are acting individually; they do not discuss the proposal with each other, nor with any third party. The experts record their individual opinions in an Individual Assessment Report (IAR), giving scores and also comments against the evaluation criteria. Experts will assess and mark the proposal exactly as it is described and presented. They do not make any assumptions or interpretations about the project in addition to what is in the proposal. Concise but explicit justifications will be given for each score. Recommendations for improvements to be discussed as part of a possible negotiation phase will be given, if needed. The experts will also indicate whether, in their view, the proposal deals with sensitive ethical issues, or if it requires further scrutiny with regard to security considerations. Signature of the IAR also entails a declaration that the expert has no conflict of interest in evaluating the particular proposal. Proposal X Copy 3 IAR Expert 3 *IAR= Individual assessment report Poznan, 23rd January 2007

2. Consensus Built on the basis of the individual assessments of all the evaluators Usually involves a discussion Moderated by a Commission representative One expert acts as a rapporteur Agreement on consensus marks and comments for each of the criteria Once all the experts to whom a proposal has been assigned have completed their IAR, the evaluation progresses to a consensus assessment, representing their common views. This entails a consensus meeting to discuss the scores awarded and to prepare comments. The consensus discussion is moderated by a representative of the Commission. The role of the moderator is to seek to arrive at a consensus between the individual views of experts without any prejudice for or against particular proposals or the organisations involved, and to ensure a confidential, fair and equitable evaluation of each proposal according to the required evaluation criteria. The moderator for the group may designate an expert to be responsible for drafting the consensus report ("rapporteur"). The experts attempt to agree on a consensus score for each of the criteria that have been evaluated and suitable comments to justify the scores. Comments should be suitable for feedback to the proposal coordinator. Scores and comments are set out in a consensus report. They also come to a common view on the questions of scope, ethics and security. If during the consensus discussion it is found to be impossible to bring all the experts to a common point of view on any particular aspect of the proposal, the Commission may ask up to three additional experts to examine the proposal. The outcome of the consensus step is the consensus report. This will be signed (either on paper, or electronically) by all experts, or as a minimum, by the rapporteur and the moderator. The moderator is responsible for ensuring that the consensus report reflects the consensus reached, expressed in scores and comments. In the case that it is impossible to reach a consensus, the report sets out the majority view of the experts but also records any dissenting views. Poznan, 23rd January 2007

3. Panel review Panel Meeting Compare consensus reports Examines proposals with same consensus score (if needed) Final marks and comments for each proposal Suggestions on order of priority, clustering, amendments, etc. Hearings with proposers may be convened Questions to the invited proposal coordinators Small number of proposal representatives PANEL REVIEW= The main task of the panel is to examine and compare the consensus reports in a given area, to check on the consistency of the marks applied during the consensus discussions and, where necessary, propose a new set of consensus scores. The panel is chaired by a representative of the Commission. The outcome of the panel meeting is a report recording, principally: • An evaluation summary report (ESR) for each proposal, including, where relevant, a report of any ethical issues raised and any security considerations; • A list of proposals passing all thresholds, along with a final score for each proposal passing the thresholds and the panel recommendations for priority order. • A list of evaluated proposals having failed one or more thresholds; • A list of any proposals having been found ineligible during the evaluation by experts; • A summary of any deliberations of the panel; • A record of the hearings (where applicable) HEARINGS= Applicants whose proposals have been favourably evaluated are sometimes invited to Brussels to answer any specific questions raised by the experts. Poznan, 23rd January 2007

4. Commission Follow-up Evaluation summary reports (ESR) sent to applicants (“Initial information letter”) Draw up final ranking lists Information to the Programme Committee Commission decisions on rejected proposals Contract negotiation Formal consultation of Programme Committee (when required) Commission decisions on proposals selected for funding Survey of evaluators INITIAL INFORMATION LETTER=EVALUATION SUMMARY REPORT Poznan, 23rd January 2007

Thank you for your attention Thank you for your attention! Sabrina Bozzoli APRE +39 06 5911817 bozzoli@apre.it Poznan, 23rd January 2007