A Focus on the remote evaluation phase 7th Framework Programme Cooperation Specific Programme Energy Work Programme PROPOSAL EVALUATION Experts Briefing A Focus on the remote evaluation phase FP7-ENERGY-2013-1

Energy Theme Overall objective To adapt the current fossil-fuel based energy system into a more sustainable one: less dependent on imported fuels, based on a diverse mix of energy sources and carriers, with particular attention being paid to lower and non-CO2 emitting energy technologies, combined with enhanced energy efficiency and conservation To address the pressing challenges of security of supply and climate change, whilst increasing the competitiveness of Europe’s industries. Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #2

Energy Theme Overview 2. Renewable electricity generation 10 “activities” implemented jointly by DG RTDI and ENER: FP7-ENERGY-2013-1 activities 2,3,4,5, 6,7 2. Renewable electricity generation 1. Hydrogen and fuel cells => JTI 3. Renewable fuel production 5. CO2 capture and storage technologies for zero emission power generation 7. Smart energy networks 6. Clean coal technologies 4. Renewables for heating and cooling 8. Energy savings and energy efficiency 9. Knowledge for energy policy making 10. Horizontal Programme Actions Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #3

Energy Theme Work Programme (WP) Legal basis for the WP is the Cooperation Specific Programme, which sets the overall scope, objectives and the subjects to be covered WP may be revised annually (as required) and constitutes the basis of annual Calls for Proposals WP Terminology: Theme (Energy) Activity (ten in the Energy Theme) Area (stable throughout FP7, but with flexibility) Topic (subjects open for proposals in the calls, only in the year in question) Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #4

International Cooperation Associated countries to the FP7 with an international agreement (or equivalent) and contribution to FP7 budget Switzerland, Norway, Iceland, Liechtenstein, Israel, Former Yugoslav Republic of Macedonia, Faroe Islands, Serbia, Croatia, Turkey, Albania, Montenegro, Bosnia & Herzegovina, Moldova The same rules as for MS are applied Third Countries All topics are open to the participation of International Cooperation Partners Countries (ICPC) and Industrialized countries; ICPC may receive EU funding in all cases; Industrialized countries USA, Canada, Australia, New Zealand, Japan, South Korea, etc.. – in principle no EC funding Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #5

Energy Theme 2013 Calls for Proposals This presentation covers evaluations of only the following calls R&I Energy call (FP7-ENERGY-2013-1) Other calls are : ENER Energy call (FP7-ENERGY-2013-2) R&I Energy call on International Research Programming (FP7-ENERY-2013-IRP) ENER Smart Cities and Communities cross thematic call (FP7-SMARTCITIES-2013) : Energy topics Also involved in Joint Calls: ERANET Call (FP7-ERANET-2013-RTD) OCEAN FOR TOMORROW (FP7-OCEAN-2013) Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #6

Overview of Calls FP7-ENERGY-2013 -1 (1/2) Publication date: 10 July 2012 One type of evaluation process - two funding schemes: Collaborative Projects (CPs) Collaborative projects for Specific International Actions (SICA /MPC) Single stage evaluation process: full proposals once Deadline:  28 November 2012 at 17.00.00 (Brussels local time) Budget: FP7-ENERGY-2013-1: EUR 107,5 million Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #7

Overview of Calls FP7-ENERGY-2013 -1 (2/2) FP7-ENERGY-2013-1 /18 topics in different areas: Renewable Electricity generation (Photovoltaics (1), Wind(2), Geothermal (1), Ocean (1), Cross-Cutting issues (2)) and Renewables for Heating and Cooling (Low/Medium Temperature Solar Thermal (1) ) : EUR 51 million – 8 topics Renewable Fuel production (Cross-Cutting issues): EUR 4 million – 1 topic CO2 capture & storage: (Capture (2), Storage (1)) and Clean Coal Technologies ( Conversion technologies for Power (1)): EUR 28,5 million – 4 topics Renewable Electricity generation (Hydro (1)) and Smart Energy networks (Pan-European Energy Networks (3)) and Cross-Cutting (1): EUR 24 million – 5 topics Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #8

Eligibility Checks (Annex 2 of the Work Programme Receipt before deadline Minimum number of partners Collaborative Projects: At least 3 independent entities from MS or AC, no 2 of these in the same MS or AC (CP) Collaborative Projects for specific international cooperation (SICA): At least 4 independent entities, 2 from different MS or AC, the other 2 from different international cooperation partner countries based on Part A of the proposal only! Completeness of proposal (Part A and Part B) “Out of scope” : a proposal will only be deemed ineligible in clear cut case and duly explained Note: Normally already been checked by the Commission However a proposal can be declared ineligible by the experts throughout the evaluation process, notably regarding its conformance to the scope of the topic Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #9

Additional Eligibility Criteria Budget thresholds Topic ENERGY.2013.2.9.2: Methods for the estimation of the Direct Normal Irradiation (DNI): Requested EU contribution per project shall not exceed EUR 3 million Topic ENERGY.2013.5.1.1: Scale-up of advanced high-efficiency capture process: Requested EU contribution per project shall not exceed EUR 8 million Based on Part A of the proposal only! Basis for eligibility is the Energy Theme Work Programme and the Rules for participation including third countries (INCO) Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #10

Evaluating a proposal Fundamental Principles Objectivity, Fairness and Impartiality Each proposal is evaluated as it is written. However, statement of achievements not to be taken for granted Excellence and Accuracy You make your judgment against the state-of-the-art in the field applying the official evaluation criteria, and nothing else Consistency and Equal treatment You apply the same standard of judgment to each proposal Efficiency and speed Confidentiality Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #11

Evaluation Criterion 1 S&T Quality Scientific and/or technological excellence (relevant to the topics addressed by the call) sub-Criteria: all to be assessed and commented! Soundness of concept, and quality of objectives Progress beyond the state-of-the-art Quality and effectiveness of the S/T methodology and associated work plan Threshold : 3/5 Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #12

Evaluation Criterion 2 Implementation Quality and efficiency of the implementation and the management sub-Criteria: all to be assessed and commented! Appropriateness of the management structure and procedures Quality and relevant experience of the individual participants Quality of the consortium as a whole (including complementarity, balance) Appropriateness of the allocation and justification of the resources to be committed (staff, equipment..) Threshold: 3/5 Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #13

Evaluation Criterion 3 Impact (1/2) Potential impact through the development, dissemination and use of projects results sub-Criteria: all to be assessed and commented! Contribution, at the European (and/or international) level, to the expected impacts listed in the work programme under the relevant topic/activity Appropriateness of measures for the dissemination and/or exploitation of project results, and management of intellectual property Threshold : 3/5 Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #14

Evaluation Criterion 3 Impact (2/2) Innovation impacts Regarding sub-criterion a: Potential areas and markets of application of the project results and the potential advantages of the resulting technologies/ solutions compared to those that are available today Regarding sub-criterion b: Measures to increase the likelihood of market uptake of project results, such as: verification, testing, and prototyping; supporting the development of technical standards; identifying and collaborating with potential users; identifying potential partners and sources of finance for commercialisation Note: Both sub-criteria and innovation impacts should be well elaborated on in the proposal. All relevant aspects should be elaborated on in the proposal, and in such a manner that evaluators are fully able to make a judgment on the expected impact based on the actual proposal workplan Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #15

Proposal Marking When assigning a mark to a criterion Take account of all sub-criteria and ensure that failure to meet any of the sub-criteria is reflected in the overall criterion mark Use full marking range: 0 to 5, including half-marks Corresponding comments should be concise, should outline strengths and weaknesses and should be consistent with the criterion mark Thresholds apply to individual criteria …and to total score Overall Threshold: 10/15 A proposal will not be considered for funding if it fails to meet any of the individual criteria thresholds and/or the overall threshold Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #16

Marking Range Interpretation 0 = The proposal fails to address the criterion under examination or cannot be judged due to missing or incomplete information. 1 = Poor. The criterion is addressed in an inadequate manner, or there are serious inherent weaknesses 2 = Fair. While the proposal broadly addresses the criterion, there are significant weaknesses. 3 = Good. The proposal addresses the criterion well, although improvements would be necessary 4 = Very good. The proposal addresses the criterion very well, although certain improvements are still possible 5 = Excellent. The proposal successfully addresses all relevant aspects of the criterion in question. Any shortcomings are minor Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #17

EC staff / Moderator Responsibilities Ensure the Evaluation Process is carried out in accordance with established procedures and manage the entire evaluation process Check proposals for eligibility, identify possible transfers Organise & oversee work of experts Assign proposals to experts Assist experts in their work Ensure evaluation criteria are understood and applied the same way by the different evaluation groups Ensure implementation of fundamental principles Accurately record evaluation results Ensure all experts give their views Maintain the documentation for the “audit trail” Do not attempt to influence the evaluation results Do not express opinions on proposals or applicants Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #18

Role of Chairs and Vice-chairs To assist the Commission with the management of the evaluation process. Their tasks will include the following: Assisting in overseeing the entire evaluation process, notably ensuring the quality and consistency of the Consensus Reports and Evaluation Summary Reports Chairing/co-chairing the panel meetings (this includes the preparation and reporting) Do not attempt to influence the evaluation results Do not express opinions on proposals or applicants Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #19

Role of Rapporteurs One of the experts having evaluated the proposal Guide the scientific discussion and seek for consensus amongst individual evaluators Produce Consensus Report If consensus not achieved, record majority/minority view and confer with Moderator Attend the panel meeting Rapporteurs will be assigned in RIVET only at the time of the consensus meetings Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #20

Role of Independent observers Provide assurance that the process is fair and can provide constructive advice Not normally experts in the scientific domain concerned Do not take part in the actual evaluation of proposals Their reports are made available to Programme Committee(s) Independent Observers from all Calls take part in a joint annual meeting to review evaluation procedures Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #21

Evaluators Responsibilities Follow the “Code of Conduct for independent experts appointed as evaluators”, notably Declare conflicts of interest (Precise conditions, as spelled out in the appointment letter -You must inform your Panel Moderator immediately) Provide independent, impartial and objective advice to the Commission Represent your knowledge and skills, not your employer, nor your country Confidentiality: no discussions about any proposal except in consensus meeting / final panel no external contacts during or after the evaluation the EC does not reveal your individual opinions (the EC just publishes evaluator names annually) Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #22

«Disqualifying» Conflict of Interest You must inform your Panel Moderator immediately Precise conditions, as spelled out in the appointment letter: Involved in preparation of proposal Stands to benefit directly Close family relationship Director/trustee/partner Employee (but, possible exception…) Member of Advisory Group (and Commitology Committees) Any other situation that compromises impartiality You cannot evaluate the concerned proposal or any other proposal in the same call and hence neither participate in the consensus group… nor in the final panel Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #23

Potential Conflict of Interest You must inform your Panel Moderator immediately Precise conditions, as spelled out in the appointment letter: Employed in last 3 years Involved in contract or research collaboration in previous 3 years Any other situation that casts doubt…or that could reasonably appear to do so… The Commission will consider circumstances of case to see if whether or not there is an effective Conflict of Interest Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #24

Individual reading You first carry out readings on your own: Evaluate the proposal individually Check whether the proposal is ‘in scope’ (relevant to topic) Complete using RiVet an Individual Evaluation Report Form (IER) giving scores and precise but concise comments on each of the criteria and overall Scores should be in line with comments If in any doubt, consult your moderator Sign and date the form in RiVet (electronically by finalising it) IERs will be checked by the moderator and, if necessary, returned with a request to further justify the score given or to clarify comments Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #25

Quality of Consensus Reports (CR) (1/2) Built on the basis of the comments of all individual evaluators having examined the same proposal, without knowledge of the individual marks One expert acts as rapporteur to prepare the Consensus Report (CR), but this is a collective exercise Involves a discussion moderated by a Commission staff-member who impartially provides necessary information but does not contribute with opinions Agreement on consensus comments and marks for all criteria Not just a simple averaging exercise Consensus should be actively reached However, a minority viewpoint can be recorded alongside the majority position Signing of the final CR in RiVeT Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #26

Quality of Consensus Reports (CR) (2/2) The quality of the CR is paramount, make sure: it contains marks and comments for each relevant criteria Comments address both the strengths and weaknesses Comments are clear, concise and consistent with marks CR should not contain: Recommendations in view of re-submission Summary of the proposal Proof reader for quality control (EC staff , (Vice-)-Chairs) Memo note for rapporteurs and checklist for moderators available Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #27

Writing Tips of Consensus Reports (CR) (1/3) Comments should be written for ALL sub-criteria Poor comments merely address the score Good comments explain it Examples: This proposal does not adequately describe a proper S/T methodology This proposal fails to address different steps of the S/T methodology regarding X or Y, it does not take Z into account Poor comments are ambiguous Good comments are clear The allocated resources are unrealistic The resources allocated to Work packages X and Z are seriously underestimated given the complexity of the work described Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #28

Writing Tips of Consensus Reports (CR) (2/3) Comments should be written for ALL sub-criteria Poor comments are vague Good comments are precise and final Examples: The management plan seems inadequate given the project duration and the number of partners involved The management plan is inadequate. It does not include clear overall responsibility for the administrative tasks. Furthermore it does not establish a risk assessment process and/or a mitigation procedure. It does not foresee a decision making mechanism nor a problem-solving procedure in case of disputes between partners. In addition, it does not bring in the expertise in X needed to carry out the tasks referred at Work package Y Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #29

Writing Tips of Consensus Reports (CR) (3/3) Comments should be written for ALL sub-criteria Poor comments provide the opening to complaints (redress procedure) Good comments provide final answers to the question Examples: There is no mention of dissemination activities Dissemination activities are not adequately described and substantiated by a strategic plan There is no mention of Intellectual Property Intellectual Property is not adequately described and there is no plan to manage it The coordinator is not adequately experienced The coordinator does not demonstrate in the proposal an adequate level of expertise in this field / or as a coordinator Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #30

Panel meeting Quality control” on the evaluation process Attended by all rapporteurs Review Consensus Reports Resolve any cases where a minority view was recorded Review comments and marks for consistency Confirm final marks and comments for each proposal Any adjustment of scores (if necessary) should be carefully justified Recommend list(s) of proposals in order of priority The single stage evaluation is about “ranking” decision Prepare and sign Evaluation Summary Reports (ESRs) – each rapporteur Prepare and sign – collectively - the Panel Report Comment on evaluation process Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #31

Evaluation Summary Report (ESR) As a general rule, ESR = CR Changes from CR should be exceptional and duly justified ESR must be of high quality as it is the only feedback given to proposers following evaluation (minimize the use of redress procedure) The process is important… …but it’s the outcome that counts! Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #32

Thank you for your attention ! Note: Document not legally binding 2013 Energy Calls for proposals – Expert Briefing (January 2013) #33