1. A Focus on the remote evaluation phase
7th Framework Programme Cooperation Specific Programme Energy Work Programme
PROPOSAL EVALUATION
Experts Briefing
A Focus on the remote evaluation phase
FP7-ENERGY

2. Energy Theme Overall objective
To adapt the current fossil-fuel based energy system into a more sustainable one:
less dependent on imported fuels,
based on a diverse mix of energy sources and carriers, with particular attention being paid to
lower and non-CO2 emitting energy technologies, combined with enhanced energy efficiency and conservation
To address the pressing challenges of security of supply and climate change, whilst increasing the competitiveness of Europe’s industries.
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #2

3. Energy Theme Overview 2. Renewable electricity generation
10 “activities” implemented jointly by DG RTDI and ENER: FP7-ENERGY activities 2,3,4,5, 6,7
2. Renewable
electricity
generation
1. Hydrogen and fuel cells => JTI
3. Renewable fuel production
5. CO2 capture and storage technologies for zero emission power generation
7. Smart energy networks
6. Clean coal technologies
4. Renewables for heating and cooling
8. Energy savings and energy efficiency
9. Knowledge for energy policy making
10. Horizontal Programme Actions
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #3

4. Energy Theme Work Programme (WP)
Legal basis for the WP is the Cooperation Specific Programme, which sets the overall scope, objectives and the subjects to be covered
WP may be revised annually (as required) and constitutes the basis of annual Calls for Proposals
WP Terminology:
Theme (Energy)
Activity (ten in the Energy Theme)
Area (stable throughout FP7, but with flexibility)
Topic (subjects open for proposals in the calls, only in the year in question)
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #4

5. International Cooperation
Associated countries to the FP7
with an international agreement (or equivalent) and contribution to FP7 budget
Switzerland, Norway, Iceland, Liechtenstein, Israel, Former Yugoslav Republic of Macedonia, Faroe Islands, Serbia, Croatia, Turkey, Albania, Montenegro, Bosnia & Herzegovina, Moldova
The same rules as for MS are applied
Third Countries
All topics are open to the participation of International Cooperation Partners Countries (ICPC) and Industrialized countries;
ICPC may receive EU funding in all cases;
Industrialized countries USA, Canada, Australia, New Zealand, Japan, South Korea, etc.. – in principle no EC funding
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #5

6. Energy Theme 2013 Calls for Proposals
This presentation covers evaluations of only the following calls
R&I Energy call (FP7-ENERGY )
Other calls are :
ENER Energy call (FP7-ENERGY )
R&I Energy call on International Research Programming (FP7-ENERY-2013-IRP)
ENER Smart Cities and Communities cross thematic call (FP7-SMARTCITIES-2013) : Energy topics
Also involved in Joint Calls:
ERANET Call (FP7-ERANET-2013-RTD)
OCEAN FOR TOMORROW (FP7-OCEAN-2013)
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #6

7. Overview of Calls FP7-ENERGY-2013 -1 (1/2)
Publication date: 10 July 2012
One type of evaluation process - two funding schemes:
Collaborative Projects (CPs)
Collaborative projects for Specific International Actions (SICA /MPC)
Single stage evaluation process: full proposals once
Deadline: 
28 November 2012 at (Brussels local time)
Budget:
FP7-ENERGY : EUR 107,5 million
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #7

8. Overview of Calls FP7-ENERGY-2013 -1 (2/2)
FP7-ENERGY /18 topics in different areas:
Renewable Electricity generation (Photovoltaics (1), Wind(2), Geothermal (1), Ocean (1), Cross-Cutting issues (2)) and Renewables for Heating and Cooling (Low/Medium Temperature Solar Thermal (1) ) : EUR 51 million – 8 topics
Renewable Fuel production (Cross-Cutting issues): EUR 4 million – 1 topic
CO2 capture & storage: (Capture (2), Storage (1)) and Clean Coal Technologies ( Conversion technologies for Power (1)): EUR 28,5 million – 4 topics
Renewable Electricity generation (Hydro (1)) and Smart Energy networks (Pan-European Energy Networks (3)) and Cross-Cutting (1): EUR 24 million – 5 topics
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #8

9. Eligibility Checks (Annex 2 of the Work Programme
Receipt before deadline
Minimum number of partners
Collaborative Projects: At least 3 independent entities from MS or AC, no 2 of these in the same MS or AC (CP)
Collaborative Projects for specific international cooperation (SICA): At least 4 independent entities, 2 from different MS or AC, the other 2 from different international cooperation partner countries
based on Part A of the proposal only!
Completeness of proposal (Part A and Part B)
“Out of scope” : a proposal will only be deemed ineligible in clear cut case and duly explained
Note: Normally already been checked by the Commission
However a proposal can be declared ineligible by the experts throughout the evaluation process, notably regarding its conformance to the scope of the topic
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #9

10. Additional Eligibility Criteria
Budget thresholds
Topic ENERGY : Methods for the estimation of the Direct Normal Irradiation (DNI): Requested EU contribution per project shall not exceed EUR 3 million
Topic ENERGY : Scale-up of advanced high-efficiency capture process: Requested EU contribution per project shall not exceed EUR 8 million
Based on Part A of the proposal only!
Basis for eligibility is the Energy Theme Work Programme and the Rules for participation including third countries (INCO)
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #10

11. Evaluating a proposal Fundamental Principles
Objectivity, Fairness and Impartiality
Each proposal is evaluated as it is written. However, statement of achievements not to be taken for granted
Excellence and Accuracy
You make your judgment against the state-of-the-art in the field applying the official evaluation criteria, and nothing else
Consistency and Equal treatment
You apply the same standard of judgment to each proposal
Efficiency and speed
Confidentiality
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #11

12. Evaluation Criterion 1 S&T Quality
Scientific and/or technological excellence (relevant to the topics addressed by the call)
sub-Criteria: all to be assessed and commented!
Soundness of concept, and quality of objectives
Progress beyond the state-of-the-art
Quality and effectiveness of the S/T methodology and associated work plan
Threshold : 3/5
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #12

13. Evaluation Criterion 2 Implementation
Quality and efficiency of the implementation and the management
sub-Criteria: all to be assessed and commented!
Appropriateness of the management structure and procedures
Quality and relevant experience of the individual participants
Quality of the consortium as a whole (including complementarity, balance)
Appropriateness of the allocation and justification of the resources to be committed (staff, equipment..)
Threshold: 3/5
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #13

14. Evaluation Criterion 3 Impact (1/2)
Potential impact through the development, dissemination and use of projects results
sub-Criteria: all to be assessed and commented!
Contribution, at the European (and/or international) level, to the expected impacts listed in the work programme under the relevant topic/activity
Appropriateness of measures for the dissemination and/or exploitation of project results, and management of intellectual property
Threshold : 3/5
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #14

15. Evaluation Criterion 3 Impact (2/2)
Innovation impacts
Regarding sub-criterion a: Potential areas and markets of application of the project results and the potential advantages of the resulting technologies/ solutions compared to those that are available today
Regarding sub-criterion b: Measures to increase the likelihood of market uptake of project results, such as: verification, testing, and prototyping; supporting the development of technical standards; identifying and collaborating with potential users; identifying potential partners and sources of finance for commercialisation
Note:
Both sub-criteria and innovation impacts should be well elaborated on in the proposal. All relevant aspects should be elaborated on in the proposal, and in such a manner that evaluators are fully able to make a judgment on the expected impact based on the actual proposal workplan
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #15

16. Proposal Marking When assigning a mark to a criterion
Take account of all sub-criteria and ensure that failure to meet any of the sub-criteria is reflected in the overall criterion mark
Use full marking range: 0 to 5, including half-marks
Corresponding comments
should be concise, should outline strengths and weaknesses and
should be consistent with the criterion mark
Thresholds apply to individual criteria …and to total score
Overall Threshold: 10/15
A proposal will not be considered for funding if it fails to meet any of the individual criteria thresholds and/or the overall threshold
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #16

17. Marking Range Interpretation
0 = The proposal fails to address the criterion under examination or cannot be judged due to missing or incomplete information.
1 = Poor. The criterion is addressed in an inadequate manner, or there are serious inherent weaknesses
2 = Fair. While the proposal broadly addresses the criterion, there are significant weaknesses.
3 = Good. The proposal addresses the criterion well, although improvements would be necessary
4 = Very good. The proposal addresses the criterion very well, although certain improvements are still possible
5 = Excellent. The proposal successfully addresses all relevant aspects of the criterion in question. Any shortcomings are minor
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #17

18. EC staff / Moderator Responsibilities
Ensure the Evaluation Process is carried out in accordance with established procedures and manage the entire evaluation process
Check proposals for eligibility, identify possible transfers
Organise & oversee work of experts
Assign proposals to experts
Assist experts in their work
Ensure evaluation criteria are understood and applied the same way by the different evaluation groups
Ensure implementation of fundamental principles
Accurately record evaluation results
Ensure all experts give their views
Maintain the documentation for the “audit trail”
Do not attempt to influence the evaluation results
Do not express opinions on proposals or applicants
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #18

19. Role of Chairs and Vice-chairs
To assist the Commission with the management of the evaluation process. Their tasks will include the following:
Assisting in overseeing the entire evaluation process, notably ensuring the quality and consistency of the Consensus Reports and Evaluation Summary Reports
Chairing/co-chairing the panel meetings (this includes the preparation and reporting)
Do not attempt to influence the evaluation results
Do not express opinions on proposals or applicants
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #19

20. Role of Rapporteurs One of the experts having evaluated the proposal
Guide the scientific discussion and seek for consensus amongst individual evaluators
Produce Consensus Report
If consensus not achieved, record majority/minority view and confer with Moderator
Attend the panel meeting
Rapporteurs will be assigned in RIVET only at the time of the consensus meetings
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #20

21. Role of Independent observers
Provide assurance that the process is fair and can provide constructive advice
Not normally experts in the scientific domain concerned
Do not take part in the actual evaluation of proposals
Their reports are made available to Programme Committee(s)
Independent Observers from all Calls take part in a joint annual meeting to review evaluation procedures
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #21

22. Evaluators Responsibilities
Follow the “Code of Conduct for independent experts appointed as evaluators”, notably
Declare conflicts of interest (Precise conditions, as spelled out in the appointment letter -You must inform your Panel Moderator immediately)
Provide independent, impartial and objective advice to the Commission
Represent your knowledge and skills, not your employer, nor your country
Confidentiality:
no discussions about any proposal except in consensus meeting / final panel
no external contacts during or after the evaluation
the EC does not reveal your individual opinions (the EC just publishes evaluator names annually)
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #22

23. «Disqualifying» Conflict of Interest You must inform your Panel Moderator immediately
Precise conditions, as spelled out in the appointment letter:
Involved in preparation of proposal
Stands to benefit directly
Close family relationship
Director/trustee/partner
Employee (but, possible exception…)
Member of Advisory Group (and Commitology Committees)
Any other situation that compromises impartiality
You cannot evaluate the concerned proposal or any other proposal in the same call and hence
neither participate in the consensus group…
nor in the final panel
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #23

24. Potential Conflict of Interest You must inform your Panel Moderator immediately
Precise conditions, as spelled out in the appointment letter:
Employed in last 3 years
Involved in contract or research collaboration in previous 3 years
Any other situation that casts doubt…or that could reasonably appear to do so…
The Commission will consider circumstances of case to see if whether or not there is an effective Conflict of Interest
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #24

25. Individual reading You first carry out readings on your own:
Evaluate the proposal individually
Check whether the proposal is ‘in scope’ (relevant to topic)
Complete using RiVet an Individual Evaluation Report Form (IER)
giving scores and precise but concise comments on each of the criteria and overall
Scores should be in line with comments
If in any doubt, consult your moderator
Sign and date the form in RiVet (electronically by finalising it)
IERs will be checked by the moderator and, if necessary, returned with a request to further justify the score given or to clarify comments
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #25

26. Quality of Consensus Reports (CR) (1/2)
Built on the basis of the comments of all individual evaluators having examined the same proposal, without knowledge of the individual marks
One expert acts as rapporteur to prepare the Consensus Report (CR), but this is a collective exercise
Involves a discussion moderated by a Commission staff-member who impartially provides necessary information but does not contribute with opinions
Agreement on consensus comments and marks for all criteria
Not just a simple averaging exercise
Consensus should be actively reached
However, a minority viewpoint can be recorded alongside the majority position
Signing of the final CR in RiVeT
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #26

27. Quality of Consensus Reports (CR) (2/2)
The quality of the CR is paramount, make sure:
it contains marks and comments for each relevant criteria
Comments address both the strengths and weaknesses
Comments are clear, concise and consistent with marks
CR should not contain:
Recommendations in view of re-submission
Summary of the proposal
Proof reader for quality control (EC staff , (Vice-)-Chairs)
Memo note for rapporteurs and checklist for moderators available
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #27

28. Writing Tips of Consensus Reports (CR) (1/3)
Comments should be written for ALL sub-criteria
Poor comments merely address the score
Good comments explain it
Examples:
This proposal does not adequately describe a proper S/T methodology
This proposal fails to address different steps of the S/T methodology regarding X or Y, it does not take Z into account
Poor comments are ambiguous
Good comments are clear
The allocated resources are unrealistic
The resources allocated to Work packages X and Z are seriously underestimated given the complexity of the work described
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #28

29. Writing Tips of Consensus Reports (CR) (2/3)
Comments should be written for ALL sub-criteria
Poor comments are vague
Good comments are precise and final
Examples:
The management plan seems inadequate given the project duration and the number of partners involved
The management plan is inadequate. It does not include clear overall responsibility for the administrative tasks. Furthermore it does not establish a risk assessment process and/or a mitigation procedure. It does not foresee a decision making mechanism nor a problem-solving procedure in case of disputes between partners. In addition, it does not bring in the expertise in X needed to carry out the tasks referred at Work package Y
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #29

30. Writing Tips of Consensus Reports (CR) (3/3)
Comments should be written for ALL sub-criteria
Poor comments provide the opening to complaints (redress procedure)
Good comments provide final answers to the question
Examples:
There is no mention of dissemination activities
Dissemination activities are not adequately described and substantiated by a strategic plan
There is no mention of Intellectual Property
Intellectual Property is not adequately described and there is no plan to manage it
The coordinator is not adequately experienced
The coordinator does not demonstrate in the proposal an adequate level of expertise in this field / or as a coordinator
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #30

31. Panel meeting Quality control” on the evaluation process
Attended by all rapporteurs
Review Consensus Reports
Resolve any cases where a minority view was recorded
Review comments and marks for consistency
Confirm final marks and comments for each proposal
Any adjustment of scores (if necessary) should be carefully justified
Recommend list(s) of proposals in order of priority
The single stage evaluation is about “ranking” decision
Prepare and sign Evaluation Summary Reports (ESRs) – each rapporteur
Prepare and sign – collectively - the Panel Report
Comment on evaluation process
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #31

32. Evaluation Summary Report (ESR)
As a general rule, ESR = CR
Changes from CR should be exceptional and duly justified
ESR must be of high quality as it is the only feedback given to proposers following evaluation (minimize the use of redress procedure)
The process is important…
…but it’s the outcome that counts!
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #32

33. Thank you for your attention !
Note: Document not legally binding
2013 Energy Calls for proposals – Expert Briefing (January 2013) #33