1. “Thematic Priority 3” Draft 19.12.02 Evaluation of IP + NoE

2. Evaluation planning 1st Call 1st stage: 2nd stage: Other instruments: Call IP/SMEs 1st stage: 2nd stage: Call P2/P3 1st stage: 2nd stage: Other instruments: 06/3/03beginning April 26/6/032nd week July 10/4/03end May / beg June 10/4/03beginning May 03/9/03mid September 24/4/033rd week May 16/9/032nd week October 24/4 + 16/93rd week May + Oct. DeadlineEvaluation period

3. Stage 1 : Outline proposal - not “incomplete” - all criteria will be applied Stage 2 - S&T and financial details, including work/financial plan for first 18 months;further/more details through hearing - all criteria will be applied Two Stage Evaluation...

4. Evaluation procedures for the new instruments Common approach Eligibility Criteria Criteria & thresholds External experts:min 5 Priority 3 approach 2 stage submission & evaluation Hearings at 2nd stage Reduced time to contract, i.e: finalise Description of Work or JPA, administrative and financial plan)

5. Stage 1 : Outline proposal - not “incomplete” - all criteria will be applied Stage 2 - S&T and financial details, including work/financial plan for first 18 months;further/more details through hearing - all criteria will be applied Two Stage Evaluation...

6. Evaluation issues... Integrated Projects Relevance to call 3/5 Potential impact 3/5 S & T excellence 4/5 Quality of the consortium 3/5 Quality of the management 3/5 Mobilisation of the resources 3/5 24/30 Common for all priorities

7. IP evaluation criteria A) Relevance to the objectives of the call, in terms of: –scope of the IP vis à vis the WP/call (including political objectives - Lisbon, Gothenburg, ERA…) –support to transformation of industry, addressing real bottlenecks and key problems –deliverables: breakthrough in new applicable knowledge (new products, processes and services) –ambition (critical mass; relationship and coherence between modules; multi-sectoral approach….) –appropriateness of the use of an IP: flagship project? particular case for “IP for SMEs” ?

8. IP Evaluation criteria (cont) B) Potential impact, in terms of: –Competitiveness (i.e: new markets and new products; impacts on industry; clear relevance for SME intensive sectors in the case of “IP for SMEs”) –Support to EU policies (e.g:health, employment, sustainability, enlargement, etc..) as well as S tandardisation/regulations –Potential impact at international level –Commitment of partners related to exploitation of potential impacts (corresponding to their strategy?) –Science & Society issues including gender

9. IP Evaluation criteria (cont) C) S & T excellence, in terms of: –S &T objectives, content and achievements (work plan, milestones & deliverables); progress beyond state-of-the-art; –multidisciplinary, efficient and coherent approach (key to achieving the objectives) –clear implementation plan; interrelation between the various components of the IP to ensure radical innovation (TT, education, IPR aspects, etc) –recognition of ERA excellence

10. IP Evaluation criteria (cont) D) Quality of the consortium, in terms of: –expertise of partners (suited to role in the project) –efficient groupings of partners –active industrial involvement, including SMEs (substantial in the case of “IP for SMEs”) –allocation of activities and scheduling (complementarity and effectiveness) –high level commitment of partners (credible, realistic, in line with organisations’ long term strategy) … –… based on a clear consortium agreement

11. IP Evaluation criteria (cont) E) Quality of the management, in terms of: –effectiveness of the organisational, management and governance structure vis à a vis the “complexity” of IP’s activities: co-ordination and follow-up; performance indicators; procedures for decision-making and conflict resolution; allocation and distribution of responsibilities; articulation between the different modules including use of knowledge and dissemination /exploitation, communication strategies….. –evolution of the partnership during life of project –knowledge management, including IPR issues, education and training, communication, etc.

12. IP Evaluation criteria (cont) F) Mobilisation of resources, in terms of: –availability of necessary resources (critical mass): human/financial/equipment –balance and cost-effectiveness –industrial funding –consistency of the financial plan –potential to mobilise complementary funds, during and after the project to achieve expected results –key personnel involvement

13. Relevance to call 3/5 Potential impact 3/5 Degree of integration & the J P A 4/5 Excellence of the participants 3/5 Organisation and management 3/5 20/25 Common for all priorities Evaluation issues... Networks of Excellence

14. NoE evaluation criteria A) Relevance to the objectives, in terms of: –scope of the NoE vis à vis the WP/call –support to competitiveness and sustainable development through strengthening of scientific and technical excellence –appropriateness of the use of an NoE: real needs for EU industry and society to integrate activities and research teams in the field? –ambition towards ERA (overcome fragmentation) –deliverables: progressive/durable integration; world-wide recognised ‘virtual’ centre of excellence

15. NoE Evaluation criteria (cont) B) Potential impact, in terms of: –effective integration of research capacities in the field and strategic relevance to ERA –addressing problems of knowledge of tomorrow –socio-economic/industrial impact –spreading of excellence within/beyond the NoE –skills development/generation –contribution to Community societal objectives –potential for TT and dissemination towards industry and in particular SMEs, in particular contribution to standards

16. NoE Evaluation criteria (cont) C) Quality of the integration, in terms of: –appropriateness of the JPA: °(1) integrating activities; °(2) jointly executed research (S&T excellence, coherence of research activities); °(3) spreading of excellence, including through education and training °(4) management activities –adequacy of the number/profile of researchers to be integrated with NoE’s objectives –degree of commitments from partners

17. NoE Evaluation criteria (cont) D) Excellence of the participants, in terms of: –expertise of partners (suited to role in the network and achievement of potential impacts) –high level commitment of partners (credible, realistic) during and after the project –past experience of networking with reference to management capacitities –quality of the grouping, expertise, experience and complementarity, in particular potential for TT and dissemination towards industry, in particular SMEs –possible impact on international cooperation

18. NoE Evaluation criteria (cont) E) Organisation and Management, in terms of: –effectiveness of the organisational, management and governance structure vis à a vis the “complexity” of NoE’s activities: co-ordination and follow-up; performance indicators; response to emerging research needs; procedures for decision-making and conflict resolution(consortium agreement); allocation and distribution of responsibilities, communication strategies… –Management of resources, and evolution of the partnership during the life of the network –knowledge management –adequacy of the requested grant

19. Evaluation procedures for the other instruments Common approach single stage Eligibility Criteria Criteria & thresholds External experts:min 3

20. Specific Targeted Research Projects Relevance to call 3/5 S & T excellence 4/5 Potential impact 3/5 Quality of the consortium 3/5 Quality of the management 3/5 Mobilisation of the resources 3/5 21/30 Common for all priorities

21. Evaluation issues... Co-ordination Actions Relevance to call 3/5 Quality of the co-ordination 4/5 Potential impact 3/5 Quality of the consortium 3/5 Quality of the management 3/5 Mobilisation of resources 3/5 21/30 Common for all priorities

22. Evaluators’ profile: a key point for IPs….. – S/T expertise not sufficient ! Ability to evaluate also ‘integrated’ set of activities: – potential impact (on competitiveness, sustainable development, etc) – critical mass; financial engineering, IPR issues, training, etc.); – risks; management, communication...

23. Evaluators’ profile: a key point for NoEs... Ability to evaluate: –integration in the field; –potential impact (on ERA, competitiveness, sustainable development, etc); –management (financial, human resources) –… Which other expertise (apart of previous experience in networking) is needed?

24. Evaluators’ profile: as usual for the other instruments Ability to evaluate: –scientific & technical excellence; –potential impact (on competitiveness, sustainable development, etc); –management (financial, human resources) –balance of research consortia and proposed resources