Mapping of S&T excellence in Europe Brussels, 23 November 2000 CENTER FOR INNOVATION, TECHNOLOGY AND POLICY RESEARCH, IN+ Instituto Superior Técnico, Technical University of Lisbon Mapping of S&T excellence in Europe: building on the Portuguese experience in evaluating research excellence Pedro Conceicão and Manuel Heitor

The scope accelerating technological change: strategic development of technology platforms: the context: collaborative learning and the emergence of the learning society! knowledge creation and diffusion is increasingly recognized as a major driver of economic growth The need to access to new knowledge and the science base, together with the requirements to speed-up knowledge diffusion and technology commercialization The result: the need for a competitive science base

The question What can we learn from the portuguese experience in research evaluation, in a way to stimulate the European Research Area and to promote research excellence in Europe?

Our argument... The locus of excellence? Varied and multidimensional information! The need to identify the potential for excellence in the coming decades: international peer review The scope:... the globalized “learning society”! Knowledge Institutions Learning Organisations Intellectual Property Learning Networks BUT, stimulating the inclusive development ! Conceicão, Gibson, Heitor & Sirilli (2000)

The contents 1. The objective 2. A Portrait in Figures: A Quantitative Perspective of Portuguese S&T in OECD 3. The evaluation of research excellence in Portugal 4. The context: Building the European Research Area 5. Concluding

Resources: Scale vs Intensity R&D Expenditure (OECD) Sweden Finland Japan France Germany the Netherlands Denmark UK US Belgium Austria Ireland Italy Spain Portuga l Greece 0 0,005 0,01 0,015 0,02 0,025 0,03 0,035 0, Scale- Total Expenditure in R&D ($PPP; logarithmic scale) Intensity- Share of GDP spent on R&D

Output: publications vs expenditure

Scientific and technological capacities in OECD Fonte: Caracostas & Muldur (1998)

Innovation in EU source: Community Innovation Survey, OCT-PT

Growth of Knowledge-based industries OECD(2000) UK* Belgium Portugal** Greece* Sweden*** NL* Austria Japan Norway Canada Denmark Spain *** Denmark Mexico Italy US France Korea Germany Share in Business Sector Value Added of Knowledge Based Industries (share year 1996 except: *1995;**1993; ***1994) Average Annual Real Value Added Growth of Knowledge Based Industries (1985-share year)

Incentives to BERD Australia US Korea Denmark Switzerland France Belgium Iceland UK Canada Germany Finland Sweden Italy Norway Mexico Austria Ireland Spain Netherlands Japan Portuga l Change in the Amount of Tax Subsidies for 1 US Dollar of R&D SOURCE: OCDE (1999).

…further evidence Le Monde, 21 November 2000 Based on positive correlation of productivity growth and the increase in BERD: “les bons élèves da le nouvelle économie”:...en Europe, la Scandinavie, l’Irlande et le Portugal tirent le mieux leur épingle du jeu,...

The issue… Evaluating scientific excellence in Portugal We argue: Portugal is facing the growing importance of digital technologies The evidence : Structural debilities in the education and R&D system still persist, despite the recent growth of R&D expenditures and the effort to provide incentives for private spending on R&D The challenges: - development of new skills - involvement in international networks of excellence The question for S&T: the extent to which the existing research excellence in Portugal allows the involvement in European (and international) networks of excellence?

The contents 1. The objective 2. A Portrait in Figures: A Quantitative Perspective of Portuguese S&T in OECD 3. The evaluation of research excellence in Portugal 4. The context: Building the European Research Area 5. Concluding

Evaluating scientific excellence in Portugal: the concept critical analysis of research activities, based on recommendations from external experts with experience in scientific assessment. a process of regular assessment, encouraging the development of a dynamic scientific and technological base: launching the process:1996 promoting reorganization of the S&T system:1997/98 a continued process :1999

Evaluating scientific excellence in Portugal: the methodology International peer-review: panels of foreign scientists using international reference criteria documentation Results quality and quantity relative to number of PhD’s Relevance of the research activity Internationalization - quality and quantity visit Potential for developing excellence Intrinsic merit of the Unit activities Attitudes and work environment young researchers Resources for the research activity Evaluation based on: Multidisciplinarity: multiple peer review

Evaluating scientific excellence in Portugal: the methodology Grades Description Excellent Research activities at a high international level, with publications in internationally leading journals Very Good Research activities at a good international level and at a high national level, with publications in inteinternationally leading journals Good Research activities at a high national level and at a fair international level, with publications in internationally well- known journals Fair Research activities at a fair national level, with publications only partially in internationally well -known journals Poor Research activities of insufficient quality

Evaluating scientific excellence in Portugal: areas and number of PhD’s

Evaluating scientific excellence in Portugal quantitative results: grading

Evaluating scientific excellence in Portugal: PhD’s per grade ExcellentVery GoodGoodFairPoor

Evaluating scientific excellence in Portugal: the issue…2 the usage of indicators of shares of national resources used in R&D (as inputs) and the usage of bibliometric indicators and number of patents (as outputs) are likely to consistently bias an accurate portrait of the size and, especially, quality of Portuguese research The evidence: a number of excellent R&D units The method: international peer-review

The contents 1. The objective 2. A Portrait in Figures: A Quantitative Perspective of Portuguese S&T in OECD 3. The evaluation of research excellence in Portugal 4. The context: Building the European Research Area 5. Concluding

Discussing S&T excellence in Europe… 3 main ideas: European Research Area as a way to promote the inclusive development of the European science base Scale and diversity Competitiveness at average level Benchmarking: learning by comparing A collective process leading to mutual learning Metrics for knowledge Centres of Excellence with a European dimension Networks of excellence

Conclusions Mapping excellence requires the identification of the locus of excellence across Europe. The information needed to identify these poles of excellence is, naturally, varied and multidimensional We argue that: in the case of Portugal, scientific and technological excellence may be lost in aggregate quantitative indicators traditionally used to measure scientific and technological activities. peer review, with international criteria, is able to identify to enablers of new competences and the potential to develop excellence in the coming decades. European Research Area requires poles of scientific competences, and the development of networks of research excellence

Mapping of S&T excellence in Europe Brussels, 23 November debate

Human Development and Inputs to S&T

Public vs private R&D expenditures P97 P95 P81 ES IR SE JP USA FR D UK

Regional diversity in Western Europe Source: Sixth Periodic Report DG XVI, 1998