1. www.taftie.org www.parp.gov.pl 2014 TAFTIE Policy Forum “Measuring innovation” Can we measure innovation? Lessons from innovation scoreboards Hugo Hollanders Brussels, November 25, 2014

2. www.taftie.org www.parp.gov.pl What is an innovation scoreboard? “Innovation scoreboards consist of a collection of national or regional indicators of relevance to innovation” Arundel, A. and H. Hollanders, "Innovation Scoreboards: Indicators and Policy Use", in C. Nauwelaers and R. Wintjes (eds.), Innovation Policy in Europe, Edward Elgar: Cheltenham, 2008 “Innovation scoreboards are an attempt to summarize innovation indicators and to compare innovation performance of countries, regions or sectors” Hollanders, H. and N. Janz, "Scoreboard and indicator reports", in F. Gault (ed.), Handbook of Innovation Indicators and Measurement, Edward Elgar : Cheltenham, 2013

3. www.taftie.org www.parp.gov.pl Purpose of innovation scoreboards The main purpose of an innovation scoreboard is to assist policy by summarizing a range of innovation indicators at the national, regional or sectoral level by permitting a comparison of the relative success or failure of the innovation system, or through the identification of specific aspects of the innovation system which perform well or poorly

4. www.taftie.org www.parp.gov.pl Policy relevance Innovation scoreboards serve three policy needs They act as an ‘early-warning’ system for potential problems If used over time, they can track changes in innovation strengths and weaknesses They can attract the interest of policy-makers and as such can help build a consensus to introduce actions and policies to improve the innovation capabilities of firms and institutions Policy use Innovation scoreboards will be most useful for policy when combined with an integrated system for evaluating innovation policies

5. www.taftie.org www.parp.gov.pl Principles of scoreboard design The indicators should Be of similar importance as measures of the drivers of innovation activity Be based on reliable statistics Hold their value over time Be of relevance to medium- and long-run policy issues

6. www.taftie.org www.parp.gov.pl Examples of innovation scoreboards European Commission Innovation Union Scoreboard (IUS) Innovation Union Competitiveness Report OECD Science, Technology and Industry Scoreboard OECD Science, Technology and Industry Outlook INSEAD/WIPO Global Innovation Index World Economic Forum Global Competitiveness Report World Bank Knowledge Assessment Methodology Deutsche Telekom Stiftung, BDI Innovationsindikator

7. www.taftie.org www.parp.gov.pl Examples of innovation scoreboards

8. www.taftie.org www.parp.gov.pl IUS uses 25 indicators which are grouped into 3 main categories: Enablers capture the main drivers of innovation performance external to the firm Firm activities capture the innovation efforts at firm level Outputs capture the effects of firms’ innovation activities Average performance is measured by a composite indicator, the Summary Innovation Index 8 Innovation Union Scoreboard IUS monitors the implementation of the Europe 2020 Innovation Union flagship by providing a comparative assessment of the innovation performance of the EU28 Member States and the relative strengths and weaknesses of their innovation systems

9. www.taftie.org www.parp.gov.pl 9 IUS Measurement framework

10. www.taftie.org www.parp.gov.pl IUS Performance groups 10

11. www.taftie.org www.parp.gov.pl Differences in rankings

12. www.taftie.org www.parp.gov.pl IUS International comparison Europe’s performance gap with the US and Japan is closing but the gap with South Korea is increasing Europe is

13. www.taftie.org www.parp.gov.pl Lessons from innovation scoreboards What are the strengths and weaknesses of the innovation measurement tools and indicators we use today? Strengths Country coverage, measuring variety of innovation activities, good coverage of inputs and throughputs, use of innovation survey data Weaknesses Partial coverage of outputs, use of innovation survey data

14. www.taftie.org www.parp.gov.pl Lessons from innovation scoreboards What’s the state of play on the development of new indicators? Community Innovation Survey 2014 Better coverage of public sector as a driver of innovation Public procurement Ad-hoc modules: 2012 on growth, 2014 on innovations with environmental benefits EC’s Innovation Output indicator Combines 5 of the IUS indicators into one indicator measuring outputs of which one is a new indicator: Employment in fast-growing firms of innovative sectors

15. www.taftie.org www.parp.gov.pl Fast-growing firms innovative sectors 2010: High-Level Panel asked to recommend new headline innovation indicator 2011-2013: work by OECD/Eurostat to identify the innovativeness of sectors 2013: indicator launched by EC Employment in fast-growing firms of innovative sectors provides an indication of the dynamism of fast-growing firms in innovative sectors as compared to all fast-growing business activities. It captures the capacity of a country to transform rapidly its economy to respond to new needs and to take advantage of emerging demand

16. www.taftie.org www.parp.gov.pl Lessons from innovation scoreboards What has not been measured yet? Outputs, current indicators focus on Employment in high-tech and knowledge-intensive activities Exports in high-tech and knowledge-intensive activities Both of which do not necessarily have to be directly linked to innovation Sales of innovative products Data are not available for all countries and there are issues about the comparability of results across countries Commercialization activities Innobarometer 2014

17. www.taftie.org www.parp.gov.pl Commercialization of innovations

18. www.taftie.org www.parp.gov.pl Commercialization of innovations

19. www.taftie.org www.parp.gov.pl Thank You

20. www.taftie.org www.parp.gov.pl Lessons from innovation scoreboards Is there a better taxonomy and nomenclature for the innovation process? How can we develop the process of finding the exact tools to observe, monitor and describe policy relevance, data quality (international availability and cross-country comparability) and robustness of results? What could be the role of TAFTIE in developing innovation indicators and ensuring the most relevant picture of impact of TAFTIE support?