The non-trivial choice among innovation indicators by Alfred Kleinknecht, Professor, Economics of Innovation, TU Delft & Visiting Professor, Université Panthéon Sorbonne, Paris I Key note speech to the Ester/ESF Workshop: "The Economic History of Patents and Innovation", Eindhoven University of Technology, June 2009 Related literature can be downloaded from:

Indicators of innovation: R&D (Research & Development): Man hours of R&D as a percentage of total personnel; or R&D budget as a percentage of total sales; or, a country's R&D spending as a percentage of its National Product Total innovations costs (Including non-R&D innovation costs) Patents granted or patent applications (at the European, US or national patent offices) Sales of new products: The share in total sales taken by technologically new or substantially improved products, introduced during the past 3 years. Distinction between "New to the firm" (→Imitation) and "New to the market" (→Innovation) New product announcements in trade journals

Note: These five indicators have a very weak correlation with each other: it does make a difference which one you use!

R&D (personnel or budgets): Strengths: + Regularly collected since the 1950s + Can be subdivided by product and process effort + Easily accessible (in aggregated form); + Time series available + Analysis of inter-sectoral technology flows is possible

R&D (personnel or budgets): Weaknesses: - It is an input indicator - Usually it cannot be split by technical fields - It is only one of several input indicators (besides design, trial production, tooling-up, market analysis, training, investment in fixed assets etc.) - Undercounting of small firm R&D (calls into question usefulness for comparison across firm size classes, sectors, regions and even countries): serious measurement problems due to subsidy schemes. - Problems with the interpretation of the Frascati definition - Use of micro data by researchers is often hampered by secrecy - “Singapore effect” - Hard to disaggregate by regions (multi-plant firms)

Total innovation expenditures: Strength: + Measures many more aspects than just R&D, e.g. design, trial production, tooling-up, market analysis, training, investment in fixed assets. Weaknesses: - Input measure - Difficult to report in surveys (high item non-response) - Inclusion in a questionnaire may have negative impact on overall response

Patents granted or patent applications: Strengths: + Long historical time series available + Publicly available (no secrecy problems) + Relatively consistent over time + Classification by technical field is possible + Citation analysis is possible

Patents granted or patent applications: Weaknesses: - Misses non-patented innovations; note that the probability to patent an innovation differs (substantially!): - Across firm size class; - Between firms that do or do not collaborate on R&D; - Between sectors - Patent figures may be obscured by strategic behaviour - Hard to classify by economic sectors

Sales of innovative products: Strengths: +Direct measure of innovation +Only successful innovations (positive cash flow) +Interesting distinction between imitation and innovation +By relating R&D input to innovative output, we can say something about factors that influence efficiency of the innovation process

Sales of innovative products: Weaknesses: - Sometimes suffering from low response rates - Will the firms adequately interpret our definitions? - Accuracy of answers? - Probably quite sensitive to the business cycle - Inter-sectoral comparisons can be problematic (length of life cycles differs)

New product announcements in trade journals: Strengths: + Direct measure of innovation + Not expensive to collect + No secrecy problems + Can be classified by type of innovation + Data can be extended to the past + Covers innovations in very small firms + Tracing of inter-sectoral flows is easily done + Easy to disaggregate by regions + Allows analysis of success and failure of innovation

New product announcements in trade journals: Weaknesses: - Less suitable for comparative studies as statistical properties of database appear dubious - Reliability depends on adequate selection of journals - Process innovations are neglected - Innovations from small market niches may be missed - Influenced by publication policy of journals and firms - Larger firms appear to be clearly under-represented

Final remarks: Note that the various innovation indicators have a weak correlation with each other. It therefore matters which one you use; the choice between indicators depends on your research purpose. Further reading: Kleinknecht, A., K. van Montfort & E. Brouwer: "The non-trivial choice between innovation indicators", in Economics of Innovation and New Technology, Vol. 11 (2002), p OECD Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd edition (2007); download: