The fork in the road for innovation measurement: which way should we go? Jonathan Haskel Imperial College Business School, Imperial College London and CEPR, UKIRC National Academy of Sciences, Washington DC, 11 th July 2011 Slides plus spares, links to papers etc. in spares section 1

The exam questions (to be answered in final slide…) The objective of this session is to identify – recent developments in measuring STI and – what is currently planned for the future. Discussion should reveal – what has been successfully and unsuccessfully measured. – What are critical bottlenecks and perceived opportunities? – What global STI metrics and indicators should NCSES develop in the near and medium term (the next 5-10 years)? 2

The fork in the road for innovation measurement Approach 1: EU-type innovation surveys – Tried for 15+ years in EU, Applied in latest US work – Key questions “have you process/product innovated?” Spending on innovation e.g. software, marketing, training Potpourri of other questions – Information for innovation (e.g. importance of information from universities high/med/low) – What stopped you from innovating? Approach 2: extended R&D/spending type surveys – Follows Griliches/Jorgenson/CHS/Nat Accounts to measure knowledge capital – Implementation: software and (forthcoming) R&D spend – CHS agenda: need coinvestments with R&D e.g. design, training, marketing, business process Current method: potpourri of surveys UK: intangibles investment survey, similar style to R&D survey Kauffman 3

What theory do we have to guide measurement? Innovation as TFP i.e. output net of tangible K and labour – Logically consistent, suggests measurement (e.g. depreciation, investment) – Appropriate if all knowledge from free spillovers Corrado, Hulten, Sichel – Recognises innovation is routinised in many firms e.g. organised spending on marketing, training, design etc. – Helps understand innovation in non-R&D industries e.g. finance – Strength: CHS categories are matched to functional bodies in (large) firms e.g. HR dept for training, marketing dept for brand etc. – Still scope for knowledge spillovers Measurement implications – Spending, investment, depreciation, asset and rental prices

European Innovation survey questionnaires UK example: product innovation 5 Process innovation

6 US data: Boroush (2010), Other: OECD Measuring innovation (2010), p.26 Michael Mandel: “You can’t be an innovative economy if only 9% of your companies are innovating” What do innovation surveys find?

What does the “have you innovated” question measure? Other findings – the total is procyclical: rises strongly in IT boom end of 1990s – Item non-response can be very high e.g. 70% to intangible spending questions – Large firms show poor response rates So what does the question measure? Crespi, Criscuolo, Haskel (2007, Appendix 1), UK CIS3 (98-00) – 45% of firms reported technological product/process – they were asked for text describing their innovation – They are given instructions on what to include and exclude e.g. include “robotised welding”, exclude “packaging” – Analysis of 874 firms showed 7

The “have you process innovated “ question measures new capital equipment Summary: – of firms who provided the information, reported process innovation is mostly new capital spending – This is why measured “innovation” is highest in IT boom – Conjecture: we would get same/better(?) information from a standard investment survey (+ good deflators) 8

An alternative approach guided by CHS What does this framework suggest you need to know? – Knowledge services within the firm = knowledge capital stock and rental rates Innovation spending on broad range of knowledge assets: software, design etc. Convert to investment Convert to real stock: need depreciation and deflators – Implication: need a spending survey. Can piggy back onto R&D survey or new one (or modify innovation survey) – Knowledge from outside the firm Information flows – Patents is one source: but service sector? – CIS information flows

Possible model: Investment in Intangible Assets Survey ( ) Extension of UK R&D survey, Autumn 2009, through ONS: so linked to national accounts data and R&D and CIS survey Survey follows (for large firms) typical business functions 10

Each section has a filter question which defines the asset with examples 11

Then asks purchased and own-account 12

Finally life lengths 13

14 Life lengths All asset life lengths are greater than one year and range between 2.75 years for training and reputation to around 4.5 years for R&D.

My answers to the exam questions The objective of this session is to identify – recent developments in measuring STI US innovation survey raises puzzles for survey design. Intangible assets framework gaining policy traction. Some surveys moving in this direction. – what is currently planned for the future Review by OECD of innovation surveys. More UK intangible surveys. Kauffman survey to continue? Discussion should reveal – what has been successfully and unsuccessfully measured. yes/no innovation questions have not worked. I Intangible spending can work if asked to business functions, own-account explained. Business process has not worked in UK survey – What are critical bottlenecks and perceived opportunities? Shortages of funding for more surveys. Opportunity (a) in EU, convert Innovation surveys to intangible spending surveys, (b) in US to convert supplement to BERDIS to intangible spending survey. General lesson: take best practice from each survey – What global STI metrics and indicators should NCSES develop in the near and medium term (the next 5-10 years)? Intangible spending, life lengths. Integrate with National Accounts work on deflators, service sector output. 15

Spares 16

What do innovation surveys find? 17 US data: Boroush (2010), Other: OECD Measuring innovation (2010), p.26

Crespi et al, What do firms write down as their most significant process innovation? Replies – 53% are other, mostly including capital equipment (e.g. steel wire machine, spectrophotometer..) – 23% are hardware related (computer, CAD, PC) and software related (website, , software, internet etc.) Summary: of firms who provided the information, reported process innovation it new capital spending Not surprising that measured “innovation” is highest in IT boom We likely get equivalent information from a standard investment survey (and spend resources on better deflators) 18

The future Take best practice from elements of different surveys – Innovation surveys Don’t ask % new, % innovation Do ask information sources – R&D/IAS Do ask for spending Keep structure of questionaire allied with business function: ICT dept for software, HR dept for training, Marketing dept for branding Complement with – What do firms do? Innovation data at project level ? – What do young firms do? Likely very intangible intensive. Kauffman survey. – What do consultants do, especially intangible consultants? – What do financial institutions do esp young firms, they are trying to value innovation? – Measuring business process huge challenge Chance for NAS/OECD to lead as OECD did on software