A Methodology for Evaluating Ex Ante the Regional Benefits of R&D Projects Stephen Roper, Nola Hewitt-Dundas and James H Love Aston Business School, Birmingham

Why is ex ante evaluation of R&D projects particularly difficult? Uncertain outcomes or success, i.e. risk Uncertain timing of outcomes or benefit stream Difficulty of assessing chance of technology developed being exploited locally by the R&D performer And, crucially, difficulty of assessing the extent of local spillovers through: –technology or knowledge transfer –supply chains –development of research trained labour

The Task Key questions: –What is best practice in ex ante evaluation of publicly funded R&D projects? –Can we create a framework/tool to enable us to better evaluate the returns to R&D projects? Elements of study –Discussions with agencies in Scotland, Wales and DTI –International consultation – Finland, Israel, US –Extensive review of evaluation/academic literature –Development of tools for Invest NI –Calibration and revision of tools in seven case studies of existing R&D Centres

Best Practice? Best practice/ international/ lit survey - no existing systems for predicting returns (or ex ante evaluation) – No ‘silver bullet’ Instead we found different strategies: –Avoidance of issue (Israel) –Peer review of projects (US, Finland, Scotland) –Strategic rather than economic evaluation (DTI, Wales) But: evaluation studies do provide rich knowledge base of ex post effects of different types of R&D projects.

Key Questions So.. can we use existing knowledge base to draw inferences ex ante about the likely regional benefits of different types of R&D project. RQ1: What (global) benefits stem from the establishment of a new R&D centre or facility? RQ2: What determines the share of these benefits which accrue to the host region? How do we codify the answers into a ‘tool’ usable by a regional development agency?

RQ1: What global benefits stem from the establishment of a new R&D centre or facility? - An Inventory Approach

RQ2: What determines the share of these benefits which accrue to the host region? The Profile of the R&D Centre or Project –Type of R&D –Institutional or Organisational Setting –Applicability –Business Ownership Embeddedness of the R&D centre –Parallels with high-tech plants? –Effects of region size Absorptive capacity of the RIS –Industrial composition –Absorptive capability of local firms –Public and private knowledge mediating institutions –Networks

Key Conclusions from Research Phase Ex ante impact assessment needs –benefits inventory –knowledge base from prior evaluations/studies –profile/landscape/synergy consideration The bottom line? –Private benefits depend crucially on type of business –Rent based spillovers depend on embeddedness (labour may be stronger than product market) –Pure knowledge spillovers depend on synergy/RIS

Tool Building and Validation Spreadsheet based scoring tool for agency staff Developed using inventory x regional share approach ‘Validated’ using 7 existing Centres and comparison of outcomes to ex ante predictions Tool developed can – at best- give a broad indication of expected effects So, use alongside a narrative template developed to reflect same structure

End Notes As yet no way of making reliable ex ante predictions of social rate of return from individual R&D projects But we can use our framework to identify a comprehensive list of potential benefits and a general idea of their likely importance to host region. Individual case-specific factors will still be important, however.

Spreadsheet Model Step 1: Global benefits of R&D Project

Spreadsheet Model Step 2: Regional Share of Benefits

Narrative Template – Private Benefits

Narrative Template – Wider Benefits