Taking the Road Less Travelled: Evaluating and Monitoring the Socio-Economic Impact of Investment in Research Infrastructures Dr Jelena Angelis Deputy Director, Technopolis Group Baltics ICRI2016 Cape Town, 4 October 2016

Introduction Methodological guidance document: Griniece, E., Reid, A. and Angelis, J. (2015) Evaluating and Monitoring the Socio-Economic Impact of Investment in Research Infrastructures Work-in-progress based on: Systematic literature reviews Policy studies, foresights and evaluations on research infrastructure topics (notably in NL, UK, IE and EU level) Hand-on work for monitoring investment in RI in Lithuania (3.5 year long technical assistance project) Exchanges with RI practitioners at workshops Contributions from OECD Global Science Forum workshop on methodologies and tools for IA of RI and numerous conferences 2

Agenda 3 1. Why evaluate impact of RI? 2. Conceptual framework 3. Methodologies and tools 4. Limitations

Why to evaluate impacts of RIs? 4 2. Conceptual framework 4. Limitations Increasing importance: RIs are high on political agendas Increasingly form important part of strategic RDI visions and roadmaps Concern for value added: Generally investment in RI requires considerable level of financial investment and long-term operating strategy Broad impacts: Designed for research needs but impacts reach beyond fueling scientific excellence Yet, difficult to quantify this return in conventional commercial terms

Suggested framework – finding the pathways and connections 5 To disentangle and characterise the concrete benefits from investment in RI over the whole life-cycle of RI To build a schematic impact assessment framework that can be used as reference to trace the core impact pathways Predominantly designed for single-sited and geographically confined national or macro-regional distributed RI A step towards defining a typology of possible effects and accounting for conditions and patterns that enable their creation and diffusion

6

Methodologies and Tools for IA of RI (1/2) 7 2. Conceptual framework Economic impacts: econometric techniques, e.g. CBA, input- output models, economic multiplier estimates. However, they have to rely on uneven data input and also face validation issues Impacts on human resources: statistics, no. of PhDs, training activities, etc. However, more elaborated longitudinal studies e.g. on graduate career paths are useful Impacts on innovation: standard statistics, e.g. patents, licensing, spin-offs, etc. Much more difficult to gain insights of innovation activities induced in companies

Methodologies and Tools for IA of RI (2/2) Impacts on research activities: bibliometrics, peer reviews, scientific collaborations and other standard indicators are routinely collected. However, the specificities of RI-based scientific output should be better taken into account (e.g. time lag for publishing results and traceability of RI where the research was carried out) Broader societal impacts: rely heavily on qualitative methods and data; there is large heterogeneity among social parameters/effects to be considered 8

Limitations – stuck in a box? 9 2. Conceptual framework 3. Methodologies and tools Limited data caption during the lifetime of research facility Delineation between short-, medium- and long-term effects difficult due to restraints in data availability in time series Difficulty of capturing multidimensional and complex research outcomes and impacts with simple metrics that are used as core indicators Issue of displacement – or the extent to which benefits occur at the expense of other economic/societal activities – is hard to address in the analysis

10 Key messages RI is embedded in specific socio-economic conditions. Hence, no ‘one-size fits all’ approach to mapping all socio-economic impacts Reflection on impact pathways is recommended to scope challenging IAs in which socio-economic impact may seem remote at the first sight Need for diverse evidence and interdisciplinary expertise. There are signals but very scarce empirical evidence of wider societal benefits, such as impact on science and education system, healthcare system, environment, cultural tourism, public awareness of science

Thank you! 11 technopolis |group| with offices in Amsterdam, Bogotá, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna