Integration of Information Systems for the Benefit of Researchers and Policymakers A Need for a European Approach and Coordination? Erik Arnold European Parliament, Brussels 26 March 2013

2 Why are we interested in information and measurement? Drive from the New Public Management for performance measurement Increased effort to link research with innovation A need for output data to feeding the growing use of performance- based research funding systems More use of evaluation in managing and measuring funding programmes Increased autonomy of research-performing institutions leads to a desire for more management information at institutional level Open access to the results of publicly-funded research ICT systems increasingly enable access to, and research use of, others’ research materials

Stakeholders’ use of research information Governments and government agencies Research institutions management & governance EU and National Governments Ministries responsible for research/innovati on Government Agencies Governing Bodies/ Councils Executives/ Management Research Groups / departments & institutes Inform policies & strategies Determine competitiveness / strategic positioning xxxxxx Improve performance and qualityxxx Improve system functionalityxx Aid resource allocationx Assess per- formance Quality, sustainability, relevance and impact of research activity xx Other Investor confidence/value-for- money and efficiency xxxxx Recruitmentxx 3 Based upon: Expert Group on Assessment of University-Based Research, Assessing Europe’s University-Based Research, European Commission, DG Research, 2010

Types of system are proliferating Research information systems Institutional level Funder or national level Indicator systems and databases at national and European level National research evaluation databases International bibliometric databases Web of Science, Scopus Web-based publication databases Scholar Google – a game-changer? Open access repositories Others, eg CV and patent databases 4

Use of bibliometrics in performance-based funding MethodUnit of analysisType of bibliometricsCountry Indicator-basedInstitutionPapersDenmark Finland Norway Papers & citationsBelgium (Flanders) Poland Slovak republic Sweden Peer reviewDepartment/ field of research UK RAE Italy Informed by metricsPoland Czech Republic IndividualSpain New Zealand 5 Based on Diana Hicks, 'Overview of models of performance-based research funding systems', Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings, OECD Working Party on Research Institutions and Human Resources (RIHR), 2010

Why would we need better systems? Better measurement of the broad impacts of research Support better evaluation methods More data availability Better, more sophisticated indicators Improved understanding of research and innovation systems Availability of fine-grained, internationally comparable data Obtain input data (especially funding), not just outputs Enable policy coordination and measurement at a trans-national level 6

Data sources for research and innovation assessments 7

Growing interlinkage of systems, enabled by emerging standards 8

Use and sources of tools for evaluation 9

Questions for today’s workshop 1.Is our view on the current landscape of systems for research- related information collection correct? Did we miss out something? 2.Is it desirable to interconnect the various types of information? Is it practical? What would be/are the benefits? 3.Is interconnection of national systems at European level desirable? Is it practical? What would be the benefits? 4.Is there a coordination deficit at European level? 5.Where should coordination come from? 10

11 Thank you technopolis |group| has offices in Amsterdam, Ankara, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna