Knowing science Synopsis of the state of the art based on collected research results of the team

Purpose of the presentation To provide a summary overview of the research findings of the group To signal potential policy relevant aspects of the findings

Structure of the presentation Research findings are grouped into three categories Higher education and research governance: Setting the scene Problems in the governance of research Comparing research systems and evaluating research: State of the Art

Structure of the presentation Project information –title of project, principal investigator, main questions tackled, empirical site(s) Most policy significant results

Higher education and research governance (Kauppi) Project: Higher education rankings Principal Investigator: Niilo Kauppi Problem(s): What is the impact of ranking on higher education institutions (HEI)?

Higher education and research governance (Kauppi) The impact of rankings is difficult to measure because they are part of a broader tool kit and set of transformations that HEIs and research are currently undergoing. Rankings enable some HEIs to position and navigate themselves in an increasingly complex environment. Rankings also provide governments with tools to govern their HEIs. Rankings function as regulatory value standards to help research universities set strategic goals.

Problems in the governance of research (Hellström) Project: Valuing Research and Development Principal Investigator: Tomas Hellström Problem(s): How is value assigned to scientific work (products and practices) and in selecting research? How are such values expressed? Empirical sites: Strong research environments, Centres of Excellence (Sweden

Problems in the governance of research (Hellström): policy significant findings The contribution of these environments to the renewal of the traditional university structure is unclear and dependent on how they are positioned in the university structure Not all fields are able to organize in the fashion prescribed by strong research environment and excellence schemes Tension between (1) strategies (global) and execution/impacts (local), (2) excellence and impact, (3) organizational goals and cognitive/social goals … in CoE schemes

Problems in the governance of research (Braun) Project: Institutional conditions for governing science Principal Investigator: Dietmar Braun Problem(s): Relationship between societal governance and epistemic production Empirical sites: Funding Agencies and Universities

Problems in the governance of research (Braun): policy significant findings Project shows the existence of 3 layers of governance: social, epistemic and organisational Every layer of governance has a preferred organisational model Self governance in science did not in all cases support creativity NPM variant of societal governance offers the advantage of direct support for science through its interventions via evaluation but the costs to policy and science are high in relation to benefits

Funding research systems and evaluating research: State of the Art (Jacob) Project: Analysis of funding instruments and modalities Principal Investigator: Merle Jacob Problem(s): Analysis of impact of funding instruments and modalities on research systems

Funding research systems and evaluating research (Jacob): policy significant findings Funding instruments are standardised, modalities represent opportunities for experimentation Funders can and do use instruments and modalities to introduce new behaviours, standards to the research community. More attention needs to be paid to match instrument/modality to scientific field Grouping of objectives on to same instrument may disadvantage some fields

Funding research systems and evaluating research (Thomas): State of the Art Project: Characterising researchers Principal Investigator: Duncan Thomas Problem(s):To understand how social conditions (policy, funding) manifest in research organisation and affect researchers’ knowledge production (particular epistemic properties with particular people [group, community, utilisation, commercialisation

Funding research systems and evaluating research (Thomas): Results of significance to policy Multiple pressures for change at any one time... not only specific funding pressures (e.g. ERC grant schemes)... Policy/funding pressures not sole cause of organisational changes but can affect the speed and scope of changes... Performance of research organisations mediates susceptibility to policy/funding pressures... different orgs affected differently... Internal funding decisions, steering of career pathways, organisational decisions, prioritisation of various markers of quality... all can be affected by policy/funding

Funding research systems and evaluating research (Nedeva) Project: Effect of policy and funding on science Principal Investigator: Maria Nedeva Problem: To understand how policy and funding schemes constitute different pressures for change of the organisational conditions of science and relate organisational trasformation to changes in the nature and characteristics of knowledge.

Funding research systems and evaluating research (Nedeva): policy significant findings Fine grain analyses of research systems provide actionable knowledge that can: Improve the policy making process by enabling better targeting of incentives; Inform expectations of desired outcomes and effects; Enable the development and implementation of better adjusted policy instruments; Enable the development of policies (and funding schemes) that send clear signals to potential beneficiaries in noisy policy spaces Have consequences for the way in which the effectiveness and efficiency of science and innovation policy are assessed.

Funding research systems and evaluating research (de Rijcke) Project: Effects of performance indicators on knowledge creation Principal Investigator: Sarah de Rijcke Problem: The project aims to understand intended and unintended effects of the proliferation of metrics on knowledge production

Funding research systems and evaluating research (de Rijcke): policy significant findings The worth of research activities becomes increasingly assessed and defined by their potential for translation into quantitative measures of quality This general trend holds most strong in fields that have greater degrees of interdependencies, division of labour, need to pool expertise, work with large numbers of collaborators Indicators of scientific quality such as innovativeness, societal relevance become redefined through their relations to quantitative indicators There is a risk that ease of comparability afforded by metrics involves trade offs which could undermine long term policy goals such as encouraging innovative, societally relevant and responsible research.