1. Knowing science
Synopsis of the state of the art based on collected research results of the team

2. Purpose of the presentation
To provide a summary overview of the research findings of the group
To signal potential policy relevant aspects of the findings

3. Structure of the presentation
Research findings are grouped into three categories
Higher education and research governance: Setting the scene
Problems in the governance of research
Comparing research systems and evaluating research: State of the Art

4. Structure of the presentation
Project information –title of project, principal investigator, main questions tackled, empirical site(s)
Most policy significant results

5. Higher education and research governance (Kauppi)
Project: Higher education rankings
Principal Investigator: Niilo Kauppi
Problem(s): What is the impact of ranking on higher education institutions (HEI)?

6. Higher education and research governance (Kauppi)
The impact of rankings is difficult to measure because they are part of a broader tool kit and set of transformations that HEIs and research are currently undergoing.
Rankings enable some HEIs to position and navigate themselves in an increasingly complex environment.
Rankings also provide governments with tools to govern their HEIs.
Rankings function as regulatory value standards to help research universities set strategic goals.

7. Problems in the governance of research (Hellström)
Project: Valuing Research and Development
Principal Investigator: Tomas Hellström
Problem(s): How is value assigned to scientific work (products and practices) and in selecting research? How are such values expressed?
Empirical sites: Strong research environments, Centres of Excellence (Sweden

8. Problems in the governance of research (Hellström): policy significant findings
The contribution of these environments to the renewal of the traditional university structure is unclear and dependent on how they are positioned in the university structure
Not all fields are able to organize in the fashion prescribed by strong research environment and excellence schemes
Tension between (1) strategies (global) and execution/impacts (local), (2) excellence and impact, (3) organizational goals and cognitive/social goals … in CoE schemes

9. Problems in the governance of research (Braun)
Project: Institutional conditions for governing science
Principal Investigator: Dietmar Braun
Problem(s): Relationship between societal governance and epistemic production
Empirical sites: Funding Agencies and Universities

10. Problems in the governance of research (Braun): policy significant findings
Project shows the existence of 3 layers of governance: social, epistemic and organisational
Every layer of governance has a preferred organisational model
Self governance in science did not in all cases support creativity
NPM variant of societal governance offers the advantage of direct support for science through its interventions via evaluation but the costs to policy and science are high in relation to benefits

11. Funding research systems and evaluating research: State of the Art (Jacob)
Project: Analysis of funding instruments and modalities
Principal Investigator: Merle Jacob
Problem(s): Analysis of impact of funding instruments and modalities on research systems

12. Funding research systems and evaluating research (Jacob): policy significant findings
Funding instruments are standardised, modalities represent opportunities for experimentation
Funders can and do use instruments and modalities to introduce new behaviours, standards to the research community.
More attention needs to be paid to match instrument/modality to scientific field
Grouping of objectives on to same instrument may disadvantage some fields

13. Funding research systems and evaluating research (Thomas): State of the Art
Project: Characterising researchers
Principal Investigator: Duncan Thomas
Problem(s):To understand how social conditions (policy, funding) manifest in research organisation and affect researchers’ knowledge production (particular epistemic properties with particular people [group, community, utilisation, commercialisation

14. Funding research systems and evaluating research (Thomas): Results of significance to policy
Multiple pressures for change at any one time... not only specific funding pressures (e.g. ERC grant schemes)...
Policy/funding pressures not sole cause of organisational changes but can affect the speed and scope of changes...
Performance of research organisations mediates susceptibility to policy/funding pressures... different orgs affected differently...
Internal funding decisions, steering of career pathways, organisational decisions, prioritisation of various markers of quality... all can be affected by policy/funding

15. Funding research systems and evaluating research (Nedeva)
Project: Effect of policy and funding on science
Principal Investigator: Maria Nedeva
Problem: To understand how policy and funding schemes constitute different pressures for change of the organisational conditions of science and relate organisational trasformation to changes in the nature and characteristics of knowledge.

16. Funding research systems and evaluating research (Nedeva): policy significant findings
Fine grain analyses of research systems provide actionable knowledge that can:
Improve the policy making process by enabling better targeting of incentives;
Inform expectations of desired outcomes and effects;
Enable the development and implementation of better adjusted policy instruments;
Enable the development of policies (and funding schemes) that send clear signals to potential beneficiaries in noisy policy spaces
Have consequences for the way in which the effectiveness and efficiency of science and innovation policy are assessed.

17. Funding research systems and evaluating research (de Rijcke)
Project: Effects of performance indicators on knowledge
creation
Principal Investigator: Sarah de Rijcke
Problem: The project aims to understand intended and unintended effects of the proliferation of metrics on knowledge production

18. Funding research systems and evaluating research (de Rijcke): policy significant findings
The worth of research activities becomes increasingly assessed and defined by their potential for translation into quantitative measures of quality
This general trend holds most strong in fields that have greater degrees of interdependencies, division of labour, need to pool expertise, work with large numbers of collaborators
Indicators of scientific quality such as innovativeness, societal relevance become redefined through their relations to quantitative indicators
There is a risk that ease of comparability afforded by metrics involves trade offs which could undermine long term policy goals such as encouraging innovative, societally relevant and responsible research.