Open Science Dr. Dr.Phil. Rene VON SCHOMBERG Team Leader-Open science policy coordination and development   European Commission DG Research & Innovation A.6-Data, Open Access and Foresight Open Science

Open Science: a new approach to the research process Based on cooperative work and new ways of diffusing and sharing knowledge using digital technologies and new collaborative tools A systemic change to the way science is organised and research is carried out It affects virtually all components of doing science and research, from conceptual work to publishing, from empirical research to data-analysis. Shifting focus from "publishing as fast as possible" to "sharing knowledge as early as possible" 2014 Public consultation on ‘Science 2.0: Science in Transition’ Notes: tiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, Open Science

Open Science – opening up the research process Data- intensive Citizen science Open tabbooks/ workflow Open code Analysis Data-gathering Publication Open data Pre-print Open annotation Conceptualisation Open access Review Science blog Alternative reputation systems Collaborative bibliographies Open Science Source : http://ec.europa.eu/research/consultations/science-2.0/background.pdf

An emerging ecosystem of services and standards It's real! Sci-starter.com Citizens science Data-intensive Open code Runmycode.org Open workflows Pre-print Analysis Publication Review Conceptualisation Data gathering Open data Open access Open annotation Impact Story Alternative Reputation systems Openannotation.org Scientific blogs Collaborative bibliographies

Rank : the lowest need (1) to the highest need (11)

Five lines of potential policy actions Fostering and creating incentives for Open Science Removing barriers to Open Science Mainstreaming and further promoting Open Access policies Developing research infrastructures for Open Science Embedding Open Science in society as a socio-economic driver Notes: tiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, Open Science

Open Science: key issues The European Open Science Cloud Advancing Open Access and Data Policies Alternative systems to evaluate the quality and impact of research Text and Data Mining Towards better, more efficient and more Open Science Fostering Research Integrity Making science more inclusive: Citizen Science Notes: tiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, Open Science

Governance of the European Open Science Cloud Bottom-up governance Federation Legacy and sustainability Leverage of MS investment Trust Governance layer Scale of scientific activity (data-driven science) IPR and privacy protection Big data analytics Data fusion across disciplines High performance computing Data access and re-use Data manipulation and export Data discovery and catalogue Data and service layer High-speed connectivity Super-Computing Data storage Infrastructure layer Lead scientific users… …Long tail of science Applied-engineering Physics Life sciences Earth sciences Economics Social sciences Humanities Citizen science Other disciplines Open Science Source: DG Research and Innovation (2015)

Growth of Open Access Repository Mandates and Policies Policies Adopted by Quarter Number of Policies Research organisation Sub-unit of research organisation Multiple research organisations Funder Funder and research organisation Open Science Source: http://roarmap.eprints.org/

Open Science: From Open Access to Open Scholarly Communication Discovery Analysis Writing Publication Outreach Assessment Elsevier Springer Nature Digital Science Google Wikimedia Public or private initiatives at every level of the research process offering specific services to researchers Layer of "commons" New initiatives allowing the scholarly process to be carried out differently Open Science Source: http://innoscholcomm.silk.co

Towards ‘better science’ – Good, efficient and Open Science research governance changes declaring competing interests replication & reproducibility meaningful assessment effective quality checks credit where it is due no fraud, plagiarism GOOD connected tools & platforms no publ. size restrictions null result publishing speed of publication (web)standards, IDs semantic discovery Re-useability versioning open peer review open (lab)notes plain language open drafting open access CC-0/BY technical changes & standards economic & copyright changes EFFICIENT OPEN Open Science Source: http://blogs.lse.ac.uk/impactofsocialsciences/2015/11/11/101-innovations-in-scholarly-communication/

Open Science Policy Platform and European Open Science Agenda May 2016 Competitiveness Council: "NOTES the establishment of the Open Science Policy Platform by the Commission, which aims at supporting the further development of the European Open Science policy and promoting the uptake by stakeholders of best practices, including issues such as adapting reward and evaluation systems, alternative models for open access publishing and management of research data (including archiving), altmetrics, guiding principles for optimal reuse of research data, development and use of standards, and other aspects of open science such as fostering research integrity and developing citizen science"; Commissioner Moedas will inform the Council biannually on advances of the Platform (which consist of 25 Key stakeholders-European Branch Organisations)

Optimal re-use of Research Data Follow-Up by Stakeholders, EC and MS: 1. As of 2017, Open Data is the default option under H2020- Data Management Plans will be mandatory 2. Evaluation of MS advances on Open Data will be necessary 3. Evaluation of MS advances on Open Data will be necessary; Expert Group on FAIR data will advise DG RTD in course of 2016 Competitiveness Council: 1.Make research data produced by H2020 open by default 2.Encourage MS to Promote data stewardship and implement data management plans 3.Encourage MS and Commission to follow FAIR principles in research programmes and funding mechanisms

European Open Science Cloud Competitiveness Council: "CALLS on the Commission, in cooperation with Member States and stakeholders, to explore appropriate governance and funding frameworks' European Commission-Follow up of the April 2016 Communication on a European Cloud Initiative: Commission will need to have a roadmap for funding of European Open Science Cloud by end of 2016 which requires consultation of Member States.

Open Science Policy Platform European Open Science Agenda: OA publishing models FAIR open data Science Cloud Alternate metrics Rewards & careers Education & skills Citizen Science Research integrity … ERA & framework conditions for actors: European Charter for researchers Code of conduct for Research Integrity Charter for Access to Research Infra … European Commission Open Science Policy Platform DSM & framework conditions for data: Copyright - TDM Data Protection Free Flow of Data … Wide input from stakeholders: ad-hoc meetings and workshops e-platform with wider community reports and independent experts advice EG on open science cloud EG on altmetrics EG on alt. business models for OA publishing EG on FAIR open data opinions context

European Open Science Policy Agenda (1) Foster Open Science: Creating incentives, e.g. Establish an Open Science Policy Platform Promote best practices Launching a European Open Science Monitor Promote a discussion on evaluation criteria of research, prepare for next framework programme All potential actions – under consideration Establish a Policy Platform at European Level and a self-regulation/ clearinghouse mechanism for addressing Open Science issues Co-develop of Open Science Policy (Promote Best Practices, Research Integrity, Citizen Science etc.)

European Open Science Agenda (2) Remove barriers, e.g. European Copyright and Data Protection revisions: foresee appropriate exceptions for research activities - TDM Development of 'alternative' metrics Propose a European "code of conduct" Address low open data-skills amongst researchers and the underuse of professional support (librarians, repository managers etc.) Encourage the research on and collaborative development of 'alternative' metrics – and ultimately their wider use: A growing concern is that researchers should (also) be assessed based on more pertinent criteria than the traditional journal Impact Factor This is particularly relevant to support the career of young scientists Propose a European "code of conduct" setting out the general principles and requirements of how Open Science should affect the roles, responsibilities and entitlements of researchers and of their employers

Flash Report EU Expert Group Altmetrics First release September 2016 Next-generation altmetrics: responsible metrics and evaluation for open science Flash Report EU Expert Group Altmetrics First release September 2016

EU expert group members James Wilsdon, University of Sheffield (chair); Judit Bar-Ilan, Bar-Ilan University; Robert Frodeman, University of North Texas; Elizabeth Lex, Graz University of Technology; Isabella Peters, Leibniz Information Centre for Economics; Paul Wouters, Leiden University

Aims /1 assess changing role of (alt)metrics in research evaluation consider how altmetrics can be developed to support open science engage stakeholders consider implications of metrics for: diversity and equality interdisciplinarity research cultures gaming :

Aims /2 examine implications of: emerging social networks research information systems citation profiles to develop a framework for responsible metrics for research qualities and impacts for evaluation of Horizon 2020 and for wider use in the next framework programme to consider required data infrastructures

Across the research community, the description, production and consumption of ‘metrics’ remains contested and open to misunderstandings.

Quantitative evaluation should support expert assessment. Measure performance in accordance with the research mission. Protect excellence in locally relevant research Keep data collection and analytical processes open, transparent and simple. Allow for data verification Account for variation by field in publication and citation practices Data should be interpreted taking into account the difficulty of credit assignment in the case of multi-authored publications. Base assessment of individual researchers on qualitative judgment. False precision should be avoided   Systemic effects of the assessment and the indicators should be taken into account and indicators should be updated regularly

http://www.hefce.ac.uk/rsrch/metrics/

Responsible metrics Robustness: basing metrics on the best possible data in terms of accuracy and scope; Humility: recognizing that quantitative evaluation should support – but not supplant – qualitative, expert assessment; Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results; Diversity: accounting for variation by field, using a variety of indicators to reflect and support a plurality of research & researcher career paths; Reflexivity: recognizing the potential & systemic effects of indicators and updating them in response.

Ambitions for Open Science More comprehensive measurement of traditional scientific publications (eg Mendeley) Recognizing and capturing the diversity of scientific output including new forms (eg software and blogs) Opening up the whole scientific publication system (open access) and more interactive communication Opening up the very core of knowledge creation and its role in higher education and innovation (participatory science)

Measuring is changing What counts as excellence is shaped by how we measure and define “excellence” What counts as impact is shaped by how we measure and define “impact” Qualities and interactions are the foundation for “excellence” and “impact” so we should understand those more fundamental processes first We need different indicators at different levels in the scientific system to inform wise management that strikes the right balance between trust and control Context crucial for effective data standardization

Call for evidence /1 strong support for development and research of open metrics and altmetrics metrics should complement not replace human judgment of quality altmetrics currently not yet ready for routine use in assessment EU should help develop public sector based metrics diversity key criterion for metrics

Call for evidence /2 portfolios of metrics for societal interaction and impact urgently needed open standards for data and indicator infrastructure context should prevail over technical standards reflexive protection against gaming strategies strong support for Metrics Tide and Leiden Manifesto principles portfolios of indicators to support open science

Report outline Metrics: technical state of the art Use of metrics in policy and practice Data infrastructures and open standards Cultures of counting, ethics and research Next generation metrics: the way forward More information & updates on the progress of the expert panel can be found here: http://ec.europa.eu/research/openscience/index.cf m?pg=altmetrics_eg

To conclude with some problems… Good Metrics for Science 'equals' good metrics for Open Science? -Impacts of research is becoming more important, but what is a good impact? -metrics can never directly measure 'impact' and 'excellence'(whatever the definition)- are metrics not more useful for what they are not created for? Final thesis: Responsible metrics resembles responsible research ( See Von Schomberg, 2013- A Vision of Responsible Research and Innovation)