1. EU Expert Group Altmetrics
Next-generation altmetrics: responsible metrics and evaluation for open science
EU Expert Group Altmetrics

2. EU expert group members
James Wilsdon, University of Sheffield (chair); Judit Bar-Ilan, Bar-Ilan University; Robert Frodeman, University of Texas; Elizabeth Lex, Graz University of Technology; Isabella Peters, Leibniz Information Centre for Economics; Paul Wouters, Leiden University Team Leader-Open science policy coordination and development: Rene von Schomberg

3. Aims /1 assess role (alt)metrics in research evaluation
consider how altmetrics can be developed for open science
engage stakeholders
consider implications metrics for:
diversity and equality
interdisciplinarity
research cultures
gaming :

4. Aims /2 examine implications of:
emerging social networks
research information systems
citation profiles
explore altmetrics for impacts, research actions, in Horizon 2020 and in next framework programme
consider required data infrastructures

5. The Leiden Manifesto
Quantitative evaluation should support expert assessment.
Measure performance in accordance with the research mission.
Protect excellence in locally relevant research
Keep data collection and analytical processes open, transparent and simple.
Allow for data verification
Account for variation by field in publication and citation practices
Data should be interpreted taking into account the difficulty of credit assignment in the case of multi-authored publications.
Base assessment of individual researchers on qualitative judgment.
False precision should be avoided (eg. the JIF). 
Systemic effects of the assessment and the indicators should be taken into account and indicators should be updated regularly

6. Responsible metrics
Robustness: basing metrics on the best possible data in terms of accuracy and scope;
Humility: recognizing that quantitative evaluation should support – but not supplant – qualitative, expert assessment;
Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results;
Diversity: accounting for variation by field, using a variety of indicators to reflect and support a plurality of research & researcher career paths;
Reflexivity: recognizing the potential & systemic effects of indicators and updating them in response.
The Metric Tide

7. Measuring is changing
What counts as excellence is shaped by how we measure and define “excellence”
What counts as impact is shaped by how we measure and define “impact”
Qualities and interactions are the foundation for “excellence” and “impact” so we should understand those more fundamental processes first
We need different indicators at different levels in the scientific system to inform wise management that strikes the right balance between trust and control
Context crucial for effective data standardization

8. Report outline Introduction Metrics: technical state of the arts
Use of metrics in policy and practice
Data infrastructures and open standards
Cultures of counting, ethics and research
Next generation metrics: the way forward

9. Traditional metrics
Based on citation and publication counts are not sufficient
Citations take time to accumulate
IF often used as a proxy for citation count
h-index
Disciplinary differences in publication and citation culture
Ignore societal impact
DORA, Leiden Manifesto, Metric Tide

10. Altmetrics
Intend to capture and measure additional aspects of scholarly information
Altmetrics Manifesto
New forms of communication reflect and transmit scholarly impact
Expand our view of what impact looks like
Altmetric platforms
Increased visibility of researchers/publications
Exposing research to the public
Involving the public
Discussion/commenting
Readers vs. authors
Altmetric events can be measured/counted

11. Altmetrics - challenges
Coverage
Transparency
Validity
Dynamics
Disciplinary differences
Gaming
Acceptance
Research community
Decision makers :