1. Interoperable Repository Statistics
Les Carr & Tim Brody
University of Southampton
4/10/2019

2. Introduction Background to & history of our interest in statistics
The purpose of the IRS project
Progress so far
Request for participation
4/10/2019

3. Topics of Discussion Statistics - how to get them
Statistics - what services we can build with them
Statistics - what current users want from them now
Statistics - new measures of impact
and significance? and quality?
Statistics - validation for the academic community
4/10/2019

4. Some History
1999 NSF / JISC International Digital Library Project “OpCit”
provide external citation linking service for LANL (as was)
ECS, Cornell, Los Alamos
Supporting study on use of citations
Analysed download patterns from UK mirror wrt citation patterns
Were downloads influenced by citations?
Could citation impact be predicted by downloads?
4/10/2019

5. OpCit Outcome Citebase ‘OAI’ service
Used OAI to obtain article metadata
Used local UK mirror’s file system to obtain
article text (to extract citations)
article download stats from web logs
No standard way to obtain necessary data.
By now most repositories accept web spiders.
Still no agreement with arxiv.org to harvest central download data
4/10/2019

6. Stakeholders Authors - Researchers Encouragement to research
Encouragement to archive
Researchers
New Discovery & Navigation methods
New Filter mechanisms
4/10/2019

7. Stakeholders (II) Repository Administrators Fund holders
Management & maintenance decisions
Marketing, feedback to research managers
Fund holders
Assess impact, inform future funding decisions
4/10/2019

8. Internal Deposit Statistics
Who is depositing what, how frequently
Managers can collect info to feedback on effectiveness
Archives.eprints.org provides simple growth data
OpenDOAR could collect information on best practice
Evaluate policy outcomes
Are mandates necessary?
4/10/2019

9. Management Stats (the Easy Way)
Google Analytics
Repository overview
Individual document downloads over time
JavaScript invokes external service to gather stats
4/10/2019

10. Project Aims
investigate the requirements for UK and international stakeholders
design an (API) for gathering download data
build software
distribution and collection software for repositories
generic analysis and reporting tools
This project will investigate the requirements for UK and international stakeholders (in the context of open access services being global) and build distribution and collection software for repositories as well as generic analysis and reporting tools. The project will design an application programming interface (API) for gathering download data, and implement this for data providers who are using common IR software.
4/10/2019

11. Scenario 1
Forty physicists collaborate on a paper which is deposited into each of their institutional repositories plus arxiv.org
Each repository reports to its author that it has received n downloads.
How can they be aggregated?
4/10/2019

12. Scenario 2
Two repositories report that a paper has received 50 downloads.
Have they both filtered out spiders in the same way? Self-downloads? Repeated downloads from the same IP?
Are abstract and PDF downloads treated equivalently?
How can they be compared?
4/10/2019

13. Participants Project Partners International Panel
ECS (Leslie Carr, Stevan Harnad, Tim Brody), U Tasmania (Arthur Sale), Counter (David Goodman, Long Island U), Key Perspectives (Alma Swan)
International Panel
Rob Tansley (DSpace), Herbert Van de Sompel (OAI), Alberto Pepe (CERN), Laurent Romary (CNRS), Bill Hubbard (SHERPA), Leo Waaijers (DARE), Sune Karlsson (LogEc), Andrew Bennett (APSR)
4/10/2019

14. Current Progress Report on stakeholder requirements Comments requested
35 people representing 18 different institutions interviewed
Their priorities:
Origin of access (country, domain, institution)
Timing of access (date, time-series, cumulative)
Comments requested
4/10/2019

15. Comments Sought
Are the pre-occupations of new repository managers correct?
How much effort should be devoted to new bibliometrics?
We need international feedback and support to go there!
Please join expert panel!
4/10/2019

16. Impact for (e.g) UK Research Assessment exercise But to simplify:
Performed in 2001, and next in 2008
IRRA project defines roles for repository (collecting research evidence, providing it for panels)
But to simplify:
RAE results  citation impact
citation impact correlated to downloads
But academic community still very wary of ‘web logs’
but citation used to be the only auditable use of an article
4/10/2019

17. Message to UK Fundholders
Let many flowers bloom (Harnad)
i.e. Freely provide download and citation statistics
So each community can define its own statistical measures of quality, impact and success
Physicists can use journals
Computer Scientists can use conferences
All disciplines can see the impact of their own departments, individuals, projects etc.
4/10/2019

18. Next Steps Looking for feedback Looking for agreement
Looking for collaboration
How can we join in?
4/10/2019