1. Northumbria Conference 2015
IRUS-UK: enabling institutions to quantify usage of open access research
As the research environment and publication routes develop, we are seeing an increased importance placed on Open Access materials such as those available via institutional repositories. Institutional repositories enable an institution to share their research outputs with a wider audience and gain a clearer understanding of their organisational research profile. Depositing materials into the institutional repository is not the end point in the process though; what happens to the materials once they're available? This is where IRUS-UK can help.
Jo Alcock
Northumbria Conference 2015

2. Focus of today's presentation
What is IRUS-UK?
Why? Who? What? How?
IRUS-UK reports - screenshots
How IRUS-UK is being used
User feedback
Lessons learned and future priorities
Today I’m going to talk to you about what IRUS-UK is; its purpose, the people responsible, what it is, how it works, and the development approach.
Then I’ll show you a demo of some of the reports available in IRUS-UK by sharing some screenshots.
Following that I’ll share how IRUS-UK is currently being used including details of use cases and examples of user feedback.
Finally I’ll share some of the lessons we have learned and our future priorities.

3. What is IRUS-UK?
A national aggregation service for UK Institutional Repository Usage Statistics:
Collects raw download data from UK IRs for *all item types* within repositories
Processes raw data into COUNTER-compliant statistics
IRUS-UK is a national aggregator of repository usage statistics. Its purpose is to collate and provide institutions with access to reliable, accurate standard-based usage statistics for their institutional repository.
This enables them to gain a better understanding of the usage of their institution's research, which they can then share with key internal and external stakeholders.
IRUS-UK collects data on all downloads from the institutional repository, (not just articles), and processes the data into COUNTER-compliant usage statistics.

4. Who is responsible for IRUS-UK?
Funded by Jisc
Team Members:
Jisc– Project & Service Management & Host
Cranfield University - Development
Evidence Base, Birmingham City University – User Engagement & Evaluation
Bringing together key repository services to deliver a connected national infrastructure to support OA
IRUS-UK is funded by Jisc as part of the Repository Shared Services Project, which includes services to support all stages of workflows for UK institutional repositories such as deposit, metadata, discovery, and usage statistics.
The IRUS-UK team is a consortium utilising expertise from the same group supporting JUSP, the Journal Usage Statistics Portal. This includes:
Jisc, who host the service and provide project and service management
Cranfield University, who provide technical development
Evidence Base at Birmingham City University, who support evaluation and user engagement

5. The IRUS Tracker
Small piece of code added to repository software, which employs the ‘Tracker Protocol’
Patches for DSpace (1.8.x, 3.x, 4.x) and Plug-ins for Eprints ( x)
Implementation guidelines for Fedora
Gathers basic data for each download and sends to IRUS-UK server
Adheres to the processing rules specified in:
Release 4 of the COUNTER Code of Practice for e-Resources
Release 1 of the COUNTER Code of Practice for Articles
IRUS works by adding a small piece of code to repository software which employs the ‘Tracker Protocol’. The tracker is available for DSpace, EPrints, and we have implementation guidelines for Fedora.
The tracker gathers basic data for each download and sends it to the IRUS-UK server. The tracker protocol adheres to Release 4 of the COUNTER Code of Practice for e-Resources and Release 1 of the COUNTER Code of Practice for Articles.
On a daily basis, the data goes through a number of filters and checks before being added to the portal, and is then reviewed again at the end of each month. These process help ensure robot activity or other unusual activity is spotted and removed to improve the accuracy of the data.

6. Viewing IRUS data Web User Interface - The IRUS-UK Portal
Access currently behind Shibboleth authentication/authorisation
Wide range of views, slicing and dicing stats from the IRUS-UK database
Reports available for download as CSV/Excel spreadsheet files
SUSHI service
SUSHI Lite API
Under development by the NISO SUSHI Lite Technical Report Working Group (
Allows retrieval of stats snippets to be embedded into Repository (and other) web pages
Once the data has been checked it is accessible via the web user interface in the IRUS-UK portal, which is accessed via Shibboleth authentication. Within the portal there are a wide range of views, showing the data in various different forms of reports. All the reports are available for download as CSV/Excel spreadsheets.
The IRUS data can also be viewed via SUSHI service, and a SUSHI Lite API which is currently being developed in collaboration with the NISO SUSHI Lite Technical Report Working Group. The API allows retrieval of statistics to embed in other places (e.g. repository).

7. The development approach for IRUS
Delivering a service for the community
Responding to evolving user requirements
Iterative and pragmatic approach to development
IRUS is developed in collaboration with users, and has been from the beginning. The service is developed for the community, and responds to evolving user requirements. These are gathered via regular surveys, feedback on webinars, via conversations by and social media, and from our Community Advisory Group. Regular two-way communication with users is a large part of our work.
IRUS uses an iterative and pragmatic approach to development, and maintains a development wishlist that is regularly reviewed to take into consideration the changing environment (e.g. new technological developments, new collaboration opportunities).

8. IRUS-UK: Home
Over the next few screens I’ll show you some of the key features in IRUS-UK.
The home page shows how many repositories we currently collect data from and gives an overview of the downloads recorded so far.

9. IRUS-UK: Repository Statistics
The Repository Statistics Report shows an overall summary by repository of the number of items downloaded, and the total downloads IRUS-UK has recorded since they joined. It also includes details of their joining date and the downloads recorded so far in the current month.

10. IRUS-UK: Item Type Statistics
The Item Type Statistics report shows a summary across all participating repositories of the different item types downloads have been recorded for, the number of items for each item type, and the downloads for each item type.

11. IRUS-UK: Repository Report 1
The Repository Report 1 shows the number of successful item downloads requests by month and repository. You can select on the previous screen which months you would like to look at, and the report will show usage across all repositories for those months. You can sort within the portal, or you can export to Excel for further analysis (e.g. to support benchmarking).

12. IRUS-UK: Item Report 1
The Item Report 1 goes into more detail for each repository. From this report you are able to view downloads at an item level. Again you can select a date range, and sort within the portal (e.g. to see which items have received the most downloads). From this page you can also get to a more detailed view in the Item Statistics report which I’ll show in a few minutes.

13. IRUS-UK: Journal Report 1
The Journal Report 1 is a relatively new report which shows data at a journal level. I searched for Performance Measurement and Metrics and the result shows how many downloads repositories within IRUS-UK have received for articles published in Performance Measurement and Metrics. You can also see which articles if you click for more details.

14. IRUS-UK: Article Report 4
The Article Report 4 shows the data at article level. This screenshot is from the previous Journal Report 1 and it shows which articles from Performance Measurement and Metrics have been downloaded in the selected date range. If an article is included in more than one IRUS-UK repository they will each be shown here.

15. IRUS-UK: Duplicate DOI Report
The Duplicate DOI Report is useful for repositories wanting to ensure they have accurate metadata. The report shows if any of the items that have recorded a download have duplicate DOIs. Repositories can than investigate this and, if appropriate, amend the DOIs to get rid of the duplicate.

16. IRUS-UK: Search
You can search for items within IRUS-UK by title or author, and can search across all repositories and items types or specify a specific repository or item type if known. In this particular example I am looking for an item in Manchester Scholar repository as shown in the screenshot. The search will bring back a list of any relevant results which you can then click through to see item level statistics.

17. IRUS-UK: Item Statistics
The Item Statistics shows a more detailed view of usage for a particular item. You can see in the screenshot that there is both tabular and graphical representation of downloads, as well as more metadata. For some items (including this one) there is also Altmetrics information which includes use on social media.
Hopefully that quick tour has shown you some of the data you can get from IRUS - I haven’t included all reports, and more are being added all the time via user requests and feedback.

18. Providing standards-based, reliable repository statistics
How is IRUS-UK used?
Use cases:
Providing standards-based, reliable repository statistics
Reporting to Institutional Managers
Reporting to Researchers
Benchmarking
Supporting Advocacy
Repositories use IRUS-UK in a number of different ways. Last year we took time to speak to a number of our users to understand more about how they use IRUS to support them in their work. From these interviews we produced a series of use cases, each of which includes an overview of what repository managers (and other repository staff) want to be able to do, how IRUS-UK can help, and quotes from repositories using it in this way.
One of the key themes from these was the ability to provide standards-based, reliable repository statistics. Knowing that the data goes through a series of filters and is COUNTER-compliant helps repositories know they can rely on the data to use within their institutions. Many use the statistics to report internally such as to institutional management or to researchers. A number use the data for benchmarking purposes, and are able to compare their download statistics with other repositories of a similar size/focus or from institutions they benchmark against in other areas. Some also use the data to support advocacy, for example using the data to feed into newsletters, s, and online statistics pages.
These use cases are all available via the IRUS-UK website at the URL on the bottom of the slide.

19. User feedback on IRUS-UK
Best thing about IRUS-UK:
Reliable, authoritative statistics
Ability to benchmark against others
Easy to use
Improved statistical reporting for 92%
Saved time collecting statistics for 63%
Enabled reporting previously unable to do for 83%
84% currently (or plan to) use IRUS-UK for benchmarking
Data from 2015 IRUS-UK User Survey
As mentioned earlier, we regularly collect user feedback on IRUS-UK. In the 2015 user survey, we asked people what the best thing about IRUS-UK was and the responses were largely within the themes of reliable, authoritative statistics, the ability to benchmark against others, and the ease of use.
We asked about improvements to their work due to IRUS-UK and found that 92% of respondents reported that IRUS-UK has improved statistical reporting; 63% reported that IRUS-UK has saved time collecting statistics, and 83% reported that IRUS-UK has enabled reporting that they were previously unable to do. 84% of respondents either currently use IRUS-UK for benchmarking or plan to.

20. What do users say about IRUS-UK?
“The set up was quick and painless”
“High quality download stats that conform to a standard, that we can compare to other universities for benchmarking.”
“Provides useful usage stats for librarians.”
Here are a sample of quotes we have received from users; the first showing the set up is quick and painless; the second showing that IRUS-UK provides them with high quality download statistics that conform to a standard that they can compare to other universities for benchmarking, and that it simply provides useful usage statistics for librarians.

21. What lessons have we learned?
Engagement
Aim to minimise any extra work required of the IR managers – keep barriers for participation low
Consider all suggestions for improvements and regularly revisit and reprioritise based on user feedback and other opportunities
Relate functionality to the relief of pain for users
Standards
Base developments on appropriate standards and give something back to the standards bodies
Iterative development
Don’t wait until you have a full solution – start with what is there
During the time we’ve been working on IRUS-UK, we’ve learned a number of lessons that may be useful to others.
The first is around engagement. Engagement is a really key part of what we do, and crucial to the development of IRUS-UK. It’s crucial at all stages: in terms of ensuring barrier for participation is low for repository managers; to consider improvements and regular revisit and reprioritise based on user feedback and other opportunities; and to relate the functionality of IRUS-UK to the relief of pain points for our users (as evidenced in our use cases).
The second lesson is around standardisation which is crucial to usage statistics. We base IRUS-UK development on appropriate standards and in addition actively work with the standards bodies to help develop the standards or provide feedback on how they apply in practice. By working at a broader level than just one institution, we are able to provide a unique perspective on this.
The final lesson is about iterative development, which is the approach we use in IRUS-UK. The work initially stemmed from a project to bring together publisher and repository usage statistics, but the barriers to this were very high so we initially tackled the institutional repository side of things and realised there were great advantages to getting that right first. We continue to develop that whilst also looking for future opportunities to bring together publisher data.

22. What are the future priorities for IRUS?
Increase number of participating repositories in IRUS-UK
Implement IRUS tracker for other repository and CRIS software
Expand views of data & reports in response to user requirements
Leverage supplementary sources of metadata to enhance data in IRUS
Incorporate more altmetrics
International collaboration
So what are our future priorities?
We aim to increase the number of participating repositories in IRUS-UK (we currently have 60% of all eligible repositories and would like to move closer to 100%). We would also like to implement the IRUS tracker in other repository and CRIS software to enable others to join, and to benefit from IRUS. We are currently in communication with other suppliers and hope to be able to support them in implementing functionality in future.
We also intend to continue our development on IRUS-UK in terms of the views of data and reports, and this will of course be in response to user requirements. We plan to leverage supplementary sources of metadata as opportunities arise to enhance the data in IRUS.
You saw a sample of the Altmetrics ‘donut’ and we plan to incorporate more Altmetrics in future.
We would also love to bring IRUS to other countries. IRUS-UK is funded by Jisc for the UK academic community, but the IRUS tracker can be used by other services and we’d love to hear from you if you are from outside of the UK and would like to have something similar to IRUS-UK.

23. Contacts & Information
Project web site:
Contact IRUS-UK:
@IRUSNEWS
You can find lots more information about IRUS on our website, and if you’d like to contact us you can us. We’re also on Twitter so feel free to follow us and contact us that way if you prefer.