Experiences on the Ground: usage stats as a practical tool in the Library Jill Taylor-Roe Newcastle University Library UKSG Usage Statistics Training Seminar 27 June, 2006

There is nothing new under the sun…  Libraries have been collecting and using usage data for many years  Initial core data – based around print based services and physical presence of users in Library  Current data more likely to focus on remote use of e-services

Usage and Value  What do we measure and why?  How does it help?  Are we measuring the right things?  Are we making correct assumptions from the data?

What do we measure and why?  Anything which tells us about the utilisation of services in our Library  Data which might help to demonstrate whether we are getting a reasonable return on our financial investment  Data which may help to support cases for funding  Data which can inform our strategic planning

Usage data can be…  A practical management tool - helps assess effectiveness  A means of assessing performance  A planning tool  A political tool  A means of helping to demonstrate VFM  A means of highlighting user satisfaction  Support for funding bids

Traditional Library Usage Data Academic Year 94/9599/0003/0404/05 Issues574,618760,128505,044759,091 ILL Requests35,77123,33513,18012,939 Photocopies4,431,3974,498,0001,746,6481,875,358 Visits to the Library1,243,9701,175,2491,012,6701,121,759

Graphs can (sometimes!) be helpful

New Library Performance Indicators  Gross Library Expenditure per staff/student FTE  Gross Information Expenditure per staff/student FTE  Library Staff Expenditure per staff/student FTE  Cost per seat hours per annum  Cost per use of electronic journals  Expenditure on e-resources as a % of total expenditure  Uses per day of e-resources per staff/  student FTE

E-Journal Key Performance Indicators per NUL (1) METRICSCOSTS Total No. titles Total package Total full text downloadsCost as % of total serials budget Mean cost per title Max downloads (month)Mean cost per F/T download Min downloads (month)Cost per FTE user*** Mean downloads (year) PDF full text downloads HTML full text downloads PDF/HTML ratio Downloads per FTE user* Total turnaways (titles not included in licence) Medium use: (11-99 hits) High use: (= >100 hits) Top 20 best performing titles as % of total hits Low-use: (0-10 hits)

E-Journal KPI profile with data

Graphs CAN sometimes be helpful

What are these usage stats telling us?  Average cost per f/t download for large STM packages now around £ £1.75  Average cost per use of e-journals = £0.93  Usage still appears to be increasing year on year  Humanities and Social Sciences packages seem to be less well used than STM  Initially, titles we had as print were always better used than unsubscribed content, but now many of the latter are beginning to show heavy use

How does this help? (1)  Given the significant sums we are investing in e- resources – cost per use figs appear to show good value for money  Total f/t downloads per annum are still rising – ( may offset decline in traditional ILL)  Journals collection has effectively doubled in size – more choice for users  Because usage is increasing year on year – MAY help persuade paymasters to maintain and preferably increase funding

How does this help? (2)  Dealing with practical queries:  “Why are you wasting money on all these second rate journals – you should only buy the top IF rated titles in each field and cancel all the rest.”  QUESTION: does high IF correlate with high use?

How does this help? (3)  “There is a new journal called Nature Physics. This is essential for my research so please take a subscription out immediately.”  QUESTION: would it be worth trying to find funds for this title?

Building up evidence…  Average cost per use of current Nature Journal Collection: £0.74 in 2005  Overall package usage has increased year on year by over 30%  Would be worth securing trial access to the title to see how it compares with the rest  This MAY provide enough supporting evidence to warrant making a case for funding a subscription.

How does this help? (4)  “ SD is excellent – but it would be even better if you bought the back-files then we can search back to volume 1.”  QUESTION – is it worth looking at the backfiles? How could we make a case to buy them??

Building up evidence…  How many metres do our current print back-runs occupy?  If we were to charge an annual rental for this space what would it be?  Have we any evidence of usage of the print runs?  Can we trial access to the back runs?  Can we separate out usage of the backfiles from the current files?  What level of usage of the digital backfile would support a case for purchase?  Would it need to be significantly cheaper than the cost of maintaining the print runs on the shelves?

Are we measuring the right things?

Managerial Realities… (1)  Library budgets still fail to keep pace with the demands placed upon them  Most Libraries are competing for funds with other services and academic schools  Demand for new resources, especially (but not exclusively) e-resources is insatiable

Managerial Realities… (2)  In the print environment it was difficult to assess claims that particular resources were “essential” both pre and post purchase  E-usage stats are a powerful new tool to help us assess the value of the resources which are absorbing increasingly large percentages of our budgets  We are constantly having to respond to new initiatives requiring us to demonstrate the cost effectiveness of our services

Summary points  We need usage data in order to…  Provide KPIs for our paymasters  To inform collection development strategies  Evaluate whether the resources we have selected are providing good VFM  Support cases for future funding

Are we making the correct assumptions from the data?  At regular intervals you should always stop and think:  Are we taking a fair and balanced view?  Is there critical data that we are missing?  Are there any significant flaws or omissions in the data we are using?

More work is needed  More info re usage of backfiles vs current issues  Clarify the position re subscribed and non-subscribed content in usage reports  More info on turnaways – these too could influence future purchases  More comparative data to facilitate benchmarking between libraries  Greater number of publishers to be fully COUNTER compliant  Data gathering process needs further streamlining

Concluding thoughts  Usage metrics have come a long way in a short time  They are far from perfect, and need to be interpreted with care, but they are important to librarians in our role as information managers  They are important to publishers as a means of understanding and refining their market  We need to acquire and share more expertise in collating and analysing usage data  We need more and better usage data – particularly comparative data so that we can benchmark effectively  Openly collaborative work between librarians and publishers will help us to move forward faster