Building on sand? Using statistical measures to assess the impact of electronic services Dr Angela Conyers evidence base research & evaluation UCE Library Services UCE Birmingham

Why collect usage statistics? What statistics to collect? How to collect them? How to analyse them? How can libraries be assisted in this task? Key questions

E-measures project funded by HEFCE working with 25 UK higher education libraries NESLi2 study of usage statistics funded by JISC working with 17 UK higher education libraries

Because they’re there Because SCONUL has asked for them To help with promotion and user support For budgeting and decision-making To aid bench-marking Why collect usage statistics?

1. Identify type of resource: Serials Databases E-books Digital documents Virtual visits Electronic enquiries What usage statistics to collect?

2. Identify reliable statistics: COUNTER compliance: Serials: JR1 report – number of full-text article requests – 40 publishers/suppliers Databases: DB1 report – total searches and sessions by month and database – 13 publishers/suppliers - DB2 report – total searches and sessions by month and service – 15 publishers/supplier E-books: new draft COUNTER code of practice What usage statistics to collect?

Identify sources of statistics Collect passwords Download Establish where gateway statistics to be added Encourage all suppliers to be COUNTER compliant How to collect?

1.Identify supplementary information required: –Cost of deal (subscribed titles and e-access costs) –Lists of subscribed titles –Number of FTE users –Total library serials budget How to analyse?

2. Identify groupings: –Usage range –Price band –Subject category –Subscribed/unsubscribed titles How to analyse?

Nil and low use (under 10 requests) nil use 1-4 requests 5-9 requests Medium use (10-99 requests) requests requests High use (100 or more requests) ,000 plus Usage range

Unpriced: those for which no price could be found Low price – under £200 Medium price - £ High price - £ Very high price - £1,000 and over Price band

Subject category: STM (science, technology & medicine) HSS (humanities and social sciences) Subscribed and unsubscribed titles Other groupings

Average cost per request – all requests Average cost per request – subscribed titles Average cost per request – unsubscribed titles Cost per FTE user Cost per title Yield per £ for access to unsubscribed titles Assessing value for money

NESLi2 study main recommendations to the JISC: –Clearer, more transparent deals –Review of pricing and cancellations policies –Portal site for publisher usage statistics –More support to libraries in monitoring usage- toolkit, workshops etc How can libraries be assisted with the task?

NESLi2 study main recommendations to the higher education library community: –Libraries to ensure they have sufficient internal resources to manage and analyse usage data –Libraries to be encouraged to take a more joined-up approach to the management of print and e-journal resources How can libraries be assisted with the task?

Workshops Tool-kits Benchmarking groups Outsourcing Angela Conyers The future?

Some results –full-text article requests

Some results – costs

Some Conclusions Number of full text article requests considerably higher in large old universities in this study Overall increase in usage Balance of usage and costs generally led to similar range of costs across large and small universities Low average cost per request Small % of titles generated high usage Low use titles were mostly low price High use titles were generally high cost STM titles most heavily used. Data limitations