1. Dana Thomas, Ontario Council of University Libraries Alan Darnell, Ontario Council of University Libraries Understanding Use of Networked Information Content: MINES for Libraries® Implementations at Scholars Portal Terry Plum, Simmons College Martha Kyrillidou, Association of Research Libraries

2. Why measure digital content use? More than 60% of the library materials expenditures in major research libraries is spent on electronic resources This proportion is even higher for smaller college and university libraries The amount is yet higher considering consortial and state-wide and nation-wide deals from ARL Statistics Source: http://interactive.arlstatistics.org CNI Fall 2011 Membership Meeting1

3. Ontario Council of University Libraries CNI Fall 2011 Membership Meeting2

4. Consortial Licensing Canadian Research Knowledge Network – National site licensing agency – Major publishers (e.g. Elsevier) OCUL-IR – Standing committee of OCUL – Large and small publishers Knowledge Ontario – Multi-sector CNI Fall 2011 Membership Meeting3

5. Scholars Portal Shared technology infrastructure hosted at University of Toronto Shared services (e.g ILL, SFX, VR, RefWorks) Locally loaded content (aggregation and archiving) – Journals – Books – Microdata – GIS CNI Fall 2011 Membership Meeting4

6. OCUL Scholars Portal In 2004 the digital content was primarily e-journals (8.2 million articles from 7,219 full text electronic journals) In 2010 Scholars Portal has 23 million journal articles from over 11,200 full text electronic journals Extensive growth has occurred in other formats: 350,000 e- books, statistical and geospatial data collections SFX, the open-URL resolver that connects users to this digital content, is an integral component of the 2010 implementation of MINES for Libraries® since it acts as the delivery mechanism by which the patron encounters the survey CNI Fall 2011 Membership Meeting5

7. What is MINES? Action research – Historically rooted in indirect cost studies – Set of recommendations for research design – Set of recommendations for web survey presentation – Set of recommendations for information architecture in libraries – Plan for continual assessment of networked electronic resources – An opportunity to benchmark across libraries CNI Fall 2011 Membership Meeting6

8. Library User Survey CNI Fall 2011 Membership Meeting7

9. Library User Survey Patron Status CNI Fall 2011 Membership Meeting8

10. Library User Survey Affiliation CNI Fall 2011 Membership Meeting9

11. Library User Survey Location CNI Fall 2011 Membership Meeting10

12. Library User Survey Purpose 11CNI Fall 2011 Membership Meeting

13. Goals for MINES for Libraries at OCUL 2010-11 demonstrate the value of the library secure resources as needed understand use of resources within the library, on campus outside the library, and off campus understand use of resources across disciplines and user groups understand why people are using electronic resources Track the trajectory of the use of the resources since 2004-05 CNI Fall 2011 Membership Meeting12

14. OCUL MINES 2004 vs 2010 OCUL I : 2004 Randomly selected 2 hour periods each month for a year Mandatory OCUL II: 2010 SFX as delivery mechanism Systematic sampling (every nth) Mandatory and optional versions Additional question: contact pool 13CNI Fall 2011 Membership Meeting

15. Limitation: Variations in SFX as delivery mechanism 14 InstitutionUses az list?SFX from OPAC?Google Scholar?Citation linker?bx? PRIMO?/ Discovery layer? ALGOMA BROCKYESNOYES CARLETONNO YES DURHAM GUELPHYESNOYES LAKEHEADYES NO LAURENTIANYES NO LAURIERYESNOYES MCMASTERYES NO NIPISSINGYESNOYES NO OCADYESNO YESNO OTTAWAYESNOYES NO QUEENSNO YES NOYES RMC RYERSONYES NO TORONTOYESNOYES NO TRENTYESNOYES NO UOITNO YES NO WATERLOOYES WESTERNYES NO WINDSORYES NO YORKNO YES NO CNI Fall 2011 Membership Meeting

16. MINES survey using LimeSurvey software. Ryerson's optional survey User gets directed to the proper MINES Survey or to their resource, as appropriate. CNI Fall 2011 Membership Meeting15

17. SFX source with Get it! link CNI Fall 2011 Membership Meeting16

18. SFX menu with full text target and other target services CNI Fall 2011 Membership Meeting17

19. Introduction to OCUL Report and Institutional Reports 34,776 complete responses across all OCUL institutions 4,255 surveys from mandatory and 30,521 from optional implementations Summarized as: – PDF reports – Excel files – SPSS dataset(s) – SPSS syntax file(s) CNI Fall 2011 Membership Meeting18

20. Distribution of OCUL Partner Institutions' Enrollments as of July 2010, Responses to MINES for Libraries(R), and SFX Statistics (Clicks) from 2/16/2010 to 2/17/2011 CNI Fall 2011 Membership Meeting19

21. ARL Final Report Available at http://www.libqual.org/documents/LibQual/publications/MINES_ OCUL2011.pdf http://www.libqual.org/documents/LibQual/publications/MINES_ OCUL2011.pdf 20CNI Fall 2011 Membership Meeting

22. Findings: Frequency of Use of Electronic Resources by User Status (10/11) CNI Fall 2011 Membership Meeting21

23. Findings: Frequency of Use of Electronic Resources by Primary Purpose of Use (10/11) CNI Fall 2011 Membership Meeting22

24. Findings: Frequency of Use of Electronic Resources by Location (10/11) Off Campus use up 23.5% On Campus not in library down 15.9% Library use down 7.7% CNI Fall 2011 Membership Meeting23

25. Findings: Frequency of Use of Electronic Resources by Affiliation (10/11) CNI Fall 2011 Membership Meeting24

26. MINES for Libraries ® for CNI SFX provided information to add six additional variables for each survey response: ISSN / ISBN Normalized Publisher or Vendor Name Database Name Resource Type: EJournal, ebook. Database Subscription Type: Consortial, Local, or Free/Open Access Consortia: OCUL, CRKN, Knowledge Ontario CNI Fall 2011 Membership Meeting25

27. Consortia / Local / Open Access CNI Fall 2011 Membership Meeting26

28. Consortial Breakdown CNI Fall 2011 Membership Meeting27

29. CNI Fall 2011 Membership Meeting28

30. Does purpose of use vary by subscription type? CNI Fall 2011 Membership Meeting29

31. Resource Type CNI Fall 2011 Membership Meeting30

32. MINES: Publisher / Vendor Breakdown CNI Fall 2011 Membership Meeting31

33. Which resources are used for funded research? CNI Fall 2011 Membership Meeting32

34. Publisher use by discipline CNI Fall 2011 Membership Meeting33

35. Are there differences in recommendations from librarians vs. faculty? CNI Fall 2011 Membership Meeting34

36. MINES: Are there differences in recommendations from librarians vs. faculty? CNI Fall 2011 Membership Meeting35 n=2467 n=6336

37. Publisher/Vendor Uses in MINES for Libraries ® CNI Fall 2011 Membership Meeting

38. Within each publisher what is the breakdown by user role? CNI Fall 2011 Membership Meeting

39. Within each user role what is the breakdown by publisher? CNI Fall 2011 Membership Meeting

40. Within each publisher what is the breakdown by affiliation? CNI Fall 2011 Membership Meeting

41. Within each affiliation what is the breakdown by publisher? CNI Fall 2011 Membership Meeting

42. Within each publisher what is the breakdown by purpose of use? CNI Fall 2011 Membership Meeting

43. Within specific purpose of use what is the breakdown by publisher? CNI Fall 2011 Membership Meeting

44. Practical Implications/Value What are the implications of running the survey in session mode vs link-resolver? How does the use of consortial products compare to that individually-licensed content? What conclusions can be drawn about the efficacy of surveying users via an open-URL resolver? How will MINES fit into the overall assessment program for OCUL? How useful is the contextual information from MINES in understanding user workflows? Are there any lessons learned regarding discoverability? 43CNI Fall 2011 Membership Meeting

45. What else would be useful to query? – the relation between mandatory and optional survey methods – informing collection development decisions, – understanding the impact of open access, etc. – …… 44CNI Fall 2011 Membership Meeting

46. Credits The presenters thank: – Catherine Davidson for her work contributing to the slides – Vidhya Pathasarathy, Bilal Khalid, Agnes Bai, Gary Roebuck and other systems staff for their work on the survey – Kevin Wong for writing a program to complete a first pass at parsing target urls CNI Fall 2011 Membership Meeting45

47. Thank you Alan Darnell Project Director, Scholars Portal alan.darnell@utoronto.ca Dana Thomas Evaluation and Assessment Librarian, Scholars Portal dana.thomas@utoronto.cadana.thomas@utoronto.ca Terry Plum, Assistant Dean, Simmons Graduate School of Library and Information Science terry.plum@simmons.edu terry.plum@simmons.edu Martha Kyrillidou Senior Director, Statistics and Service Quality Programs, Association of Research Libraries martha@arl.org martha@arl.org 46CNI Fall 2011 Membership Meeting