1. Web-Based Usage Surveys MINES for Libraries TM Brinley Franklin University of Connecticut Brinley.franklin@uconn.edu Terry Plum Simmons GSLIS Terry.plum@simmons.edu CNI Fall 2005 Task Force Meeting Phoenix, AZ December 5, 2005

2. CNI Fall 2005 Task Force Meeting2 Overview Research problem –Assessment of networked electronic services –Open access – beyond vendor supplied data –Issues with web-based surveys in assessing the use of networked electronic services –Digital libraries – how to assess Project –MINES for Libraries TM Web based usage and user survey –Assessment infrastructure and gateway Examples – data from: –Medical libraries –Large academic libraries –Consortia

3. CNI Fall 2005 Task Force Meeting3 Assessment of networked services Vendor supplied data - standards –COUNTER - Counting Online Usage of Networked Electronic Resources http://www.projectcounter.org/ –ICOLC – International Coalition of Library Consortia http://www.library.yale.edu/consortia/ –ISO – International Standards Organization ISO 11620 Library Performance Indicators http://www.iso.org/ –NISO – National Information Standards Organization NISO Z39.7 Library Statistics http://www.niso.org/ –ARL E-Metrics http://www.arl.org/stats/newmeas/emetrics/index.html –are helpful and trusted LibQUAL+ TM –Suite of services that libraries use to solicit, track, understand, and act upon users’ opinions of service quality. http://www.libqual.org/index.cfm DigiQUAL TM –Digital library service quality – set of 180 items tested against five collections

4. CNI Fall 2005 Task Force Meeting4 Open access -- Beyond vendor supplied data ISO standards for the electronic collection include e-books, electronic databases, ejournals, and digital documents. –Free internet resources are counted separately. –Yet, library liaison and subject web pages include free resources. Other electronic resources are important, including: –Digital libraries – local, archival resources –Pre-print and post-print servers –Open access journals –Open access repositories (e.g., institutional repositories) –Electronic theses and dissertations servers Can we assess use of these resources as well as vendor supplied databases and ejournals?

5. CNI Fall 2005 Task Force Meeting5 Web-based usage surveys Advantages –Can go beyond gross usage counts, using sampling –User and use information E.g.; can query purpose of use, the discipline or affiliation, the status of the patron –Opinion Service quality perception –Can be tied to usage Point of use –Can fold in other services (e.g., ILL, Ask A Librarian) Methodological issues –Sample, not census –Replicable? –Truly random sample is difficult to obtain –High non-response rate – are the responses representative of the population? Survey not seen Survey seen, but not filled out –Users, not usage –Remembered, predicted or critical incident reports – perceived to be not as reliable as vendor census based on usage –Browsers can influence the display of the web survey And more. See for example: Holly Gunn, “Web-based Surveys: Changing the Survey Process”, First Monday, Issue 7, 2002. D.A. Dillman, Mail and Internet Surveys, The Tailored Design Method. 2 nd ed., 2000.

6. CNI Fall 2005 Task Force Meeting6 MINES for Libraries TM MINES is a transaction-based research methodology consisting of a web-based survey form and a random moments sampling plan. MINES typically measures who is using electronic resources, where users are located at the time of use, and their purpose of use. MINES measures usage as well as users. MINES is administered at the point of use, and surveys repeat usage during the sample period. MINES was adopted by the Association of Research Libraries (ARL) as part of the “New Measures” toolkit in May, 2003. MINES is different from other electronic resource usage measures that quantify total usage (e.g., COUNTER, E-Metrics) or measure how well a library makes electronic resources accessible (LibQUAL+ TM ).

7. CNI Fall 2005 Task Force Meeting7 MINES for Libraries TM

8. CNI Fall 2005 Task Force Meeting8 Assessment Infrastructure On Campus Patrons In Library Patrons Off campus patrons ?

9. CNI Fall 2005 Task Force Meeting9 Assessment Gateway online catalog federated searching

10. CNI Fall 2005 Task Force Meeting10 Demographics by Location of User U.S. Main Libraries On Campus, Not in the Library n = 8,838 Inside the Library n = 9,172 Off-Campus n = 7,688 MINES for Libraries™

11. CNI Fall 2005 Task Force Meeting11 Off-Campus n = 5,133 Demographics by Location of User U.S. Medical Libraries Inside the Library n = 6,819 On Campus, Not in the Library n = 19,582 MINES for Libraries™

12. CNI Fall 2005 Task Force Meeting12 MINES for Libraries TM Off-Campus n = 9,163 Demographics by Location of User Ontario Council of University Libraries Inside the Library n = 4,047 On Campus, Not in the Library n = 7,090

13. CNI Fall 2005 Task Force Meeting13 On-Campus, Not in the Library n = 15,423 In the Library n = 6,312 Purpose of Use By Location Medical Libraries 2003 – 2005 Off-Campus n = 4,375 *83% of sponsored research usage occurred outside the library. 92% of this use took place on-campus. Overall Use n = 26,110

14. CNI Fall 2005 Task Force Meeting14 Purpose of Use By Location Main Campus Libraries 2003 – 2005 In the Library n = 7,618 On-Campus, not in the Library n = 6,641 All Usage n = 19,271 72% of sponsored research usage of electronic resources occurred outside the library; 83% of this took place on campus. Off-Campus n = 5,012

15. CNI Fall 2005 Task Force Meeting15 MINES for Libraries TM U.S. Main Libraries Total Users n = 25,698 U.S. Medical Libraries Total Users n = 31,883 OCUL (Canada) Libraries Total Users n = 20,300 Location of Electronic Resources Users

16. CNI Fall 2005 Task Force Meeting16

17. CNI Fall 2005 Task Force Meeting17 OCUL Data from MINES

18. CNI Fall 2005 Task Force Meeting18 Web-Based Usage Surveys MINES for Libraries TM brinley.franklin@uconn.edu terry.plum@simmons.edu http://www.arl.org/stats/newmeas/mines.html