1. Utilizing OPAC Search Logs and Google Analytics Assessing OPAC Effectiveness and User Search Behavior VALE Users'/NJLA CUS/NJ ACRL Conference January 9, 2009 Rutgers University Jia Mi Cathy Weng

2. Presenters Jia Mi Electronic Resources/Serials Librarian The College of New Jersey Cathy Weng Head of Cataloging The College of New Jersey 2

3. The College of New Jersey (TCNJ) Located in Ewing, NJ 6000 students Over half a million volumes in the Library’s collection Ex Libris Voyager System in use since 2000 Activated Google Analytics in summer 2008 3

4. OPAC Search Transaction OPAC search transaction -- “Consists of a question or query by the user followed by an answer or response from the system” (*) Transaction log analysis – “Detailed and systematic examination of each search command or query by a user and the following database result output by the OPAC.”(*) *Blecic, et al, 1998 4

5. Sample Search Transaction Log page 5

6. What is Google Analytics? “Google Analytics (abbreviated GA) is a free service offered by Google that generates detailed statistics about the visitors to a website. ” GA can track visitors from all referrers, usage, entry points, searches, etc. Reports can be viewed, emailed and exported. This free version is limited to 5 million pageviews a month 6

7. Why Use Google Analytics? Find out… How users get to our site Why they use our site: Which pages are the most popular How they navigate the site How long they stay at the site What are their patterns of behavior Whether the web site effectively supports the institution’s goals and mission regarding provision of access and services 7

8. Google Analytics Dashboard 8

9. Data Collection Voyager transaction log  Two OPAC transaction logs were collected: April 2007 and November 2008  Comparisons made in many aspects  Library website was redesigned in August 2007; resulted in different search patterns Google Analytics  November 2008 data collected  Data from September to November 2008 also examined and compared 9

10. Voyager Transaction Log Can assess  Overall statistics of search types used  Query traffic, both on- and off-campus  How searches were issued and refined  Query trends Cannot assess  Queries successful or not  Queries made by librarians or other users 10

11. When Analyzing the Log Many unanswered questions  Many queries do not have session IDs  How “Simple Search” was formed?  Many queries do not have “search type” Some educated guesses  Queries from redirect (i.e., user clicked hyperlinked field) Comprehensive analysis not possible  Focus on a few areas only 11

12. Study Focus Comparison  Query statistics  Default searches  Usage of Advanced Search  Usage of Subject search Other findings related to user search skills and OPAC effectiveness 12

13. CategoriesDatesQueriesPercentage Total Searches 4/2007 53,985 11/2008 88,649 On-campus queries (159.91…) 4/2007 39,99774% 11/2008 25,59129% Queries from local host (127.0.0.1) 4/2007 6,00411% 11/2008 56,05563% Off-campus queries 4/2007 7,90215% 11/2008 7,0038% Query Statistics 13

14. April 2007November 2008 Title Begins with (Default ) Keyword – Relevance Title Begins with Keyword – Relevance (Default) 15,9105,5215,15817,698 29.4%10%6%20% Title Phrase vs. Keyword Search 14

15. Subject Searches Browse search: log does not show result hits Heading (phrase) search: difficult to use as users rarely know the established terms Search logs revealed two types: user- issued searches (Subject Heading Browse) and record redirect (Subject Browse) 15

16. Bibliographic Record Display 16

17. Subject Browse Screen 17

18. April 2007November 2008 4511 (8% of total searches) 1734 (2% of total searches) Queries typed in by user Queries via system redirect Queries typed in by user Queries via system redirect 297515366741060 65%35%39%61% Subject Queries 18

19. Advanced Search (Guided Search) April 2007: 3,331 (6% of total searches) searches issued. (3057 + 274 ) Nov. 2008: 1,761 (2% of total searches) searches issued. (1253 + 508 ) Data suggest that library users use Advanced Search interface less now than they did 18 months ago 19

20. April 2007November 2008 3,3311,761 6%2% Advanced Searches Statistics do not include searches issued by local host (127.0.0.1) which has quite a few searches using Advanced Search interface. 20

21. Call Number Browse By clicking the call number hyperlinked field, user will receive a call number browse screen of nearby items Approximately 500 queries in Nov. 2008 and 600 queries in April 2007 using call number browse hyperlink redirect It is assumed that users clicked call number hoping to get information about item shelf location in the Library (not nearby items) 21

22. Other Findings Related to system retrieving  When doing title phrase search using Advanced Search interface, it only searches main title, not subtitle.  E.g., “Rich and the super rich, a study in the power of money today”.  Main title and subtitle.  User would not know the difference.  System retrieval mechanism issue. 22

23. Other Findings Related to OPAC Display  When doing keyword search using Basic Search interface, search results are arranged by relevance.  When doing keyword search using Advanced Search interface, search results are arranged by publication date.  More relevant results might not be in the first results display page. 23

24. 24

25. 25

26. Next page 26

27. Other Findings Keyword in Basic vs. Advanced Search interface  In Basic Search words like “the,” and “and” are acceptable  In Advanced Search, typing in “the” or “and” will retrieve 0 hits  System retrieval mechanism issue 27

28. 28

29. 29

30. Other Findings In Basic Search, using “Author (last, first)” results in a browse search and the x-ref in authority record works. In Advanced search using author search key results in a heading search (instead of browse search) and the x-ref in authority record does not take effect. 30

31. Author Query in Basic Search 31

32. See Reference OPAC Display 32

33. Author Query in Advanced Search 33

34. In Advanced Search, need to use established form of name for records to be retrieved. 34

35. Other Findings Advanced Search interface  Using “as a phrase” as default might have better chance retrieving 0 records.  Should the default be changed to “All of these”? 35 Advanced Search Interface

36. What Can Be Done Encourage use of author browse in Basic Search interface to take advantage of x-ref in authority file. Provide information on search help page. Investigate making author browse possible in Advance Search interface. 36

37. What Can Be Done Look into system indexing structure more closely and their connection to OPAC searches  E.g., Title search in Advanced Search interface to change to TALL (whole title), not just 245A (main title)  E.g., Enhance Keyword search in Advanced Search to accept “stop words” (the, and, etc.) 37

38. What Can Be Done Investigate the possibility of enhancing query results display from Advanced keyword search to be ranked by relevance. 38

39. What Can Be Done OPAC search results OPAC transaction log Enhance local search skills. 39

40. Google Analytics Dashboard: Comparison 40

41. Visitors Overview 41

42. Geographic Region: International 42

43. Geographic Region: United States 43

44. Geographic Region: New Jersey 44

45. Site Visiting Time 45

46. Site Usage Trend 46

47. Where Did Users Come From? 47

48. Direct Visits to TCNJ Library Catalog 48

49. What Content are Users Looking at? 49

50. / cgi-bin/Pwebrecon.cgi?DB=local &PAGE=First This page was viewed 19 times via 1 keywords Pageview: 2816 50

51. 51

52. Pages with the Most Traffic (Homepage) 52

53. Top traffic sources (Homepage) 53

54. 54

55. Most Popular Databases 55

56. Where is the Traffic Coming From? 56

57. 57

58. 58

59. InterLibrary Loan vs. “Request Button” 59 Total 883 requests: 705 from ILL page 178 from the Library Catalog using Request Button

60. Book Request 60

61. RSS Feed and Persistent Link? 61

62. Do Users Use “Limit to” 62

63. Do Users Use “New Books” page? 63

64. Google Analytics Pros and Cons Pros – User-friendly – Free – Accurate Cons – Users with high security settings can only be partially tracked – Site visits browsed from mobile phones can not be tracked – Can slow down system operation (longer response time) 64

65. Using Google Analytics Data Help us to  Better understand Who web users are Where they come from What they do on the site How use of site may be facilitated or enhanced  Improve web site design  Provide more relevant materials for users 65

66. Combining Search Log Data and Google Analytics Data Trace traffic to TCNJ Catalog from WorldCat. Detect usage of questionable sites. 66

67. Queries from WorldCat Search log recorded searches by “ocm#” (OCLC control number) “ocm#” query is the default query type set up locally on WorldCat. Approximately 150 “ocm#” searches in November 2008. Confirming traffic from WorldCat to TCNJ OPAC. 67

68. WorldCat link to TCNJ OPAC 68

69. Referral from Worldcat 69

70. Simple Search and Builder Search Simple search and Builder search existed in TCNJ’s old OPAC interface equivalent to “Basic Search” and “Advanced Search” respectively. Old search types recorded in OPAC transaction logs, suggesting the defunct interface was still being used off campus. 70

71. TCNJ Old OPAC Interface 71

72. Builder Search A total of 274 searches in April 2007 and 508 searches in November 2008 recorded on search log are from Builder search. Majority of them are ISBN searches (coming from search engines?) 72

73. Google Analytics Data /cgi-bin/Pwebrecon.cgi Pageview: 3791 73

74. What’s Next? Work on enhancing Voyager indexing structure and OPAC search mechanism and display. Continue to find more answers to questions. More analysis to be completed. 74

75. Thank you! jmi@tcnj.edu weng@tcnj.edu 75