Utilizing OPAC Search Logs and Google Analytics Assessing OPAC Effectiveness and User Search Behavior VALE Users'/NJLA CUS/NJ ACRL Conference January 9, 2009 Rutgers University Jia Mi Cathy Weng

Presenters Jia Mi Electronic Resources/Serials Librarian The College of New Jersey Cathy Weng Head of Cataloging The College of New Jersey 2

The College of New Jersey (TCNJ) Located in Ewing, NJ 6000 students Over half a million volumes in the Library’s collection Ex Libris Voyager System in use since 2000 Activated Google Analytics in summer

OPAC Search Transaction OPAC search transaction -- “Consists of a question or query by the user followed by an answer or response from the system” (*) Transaction log analysis – “Detailed and systematic examination of each search command or query by a user and the following database result output by the OPAC.”(*) *Blecic, et al,

Sample Search Transaction Log page 5

What is Google Analytics? “Google Analytics (abbreviated GA) is a free service offered by Google that generates detailed statistics about the visitors to a website. ” GA can track visitors from all referrers, usage, entry points, searches, etc. Reports can be viewed, ed and exported. This free version is limited to 5 million pageviews a month 6

Why Use Google Analytics? Find out… How users get to our site Why they use our site: Which pages are the most popular How they navigate the site How long they stay at the site What are their patterns of behavior Whether the web site effectively supports the institution’s goals and mission regarding provision of access and services 7

Google Analytics Dashboard 8

Data Collection Voyager transaction log  Two OPAC transaction logs were collected: April 2007 and November 2008  Comparisons made in many aspects  Library website was redesigned in August 2007; resulted in different search patterns Google Analytics  November 2008 data collected  Data from September to November 2008 also examined and compared 9

Voyager Transaction Log Can assess  Overall statistics of search types used  Query traffic, both on- and off-campus  How searches were issued and refined  Query trends Cannot assess  Queries successful or not  Queries made by librarians or other users 10

When Analyzing the Log Many unanswered questions  Many queries do not have session IDs  How “Simple Search” was formed?  Many queries do not have “search type” Some educated guesses  Queries from redirect (i.e., user clicked hyperlinked field) Comprehensive analysis not possible  Focus on a few areas only 11

Study Focus Comparison  Query statistics  Default searches  Usage of Advanced Search  Usage of Subject search Other findings related to user search skills and OPAC effectiveness 12

CategoriesDatesQueriesPercentage Total Searches 4/ ,985 11/ ,649 On-campus queries (159.91…) 4/ ,99774% 11/ ,59129% Queries from local host ( ) 4/2007 6,00411% 11/ ,05563% Off-campus queries 4/2007 7,90215% 11/2008 7,0038% Query Statistics 13

April 2007November 2008 Title Begins with (Default ) Keyword – Relevance Title Begins with Keyword – Relevance (Default) 15,9105,5215,15817, %10%6%20% Title Phrase vs. Keyword Search 14

Subject Searches Browse search: log does not show result hits Heading (phrase) search: difficult to use as users rarely know the established terms Search logs revealed two types: user- issued searches (Subject Heading Browse) and record redirect (Subject Browse) 15

Bibliographic Record Display 16

Subject Browse Screen 17

April 2007November (8% of total searches) 1734 (2% of total searches) Queries typed in by user Queries via system redirect Queries typed in by user Queries via system redirect %35%39%61% Subject Queries 18

Advanced Search (Guided Search) April 2007: 3,331 (6% of total searches) searches issued. ( ) Nov. 2008: 1,761 (2% of total searches) searches issued. ( ) Data suggest that library users use Advanced Search interface less now than they did 18 months ago 19

April 2007November ,3311,761 6%2% Advanced Searches Statistics do not include searches issued by local host ( ) which has quite a few searches using Advanced Search interface. 20

Call Number Browse By clicking the call number hyperlinked field, user will receive a call number browse screen of nearby items Approximately 500 queries in Nov and 600 queries in April 2007 using call number browse hyperlink redirect It is assumed that users clicked call number hoping to get information about item shelf location in the Library (not nearby items) 21

Other Findings Related to system retrieving  When doing title phrase search using Advanced Search interface, it only searches main title, not subtitle.  E.g., “Rich and the super rich, a study in the power of money today”.  Main title and subtitle.  User would not know the difference.  System retrieval mechanism issue. 22

Other Findings Related to OPAC Display  When doing keyword search using Basic Search interface, search results are arranged by relevance.  When doing keyword search using Advanced Search interface, search results are arranged by publication date.  More relevant results might not be in the first results display page. 23

24

25

Next page 26

Other Findings Keyword in Basic vs. Advanced Search interface  In Basic Search words like “the,” and “and” are acceptable  In Advanced Search, typing in “the” or “and” will retrieve 0 hits  System retrieval mechanism issue 27

28

29

Other Findings In Basic Search, using “Author (last, first)” results in a browse search and the x-ref in authority record works. In Advanced search using author search key results in a heading search (instead of browse search) and the x-ref in authority record does not take effect. 30

Author Query in Basic Search 31

See Reference OPAC Display 32

Author Query in Advanced Search 33

In Advanced Search, need to use established form of name for records to be retrieved. 34

Other Findings Advanced Search interface  Using “as a phrase” as default might have better chance retrieving 0 records.  Should the default be changed to “All of these”? 35 Advanced Search Interface

What Can Be Done Encourage use of author browse in Basic Search interface to take advantage of x-ref in authority file. Provide information on search help page. Investigate making author browse possible in Advance Search interface. 36

What Can Be Done Look into system indexing structure more closely and their connection to OPAC searches  E.g., Title search in Advanced Search interface to change to TALL (whole title), not just 245A (main title)  E.g., Enhance Keyword search in Advanced Search to accept “stop words” (the, and, etc.) 37

What Can Be Done Investigate the possibility of enhancing query results display from Advanced keyword search to be ranked by relevance. 38

What Can Be Done OPAC search results OPAC transaction log Enhance local search skills. 39

Google Analytics Dashboard: Comparison 40

Visitors Overview 41

Geographic Region: International 42

Geographic Region: United States 43

Geographic Region: New Jersey 44

Site Visiting Time 45

Site Usage Trend 46

Where Did Users Come From? 47

Direct Visits to TCNJ Library Catalog 48

What Content are Users Looking at? 49

/ cgi-bin/Pwebrecon.cgi?DB=local &PAGE=First This page was viewed 19 times via 1 keywords Pageview:

51

Pages with the Most Traffic (Homepage) 52

Top traffic sources (Homepage) 53

54

Most Popular Databases 55

Where is the Traffic Coming From? 56

57

58

InterLibrary Loan vs. “Request Button” 59 Total 883 requests: 705 from ILL page 178 from the Library Catalog using Request Button

Book Request 60

RSS Feed and Persistent Link? 61

Do Users Use “Limit to” 62

Do Users Use “New Books” page? 63

Google Analytics Pros and Cons Pros – User-friendly – Free – Accurate Cons – Users with high security settings can only be partially tracked – Site visits browsed from mobile phones can not be tracked – Can slow down system operation (longer response time) 64

Using Google Analytics Data Help us to  Better understand Who web users are Where they come from What they do on the site How use of site may be facilitated or enhanced  Improve web site design  Provide more relevant materials for users 65

Combining Search Log Data and Google Analytics Data Trace traffic to TCNJ Catalog from WorldCat. Detect usage of questionable sites. 66

Queries from WorldCat Search log recorded searches by “ocm#” (OCLC control number) “ocm#” query is the default query type set up locally on WorldCat. Approximately 150 “ocm#” searches in November Confirming traffic from WorldCat to TCNJ OPAC. 67

WorldCat link to TCNJ OPAC 68

Referral from Worldcat 69

Simple Search and Builder Search Simple search and Builder search existed in TCNJ’s old OPAC interface equivalent to “Basic Search” and “Advanced Search” respectively. Old search types recorded in OPAC transaction logs, suggesting the defunct interface was still being used off campus. 70

TCNJ Old OPAC Interface 71

Builder Search A total of 274 searches in April 2007 and 508 searches in November 2008 recorded on search log are from Builder search. Majority of them are ISBN searches (coming from search engines?) 72

Google Analytics Data /cgi-bin/Pwebrecon.cgi Pageview:

What’s Next? Work on enhancing Voyager indexing structure and OPAC search mechanism and display. Continue to find more answers to questions. More analysis to be completed. 74

Thank you! 75