1. Crunching Numbers: OPAC Log Analysis of WebVoyage Bennett Claire Ponsford Digital Services Librarian Texas A&M University Libraries EndUser 2007 Session 34

2. Overview Why analyze your log files? How to do it What we found The changes we made What do the latest logs say? Improvements needed

3. Why Analyze? To see how your users search when you’re not watching To resolve internal disagreements over default searches, limits, etc. To see whether changes to WebVoyage really improved search results As a counterpoint to task-based user testing

4. CC.S. Lewis LLewis, C. S. (Clive Staples) LLION WITCH? LLION, WITCH? LLION, WITCH, AND WARDROBE? LLewis, C. S. (Clive Staples)

6. Issues to Think About Does Voyager capture the data you need? Privacy concerns Does your network organize data the way you need? Staff vs. public IP addresses Do you want all searches or a sample?

7. How To Read the documentation Technical Manual, Chapter 15, Popacjob Begin logging your data Extract data into Access database Clean up data as needed Run queries Scratch head and contact Tech Support

9. 5-MAR-07 WebOpac 20061016102711 Title keyword (TKEY New) AND (TKEY York) AND (TKEY Times) Y West Campus Library K Y N 7 1 W 999.999.99.999 AMDB20020820112825 N

10. Data Fields Search_date5-MAR-07 Stat_stringWebOpac Session_id20070305123912 Search_typeTitle keyword Search_string(TKEY New) AND (TKEY York) AND (TKEY Times)

11. Data Fields (cont.) Limit_flagY Limit_stringWest Campus Library Index_typeK RelevanceY HyperlinkN Hits7

12. Data Fields (cont.) Search_tab1 Client_typeW Client_ipXXX.XXX.XX.XXX DbkeyAMDB20020820112825 Redirect FlagN

14. SQL for Count of Search Type SELECT [Spring 2007 OPAC log].Search_type, Count([Spring 2007 OPAC log].Search_type) AS CountOfSearch_type FROM [Spring 2007 OPAC log] WHERE ((([Spring 2007 OPAC log].Client_type)="W") AND (([Spring 2007 OPAC log].Search_tab)="1") AND (([Spring 2007 OPAC log].Hyperlink)=“N") AND (([Spring 2007 OPAC log].Client_ip) Not Like "128.194.8[4-7].*" And ([Spring 2007 OPAC log].Client_ip) Not Like "165.91.39.*")) GROUP BY [Spring 2007 OPAC log].Search_type ORDER BY [Spring 2007 OPAC log].Search_type;

15. Results Author Browse – 334 Author headings – 943 Author keyword – 1587 Builder – 6 Call Number Browse – 777 Command – 16 Documents Call Number – 15 Expert keyword (rel) – 5 Expert keyword (rev) – 2378 Journal title keyword – 793 Journal title – 1375 Keyword – 82 Keyword (Relevance) -- 6179 Keyword Search – 1574 LC Call Number browse -- 14 Locally Assigned Call#-- 2 Simple Search -- 58 Subject Browse -- 851 Subject headings -- 128 Subject Hds keyword -- 251 Title keyword -- 2045 Title Redirect -- 615 Title starts with – 1677

16. June 2006 (Voyager 5)

17. September 2006 (Voyager 5) Changed interface Defaults Kept Tab at Simple Search Changed Search to Keyword (CMD* with javascript) Changed result sort to by relevance

18. Fall 2006 Preparing to upgrade to Voyager 6.1 New keyword searches with ^ to automatically AND words together Some people unhappy with recent changes Default search Search results sort order Decided to look at the data

19. Decisions upgrading to V6 Basic data Where are our searchers What search tab are they using How are they searching Default search Order of title searches Simple limits

20. Where Are Our Searchers?

21. What Search Tab Used?

22. Default Search: Discussion Title search (TALL) What we traditionally had used Reference’s preference General keyword search (new GKEY^*) What users are used to in a Google world More forgiving search

25. Default Search: Decision General keyword search (new GKEY^*) User preference Fewer No Hit results

26. First Title Search: Discussion Left anchored title (TALL) Preferred by Reference Title keyword (new TKEY^*) More forgiving

27. Title Search (TALL): Problems

28. Title Search: Decision Title keyword Left-anchored title had too many problems

29. Simple Limits Several additional location limits requested Concern that too many would be confusing

30. Search Limits Used

31. Simple Limits: Decision Added new limits and will evaluate with more data

33. Analysis of Voyager 6 Logs Search frequencies No hit frequencies Title search problems Journal title search hits Search limits

34. Keyword and Subject Searches

35. Author Searches

36. Title Searches

37. No Hit Title Searches: Do We Own Them?

38. No Hit Title Searches: Problems

39. Location Limits Used

40. Comparison of Limits

41. Have Changes Helped? Search frequency No hits percentage

42. Search Frequency

43. No Hit Percentages Some improvement but no major change

45. Improvements: Spelling Spellchecking Automatic searching of variant spellings “&” or “and” British vs. American spellings Numbers Abbreviations Did you mean? Suggestions based on field

46. Improvements: Help More granular no hits help Specific search types Any search with “conference” or “proceedings” in it Journal title searches including “vol.”, “no.”, or a number Searches with more than 4 or 5 words More granular help for too many hits

47. Improvements: Specific Searches Keyword searches Automatic stemming Ignore punctuation and spacing Ignore stop words Title searches Ignore initial article

48. Journal search results layout Whether to include the index field in the journal title search results Same search results but the order of the results is change by the inclusion of the index field Primarily a problem for single word titles that retrieve more than 1 screen of results – Science, Nature, etc.

51. JALL Search Results

52. What Next? Continued analysis of searches with no hits Analysis of search repair strategies Word counts

53. More Information Jansen, Bernard J. “Search log analysis: What it is, what’s been done, how to do it,” Library & Information Science Research, 28 (2006) 407-432. Yu, Holly and Margo Young, “The impact of Web search engines on subject searching in OPAC”, Information Technology and Libraries, 23 (2004) 168-180.

54. Contact Information Bennett Claire Ponsford bennett.ponsford@tamu.edu 979/845-0877 https://libcat.tamu.edu