1. NCSU Libraries Endeca and faceted browsing: Giving the user a useful catalog Scott Warren NCSU Libraries South Carolina Library Association Annual Meeting June 7, 2007

2. NCSU Libraries Outline 1.Problem and Context 2.Online searching, shopping, and examples 3.Demo 4.Faceted Navigation 5.Implementation Challenges 6.Facet Usage Statistics 7.Reflections

3. NCSU Libraries The Context

4. NCSU Libraries Online Catalogs "Most integrated library systems, as they are currently configured and used, should be removed from public view.“ - Roy Tennant, CDL

5. NCSU Libraries What is the problem? Existing catalogs are hard to use: –known item searching works pretty well, but … users often do keyword searching and get large result sets returned in system sort order (last in, etc.) catalogs are unforgiving on spelling errors, stemming Authority searching completely mystifying

6. NCSU Libraries Catalog metadata is buried Subject headings are not leveraged in searching they should be browsed or linked from, not searched Data from the item record is not leveraged should be able to filter by item type, location, circulation status, popularity

7. NCSU Libraries Word of the Day for Saturday, May 5, 2007 moil \MOYL\, intransitive verb: 1. To work with painful effort; to labor; to toil; to drudge. 2. To churn or swirl about continuously. 3. Toil; hard work; drudgery. 4. Confusion; turmoil.

8. NCSU Libraries What’s the big picture? Improve the quality of the library catalog user experience. Exploit our existing metadata infrastructure (make MARC work harder). Build a more flexible catalog tool that can be integrated with discovery tools of the future.

9. NCSU Libraries What is Endeca? Software company based in Cambridge, MA Search/information access technology provider for a number of major e-commerce websites Developers of the Endeca Information Access Platform

10. NCSU Libraries Why Endeca? Customized relevance ranking of results Better subject access by leveraging available metadata through facets Improved response time Enhanced natural language searching through spell correction, etc. Browse

11. NCSU Libraries A question “How is the new generation of library catalog being developed?” informed and enhanced by search technologies developed outside of the library based on how our users know how to search, not on how we want them to search What does search look like for our users?

12. NCSU Libraries Examples

13. NCSU Libraries

15. Faceted Navigation on the Web

16. NCSU Libraries

17. Facet Value

18. NCSU Libraries

21. Faceted Navigation in Libraries

22. NCSU Libraries Faceted Navigation in Libraries

23. NCSU Libraries Faceted Navigation in Libraries

24. NCSU Libraries Demonstration

25. NCSU Libraries Faceted Navigation

26. NCSU Libraries What is Faceted Navigation?

27. NCSU Libraries What is Faceted Navigation? Search and browse in a single interface Facets can vary in scope –What is the item about? –What kind of item is it? –Where is it? Enables users to narrow results Macroscopic behavior of results set Clues to being on the right path

28. NCSU Libraries Origins of Facets 1930s Ranganathan Colon Classification

29. NCSU Libraries Cartesian Coordinates

30. NCSU Libraries Coordinate System LCSH Format Library (x, y, z) (Library, LCSH, Format) (Branch 1, History, Book) (Branch 2, History, DVD) Multiple records could be associated with each coordinate point. Each point is associated with at least one record. Branch 1 Branch 2 DVD Book History Art (Branch 1, History, Book)

31. NCSU Libraries Another way to think about it 11 dimensional lattice space All points associated with at least one item/record Records can be associated with > 1 point Keyword search selects subset of points with word(s) in record Facets shown are those dimensions corresponding to the points in that set (nonzero values). Choosing a facet value is equivalent to slicing through the multidimensional lattice on a plane along that facet value and reducing the lattice’s dimension by 1. Choose enough facets and you will get down to a few items (never a null set)

32. NCSU Libraries Implementation

33. NCSU Libraries Implementation Challenges Facet selection Interface design Data issues

34. NCSU Libraries Endeca at NCSU Endeca used to improve the discovery portion of the library catalog Endeca software indexes 1.6 million MARC records exported nightly from Sirsi Unicorn ILS Backend functions of ILS remain intact

35. NCSU Libraries Facets Implemented at NCSU Availability Author Library Format Language Browse: New LC Classification Subject: Topic Subject: Genre Subject: Region Subject: Era

36. NCSU Libraries Facet Selection

37. NCSU Libraries Interface Design Iterative approach using wireframes Eight major revisions in a four month period Still lots of room for improvement

38. NCSU Libraries Technical Overview Endeca co-exists with SirsiDynix Unicorn ILS and Web2 online catalog Endeca handles keyword search Web2 handles authority search and detail page display Endeca indexes MARC records exported nightly from Unicorn Endeca = discovery portion of the ILS

39. NCSU Libraries Technical Overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Information Access Platform

40. NCSU Libraries Technical Overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Offline - Nightly

41. NCSU Libraries Technical Overview Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Always Online

42. NCSU Libraries Implementation Team Seven member team –5 IT staff, – 1 cataloging librarian, – 1 reference librarian Timeline –License / negotiation: Spring 2005 –Software acquisition: Summer 2005 –Implementation: Aug 2005 to Jan 2006

43. NCSU Libraries Data Issues ILS data with MARC-8 encoding => Text data with UTF-8 encoding Data consistency between ILS and Endeca catalog indexes (updates!) Data issues revealed by exposing metadata (ex: subject headings) in facets

44. NCSU Libraries Outcomes

45. NCSU Libraries Added search tools Automatic spell correction “Did you mean…” suggestions –Automatic stemming –Bookmark-ability

46. NCSU Libraries True browse Regain ability to browse catalog without entering any search terms

47. NCSU Libraries July 06 – Jan 07

48. NCSU Libraries July 06 – Jan 07

49. NCSU Libraries July 06 – Jan 07

50. NCSU Libraries July 06 – Jan 07

51. NCSU Libraries July 06 – Jan 07

52. NCSU Libraries DimensionValueRequests NewNEW56,286 FormatBook16,188 LC ClassificationQ - Science12,462 LibraryTextiles11,160 LibraryD.H. Hill11,060 AvailabilityAvailable9,276 LibraryOnline Resources8,164 LC ClassificationT – Technology8,052 Subject: TopicHistory7,915 FormatOnline7,858 LC ClassificationP - Language and literature7,005 LC ClassificationH - Social Sciences6,953 LanguageEnglish6,854 Subject: RegionUnited States6,298 FormatJournal, Magazine, or Serial4,621

53. NCSU Libraries Usability testing 10 undergraduate students –5 with new Endeca-based interface –5 with old catalog interface –Identical searching tasks Data collected –Task difficulty/failure –Task duration

54. NCSU Libraries Usability testing

55. NCSU Libraries Usability testing

56. NCSU Libraries Usability testing For students, relevance ranking is key. –July 06 – Jan 07: ~19% continued to page 2 Faceted navigation is intuitive, even for students who don’t use it. Beware of library jargon –“keyword anywhere”, “keyword in subject” User behavior is influenced by previous experience.

57. NCSU Libraries Reflections Faceted navigation enables new ways to discovery resources Library collections often contain rich descriptive metadata… exploit this! We have much to learn about how to optimize these interfaces for the user Great for collection analysis

58. NCSU Libraries Analyzing collections

59. NCSU Libraries Conclusions

60. NCSU Libraries Features Not Supported Work level aggregations / roll-up Customization / personalization Folksonomies / user contributed content Recommender functionality Shopping cart functionality

61. NCSU Libraries QuickSearch

62. NCSU Libraries Future directions Experiment with FRBR search/display through partnership with OCLC. Integrate catalog w/other tools through web services: –OpenSearch, RSS Enrich catalog through external web services: –book jackets, reviews, etc. – Amazon/OCLC Build modular shopping cart functionality. Use Endeca to index local collections.

63. NCSU Libraries Big Issues Benchmarking –Just how much better is it? For whom? When is it not better? Natural Language –Revolutionary War problem Experimenting –What is the optimal interface? –Power Search?

64. NCSU Libraries Big Wins Relevance ranking Speed / performance Locally managed presentation interface Persistent parameter based entry points Proving it could be done