© 2009 Hesburgh Libraries of Notre Dame Bringing Next Generation Unified Search to Notre Dame Bringing Next Generation Unified Search to Notre Dame AARLIN User Group Meeting October 14, 2009 Pascal Calarco Mark Dehmlow

© 2009 Hesburgh Libraries of Notre Dame

Not so long ago, in a galaxy not so far away was born the catalog and it was good. Libraries had need for automated inventory control  Bibliographic description via MARC record  Acquisitions for business processes, serials control  Circulation system for lending, returning

© 2009 Hesburgh Libraries of Notre Dame Oh, yeah, and those patrons... Let's give them an “Online Public Access Catalog”!  … but it’s mostly going to be publicly accessible version of the same search functionality in the back end system.  Designed for librarians initially, patrons later  Focused on managing, providing access to physical items Web interface added next  adding lipstick on a pig

© 2009 Hesburgh Libraries of Notre Dame The Internet is Shaping Expectations Simple searching - Google, Yahoo, Ask.com Patrons are used to doing it themselves Want fast access to results, ranked by relevance Commercial spaces are:  interactive/collaborative content creation (flickr, blogs, facebook, etc.) tagging (flickr, del.icio.us, etc.) content re-use (RefWorks/RefShare, etc.)  Syndicated Blogs News sites like cnn.com Google mash-ups

© 2009 Hesburgh Libraries of Notre Dame Where We Are Right Now OPAC: search interface on inventory system, made for librarians, not patrons – designed around the card catalog which limits its effectiveness Catalogs are usually best for known-item searching, not topic searching Improvements have typically been window dressing, don’t deal with many fundamental limitations of the catalog Users demand interfaces that they are used to (Google, Amazon, etc.) Don't have to ask Amazon how to search for items, why expect libraries?

© 2009 Hesburgh Libraries of Notre Dame Silo-ization at Every Turn Content Silos System Silos CatalogILLMeta- search eReserveWebsiteScience- Direct Web of Science ETDsEEBOJSTOR

© 2009 Hesburgh Libraries of Notre Dame How Users Search Most people make typos at least some of the time Most searches are 2, 3, 4 words with no Boolean operators Most searches use keyword Search is hesitant, iterative, often random process of discovery Most people start elsewhere Few read help screens Few use advanced search – this is true even in Google

© 2009 Hesburgh Libraries of Notre Dame Where Users Search OCLC study  84% users start in search engines vs. 2% at library websites/portals

© 2009 Hesburgh Libraries of Notre Dame Destination vs. Syndication Models The Internet Library Web Site

© 2009 Hesburgh Libraries of Notre Dame The OPAC “Sucks” The OPAC lacks common features of most search engines  Relevance ranking vs. last in, first out  Spell checking (related - did you mean?)  Popular query operators like + and –  Refine search  Sort flexibility  Faceting  Citation indexing vs full text  Developed for print materials, limitations with electronic materials or atomized items (like articles)  Difficult for certain known item search

© 2009 Hesburgh Libraries of Notre Dame Industry Trends Decouple the front end (search and discovery) from the back end (inventory and cataloging) Service Oriented Architecture – many programs loosely coupled The 5 th generation of ILS upon us  Existing systems will probably be superseded within the next 2 – 10 years

© 2009 Hesburgh Libraries of Notre Dame ILS Dis-integrating the ILS The future of the ILS could be Acquisitions eResource Mangement CirculationCatalogingSerialsAuthority ND Central Accounting

© 2009 Hesburgh Libraries of Notre Dame De-coupling – What Does This Mean? Keep the business software – the ILS, change the interface Most Next Gen features require different infrastructure, some don’t  index vs. database Speed Relevance Faceted browse Did you mean FRBR

© 2009 Hesburgh Libraries of Notre Dame “Grand Challenge” for LIS A single interface that searches the world of knowledge and brings back all of the most relevant and authoritative resources that match a user’s query It’s our “Holy Grail” - we've been working on this one for about 30 years :) Next Generation Interfaces open the possibility to get us closer

© 2009 Hesburgh Libraries of Notre Dame Characteristics of Next Generation Search Enhanced Search Functionality  Faceted browse  Relevance ranking  Did you mean / Spell Checking auto-correction, resubmit search  De-silo-ization Integrating search for books, articles, etc.  Single, Simple Search Box  FRBR – functional requirements for bibliographic record, grouping editions

© 2009 Hesburgh Libraries of Notre Dame Characteristics of Next Generation Search Enhanced Experience  Sometimes fun and engaging  Interactive/Collaborative  User centered design Enhanced Services  Find it / Get it for me  Book Covers / Synopsis  Full text  Availability on same page as results

© 2009 Hesburgh Libraries of Notre Dame Characteristics of Next Generation Search Enhanced Content  Article Searching  Commercial Data  Merging Special Collections  Harvesting Online Collections Grey Literature Free Content Enhanced Access  Syndication - Getting into users tools Course Management Systems Browser and Desktop Tool Bars Portals

© 2009 Hesburgh Libraries of Notre Dame Next Generation Search Interfaces 1.Next Generation Catalog 2.Next Generation “Unified Search” Aid Indexing System User Interface ILS OPAC MARC Vendor Data MetaSearch OAI Vendor Data Circ Data Full Text

© 2009 Hesburgh Libraries of Notre Dame Why Should We Consider This? Next Generation Systems are an evolutionary step, it is the first step toward the next generation ILS – at some point we will have to do something They are designed to:  meet our user’s search/retrieval expectations  reduce the amount of work and expertise needed to get from query to research material  improve service

© 2009 Hesburgh Libraries of Notre Dame Problems Needing Solutions Limitations with existing catalog  Coarse relevancy functionality  Non-intuitive to users used to Google  Hard to expand searches beyond local holdings Content that didn’t fit into catalog  Encoded Archival Description (manuscripts)  Image collections  Dublin Core-based descriptive collections

© 2009 Hesburgh Libraries of Notre Dame Problems Needing Solutions Wanted a platform to build new services on, not just a new UI  Social networking capabilities: review items, tag items

© 2009 Hesburgh Libraries of Notre Dame Desired functionality Ability to add plurality of metadata types: MARC, DC, XML-encoded data Extensible and modular; platform for innovation Provide clear improvements to identified problems/shortcomings with current catalog Offer a high-degree of customization options Informed by DLF recommendations on ILS & Discovery Systems

© 2009 Hesburgh Libraries of Notre Dame Background & Environmental Scan OCLC Environmental Scan (2003) Calhoun Report (2006) Rochester undergrad research project (2007) OCUL Scholar's Portal 2 discussion paper (2007) LoC Future of Bibliographic Control Report (2008) JISC & SCONUL LMS Survey (2008)

© 2009 Hesburgh Libraries of Notre Dame

Methodology Observed Courseware Group  Simplified process  No weighting, aggregate numbers, yes – no – maybe  Condensed timeframe Made Evaluation Variables Explicit Spent the first half of process getting consensus on categories  Helped handle disagreement Asked to provide recommendations  Gave comparative analysis of top 3 candidates

© 2009 Hesburgh Libraries of Notre Dame Analysis Details 44 questions in twelve areas To what extent does each solution meet/exceed our desired functionality requirements? Answered questions by:  Using the solutions at other sites  Referring to product literature  Ask on support websites  Detailed inquiries to sales people

© 2009 Hesburgh Libraries of Notre Dame Comparative Analysis Categories 1)Search functionality  Relevance ranking, FRBR, Facted Browse, Visualization, etc. 2)Spelling correction & suggestion  Did you mean?, Alternate Terms, etc. 3)User experience  Usable Interface, Pleasing Aesthetics, Customizable, etc. 4)Index capability  Metasearch Integration, API Integration, etc. 5)Record enrichment  Book Covers, TOCs, Reveiws, etc.

© 2009 Hesburgh Libraries of Notre Dame Comparative Analysis Categories 6)Supported data formats & sources  MARC, Dublin Core, EAD, TEI, I&A Data, Full text, etc. 7)ILS integration  Circulation Availablility 8)Social computing  Reviews, Ratings, Tagging, etc. 9)Enhanced services  RSS, Notification, Bibliographic Management Export, etc.

© 2009 Hesburgh Libraries of Notre Dame Comparative Analysis Categories 10) System and personnel requirements  Hardware, Support, Ease of Management, Statistics, etc. 11) APIs  Web services, SDKs, etc. 12) Support Options  Vendor, Community, etc.

© 2009 Hesburgh Libraries of Notre Dame What did we find?  A spectrum of features, functionality and integrability  Open toolkits to tightly-integrated turnkey systems  A few systems oriented to broad, inclusive content indexing and discovery  Most feature social computing/library 2.0 feature sets

© 2009 Hesburgh Libraries of Notre Dame Candidate Systems MediaLab Innovative Interfaces Inc. Villanova Univ. Ex Libris Group Endeca Technologies Inc. Index Data OCLC LibLime Aarhus Univ. Univ. of Rochester

© 2009 Hesburgh Libraries of Notre Dame Evaluating the Options NichesVisionaries Completeness of Vision ChallengersLeaders Ability to Execute Encore Evergreen ILS WorldCat Local Endeca Koha Summa Zebra Extensible Catalog (XC) Open Library Primo VUFind Aquabrowser

© 2009 Hesburgh Libraries of Notre Dame How Do You Decide?  Needs to be in the context of your institution  How much money can you spend?  What is your timeline for implementation?  How much technical expertise do you have?  How much technical time do you have?  Is open source part of your institutional culture?  What companies do you do business with?  Do you need more features than what comes “out of the box”?

© 2009 Hesburgh Libraries of Notre Dame

 MARC records: Bethel College, Holy Cross College, Saint Mary's College, Hesburgh Libraries, Kresge Law Library  Approximately 3.5 million records total  We are first Primo site to have III Millennium; some development partnership with Ex Libris  Plan for record loading enhancement plug-in to merge bibs, holdings, and availability  Working on real time availability plug-in MALC & Kresge Law Library Union Catalog March 2009 soft launch, Fall 2009 Production Phase I

© 2009 Hesburgh Libraries of Notre Dame  Encoded Archival Description (EAD): Special Collections & University Archives (1K records)‏  Art Image Library (DigiTool) (5K records)‏  Electronic Theses & Dissertations (500 records)‏  Text Encoding Initiative (TEI), and other - Special Collections content (1K records)‏  Center for Research Libraries MARC records (1M records)‏ Phase II Local Digital Content (Q1, 2009)

© 2009 Hesburgh Libraries of Notre Dame  MARC records for electronic sets (400K records)‏  These have been particularly problematic batch loading in the ILS  ECCO, EEBO, Alexander Street Press, etc.  Other licensed digital content  Full text  eJournal Metadata  A&I data  Other free content  MBooks Phase III +Phase II External Digital Content (Q2, 2009)

© 2009 Hesburgh Libraries of Notre Dame Phased Plan Primo WorldCat MALC LAW CRLIUSB Archives Latino Studies Fulltext A&I MBooksOthers Phase IPhase IIPhase III Deep Search Adapter

© 2009 Hesburgh Libraries of Notre Dame  Use Aleph X-Services, NCIP to source ILS systems  Aleph, Millenium, Unicorn integration using Primo  Direct consortial borrowing  Leverage smaller campus collections  Redirect some ILL throughout consortia  Cooperative collection development Universal Borrowing in Primo

© 2009 Hesburgh Libraries of Notre Dame

Primo as Common Framework* Model *based on California Digital Library Common Framework Common Publishing Platform Interface 1 …….. Catalog Data OAI API …….. Other Resource Database Metasearch Interface 2 Interface n Interface 3 Interface 4

© 2009 Hesburgh Libraries of Notre Dame Expansion of Services Common Publishing Platform Interface 1 …….. Catalog Data OAI API …….. Other Resource Database Metasearch Interface 2 Interface n Interface 3 Interface 4 Download Collaborate Translate Edit Define Word More

© 2009 Hesburgh Libraries of Notre Dame Other Possibilities  Search  All newspapers interface  All image collections (local and commercial)  Enhancements  Augment records with metadata from Ulrich’s and JAKE

© 2009 Hesburgh Libraries of Notre Dame Thank You