Indiana Library Federation Annual Conference ‘Make it more like Google!’: Next Generation Tools for Library Collections Indiana Library Federation Annual Conference November 19, 2008 Pascal Calarco Mark Dehmlow

Overview A [really] Short History Lesson The Internet & the Library Our Users and the OPAC The Big Picture on the Future of the OPAC Next Generation Search Notre Dame Process for Selecting NextGen Search System How Other Institutions Can Make Similar Decisions

Not so long ago, in a galaxy not so far away . . . . . . was born the catalog and it was good. Libraries had need for automated inventory control Bibliographic description via MARC record Acquisitions for business processes, serials control Circulation system for lending, returning

Oh, yeah, and those patrons . . . Let's give them an “Online Public Access Catalog”! … but it’s mostly going to be publicly accessible version of the same search functionality in the back end system. Designed for librarians initially, patrons later Focused on managing, providing access to physical items Web interface added next adding lipstick on a pig

Early Information Science Research Christine L. Borgman (1986) “Why are online catalogs hard to use? Lessons learned from information retrieval studies” Journal of the American Society for Information Science Ray R. Larsen (1991) “The decline of subject searching: Long- term trends and patterns of index use in an online catalog” Journal of the American Society for Information Science Ray R. Larsen (1992) “Evaluation of advanced retrieval techniques in an experimental online catalog” Journal of the American Society for Information Science Ray R. Larsen (1996) “Cheshire II: designing a next-generation online catalog” Journal of the American Society for Information Science Christine L. Borgman (1996) “Why are online catalogs still hard to use?” Journal of the American Society for Information Science

The Internet is Shaping Expectations Simple searching - Google, Yahoo, Ask.com Patrons are used to doing it themselves Want fast access to results, ranked by relevance Commercial spaces are: interactive/collaborative content creation (flickr, blogs, facebook, etc.) tagging (flickr, del.icio.us, etc.) content re-use (RefWorks/RefShare, etc.) Syndicated Blogs News sites like cnn.com Google mash-ups It’s not just changing them, it is creating expectations.

Where We Are Right Now OPAC: search interface on inventory system, made for librarians, not patrons – designed around the card catalog which limits its effectiveness Catalogs are usually best for known-item searching, not topic searching Improvements have typically been window dressing, don’t deal with many fundamental limitations of the catalog Users demand interfaces that they are used to (Google, Amazon, etc.) Don't have to ask Amazon how to search for items, why expect libraries? … and even then, the finding time video

Silo-ization at Every Turn Content Silos Science-Direct Web of Science JSTOR ETDs EEBO Catalog ILL Website Meta-search eReserve System Silos

How Users Search Most people make typos at least some of the time Most searches are 2, 3, 4 words with no Boolean operators Most searches use keyword Search is hesitant, iterative, often random process of discovery Most people start elsewhere Few read help screens Few use advanced search – this is true even in Google

Where Users Search OCLC study 84% users start in search engines vs. 2% at library websites/portals

Destination vs. Syndication Models The Internet Library Web Site

The OPAC “Sucks” The OPAC lacks common features of most search engines Relevance ranking vs. last in, first out Spell checking (related - did you mean?) Popular query operators like + and – Refine search Sort flexibility Faceting Citation indexing vs full text Developed for print materials, limitations with electronic materials or atomized items (like articles) Difficult for certain known item search Karen Schneider, Andrew Pace, Roy Tennant

Industry Trends Decouple the front end (search and discovery) from the back end (inventory and cataloging) Service Oriented Architecture – many programs loosely coupled The 5th generation of ILS upon us Existing systems will probably be superseded within the next 2 – 10 years

Dis-integrating the ILS The future of the ILS could be ND Central Accounting ILS Acquisitions Cataloging eResource Mangement Serials Circulation Authority

De-coupling – What Does This Mean? Keep the business software – the ILS, change the interface Most Next Gen features require different infrastructure, some don’t index vs. database Speed Relevance Faceted browse Did you mean FRBR Decoupling the interface is really the first step in disintegrating the integrated library system, which is going to happen anyway.

“Grand Challenge” for LIS A single interface that searches the world of knowledge and brings back all of the most relevant and authoritative resources that match a user’s query It’s our “Holy Grail” - we've been working on this one for about 30 years :) Next Generation Interfaces open the possibility to get us closer

Characteristics of Next Generation Search Enhanced Search Functionality Faceted browse Relevance ranking Did you mean / Spell Checking auto-correction, resubmit search De-silo-ization Integrating search for books, articles, etc. Single, Simple Search Box FRBR – functional requirements for bibliographic record, grouping editions

Characteristics of Next Generation Search Enhanced Experience Sometimes fun and engaging Interactive/Collaborative User centered design Enhanced Services Find it / Get it for me Book Covers / Synopsis Full text Availability on same page as results

Characteristics of Next Generation Search Enhanced Content Article Searching Commercial Data Merging Special Collections Harvesting Online Collections Grey Literature Free Content Enhanced Access Syndication - Getting into users tools Course Management Systems Browser and Desktop Tool Bars Portals

Next Generation Search Interfaces Next Generation Catalog Next Generation “Unified Search” Aid Vendor Data Vendor Data Full Text OAI User Interface ILS OPAC Indexing System MetaSearch MARC Circ Data

Why Should We Consider This? Next Generation Systems are an evolutionary step, it is the first step toward the next generation ILS – at some point we will have to do something They are designed to: meet our user’s search/retrieval expectations reduce the amount of work and expertise needed to get from query to research material improve service

Resource Discovery Assessment Workgroup Ten individuals, representative of Libraries and one consortia rep (Kitty!) outside Notre Dame Group Makeup Public Services, Cataloging, Collections, Systems Seven weeks to: Identify and assess commercial and open-source offerings in 'next generation' catalogs Educate ourselves on pros and cons of each Make recommendations to library administration on which 2-4 to consider as finalists, with comparative analysis Probably worth mentioning here that we didn’t do RFP and why that was beneficial – we did homework instead and why that was the best way. After experience, it would have been useful to have more time, but not much. Attended meeting last year about downsides of RFPs.

Problems Needing Solutions Limitations with existing catalog Coarse relevancy functionality Non-intuitive to users used to Google Hard to expand searches beyond local holdings Content that didn’t fit into catalog Encoded Archival Description (manuscripts) Image collections Dublin Core-based descriptive collections Add examples 23

Problems Needing Solutions Wanted a platform to build new services on, not just a new UI Social networking capabilities: review items, tag items Add examples 24

Desired functionality Ability to add plurality of metadata types: MARC, DC, XML-encoded data Extensible and modular; platform for innovation Provide clear improvements to identified problems/shortcomings with current catalog Offer a high-degree of customization options Informed by DLF recommendations on ILS & Discovery Systems

Background & Environmental Scan OCLC Environmental Scan (2003) Calhoun Report (2006) Rochester undergrad research project (2007) OCUL Scholar's Portal 2 discussion paper (2007) LoC Future of Bibliographic Control Report (2008) JISC & SCONUL LMS Survey (2008)

Candidate Systems MediaLab Villanova Univ. Innovative Interfaces Inc. Endeca Technologies Inc. Ex Libris Group OCLC Aarhus Univ. Index Data Univ. of Rochester LibLime

Analysis Details 44 questions in twelve areas To what extent does each solution meet/exceed our desired functionality requirements? Answered questions by: Using the solutions at other sites Referring to product literature Ask on support websites Detailed inquiries to sales people

Comparative Analysis Categories Search functionality Relevance ranking, FRBR, Facted Browse, Visualization, etc. Spelling correction & suggestion Did you mean?, Alternate Terms, etc. User experience Usable Interface, Pleasing Aesthetics, Customizable, etc. Index capability Metasearch Integration, API Integration, etc. Record enrichment Book Covers, TOCs, Reveiws, etc.

Comparative Analysis Categories Supported data formats & sources MARC, Dublin Core, EAD, TEI, I&A Data, Fulltext, etc. ILS integration Circulation Availablility Social computing Reviews, Ratings, Tagging, etc. Enhanced services RSS, Email Notification, Bibliographic Management Export, etc. System and personnel requirements Hardware, Support, Ease of Management, Statistics, etc.

Comparative Analysis Categories APIs Web services, SDKs, etc. Support Options Vendor, Community, etc.

What did we find? A spectrum of features, functionality and integratability I bet you thought we were going to give you all the answers

Methodology Observed Courseware Group Simplified process No weighting, aggregate numbers, yes – no – maybe Condensed timeframe Made Evaluation Variables Explicit Spent the first half of process getting consensus on categories Helped handle disagreement Asked to provide recommendations Gave comparative analysis of top 3 candidates

Evaluating the Options Challengers Leaders Aquabrowser Primo VUFind Encore Evergreen ILS WorldCat Local Ability to Execute Endeca Koha Summa Zebra Extensible Catalog (XC) Open Library Niches Visionaries Completeness of Vision

How Do You Decide? Needs to be in the context of your institution How much money can you spend? What is your timeline for implementation? How much technical expertise do you have? How much technical time do you have? Is open source part of your institutional culture? What companies do you do business with?

Thank You Questions?