1. Discovery and Discoverability
Current Issues and Trends
Marshall Breeding
Independent Consultant, Author, and
Founder and Publisher, Library Technology Guides
Computers in Libraries 2018
April 19, 2018

2. Description
Libraries make great investments in creating collections for the benefit of their communities. In order to get the most out of these investments, libraries also need to have effective strategies for discovery and access. Libraries can implement discovery systems to enable the patrons who visit their websites to search and gain access to materials. But it is also important to find ways to make their collections easily found in other contexts, such as through Google searches or course pages in learning management systems. Breeding reviews the current state of discovery products and the technologies and services able to make library collections more widely discoverable beyond the library-provided interfaces.

3. Models of Content Scope

4. Online Catalog Scope of Search
ILS Data
Online Catalog
Search:
Scope of Search
Books, Journals, and Media at the Title Level
Not in scope:
Articles
Book Chapters
Digital objects
Search Results

5. Web-scale Index-based Discovery
ILS Data
Web-scale Index-based Discovery
(2009- present)
Digital Collections
Search:
Web Site Content
Institutional Repositories
Aggregated Content packages
Search Results
Consolidated Index
Open Access …
E-Journals
Usage-generated Data
Customer Profile
Reference Sources
Pre-built harvesting and indexing

6. Bento Box Discovery Model
Aggregated Content packages
Search:
Open Access
ILS Data
VuFind / Blacklight
E-Journals
Consolidated Index
Search Results
Web Site Content
Digital Collections
Institutional Repositories
Pre-built harvesting and indexing

7. E-Book Integration Model
Aggregated Content packages
Search:
ILS Data
Library Catalog
Index
Search Results
Web Site Content
Digital Collections
Authentication
Checkout - Download
External
E-Book Lending Service
Local E-book
Repository
Discovery

8. Patron Privacy in Discovery

9. Online Privacy Concern
Operating websites and discovery services without encryption enables third parties to intercept online activity of library patrons
Search terms
Search Results
Items selected
Items downloaded and read
Important to implement https:
Provides end-to-end encryption of user session

10. HTTPS urgency for libraries
Browsers increasingly warn that http sites are insecure or untrusted
Commercial sites mostly encrypt with https:
Facebook – Twitter – Amazon - …..
Libraries continue to implement https inconsistently
Running websites and discovery services on http is inconsistent with library values to protect the privacy of patrons

11. US public libraries using HTTPS (2017)

12. US public libraries using HTTPS (2018)

13. Tracking tags
Involves the insertion of code snippets or images which allow external entities to capture information regarding the use of a web-based resource
Especially used by advertising networks
Also used in performance monitoring
Allows capture of user behavior even if page delivery is encrypted
Uses third party cookies or unique identifiers to track users
Libraries may not be aware of the tracking tags present within their pages
Ghostery is an example of a tool to detect trackers

16. Contemporary Discovery Productds

17. Discovery Services for Public Libraries

18. Public Library Discovery Trends
Most public libraries use the online catalog module or discovery services from their ILS provider
Minority of public libraries replacing native catalog with third party products
BiblioCore form BiblioCommons – larger libraries and consortia
AquaBrowser – Once popular, now almost extinct
VuFind – open source discovery interface
Pika – open source discovery interface derived from VuFind

19. Emphasis on Digital Lending Services
Most public libraries offer some type of e-book lending service
Ongoing reliance on content provided by OverDrive, Recorded Books, Bibliotheca, Odilo and others
Ability for library to maintain repository of e-books it owns (Douglas County Model)
Supported by Pika, Odilo, SimplyE
Interest in library centered e-book lending solutions
NYPL: SimplyE App (created as part of the Library Simplified initiative)

20. Phases of E-book integration
Link to provider’s platform
Load bibliographic records in catalog
API integration to enable full discovery, search, and download from library catalog
Library-centered e-book lending platform

21. BiblioCommons Interface services for public libraries:
BiblioCore
BiblioWeb
Based on extensive user experience research
Incorporates elements of social and sharing
Multi-tenant hosted service
Adopted by many major municipal libraries, consortia, as well as mid-sized

23. AquaBrowser
One of the original discovery interfaces, developed in The Netherlands
Featured a graphic word cloud of related concepts for exploring search results
Originally marketed in US by The Library Corporation
Once acquired by Bowker, TLC developed its own discovery interface (LS2 PAC)
Now owned by ProQuest, but not developed or marketed

25. Open Source Discovery Interfaces

26. VuFind
Open Source discovery interface originally created by Villanova University (2007)
Based on PHP programming framework and SOLR index for relevancy
Harvests data from ILS and other sources into SOLR to support relevancy based results
Integrates with APIs of index-based discovery services
Highly customizable
Deployed in thousands of libraries, US and internationally
Though originally developed for academics, now more popular in public libraries

28. Pika Branch of VuFind developed by Marmot Library Network
Key features:
E-book integration
Content enrichment
Recommendations (Novelist)
Record grouping
Digital archive
ILS integration
See:

31. Blacklight
Open source discovery interface originally developed by University of Virginia and Stanford University
Based on Ruby on Rails programming framework
Uses Apache SOLR for relevancy-based search results
Primarily implemented by Academic libraries
Integrates with the API of index-based discovery services

33. Library Portal Products
Console managed library websites with integrated discovery

34. From Discovery to Content Management
Extend the discovery environment to not only replace the online catalog, but to replace the full library web site
Library-specific content management system
Fully integrate discovery into the library’s web environment
Include all content and services on the site into discovery
Comprehensive library portal

35. Examples BiblioCommons: EBSCO / Global Forge Infor Axiell
BiblioCore + BiblioWeb
EBSCO / Global Forge
Stacks
Infor
Iguana
Axiell
Arena
Toronto Public Library
Endeca-powered catalog + Web site

36. Blend website and discovery
Provide a content management system for the library web site fully integrated with discovery and other library services
Avoid discordant look and feel between web site and catalog
Opportunities for better technical integration
Reduce the complexity of managing library web site
Maintain consistent branding, presentation, content

37. BiblioWeb Product introduced by BiblioCommons in 2014
Developed in partnership with Chicago Public Library
Hosted by BiblioCommons
Access via library’s domain name
Designed to handle full range of content items and services delivered through a library’s web site

39. Stacks Built in partnership with EBSCO Information Services
Web content management system created specifically for libraries
Responsive design
Drag and Drop content management and layouts
Customized presentation via CSS
Multi-Domain
Breeding, Marshall “Stacks: A New Service for Deploying Mobile-Friendly Library Websites” Smart Libraries Newsletter. Nov

41. Index-based Discovery Services
Mostly for Academic and Research libraries

42. Web-scale Index-based Discovery
ILS Data
Web-scale Index-based Discovery
(2009- present)
Digital Collections
Search:
Web Site Content
Institutional Repositories
Aggregated Content packages
Search Results
Consolidated Index
Open Access …
E-Journals
Usage-generated Data
Customer Profile
Reference Sources
Pre-built harvesting and indexing

43. EBSCO Discovery Service
Most widely deployed index-based discovery service
Extends EBSCOhost technology to the full body of scholarly literature (ideally)
Assumes the primacy of subject indexing over keyword relevancy
Designed to operate with any ILS product
Partnerships with over 60 ILS vendors

45. WorldCat Discovery Service
Based on the WorldCat bibliographic service
Largest representation of monographic content
Enhanced with article-level citation data for all scholarly literature (ideally)
Default interface for WorldShare Management Services
WorldCat Local can be integrated with other ILS products

47. Summon Originally developed by Serials Solution
Became a division of ProQuest
Full featured discovery interface, though not extensively customizable
Central index populated with citations and full text from body of scholarly literature
Relevancy rankings based on keyword algorithms optimized for scholarly resources
Product will be maintained following merger with Ex Libris
Central index will be synchronized with Primo Central

49. Primo / Primo Central Discovery interface introduced in 2007
Highly customizable, ILS content + repositories
Customized content integration via “pipes” which define indexing rules, presentation, relevancy
Primo Central index added in 2009
Designed to be integrated with any major ILS product
Default interface for Alma

50. Primo / Primo Central Discovery index introduced by Ex Libris in 2006
Primo Central index added in 2009
Designed to operate with any ILS
Default discovery interface for Alma
Highly customizable
Interface
Multiple content repositories via Pipes
Blended with results from Primo Central

51. Possibilities for Open Access discovery index
Open source tools exist for discovery Interfaces:
VuFind
Blacklight
No open access discovery indexes
High threshold of expense and difficulty to build index
Platform costs
Software development
Publisher relations
Billions of content items to index and maintain

52. Yewno Discovery
New approach to discovery based on exploring concepts rather than keywords
Spin-off from Stanford University, led by Ruggero Gramatica
Uses machine learning to extract concepts from documents
Visual interface for navigating and exploring concepts within the body of documents

53. Yewno Discovery interface
Quick demo:

54. Discoverability
Enabling discovery of library resources through general web search engines

55. Discovery happens elsewhere*
Only a portion of library users come to the libraries own website, catalog, or discovery service for research
Important to have library-provided discovery options
Also essential to ensure that library resources can be discovered and accessed through the general web.
Google Scholar, Google, Microsoft Academic Search, etc
Use Search Engine Optimization methodologies to expose library resources
Lorcan Dempsey, VP and Chief Strategist OCLC

56. More Distributed Discovery
Address the reality that discovery takes place outside of library provided interfaces
Optimized exposure in the ecosystem of search engine and social network
Not Concentrated on the Library web site
Expression of discovery services via other campus tools and portals and beyond

57. Multi-layered discovery
Native interfaces of specialized content services
Disciplinary aggregations
General library discovery tools
Global Internet-based discovery

58. Library opts out of Discovery
Utrecht University Library
Decision to not implement a discovery service but to rely entirely on Google Scholar and other general and scholarly search engines
for-literature/searching-for-articles-books-theses
(ongoing strategy as of 2017)
Kortekaas , Simone. “Thinking the unthinkable: a library without a catalogue — Reconsidering the future of discovery tools for Utrecht University library.” LIBER General Annual Conference 2012

59. Discoverability Strategies
Search Engine Optimization for library resources
Linked data models:
Schema.org
BIBFRAME
Commercial products
Demco Software DiscoveryOocal
Zepheira: Library.Link network, BIBFRAME, etc
SirsiDynix BLUEcloud Visibility
Innovative
EBSCO Information Services

60. Schema.org
Structured data methodology to deliver web-based resources that can be understood by computers as well as for presentation to humans
Originally developed by Google, now widely supported by all search engines
See:

61. Schema.org coding example
<div itemscope="itemscope" itemtype=" <h2><meta itemprop="name" content="Nashville Public Library" /> <span itemprop="legalName">Nashville Public Library</span></h2> <div itemprop="address" itemscope itemtype=" Address: </span><span itemprop="streetAddress"> 615 Church Street<br /></span> …. </div> <!-- End microdata itemscope div -->

63. Google Structured Data Testing Tool

64. Zepheira Company specializing in linked data
Involved in the development of BIBFRAME
LibHub initiative: expose library data for discoverability
Library.Link network
open, decentralized, shared data platform
Partnerships:
SirsiDynix BLUEcloud Visibility
Innovative: Linked Data
EBSCO: Linked Library Service

65. Demco DiscoverLocal
Designed to improve discoverability of library resources
Library uploads records into Demco DiscoverLocal
Records transformed into links harvested by search engines, enhanced with geolocation data
Links are harvested by search indexing bots to better direct searches to local library’s collection

66. Koios SEO products for library collections Libre Ads
Facilitates application for Google Ad Grants
Help libraries set up ad campaigns
Maintain grant
Ongoing assistance in marketing campaigns

67. BiblioCommons Built-in search engine optimization
Rich linking environment – one of the top characteristics Google considers for page ranking
Library resources from libraries using BiblioCommons are well represented in general search results on Google

68. Mobile Discovery
Google and other search engines increasingly consider search for mobile devices separately than general web search
Avoid linking mobile devices to pages that they cannot display well
Non-mobile friendly websites demoted or eliminated in mobile search results
Essential for libraries to have responsive or mobile- friendly web sites

69. Resource Discovery Trends

70. Integrate Discovery with Resource Management?
Should there be a tight bundling of Discovery Services with Library Services Platforms:
WorldCat Discovery Services + WorldShare Management Services
Alma + Primo or Summon
Also support for Blacklight or VuFind
EBSCO rejects tight bundling
Partners with almost all ILS products
Support for open source FOLIO project
Many libraries prefer providing discovery separately

71. Discovery from Content Providers
Two of the major discovery services are tied to content providers:
EBSCO Information Services
EBSCO Discovery Service
ProQuest:
Summon
Primo + Primo Central
Explicit bias would not be tolerated by libraries
Providing discovery services can provide insight into content acquisition and patron behaviors

72. Open Discovery Initiative (ODI)

73. Goals of Open Discovery Initiative
Define ways for libraries to assess the level of content provider participation and for discovery services to affirm how they use that content
Help streamline the process by which content providers work with discovery service vendors
Define models for “fair” linking from discovery services to publishers’ content
Determine what usage statistics should be collected for libraries and for content providers
From Library perspective: Web-scale discovery systems are increasingly important to the work of libraries in service of their users. As these discovery systems become more complex, librarians are less able to understand or explain to their users what content is included or how. The Open Discovery Initiative's recommended practice represents a significant opportunity to understand what is indexed, where it comes from, and how it is used.  Increasing need to ensure that the coverage meets our needs.

74. Recommended Practice
A technical recommendation for data exchange including data formats, method of delivery, usage reporting, frequency of updates and rights of use
A way for libraries to assess content providers’ participation in discovery services
A model by which content providers work with discovery service vendors via fair and unbiased indexing and linking
From Library perspective: Web-scale discovery systems are increasingly important to the work of libraries in service of their users. As these discovery systems become more complex, librarians are less able to understand or explain to their users what content is included or how. The Open Discovery Initiative's recommended practice represents a significant opportunity to understand what is indexed, where it comes from, and how it is used.  Increasing need to ensure that the coverage meets our needs.

75. Library Discovery Futures
Beyond library-provided discovery

76. The future of Resource Discovery
Discovery more integrated into a broader view of library content and services
More comprehensive discovery indexes
Discovery services enhanced by AI and machine learning
Stronger technologies for search and retrieval
Discovery beyond library-provided interfaces
Linked Data to supplement discovery indexes

77. Future of discovery service products
Remain one of the essential components of library technology infrastructure
Loosely or tightly tied to resource management
Increased sophistication in direct discovery and delivery functionality
Increased expectation to syndicate content to local and global discovery context

78. The next phase of Discovery
Now in a critical point for discovery
Current products evolve
Reaching limits of the prevailing architecture?
Current set of products and services an interim step
Important for stakeholders to engage in defining the future of library resource discovery
Future products must address expected changes in scholarly publishing, library priorities, and institutional strategies.

79. Questions and discussion