Cloud Computing in Libraries and Web-scale Library Management and Discovery Marshall Breeding Independent Consultant, Author, Speaker Founder and Publisher, Library Technology Guides March 2013 SENYLRC

Abstract This is an introduction to the concepts of cloud computing and how this suite of technologies is positioned to re- shape the way’s that libraries make use of strategic applications such as discovery and management applications. The instructor will describe the evolution of discovery systems from next-generation library catalogs that provided some improvements in the interfaces and performance of the established online catalogs toward the current wave of index-based or Web-scale discovery services. Major changes are also underway in the applications that libraries use to manage their operations and collections, with a new slate of library services platforms coming on the scene, providing an alternative to the integrated library systems that have been available for many decades.

Cloud Computing for Libraries  Volume 11 in The Tech Set  Published by Neal- Schuman / ALA TechSource  ISBN:  Book ImagePublication Info:

Appropriate Automation Infrastructure  Current automation products out of step with current realities  Majority of library collection funds spent on electronic content  Majority of automation efforts support print activities  New discovery solutions help with access to e- content  Management of e-content continues with inadequate supporting infrastructure

Key Context: Libraries in Transition  Academic Shift from Print > Electronic  E-journal transition largely complete  Circulation of print collections slowing  E-books now in play (consultation > reading)  All libraries:  Need better tools for access to complex multi-format collections  Strong emphasis on digitizing local collections  Demands for enterprise integration and interoperability

Key Text: Changed expectations in metadata management  Moving away from individual record-by-record creation  Life cycle of metadata  Metadata follows the supply chain, improved and enhanced along the way as needed  Manage metadata in bulk when possible  E-book collections  Highly shared metadata  E-journal knowledge bases, e.g.  Great interest in moving toward semantic web and open linked data  Very little progress in linked data for operational systems  AACR2 > RDA  MARC > Bibframe (

Fundamental technology shift  Mainframe computing  Client/Server  Web-based and Cloud Computing

Local Computing  Traditional model  Locally owned and managed  Shifting from departmental to enterprise  Departmental servers co-located in central IT data centers  Increasingly virtualized

Virtualization  The ability for multiple computing images to simultaneously exist on one physical server  Physical hardware partitioned into multiple instances using virtual machine management tools such as VMware  Applicable to local, remote, and cloud models

Cloud Computing  Major trend in Information Technology  Term “in the cloud” has devolved into marketing hype, but cloud computing in the form of multi- tenant software as a service offers libraries opportunities to break out of individual silos of automation and engage in widely shared cooperative systems  Opportunities for libraries to leverage their combined efforts into large-scale systems with more end-user impact and organizational efficiencies

Beyond “Cloudwashing”  Cloud as marketing hype  Cloud computing used very freely, tagged to almost any virtualized environment  Any arrangement where the library relies on some kind of remote hosting environment for major automation components  Includes almost any vendor-hosted product offering  Example: ASP now Software-as-a-Service

Cloud computing – characteristics  Web-based Interfaces  Externally hosted  Pricing: subscription or utility  Highly abstracted computing model  Provisioned on demand  Scaled according to variable needs  Elastic – consumption of resources can contract and expand according to demand

Gartner Hype Cycle 2009

Gartner Hype Cycle 2010

Gartner Hype Cycle 2011

Gartner Hype Cycle 2012

Budget Allocations  Server Purchase  Server Maintenance  Application software license  Data Center overhead  Energy costs  Facility costs  Annual Subscription  Measured Service?  Fixed fees  Factors  Hosting  Software Licenses  Optional modules Local ComputingCloud Computing

Infrastructure-as-a-service  Provisioning of Equipment  Servers, storage  Virtual server provisioning  Examples:  Amazon Elastic Compute Cloud (EC2)  Amazon Simple Storage Service (S3)  Rackspace Cloud (  EMC 2 Atmos (

Amazon EC2  Amazon Machine Instances (AMI)  Red Hat Enterprise Linux  Debian  Fedora  Ubuntu Linux  Open Solaris  Windows Server 2003/2008

Storage-as-a-Service  Provisioned, on-demand storage  Bundled to, or separate from other cloud services

Software as a Service  Multi Tennant SaaS is the modern approach  One copy of the code base serves multiple sites  Software functionality delivered entirely through Web interfaces  No workstation clients  Upgrades and fixes deployed universally  Usually in small increments

Data as a service  SaaS provides opportunity for highly shared data models  Bibliographic knowledgebase: one globally shared copy that serves all libraries  Discovery indexes: article and object-level index for resource discovery  E-resource knowledge bases: shared authoritative repository of e-journal holdings  General opportunity to move away from library-by- library metadata management to globally shared workflows

Software-as-a-Service  Complete software application, customized for customer use  Software delivered through cloud infrastructure, data stored on cloud  Eg: Salesforce.com—widely used business infrastructure  Multi-tenant: all organizations that use the service share the same instance (codebase, hardware resources, etc)  Often partitioned to separate some groups of subscribers

Application service provider  Legacy business applications hosted by software vendor  Standalone application on discrete or virtualized hardware  Staff and public clients accessed via the Internet  Same user interfaces and functionality as if installed locally  Established as a deployment model in the 1990’s  Can be implemented through Infrastructure-as-a Service  Individual instances of legacy system hosted in EC2

ASP vs SaaS From: THINKstrategies: CIO’s Guide to Software-as-a-Service

Platform-as-a-Service  Virtualized computing environment for deployment of software  Application engine, no specific server provisioning  Examples:  Google App Engine SDKs for Java, Python  Heroku: ruby platform  Amazon Web Service  Library Specific platforms  WorldShare Platform

Library Context Cloud Computing

Library automation through SaaS  Almost all library automation products offered through hosted options  SaaS or ASP?

ILS Products offered as SaaS (mostly ASP)  SirsiDynix Symphony  SirsiDynix Horizon  Innovative Interfaces Millennium  Ex Libris Aleph  EOS International EOS.Web  Evergreen – Equinox Software  Koha – LibLime, ByWater, many others internationally  …many other examples …

Multi-tenant SaaS  Serials Solutions  Summon  Intota (Announced for )  360 Search, 360 Link, KnowledgeWorks  Ex Libris  Alma  Primo Central  BiblioCommons  OCLC WorldShare Management Services

Platform as a Service  OCLC WorldShare Platform  WorldShare Management Services  WorldShare License Manager  Library-created applications

Library Management in the Cloud  Almost all library automation vendors offer some form of “cloud-based” services  Server management moves from library to Vendor  Subscription-based business model  Comprehensive annual subscription payment  Offsets local server purchase and maintenance  Offsets some local technology support

Leveraging the Cloud  Moving legacy systems to hosted services provides some savings to individual institutions but does not result in dramatic transformation  Globally shared data and metadata models have the potential to achieve new levels of operational efficiencies and more powerful discovery and automation scenarios that improve the position of libraries overall.

Transition to Web-scale Technologies  Web-scale: a characterization or marketing tag that denotes a comprehensive, highly-scalable, globally shared model  Web-scale: One of the key characteristics of emerging library management and discovery services  Displaces applications or data models targeting individual libraries in isolation  Discovery: index-based search  Management: Library Services Platforms

Repositories in the cloud  Dspace – institutional repository application  Fedora – generalized repository platform  DuraSpace – organization now over both Dspace and Fedora  DuraCloud – shared, hosted repository platform  Pilot since 2009, production in early 2011 

Caveats and concerns with SaaS  Libraries must have adequate bandwidth to support access to remote applications without latency  Quality of service agreements that guarantee performance and reliability factors  Configurability and customizability limitations  Access to API’s  Ability to interoperate with 3 rd party applications  Eg: Connect SaaS ILS with discovery product from another vendor

Benefits of Cloud Computing  Elimination of capital expenses for equipment  Lower annual costs  Redeployment of technical staff to more meaningful activities  Higher revenues relative to software-only arrangements  Provision of infrastructure at scale with lower unit costs  Longer-term relationships with customers Libraries Providers / Vendors

Cost implications  Total cost of ownership  Do all cost components result in increased or decreased expense  Personnel costs – need less technical administration  Hardware – server hardware eliminated  Software costs: subscription, license, maintenance/support  Indirect costs: energy costs associated with power and cooling of servers in data center  IaaS: balance elimination of hardware investments for ongoing usage fees  Especially attractive for development and prototyping

Risks and concerns  Privacy of data  Policies, regulations, jurisdictions  Ownership of data  Avoid vendor lock-in  Integrity of Data  Backups and disaster recovery

Security issues  Most providers implement stronger safeguards beyond the capacity of local institutions  Virtual instances equally susceptible to poor security practices as local computing

Cloud computing trends for libraries  Increased migration away from local computing toward some form of remote / hosted / virtualized alternative  Cloud computing especially attractive to libraries with few technology support personnel  Adequate bandwidth will continue to be a limiting factor

Increased pressure  Library automation vendors promoting SaaS offerings  Some companies already exclusively SaaS  Software pricing increasingly favorable to SaaS

Caveat  technologies promoted by companies and organizations have a vested interest in their adoption  Critically assess viability of the technology and its appropriateness for your organization

A New Generation of Resource Discovery

Next-Gen Library Catalogs Marshall Breeding Neal-Schuman Publishers March 2010 Volume 1 of The Tech Set

Online Catalog  Books, Journals, and Media at the Title Level  Not in scope:  Articles  Book Chapters  Digital objects Scope of Search Search: Search Results ILS Data

Next-gen Catalogs or Discovery Interface  Single search box  Query tools  Did you mean  Type-ahead  Relevance ranked results  Faceted navigation  Enhanced visual displays  Cover art  Summaries, reviews,  Recommendation services  Books, Journals, and Media at the Title Level  Other local and open access content  Not in scope:  Articles  Book Chapters  Digital objects  Scope of Search

Discovery from Local to Web-scale  Initial products focused on interface improvements  AquaBrowser, Endeca, Primo, Encore, VuFind,  LIBERO Uno, Civica Sorcer, Axiell Arena  Mostly locally-installed software  Current phase is focused on pre-populated indexes that aim to deliver Web-scale discovery  Primo Central (Ex Libris)  Summon (Serials Solutions)  WorldCat Local (OCLC)  EBSCO Discovery Service (EBSCO)  Encore Synergy (no index, though)

Discovery Interface search model Search: Digital Collections ProQuest EBSCOhost … MLA Bibliography ABC-CLIO Search Results Real-time query and responses ILS Data Local Index MetaSearch Engine

Web-scale Index-based Discovery Search: Digital Collections Web Site Content Institutional Repositories … E-Journals Reference Sources Search Results Pre-built harvesting and indexing Consolidated Index ILS Data Aggregated Content packages (2009- present)

Public Library Information Portal Search: Digital Collections Web Site Content Community Information … Customer- provided content Reference Sources Search Results Pre-built harvesting and indexing Consolidated Index LMS Data Aggregated Content packages Archives Usage- generated Data Customer Profile

Web-scale Search Problem Search: Search Results Pre-built harvesting and indexing Consolidated Index ?? ? Non Participating Content Sources Non Participating Content Sources Problem in how to deal with resources not provided to ingest into consolidated index Digital Collections Web Site Content Institutional Repositories … E-Journals ILS Data Aggregated Content packages

Discovery Products

Challenge for Relevancy  Technically feasible to index hundreds of millions or billions of records through Lucene or SOLR  Difficult to order records in ways that make sense  Many fairly equivalent candidates returned for any given query  Must rely on use-based and social factors to improve relevancy rankings

Challenges for Collection Coverage  To work effectively, discovery services need to cover comprehensively the body of content represented in library collections  What about publishers that do not participate?  Is content indexed at the citation or full-text level?  What are the restrictions for non-authenticated users?  How can libraries understand the differences in coverage among competing services?

Open Discovery Initiative  NISO Work Group to Develop Standards and Recommended Practices for Library Discovery Services Based on Indexed Search  Informal meeting called at ALA Annual 2011  Co-Chaired by Marshall Breeding and Jenny Walker  Term: Dec 2011 – May

Balance of Constituents 57 Marshall Breeding, Vanderbilt University Jamene Brooks-Kieffer, Kansas State University Laura Morse, Harvard University Ken Varnum, University of Michigan Anya Arnold, Orbis Cascade Alliance Sara Brownmiller, University of Oregon Lucy Harrison, College Center for Library Automation (D2D liaison/observer) Michele Newberry, Florida Virtual Campus Lettie Conrad, SAGE Publications Beth LaPensee, ITHAKA/JSTOR/Portico Jeff Lang, Thomson Reuters Linda Beebe, American Psychological Assoc Aaron Wood, Alexander Street Press Roger Schonfeld, JSTOR, Ithaka Jenny Walker, Ex Libris Group John Law, Serials Solutions Michael Gorrell, EBSCO Information Services David Lindahl, University of Rochester (XC) Jeff Penka, OCLC (D2D liaison/observer)

ODI Project Goals:  Identify … needs and requirements of the three stakeholder groups in this area of work.  Create recommendations and tools to streamline the process by which information providers, discovery service providers, and librarians work together to better serve libraries and their users.  Provide effective means for librarians to assess the level of participation by information providers in discovery services, to evaluate the breadth and depth of content indexed and the degree to which this content is made available to the user.

New-generation Library Management

Fragmented Library Management  LMS for management of (mostly) print  Duplicative financial systems between library and local government or other parent organization  E-book lending platform (multiple?)  Interlibrary loan (borrowing and lending)  Self-service and AMH infrastructure  Electronic Resource Management  PC Scheduling and print management  Event scheduling  Digital Collections Management platforms (CONTENTdm, DigiTool, etc.)  Discovery-layer services for broader access to library collections  No effective integration services / interoperability among disconnected systems, non-aligned metadata schemes

Integrated (for print) Library System Circulation BIB Staff Interfaces: Holding / Items Circ Transact UserVendorPolicies $$$ Funds CatalogingAcquisitionsSerialsOnline Catalog Public Interfaces: Interfaces Business Logic Data Stores

LMS / ERM: Fragmented Model Circulation BIB Staff Interfaces: Holding / Items Circ Transact UserVendorPolicies $$$ Funds CatalogingAcquisitionsSerialsOnline Catalog Public Interfaces: Application Programming Interfaces ` License Management License Terms E-resource Procurement Vendors E-Journal Titles Protocols: CORE

Common approach for ERM Circulation BIB Staff Interfaces: Holding / Items Circ Transact UserVendorPolicies $$$ Funds CatalogingAcquisitionsSerialsOnline Catalog Public Interfaces: Application Programming Interfaces Budget License Terms Titles / Holdings Vendors Access Details

Comprehensive Resource Management  No longer sensible to use different software platforms for managing different types of library materials  ILS + ERM + OpenURL Resolver + Digital Asset management, etc. very inefficient model  Flexible platform capable of managing multiple type of library materials, multiple metadata formats, with appropriate workflows

Academic Libraries need a new model of library management  Not an Integrated Library System or Library Management System  The ILS/LMS was designed to help libraries manage print collections  Generally did not evolve to manage electronic collections  Other library automation products evolved:  Electronic Resource Management Systems – OpenURL Link Resolvers – Digital Library Management Systems -- Institutional Repositories

Library Services Platform  Library-specific software. Designed to help libraries automate their internal operations, manage collections, fulfillment requests, and deliver services  Services  Service oriented architecture  Exposes Web services and other API’s  Facilitates the services libraries offer to their users  Platform  General infrastructure for library automation  Consistent with the concept of Platform as a Service  Library programmers address the APIs of the platform to extend functionality, create connections with other systems, dynamically interact with data

Library Services Platform Characteristics  Highly Shared data models  Knowledgebase architecture  Some may take hybrid approach to accommodate local data stores  Delivered through software as a service  Multi-tenant  Unified workflows across formats and media  Flexible metadata management  MARC – Dublin Core – VRA – MODS – ONIX  Bibframe  New structures not yet invented  Open APIs for extensibility and interoperability

Open Systems  Achieving openness has risen as the key driver behind library technology strategies  Libraries need to do more with their data  Ability to improve customer experience and operational efficiencies  Demand for Interoperability  Open source – full access to internal program of the application  Open API’s – expose programmatic interfaces to data and functionality

Consolidated index Unified Presentation Layer Search: Digital Coll ProQuest EBSCO … JSTOR Other Resources New Library Management Model ` API Layer Library Services Platform Learning Management Enterprise Resource Planning Stock Management Self-Check / Automated Return Authentication Service Smart Cad / Payment systems Discovery Service

Library Services Platforms Category WorldShare Management Services AlmaIntota Sierra Services Platform Kuali OLE Responsible Organization OCLC.Ex Libris Serials Solutions Innovative Interfaces, Inc Kuali Foundation Key precepts Global network-level approach to management and discovery. Consolidate workflows, unified management: print, electronic, digital; Hybrid data model Knowledgeba se driven. Pure multi- tenant SaaS Service-oriented architecture Technology uplift for Millennium ILS. More open source components, consolidated modules and workflows Manage library resources in a format agnostic approach. Integration into the broader academic enterprise infrastructure Software model Proprietary Open Source

Library Services Platforms Category WorldShare Management Services AlmaIntota Sierra Services Platform Kuali OLE Responsible Organization OCLC.Ex Libris Serials Solutions Innovative Interfaces, Inc Kuali Foundation Key precepts Global network-level approach to management and discovery. Consolidate workflows, unified management: print, electronic, digital; Hybrid data model Knowledgeba se driven. Pure multi- tenant SaaS Service-oriented architecture Technology uplift for Millennium ILS. More open source components, consolidated modules and workflows Manage library resources in a format agnostic approach. Integration into the broader academic enterprise infrastructure Software model Proprietary Open Source

Development / Deployment perspective  Beginning of a new cycle of transition  Over the course of the next decade, academic libraries will replace their current legacy products with new platforms  Not just a change of technology but a substantial change in the ways that libraries manage their resources and deliver their services

 Traditional Proprietary Commercial ILS  Aleph, Voyager, Millennium, Symphony, Polaris,  BOOK-IT, DDELibra, Libra.se  LIBERO, Amlib, Spydus, TOTALS II, Talis Alto, OpenGalaxy  Traditional Open Source ILS  Evergreen, Koha  New generation Library Services Platforms  Ex Libris Alma  Kuali OLE (Enterprise, not cloud)  OCLC WorldShare Management Services,  Serials Solutions Intota  Innovative Interfaces Sierra (evolving) Competing Models of Library Automation

Convergence  Discovery and Management solutions will increasingly be implemented as matched sets  Ex Libris: Primo / Alma  Serials Solutions: Summon / Intota  OCLC: WorldCat Local / WorldShare Platform  Except: Kuali OLE, EBSCO Discovery Service  Both depend on an ecosystem of interrelated knowledge bases  API’s exposed to mix and match, but efficiencies and synergies are lost

How do libraries make the transition? Migrating to the Cloud

Infrastructure  Move existing applications to cloud hosting?  Infrastructure as a service  Marginal gains  Create platforms designed for cloud deployment  Multi-tenant software as a service

Transition of services  Identify specific library services as candidates  What activities are performed by individual libraries that could be done more effectively collaboratively

Candidate services  Bibliographic support  Reference / Research support  Resource sharing  E-resource management  Resource Discovery  Library Management

Organizational strategy  Individual institutions make gains by moving legacy applications to hosted services  Amplify impact as new collaborative services are built that span organizations

Partnership opportunities  When to partner with existing service providers?  When to create services for a specific country or sector?

More than a technical transition  Transforming infrastructure  Transform resources  Working toward shared infrastructure  Identify areas where libraries can collaborate to share resources  Infrastructure transformation  Bandwidth  Shared services  Refocus development from stand-alone applications to platforms  Platform development  APIs that allow individual libraries or campuses to consume content or services according to local needs

New conceptual models  Think beyond moving existing functionality  Re-evaluate the way that technical and information infrastructure supports the library in its strategic services to its parent institution

Candidates for Cloud-based Services  Identify services that can be provided at the national or international level  Resource sharing  Document delivery  Interlibrary Loan  E-resource knowledgebase  Index-based discovery

Infrastructure  Robust Interconnectivity  Development and support capacity  Distributed data centers

Organization and personnel issues  Refocus efforts of technologists and technicians  Away from redundant local implementations  Toward collaborative broad-based cross-institutional services  Deployment and maintenance of conventional systems consumes all available resources  Library-by-library model least efficient

From software development to Platform development  Multi-tenant software as a service platforms that scale to meet the needs of the largest organizations or clusters of organizations  Consume platform services when available and appropriate  Create strategic platforms

Progressive consolidation of library services  Centralization of technical infrastructure of multiple libraries within a campus  Resource sharing support  Direct borrowing among partner institutions  Shared infrastructure between institutions  Examples: 2CUL (Columbia University / Cornell University)  Orbis Cascade Alliance (37 independent colleges and universities to merge into shared LSP)

Consolidation of library automation services  Centralized library services within institutions  Strategically cooperate between institutions  From software development to platform development  Refocus efforts of technology personnel  Less attention to deployment of conventional systems  More attention on broad-based services  Library-by-library automation model least efficient

Open source and Open Access  Open source development of platform services  Open source infrastructure components  Open APIs to expose platform services  Knowledge base components  Open access  Community maintained  Adequately resourced

Reassess expectations of Technology  Many previous assumptions no longer apply  Technology platforms scale infinitely  No technical limits on how libraries share technical infrastructure  Cloud technologies enable new ways of sharing metadata  Build flexible systems not hardwired to any given set of workflows

Reassess workflow and organizational options  ILS model shaped library organizations  New Library Services Platforms may enable new ways to organize how resource management and service delivery are performed  New technologies more able to support strategic priorities and initiatives

Time to engage  Transition to new technology models just underway  More transformative development than in previous phases of library automation  Opportunities to partner and collaborate  Vendors want to create systems with long-term value  Question previously held assumptions regarding the shape of technology infrastructure and services  Provide leadership in defining expectations

Questions and discussion