Free the Data: creating a web services interface to the online catalog Emily Lynema NC State University Libraries Code4lib 2007 February 28, 2007

#code4lib: 2007 Context Endeca ‘Information Access Platform’ Enterprise search and faceted navigation Home Depot, Lowe’s, Circuit City, Dice [etc.] FCLA, McMaster

#code4lib: 2007 Features Stopwords and automatic stemming (nouns) Automatic spell correction & did you mean suggestions Customizable relevance ranking algorithms Faceted navigation and true browse Improved response time Persistent URLs (no sessions!)

#code4lib: 2007 Architecture Raw MARC data NCSU exports and reformats Flat text files Data Foundry Parse text files Indices MDEX Engine NCSU Web Application HTTP Information Access Platform

#code4lib: 2007 The very beginning OCLC Research Software Contest The idea of an availability web service that could report on holdings to other sites Functionality Submit ISBN XML response returns availability and location If not owned or no copies available, looks for similar ISBN via xISBN service.

#code4lib: 2007 Catalog Availability More details: Try it out:

#code4lib: 2007 Introducing CatalogWS Rest web API for dynamically querying information from the NCSU Libraries Catalog Have fun!

#code4lib: 2007 Motivations Initial impetus – 2 requests Can we have RSS feeds for the catalog? Can we integrate catalog results into library website QuickSearch? Where did we end up? Generic XML layer on top of catalog searching Capability for server-side user-defined XSL transformations

#code4lib: 2007 Why go there? More open access to the data available in our library catalog Core XML schema can be re-used and modified via stylesheets Enable other developers in the library to build applications using catalog data Reduce bottleneck

#code4lib: 2007 Using the service Base: Parameters: service (required) availability | search query (required) Any term(s) output (opt) Default: xml | rss | opensearch | json

#code4lib: 2007 Additional functionality count default: 30 max: 50 offset default: 0 sort default: relevance | date_desc | date_asc | call_number | most_popular style URL of XSL to transform to custom output

#code4lib: 2007 Technical overview Separate web application handles web service requests Java and Tomcat XOM for XML creation and XSL transformation Saxon 8.8 for XSLT 2.0 functionality org.json Java package for easy XML => JSON

#code4lib: 2007 XML response Defined with Relax NG SchemaRelax NG Schema Data from search results page Search information Results Facets

#code4lib: 2007 RSS

#code4lib: 2007 OpenSearch

#code4lib: 2007 QuickSearch

#code4lib: 2007 Mobile device searching

#code4lib: 2007 I promised I would talk about… Experimenting with facet data in OpenSearch Early plan: 2 OpenSearch requests for QuickSearch integration: 1 for results, 1 for facets Why request twice when you could do it once? But what if OpenSearch could do both… Existing query role=subset Extended OpenSearch parameters to create a facet parameter for use in the OpenSearch URL template. <opensearch:Query xmlns:custom=“ role=“subset” searchTerms=“deforestation” custom:facet=“ ” />

#code4lib: 2007 Questions? NCSU Endeca project site (w/slides): CatalogWS project site: Emily Lynema Systems Librarian for Digital Projects