Collection & Service Description and the NISO Metasearch Initiative Juha Hakala, Director (IT), Helsinki University Library Chair, NISO Metasearch Initiative Task Group 2 Pete Johnston, UKOLN, University of Bath Member, NISO Metasearch Initiative Task Group 2 Special Session, DC-2004, Shanghai, China, Wednesday 13 October

Collection & Service Description for the NISO Metasearch Initiative The Metasearch problem The NISO Metasearch Initiative Collections and Services Collection Description & Service Description

The Metasearch problem

The problem Content providers make their collections available through their own separate “presentation services” User wants to access/use items from multiple content providers User has to discover, access and interact with multiple presentation services But –Each service has different user interface for discovery –Results human-readable (HTML), but difficult to merge, reuse, manipulate –Different authentication/access requirements

The solution (ideally…) The provision of "Metasearch" services that –enable user to search across the metadata databases of multiple content providers from a single interface –manage multiple result sets and present to user –manage authentication/access –(etc!) Technologies exist e.g. –(Real-time) cross-searching (Z39.50, SRW/U, service-specific APIs) –Harvesting (OAI-PMH) Seamless (to the user) discovery of and access to heterogeneous, distributed resources! However…..

The problems with Metasearch today User requires/expects resources from increasing range of content providers Many content providers have not implemented standards- based search interfaces –Many proprietary APIs –Some "screen scraping" (parsing of HTML) Metasearch services do work, but –fragile, susceptible to changes by content provider –labour-intensive, scalability issues –duplication of effort Also content provider concerns about –efficiency/effectiveness of search –access management, logging etc –branding/IPR/presentation of results

What is needed For effective Metasearch services, content providers and service providers need agreement on (at least…) –Transport protocol(s) –Query language(s) syntax and semantics –Metadata schemas syntax and semantics –Intellectual property rights issues how metadata records and resources are presented, used –Authorisation / authentication –Disclosure / discovery of collections and services

The NISO Metasearch Initiative

The NISO Metasearch Initiative Response to content provider/service provider concerns Bring together –Content providers –System vendors –Library service providers –Standards developers "To identify, develop, and frame the standards and other common understandings that are needed to enable an efficient and robust information environment"

The NISO Metasearch Initiative Aims to enable –metasearch service providers to offer more effective and responsive services –content providers to deliver enhanced content and protect their intellectual property –libraries to deliver services that distinguish their offerings from other free web services

The NISO Metasearch Initiative Standardisation of metasearch applications (portals) must be accomplished –Traditional integrated library systems are fairly well standardised ISO 2709 Exchange format (since early 1970s) MARC formats & AACR2 cataloguing rules Z39.50 Information retrieval protocol ISO ILL, NCIP –For metasearch applications, many relevant standards will be developed in the NISO MI This will e.g. enable libraries and other users to exchange metadata between them

The NISO Metasearch Initiative: Current Activity Task Group 1: Access Management –Gather requirements for access/authentication –Describe existing processes –Develop use cases Task Group 2: Collection Description –Establish metasearch services' requirements for description of Collections Services which provide access to Collections ("Informational Services") –Select/develop metadata schemas –Recommend syntax for representation & data exchange Task Group 3: Search & Retrieval –Describe existing practice –Metadata to describe result sets –Metadata to describe article-level citations

Collections and Services

Collections and Services Item –A physical or digital entity Collection –An aggregation of one or more items Service –The provision of, or system of supplying, one or more functions of interest to an end-user or software application. –Physical or digital –Digital services may be "structured" or "unstructured" Informational services –Services that provide access to, or metadata about, items and/or collections –JISC Information Environment Architecture: Glossary

OAI repository Harvest via OAI-PMH Z39.50 target Search/retrieve via Z39.50 Collection of digital metadata records Collection of digital or physical items Informational services unstructured network service structured network service RSS channel Alert via RSS/HTTP Web site "Screen- scrape"

Harvest Search Alert "Screen- scrape" Web site OAI repository Z39.50 target RSS channel

Functional Model: “Surveying the landscape” Agent –"Enters" information landscape Views a default set of collections, based on information about the agent –"Surveys" landscape Modifies landscape by adding/removing collections, based on information about the collections –"Discovers" items of interest within collections "Drills down" into selected collections N.B. Agent may be –Human researcher –Human administrator of presentation service –Software application acting on behalf of human researcher ie/arch/functional-model/

My default landscape Coll BColl CColl DColl E Surveying the information landscape

My default landscape Coll BColl CColl DColl E My default landscape Coll BColl CColl DColl E My default landscape Coll AColl BColl CColl D modified

Functional Requirements Allow an agent to –Discover collections of potential interest –Identify a collection –Select one or more collections from amongst a number of discovered collections –Identify the informational services that provide access to the collection –Select a service with which to interact –Interact with service Subject to "knowledge" of interface semantics Collection description Service description

Relations between Collections and Services Relationships exist –Between collections and services –Between collections In NISO MI conceptual model –A collection is-made-available-by zero or more services –A service makes-available exactly one collection –A collection is-part-of zero or more (super-) collections (parent) –A collection has-parts zero or more (sub-) collections (child)

Collection is-Part-Of Collection is-Part-Of Service is-Made- Available-By Service is-Made- Available-By Service is-Made- Available-By Collection is-Part-Of Collection is-Part-Of Service is-Made- Available-By

Collection Description & Service Description

Collection Description & Service Description NISO MI TG2 specifying metadata for collections & services –Data model –Metadata semantics –Syntax(es) for representation and data exchange –Guidelines for use Should build on/reuse existing work where possible Make recommendations for future work N.B. TG2 is not –building a service; or –specifying the architecture within which a service might operate –specifying the protocols for the exchange of collection/service metadata

Collection Description

Collection Description Collection as “an aggregation of one or more items” –"functional granularity" Collection-level description –Description of the collection as a whole –Unitary finding-aid Considerable recent work on collection-level description Research Support Libraries Programme (UK, ) –support for academic research –improve disclosure/discovery of library/archive collections –also collaborative collection management –recognition of CLD as important mechanism for disclosure/discovery

RSLP Collection Description Project, Funded by RSLP, OCLC RSLP CD Model –Entity-Relation model (Michael Heaney, University of Oxford) –Implementation independent –Intended to be applicable to wide range of collections –Informed by IFLA FRBR approach as well as existing descriptive standards RSLP CD Schema –DC-based metadata schema (Andy Powell, UKOLN) –Expresses subset of RSLP model –Simplification of model Significant influence on other initiatives But concerns over status, ownership, visibility, persistence, maintenance, etc

RSLP CD Schema v Model ContentCreator creates Collector Owner collects owns Administrator administers ItemProducer produces is-embodied-in Collection is-gathered-into Location is-located-in

DC Collection Description Working Group Active 2001 (really 2003!) - Provide forum for sharing information about CLD activity Develop a DC Application Profile for collection- level description –Specification of how DC (and other) properties are used for describing collections Develop supporting materials for use of AP Informed by experience of RSLP CD implementers and other CLD initiatives –RSLP projects, TEL, JISC IESR, IMLS, others

DC Collection Description Application Profile (DC CD AP) A "core" set of collection description properties –For simple collection-level descriptions –Suitable for a broad range of collections –Primarily to support discovery of collections Examine collection attributes (only) of RSLP CD Schema as starting point DC CD AP building on Heaney E-R model –introduces Service as entity-type –describes Collection-Location, Collection-Service, Collection-Agent relationships –but excludes Location, Service, Agent description

Item Collection is-gathered-into m n Location is-located-in m n Service is-Made-Available-By m 1 m provides n administers n m Agent collects m n owns m n Collection Description is-described-by 1 m

DC Collection Description Application Profile (DC CD AP) Draft covers –Identification of collection –Content of items in collection –Form of items in collection –Process by which items gathered into collection –Ownership of collection –Rights of access to/use of collection –Location of collection –Services that provide access to collection –Relationships between collections Instances can be represented using DC guidelines (RDF/XML, DC-in-XML)

NISO MI TG2 & DC CD AP DC CD AP still work in progress –Some issues data model (Location/Service) one-to-one rule –Some terms still to be assigned URIrefs Scope and specificity of DC CD AP –NISO MI addressing (primarily) library service providers –Some library-specific requirements e.g. completeness of collection –NISO MI may require superset of DC CD AP –May require non-DCMI naming authority for some metadata terms

Service Description

Service Description Led by Larry Dixson, Library of Congress (Informational) Service –Means of accessing collection Service description must provide –Indication of protocol used –Access point for service –Authentication/authorisation information –Operations/queries supported N.B. does not describe the "syntax" of service –Assumption that protocols described elsewhere NISO MI evaluating use of Zeerex

Zeerex background Z39.50-based specification Based on earlier work, including Z39.50 Explain Service and Explain Lite, developed in the ONE2 project Relatively easy to implement, yet allows detailed description of services (e.g. Z39.50 servers) Sufficiently expressive/flexible to describe similar types of service –Services that provide access to a “database”

Zeerex ServerInfo –protocol –host/IP –port –database/service –authentication* DatabaseInfo –access restrictions IndexInfo –search –scan (browse) –sort RecordInfo –record syntax –element set name

Zeerex and access protocols Access protocols “in scope” for NISO MI –Z39.50 –SRW/SRU –OAI-PMH –HTTP –LDAP/X.500? –GRID metasearch? –GIS search facility? Can Zeerex support description of services using all of these protocols? Tests currently in progress

Summary NISO MI bringing together different stakeholders to develop shared approaches to common problems Disclosure/discovery of collections and informational services critical to effective metasearch services

Acknowledgements UKOLN is funded by the UK Museums, Libraries and Archives Council (MLA), the Joint Information Systems Committee (JISC) of the UK higher and further education funding councils, as well as by project funding from the JISC and the European Union. UKOLN also receives support from the University of Bath where it is based.

Collection & Service Description and the NISO Metasearch Initiative Juha Hakala, Director (IT), Helsinki University Library Chair, NISO Metasearch Initiative Task Group 2 Pete Johnston, UKOLN, University of Bath Member, NISO Metasearch Initiative Task Group 2 Special Session, DC-2004, Shanghai, China, Wednesday 13 October