Amy Benson NELINET, Inc. November 7, 2005 Metadata 101 Amy Benson NELINET, Inc. November 7, 2005

Standards Increase interoperability Lower use and participation barriers Build larger communities of users which can drive creation of a wider range of relevant services and tools (Windows vs Mac) Improve chances of long term survival of materials Prefer open over proprietary

Categories Metadata containers Metadata standards XML, RDF Metadata standards MARC, MODS, DC, EAD, TEI, ONIX, FGDC, GILS Metadata content standards Transmission standards and protocols METS, OAI, SOAP, Z39.50, SRW Identifiers URI, URL, PURL, URN, DOI, ISTC

Metadata - What is it? Data about data Information about any aspect of a resource - size, location, attributes, topic, origin, use, audience, creator, quality, access rights, reviews… the list is endless An aid to the discovery, identification, assessment, and management of described entities

Types of Metadata Descriptive Discovery Structural Administrative What is it? Discovery How can I find it? Structural What files comprise it? Administrative When was it created?

Types of Metadata Identifiers Terms & conditions Preservation How can I get to it? Terms & conditions Can I use it? Preservation Which key characteristics of the resource need to be maintained?

MARC Advantages Disadvantages Rich set of descriptive elements Highly interoperable within library community Long, established history Disadvantages Low extensibility As is, not interoperable beyond the library world Weak on administrative, rights, and other kinds of metadata important for digital resources

MARC Future of MARC MARC XML from the Library of Congress (LC) Must MARC die? No. New life through XML MARC XML from the Library of Congress (LC) MODS: a version of MARC encoded in XML, developed by the Library of Congress Crosswalks between MARC and many other metadata schemas already exist

MARC XML LC has developed a MARC XML schema, stylesheets, and tools The schema allows representation of a complete MARC record in XML Lossless conversion Will support new transformations to new uses of MARC data MARC to MARCXML to Dublin Core and MODS

Metadata Object Description Schema (MODS) Set of 20 bibliographic elements - a subset of the MARC 21 Format for Bibliographic Data Not as complete as the full MARC format, but richer than Dublin Core (for example) Highly interoperable with existing MARC records Uses language-based tags, rather than numbers like MARC 21 (245, 650, etc.) Under development by the LC Network Development and MARC Standards Office

MODS XML-based Intended to work with/complement other metadata formats Can be used for conversion of existing MARC records or to create new resource description records Useful particularly for library applications that want to go beyond the OPAC Shares features of MARC and Dublin Core

MODS Elements TitleInfo Name TypeOfResource Genre PublicationInfo Language PhysicalDescription Abstract TableOfContents TargetAudience Note Cartographics Subject Classification RelatedItem Identifier Location AccessCondition Extension RecordInfo

MODS Elements Title element is mandatory, all others are optional Elements can have subelements and attributes which provide refining detail for the element Elements and sub-elements are repeatable, except in certain cases Elements display in any order

MODS Example

MODS Implementation MODS User Guidelines MODS Implementation Registry http://www.loc.gov/standards/mods/registry.html MODS Implementation Registry Contains descriptions of MODS projects planned, in progress, and fully implemented

Dublin Core (DC) A method of describing resources intended to facilitate the discovery of electronic resources Designed to allow simple description of resources by non-catalogers as well as specialists National and International standard ANSI/NISO standard Z39.85-2001 ISO standard 15836 Includes 15 “core” elements

Dublin Core Elements Title Creator Subject Description Publisher Contributor Date Type Format Identifier Source Language Relation Coverage Rights

Dublin Core All elements optional and repeatable Elements display in any order Authority control not required Simple and Qualified DC Extensible Flexible International

Dublin Core Simple Qualified Lowest common denominator Less rich Discovery role – leads to resource or more complete description of resource Qualified More precise Less interoperable

Dublin Core Examples Generic Title=“The sound of music” HTML <meta name = "DC.Title" content = “The sound of music”> XML <?xml version="1.0"?> <metadata xmlns:dc="http://purl.org/dc/elements/1.1/"> <dc:title> The Sound of Music</dc:title> </metadata>

Dublin Core Examples - HTML Yale example.

Dublin Core Examples - XML Yale example.

Other Metadata Standards Encoded Archival Description (EAD) Text Encoding Initiative (TEI) Visual Resources Association (VRA) Global Information Locator Service (GILS) Online Information Exchange (ONIX) Content Standards for Digital Geospatial Metadata (CSDGM) aka FGDC Document Data Initiative (DDI)

Crosswalks Crosswalks map an element from one scheme to its closest equivalent in another scheme Example: MARC 1XX field is mapped to DC ‘creator’ Instrumental for converting data in one format to another format - one that is potentially more widely accessible Support the demand for cross-domain searching and interoperability

Crosswalks There is rarely a one-to-one correlation between elements of different schemes One to many - DC to MARC Many to one or none - MARC to DC None to one or many MARC to DC http://www.loc.gov/marc/marc2dc.html#unqualif

Content Standards AACR (Anglo-American Cataloguing Rules) “The rules cover the description of, and the provision of access points for, all library materials commonly collected at the present time.” The current text is the 2nd ed, 2002 Revision (with 2003, 2004, and 2005 updates) The Joint Steering Committee for Revision of AACR (JSC) is working on a new code, “RDA: Resource Description and Access” scheduled to be published in 2008

Content Standards International Standard Bibliographic Description (ISBD) A family of standards to regularize the form and content of bibliographic descriptions Available for different material types: monographs, computer files, etc. Designed to promote record sharing and exchange

Content Standards Describing Archives: A Content Standard (DACS) Designed to facilitate consistent, appropriate, and self-explanatory description of archival materials and creators of archival materials Replaces Archives, Personal Papers, and Manuscripts (APPM)

Metadata Encoding & Transmission Standard (METS) A system for packaging metadata necessary for both the management of digital library objects within a repository and the exchange of such objects between repositories, or between repositories and their users Used for: Digital collection repositories Developed by the Digital Library Federation (DLF) and Library of Congress (LC)

Metadata Encoding & Transmission Standard (METS) METS can be understood as a binder that unites metadata about a particular resource A METS record includes six parts: Header Descriptive metadata Administrative metadata File groups Structural map Behavior section

From Patrick Yott.

METS Schema From Patrick Yott.

Open Archives Initiative (OAI) A tool that supports interoperability among multiple databases OAI goal: coarse-granularity resource discovery OAI handles simple discovery from multiple community-specific repositories with metadata crosswalked to unqualified Dublin Core

OAI Roots are in the science community interested in locating and searching multiple repositories of pre- and e-prints of scientific papers Not really an archive, the way we traditionally think of the word

OAI Data providers expose (make available) the metadata for their collections Service providers harvest the exposed metadata and aggregate it (so that one search does it all) and/or provide additional services related to the harvested metadata, such as providing easy access to recent additions, updated materials, pre-set searches, etc.

OAI OAI Protocol for Metadata Harvesting Metadata content must be encoded in XML and have a corresponding XML schema for validation Metadata must be supplied in unqualified Dublin Core format, at least Other metadata formats are optional Metadata may optionally include a link to the actual content / resource

OAI Infrastructure Harvester Service Provider repository repository DC DC Harvester From Patrick Yott. repository DC repository DC

OAI Infrastructure user search Repository From Patrick Yott.

OAI Infrastructure Repository user repository search From Patrick Yott. repository

Z39.50 Z39.50 is a search and retrieval protocol, maintained by LC, capable of operating over TCP/IP Negotiates queries with multiple, separate databases – does not harvest + create new db Built in to some library software systems OAI not intended to replace other approaches, but to provide an easy-to-use alternative for different constituencies and purposes

Search/Retrieve Web Service The primary function of SRW is to allow a user to search remote databases of records Protocol uses easily available technologies -- XML, SOAP, HTTP, URI -- to perform tasks traditionally done using proprietary solutions such as database queries and responses Builds on Z39.50 and moves it forward ZING: Z39.50 International: Next Generation

Functional Requirements for Bibliographic Records (FRBR) A study by IFLA (International Federation of Library Associations) of the full range of functions performed by the bibliographic record What do we use bibliographic records for? Description, access, location, identification, annotations ... The report provides a framework for the nature of and uses for bibliographic records A conceptual model that can be used as a means to meet user needs and expectations

Functional Requirements for Bibliographic Records (FRBR) Tasks we use bibliographic records for: Finding Identifying Selecting Obtaining access to resources FRBR should allow systems to handle bibliographic data in new, useful ways that fulfill these tasks

Functional Requirements for Bibliographic Records (FRBR) Conceptual model of relationships between bibliographic entities Hierarchical relationships Work The intellectual product Expression An ‘expression’ of the parent work such as a translation, edition, revisions, annotated text, etc. – Expressions entail additional intellectual effort

Functional Requirements for Bibliographic Records (FRBR) Hierarchical relationships Manifestation Published runs of each expression in multiple formats over time The level at which we traditionally create a catalog record Item Each copy of a specific manifestation Circulation records track items

Functional Requirements for Bibliographic Records (FRBR) OCLC is researching the application of FRBR to WorldCat “FRBRization” They have created an algorithm that groups records automatically based on the Work/Expression/Manifestation/Item model http://www.oclc.org/research/projects/frbr/algorithm.htm

Identifiers Four potential purposes Locator Identifier Gatherers Where is the document I seek? Identifier Unique label for a resource Gatherers Groups like resources similar to a uniform title Differentiator Helps identify different versions of same resource

Identifiers Uniform Resource Identifiers (URI) OpenURL DOI ISTC Generic set of all names/addresses that refer to resources on the Web including: Uniform Resource Locator (URL) Persistent Uniform Resource Locator (PURL) Uniform Resource Name (URN) OpenURL DOI ISTC

Uniform Resource Locator (URL) Web address or location at which a resource is held, not an identifier for the resource itself Most common way to locate documents / items on the Web (http, ftp, mailto, etc.) Not particularly stable or permanent Error 404: File not Found No metadata, but important starting point as we look at some of the related technologies

Persistent Uniform Resource Locator (PURL) PURL Service is managed by OCLC Functionally, a PURL is a URL The PURL remains constant even if the URL changes - its function is to automatically re-direct a user to the current URL PURL system/resolver is updated by resource manager to reflect any changes to location of the file, or URL

PURLs PURLs can be used both in documents and in cataloging systems PURLs increase the probability of correct resolution and long-term access to resources Use of PURLs can reduce the burden and expense of catalog maintenance (and business card printing)

PURL - Example US Government is a big user of PURLs http://www.ccny.cuny.edu/library/Divisions/Government/iraqbib.html

Uniform Resource Name (URN) Uniform Resource Names (URNs) are intended to serve as persistent, location-independent resource identifiers Globally unique Never change Format urn:<namespace identifier>:<namespace specific string> Use a resolver system to indicate current location of resource

Digital Object Identifier (DOI) Overseen by the International DOI Foundation DOIs are persistent, location-independent identifiers of resources Developed to enable management of copyrightable materials in an electronic environment (locate, buy, sell, track, license) Specific type / implementation of a URN

DOI A two-part number with a prefix identifying the original publisher and a suffix identifying the specific work Similar to the ISBN A DOI resolution request for a specific resource would return one or more URLs - *locations* where a user could obtain access to the resource Appropriate copy: online, text, free, illustrated, etc.

DOI Applications of the DOI will require metadata The basis of the DOI metadata scheme is a minimal "kernel" of elements DOI minimal kernel elements of metadata: DOI, DOI genre, identifier, title, type, origination, primary agent, agent role, and administrative data such as registrant, and date of registration

NELINET Digital Services Questions? Amy Benson Program Director NELINET Digital Services NELINET, Inc. benson@nelinet.net 508.597.1937 800.635.4638 x1937