Metadata Interaction, Integration, and Interoperability NISO Workshop: Metadata Practices on the Cutting Edge, May 20, 2004, Washington, DC William E. Moen School of Library and Information Sciences Texas Center for Digital Knowledge University of North Texas Denton, TX 72603

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 2 Overview The problem space Interoperability and communities Mechanisms to address interoperability

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 3 Is there a problem? Many metadata schemes and element sets Well known & documented Less known and little public documentation Similar/same content described by different metadata schemes and vocabularies No canonical metadata record for an object Varied syntaxes for encoding metadata No canonical syntax A vital and diverse metadata ecology! No problem, unless….

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 4 Metadata in the networked environment Interaction between systems that use metadata Harvesting Searching Integrating different types of metadata for local information management Technical metadata for digital asset mgmt Reusing metadata in local applications ONIX metadata in library systems Interoperability?

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 5 Importance of interoperability Systems and organizations will interoperate One should actively be engaged in the ongoing process of ensuring that the systems, procedures and culture of an organisation are managed in such a way as to maximise opportunities for exchange and re-use of information, whether internally or externally. Paul Miller, 2000 Metadata interoperability has to be the underlying principle of networked information management. Marcia Lei Zeng, 2001

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 6 Interoperability System-oriented definition: The ability of two or more systems or components to exchange information and use the exchanged information without special effort on either system User-oriented definition: The condition achieved when two or more technical systems can exchange information directly in a way that is satisfactory to users of the systems (AAP)

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 7 Interoperability factors In the context of networked information retrieval Multiple and disparate systems (operating systems, information retrieval systems, etc.) Multiple protocols Multiple formats of data Multiple metadata schemes Multiple vocabularies, ontologies, disciplines Multiple languages Multiple character sets

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 8 Preliminary framework for interoperability In the context of networked information retrieval Within and across communities Information communities/Communities of practice Focal community Extended community Extra community Costs to achieve interoperability vary

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 9 Interop Among and Across Communities Focal Community (e.g., Libraries) Focal Community (e.g., Archives) Focal Community (e.g., Museum) Extended Community (e.g., Cultural Heritage) Focal Community (e.g., Geospatial ) Focal Community (e.g., Geospatial) Focal Community (e.g., Natural History Museums) Extended Community Extra Community

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 10 Communities Communities of practice (Wenger) Network of professionals work on common problems speak a common language share similar values produce shared meanings Information communities Looser affiliation of people creators information managers users Membership in multiple information communities

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 11 Rust’s people & stuff (& agreements) model PeopleStuff Create Manage Use People creating stuff for specific information community; stuff used by multiple communities People managing stuff within context of community of practice Different communities of practice interested in same stuff

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 12 Interoperability cost vs. functionality Adoption of common standard low cost with low functionality higher functionality but with a greater cost of adoption No best point on the curve – every point is optimal for some purpose Functionality Cost of acceptance Many adopters Few adopters Arms, et al., 2002

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 13 So we have … Many metadata schemes and element sets Similar/same content described by different metadata schemes and vocabularies Varied syntaxes for encoding metadata Which reflect: Community practices, needs, meaning Cost barriers to adopting common standards Lack of knowledge of available standards Not invented here syndrome

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 14 Mechanisms for addressing interoperability Crosswalks and mapping Application profiles Registries Resource Description Framework (RDF)

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 15 Mapping and crosswalks Mapping: Intellectual activity that identifies semantically equivalent elements in different metadata schemes Crosswalk: Documentation resulting from mapping showing the equivalencies and conversion specifications 1998 NISO White Paper on Crosswalks Unfortunately, the specification of a crosswalk is a difficult and error-prone task requiring in-depth knowledge and specialized expertise in the associated metadata standards St. Pierre & LaPlant, 1998

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 16 Mapping issues Semantic, structural, and data conversion One-way or reversible mappings? Mapping between any two elements: One-to-one One-to-many (repeatable elements; unique more narrowly defined elements) Many-to-one (complete mapping; incomplete mapping) One-to-zero (no semantically equivalent element) Data conversion From less inclusive to more inclusive format From uncontrolled to controlled vocabulary Correct and efficient mapping of metadata elements among various formats is the essential condition for ensuring metadata interoperability Zeng & Xiao, 2001

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 17 Mapping to an interoperable core OCLC Office of Research’s Metadata Switch Project Experimental modular services that add value to metadata Metadata Schema Transformation Web Service (Godby, et al., 2003) An interoperable core Translations between metadata standards via mapping to and from the core Reducing the number of separate mappings between metadata standards Design of the interoperable core is an open issue

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 18 Application profiles Reuse of elements from different sets, but cannot define new elements Specify permitted schemes (e.g., date/time formats, controlled vocabulary) for data values Can refine standard definitions Application profiles consist of data elements drawn from one or more namespace schemas combined together by implementors and optimised for a particular local application. Heery & Patel, 2000 By defining application profiles and, most importantly by declaring them, implementers can start to share information about their schemas in order to inter-work with wider groupings. Heery & Patel, 2000

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 19 Registries Metadata registry: An index of metadata terms, official definitions, local variations extensions Can enable the reuse of existing elements rather than users/communities reinventing their own UK Schemas Project: Includes registry of several metadata element sets EU Cores Project: Includes registry of core vocabularies and profiles; a schema creation tool and Web interface to register schemas Dublin Core Metadata Registry: Authoritative source for DC; Designed to promote the discovery and reuse of exiting metadata definitions; The term "registry" covers a broad range of databases, documentation services, or Web- based portals providing access to schemas. Baker, et al., 2001 Almost universally, registries are seen as our best hope in the medium term for a scalable solution to the problem of mapping and translating between a diversity of schemas. Baker, et al., 2001

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 20 Resource Description Framework (RDF) Provides a basic grammar for representing metadata terms, their semantics, relationships, etc. Use of Uniform Resource Identifiers (URIs) to identify namespace schemas where terms are declared and defined RDF Schemas and XML Schemas, see: Heery & Johnston, 2003 Hunter and Lagoze, 2001 Baker, et al., 2001 SchemaWeb: gathers information about schemas published on the web

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 21 Metadata interoperability? In this context, interoperability means Effective reuse and integration of existing metadata from various sources Ability to discover and act on authoritative specifications of metadata vocabularies Support for interaction with one or more metadata sets created for other communities The diverse metadata ecology presents challenges but reflects real-world needs of diverse information communities

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 22 References Application Profiles: Mixing and Matching Metadata Schemas. Heery & Patel  Combining RDF and XML Schemas to Enhance Interoperability Between Metadata Application Profiles. Hunter & Lagoze  CORES Project: A Forum on Share Metadata Vocabularies  The Dublin Core Metadata Registry  Issues in Crosswalking Content Metadata Standards. St. Pierre & LaPlant  Mapping Metadata Elements of Different Formats. Zeng, M. L. & Xiao, L A Metadata Registry for the Semantic Web. Heery & Wagner 

Moen NISO Workshop: Metadata Practices on the Cutting Edge -- May 20, Washington, DC 23 References Metadata Schema Registries in the Partially Semantic Web: The CORES Experience. Heery & Johnston  Metadata Switch Project. OCLC  SCHEMAS Project: Forum for Metadata Schema Implementers  SchemaWeb.  A Spectrum of Interoperability. Arms, et al  Two Paths to Interoperable Metadata. Godby, et al  What Terms Does Your Metadata Use? Application Profiles as Machine-Understandable Narratives. Baker, et al 