Lifecycle Metadata for Digital Objects With an emphasis on preservation…

What is metadata? Data about data Database usage Web usage (metatags) Functions? Kinds?

First-order metadata Written language Layout conventions Separation of words Arrangement of groups of words Punctuation, capitalization, etc. Note that this is usually considered to belong to an external standard, and nobody worries about it!

Second-order metadata Encoding (ASCII, proprietary formatting schemes) Compression schemes Encryption or other intentional distortion schemes Note that when these are referred to as external standards, everyone worries about it.

Third-order metadata “Connections to the world” Meaning Semantics Pragmatics

Fourth-order metadata Functions What can you do with the digital object? What is its purpose? How does it work? Functionality significant for preservation Explicit digital object types

Fifth-order metadata Groups of digital object types Archival series Project files “Complex documents”

Classic objects of preservation in archives  Content  Context  Structure

Functional types of metadata  Administrative  Descriptive  Preservation  Technical  Use

Life cycle view of metadata  Appraisal/Inventory/Scheduling  Creation and versioning  Transfer/Authenticity  Descriptive  Use  Rights management  Preservation and disposition

Attributes of metadata items  Source of metadata (internal or external)  Method of metadata creation (auto or manual)  Nature of metadata (lay or expert)  Status (static or dynamic)  Structure (structured or unstructured)  Semantics (controlled or uncontrolled)  Level (item or collection)

Major Archival Metadata Schemes

University of Pittsburgh metadata reference model in six layers  Handle  Terms & Conditions  Structural  Contextual  Content  Use History

Structural Layer specifies technical details File identification metadata File encoding metadata File rendering metadata Record rendering metadata Content structure metadata Source metadata

InterPARES Project Authenticity template Documentary form Extrinsic elements Intrinsic elements Annotations Medium Context

Dublin Core Metadata Initiative Supported by OCLC Primarily a surrogate/discovery metadata scheme Does not aim to document everything Useful for management of active digital objects

Dublin Core elements  Title  Creator  Subject  Description  Publisher  Contributor  Date  Type  Format  Identifier  Source  Language  Relation  Coverage  Rights

Dublin Core development Initial development of simple elements Subelements and user communities Warwick Framework RDF and XML

Metadata Encoding and Transmission Standard (METS)  Developed out of LoC’s MOA project  Designed to support maintenance of library of digital objects  Three overall types of metadata Descriptive Administrative Structural

METS Descriptive metadata  External (e.g., finding aid)  Internal (part of the document)

METS Administrative metadata  Technical metadata  Intellectual property rights metadata  Source metadata (re analog source)  Digital provenance metadata Relations between files Migration/transformation data

METS Structural metadata  File groups list  Structural map (defines relations between files and METS element structure)  Behavior segment (associates executable methods with specific content elements, e.g. for display)

METS and XML The METS XML schema sd/mets.html sd/mets.html Why is it all so complicated? How can anyone ever keep track of all this metadata?

XML in 10 Points 1. XML is for structuring 2. XML looks like HTML 3. XML is text for computers 4. XML is purposely verbose 5. XML is a family 6. XML is only partly new 7. XHTML->XML 8. XML is modular 9. XML is base for RDF, Semantic Web 10. XML is free, universal, supported

Appraisal / Inventory / Retention Schedule Metadata

Digital Appraisal Decisions Keep (costs of carrying into the future) Allow to Die (keep but do nothing) Repurpose (separating content and form) Destroy (microwave the disk?)

Digital Appraisal: What to Appraise Content (as with paper?) Technical support System Creating application Display requirements Functionality

What is a Retention Schedule? Classic record statuses: active, semiactive, inactive Keep Alter function of custodian Alter custodianship Allow to Die Leave with creator? Why not always do this? Destroy Determine when to destroy Almost always a method for reprieve

Record-level vs Group-level Metadata Record-level: Metadata orders written (content) 2 encoded (content) 3 meaning (ontology) 4 function/purpose=type (form) Group-level: Metadata order 5 5 Object grouping schemes (categories) Record groups, record series (intellectual management) Format, security concerns (physical management)

Transfer / Authenticity Metadata

The central problem: Security guaranteeing Authenticity Guarding the object (authenticity, integrity) Proving the identities of the people responsible for transferring the object (authentication, non-repudiation) Transferring the object in a secure way

What is transfer about? What is a digital copy? What qualifies? Data compression issues Data segmentation issues Creating application vs file-management application How can a digital copy be guaranteed? Digital object as string of bits Message digest of object as math on the bits Ship the message digest with the object Recalculate and compare at the other end

Guaranteeing the authenticity of the object (Integrity) Object as open or secret Must we disguise the object? Can we move it around in clear? Message digest Creates single number: “one-way hash” Number will change with the slightest change in the object on which it was calculated Encryption (Confidentiality) Asymmetric Symmetric

Accession Metadata

What is the nature of the accession task? The object received has been uprooted from its former context Object is equipped with enough metadata to reconstruct that context Contextual metadata now is no longer functional but descriptive of the old context Object must be integrated into a new context New functions must be provided for

Validation of the object Validation test suite Validation tools Formal validation process Validation outcomes Rejection Re-transfer Acceptance

Preparation of the object for storage Metadata as data and processing instructions Object and use copy Storage issues

Descriptive Metadata

Descriptive metadata for what? WWW (metatags, Dublin Core [Colorado], RDF) Finding aids (EAD) Books and other chunks (MARC) Multimedia objects (METS/Colorado) Individual objects

What about the single object? Is Dublin Core enough? What for? Who will describe at the object level? Zillions of archivists? Automatic analysis? Ad hoc analysis?

Preservation Metadata

What is Preservation Metadata? Object stability (OAIS “content data object”) What elements of the object’s content should be preserved? What is it? What is it for? What functions of the object should be preserved? (i.e., how can it remain itself into the future, and what do we mean by “itself”?) Environmental support (OAIS “environment”) What kind of environmental characteristics does the object need to stay alive (software, hardware)? (i.e., how do we specify its life support system?)

Object Stability I: Content Authenticity revisited: stability for what? Access to genuine article Historical truth Guarantee of prior art Intellectual property guarantee Range of attributes needed for each What does “content” mean?

Object Stability II: Functionality Static objects (e.g. text) Look and feel Dynamic objects (e.g. computer game) Look and feel Connectivity Interactivity

Environmental Support I: Emulation Making it possible to see the object as it was originally seen Making it possible for the object to function as it originally did Providing software support for that to happen Running the original program (in an environment that emulates the original environment) Running something that looks like (emulates) the original program

Environmental Support II: Migration Deciding what to migrate (deciding what to lose) Transformations to the object If reversible, no need to keep original object If not, retention of original object necessary

Documentation requirements for preservation What the object was What the object is What happened in between

OAIS metadata model I

OAIS metadata model II SIP (send), AIP (archive), DIP (disseminate) Parts of an object Content Preservation description Reference (unique identifier) Provenance (history in and out of repository) Context (archival bond) Fixity (message digest) Packaging Descriptive

OAIS metadata model III What is “representation information”? How much must be kept? Monitoring changes What is the “knowledge base”? Designated user community DUC as “the public”

Usage Metadata

What is Usage Metadata? Internal users (with respect to the creator External users (with respect to the creator) Internal users (with respect to the repository) External users (with respect to the repository

Creator Usage The creator’s actual use of the object Version control The creator’s colleagues’ use of the object Object function Object used for reference, model The creator’s customers’ use of the object Object function: mediates relationship

Repository Usage Management usage Object maintenance and preservation Object analysis Designated user community Object viewing Object acquisition

Rights Management Metadata

What is Rights Management? Protection of copyright Protection of patent Protection of the integrity of the digital object (and thereby reputation of the author/creator herself)

What is being protected? Object itself (integrity) Uses of the object (access controls) Limiting use (protecting rights of the owner) Enabling use (protecting rights of the user)

Protection against theft Threats of the law Fully document with metadata and protect the metadata Authentication of users and user requests Watermarking/steganography

What about integrity of the digital object? Relevant even in public domain E.g. “copyleft” agreement: See but not change, or change only with notification