Archive Ingest and Handling Test: ODU’s Perspective Michael L. Nelson Department of Computer Science Old Dominion University

Slides:

Advertisements

Similar presentations

Research Data Access and Preservation Summit Panel 2 - Promoting Re-Use of Scientific Collections Some responses to the questions posed... John Harrison.

Advertisements

Long-Term Preservation. Technical Approaches to Long-Term Preservation the challenge is to interpret formats a similar development: sound carriers From.

An Introduction to Repositories Thornton Staples Director of Community Strategy and Alliances Director of the Fedora Project.

Using OAI-PMH for Resource Exchange OAI Metadata Harvesting Workshop, JCDL 03 Michael L. Nelson, Terry L. Harrison Old Dominion University Norfolk VA

The Open Archives Initiative DRIADE Workshop, Durham NC, May 16-17, 2007 Michael L. Nelson The Open Archives Initiative Michael L. Nelson Computer Science,

National Geospatial Digital Archive Greg Janée. Greg Janée May 31, Outline Two preservation misadventures Digital preservation problems Genesis.

Depositing e-material to The National Library of Sweden.

Preservation Metadata Extraction and Collection : Tools and Techniques Mat Black National Library of New Zealand Te Puna Matauranga o Aotearoa.

Merrilee Proffitt e(X)literature / Digital Cultures Project April 2003 News from the Digital Library The Metadata Encoding and Transmission Standard; the.

WMES3103 : INFORMATION RETRIEVAL

William Y. Arms Corporation for National Research Initiatives March 22, 1999 Object models, overlay journals, and virtual collections.

1 CS 502: Computing Methods for Digital Libraries Lecture 27 Preservation.

Incompatible or Interoperable? A METS bridge for a small gap between two digital preservation software packages Lucas Mak Metadata & CatalogLibrarian

Metadata Standards and Applications 4. Metadata Syntaxes and Containers.

Ingest and Dissemination with DAITSS Presented by Randy Fischer, Programmer, Florida Center for Library Automation, University of Florida DigCCurr2007.

Implementing an Integrated Digital Asset Management System: FEDORA and OAIS in Context Paul Bevan DAMS Implementation Manager

Digital Preservation 101, or, How to Keep Bits for Centuries Julie C. Swierczek Digital Asset Manager and Digital Archivist Harvard Art Museums.

U.S. Department of the Interior U.S. Geological Survey Information Technology Exchange Meeting May 24 – 28, 2010 A New Decade in Support of Science Thinking.

University of North Texas Libraries Building Search Systems for Digital Library Collections Mark E. Phillips Texas Conference on Digital Libraries May.

XML and Digital Libraries M. Zubair Department of Computer Science Old Dominion University.

Dynamic Web Pages & JavaScript. Dynamic Web Pages Dynamic = Change Dynamic Web Pages are web pages that change. More than just moving graphics around.

The Library of Congress Martha Anderson Program Officer, NDIIPP Office of Strategic Initiatives Library of Congress April 2005 LC Perspective : Preservation.

NCSU Libraries 27 March 2006 Digital Preservation in State Government – Wilmington, NC North Carolina Geospatial Data Archiving Project Workflow, Tools,

Ocean Observatories Initiative Data Management (DM) Subsystem Overview Michael Meisinger September 29, 2009.

Elements of a Data Management Plan: Roles and Responsibilities Ruth Duerr National Snow and Ice Data Center Version 1.0 Review Date.

Implementor’s Panel: BL’s eJournal Archiving solution using METS, MODS and PREMIS Markus Enders, British Library DC2008, Berlin.

Metadata hussein suleman uct cs honours Data and Metadata  Data refers to digital objects that contain useful information for information seekers.

National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Management of Distributed Data Reagan W. Moore.

Vidispine Data Model Vidispine Bootcamp. Overview Collection Storage File Item Shape Item Component Shape Component Metadata abstract entity physical.

Repository Synchronization Using NNTP and SMTP Michael L. Nelson, Joan A. Smith, Martin Klein Old Dominion University Norfolk VA

INTELLECTUAL RIGHTS AND HISTORIC CORPORA Mark Sandler University of Michigan ICOLC, March, 2003.

DRS 2 Project (2008 – Present!) Andrea Goethals, Harvard Library Digital Preservation Management Workshop, MIT June 13, 2013.

OAIS Rathachai Chawuthai Information Management CSIM / AIT Issued document 1.0.

Challenges in the Nursery: Linking a Finding Aid with Online Content Elizabeth Johnson, Lilly Library Jenn Riley, Digital Library Program DL Brown Bag,

26/05/2005 Research Infrastructures - 'eInfrastructure: Grid initiatives‘ FP INFRASTRUCTURES-71 DIMMI Project a DI gital M ulti M edia I nfrastructure.

JISC/NSF PI Meeting, June Archon - A Digital Library that Federates Physics Collections with Varying Degrees of Metadata Richness Department of Computer.

Digital Collections Forum Doug Moncur AIATSIS September 2004.

Archive Ingest and Handling Test: ODU’s Perspective Michael L. Nelson Department of Computer Science Old Dominion University

OAI-PMH for Resource Harvesting Tutorial OAI4, October 20 th 2005, CERN, Geneva, Switzerland The American Physical Society Project: Standards-based Mirroring.

UKOLN is supported by: Content packaging and MPEG-21 DID Andy Powell, UKOLN, University of Bath JISC Joint Programmes Meeting, July.

Evaluating Ingest Success: Using the AIHT Michael L. Nelson, Joan A. Smith Department of Computer Science Old Dominion University Norfolk VA DCC.

Lifecycle Metadata for Digital Objects November 15, 2004 Preservation Metadata.

Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.

Collection-Based Persistent Archives Arcot Rajasekar, Richard Marciano, Reagan Moore San Diego Supercomputer Center Presented by: Preetham A Gowda.

Mod_oai: Metadata Harvesting for Everyone Michael L. Nelson, Herbert Van de Sompel, Xiaoming Liu, Aravind Elango

Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.

Introduction to Digital Libraries Assignment #1 Old Dominion University Department of Computer Science CS 751/851 Spring 2015 Michael L. Nelson 01/22/15.

Transparent Format Migration of Preserved Web Content D. S. H. Rosenthal, T. Lipkis, T. S. Robertson, S. Morabito Lib Magazine, 11(1), 2005

The Multi-Faceted Use of the OAI-PMH in the LANL Repository Written By: Henry, Xiaoming,Patrick Henry, Xiaoming,Patrick and Herbert. Presented By: Shashi.

Building Digital Archives Mark Phillips Cathy Hartman June 6, 2008.

Ingest and Dissemination with DAITSS

knowledge organization for a food secure world

Bentley Project Reel Digitization Bentley Historical Library t

Flexible Extensible Digital Object Repository Architecture

CS 501: Software Engineering Fall 1999

Flexible Extensible Digital Object Repository Architecture

Introduction to Digital Libraries Week 6: Complex Objects, Part 1

Introduction to Digital Libraries Week 8: Complex Objects, Part 1

Introduction to Digital Libraries Assignment #3

Metadata to fit your needs... How much is too much?

Characterization of Search Engine Caches

Introduction to Digital Libraries Assignment #3

Introduction to Digital Libraries Assignment #3

Introduction to Digital Libraries Assignment #3

Introduction to Digital Libraries Assignment #2

Introduction to Digital Libraries Assignment #3

Introduction to Digital Libraries Assignment #3

Introduction to Digital Libraries Assignment #1

Introduction to Digital Libraries Assignment #4

Introduction to Digital Libraries Assignment #2

Presentation transcript:

Archive Ingest and Handling Test: ODU’s Perspective Michael L. Nelson Department of Computer Science Old Dominion University NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Preservation: Fortress Model 1.Get a lot of $ 2.Buy a lot of disks, machines, tapes, etc. 3.Hire an army of staff 4.Load a small amount of data 5.“Look upon my archive ye Mighty, and despair!” image from: Five Easy Steps for Preservation:

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July ODU’s Research Goals We’re in the CS department, not the library –Less infrastructure (bad) –More freedom (good) Interested in repository/object interaction –Long-range vision: repositories fade away; objects are responsible for their own preservation –Could we accomplish this with our “bucket” technology? Significant questions about archive granularity Transition to MPEG-21 Digital Item Declaration Language (DIDL) based buckets New models for digital preservation?

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Buckets Buckets: self-contained, web-accessible objects –Grew out of research for serving NASA documents, esp. NACA Reports CACM, 2001: – implicit assumptions: 1 bucket = 1 logical item (N physical items) Display is for human use Bucket contents are DOM-parsable

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Which Interface? Display based on web useDisplay based on archival use

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July MPEG-21 DIDL A generic, powerful complex object metadata format –Based on an abstract data model –Semantics separated from syntax i.e. the tags don’t mean anything -- a little disconcerting at first glance –Digital library use championed by LANL

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July MPEG-21 DIDL Data Model How to encode Archive? 1 file = 1 DID 1 archive = 1 container 1 archive = 1 component 1 file = 1 component

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July File = 1 Component 8 file archive for demo purposes…

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Looking Inside the Archive

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Looking at a Single File…

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Design Decisions: File Storage Store each file as a –Big: each file is base64’d into the DIDL –Small: each file is ref’d from the DIDL to a directory Filename = MD5 hash of the original file name (not contents!) + a version number Example:

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Design Decisions: Ingestion For every program/process to apply to a file, create a corresponding –Jhove –Unix “file” –Fred URI –MD5 of file contents Expandable, scriptable list of metadata extraction / analysis programs

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July perl/Digest::M D a1bcd2b e7cf05f36066d4cdc9cf

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July In Vivo Preservation

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Harvard Ingest

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July Bucket / MPEG-21 Model

Archive Ingest and Handling Test: ODU’s Perspective NDIIP Partners Meeting, Airlie House, VA, July METS/MPEG-21 / mod_oai Use mod_oai as DIP