The NDIIP Plan The National Digital Information Infrastructure and Preservation Program Terry Harrison

Time and accident are committing daily havoc on the originals deposited in our public offices. … The lost cannot be recovered; but let us save what remains; not by vaults and locks which fence them from the public eye and use, in consigning them to the waste of time, but by such a multiplication of copies, as shall place them beyond the reach of accident. --Thomas Jefferson to Ebenezer Hazard, February 18, 1791

Preservation’s 2 nd Face: Electronic Information 1960s: Data centers appear : CPA/RLG Task Force on Archiving of Digital Information 1997: “Born digital” preservation identified

Digital Preservation Today Focus is on “born digital” –“Reborn digital” is a subset – In government…  Broad public interest – In the popular press…

Background May 1996 – Commission on Preservation and Access and The Research Libraries Group published Preserving Digital Information. July 2000 – National Academy of Science published LC21: A Digital Strategy for the Library of Congress.

Legislation Dec Congress passes legislation establishing the National Digital Information Infrastructure and Preservation Program (NDIIPP) LOC tasked for long-term preservation of digital content and capture of digital content at risk of disappearing.

National Digital Information Infrastructure and Preservation Program (NDIIPP) An initiative of the Library of Congress

NDIIPP Mission: Develop a national strategy to collect, archive and preserve the burgeoning amounts of digital content, especially materials that are created only in digital formats, for current and future generations. Funding: $150 million (half private, half Federal match)

The Challenge Libraries and archives must ensure access to digital collections in an environment of exponential growth and volatile technology.

National Digital Information Infrastructure and Preservation Program (NDIIP) Part of Consolidated Appropriations Act Fy2001 Federal partners jointly assess planning considerations Strategy to be executed in cooperation with library, creative, publishing, technology, and copyright communities nationally and internationally

NDIIP Strategic Planning Library of Congress to develop and execute Congressionally-approved strategic plan ($25 million total) –Initial planning –Acquisition and preservation of digital information that might otherwise “be uncollectible” Federal funding available for matching by non-federal donations, including in-kind contributions, through March 31, 2003 ($75 million)

Digital preservation raises issues that cannot be addressed fully within the walls of any one institution. --LC21, July 2000

Joint and Participating Partners Federal and private, libraries and archives, research and business organizations Expertise in telecommunications and e- commerce policy and practices Expertise in collection and maintenance of archives of digital materials Involved in efforts to preserve, collect and disseminate digital information

Opportunities for Partnerships Promote development of solutions for digital libraries and archives Exploit infrastructure technologies being developed for e- commerce and e-government Pursue value from existing research partnerships Develop standards and best practices Cooperate with a variety of communities with related interests

Partnering ISS Internet Archives Dr. Michael L. Nelson

Long-term preservation of digital information on a scale adequate for the demands of future research and scholarship will require a deep infrastructure capable of supporting a distributed system of digital archives. --Preserving Digital Information, Task Force on Archiving of Digital Information, May 1, 1996 The Architecture

NDIIPP Architecture To support institutions who are inclined to cooperate but have different technological systems in place –Heterogeneity guards against systemwide failure –Having everyone agree on same format/tools should not be the goal Need to make digital preservation systems interoperable –Importing/Exporting objects and collections Designed to be a “minimum requirement” subset of functionality for the LOC to describe and evaluate systems. –No longer “triple-access control dark-archives”

Architecture 2 terms Identifier - A label for an object within a system. It does not necessarily specify a location of content within the system. An ISBN is a type of identifier. Pointer- A reference to an identifier. A URL is a type of pointer. Object - Anything stored in the system that has a pointer. Unit - The smallest kind of object contained in the system. A unit is an object that contains no other objects (analogous to a file in a file system). Container - An object that contains other objects, whether units or other containers or both (analogous to a folder in a file system). (from: Update to the NDIIPP Architecture)

Layers Upper - closest to end user Middle - where most preservation services exist Lower - closest to the storage of digital objects

Architecture Near-term federation Metadata exists at Lower (repository) layer Metadata evolves over time A Container is also an Object –An atomic Object is a “unit” Diffusion of Gateway Layer functions –U may want more or less security Less Middle Layer functions (more abstract) Local vs. Public access at the Upper Layer

The NDIIP is developing a framework for national information infrastructure for preservation of digital materials through a network of partners in library and publishing communities, federal agencies and private-sector business organizations.

END

NARA’s Electronic Record Archive Vision : “The Electronic Records Archives will authentically preserve and provide access to any kind of electronic record, free from dependency on any specific hardware or software, enabling NARA to carry out its mission into the future.”

ERA at work Concerned with volume, authenticity, access Heavily focused on technology and infrastructure –Partnerships with SDSC, US Army Research Lab, NIST, NASA Special interest in developing hardware and software independent digital objects $38 million dollars in FY2003 budget

“The strategy envisages developing an information management architecture capable of preserving and delivering digital information across generations of information technologies and that is applicable across as broad a range of requirements as possible”

Persistent Object Preservation +Aims at independence of technological infrastructure +Reduce threats to integrity and authenticity by minimizing changes over time. +Embeds changes in a comprehensive information management architecture designed for preservation +Inherently extensible +Facilitates use of future, advanced technologies, without requiring change in what is preserved. -Currently beyond state of the art of information technology.

How will we develop the Electronic Records Archives? Electronic Records Archives Framework Information Technology Architecture for Persistent Digital Collections NARA ERA System

NARA Partnerships Open Archival Information System (OAIS) Reference Model –NASA, Consultative Committtee on Space Data Systems Distributed Object Computation Testbed (DOCT) –Defense Advanced Research Projects Agency, U.S. Patent and Trademark Office National Partnership for Advanced Computational Infrastructure (NPACI) –National Science Foundation Presidential Electronic Records Processing Operational System (PERPOS) –Army Research Laboratory, Georgia Tech Research Institute Archivist’s Workbench –NHPRC Grant to San Diego Supercomputer Center International research on Permanent Authentic Records in Electronic Systems (InterPARES) –7 international, multidisciplinary research teams, 10 national archives

ERA Concept model

ERA: Reference Process Request for records Retrieve collection meta-data Rebuild collection structure Put content into collection Metadata Archival Repository Reference Workbench Query Rebuild Present Collection User