Preserving eScholarship and Digitized Special Collections Distributed Digital Preservation Bill Donovan

25 March 2010Bill Donovan Boston College2 Summary As stewards of eScholarship and digitized special collections, we are responsible for saving these and other treasures effectively and economically. One approach for digital preservation is being spearheaded by the MetaArchive Cooperative; collections are replicated by peer institutions to guard against loss. The MetaArchive approach is one model for cultural memory organizations to consider adopting/adapting for their own use.

25 March 2010Bill Donovan Boston College3 Rationale for this talk Not recruiting for MetaArchive Cooperative Not recruiting for MetaArchive Cooperative DDP = a work in progress DDP = a work in progress Just one approach, but promising Just one approach, but promisingpromising –Adaptable for other “CMO” consortia? –Cultural memory organizations (CMOs) Perspective of just one member Perspective of just one member Ulterior motive: convince management Ulterior motive: convince management

25 March 2010Bill Donovan Boston College4

25 March 2010Bill Donovan Boston College5 Special Collections

25 March 2010Bill Donovan Boston College6 “Digital Preservation” defined “Digital preservation” combines policies, strategies and actions that ensure access to digital content over time. “Digital preservation” combines policies, strategies and actions that ensure access to digital content over time. esources/preserv/defdigpres0408.cfm esources/preserv/defdigpres0408.cfm esources/preserv/defdigpres0408.cfm esources/preserv/defdigpres0408.cfm

25 March 2010Bill Donovan Boston College7 Distributed Digital Preservation (DDP) geographically dispersed sites

25 March 2010Bill Donovan Boston College8 “MetaArchive Cooperative”? low-cost, high-impact DDP for “CMOs” – –e.g. libraries, research centers, and museums founded in 2004; funding from: – –NDIIPP (Library of Congress) – –NHPRC (National Archives) Not vendor-based; enable CMOs to own and control the process of digital preservation for themselves.

25 March 2010Bill Donovan Boston College9 MetaArchives’s networks

25 March 2010Bill Donovan Boston College10 MetaArchive’s ETD network

25 March 2010Bill Donovan Boston College11 Policies & Strategy  Flat, Trim, Tight-Knit organization P2P: no supermember, no host institutionP2P: no supermember, no host institution Minimal overhead, bureaucracyMinimal overhead, bureaucracy Emphasis on communication & collaborationEmphasis on communication & collaboration Committees: steering, technical, content, preservationCommittees: steering, technical, content, preservation  Self-sufficiency avoid outsourcing; retain controlavoid outsourcing; retain control cost containment, understand & refine processcost containment, understand & refine process sustainable sources of fundingsustainable sources of funding

25 March 2010Bill Donovan Boston College12 Policies & Strategy Caches (dark archives) Caches (dark archives) –6 replications –Access only via contributing member Active monitoring of the integrity of stored digital content --- NOT just back-ups Active monitoring of the integrity of stored digital content --- NOT just back-ups For ETDs, discovery via Networked Digital Library of Theses & Dissertations, NDLTD For ETDs, discovery via Networked Digital Library of Theses & Dissertations, NDLTDNDLTD

25 March 2010Bill Donovan Boston College13 Local actions/responsibilities Skills & infrastructure Skills & infrastructure Copyright responsibility Copyright responsibility Data wrangling Data wrangling –Format choices  Proprietary versus open formats open –Bit preservation versus migration –Filenaming & directories Preservation information (OAIS) Preservation information (OAIS)

25 March 2010Bill Donovan Boston College14 Adapted from: “Reference Model for an Open Archival Information System” CCSDS B-1 (2002) OAIS = Open Archival Information System

25 March 2010Bill Donovan Boston College15 OAIS preservation information Preservation Description Information Reference Information Provenance Information Context Information Fixity Information

25 March 2010Bill Donovan Boston College16 OAIS preservation information Preservation Description Information Reference Information Provenance Information Context Information Fixity Information … identifies, and if necessary describes, one or more mechanisms used to provide assigned identifiers for the Content Information. It also provides identifiers that allow outside systems to refer, unambiguously, to a particular Content Information. An example of Reference Information is an ISBN.

25 March 2010Bill Donovan Boston College17 OAIS preservation information Preservation Description Information Reference Information Provenance Information Context Information Fixity Information … documents the history of the Content Information. … tells the origin or source of the Content Information, any changes that may have taken place since it was originated, and who has had custody of it since it was originated. Examples of Provenance Information are the principal investigator who recorded the data, and the information concerning its storage, handling, and migration.

25 March 2010Bill Donovan Boston College18 OAIS preservation information Preservation Description Information Reference Information Provenance Information Context Information Fixity Information … documents the relationships of the Content Information to its environment. This includes why the Content Information was created and how it relates to other Content Information objects.

25 March 2010Bill Donovan Boston College19 OAIS preservation information Preservation Description Information Reference Information Provenance Information Context Information Fixity Information … documents the authentication mechanisms and provides authentication keys to ensure that the Content Information object has not been altered in an undocumented manner. Example: Cyclical Redundancy Check code for a file.

25 March 2010Bill Donovan Boston College20 MetaArchive hierarchy Archive (6 + caches per network) Archive (6 + caches per network) –Genre- or Format-based Collections (1 + per member) Collections (1 + per member) –Collection level metadata Archival unit (1 + per ingest) Archival unit (1 + per ingest) –e.g., all ETDs for each year

25 March 2010Bill Donovan Boston College21 Lots of Copies Keep Stuff Safe LOCKSS open-source software/support to preserve web-published materials LOCKSS open-source software/support to preserve web-published materials LOCKSS decentralized digital preservation infrastructure decentralized digital preservation infrastructure migrates content forward in time migrates content forward in time migrates content forward in time migrates content forward in time bits & bytes continually audited & repaired bits & bytes continually audited & repairedcontinually audited & repairedcontinually audited & repaired MetaArchive members also join LOCKSS MetaArchive members also join LOCKSS

25 March 2010Bill Donovan Boston College22 Private LOCKSS network (PLN) PLN is a LOCKSS network deployed by a set of like-minded institutions in order to preserve content in a closed preservation network. PLN is a LOCKSS network deployed by a set of like-minded institutions in order to preserve content in a closed preservation network. Not maintained by the Stanford University- based LOCKSS staff Not maintained by the Stanford University- based LOCKSS staff

25 March 2010Bill Donovan Boston College23 Manifest page

25 March 2010Bill Donovan Boston College24 Archival unit An independent collection of content in a LOCKSS cache. Archival units are maintained as a whole by LOCKSS daemons. They are defined by the plugin and plugin parameters.

25 March 2010Bill Donovan Boston College Digital object and its metadata

25 March 2010Bill Donovan Boston College26 Metadata xml file

25 March 2010Bill Donovan Boston College27 ETD (electronic thesis/dissertation)

25 March 2010Bill Donovan Boston College28 Plug-in An XML file that instructs the LOCKSS software how to ingest and preserve content. Each cache on the network writes a plug-in for its collection, enabling other caches to replicate its content

25 March 2010Bill Donovan Boston College29 Security Copies on different power grids Copies on different power grids All copies not accessible to one person All copies not accessible to one person Each cache secure and for DDP-only Each cache secure and for DDP-only Security-enhanced Linux Security-enhanced Linux SSL-encrypted inter-cache communication SSL-encrypted inter-cache communication IP address based Firewall exceptions IP address based Firewall exceptions

25 March 2010Bill Donovan Boston College30 For more details…

25 March 2010Bill Donovan Boston College31 MA regional library systems Massachusetts Networks: CLAMS*MBLNSAILS* NOBLE*C/W MARS*MVLC Minuteman* OCLN