Core Issues in Digital Preservation Jacob Nadal, Preservation Officer UCLA Library

PERSPECTIVE Is there a general preservation framework that applies to all records? How does it differ in application between artifactual and digital preservation?

-ed preserv--ation Preservation consists in sustainable efforts, optimized over time

-ed-ing -ablepreserv--ation Preservation consists in sustainable efforts, optimized over time

Framework Materials: tangible substance that carries media Media: materials that record information Transport: means(s) for perceiving media Language: system for interpretation of media

LINEAR B: Digital Preservation Analog Photo: British Museum

Linear B Bronze Age Cretan script: c to 1375 B.C. No cribs, such as the Rosetta Stone, an almost entirely logical decipherment This is the essential problem that digital preservation tries to avert or mitigate Show all four parts of our preservation framework Discovered by Sir Arthur Evans, in spring of 1900 on numerous inscribed (media) clay (material) tablets.

Linear B Tablet and Transcribed Glyphs Photo: Dennis Jarvis

First successes Counting system was easy to determine Analog to Digital: Some formats & encodings are favored because they’re easy to identify 90 distinct characters, indicative of a syllabic system, with a writing direction from left to right Debate over relation to Greek or Cypriot. Most felt it was a unique Cretan language. Analog to Digital: Encoding and File Format

Alice Kober: Pattern Recognition Alice Kober identifies word triplets Same word stem with different endings, presumably for case (e.g. accusative, or nominative) Kober separated symbols into modifiers and word stems Analog to Digital: Metadata, Headers, Content blocks, Structured data

Michael Ventris: Patterns to Prose Consonant-vowel patterns established Problem of missing vowels and leading vowels: e.g. di-vi-si-b(i)-le or i-n(i)-di-vi-si-b(i)-le Analog to Digital: The problem of compression Developed refinements of Kober’s chart to manage these relationships

A few good guesses Refinement of relationships gave Ventris enough confidence to take a guess at three words, the towns of Anisos, Knossos, and Tulissos Assigning consonant values opened up more words Greek philologist John Chadwick partnered to carry forward the decoding of a Greek dialect from the time of the Trojan War.

In effect, those are the issues in digital preservation: Began with identification of parts… Digital Forensics and Analysis … associated with possible informational content … Metadata and Contextual Information … then instantiated by a subject expert and translated into a known, contemporary language. Digital Curation and Migration

Defining Digital Preservation Photo: John Keogh

Short Definition of Digital Preservation Digital preservation combines policies, strategies and actions that ensure access to digital content over time. Remove digital and it’s a generic definition of preservation The medium definition adds some strongly digital concepts, that do not bear heavily on artifactual preservation.

Medium Definition of DIgital Preservation Digital preservation combines policies, strategies and actions to ensure access to reformatted and born digital content regardless of the challenges of media failure and technological change. The goal of digital preservation is the accurate rendering of authenticated content over time.

Long Definition Core Digital preservation combines policies, strategies and actions to ensure the accurate rendering of authenticated content over time, regardless of the challenges of media failure and technological change. Digital preservation applies to both born digital and reformatted content. Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information. Digital preservation strategies and actions address content creation, integrity and maintenance.

Long Form Details: Content Creation Content creation includes: Clear and complete technical specifications Production of reliable master files Sufficient descriptive, administrative and structural metadata to ensure future access Detailed quality control of processes

Long Form Details: Content Integrity Content integrity includes: Documentation of all policies, strategies and procedures Use of persistent identifiers Recorded provenance and change history for all objects Verification mechanisms Attention to security requirements Routine audits

Long Form Details: Content Maintenance Content maintenance includes: A robust computing and networking infrastructure Storage and synchronization of files at multiple sites Continuous monitoring and management of files Programs for refreshing, migration and emulation Creation and testing of disaster prevention and recovery plans Periodic review and updating of policies and procedures

Content creation includes: → Clear and complete technical specifications → Production of reliable master files → Sufficient descriptive, administrative and structural metadata to ensure future access → Detailed quality control of processes Content integrity includes: → Documentation of all policies, strategies and procedures → Use of persistent identifiers → Recorded provenance and change history for all objects → Verification mechanisms → Attention to security requirements → Routine audits Content maintenance includes: → A robust computing and networking infrastructure → Storage and synchronization of files at multiple sites → Continuous monitoring and management of files → Programs for refreshing, migration and emulation → Creation and testing of disaster prevention and recovery plans → Periodic review and updating of policies and procedures Digital preservation combines policies, strategies and actions to ensure the accurate rendering of authenticated content over time, regardless of the challenges of media failure and technological change. Digital preservation applies to both born digital and reformatted content. Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information. Digital preservation strategies and actions address content creation, integrity and maintenance. Long Form (Detail)Long Form (Core)

Content creation includes: → Clear and complete technical specifications → Production of reliable master files → Sufficient descriptive, administrative and structural metadata to ensure future access → Detailed quality control of processes Content integrity includes: → Documentation of all policies, strategies and procedures → Use of persistent identifiers → Recorded provenance and change history for all objects → Verification mechanisms → Attention to security requirements → Routine audits Content maintenance includes: → A robust computing and networking infrastructure → Storage and synchronization of files at multiple sites → Continuous monitoring and management of files → Programs for refreshing, migration and emulation → Creation and testing of disaster prevention and recovery plans → Periodic review and updating of policies and procedures Digital preservation combines policies, strategies and actions to ensure the accurate rendering of authenticated content over time, regardless of the challenges of media failure and technological change. Digital preservation applies to both born digital and reformatted content. Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information. Digital preservation strategies and actions address content creation, integrity and maintenance. Long Form (Detail)Long Form (Core) Create Good Files Keep an Eye on Them Store Them Safely Don’t get lost in the fine print

Text

UTF-8, a way of representing Unicode, is standard Digital text is purely character data No font or layout information is stored in a pure text file Critical for searching and manipulation XML is a UTF-8 text format

Images

TIFF standard preservation format; JPEG2000 emerging as a new alternative Must be uncompressed image data (TIFF and JP2K can both store compressed data) At least 300 pixels per inch (ppi/dpi), 24-bit color More pixels allows more magnification without pixelation Color should be calibrated and profiled with an ICC color profile.

Audio

Broadcast WAV (BWAV) – Wave file with a metadata header WAV audio is Pulse Code Modulation (PCM), the universal format for uncompressed audio Resolution of at least 44.1 kHz (CD quality), preferably 96 kHz Bit Depth of at least 16-bit (CD quality), pref. 24-bit

Video & Moving Image

Standards and practices developing Uncompressed desirable, but high storage costs Compression is normal in video, but may cause preservation problems Uncompressed.AVI is the current safe bet Motion JP2K & MPEG21 may be options H.264 becoming the standard for service copies Pick one, but plan on a migration

Data and Interactivity

Need to decide if fixed points in time are required: Are you storing an instance of data? Need to decide if active system is required: Are you maintaining and experience or immersive environment? Or, are you doing both? ICPSR: CDL: Variable Media Network: variablemedia.netvariablemedia.net

Metadata

PREMIS: PREservation Metadata: Implementation Strategies

Storage and Maintenance

Lots of options LOCKSS Networks Digital archives (OCLC digital archive, DuraSpace) DIY systems, from a couple removable hard drives, to cloud storage, to building your own data center.

1. OAIS compliance 2. Administrative responsibility 3. Organizational viability 4. Financial sustainability 5. Technological and procedural suitability 6. System security 7. Procedural accountability RLG: Trusted Digital Repositories

The OAIS Reference Model

Digital Preservation: Reasonable Expectations Digital preservation has strong points and weak points; so does artifactual preservation. With digital preservation, we should expect High day to day reliability Low incidence of acid decay, mold, or biohazards Some preservation problems in the future; stick to standards and the impact will be mitigated

Methods of Preservation Digital Archaeology: Recovery and forensic analysis of data from damaged media Conservation: Maintaining original equipment for access Bit preservation: Storage, transfer and refresh of data Migration: Transformation of data into new formats to allow for continued access Emulation: Recreation of original operating environment for continued access

Methods of Preservation Digital Archaeology: Recovery and forensic analysis of data from damaged media Conservation: Maintaining original equipment for access Bit preservation: Storage, transfer and refresh of data Migration: Transformation of data into new formats to allow for continued access Emulation: Recreation of original operating environment for continued access Now, but also never. This is what’s next. Step one (and 2… n)

THANK YOU! Questions & Comments: jacobnadal.com/247