Besser--Lazerow 12/10/02 1 The Challenge of Media Preservation: Digital Works and Time-Based Media Howard Besser NYU Archiving and Preservation Program and Library Senior Scientist

Besser--Lazerow 12/10/02 2 The Challenge of Media Preservation: Digital Works and Time-Based Media _ How these works differ from conventional works _ Preservation problems they pose _ Necessary paradigm shifts _ Importance of managed environments, metadata, ancillary materials

Besser--Lazerow 12/10/02 3 Conventional Works _ Manuscripts, books, paintings, sculpture _ We have a good sense of what the original object is _ Objective is to make object itself endure (temperature/humidity control, chemicals/pigments/fibers/adhesives, …) _ Goal is to keep object as close as possible to original state (though occasionally contraversy arises over whether to let aging show)

Besser--Lazerow 12/10/02 4 Electronic Media _ Video, audio, digital, new media _ Often difficult to determine what the original object is _ Difficult to make the original object endure (magnetic particle deterioration, warping, etc.) _ Even if we could make the original object endure, we wouldn’t have the infrastructure to view it in the future _ Need to develop a paradigm shift from preserving the original object to preserving info content _ Need to pay more attention to maintaining authenticity and replicating user experience

Besser--Lazerow 12/10/02 5 Preserving electronic media: What’s the problem & What can we do about it? _ The Problem: Maintaining Accessibility to a Work over time _ New Products and New Expectations necessitate New Paradigms _ The Easy Part of the Problem: Physical storage devices & hardware players _ The Hard Part of the Problem: File Formats _ Problems with Many Types of Digital Works Today _ Special Characteristics of Electronic Works _ What is the Work? _ Some approaches to solutions

Besser--Lazerow 12/10/02 6 Serious Longevity Problems  What we know from prior widespread digital file formats  Images separating from their metadata  Inaccessibility of software needed to view an image  Inability to even decode the file format of an image

Besser--Lazerow 12/10/02 7 The Problem: Maintaining Accessibility to a Work over time _ Preservation techniques for Physical artifacts do NOT address the problem of preserving digital works _ Need to shift from preserving Physical Artifact to preserving disembodied content

Besser--Lazerow 12/10/02 8 Electronic Art is not like canvas paintings _ In the future these may include –Moving image materials –Multimedia –Interactive programs –Computer generated art _ These works share some common characteristics with other “strange” works like –Performance Art –Conceptual Art –Site-specific installations –Experiential Art

Besser--Lazerow 12/10/02 9 Thinking of the Future (1/2) _ Screens will be different resolutions and different aspect ratios _ CRTs won’t exist _ A decade or 2 from now, today’s user interfaces will look like arrow-key navigation looks like today

Besser--Lazerow 12/10/02 10 Thinking of the Future (2/2) _ Today’s streaming media are small windows, slow speeds _ As bandwidth increases, viewers will expect higher quality streams _ Creators may need to consider how they’ll be able to deliver higher-bandwidth streams –Delivery Derivatives vs. Masters encoded w/standards –May also want to re-edit the piece to take advantage of changes in technology, viewer expectations, society-

Besser--Lazerow 12/10/02 11 The Easy Part of the Problem: Physical storage devices & hardware players

Besser--Lazerow 12/10/02 12 Analog - EIAJ 30

Besser--Lazerow 12/10/02 13 Analog - U-Matic

Besser--Lazerow 12/10/02 14 Analog - VHS & Betacam

Besser--Lazerow 12/10/ mm video

Besser--Lazerow 12/10/02 16 Digital -128M Optical

Besser--Lazerow 12/10/02 17 Digital - Optical R/W

Besser--Lazerow 12/10/02 18 Digital - DAT

Besser--Lazerow 12/10/02 19 Digital - Syquest

Besser--Lazerow 12/10/02 20 Digital - Zip 100

Besser--Lazerow 12/10/02 21 Digital - Writeable CD-ROM (650M)

Besser--Lazerow 12/10/02 22 Edison

Besser--Lazerow 12/10/02 23 Early Wax

Besser--Lazerow 12/10/02 24 Serious Longevity Problems  What we know from prior widespread digital file formats  Previous formats required little ongoing intervention (remote storage facilities, Iron Mtn); digital formats require intense ongoing management  The Short Life of Digital Info-

Besser--Lazerow 12/10/02 25 The Hard Part of the Problem: File Formats _ even with text like Wordstar & Word

Besser--Lazerow 12/10/02 26 Problems with Many Types of Digital Works Today-  Disappearing Information  The Viewing Problem  The Scrambling Problem  The Inter-relation Problem  The Custodial Problem  The Translation Problem

Besser--Lazerow 12/10/02 27 The Viewing Problem  Digital Info requires a whole infrastructure to view it  Each piece of that infrastructure is changing at an incredibly rapid rate  How can we ever hope to deal with all the permutations and combinations

Besser--Lazerow 12/10/02 28 The Scrambling Problem Dangers from:  Compression to ease storage & delivery  Container Architecture to enhance digital commerce

Besser--Lazerow 12/10/02 29 The Inter-relation Problem  -Info is increasingly inter-related to other info  -How do we make our own Info persist when it points to and integrates with Info owned by others?  -What is the boundary of a set of information (or even of a digital object)?

Besser--Lazerow 12/10/02 30 The Custodial Problem  In the past, much of survival was due to redundancy  How do we decide what to save?  Who should save it?  Mellon-funded E-Journal Archives  How should they save it?-

Besser--Lazerow 12/10/02 31 The Custodial Problem: How to save information?  Methods for later access  Refreshing  Migration  Emulation  Issues of authenticity and evidence

Besser--Lazerow 12/10/02 32 The Translation Problem  Content translated into new delivery devices changes meaning – -A photo vs. a painting – -If Info is produced originally in digital form in one encoded format, will it be the same when translated into another format? – Behaviors

Besser--Lazerow 12/10/02 33 Conceptual Approaches to Digital Preservation _ Refreshing always necessary due to volatility of physical strata –Impact on evidential value _ Migration -- advantages & disadvantages _ Emulation -- advantages & disadvantages _ And will need a long-term managed environment

Besser--Lazerow 12/10/02 34 OCLC/RLG Digital Repository Attributes _ Administrative responsibility _ Organizational viability _ Financial sustainability _ Technological suitability _ System security _ Procedural accountability

Besser--Lazerow 12/10/02 35 OCLC/RLG Selected Recommendations _ Policies, Certification processes, Risk management, Persistent ID, Migration/Emulation experiments _ Stakeholders meet to decide how to describe what is in a dig repository _ Examine special properties of particular classes of digital objects _ Technical standards for exchange and interoperability btwn repositories _ Develop projects and case studies _ Copyright issues

Besser--Lazerow 12/10/02 36 Preservation Repositories: Open Archival Info System Model  High-level reference model describing submission, organization and management, and continuing access  Conceptual framework for different organizations to share discussions with a common language  Producers, consumers, management, actual repository  SIP, DIP, AIP  AIP consists of data objects plus representation info (Content, Preservation Description, Packaging, Descriptive)  Originally developed for Space Science community

Besser--Lazerow 12/10/02 37 Preservation Repositories: Open Archival Info System Model Producer Management Consumer

Besser--Lazerow 12/10/02 38 Preservation Repositories -- AIP Metadata _ Preservation Description Info – reference info – context info – provenance info – fixity info _ Packaging Info _ Descriptive Info _ Content Info

Besser--Lazerow 12/10/02 39 Preservation Repositories: Projects based on OAIS Model  CEDARS  NEDLIB  Pandora  CDL  OCLC/RLG Working Group on Preservation Metadata, Attributes of a Trusted Digital Repository, August 2001-

Besser--Lazerow 12/10/02 40 Preservation Metadata  OCLC/RLG Working Group on Preservation Metadata, Preservation Metadata for Digital Objects: A Review of the State of the Art, January  OCLC/RLG Working Group on Preservation Metadata, A Recommendation for Content Information, October 2001

Besser--Lazerow 12/10/02 41 Older Longevity Projects  CPA Task Force  Getty “Time & Bits” Conference & Follow-ups-  Preservation experiments in US and Europe  NEDLIB, CURL, Michigan  Internet Archive  Long Now

Besser--Lazerow 12/10/02 42 Other Digital Preservation Activities-  LC Natl Dig Info Infrastructure & Preservation  InterPARES  Emulation Projects  E-Journal Archiving  ERPANET  Persistent Naming

Besser--Lazerow 12/10/02 43 LC’s National Digital Information Infrastructure and Preservation Program _ Authorized Dec 2000 _ LC, Dept of Commerce, NARA, White House Office of Sci & Tech Policy _ with help from CLIR, NLM, NAL, OCLC, RLG _ Ongoing collab process _ Commissioned papers on preserving: the Web, periodicals, digital sound, E-Books, Digital TV, Digital Video

Besser--Lazerow 12/10/02 44 InterPARES International Research on Permanent Authentication Records in Electronic Systems _ Ongoing international archival world project examining how to make electronically-generated records last over time _ Developing the theoretical and methodological knowledge needed, then will formulate model policies, strategies, and standards _ Next year will be extended to include images and rich media

Besser--Lazerow 12/10/02 45 Emulation Projects _ CAMiLEON (Michigan/Leeds) _ NEDLIB

Besser--Lazerow 12/10/02 46 E-Journal Archiving _ Issues – License, don’t own; may not be even able to obtain right to make archival copy – Increasingly no paper back-up at all – Usually we don’t have the important redundancy factor _ Mellon funded projects (2001) – Yale, Harvard, Penn working w/individual publishers – Cornell, NYPL--specific disciplines – MIT exploring characteristics that change (dynamic)\ – Stanford--archiving software tools

Besser--Lazerow 12/10/02 47 Electronic Resource Preservation and Access NETwork (ERPANET) _ Best practices and skills development for digital preservation of cultural heritage and scientific objects _ 3 year project launched Nov 2001; 1.2 million Euros

Besser--Lazerow 12/10/02 48 NEA 2001 grant to BAVC for $150,000 _ “To support development and dissemination of a DVD that contains a curriculum for the preservation of electronic art. The DVD will feature a preservation overview; discussions with conservators, artists, curators and technicians; a curriculum to train professionals in the field and project case studies to conserve electronic art.”

Besser--Lazerow 12/10/02 49 Special Characteristics of Electronic Works _ What Really is the Work?- _ Disappearing software _ Enormous number of elements can, at times, be very important to preserve (randomness, interactivity, pacing, color, format, original artifact, elements used to construct the artifact) _ Pieces and Boundaries _ Recontextualization (Postmodernism)--which rendition to save? _ Dynamic & Lack of Fixity (evolving works) _ Interactivity _ Historical context _ Difficulty of authentication over time

Besser--Lazerow 12/10/02 50 What’s special about Cult Heritage Materials? _ Images & rich media _ Inter-relationships btwn parts _ For Contemporary Art: What is the Work?-

Besser--Lazerow 12/10/02 51 What Really is the Work?-

Besser--Lazerow 12/10/02 52 LeWitt: Wall Drawing 340

Besser--Lazerow 12/10/02 53 Installing LeWitt

Besser--Lazerow 12/10/02 54 LeWitt Install Directions

Besser--Lazerow 12/10/02 55 ECI - Imagespace (early 80s)

Besser--Lazerow 12/10/02 56 ECI - Hole in Space (both)

Besser--Lazerow 12/10/02 57 ECI - 84-locations

Besser--Lazerow 12/10/02 58 ECI - 84-Community Memory

Besser--Lazerow 12/10/02 59 ECI - 84-kids

Besser--Lazerow 12/10/02 60 ECI - 84-MOCA

Besser--Lazerow 12/10/02 61 ECI - 84-Annotating Video

Besser--Lazerow 12/10/02 62 ECI - Avatars & Humans

Besser--Lazerow 12/10/02 63 ECI - Avatar Stage

Besser--Lazerow 12/10/02 64 Video Technology to Make the Head Spin (NYT 3/2/00)

Besser--Lazerow 12/10/02 65 Special Characteristics of Electronic Works (again) _ What Really is the Work?- _ Disappearing software _ Enormous number of elements can, at times, be very important to preserve (randomness, interactivity, pacing, color, format, original artifact, elements used to construct the artifact) _ Pieces and Boundaries _ Recontextualization (Postmodernism)--which rendition to save? _ Dynamic & Lack of Fixity (evolving works) _ Interactivity _ Historical context _ Difficulty of authentication over time

Besser--Lazerow 12/10/02 66 More Technical Issues _ Complexity of formats (storage & compression) _ Synchronicity between media/streams _ Persistent Ids- _ Website mgmt-

Besser--Lazerow 12/10/02 67 Persistent IDs--the Problem _ Need to separate work ID from work location _ URNs probably won’t be ready until 2003 _ Becomes a business process issue when one organization maintains the resource and another organization references it (ie. licensed from vendors or managed by separate administrative structures)

Besser--Lazerow 12/10/02 68 More Persistent IDs --the Approach for today _ PURLs _ Handles _ HTTP redirects _ And worry about costs now and conversion costs when URNs become feasible

Besser--Lazerow 12/10/02 69 Website Management More issues with referencing IDs _ References for mirror sites _ References for back-up sites when main site is down or bottle-necked _ References for off-site copies and archival copies

Besser--Lazerow 12/10/02 70 Pieces of the General Solution (1/2)  -We need to insist upon clearly readable standardized ways for digital objects to self- identify their formats  -We should discourage scrambling  -We need to better understand information inter-relates to other Info, and what constitutes “boundaries” of Info objects

Besser--Lazerow 12/10/02 71 Pieces of the General Solution (2/2)  -People and organizations wishing to make information persist need guidelines of how to go about doing it  -We need to better understand how translating from one storage or display format to another affects the meaning of a work  -We need to save the “behaviors” of a digital object, not just it’s “contents”

Besser--Lazerow 12/10/02 72 Metadata & Standards can be the first line of defense  Can tell you – where the file is (if you can’t find the file) – where more info about the file is (if you have the file but most other metadata has become separated) – what the file format is – what the compression scheme is – what application program and version is needed for the file

Besser--Lazerow 12/10/02 73 From the technological point of view: Standards offer the best hope of overcoming Impediments _ Easier to maintain a single set of standards over long periods of time _ Puts your institution in the same large boat with lots of other institutions who will face obsolescence and migration problems periodically throughout the future

Besser--Lazerow 12/10/02 74 What can we do specific to electronic media? _ Works themselves may no longer even exist; in many cases, what we can save amounts to forensic evidence _ Enormous number of elements can, at times, be very important to preserve (pacing, original artifact, elements used to construct the artifact) _ Too complex to save every one of these aspects for every type of material _ Importance of saving pieces, representations, and documentation _ Involve the artists to capture their intentions _ Importance of Standards _ Familiarize ourselves with recent conservation developments (Guggenheim’s Variable Media, Who Knows?, TechArcheology, Tate, IMAP)-

Besser--Lazerow 12/10/02 75 Standards for encoding artists intentions (group efforts w/i Cult Heritage community) _ Artists Interviews Project, Netherlands Institute for Cultural Heritage , Modern Art: Who Cares ( _ TechArcheology: A Symposium on Installation Preservation (SFMOMA) _ More recent SFMOMA/Tate collaborations _ IMAP _ Guggenheim’s Variable Media

Besser--Lazerow 12/10/02 76 Authors Intentions: Kendall Example

Besser--Lazerow 12/10/02 77 Standards for encoding artists intentions (possible ways to proceed) _ initially probably a few fixed fields and a lengthy open field (similar to Technical Imaging Standards for “reformatting intention” _ should identify certain types of info that should be only for museum personnel, some for general public viewing, and others unknown

Besser--Lazerow 12/10/02 78 Things that can be done _ Save documentation about the work and its context _ Save interviews with viewers about the experience _ Construct repositories that save software, works, hardware, and engage in ongoing emulation _ Encode creators’ intentions- _ Adhere to non-proprietary software/standards as much as possible-

Besser--Lazerow 12/10/02 79 Tensions around Standards Follow Standards (No Quicktime, Realmedia, DHTML, Flash, …) _ innovative, new functions

Besser--Lazerow 12/10/02 80 Structural Metadata Standards for Encoding Multimedia- _ SMIL _ MPEG 4

Besser--Lazerow 12/10/02 81 Synchronized Multimedia Integration Language (SMIL) _ For repurposing and reuse in different ways _ Use XML to reference various pieces in different ways _ Supported by Realmedia but not Microsoft or Macromedia

Besser--Lazerow 12/10/02 82 MPEG 4 _ Object-oriented _ Very low level of granularity (even objects vs backgrounds) _ Scaleable bandwidth use _ Binary Format for Scenes (BIFS) borrows concepts from VRML

Besser--Lazerow 12/10/02 83 Identification/Provenance (Images)-  The number of variant forms of a work can be enormous  Image Families  A digital image frequently has many layers of parentage  Information about the parentage that can indicate the quality and veracity of the image (Dublin Core "Source" and "Relation")  how to deal with different versions derived from the same scan or different encoding schemes  Vocabulary Standards to express this

Besser--Lazerow 12/10/02 84 The number of variant forms of a work can be enormous  different views of the same object  different scans of the same photo  different resolutions  different compression schemes  different compression ratios  different file storage formats  different details of the same image ...

Image Families

Besser--Lazerow 12/10/02 86 Identification/Provenance  how to deal with different versions (browse, hi-res, medium res) derived from the same scan or different encoding schemes (TIFF, PICT, JFIF)  Vocabulary Standards to express this – VRA Surrogate Categories – CIMI's "Image Elements”

Besser--Lazerow 12/10/02 87 Incorporate parts of Functional Requirements for Bibliographic Records (FRBR) _ work _ expression _ manifestion _ item _ (and push into “change history” section of Technical Image Metadata)

Besser--Lazerow 12/10/02 88 NISO/DLF Technical Image Metadata Workshop--4/99 ( Z draft)  create metadata needed to manage images in digital repositories over long periods of time (full life-cycle mgmt)  document image provenance & history  ensure that the images will be rendered accurately on any output device

Besser--Lazerow 12/10/02 89 Reference Models for Digital Libraries: Actors and Roles DELOS/NSF Working Group

Besser--Lazerow 12/10/02 90 NSF/DELOS Actors/Roles Project _ Classes of Actors, including – Persons – Organizations – automata _ Roles & implications – Production – Dissemination – Management – use

Besser--Lazerow 12/10/02 91 Multimedia & Collaborative Authorship imply _ Not only: – Authors – Editors – Publishers _ But also creators of – Text – Illustrations – Composers – Musicians...

Besser--Lazerow 12/10/02 92 New products & Shifts in user expectations _ VCR led to user expectation of time-shifting, which in turn created market for video-on-demand and cinema multiplexes with staggered times _ WWW will further increase expectation of immediacy, as well as: –viewing massive amounts of related materials –viewing material in fragmented ways _ DVDs are even further increasing need for ancillary materials, as well as for interactivity and viewing fragments

Besser--Lazerow 12/10/02 93 Key Paradigm Shifts for Libraries/Museums/Archives (also implications for creators) _ Maintain long-term managed environment (life-cycle if possible) _ From archiving completed whole works to asset mgmt of both component parts of works and ancillary materials related to the work _ From preserving a physical artifact to saving a digital work that has no tangible embodiment

Besser--Lazerow 12/10/02 94 Saving Component Parts & Ancillary Materials _ Edit decision list _ Out-takes _ Special effects _ Initial casting calls _ Sketches of sets _ Interviews _ Scripts... _ If the Archivists don’t act to save these, their role will be marginalized. Need to proactively seek out material that may be routinely tossed out. _ These materials may become important for either re-issuing the piece in a later version, or for future curators trying to understand the artist’s intention enough to re-display the piece when all the software used is no longer available.

Besser--Lazerow 12/10/02 95 Not just saving the Work, but also saving Conservation info about the Work

Besser--Lazerow 12/10/02 96 Condition Report UK National Trust

Besser--Lazerow 12/10/02 97 Annotated Image Mississippi State, Heather Gray

Besser--Lazerow 12/10/02 98 Polarizws Image Mississippi State, Heather Gray

Besser--Lazerow 12/10/02 99 X-Ray Investigating the Renaissance--Portrait of a Man, Fogg Museum

Besser--Lazerow 12/10/ Underdrawing Investigating the Renaissance--Last Judgment, Fogg Museum

Besser--Lazerow 12/10/ Cross-section Investigating the Renaissance--Last Judgment, Fogg Museum

Besser--Lazerow 12/10/ Helping to make sure that the electronic media (Orphan/Ephemeral/Experimental) Films of the Future Survive _ Do the research to develop better methods for saving electronic media _ Develop training programs that show how to help this material to survive _ Make sure that the archivists of the future understand the importance of this type of material _ Generate publicity to make filmmakers and institutions aware of the peril of losing these (and of how to save them)

Besser--Lazerow 12/10/ How Best to save these? _ Use Standards wherever possible _ Be aggressive about asset mgmt -- saving component parts and ancillary materials _ Both filmmaker and Archive should develop an institution-wide plan for saving electronic works –Refreshing and either migration or emulation –Standard encoding schemes –What is the work? And prioritize what needs to be saved –Save ancillary materials and records

Besser--Lazerow 12/10/ NYU Moving Image Archiving Program (MIAP) _ 2 year MA Program _ train future professionals to manage preservation- level collections of film, video, new media, and other types of digital works _ Provide prospective collection managers and archivists with an international, comprehensive education in the theories, methods, and practices of moving image archiving and preservation _ Curriculum will cover all aspects of moving image archiving

Besser--Lazerow 12/10/ MIAP Program Training _ Film History/Historiography and Film Style _ Conservation, Preservation, Storage, and Management _ Legal Issues and Copyright _ Laboratory Techniques _ Moving Image Cataloging _ Curatorial Work and Museum Studies _ Programming _ New Media and other Digital Technologies _ Access to Archival Holdings

Besser--Lazerow 12/10/ NYU MIAP Courses _ Introduction to Moving Image Archiving and Preservation _ Film History/Historiography _ Film Form and Film Sense _ Television History and Culture _ History and Culture of Museums, Archives, and other Repositories _ Conservation & Preservation of Moving Image Material--Principles _ Collection Management _ Access to Moving image Collections _ Copyright, Legal Issues, and Policy _ Handling New Media _ The Archive, the Collection, the Museum _ Curating, Programming, Exhibiting, and Repurposing/Recontextualizing Moving Image Material _ Film Restoration _ Video Restoration _ Digital Preservation and Restoration _ Elective or Independent Study _ Advanced Preservation Studies Workshop _ Thesis or Portfolio

Besser--Lazerow 12/10/ Howard Besser NYU Archiving and Preservation Program and Library Senior Scientist The Challenge of Media Preservation: Digital Works & Time-based Media

Besser--Lazerow 12/10/02 108

Besser--Lazerow 12/10/ Sustainable, Interoperable, Accessible Repositories -  Models for Digital Repositories  Importance of Metadata Standards & Philosophies  Discovery Metadata: The Dublin Core  Administrative and Structural Metadata: MOA2/METS  Actors Metadata  Longevity Metadata & Preservation Repositories  Preserving Electronic Art  Identification/Provenance  Technical Imaging Metadata  Various other Metadata

Besser--Lazerow 12/10/ Recent Digital Preservation Activities -  The Problem  Preservation Repositories  Preservation Metadata  Other Digital Preservation Activities  Special concerns of Cult Heritage community

Besser--Lazerow 12/10/ Complexity of Rich Media _ Works often have artistic nature (including video games) _ Enormous number of elements can, at times, be very important to preserve (pacing, original artifact, elements used to construct the artifact) _ Too complex to save every one of these aspects for every type of material _ Importance of saving documentation

Besser--Lazerow 12/10/02 112

Besser--Lazerow 12/10/ A few questions the NINCH community should address _ Special issues raised by non-library institutions _ Special issues raised by images and rich media _ What is the work (or salient points we need to preserve)? _ Bring the arts communities (artist intent, BAVC) together with the preservation repository communities and the preservation metadata communities _ Specifically get Cult Heritage communities involved with the selected OCLC/RLG recommendations _ Get cult heritage groups started on working to make sure that structure standards incorporate our works _ What organizations will take responsibility to save today’s digital “ephemeral” materials (online ‘zines, arts discussion groups, etc.)?

Besser--Lazerow 12/10/ From Digital Collections to Digital Libraries, Museums, and Archives _ No longer merely experiments _ Adhere to our fields’ traditions (access, interoperability, sustainable, privacy, …) _ Provide services

Besser--Lazerow 12/10/ To respond to our needs for both Service & Traditions, we face the challenges of:  Access (discovery)  Sustainability (longevity)-  Interoperability-