Preservation, New Media, Oral Cultures How to Build a Digital Library Ian H. Witten and David Bainbridge

Mission of a Library The mission of a library is twofold: The mission of a library is twofold: To collect, organize, and provide access to information To collect, organize, and provide access to information To pass it down to succeeding generations as a record of culture To pass it down to succeeding generations as a record of culture

The Librarian’s Duty The librarian has twin duties: The librarian has twin duties: Access to the world’s literature for today’s readers Access to the world’s literature for today’s readers Preservation for future generations Preservation for future generations

The Problem of Preservation Paper Paper Acid-based paper decomposes after only a few decades Acid-based paper decomposes after only a few decades Film Film Film containing nitrate decays quickly Film containing nitrate decays quickly Analog audio Analog audio Wax cylinders or magnetic tapes must be preserved by transferring into digital formats Wax cylinders or magnetic tapes must be preserved by transferring into digital formats

The Problem of Preservation Technological progress comes at the expense of preservation Technological progress comes at the expense of preservation A process of regular copying can be established to preserve digital material without loss A process of regular copying can be established to preserve digital material without loss “No one understands how to archive digital documents” “No one understands how to archive digital documents”

Preservation Technology Enormous amounts of digital information are already lost forever Enormous amounts of digital information are already lost forever Information technologies become obsolete very quickly Information technologies become obsolete very quickly Document and media formats continue to proliferate Document and media formats continue to proliferate Technology standards will not solve fundamental issues in the preservation of digital information Technology standards will not solve fundamental issues in the preservation of digital information

Availability of Material Libraries will shortly see a demographic bulge of electronic material as the baby boom generation of authors and academics contribute material gathered during their careers Libraries will shortly see a demographic bulge of electronic material as the baby boom generation of authors and academics contribute material gathered during their careers Much material will never make it into library collections for preservation because of increasingly restrictive intellectual property and licensing regimes Much material will never make it into library collections for preservation because of increasingly restrictive intellectual property and licensing regimes Archiving and preservation functions in a digital environment will increasingly become privatized as information continues to be commodified Archiving and preservation functions in a digital environment will increasingly become privatized as information continues to be commodified

Traditional Library Functions Financial resources available to libraries and archives continue to decrease Financial resources available to libraries and archives continue to decrease Libraries and archives will be required to continue their existing archival and preservation practices as the current paper publishing boom continues Libraries and archives will be required to continue their existing archival and preservation practices as the current paper publishing boom continues

Preservation Strategies Digital documents are vulnerable to loss because the media on which they are stored decays and becomes obsolete Digital documents are vulnerable to loss because the media on which they are stored decays and becomes obsolete They become inaccessible when the software or hardware becomes obsolete They become inaccessible when the software or hardware becomes obsolete

Preservation Strategies Digital formats have advantages over analog formats Digital formats have advantages over analog formats Digital formats seem to promote preservation Digital formats seem to promote preservation The advantages make digital preservation even harder The advantages make digital preservation even harder

Preservation Strategies Ease of creation causes information glut Ease of creation causes information glut Easy of copying makes “copies” seem dispensable Easy of copying makes “copies” seem dispensable Improvements in hardware and software promote obsolescence Improvements in hardware and software promote obsolescence

Preservation Strategies “May all your problems be technical ones” “May all your problems be technical ones” Computer people recognize that the technical problems can be solved Computer people recognize that the technical problems can be solved It’s the human part that causes problems It’s the human part that causes problems Administrative and political processes take time and cause frustration Administrative and political processes take time and cause frustration Technical problems have solutions which yield to honest intellectual work Technical problems have solutions which yield to honest intellectual work

Preservation Strategies Paper and Museums Paper and Museums Involves printing the material on paper or microfilm and storing in museums Involves printing the material on paper or microfilm and storing in museums Not considered a long-term preservation strategy Not considered a long-term preservation strategy Emulation and Migration Emulation and Migration Involves preserving the physical stream of bits and/or the logical means by which the bits are interpreted as a document Involves preserving the physical stream of bits and/or the logical means by which the bits are interpreted as a document

Preservation Strategies Preserving the physical bit stream Preserving the physical bit stream Regular copying to new media Regular copying to new media Error detection to determine if degradation is occurring Error detection to determine if degradation is occurring Error correcting codes to ensure new generations are faithful copies of the original Error correcting codes to ensure new generations are faithful copies of the original

Preservation Strategies Emulation Emulation Keeping the documents in exactly the same form Keeping the documents in exactly the same form Emulate the functionality of the original, obsolete system on future, unknown systems Emulate the functionality of the original, obsolete system on future, unknown systems

Preservation Strategies Migration Migration Translating the document from the old format to a format accepted by new software Translating the document from the old format to a format accepted by new software Designed for near-obsolete software Designed for near-obsolete software Involves copying the physical bit stream to new media Involves copying the physical bit stream to new media Involves translation to a new logical format Involves translation to a new logical format

Preservation Strategies Emulation or Migration? Emulation or Migration? Migration may be cheaper Migration may be cheaper No special emulation software needs to be written No special emulation software needs to be written Conversion software is usually available Conversion software is usually available Conversion is a kind of translation Conversion is a kind of translation May lose features of the data May lose features of the data

Preservation Strategies An important feature of any format used for preserving documents it that it is open - the details are made publicly available An important feature of any format used for preserving documents it that it is open - the details are made publicly available It must be open in principle as well as practice It must be open in principle as well as practice Documented well enough for others to understand and build their own interpreters Documented well enough for others to understand and build their own interpreters Examples: PostScript and PDF Examples: PostScript and PDF

New Media Text remains the principal means for searching and browsing collections, even when they contain documents in other media Text remains the principal means for searching and browsing collections, even when they contain documents in other media

Music Music information retrieval Music information retrieval Motifs in music are analogous to key phrases in text Motifs in music are analogous to key phrases in text OMR OMR Optical music recognition Optical music recognition Music analog of OCR Music analog of OCR

Images Thumbnails Thumbnails Visual material can be rapidly browsed using thumbnails Visual material can be rapidly browsed using thumbnails Captures the readers attention Captures the readers attention Gives a feeling for what the collection is about Gives a feeling for what the collection is about Difficult to automatically search images rather than manually browse them Difficult to automatically search images rather than manually browse them Collection Understanding … Collection Understanding …

Videos Video Video a sequence of pictures? a sequence of pictures? Cut detection Cut detection Locating techniques where the scene changes Locating techniques where the scene changes Movies Movies Browsed and manipulated using thumbnails Browsed and manipulated using thumbnails Each thumbnail represents a typical image or the initial image in a scene Each thumbnail represents a typical image or the initial image in a scene

Realia Real artifacts Real artifacts Computer graphics allow three-dimensional objects to be captured in the form of a data set Computer graphics allow three-dimensional objects to be captured in the form of a data set Artifacts are indexed and located on the basis of metadata Artifacts are indexed and located on the basis of metadata

Other Document Types Teaching material Teaching material Multimedia elements Multimedia elements Research material Research material Laboratory notebooks Laboratory notebooks Scientific and engineering data Scientific and engineering data Results of experiments, simulations, and surveys Results of experiments, simulations, and surveys Information is expressed in many forms Information is expressed in many forms

Digital Libraries for Oral Cultures Libraries are about literature Libraries are about literature Literature: Literature: The writings of a society, in prose or verse The writings of a society, in prose or verse Broadly speaking, literature includes all types of fiction and nonfiction writing intended for publication Broadly speaking, literature includes all types of fiction and nonfiction writing intended for publication

Digital Libraries for Oral Cultures It should be possible to create digital library collections intended for people in oral cultures It should be possible to create digital library collections intended for people in oral cultures Useful for people who may be illiterate or semi- literate Useful for people who may be illiterate or semi- literate Useful for people who cannot speak or read the language of the digital library Useful for people who cannot speak or read the language of the digital library

Digital Libraries for Oral Cultures Iconic Form Iconic Form Serious practical information can be conveyed in a purely iconic form Serious practical information can be conveyed in a purely iconic form Examples Examples How to splint a broken forearm How to splint a broken forearm User manual for underground transport system User manual for underground transport system Historical precedent of Beggar’s Bibles Historical precedent of Beggar’s Bibles

Digital Libraries for Oral Cultures Libraries for the illiterate Libraries for the illiterate We are all illiterate with respect to some other languages and cultures We are all illiterate with respect to some other languages and cultures Media types: Media types: Static images Static images Motion, sound, video, interaction, 3D objects, simulations, virtual reality Motion, sound, video, interaction, 3D objects, simulations, virtual reality