Besser--LITA Dig Imaging Preconference 7/7/00 1 Creating Working Digital Libraries Howard Besser UCLA School of Education & Information

Besser--LITA Dig Imaging Preconference 7/7/00 2 Creating Working Digital Libraries- _ Moving from Digital Collections to Digital Libraries _ Interoperability _ Importance of Standards _ Longevity _ Best Practices for Managing Digital Projects _ Some Wild Musings

Besser--LITA Dig Imaging Preconference 7/7/00 3 Moving from Digital Collections to Digital Libraries _ What’s the difference? _ Recent history of Library Automation-

Besser--LITA Dig Imaging Preconference 7/7/00 4 Developmental Stages _ Experiment with methods _ Build real operational systems _ Build interoperable operational systems

Besser--LITA Dig Imaging Preconference 7/7/00 5 Traditional Digital Library Model DL user search & presentation

Besser--LITA Dig Imaging Preconference 7/7/00 6 Ideal Digital Library Model DL user search & presentation

Besser--LITA Dig Imaging Preconference 7/7/00 7 Developmental Stages _ Experiment with methods _ Build real operational systems _ Build interoperable operational systems – For DL Initiatives – For OPACs – For I & A Services – For Image Retrieval

Besser--LITA Dig Imaging Preconference 7/7/00 8 Key problems we’re facing  Discovery  Interoperability-  Longevity-

Besser--LITA Dig Imaging Preconference 7/7/00 9 For Interoperability Digital Libraries Need Standards  Descriptive Metadata for consistent description  Discovery Metadata for finding  Administrative Metadata for viewing and maintaining  Structural Metadata for navigation ... Terms & Conditions Metadata for controlling access...

Besser--LITA Dig Imaging Preconference 7/7/00 10 Metadata is not just indexing terms _ CBIR attributes used for retrieval on color, shape, texture, etc. _ Structural attributes used for page-turning _ Administrative attributes used for managing a digital work over time _ IPR attributes to limit unauthorized use _ Identification attributes to determine what application software is needed to view a particular digital work _ Can be located anywhere

Besser--LITA Dig Imaging Preconference 7/7/00 11 Why are Standards and Metadata consensus important?  Managing digital files over time  Longevity  Interoperability  Veracity  Recording in a consistent manner  Will give vendors incentive to create applications that support this

Besser--LITA Dig Imaging Preconference 7/7/00 12 Why Standards?  Why do we need standards? – To make information universally available to users – facilitate sharing and interchange of information – To preserve information (make it safe from changes in hardware and software)  Standards only work if communities widely accept them, but they’re necessary for communities to work together

Besser--LITA Dig Imaging Preconference 7/7/00 13 Serious Longevity Problems  What we know from prior widespread digital file formats  Images separating from their metadata  Inaccessibility of software needed to view an image  Inability to even decode the file format of an image

Besser--LITA Dig Imaging Preconference 7/7/00 14 Journal Archiving _ License, don’t own; may not be even able to obtain right to make archival copy _ Increasingly no paper back-up at all _ Usually we don’t have the important redundancy factor _ Stanford’s LOCKSS Project (Lots of Copies Keeps Stuff Safe) and its problems (

Besser--LITA Dig Imaging Preconference 7/7/00 15 The Short Life of Digital Info: Digital Longevity Problems-  Disappearing Information  The Viewing Problem  The Scrambling Problem  The Inter-relation Problem  The Custodial Problem  The Translation Problem

Besser--LITA Dig Imaging Preconference 7/7/00 16 The Viewing Problem  Digital Info requires a whole infrastructure to view it  Each piece of that infrastructure is changing at an incredibly rapid rate  How can we ever hope to deal with all the permutations and combinations

Besser--LITA Dig Imaging Preconference 7/7/00 17 The Scrambling Problem Dangers from:  Compression to ease storage & delivery  Container Architecture to enhance digital commerce

Besser--LITA Dig Imaging Preconference 7/7/00 18 The Inter-relation Problem  -Info is increasingly inter-related to other info  -How do we make our own Info persist when it points to and integrates with Info owned by others?  -What is the boundary of a set of information (or even of a digital object)?

Besser--LITA Dig Imaging Preconference 7/7/00 19 The Custodial Problem  How do we decide what to save?  Who should save it?  How should they save it? – -methods for later access: emulation, migration, etc. – -issues of authenticity and evidence

Besser--LITA Dig Imaging Preconference 7/7/00 20 The Translation Problem  Content translated into new delivery devices changes meaning – -A photo vs. a painting – -If Info is produced originally in digital form in one encoded format, will it be the same when translated into another format? – Behaviors

Besser--LITA Dig Imaging Preconference 7/7/00 21 Pieces of the Solution (1/2)  -We need to insist upon clearly readable standardized ways for digital objects to self- identify their formats  -We should discourage scrambling  -We need to better understand information inter-relates to other Info, and what constitutes “boundaries” of Info objects

Besser--LITA Dig Imaging Preconference 7/7/00 22 Pieces of the Solution (2/2)  -People and organizations wishing to make information persist need guidelines of how to go about doing it  -We need to better understand how translating from one storage or display format to another affects the meaning of a work  -We need to save the “behaviors” of a digital object, not just it’s “contents”

Besser--LITA Dig Imaging Preconference 7/7/00 23 Metadata can be the first line of defense  Can tell you – where the file is (if you can’t find the file) – where more info about the file is (if you have the file but most other metadata has become separated) – what the file format is – what the compression scheme is – what application program and version is needed for the file

Besser--LITA Dig Imaging Preconference 7/7/00 24 Groups Working on the Big Longevity Problem  CPA Task Force  Getty “Time & Bits” Conference & follow- up  NEDLIB, CURL, Michigan  Internet Archive  Long Now

Besser--LITA Dig Imaging Preconference 7/7/00 25 Migration/Refreshing  Impact on evidential value

Besser--LITA Dig Imaging Preconference 7/7/00 26 Best Practices for Managing Digital Projects- _ Who will your users be? _ Best Practices Guidelines _ Workflow and Management Issues

Besser--LITA Dig Imaging Preconference 7/7/00 27 Why are you Managing this Information?  Organizational mission & type  Users  Uses

Besser--LITA Dig Imaging Preconference 7/7/00 28 Scanning Best Practices _ Think about users (and potential users), uses, and type of material/collection _ Scan at the highest quality that does not exceed the likely potential users/uses/material _ Do not let today’s delivery limitations influence your scanning file sizes; understand the difference between digital masters and derivative files used for delivery _ Many documents which appear to be bitonal actually are better represented with greyscale scans _ Include color bar and ruler in the scan _ Use objective measurements to determine scanner settings (do NOT attempt to make the image good on your particular monitor or use image processing to color correct) _ Don’t use lossy compression _ Store in a common (standardized) file format _ Capture as much metadata as is reasonably possible (including metadata about the scanning process itself)

Besser--LITA Dig Imaging Preconference 7/7/00 29 Why Scale is important

Besser--LITA Dig Imaging Preconference 7/7/00 30 Digital Object Behaviors _ Book example

Besser--LITA Dig Imaging Preconference 7/7/00 31 Metadata Standards (from MOA2) _ Administrative Metadata – for enhancing resource management _ Structural Metadata – for reflecting internal hierarchies and relationships btwn parts _ Raw/Seared/Cooked

Besser--LITA Dig Imaging Preconference 7/7/00 32 Workflow and Management Issues- _ Managing multiple image files _ Persistent Identification _ Making your works accessible throughout the Net

Besser--LITA Dig Imaging Preconference 7/7/00 33 The number of variant forms of a work can be enormous  different views of the same object  different scans of the same photo  different resolutions  different compression schemes  different compression ratios  different file storage formats  different details of the same image ...

Image Families

Besser--LITA Dig Imaging Preconference 7/7/00 35 Identification/Provenance  how to deal with different versions (browse, hi-res, medium res) derived from the same scan or different encoding schemes (TIFF, PICT, JFIF)  Vocabulary Standards to express this – VRA Surrogate Categories – CIMI's "Image Elements”

Besser--LITA Dig Imaging Preconference 7/7/00 36 Persistent IDs--the Problem _ Need to separate work ID from work location _ URNs probably won’t be ready until 2003 _ Becomes a business process issue when one organization maintains the resource and another organization references it (ie. licensed from vendors or managed by separate administrative structures)

Besser--LITA Dig Imaging Preconference 7/7/00 37 More Persistent IDs --the Approach for today _ PURLs _ Handles _ HTTP redirects _ And worry about costs now and conversion costs when URNs become feasible

Besser--LITA Dig Imaging Preconference 7/7/00 38 Data Set Management More issues with referencing IDs _ References for mirror sites _ References for back-up sites when main site is down or bottle-necked _ References for off-site copies and archival copies

Besser--LITA Dig Imaging Preconference 7/7/00 39 Making your works accessible throughout the Net _ The DLF/Mellon meeting _ An administrative and political issue as much as a a technical one

Besser--LITA Dig Imaging Preconference 7/7/00 40 Some Wild Musings- _ Movement towards packages and away from MARC _ The disappearance of OPACs

Besser--LITA Dig Imaging Preconference 7/7/00 41 Containers and Packages of Metadata Warwick, not MARC _ modular _ overlapping _ extensible _ community-based _ designed for a networked world to aid commonality btwn communities while still providing full functionality within each community

Besser--LITA Dig Imaging Preconference 7/7/00 42 DC Qualifiers _ allows one community to express important nuances and qualifications, while still making the basic importance available to communities with simple needs _ our community can reflect alternate title, transliterated title, and main title, yet they will all be found under a simple Web search under “title”

Besser--LITA Dig Imaging Preconference 7/7/00 43 Crosswalks  mapping btwn differing metadata structures  eliminate the need for monolithic, universally adopted standards  focus on flexibility and interoperatiblity  RDF-based metadata registries

Besser--LITA Dig Imaging Preconference 7/7/00 44 Crosswalk Example

Besser--LITA Dig Imaging Preconference 7/7/00 45 Do we still need OPACs? _ Why repeat almost identical bibliographic descriptions in each local system? _ Why not store only local information locally, and link to bibliographic descriptions stored in the major utilities? _ Could our acquisition systems for monographs begin to use the acquisition systems imposed on us by our parent organizations (like those for supplies)?

Besser--LITA Dig Imaging Preconference 7/7/00 46 Creating Working Digital Libraries- _ Moving from Digital Collections to Digital Libraries _ Interoperability _ Importance of Standards _ Longevity _ Best Practices for Managing Digital Projects _ Some Wild Musings

Besser--LITA Dig Imaging Preconference 7/7/00 47 Creating Working Digital Libraries Howard Besser UCLA School of Education & Information