A Preservation Repository in Prose Being a Story of the DRS Past, Present and Future By Andrea Goethals, Wendy Gogel In Cambridge, Massachusetts 2009

Today’s Agenda DRS 1: Being a Story of the Past A Transition: Being a Story of the Present DRS 2 and You!: Being a Story of the Future Questions?

DRS 1: Being a Story of the Past

The Formative years - LDI November 1997 Proposal for the Library Digital Initiative “…create the first-generation technical infrastructure to support storage of and access to digital library materials.” In July 1998, LDI was approved and funded In December 1998, planning for DRS began

Digital Repository Service (DRS) provides a set of professionally managed services to ensure the usability of securely stored digital objects over time. is both a preservation and an access repository includes the bundled delivery services October 2000 Launch

LDI Grant projects 49 Grants were awarded Digitizing analog collections Images Text Audio Music scores Born Digital Biomedical images Geospatial data Web sites Online cataloging projects

Digitizing Facilities June 1999 Harvard College Library Imaging Services HCL Fine Arts Library Digital Imaging Lab (FAL DIL) Harvard Art Museum Digital Imaging and Visual Resources (DIVR) Harvard College Library Audio Preservation Services (HCL APS) Peabody Museum of Archaeology and Ethnology

The first Deposit and the first object was deposited on October 23, 2000…

w/ Metadata Administrative Stewardship, contacts (e.g., HCL Harvard-Yenching Library, Ray Lum, etc.) Billing account (e.g., 33-digit account number) Access flag (e.g., open to the public, restricted to the Harvard community, no access) Technical Physical characteristics (e.g. for images, x and y resolution, MD5 signature, pixel width and height, compression, bit sample rate, etc.) Production methods (e.g. for images, Scitex; Leaf Volare; Leaf Colorshop 5.x )

The first Book was deposited on June 29, 2001

The first Audio was deposited on January 28, 2003 Matins for Sunday after the Elevation of the Holy CrossMatins for Sunday after the Elevation of the Holy Cross Laura Boulton ( ) Collection of Byzantine and Orthodox Musics Archive of World Music One of a series of Byzantine hymns and liturgies recorded in a monastery on Patmos, Logbook (Part I, p. 1-10)

The first georeferenced map was deposited on January 14, 2005 Barnstable, Massachusetts 15 Minute Digital Raster Graphic From an 1893 Historic USGS map reprinted in 1907

Systems and Services 1985 HOLLIS –our OPAC VIA Visual Information Access– union catalog OASIS Online Archival Search Information System – union catalog OLIVIA – image cataloging tool

Systems and Services DRS Digital Repository Service – preservation and access repository NRS Name Resolution Service – to resolve persistent identifiers AMS Access Management Service – to provide access controls IDS Image Delivery Service PDS Page Delivery Service FTS Full-text Search Service NRS Web Admin Policy Web Admin

Systems and Services DRS Web Admin – staff interface to DRS PDS Maint Harvard Geospatial Library – union catalog TED TEmplated Database – collection building tool SDS Streaming Delivery Service – for audio delivery ADS Asynchronous Delivery – for large files Cross-catalog search – for federated searching

Systems and Services Dynamic IDS – for zoom and pan features w/ JP2 DMART- Audio deposit tool RList – Course reserves tool Virtual Collections Batch Builder Google data loading WAX

A Transition: Being a Story of the Present

2008: new DRS storage system New servers, new storage arrays, new tape library, new storage software Increased storage capacity Less complex - DRS loader doesn’t need to know the details of storage system anymore Higher availability for deliverable content Copies stored in 3 different geographic locations 3 “low use” copies, 4 “high use” copies

Cumulative file count per format type

Annual file size per harvard unit (gb) HCL Art Museums

Cumulative non-Google file size per use (gb) April 2009: 45,742 GB

Cumulative file size (gb) April 2009: 105,652 GB

DIY --

2008: new program, new position HUL takes next step in its commitment to digital preservation and establishes: 1.Digital Preservation and Repository Manager Position March 2008 Andrea Goethals 2.Digital Preservation Program June 2008 Established within OIS

2008/9 priorities of new digital preservation program 1.Define additional infrastructure requirements to support digital preservation DRS enhancements Global digital format registry (GDFR) 2.Identify and analyze new formats for the DRS to support PDF, , audio, architectural drawings, etc. 3.Establish communication network with the 2 communities we inhabit Broader digital preservation community Harvard community

Avenues of communication Broader digital preservation community Conferences and meetings Collaborative projects conversations, blogs, newsgroups Harvard community Committees (ULC, CCCC, DMCC, DCSWG, etc.) Digital project librarians Ad-hoc focus groups, meetings and with stakeholders (depositors, curators and collection managers) Customer surveys

These communities inform our thinking about: Concepts and terms Metadata Data models Content Recommended & supported formats Best practices Preservation planning and actions Storage, management and monitoring Certifications Registries Tools and services

DRS customer survey 2008 August - September 2008 Users of DRS tools or services To evaluate the level of satisfaction with DRS tools, services, and websites To understand any unmet needs

Survey findings Question 1: What word or phrase best describes the DRS? In general the DRS is valued for its preservation services and perceived as stable, secure and trusted.

Other key findings of survey DRS Customers want: Support for more formats Guidance on preservation formats and content creation Better search and editing management tools Delivery services that use common or popular third-party applications

Trends in DRS customer needs 1.Problem of abundance 2.Remote creators 3.Diversity of formats

1. Problem of abundance DRS owners and depositors: Are increasingly overwhelmed by the amount of digital content to preserve Can’t fully process the material they want to deposit into the DRS Can’t go through a deposit process that is time-consuming

2. Remote creators Increasingly DRS owners and depositors are acquiring content they did not create DRS staff can not influence the formats or technical properties of this content during creation

3. Diversity of formats DRS owners and depositors increasingly need to preserve formats and genres that aren’t currently supported by the DRS CAD formatsSpreadsheet formats 3D visualization formatsPresentation formats Additional audio formatsDatabases Video formatsLocally archived websites Executable file formatsRaw survey data Word processing formatsRaw camera files

Implications of these trends The DRS needs to: accept and preserve minimally-processed content provide a time-efficient deposit process support a broad range of formats and genres And: can’t rely on the content being in “preservable” formats prior to deposit into the DRS

DRS 2 and You!: Being a Story of the Future

DRS 2 changes Why? 1.To better support digital preservation 2.To better support needs of DRS depositors, curators and collection managers

DRS 2 changes 1.New conceptual foundation Objects Content models 2.User improvements Support for opaque objects Support for new file formats Deposit, management & delivery tools Guidance & user community 3.A new approach to metadata 4.Increased preservation planning and activities

Objects Currently only a file level in the DRS All management has to be done at the individual file level Objects are aggregations of files Page-turned object Still image object More intuitive unit for management, reporting and searching Example: How many Page-turned objects do I have in the DRS?

Content models Types of objects Example: audio content model

Support for opaque objects A special content model Allows files in any format The digital equivalent of buying time at HD Content can be minimally processed Must be intended for long-term preservation The content could be fully processed by depositors but not supported yet by DRS Will receive some preservation services Will be on a path to fuller DRS preservation

Support for new file formats PDF Audio MP3, MP4/AAC Drawings AutoCAD Adobe Illustrator Video What’s next?

Deposit, management & delivery tools Enhanced Batch Builder Integrated with File Information Tool Set (FITS) Enhanced DRS Web Admin Better searching Richer management and reporting Ability to perform batch updates File Delivery Service (FDS) Created for PDF delivery Delivers a file to user’s web browser

Future of

Guidance & user community New website for digital preservation Formats central Content models DRS practices HUL digital preservation projects Emerging standards and best practices Tools, services, registries Resources & Experts

A new approach to metadata Moving towards community-standard schemas PREMIS, MODS, MIX, textMD, etc. Metadata files on the file system alongside content files “object descriptors” Preservation, rights, descriptive metadata More reliance on embedded metadata Automatic extraction at deposit time by FITS Third party delivery applications are becoming aware of file-embedded metadata

Increased preservation planning and activities More granular format identification Sub-file characterization Preservation plans per content model Digital first aid (content & metadata) “Localization,” migrations, normalizations Technology watch Virus checking

DRS 2 process Phases of work DRS 2.1, 2.2, 2.3, etc. Themed phases DRS 2.1: “Object Security and Integrity” DRS 2.2: “Management and Monitoring” Includes support for new formats DRS 2.1: PDFs, opaque objects DRS 2.2: more audio formats (MP3, MP4/AAC)

Questions?

Image credits Future ghost Marley’s ghost _marley27s_ghost.jpghttp://cueballcol.files.wordpress.com/2007/12/435px-a_christmas_carol_- _marley27s_ghost.jpg Ghost of the past Ghost-Of-Marley,-From-Dickens-A-Christmas-Carol.jpghttps:// Ghost-Of-Marley,-From-Dickens-A-Christmas-Carol.jpg Ignorance and want Weight of wikipedia pghttp://images.theage.com.au/ftage/ffximage/2008/05/26/300_wikipedia1.j pg Lots of people Ghost of the future Mr. Magoo re_small.jpghttp:// re_small.jpg