PeDALS Persistent Digital Archives & Library System GladysAnn Wells, Director and State Librarian Lisa Maxwell, Division Director, Records Management Division Arizona State Library, Archives and Public Records

A Word from Our Sponsors  Library of Congress National Digital Information and Infrastructure Preservation Program (NDIIPP)  Institute for Museum and Library Services Library Services and Technology Act

Project Partners  Arizona  Florida  New York  South Carolina  Wisconsin  Two states to be named  Kudos to the Washington State Archives

Technical Goals  To develop a OAIS compliant curatorial rationale that can be implemented in software to support an automated, integrated workflow to process collections of digital records and publications  Best suited for collections of records that grow out of routine process and have associated metadata  May be adapted to publications  Not immediately appropriate for ad hoc records

Technical Goals  To build “digital stacks” using LOCKSS as the basis of an inexpensive storage network that can preserve the authenticity and integrity of the materials.

Additional Goals  To build a community of shared practice that meets the needs of a wide range of repositories For best practices ~ what works, what’s practical For resource sharing ~ avoid redundant work  To remove barriers to preservation by keeping costs as low as possible

Curatorial Rationale  Transformation of traditional, paper-based practices into the digital arena that focuses on explaining why we do things in a particular manner Appraisal Acquisition Arrangement and description Housing and storage Reference and access Preservation

Architecture

Automated Processing  Curators work with rules, not records Describe business processes (rules) Monitor the process for quality assurance  Rules expressed in software A “pipeline” that transforms records as they move through the system  Based on Microsoft BizTalk middleware

Preparatory Work with Provenance  For each series of records selected for transfer Negotiate metadata you will receive Negotiate format of the records Negotiate format of the submission information package (records, metadata, shipping manifest) Negotiate frequency and manner of transfer  Provenance develops procedures to create SIPs Exploring the use of LC’s BagIt  Archives describes business rules in middleware

Preliminary Processing  Describe provenance  Describe series-level Creator, Provenance, Source of Records Series title, Date ranges Scope note Access points: activities, topics  Describe accession Assign accession number, date Assign unique system number Record source, transfer authority, restrictions

Submission  Provenance transfers submission information packages (SIPs) to a drop box FTP, sFTP, scp Tape CD, DVD  Each Provenance has a directory Each series has a subdirectory  Isolated for virus scanning  Option: simple Linux box

Data Wrangling  Before business rules are launched, the curator will validate the SIPs to ensure they conform to the negotiated specification.  Includes running New Zealand Metadata Extractor to create Software format Version File size Mime type  Some problems we’ve encountered during this step License number did not link to the file containing the record XML contained invalid control characters TIFFs were delivered instead of PDFs

Automated Item-Level Description: PeDALS Core Metadata Schema  Administration Accession number Transfer authority Restrictions  Discovery Provenance Series Title Date Summary (abstract, scope) Access points  Preservation File format, version Hash  Single schema used for all records, contains elements common to most records  Metadata/Default.aspx  Version 2.0 out mid August

Mapping and Creating Metadata Bride First Name ROBIN Bride Last Name STERNBERGER Groom First Name MANUEL Groom Last Name AVILES Bride First Name ROBIN Bride Last Name STERNBERGER Groom First Name MANUEL Groom Last Name AVILES License # Marriage Date Recording Date Batch Name ML01 SAD 01/13/06

Mapping and Creating Metadata  Title MARY ALICE CICERALE and ROBERT PORTER : marriage certificate, 2008  Extent 1 Adobe Acrobat PDF (34379 bytes)  Date 3 Jan 2008 : Marriage date 12 Jan 2008 : Recording date  Access CICERALE, MARY ALICE : Bride PORTER, ROBERT : Groom

Create Archival Information Package  Update Accessions Register database  Create AIP

Ingest into the Digital Stacks  AIPs transferred to a LOCKSS Cluster  LOCKSS Redundant Array of Inexpensive Servers Automatic integrity checking Automatic error-correction Geographically dispersed copies Bitstream preservation See LOCKSS.ORG

Public Access  For records that are not confidential, the middleware creates dissemination information packages, puts them on public webserver Doesn’t include administrative, preservation metadata that users will not likely want In format easily supported by common browsers  Middleware updates public catalog for website We assume most partners will integrate these search pages into their existing website.

For more information   Principal Investigator Richard Pearce-Moses  Project Coordinator Sara Muth