the ECHO DEPository Project A project of the University of Illinois at Urbana-Champaign and OCLC in partnership with the Library of Congress ALA Annual Chicago June 2005 Taylor Surface, OCLC

ECHO DEPository UIUC / OCLC The digital preservation problem Information is being produced in greater quantities and with greater frequency than at any time in history. –How will society preserve this information and make it available to future generations? –How will libraries and other repositories classify this information so that their patrons can find it with the same ease that they can locate a book on a shelf? The ease with which electronic information can be created and "published" makes much of what is available today, gone tomorrow. Thus there is an urgent need to preserve this information before it is forever lost. [Library of Congress (

ECHO DEPository UIUC / OCLC About NDIIPP The National Digital Information Infrastructure Preservation Project is a $99.8M national digital strategy effort led by the Library of Congress. Its mission: Develop a national strategy to collect, archive and preserve the burgeoning amounts of digital content, especially materials that are created only in digital formats, for current and future generations.

ECHO DEPository UIUC / OCLC Library of Congress NDIIPP Program Building Digital Preservation Infrastructure Partnerships Policy Standards Technical components

ECHO DEPository UIUC / OCLC NDIIPP key areas of interest Digital Preservation … Practical applications and models National technical architecture Basic research

ECHO DEPository UIUC / OCLC ECHO DEPository – Overview Design selection methodology Develop software implementing theory –Machine-assisted –Open source Evaluate various repositories –Using content gathered from tools –Other content providers Study semantic preservation techniques

ECHO DEPository UIUC / OCLC Three objectives Comparative test of repositories with various digital collections Development of Web Archives Workbench Investigations of semantic digital preservation and alternate applications of workbench tools

ECHO DEPository UIUC / OCLC Project Partners University of Illinois, Urbana-Champaign –Libraries, GSLIS, NCSA, WILL, DMI OCLC State Libraries of Arizona, Connecticut, Illinois, North Carolina and Wisconsin Tufts – Perseus Project Michigan State – Sounds Archives Library of Congress, NDIIPP Program –$3 million funding over 3 years

ECHO DEPository UIUC / OCLC ECHO DEPository Project Universe of Content Tools from this project Service Provider Repository SRB Greenstone Fedora DSpace Digital Archive NCSA UIUC OCLC Digitize d Texts G.I.S. Photos Vide o Audio Admin Data Comparative repository testing

ECHO DEPository UIUC / OCLC ECHO DEPository Project Universe of Content State Pubs Tools from this project Service Provider Repository SRB Greenstone Fedora DSpace Digital Archive NCSA UIUC OCLC Digitize d Texts G.I.S. Photos Vide o Audio Admin Data Web Archives Workbench “ Arizona model” W.A.W. development

ECHO DEPository UIUC / OCLC The Arizona Model Web domains as “archival collections” Creates efficiencies for … –Selection of “documents” –Name authority & other metadata –Browseable access

ECHO DEPository UIUC / OCLC Arizona Model: a new approach Assumptions –Content creators won’t help –Item by item selection is unsatisfactory –Bulk harvesting is unsatisfactory An archival approach –Identifying groups of similar material (series) –Automatic identification of new series items –Series description Item level description is possible if warranted –Ingest of documents into an archive

ECHO DEPository UIUC / OCLC Web Archives Workbench Apache Linux Packager Tool Packager Tool Heritrix Harvester Heritrix Harvester Cloudscape DB Cloudscape DB TomCat Discovery Tool Discovery Tool Analysis Tool Analysis Tool Properties Tool Properties Tool

ECHO DEPository UIUC / OCLC Web Archives Workbench (WAW) Tools for curators … Discovery – identify & manage domains Properties – associate metadata, content, and providers Analysis – select content from structure Packager – package content & metadata

ECHO DEPository UIUC / OCLC WAW - Discovery Tool Currently available (May 2005) Helps curators identify domains that are within their collecting scope Crawls web sites and extracts domains of possible interest from content Maintains lists of domains Monitors selected domains for changes

ECHO DEPository UIUC / OCLC WAW - Properties Tool Currently available (May 2005) Relates content providers to web sites Organizes a ‘group’ of web sites hierarchically Associates metadata to content providers and, later, to selected content Metadata can be subject headings, preferred names, aliases, etc.

ECHO DEPository UIUC / OCLC WAW - Analysis Tool Available January 2006 Content selection at varying levels of granularity –Harvests an entire site or one document Scheduled harvesting of content Shows site structure Understands serials Content is automatically associated to content provider’s metadata

ECHO DEPository UIUC / OCLC WAW - Packager Tool Available January 2006 Combines descriptive metadata about content creator, series, and object Creates administrative and preservation metadata Packages web content and metadata into an XML standard package (METS) Neutral format for ingest into OCLC archive and other repositories

ECHO DEPository UIUC / OCLC ECHO DEPository Project Universe of Content State Pubs Tools from this project Service Provider Repository SRB DSpace Digital Archive NCSA UIUC OCLC Digitize d Texts G.I.S. Photos Vide o Audio Admin Data Web Archives Workbench Digital preservation investigation Fedora Greenstone

the ECHO DEPository Project A project of the University of Illinois at Urbana-Champaign and OCLC in partnership with the Library of Congress ECHO DEPository project web site: NDIIPP web site: Me: Taylor Surface, OCLC