A Digital Library Repository Utilizing the Open Archives Initiative Developed to meet the needs of UTK Library Special Collections

Tremendous quantities of valuable information exist in Museums, Libraries, and Research Centers which are not available in a standardized format via centralized search engines Photos and videosScientific recordsMathematical findings Musical scores and sound tracks Historical Documents Theses and Dissertations The Problem : How to make the connection???

 Translation of records: Into a Common Format and Language: XML & Unqualified Dublin Core  Storage: of these translations  Response: to a standardized set of queries The Open Archives Solution:  Gather document descriptions from Repositories into large databases, using OAI Harvesters  Set up search engines to offer up information in these databases

Photos and videosScientific recordsMathematical findings Musical scores and sound tracks Historical Documents Theses and Dissertations Required For Translation:  Understanding of XML and XML schemas  Determining correct mapping of information to Unqualified Dublin Core Elements, in order to translate legacy files into a metadata format supported by the Open Archives Initiative  Scripts to reduce the labor of translation

The 15 elements of Dublin Core Unqualified: A Common Language…. Dublin Core Content: Title Description Coverage Relation Source Subject Type Intellectual Property: Contributor Creator Publisher Rights Instantiation: Date Format Identifier Language

A Common Framework: XML schemas The XML schema constrains each element of the document, providing rules and framework for parsing:

SCHEME="LCSH"> Letters Cherokee Indians—Claims against Tennessee From a TEI Lite SGML file segment: To an Unqualified Dublin Core XML file segment: Letters Cherokee Indians Claims against Tennessee A Common Format…. XML

[Letter] July 8, 1839, Washington City DC, [to] HP King, Qualla Town / William Holland Thomas: a machine-readable transcription of an image … Thomas, William Holland … The University of Tennessee Libraries wt025 … This work is the property of the Special Collections Library, University of Tennessee, Knoxville, TN. It may be used freely by individuals for research, teaching, and personal use as long as this statement of availability is included in the text. … July 8, 1839 … This document is a letter dated July 8, 1839 to H.P. King from William Holland Thomas with instructions for running the Indian Store. … KEYWORDS SCHEME="LCSH"> Cherokee Indians Government relations /KEYWORDS> … … Selected Portions of a TEI-Lite SGML record

… Translated to XML Unqualified Dublin Core [Letter] July 8, 1839, Washington City DC, [to] HP King, QuallaTown The University of Tennessee Libraries, Knoxville Southeastern Native American Documents Collection (GALILEO (Georgia statewide project)) GAGAL Thomas, William Holland The University of Tennessee Libraries July 8, 1839 This document is a letter dated July 8, 1839 toH.P. King from William Holland Thomas with instructions for running the Indian Store. Document ID: wt025 Cherokee Indians Government relations This work is the property of the Special Collections Library, University of Tennessee, Knoxville, TN. It may be used freely by individuals for research, teaching, and personal use as long as this statement of availability is included in the text. letter computer file

Crosswalks available: MARC to DC: Shown in action at: OTHERS: _crosswalks/index.html Translation Tools:

The Open Archives Solution:  Gather document descriptions from Repositories into large databases, using OAI Harvesters  Set up search engines to offer up information in these databases  Translation of records: Into a Common Format and Language: XML & Unqualified Dublin Core  Storage: of these translations  Response: to a standardized set of queries

 Storage of OAI Records mysql> create table gsm( -> id char(10) not null, -> primary key (id), -> date char(10), -> path char (80), -> listit text); $sth = $dbh->prepare("select listit from $set where date <= '$until' and date >= '$from' order by id"); MySQL: small, fast, and free: Use scripts to load database and retrieve information Store entire records, already marked up in Unqualified Dublin Core, for quick response; …or Store fields untagged, multiple values for a field separated by tags, and retag upon request: flexibility. This structure allows for a record to be entered once and retrieved in various formats upon request. For local search engines, also store hardcoded xml files in a directory.

The Open Archives Solution:  Gather document descriptions from Repositories into large databases, using OAI Harvesters  Set up search engines to offer up information in these databases  Translation of records: Into a Common Format and Language: XML & Unqualified Dublin Core  Storage: of these translations  Response: to a standardized set of queries

 Response: Offer up document descriptions via a standardized set of queries & responses: the Open Archives Initiative Protocol 1)6 Verbs, with 5 required and/or optional arguments 2) Unique Identifiers, Optional Sets, and Metadata Prefixes 3) Flow control & Resumption Tokens 4) Error Codes

 Verbs and arguments: The Open Archives Protocol 1)Identify 2)ListSets 3)ListMetadataFormats: optional: identifier 4)ListIdentifiers: required: metadata prefix (oai_dc); optional: from, until, set, resumption token 5)ListRecords: required: metadata prefix (oai_dc); optional: from, until, set, resumption token 6)GetRecord: required: identifier and metadata prefix

 Identifiers, Sets, and Metadata Prefixes oai:tkn:har/har0001 oai:tkn:che/che0003 oai:tkn:civ/civ0001 oai:tkn:etd/etd0002 oai:tkn:emn/emn0001 oai:tkn:ead/ead0003 oai:tkn:gsm/gsm0045 oai:tkn:ldr/ldr0002 oai:tkn:rth/rth0034 oai:tkn:tdh/tdh0005 oai:tkn:vid/vid0001 har che civ etd emn ead gsm ldr rth tdh vid Bessie Harvey Collection Cherokee Civil War Collection Electronic Theses and Dissertations Emancipator Encoded Archival Description Great Smoky Mountains Library Development Review Roth Photography Collection Tennessee Documentary History Videos Sample Identifiers: Input as "Set": Current Sets: Supported Metadata prefix: oai_dc

 Flow Control and ResumptionTokens For ListIdentifiers, ListSets and ListRecords LRrtdc20f u LR or LI for ListRecord or ListIdentifier rt: Number or letter combination: which set next dc: Metadata format 20: Which record number to start with this time f = From date U = Until date Specifies the call to the database when this Resumption token is returned!!

badResumptionToken badVerb badArgument idDoesNotExist cannotDisseminateFormat noMetadataFormats noRecordsMatch noSetHierarchy  Error Codes: version 2.0

OAI 1.1 Test interface and Local Search Engine: Search by: word or phrase Searching by all or any field and set, Sorting by date or set Returning: Lists of identifiers or short file descriptions, each with links to full file in HTML, XML, and online document Scientific recordsMathematical findings Musical scores and sound tracks Historical Documents Theses and Dissertations Videos and Photos

The Open Archives Solution:  Gather document descriptions from Repositories into large databases, using OAI Harvesters  Set up search engines to offer up information in these databases  Translation of records: Into a Common Format and Language: XML & Unqualified Dublin Core  Storage: of these translations  Response: to a standardized set of queries

CrossWalks: _crosswalks/index.html More Information: Pre-developed repositories, harvesters, search engines, and more: Current Service Providers, who can offer searches of your records from your repository responses;