ComPADRE Experiences developing an OAI server over an existing database repository Resources for Physics and Astronomy Education Lyle Barbato American Association of Physics Teachers

The ComPADRE Project NSF/NSDL funded repository for Physics and Astronomy specific resources Partnerships and collaborations – AAPT: American Association of Physics Teachers – AAS: American Astronomical Society – APS: American Physical Society – AIP/SPS: Society of Physics Students – Merlot: Multimedia Educational Resource for Learning and Online Teaching Multiple collection interfaces for different target audiences Collection interfaces share a common architecture so we can build once and use everywhere

Why OAI? Fits well as a shared service Necessary step to becoming a service provider for physics and astronomy data providers NSDL harvesting Increased content visibility Theoretically, it has a ‘simple’ and ‘easy’ implementation

Our Backend Structure Database driven – Our records are generated on-the-fly Database model based on the Learning Object Model (LOM) metadata standard

OAI Implementation Experience Research of OAI tools and protocol OAI Server Implementation and corresponding database changes Implementation of metadata formats – OAI_DC A few hours – NSDL_DC A few weeks

OAI Server Implementation Only 6 Verbs! – But opportunities still exist for problems and confusion Focusing on implementation problems, they were either: – Related to our database infrastructure Datestamps for exported ‘on the fly’ records General deletion handling – Or related to supporting shared item records for multiple collections with separate workflows

Datestamp Issues OAI requires datestamps be applied on the metadata record level. This raises issues with database-based items: – When exports are done on the fly, item level last modified datestamps are not good enough since fields that change in one format may not be included in another – When a new metadata format is created, it will have a different OAI datestamp for creation, however the database record will already exist

Deleted Record Issues The goal is for ‘persistent’ deleted records However: – The datestamp for a deleted record must be the time it was deleted Thus we could not use the ‘Last Modified’ date of our records due to potential future database-wide global vocabulary updates. – Metadata versioning It happens. Plan to consistently export all metadata format versions forever Or plan to have ‘deleted’ record holders for existing records at the time of the metadata format removal

Additional recommendations For Service Providers – To avoid loss of context and information: Place set info into the record using IsPartOf to alleviate “On the Horse” problems If possible, Indicate the set size in the setDescription – Provide internal unique ids for de-duping – Indicate source of metadata – Expose the richest metadata format possible

Provide Richer Metadata – How? Expose the richest metadata format possible – However, the OAI literature has little on how to create your own xml schema if you need to: has a great tutorial on how to create a new schema by extending the oai_dc schema to add new elements at – Also, if you have an internal vocabulary that you wish to expose, a page that gives instructions for creating an xml schema ‘vocabulary’ via enumeration:

When it’s all said and done There are tools which can do this for you BUT If your infrastructure doesn’t mesh with existing tools, OAI is simple enough to implement yourself.

References Compadre – Open Archives Initiative – Digital Library Federation OAI and Shareable Metadata Best Practices Working Group – NSDL Metadata Resources Page – Dublin Core Metadata Initiative –

References The Basics of OAI: An introduction to the Protocol for Metadata Harvesting – Web-Wise 2004: Sharing Digital Resources Tim Cole, Sarah Shreeves, Martin Halbert Bitter Harvest: Problems & Suggested Solutions for OAI-PMH Data & Service Providers – Roy Tennant California Digital Library Challenges for Service Providers When Importing Metadata in Digital Libraries – Marilyn McClelland, David McArthur, Sarah Giersch, Gary Geisler

References JCDL OAI Workshop 2003 – Naomi Dushay _OAI_Workshop_Naomi_Dushay.ppt Improving Metadata Quality: Augmentation and Recombination – Naomi Dushay, Diane Hillmann, Jon Phipps DC2004.pdf DLF Best Practices for OAI Data Providers and for Sharable Metadata - Proposal for a Workplan – Sarah Shreeves Set Best Practices –

References Service Provider (SP) Issues – Katrina (Kat) Hagedorn CIC-OAI project recommendations for Dublin Core metadata providers – Muriel Foulonneau & Timothy W. Cole Description Guidelines for RLG Cultural Materials (Metadata Recommendations) – Open Archives Forum –

Remaining questions Selective Harvesting issue regarding moving in and out of a set Persistence of a record’s ‘deleted’ status