1. ComPADRE Experiences developing an OAI server over an existing database repository Resources for Physics and Astronomy Education Lyle Barbato American Association of Physics Teachers lbarbato@aapt.org

2. The ComPADRE Project NSF/NSDL funded repository for Physics and Astronomy specific resources Partnerships and collaborations – AAPT: American Association of Physics Teachers – AAS: American Astronomical Society – APS: American Physical Society – AIP/SPS: Society of Physics Students – Merlot: Multimedia Educational Resource for Learning and Online Teaching Multiple collection interfaces for different target audiences Collection interfaces share a common architecture so we can build once and use everywhere

3. Why OAI? Fits well as a shared service Necessary step to becoming a service provider for physics and astronomy data providers NSDL harvesting Increased content visibility Theoretically, it has a ‘simple’ and ‘easy’ implementation

4. Our Backend Structure Database driven – Our records are generated on-the-fly Database model based on the Learning Object Model (LOM) metadata standard

5. OAI Implementation Experience Research of OAI tools and protocol OAI Server Implementation and corresponding database changes Implementation of metadata formats – OAI_DC A few hours – NSDL_DC A few weeks

6. OAI Server Implementation Only 6 Verbs! – But opportunities still exist for problems and confusion Focusing on implementation problems, they were either: – Related to our database infrastructure Datestamps for exported ‘on the fly’ records General deletion handling – Or related to supporting shared item records for multiple collections with separate workflows

7. Datestamp Issues OAI requires datestamps be applied on the metadata record level. This raises issues with database-based items: – When exports are done on the fly, item level last modified datestamps are not good enough since fields that change in one format may not be included in another – When a new metadata format is created, it will have a different OAI datestamp for creation, however the database record will already exist

8. Deleted Record Issues The goal is for ‘persistent’ deleted records However: – The datestamp for a deleted record must be the time it was deleted Thus we could not use the ‘Last Modified’ date of our records due to potential future database-wide global vocabulary updates. – Metadata versioning It happens. Plan to consistently export all metadata format versions forever Or plan to have ‘deleted’ record holders for existing records at the time of the metadata format removal

9. Additional recommendations For Service Providers – To avoid loss of context and information: Place set info into the record using IsPartOf to alleviate “On the Horse” problems If possible, Indicate the set size in the setDescription – Provide internal unique ids for de-duping – Indicate source of metadata – Expose the richest metadata format possible

10. Provide Richer Metadata – How? Expose the richest metadata format possible – However, the OAI literature has little on how to create your own xml schema if you need to: http://www.oaforum.org has a great tutorial on how to create a new schema by extending the oai_dc schema to add new elements at http://www.oaforum.org/tutorial/english/page5.htm#section1 http://www.oaforum.org http://www.oaforum.org/tutorial/english/page5.htm#section1 – Also, if you have an internal vocabulary that you wish to expose, a page that gives instructions for creating an xml schema ‘vocabulary’ via enumeration: http://www.xml.com/pub/a/2003/02/05/wxs-enum.html

11. When it’s all said and done There are tools which can do this for you BUT If your infrastructure doesn’t mesh with existing tools, OAI is simple enough to implement yourself.

12. References Compadre – http://www.compadre.org/portal http://www.compadre.org/portal Open Archives Initiative – http://www.openarchives.org/ http://www.openarchives.org/ Digital Library Federation OAI and Shareable Metadata Best Practices Working Group – http://oai-best.comm.nsdl.org/cgi-bin/wiki.pl http://oai-best.comm.nsdl.org/cgi-bin/wiki.pl NSDL Metadata Resources Page – http://metamanagement.comm.nsdlib.org/IntroPage.html http://metamanagement.comm.nsdlib.org/IntroPage.html Dublin Core Metadata Initiative – http://dublincore.org http://dublincore.org

13. References The Basics of OAI: An introduction to the Protocol for Metadata Harvesting – Web-Wise 2004: Sharing Digital Resources Tim Cole, Sarah Shreeves, Martin Halbert http://imlsdcc.grainger.uiuc.edu/OAI_Tutorial_WebWise.ppt Bitter Harvest: Problems & Suggested Solutions for OAI-PMH Data & Service Providers – Roy Tennant California Digital Library http://www.cdlib.org/inside/projects/harvesting/bitter_harvest.html Challenges for Service Providers When Importing Metadata in Digital Libraries – Marilyn McClelland, David McArthur, Sarah Giersch, Gary Geisler http://www.dlib.org/dlib/april02/mcclelland/04mcclelland.html

14. References JCDL OAI Workshop 2003 – Naomi Dushay http://www.cs.cornell.edu/people/simeon/workshops/JCDL2003/JCDL2003 _OAI_Workshop_Naomi_Dushay.ppt Improving Metadata Quality: Augmentation and Recombination – Naomi Dushay, Diane Hillmann, Jon Phipps http://metamanagement.comm.nsdl.org/Metadata_Augmentation-- DC2004.pdf DLF Best Practices for OAI Data Providers and for Sharable Metadata - Proposal for a Workplan – Sarah Shreeves http://imlsdcc.grainger.uiuc.edu/DLF_OAI/DLF_OAI_Workplan.pdf Set Best Practices – http://oai-best.comm.nsdl.org/cgi-bin/wiki.pl?SetPractices http://oai-best.comm.nsdl.org/cgi-bin/wiki.pl?SetPractices

15. References Service Provider (SP) Issues – Katrina (Kat) Hagedorn http://www.kathagedorn.com/SP_issues.html CIC-OAI project recommendations for Dublin Core metadata providers – Muriel Foulonneau & Timothy W. Cole http://cicharvest.grainger.uiuc.edu/dcguidelines.asp Description Guidelines for RLG Cultural Materials (Metadata Recommendations) – http://www.rlg.org/en/page.php?Page_ID=214 http://www.rlg.org/en/page.php?Page_ID=214 Open Archives Forum – http://www.oaforum.org/ http://www.oaforum.org/

16. Remaining questions Selective Harvesting issue regarding moving in and out of a set Persistence of a record’s ‘deleted’ status