1. Search Interoperability, OAI, and Metadata An Introduction to the OAI Protocol for Metadata Harvesting Sarah Shreeves University of Illinois at Urbana-Champaign November 30, 2006 This work is licensed under the Creative Commons Attribution- NonCommercial-ShareAlike 2.5 License.

2. November 30 2006 Scenario: An undergraduate is writing a paper comparing immigration in the early 20 th century to immigration now and has to include a variety of primary sources

3. November 30 2006 IMLS funded digital collections with relevant content The problem: The user has to access each collection individually. Wastes time and makes it harder to get work done. A partial solution: The OAI Protocol for Metadata Harvesting provides a relatively low barrier means for integrated access to the metadata describing items in these collections.

4. November 30 2006 Outline  Search interoperability basics  What the OAI protocol is & what it is not  Examples of OAI enabled services  How it works (basically)  Challenges for data / service providers

5. November 30 2006 Search interoperability “the ability to perform a search over diverse sets of metadata records and obtain meaningful results.” – Priscilla Caplan Metadata Fundamentals for All Librarians

6. November 30 2006 Keys to Search Interoperability  Communication protocol (Z39.50, OAI, etc.)  Organizational commitment  Standards  And more Standards

7. November 30 2006 Sharing metadata: Federated search The distributed databases are searched directly. Mill? My resource 04 For Example: Z39.50, SRU/SRW

8. November 30 2006 Sharing metadata: Data aggregation The user searches a pre-aggregated database of metadata from diverse sources. Mill? My resource 04 For Example: Search engines, union catalogs, OAI

9. November 30 2006 Why share metadata?  Benefits to users One-stop searching Aggregation of subject-specific resources  Benefits to institutions Increased exposure for collections Broader user base Bringing together of distributed collections Don’t expect users will know about your collection and remember to visit it.

10. November 30 2006 Examples of OAI Service Providers  OAIster: http://oaister.umdl.umich.edu/o/oaister/ http://oaister.umdl.umich.edu/o/oaister/  Engineering, Computer Science, and Physics: http://g118.grainger.uiuc.edu/engroai/ http://g118.grainger.uiuc.edu/engroai/  CIC Metadata Portal: http://nergal.grainger.uiuc.edu/cgi/b/bib/oaister http://nergal.grainger.uiuc.edu/cgi/b/bib/oaister  IMLS Digital Collections and Content: http://imlsdcc.grainger.uiuc.edu/ http://imlsdcc.grainger.uiuc.edu/

11. November 30 2006 The OAI-PMH is a tool  Moves metadata (not content for the most part yet) from a data provider to a service provider (or harvester)  A set of rules that defines the communication between two systems (like FTP and HTTP)  Facilitates the aggregation of metadata (like a union catalog)  Developed in 2001 out of the eprint/pre-print community

12. November 30 2006 Basic OAI-PMH Concepts  “Aggregated search” rather than “Federated search”  OAI-PMH based upon HTTP and XML  Data providers – support OAI PMH as a means to expose metadata  Service providers – ‘harvests’ metadata from data providers via the OAI-PMH  OAI-PMH requires use of simple Dublin Core BUT supports and encourages use of other metadata schemas

13. November 30 2006 Sample OAI Request

14. November 30 2006 OAI-PMH is not…. Metadata A search tool A database Open Access

15. November 30 2006 UIUC Library and OAI  Early testers of the protocol in 2000 and 2001  Received Mellon grant in 2001 in first wave of establishing the protocol and have since received several grants to build OAI aggregations  Currently have data providers for CONTENTdm, IDEALS, Archives, Aerial Photographs, and others.  Have been active in the continued development of the protocol and associated activities since Static repository development Best practices for OAI which led to an IMLS training grant Best practices for OAI Implementation Guidelines for Shareable MODS Records  Will be working on the next initiative out of the OAI: ORE (Object Reuse and Exchange): http://www.openarchives.org/ore/ http://www.openarchives.org/ore/  Tim Cole and Muriel Foulonneau currently working on a book on OAI

16. November 30 2006 Metadata challenge “the ability to perform a search over diverse sets of metadata records and obtain meaningful results.” – Priscilla Caplan Metadata Fundamentals for All Librarians

17. November 30 2006 OAI ≠ Dublin Core DC is OAI’s lowest common denominator BUT OAI supports & encourages use of other community-driven metadata schemas

18. November 30 2006 Metadata Interoperability  Semantics What is the metadata format used? Mapping from one format to another  Content rules How are values for the metadata elements selected and represented?  Syntax How are the metadata elements encoded in machine readable form?  Documentation

19. November 30 2006 What does this record describe? identifier: http://name.university.edu/IC-FISH3IC- X0802]1004_112 publisher: Museum of Zoology, Fish Field Notes format:jpeg rights: These pages may be freely searched and displayed. Permission must be received for subsequent distribution in print or electronically. type:image subject: 1926-05-18; 1926; 0812; 18; Trib. to Sixteen Cr. Trib. Pine River, Manistee R.; JAM26-460; 05; 1926/05/18; R10W; S26; S27; T21N language: UND source: Michigan 1926 Metzelaar, 1926--1926; description: Flora and Fauna of the Great Lakes Region Dublin Core record retrieved via the OAI Protocol

20. November 30 2006

21. How about this one? title: (Woman Holding a Pie) LNG42122.5 subject: Berkeley; male; outdoors; yard; stair subject: Dorothea Lange Collection subject: The War Years (1942-1944) subject: Office of War Information (OWI) subject: Woman Holding a Pie publisher: Museum of [state] date: 1944 type: image identifier:http://www.orgname.org/idnumber relation: http://orgname.org/findaid/idnumber relation: id:/13030/tf9779p783 relation: http://www.orgname.org/ relation: http://findaid.org.org/findaid/... relation: http://www.orgname.edu/project/ Dublin Core record harvested via OAI

22. November 30 2006

24. Metadata for different communities

25. November 30 2006 Metadata for different communities

26. November 30 2006 Loss of Context: Record in OAI aggregation

27. November 30 2006 Context: Record in native database

28. November 30 2006 Loss of context / data

29. November 30 2006 Loss of context / data

30. November 30 2006 Granularity of Description: Excerpt of Metadata Record Describing “American Woven Coverlet”

31. November 30 2006 Granularity of Description: Excerpt of Metadata Record Describing "Cotton coverlet with embroidered butterfly design"

32. November 30 2006 Collection Registries ????? GEM Photograph from Indiana University Charles W. Cushman Collection

33. November 30 2006 Shareable metadata defined  Promotes search interoperability - “the ability to perform a search over diverse sets of metadata records and obtain meaningful results” (Priscilla Caplan)  Is human understandable outside of its local context  Is useful outside of its local context  Preferably is machine processable

34. November 30 2006 Recap  OAI protocol is a tool  OAI is easy - metadata is hard  Better metadata = better interoperability

35. November 30 2006 Sarah Shreeves Coordinator, IDEALS University of Illinois Library at Urbana-Champaign Email: sshreeve@uiuc.edusshreeve@uiuc.edu Phone: 217-244-3877 This work is licensed under the Creative Commons Attribution- NonCommercial-ShareAlike 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Contact Information