1. Designing Protocols in Support of Digital Library Componentization Hussein Suleman and Edward A. Fox Digital Library Research Laboratory Virginia Tech

2. Acknowledgements §Portions of this work were funded in part by the US National Science Foundation through grants DUE-9752190, 9752408, 0121679, 0121741, and 0136690; and IIS-0002935, 0080748 and 0086227. Among these are subcontracts with original funding to UNC Wilmington, U. of Arizona, and U. of Florida. §Portions of this work were funded in part by the Mellon Foundation through a subcontract with original funding to SOLINET for AmericanSouth.org.

3. 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video usersdigital objects ?

4. ? 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video ? digital library Monolithic and/or Custom-built web-based application

5. 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video componentized digital library ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

6. Open Archives Initiative (OAI) §Advocacy for interoperability §Standard for transferring metadata among digital libraries l Protocol for Metadata Harvesting (PMH) Simplicity Generality Extensibility §Support for PMH => Open Archive (OA)

7. 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video open digital library OA PMH XPMH

8. Open Digital Library Protocol Extended OAI-PMH Protocol for Metadata Harvesting

9. Open Digital Library Component Extended OPEN ARCHIVE OPEN ARCHIVE

10. Open Digital Library §Network of Extended Open Archives where each node acts as either a provider of data, services or both. §Component = Node §Protocol = Arc

11. 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 Document 1010100101 0100101010 1001010101 0101010101 ETD-1 1010100101 0100101010 1001010101 0101010101 Program 1010100101 0100101010 1001010101 0101010101 ETD-2 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 Image 1010100101 0100101010 1001010101 0101010101 ETD-3 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 Video 1010100101 0100101010 1001010101 0101010101 ETD-4 ETD Digital Library Search Filter Union Recent Browse PMH ODLRecent ODLBrowse ODLUnion ODLSearch ODLUnion PMH USER INTERFACE Students and researchers ETD collections Example Open Digital Library

12. Prototype - FrontPage

13. Prototype - Search

14. Prototype - Browse

15. ODL Component Requirements §Search l Retrieve a list of items l Index new items §Annotate l Add annotation to item l Retrieve a list of annotations for an item

16. Open Digital Library Components §Running now l XML-File (data provider from file system) l Union, search, browse, recent, filter l E-journal/review, Submit, Edit, Annotation §Class projects l High performance multilingual search l Recommender, Rating; Mirroring (see JCDL’02) l Working with NCSA: from DB, unstructured text §Others discussed l Classification/categorization l DL-Viz interconnection (VIDI – Jun Wang ETD)

17. Harvest from data providers DBUnion Archive Merger Component DBBrowse Browse Engine IRDB-1 Search Engine As Metadata Search Service Provider As Metadata Browse Service Provider XML File Coll. & Data Provider 1 XML File Coll. & Data Provider 2 XML File Coll. & Data Provider 3 Open Digital Library: Extended What’s New Engine As What’s New Service Provider OAI-PMH Data Provider Submit Archive OAIB (NCSA: from RDBMS) Filter Recommend Rate Engine Annotation Engine IRDB-2 Search Engine As Annotation Search Service Provider As Recommend & Rate Service Provider

18. Layer 1 : OAI PMH §Protocol for Metadata Harvesting l Transfer stream of metadata from one archive or component to another §Service Requests l Identify, ListSets, ListMetadataFormats l GetRecord, ListIdentifiers, ListRecords

19. Layer 2 : Extended OAI-PMH §OAI-PMH + extensions for general-purpose inter-component communication l Added in generic containers in every response for additional information l Added “PutRecord” to submit a record l Increased granularity to support times as well as dates (same as OAI-PMH v2.0) l Ignored DC requirement

20. Layer 3 : ODL Protocols §Specialized protocol semantics for different components, e.g.: l Search component uses ODLSearch protocol ListRecords and ListIdentifiers embed query terms in “set” parameter l Annotation component uses ODLAnnotate protocol ListRecords and ListIdentifiers specify the item for which annotations are requested in the “set” parameter PutRecord adds an annotation to an item

21. Case Study: ETD ODL Prototype §Electronic Thesis and Dissertation Open Digital Library

22. Case Study: CSTC §Computer Science Teaching Center

23. CSTC User Interface

24. Performance Optimizations §Caching of responses §Persistent CGI mechanisms l FastCGI l SpeedyCGI §Request multiple records in a single operation (proposed)

25. What have we accomplished ? §Complete protocol-level separation among components within the DL §Seamless integration with little “glue” §Simple extensions of OAI-PMH §Modular and portable components §Efficient in speed - not as efficient in storage

26. Projects Using ODL §NDLTD (www.ndltd.org) l Union Catalog for Electronic Theses and Dissertations – prototype ODL site §Computer Science Teaching Center (www.cstc.org, part of www.citidel.org) l Digital library of peer-reviewed teaching resources in the computing sciences §AmericanSouth.org l Portal to meta-collection of resources related to Southern History and Culture

27. (Somewhat) Open Issues §Is this scalable? Portable ? Extensible ? §Can we define all popular DL services using such a methodology? (completeness problem) §Can we define DLs as configurations of ODL components? (composition problem) §Is OAI-PMH a good baseline protocol ? Can we design a better baseline protocol upon which to base harvesting and repository access? §To what degree is an ODL network equivalent to a monolithic system? (comparison problem)

28. Ultimate Goal §Package different configurations into instant DL systems or subsystems §DL building = component configuration §All DLs speak the same language(s) §Basic services are trivial to provide so more effort is spent on advanced capabilities of DLs

29. Discussion? §Questions? §How can we extend this discussion, in context of the OCKHAM effort? §Will you please join: l dl-in-a-box-l@listserv.vt.edu l dl-log-l@listserv.vt.edu l OCKHAM-SYS@listserv.cc.emory.edu §Will you add to http://oai.dlib.vt.edu/odl/ ?