DEF System Architecture XML Web Services Fedora and the Zebra Search Engine in an OAI Eprints Application by Gert Schmeltz Pedersen, DTV

2 Contents 1. XML Web Services and the 3-tier Architecture 2. The DEF Eprints Service 3. DEF-XWS Eprints 4. Generic Search Service 5. Repository Federation

3 DEF-XWS project suite "XML Web Services and the 3-tier architecture" a project suite within the programme area System Architecture at Denmark's Electronic Research Library (DEFF) ( a collaboration with The Royal Library, The State and University Library, Aarhus Business School Library a.o. Get web services hands-on and experience. Get Fedora hands-on and experience. Use Fedora to implement a web service version of DEF Eprints - International eprints metadata harvested from Open Archives, a DEF project carried out at DTV. Add full text indexing and retrieval.

4 Data base Common service Local Portal Common service Local service Common service Central Portal Web browser DEff 3-tier Service Oriented Architecture

5 The DEF Eprints Service Architecture of the DEF Eprints Service Provider OAI Manager Full set Sub set Librarian DEF Portal User OAI Harvester Open Archives Initiative Data Providers MYSQLMYSQL Z39.50 OAI-PMH Eprint Service Provider Zebra server Web UI w/Z39.50 InfoNet User Zebra server Web UI w/Z39.50 EXPORTEXPORT Protocol for Metadata Harvesting

6 DEF-XWS Eprints OAI Manager Full set Sub set Librarian DEF Portal User OAI Harvester Open Archives Initiative Data Providers MYSQLMYSQL Z39.50 OAI-PMH Eprint Service Provider Zebra server Web UI w/Z39.50 InfoNet User Zebra server Web UI w/Z39.50 EXPORTEXPORT Fedora server Zebra server Full text retrieval Batch ingest EXPORTEXPORT AppXYZ User DEF-XWS Eprints User SOAP/REST Web UI w/SOAP java Web UI w/REST php AppXYZ w/SOAP perl

7 DEF-XWS Eprints ZebraForFedora, a module for Fedora ( Purpose: to obtain powerful text index and search functionality and performance. The original text index and search functionality in Fedora is simple SQL on a table, where DC element texts are stored in fields. ZebraForFedora is a set of Java classes that deploys over existing Fedora and Zebra installations by the running of an Ant target. In the Fedora configuration file: <module role="fedora.server.search.FieldSearch" class="dk.defxws.eprints.fedora.server.search.FieldSearchZebraModule"> Instead of fedora.server.search.FieldSearchSQLModule Zebra server

8 DEF-XWS Eprints

9 Purpose achieved Fedora hands-on and experience web services hands-on and experience DEF-XWS Eprints available from web services ready for 3-layered system architecture applications combining many web services Lesson Do not override field search, provide generic search service instead... DEF-XWS Eprints

10 Generic Search Service Core Fedora Repository Service new services are deployed as web applications (.war files), with a configuration file. The Generic Search Service shall be a webapp, configurable to use an existing Fedora repository and an existing installation of an indexing and searching engine, like Zebra, Lucene, and others. Functionality to be decided by a working group of Fedora users and developers. Generic Lucene Zebra...

11 Generic Search Service preliminary analysis of what has been done by others already, approaches and issues people have taken in the following areas a. what kinds of search engines? b. how is indexing done and how is it kept up to date? c. configuration options? How can you specify what datastreams/disseminations to index? d. what interfaces for doing searches? e. how do you deal with security in terms of the service interacting with Fedora? f. what are problems with current approaches? g. what would be desirable in a generic search service that would be delivered with Fedora? gathering of requirements and issues for moving towards a reference implementation - ZebraForFedora may serve as a reference implementation from a broader perspective, how to deal with search for federations of repositories - P2P search in EU project Alvis may be relevant things that the Fedora Dev Team might need to do for new services in the Framework: - a notification/messaging module in the core Fedora repository service so that other services can find out when objects are added or changed. - how the services run securely with Fedora, a Basic Auth approach is used now

12 Repository Federation Idea under elaboration: Fedora as Superpeer in an ALVIS peer-to-peer system

13 DEF-XWS Thank you!

14 future DEF-XWS Pilot Web Service-Oriented Architecture Graphics from Web Services: A Manager's Guide, by Anne Thomas Manes, Addison-Wesley, 2003 DEF-XWS Pilot Java Eprint WS php Test UI Java Test UI Simple Object Access Protocol or REST Representational State Transfe r Web Services Description Language DEF-XWS Eprints