IST Open Archives Initiative in Europe and Germany Uwe Müller Humboldt University Berlin, Germany Electronic Publishing Group University Library / Computer and Media Service

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Agenda 1.Open Archives 2.The Open Archives Initiative 3.OAForum: European Activities 4.DINI: German Activities

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Open Archives Archive repository of digital information text documents, images, revords, audio/video sequences,... Open Archive provides open machine interface for making content externally available not necessarily: open (= free) usage of metadata (and digital objects) mostly: usage of open standards as exchange protocols

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Value of Open Archives open archive Aopen archive Bopen archive C external service Interface

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Open Archives Different types of archives scholarly publication server (pre-prints, e-prints) libraries (OPAC, e-journals) museum databases (object metadata) archives (historical documents) and cultural heritage education Origin self archiving unclosing existing databases establishing new databases open archives approach gains popularity cross archive access low cost dissemination of previously hidden resources building of new service provision

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Open Archives: Problems lacking interoperability different metadata standards formats (DC, MAB, Marc...) interpretations (Creator: Author vs. Artist vs. Photographer) different terminology different languages different access strategies different interfaces / transfer protocols different copyright regulations Difficulty to establish joint services based on open archives

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" General Search Methods Cross Search Approach (e.g. Z39.50) Harvest Approach (e.g. DIENST protocol) service Request Answer archive service Request Answer archive database 35 2

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Agenda 1.Open Archives 2.The Open Archives Initiative 3.OAForum: European Activities 4.DINI: German Activities

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" The Open Archives Initiative (OAI) Main ideas world-wide consolidation of scholarly archives free access on the archives (at least: metadata) consistent interfaces for archives and service provider effortless implementation based on existing standards (e.g. HTTP, XML, DC) Basic functioning HarvesterRepository Requests (based on HTTP) Metadata (encoded in XML) Metadata (Documents) Metadata Service ProviderData Provider Service

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI: General Assumptions exchange metadata, not digital objects themselves based on harvest approach (asynchronous) two groups of participants Data Providers (Open Archives, Repositories) free access of metadata not necessarily: free access and usage of resources easy to implement, low barriers (useable for small institutions) Service Providers use OAI interfaces of the Data Providers harvest and store metadata may select certain subsets from Data Providers (set hierarchy, date stamp) may enrich metadata offer (value-added) service on the basis of the metadata

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI: Technical Model Service Provider e-print Data Provider e-prints e-print Data Provider Images e-print Data Provider OPAC e-print Data Provider Museum e-print Data Provider Archive Requests: Identify ListMetadataformats ListSets ListIdentifiers ListRecords GetRecord Responses: General information Metadata formats Set structure Record identifier Metadata Data Provider Harvester Repository

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI-Protocol for Metadata Harvesting Basics of OAI-PMH protocol based on HTTP request arguments as GET or POST parameters six request types e.g. responses are encoded in XML syntax supports any metadata format (at least: Dublin Core) Details of OAI-PMH logical set hierarchy (definition: data providers) date stamps (last change of metadata set) error messages flow control

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI-Protocol for Metadata Harvesting Metadata sets (Records) 1.header unique identifier (key for further archive requests), e.g. oai:HUBerlin.de: datestamp, e.g logical sets in which the record is contained 2.metadata metadata prefix (identifier for metadata format) metadata set (at least: Dublin Core, but arbitrary other metadata formats can be transmitted)

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Example: verb=ListIdentifiers&from= &until= & metadataPrefix=oai_dc&set=doctypes:dissertations T17:49:49+01:00 oai:HUBerlin.de: doctypes doctypes:dissertations dnb dnb:dnb33 oai:HUBerlin.de: doctypes doctypes:dissertations dnb dnb:dnb27

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI: Data Provider Architecture SQL- Database OAI Data Provider Web server (e.g. Apache, IIS) OAI request (HTTP request) Programming extension (e.g. PHP, Perl) SQL request DB response OAI response (XML instance)

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI: Service Provider Architecture Data Provider Scheduler Flow control XML Parser Normaliser Database Service module UserHarvesterUser OAI Service Provider Dublication checker Update mechanism

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Problems beyond the OAI-PMH agreement on metadata usage (except DC) semantics XML schema agreement on set definitions selective harvesting e.g. subject gateways definition of rights statements agreement on different right states machine readable information

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI: Examples / Participants Data Provider / Repositories see Service Provider Repository Explorer: Cross Archive Searching Service: MyOAI: DINI: Physnet: …

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Agenda 1.Open Archives 2.The Open Archives Initiative 3.OAForum: European Activities 4.DINI: German Activities

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Background Project Open Archives Forum European Union Information Society Technologies (IST) Programme accompanying measure project start: October 2001 (duration: 2 years) project partners: UKOLN, University of Bath, United Kingdom I.E.I.-CNR, Pisa, Italy Humboldt University Berlin, Germany Motivation increasing discussion about open archives approach setting up a framework for the approach in general promotion of the open archives approach European view...

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Project Partners UKOLN, Bath various projects contexts of metadata and interoperability, cross searching Renardus project, Schemas, DESIRE I.E.I.-CNR, Pisa CYCLADES project, DELOS development of services on top of OAI specification Humboldt University, Berlin Dissertation Online, NDLTD DINI workshops on OAI

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Objectives raise and sharpen awareness on the open archives approach promote low barrier interoperability opening cultural resources detecting potentials for new services encourage collaborative development of solutions discussion of problems exchange of experiences and information support European liaison with OAI … build community of interest

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Objectives (2) Establish an information repository activities related to the open archives approach experiences, developed services (e.g. document delivery, searching, browsing, summarisation, linking) share the database with other organisations (e.g. OAI) Validation of European experience concerning implementing and using the OAI-PMH and other similar approaches requirements from implementers and users Organisational review and analysis possible business models Intellectual Property Rights Dissemination of the open archives approach workshops

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Ask and Answer Questions Not: yet another OAI implementation project … supporting activities European initiatives with open archives based approach clustering activities existing and new communities IST projects national initiatives dissemination activities share experiences on Open Archives investigation of usage: different paradigms global availability share developments

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Benefits tool to reach communities bring together what is happening in Europe raise awareness on and discuss main issues common terminology on digital repositories metadata / full text harvesting models needs of users and communities advanced services make European projects ready for action develop possible solutions establish business models

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Communities institutions of cultural heritage museums European digitising projects scholarly institutions public libraries special user groups publishers commercial sector education Service Provider e-print archives subject gateways with aggregating functions value-added services Data Provider existing metadata repositories new metadata collections OAForum: Participants Carriers and Users of Open Archives

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Topics 1.Workshops, Distribution distribute information and experiences on technologies, etc. produce interest for topics connected with open archives articles, talks, etc. 2.Organisational Evaluation analyse business models tackle issues of copyrights (IPR …) 3.Technical Evaluation apply the technical framework of OAI develop an information portal with information on projects, repositories, service provider metadata schemas software and implementations discuss problems of interoperability

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Workshops, Distribution Contact: I.E.I.-CNR, Pisa – Donatella Castelli May 2002, Pisa, Italy experiences from the European e-prints community establish the forum December 2002, Lisbon, Portugal archives and libraries open access to hidden archives March 2003, Berlin, Germany metadata schemas networking multimedia resources September 2003, Bath, United Kingdom In Practice – Best Practice final workshop, EU review

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Organisational Evaluation Contact: UKOLN, Bath – Philip Hunter Business models co-operations between data providers to establish service networks metadata exchange between archives and services provision of value-added services (e.g. enrichment of metadata, automatic classification, OpenURL) Copyright issues IPR and copyright (influence on producers and distributors) property rights on metadata (collective use of metadata, metadata exchange, agreements with publishers etc.) long-term availability of digital resources Discussion group Dennis Nicholson (University Glasgow)

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAForum: Technical Evaluation Contact: Humboldt University, Berlin – Susanne Dobratz Interoperability integration of new (open archive) approaches with existing technologies Is unqualified Dublin Core sufficient? Issues on database management concurrency and update problems scalability de-dublication Software and tools collect and share experiences and existing solutions estimation of necessary expenditure establish data provider / service … skills, manpower, time Tutorials on OAI technologies online tutorial – will be presented at Bath Workshop

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" OAI Activities in Europe (numbers from November 2002)

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Agenda 1.Open Archives 2.The Open Archives Initiative 3.OAForum: European Activities 4.DINI: German Activities

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" DINI recommendations for usage of OAI-PMH created by DINI-OAI working group target: agreement on syntax and semantics of OAI set definitions for German data and service providers enhance retrieval quality and support subject gateways (e.g. Physnet, Dissertation search engine,...) definition of three classification types subjects (according to DNB) formal publication types (e.g. dissertation) formal document types (e.g. text, audio) example service provider based on recommended sets:

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Classification according to subjects SetSpecSetName dnb:01 Knowledge and Culture in General dnb:02 Books and Libraries, Information and Documentation dnb:03 Reference Books, Bibliographies dnb:04 Directories and Phone Books dnb:05 Calendars dnb:06 Journalism dnb:07 Children's and Youth Literature dnb:08 Comics, Cartoons, Caricatures Miscellanea dnb:09 Esoterica Manuscripts, Book Art dnb:10 Philosophy dnb:11 Psychology dnb:12 Christianity dnb:13 General and Comparative Theology, Non-Christian Religion dnb:14 Sociology, Sociography... dnb:65 Economic History

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Classification according to formal publication types SetSpecSetName pub-type:monograph Books, Monographs pub-type:article Journal Articles pub-type:dissertationDissertations and Professional Dissertations pub-type:masterthesis Diploma Theses pub-type:report Report pub-type:paper Paper pub-type:conf-proceeding Conference Proceedings pub-type:lecture Lectures pub-type:music Music pub-type:program Programs Pub-type:play Play Pub-type:news News Pub-type:standards Standards

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Classification according to formal document types SetSpec SetName doc-type:text Text doc-type:notes Notes doc-type:image Image doc-type:audio Audio doc-type:video Video doc-type:multimedia Multimedia doc-type:data Data doc-type-binary Binary data, (executable) program

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Multiple Data and Service Providers Data providers Service providers Harvesting based on OAI-PMH

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Aggregators – Example: HBZ Köln Data providers Service providers Aggregator

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Hybrid Search – Example: Metalib at HU Data providers Service providers Harvesting based on OAI-PMH Searching based on Z39.50 or SRW

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" A German OAI Example: ProPrint project of SUB Göttingen and CMS of HU Berlin duration: 2000 – 2003 target: integrate heterogenous document servers in order to provide PoD service components: search engine for documents PDF documents preview generation of compound PDF file with front page and table of contents production and delivery by selected print service provider underlying technology: extension of Dublin Core and OAI- PMH (with an extension for document exchange)

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Thank you … Questions? Uwe Müller Humboldt University Berlin, Germany

IST Uwe Müller, : Information Technology and DCMI - "Open Archives Initiative in Europe and Germany" Additional Information Open Archives Initiative (OAI- PMH) (Service Providers) (Data Providers) (Tools,...) Open Archives Forum (workshop, Bath, September 2003) (Technical Validation Questionnaire) DINI (OAI search engine)