BASE: Institutional Repositories Bielefeld Academic Search Engine (BASE): an End-user Oriented Institutional Repository Search Service Dirk Pieper/Friedrich Summann Bielefeld UL

BASE: Institutional Repositories Part 1: Institutional Repository Servers BASE: concept and content Creating a special view on institutional repository server collections Demo: BASE user-interface and further visions Part 2: OAI dataflow, BASE dataflow Repository information in registries OAI harvesting problems Further developments of BASE Overview:

BASE: Institutional Repositories Definition: “A digital collection capturing and preserving the intellectual output of a single or multi-university community.” (Raym Crow, IR servers exist of course also outside the university community IR servers appear as simple web sites, database systems with OAI interface, … Institutional Repository Servers:

BASE: Institutional Repositories BASE uses Fast Data Search BASE contains intellectual selected resources with focus on OAI-Servers but also web crawled content BASE displays result lists as bibliographic data and full text hits BASE frontend is written in PHP using the search API from Fast Data Search BASE offers sorting, search refinement and search history BASE: concept and content

BASE: Institutional Repositories Search API Pipeline QUERY & RESULT PROCESSING DOCUMENT PROCESSING Pipeline FILE TRAVERSER FILTER SEARCH INDEX FILES CONNECTORS TUNING, ADMINISTRATION and DEBUGGING WEB CRAWLER BASE: concept and content

BASE: Institutional Repositories BASE: concept and content At present 2,7 mio documents in 189 collections, 15 of them web crawled data

BASE: Institutional Repositories Projekt Gutenberg-DE Internet Library of Early Journals Oxford Various Institutional Repositories Springer Link Metadata Cornell HistMath Fulltext Crawl University Michigan Historical Math CiteSeerZentralblatt Mathematik Bielefeld Univ: Math. Preprints ArXivOPAC UL Bielefeld Ifo Institute Munich Zeitschriften der Aufklärung (Bielefeld UL) BASE: concept and content

BASE: Institutional Repositories Special view on IR server collections Collections are listed in configuration file [ftubirmingham] url = " desc_de = "The Univ. of Birmingham: Eprints Archive" desc_en = "The Univ. of Birmingham: Eprints Archive" descdd_de = "Birmingham Univ." descdd_en = "Birmingham Univ."  Collections can be clustered for user-interface, e.g. “Institutional Repositories Europe” consists of [ftubarcelona], [ftubath], [ftubristol], [ftuhelsinki], …  Parametric search possible  Frontend is ready for multi view (independent views with own configuration and layouts on the same backend)

BASE: Institutional Repositories Try your search on Google Scholar... Vision: search in Google Scholar

BASE: Institutional Repositories Check citations (citing articles) in Google Scholar... Vision: check citations in Google Scholar

BASE: Institutional Repositories OAI-Data Harvesting BASE Internal Index (FAST) OPAC Article Database Dissertations, monographs (fulltext) Articles (fulltext) PubMed, Euclid, ArXiv, CiteSeer, Citebase, DOAJ articles All ressources (texts, images, video,references.... OAI dataflow at Bielefeld UL

BASE: Institutional Repositories OAI-Data Web Pages Database Records Harvesting Pre-Processing Processing Internal Index (FAST) User interface (PHP) BASE dataflow

BASE: Institutional Repositories  Eprints Registry (607)  Openarchives.org (383)  DSpace Registry (28)  Directory of Open Archive Repositories (324)  Univ. of Illinois Registry (1000) Repository information in registries

BASE: Institutional Repositories USA 76 Canada 13 South America 2 Africa 2 India 3 Australia 11 New Zealand 1 3 OAI-compliant univ. repositories in BASE

BASE: Institutional Repositories  OAI Registry Watcher (Bielefeld UL, Perl)  Open Source Harvester (FS Consulting, Perl with modifications)  XML Validator and Repairer (Bielefeld UL, based on Perl XML modules  OAI Harvest Watcher (Bielefeld UL, Perl)  OAI Resource Updater (Bielefeld UL, Perl) Tools for the Harvesting Environment

BASE: Institutional Repositories  Repositories do not response or deliver Error Messages  Data contain only References without any Fulltext  Links to the Document do not work  Access to fulltext is restricted  XML file is not well-formed  Field content varies OAI harvesting challenges

BASE: Institutional Repositories ELibD905_diplom_allnoch.pdf Barry Wellman,Jeffrey Boase,Kakuko Miyata Barry Wellman,Jeffrey Boase,Kakuko Miyata The Mobile-izing.... Talk P. Bruzzone Bruzzone Pierluigi Reproductive Biology and Endocrinology 2004, 2:52 doi: / Review OAI Harvesting: Problems in Practice 1

BASE: Institutional Repositories EN: 9910 ENG: 771 En: 566 Eng: 1 English: English (United States): 63 English and Greek: 1 English and Russian: 1 English/Japanese: 1 English; Russian: 1 English=en: 1 Translation into English: 2 en: en-CA: 865 en-US: 3 en-es: 5 en-us: 8 en;: 2 en_UK: 618 en_US: eng: eng : 92 eng + dut: 2 eng;: 17 eng; fre; ger;: OAI Harvesting: Problems in Practice 2 - Variations of

BASE: Institutional Repositories  Standard repository software is great - for OAI harvesting as well  Small collections – small problems  Getting the related fulltext is complicated  Libraries produce better metadata  Data aggregation may produce problems  Writing s helps - sometimes Some Rules from Harvesting Practice

BASE: Institutional Repositories  Search form (working)  HTTP calls (working)  Web Service (in development)  Federated Search (Vascoda) (in discussion) Further Developments: BASE Interfaces

BASE: Institutional Repositories <form action=" method="post" accept-charset="UTF-8"> Local Integration: Search Form

BASE: Institutional Repositories Thank you!