Open Scholarship 2006 Bielefeld Academic Search Engine a Scientific Search Service for Institutional Repositories Open Scholarship 2006 New Challenges for Open Access Repositories Univ. of Glasgow, October 2006 Friedrich Summann Bielefeld University Library

Open Scholarship 2006 BASE: concept and content Overview BASE user-interface and further visions BASE dataflow OAI harvesting challenges BASE interfaces Demo Overview:

Open Scholarship 2006 BASE uses Fast Data Search BASE uses Linux-based multi-node system BASE contains intellectual selected resources with focus on OAI Servers but also web crawled content BASE displays result lists as bibliographic data and full text hits BASE frontend is written in PHP using the search API from Fast Data Search BASE offers sorting, search refinement and search history BASE: concept and content

Open Scholarship 2006 Search API Pipeline QUERY & RESULT PROCESSING DOCUMENT PROCESSING Pipeline FILE TRAVERSER FILTER SEARCH INDEX FILES CONNECTORS TUNING, ADMINISTRATION and DEBUGGING WEB CRAWLER BASE: concept and content

Open Scholarship 2006 BASE: concept and content At present 3.8 mio documents in 274 collections, 15 of them web crawled data

Open Scholarship 2006 Projekt Gutenberg-DE Internet Library of Early Journals Oxford Various Institutional Repositories Springer Link Metadata Cornell HistMath Fulltext Crawl University Michigan Historical Math CiteSeerZentralblatt Mathematik Bielefeld Univ: Math. Preprints ArXivOPAC UL Bielefeld Ifo Institute Munich PubMedJournals of Enlightment (Digital Collection of Bielefeld UL) BASE: concept and content

Open Scholarship 2006 Special view on IR server collections Collections are listed in configuration file [ftubirmingham] url = " desc_de = "The Univ. of Birmingham: Eprints Archive" desc_en = "The Univ. of Birmingham: Eprints Archive" descdd_de = "Birmingham Univ." descdd_en = "Birmingham Univ." Collections can be clustered for user-interface, e.g. Institutional Repositories Europe consists of [ftubarcelona], [ftubath], [ftubristol], [ftuhelsinki], … Parametric search possible Frontend is ready for multi view (independent views with own configuration and layouts on the same backend)

Open Scholarship 2006 BASE: end-user interface (1) Displays search results as bibliographic data and full text hits

Open Scholarship 2006 BASE: end-user interface (2) The result list (left hand side) If the document contains meta data (e.g. title, author, abstract) the displayed description is highlighted

Open Scholarship 2006 BASE: end-user interface (3) Various options to sort the result set Search refinement by author, keyword, document type, language etc. Search history comprises up to 10 queries The result list (right hand side)

Open Scholarship 2006 BASE: end-user interface (4) Search Refinement Select an author only documents by this author are displayed

Open Scholarship 2006 Check citations (citing articles) in Google Scholar... Google Scholar integration

Open Scholarship 2006 Vision: DDC Browsing

Open Scholarship 2006 OAI-Data Web Pages Database Records Harvesting Pre-Processing Processing Internal Index (FAST) User interface (PHP) BASE dataflow

Open Scholarship USA 82 Canada 14 South America 2 Africa 3 India 5 Australia 11 New Zealand 1 OAI-compliant university repositories in BASE 3 1 1

Open Scholarship 2006 OAI harvesting challenges Repositories do not response or deliver Error Messages Data contain only References without any Fulltext Links to the Document are not included or do not work Access to fulltext often is restricted XML file is not well-formed Field content varies

Open Scholarship 2006 Some Rules from the Harvesting Practice Standard repository software is great - for OAI harvesting as well Small collections – small problems Getting the related fulltext is complicated Libraries produce better metadata Data aggregation may produce problems Writing s helps - sometimes

Open Scholarship 2006 BASE interfaces Search form HTTP calls Web Service

Open Scholarship 2006 <form action=" method="post" accept-charset="UTF-8"> Local integration (via search form) E-Repository Integration

Open Scholarship 2006 Prototype: Search Based on SOAP interface (EU project DRIVER)

Open Scholarship 2006 Thank you!