Assessing a human mediated current awareness service International Symposium of Information Science (ISI 2015) Zadar, 2015-05-20 Zeljko Carevic 1, Thomas.

Slides:



Advertisements
Similar presentations
ELIBRARY CURRICULUM EDITION The ultimate K-12 curriculum and reference solution.
Advertisements

Distributed Current Awareness Services Thomas Krichel
Rclis in vision and reality Thomas Krichel
Current Awareness in a Large Digital Library José Manuel Barrueco Cruz Thomas Krichel Jeremiah Trinidad.
Use your bean. Count it. Thomas Krichel
Four slides for the future Thomas Krichel given at 4 th International Socionet seminar Novosibirsk
Jane Long, MA, MLIS Reference Services Librarian Al Harris Library.
Slides will automatically advance Back to Online demo Welcome to the Safety Insite. com.
How to Read a Scientific Research Paper : an overview Asst.Prof.K.Chinnasarn, Ph.D.
JINR / CERN Grid and advanced information systems 2012 Anne Gentil-Beccot CERN Library GS/SIS The Library behind the scene Opportunities for Scientific.
Elibrary.worldbank.org World Bank eLibrary User Guide Take full advantage of your eLibrary subscription!
Exploring the Academic Invisible Web Das wissenschaftliche Invisible Web erkunden Dr. Dirk Lewandowski Heinrich-Heine-Universität Düsseldorf, Information.
Engineering Village ™ ® Basic Searching On Compendex ®
Best Web Directories and Search Engines Order Out of Chaos on the World Wide Web.
285 Final Project. Document Specification: Rough Draft Due April 10th Purpose: Purpose: Economy of effort Economy of effort Input from instructors and.
Reference Collections: Task Characteristics. TREC Collection Text REtrieval Conference (TREC) –sponsored by NIST and DARPA (1992-?) Comparing approaches.
How to find a worthy research subject Sadeghi Ramin, MD Nuclear Medicine Research Center, Mashhad University of Medical Sciences.
Swets Information Services SwetsWise Title Bank 13 th Panhellenic Libraries Conference th October Corfu.
1 Using Scopus for Literature Research. 2 Why Scopus?  A comprehensive abstract and citation database of peer- reviewed literature and quality web sources.
1 CS 430: Information Discovery Lecture 2 Introduction to Text Based Information Retrieval.
What is so good about Archie and RevMan 5
KNOWLEDGE FOR LIFE Leisure Tourism Database CABI product training Tom Corser.
Online Resources From Oxford University Press This presentation gives a brief description of Oxford Journals. It tells you: what the journals are; how.
Introduction to Current Contents Connect. What is CCC? A multidisciplinary current awareness resource –Browse and search journals, books and websites.
0 1 Presented by MANSOUREH SERATI Faculty Member of Islamic World Science Citation Center (ISC) shiraz, Iran.
Are downloads and readership data a substitute for citations? The case of a scholarly journal? Christian Schlögl Institute of Information Science and Information.
Title of the Poster. “Digital library services and their impact with reference to a developing country: The case of the Faculty of Health Sciences library,
LIS618 lecture 4 before searching + introduction to dialog Thomas Krichel
Springerlink.com Introduction to SpringerLink springerlink.com.
Research evaluation requirements José Manuel Barrueco Universitat de València (SPAIN) Servei de Biblioteques i Documentació May, 2011.
1 Scopus as a Research Tool March Why Scopus?  A comprehensive abstract and citation database of peer-reviewed literature and quality web sources.
Jacqueline A. Gill, Associate Professor Slides will change automatically or you may click the screen to move forwards.
1 ScopusScopus Empowering Your Research. 2 As a Comprehensive Abstracts Database ~18,000 sources (90% peer-reviewed journals) from 5,000 publishers Comprehensive.
NC LIVE Medical & Health Resources Consumer Health Christie Silbajoris UNC-CH Health Sciences Library March 2010.
University of Antwerp Library TEW & HI UA library offers... books, journals, internet catalogue -UA catalogue, e-info catalogue databases -e.g.
Karen Herter (HMG) Mike Langley (DGS) April 15, 2008 Portfolio Manager for California State Buildings Meeting the Requirements of Executive Order S
1 Information Retrieval Acknowledgements: Dr Mounia Lalmas (QMW) Dr Joemon Jose (Glasgow)
25/10/20151Gianluca Demartini Desktop Search Evaluation Sergey Chernov and Gianluca Demartini TREC 2006, 16th November 2006 Pre-Track Workshop.
The ISI Web of Knowledge nce/training/wok/#tab3.
Web Image Retrieval Re-Ranking with Relevance Model Wei-Hao Lin, Rong Jin, Alexander Hauptmann Language Technologies Institute School of Computer Science.
LOGO A comparison of two web-based document management systems ShaoxinYu Columbia University March 31, 2009.
EconLit Using indexes University Library click = next.
1 Automatic indexing Salton: When the assignment of content identifiers is carried out with the aid of modern computing equipment the operation becomes.
Information Retrieval
Advantages of Query Biased Summaries in Information Retrieval by A. Tombros and M. Sanderson Presenters: Omer Erdil Albayrak Bilge Koroglu.
 Service Learning: Research Paper Rough Draft Steps to Starting a Google Doc.
1 e-Resources on Social Sciences: Scopus. 2 Why Scopus?  A comprehensive abstract and citation database of peer-reviewed literature and quality web sources.
Alma Analytics Usage Yoel Kortick | Senior Librarian.
CitEc as a source for research assessment and evaluation José Manuel Barrueco Universitat de València (SPAIN) May, й Международной научно-практической.
Major Issues n Information is mostly online n Information is increasing available in full-text (full-content) n There is an explosion in the amount of.
Databases- presentation and training
TJTS505: Master's Thesis Seminar
journal metrics university of sulaimani college of science geology dep by Hawber Ata
Electronic Services at the Central Library.
CS 430: Information Discovery
Journals online via Wiley InterScience
The RePEc database about Economics
Building an autonomous citation index for grey literature: the
Introduction of KNS55 Platform
Searching for print and electronic books
Searching for books and electronic books
Discovery – Using Limiters to Refine Your Search
5. Setting up Alerts.
Searching for print and electronic books
USER MANUAL - WORLDSCINET
This presentation document has been prepared by Vault Intelligence Limited (“Vault") and is intended for off line demonstration, presentation and educational.
This presentation document has been prepared by Vault Intelligence Limited (“Vault") and is intended for off line demonstration, presentation and educational.
USER MANUAL - WORLDSCINET
Presentation transcript:

Assessing a human mediated current awareness service International Symposium of Information Science (ISI 2015) Zadar, Zeljko Carevic 1, Thomas Krichel 2 and Philipp Mayr

Outline 1.Introduction 2.RePEc and NEP 3.Results 3.1 Editing time 3.2 Indicators for report success 3.3 Editing effort 4.Conclusion and Outlook Slide 2 / 31

Motivation Thomas Krichel, the founder of RePEc, visited GESIS – Cologne in Oct Sharing his Russian souvenir ~100 GB of XML log files Slide 3 / 31

1. Introduction Current awareness in digital libraries –To inform users / subscribers about new / relevant acquisitions in their libraries [1]. Current awareness services allow subscribers to keep up to date with new additions in a certain area of research. Selection of relevant documents can be done (semi- )automatically or manually. For this work we focus on the intellectual editing process Aim of this work: How do editors work when creating a subject specific report in Digital Libraries (DL)? Slide 4 / 31

2. Use case: RePEc RePEc (Research Papers in Economics) is a DL for working papers in economics research. Covers metadata for working papers and journal articles. Usually document metadata contains links to full texts Slide 5 / 31

2. RePEc statistics Contr. ArchivesDocumentsFull text Documents Regist. AuthorsAbstract views (April 2015) ~1, mio1.63 mio~45,000>2 mio Slide 6 / 31

2. Current awareness service NEP NEP (New Economics Papers) is a current awareness service for new additions in RePEc. NEP covers subject specific reports from over 90 specific fields. –Business, Economic and Financial History –Public Economics –Social Norms and Social Capital Issues are sent to subscribers via , RSS and Twitter Reports to new additions are generated by subject specific editors. Relevant document selection is done manually by the editor! Slide 7 / 31

Nep-acc Nep-afr Nep-all Contains all new RePEc docs Created roughly on weekly base Contains avg. 488 doc Notified Selects Notified Nep-upt Nep-ure Selects Sends issue Manual selection of relevant documents is a time consuming task. Slide 8 / 31

ERNAD ERNAD (Editing Reports on New Academic Documents) is a purposed built system Re-rank nep-all for each editor based on the specific report topic Looking at past issues of a report to produce a ranked nep-all If presorting works well editors select highly ranked documents from nep-all Slide 9 / 31

ERNAD example for Nep-Africa (NEP-AFR) 1. Tax compliance.. 2. Mental accounting.. … 212. Ethnic..in Africa 317. Sino-African relations: Nep-all unsorted Nep-all presorted Slide 10 / Ethnic..in Africa 2. Sino-African relations: … 50. Tax compliance Mental accounting..

Editing stages Slide 11 / 31

Research questions RQ 1: How long is the editing duration? RQ 2: What influences the success of a report? –Editing duration –Issue size RQ 3: How much effort is invested for selecting and sorting papers per issue? N –Relative search length Slide 12 / 31

RQ 1: Editing time How much time do editors invest to create a report? Slide 13 / 31

Pre-selection Editing an issue can be interrupted This would distort the results Exclude interrupted issues by separating the edit duration in 3-minute chunks Slide 14 / 31

Pre-selection Limit edit time < 90 min Slide 15 / 31

RQ 1: Editing time Avg minutes. (sd = 10.1) Min. 2.5 minutes NEP- RES (Resource economics) Max. 53 minutes NEP-ETS (Economic time series) Slide 16 / 31

Summarize RQ 1 Average editing time is comparable low with 15.5 minutes Huge scattering between the reports: –Min. 2.5 minutes –Max. 53 minutes Slide 17 / 31

RQ 2: Influences to successful reports Popularity of a report can be measured by the number of subscribers. Huge scattering between number of subscribers per report –Max NEP-HIS Business, Economic and Financial History –Min. 75 NEP-CIS Confederation of Independent States Factors influencing reports success for example: topic, age of a report.. Does the issue size or the editing time influence the report success? Slide 18 / 31

Editing time Education 2198 sub. (avg. 836) Project, Program and Portfolio Management 43,5 min (avg. 15.5) Slide 19 / 31

Issue size Sports issue size 2.5 (avg. 12.4) Demographic Economic issue size 21 (avg. 12.4) Slide 20 / 31

Summarize RQ 2 There is no correlation between: – Issue size and number of subscribers – Editing time and number of subscribers We assume that the success of a report is mainly driven by topic and age. Slide 21 / 31

RQ 3: Effort in selecting and sorting How much effort is invested in selecting and sorting relevant documents from nep-all? Two measures are used: Relative search length Slide 22 / 31

N How many of the top n documents from pre-sorted nep-all are selected for the issue? N set to: 5, 10, 15, 20 We only consider issues where issue size > N A document is relevant if its index position in nep-all is < N. Slide 23 / 31

Example: 5 M={(D1, 4), (D2, 1), (D3, 7), (D4, 3), (D5, 9)} for issue I in report J = ⅗ Editors vary between using pre-sorted and un-sorted nep-all. Therefore: –Only consider issues with pre-sort usage > 50 Slide 24 / 31

Results for Avg. (82 rep) Avg. (64 rep) Avg. Avg. (31 rep) Max. found for nep-env (Environmental Economics) with = 0.99 Min. found for nep-cba (Central Bank) with = 0.35 Slide 25 / 31

Summarize Editors work comfortably with the presorting in nep-all. The number of papers per issue has no significant influence for the precision. Slide 26 / 31

Relative Search Length We know how many of the top N document from nep-all selected. To what depth do editors inspect nep-all? Ratio between the highest index position (hin) of the last relevant document in nep- all and the length of nep-all Slide 27 / 31

Example RSL Editor is given a nep-all containing 300 documents. M={(D1, 4), (D2, 10), (D3, 7)} RSL = 10/300 We assume that the editor has inspected nep-all to document 10. Slide 28 / 31

Relative Search Length NEP-MAC (Macroeconomics) RSL = 0.35 NEP-SPO (Sports and Economics) RSL = 0.01 Avg. RSL = 0.08 Slide 29 / 31

Summarize RSL The relative search length is comparable low with 0.08 Editors select papers from the very upper part of nep-all. Slide 30 / 31

Conclusion Focused on observable system features –Editing time –Influences on report success –Effort in creating an issue Summarize: The system supports the editor well in creating an issue A complete view requires a more user-centred observation. Future work: –Why and under what conditions is a document relevant? NEP provides many opportunities for further research on data that is relatively easily available. Slide 31 / 31

Thank you! Questions?