3. Challenges of bibliometrics DATA: many problems linked to the collection of data FIELDS – DISCIPLINES : various classifications, not satisfactory INDICATORS.

Slides:

Advertisements

Similar presentations

The building blocks What’s in it for me? Bibliometrics – an overview Research impact can be measured in many ways: quantitative approaches include publication.

Advertisements

Bibliometrics Toolkit Google Scholar (GS) is one of three central tools (the others being ISI and Scopus) used to generate bibliometrics for researchers.

Ronald L. Larsen May 22, Trace relationships amongst academic journal citations Determine the popularity and impact of articles, authors, and publications.

N EW WAYS TO TRACK SCHOLARLY PRODUCTIVITY : T HE H - AND G - INDICES.

Bibliometrics: Measuring the Impact of Your Publications Jane Buggle Deputy Librarian.

ANALYSING RESEARCH – A GLOBAL PERSPECTIVE Krzysztof Szymanski – Country Manager Thomson Reuters October 2009.

Patterns of Research Collaboration in U.S. Universities, AAAS Meetings Denver, Colorado February 18, 2003 James D. Adams, University of Florida.

Bryan Coad Research Fellow Ian Wark Research Institute University of South Australia 18 th July 2013 Publishing during your PhD and during your post-doc.

SCIENTROMETRIC By Preeti Patil. Introduction The twentieth century may be described as the century of the development of metric science. Among the different.

Information Retrieval to Informed Action with Research Metrics Thomson Scientific Research Services Group 2007.

1 Guide to exercise 10 Bibliometric searching on indicators for journals, papers, and institutions Tefko Saracevic.

Bibliometrics overview slides. Contents of this slide set Slides 2-5 Various definitions Slide 6 The context, bibliometrics as 1 tools to assess Slides.

Aims Correlation between ISI citation counts and either Google Scholar or Google Web/URL citation counts for articles in OA journals in eight disciplines.

Bureau du F.R.S. – FNRS 23 juin 2009 Possible role of a research funding agency The role of evaluation and ranking of Universities in the quality culture.

1 Using metrics to your advantage Fei Yu and Martin Cvelbar.

Using Journal Citation Reports The MyRI Project Team.

Web of Science Pros Excellent depth of coverage in the full product (from 1900-present for some journals) A large number of the records are enhanced with.

A tutorial on how to compute H-index using Web of Science database.

T H O M S O N S C I E N T I F I C Editorial Development James Testa, Director.

Bibliometrics in Computer Science MyRI project team.

Journal Impact Factors and H index

The Changing Role of Intangibles over the Crisis Intangibles & Economic Crisis & Company’s Value : the Analysis using Scientometric Instruments Anna Bykova.

Guillaume Rivalle APRIL 2014 MEASURE YOUR RESEARCH PERFORMANCE WITH INCITES.

SCOPUS AND SCIVAL EVALUATION AND PROMOTION OF UKRAINIAN RESEARCH RESULTS PIOTR GOŁKIEWICZ PRODUCT SALES MANAGER, CENTRAL AND EASTERN EUROPE KIEV, 31 JANUARY.

Scientometric study of CSIR-NAL Scholarly Literature.

Orientation to Web of Science Dr.Tariq Ashraf University of Delhi South Campus

Difficulties and Possibilities of University Rankings in Hungary Magdolna Orosz (Eötvös Loránd University Budapest, Hungary) Academic cooperation and competitiveness.

The Web of Science database bibliometrics and alternative metrics

The Role of Citations in Warwick’s Strategy and Improving Them Nicola Owen (Academic Registrar) Professor Mark Smith (PVC Research: Science and Medicine)

Welcome to Scopus Training by : Arash Nikyar June 2014

Making an impact ANU Library What is impact What (the heck) are bibliometrics Publish with impact – an overview Debate on impact How innovative are you.

Standards in science indicators Vincent Larivière EBSI, Université de Montréal OST, Université du Québec à Montréal Standards in science workshop SLIS-Indiana.

Experiences with a bibliometric indicator for performance-based funding of research institutions in Norway Gunnar Sivertsen Nordic Institute for Studies.

Bibliometrics toolkit: ISI products Website: Last edited: 11 Mar 2011 Thomson Reuters ISI product set is the market leader for.

2 Journals in the arts and humanities: their role and evaluation Professor Geoffrey Crossick Warden Goldsmiths, University of London.

Rajesh Singh Deputy Librarian University of Delhi Measuring Research Output.

Bibliometrics: coming ready or not CAUL, September 2005 Cathrine Harboe-Ree.

A month in the life of a university bibliometrician Dr Ian Rowlands University of Leicester, UK.

Bibliometric research methods Faculty Brown Bag IUPUI Cassidy R. Sugimoto.

The Web of Science, Bibliometrics and Scholarly Communication 11 December 2013

Ph. D. Completion and Attrition: Analysis of Baseline Data NSF AGEP Evaluation Capacity Meeting September 19, 2008 Robert Sowell Council of Graduate Schools.

SCOPUS AND SCIVAL EVALUATION AND PROMOTION OF UKRAINIAN RESEARCH RESULTS PIOTR GOŁKIEWICZ PRODUCT SALES MANAGER, CENTRAL AND EASTERN EUROPE LVIV, 11 SEPTEMBER.

THOMSON SCIENTIFIC Patricia Brennan Thomson Scientific January 10, 2008.

Bibliometric methods of (research) assessment Themis Lazaridis Chemistry Department City College of New York/CUNY

Google Scholar as a cybermetric tool Alastair G Smith Victoria University of Wellington New Zealand

1 NEST New and emerging science and technology EUROPEAN COMMISSION - 6th Framework programme : Anticipating Scientific and Technological Needs.

Theme 2: Investing in Science and Innovation Summary December 4, 2008.

An overview of main bibliometric indicators: Dag W. Aksnes Data sources, methods and applications Nordic Institute for Studies in Innovation, Research.

Where Should I Publish? Journal Ranking Tools eigenfactor.org SCImago is a freely available web resource available at This uses.

A Bibliometric Comparison of the Research of Three UK Business Schools John Mingers, Kent Business School March 2014.

Bibliometrics for your CV Web of Science Google Scholar & PoP Scopus Bibliometric measurements can be used to assess the output and impact of an individual’s.

How to use Bibliometrics in your Career The MyRI Project Team.

Mapping New Strategies: National Science Foundation J. HicksNew York Academy of Sciences4 April 2006 Examples from our daily life at NSF Vision Opportunities.

The Web of Science, Bibliometrics and Scholarly Communication

Database collection evaluation An application of evaluative methods S519.

1 Making a Grope for an Understanding of Taiwan’s Scientific Performance through the Use of Quantified Indicators Prof. Dr. Hsien-Chun Meng Science and.

RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010.

Bibliometrics and Publishing Peter Sjögårde, Bibliometric analyst KTH Royal Institute of Technology, ECE School of Education and Communication in Engineering.

Round Table Discussion Bibliometric Indicators in Research Evaluation and Policy Colloque Evolution des publications scientifiques Académie des sciences,

Web of Science: The Use & Abuse of Citation Data Mark Robertson & Adam Taves Scott Library Reference Dept.

Research assessment for at the Academy of Sciences of the Czech Republic Eva Zažímalová The Academy of Sciences of the Czech Republic.

MEASURING RESEARCHERS: FROM RESEARCH PUBLICATIONS AND COMMUNICATION TO RESEARCH EVALUATION Lucie Vavříková 1.

1 e-Resources on Social Sciences: Scopus. 2 Why Scopus?  A comprehensive abstract and citation database of peer-reviewed literature and quality web sources.

الله الرحيم بسم الرحمن علیرضا صراف شیرازی دانشیار و مدیر گروه دندانپزشکی کودکان رئیس کتابخانه مرکزی و مرکز علم سنجی دانشگاه علوم پزشکی مشهد.

NIFU STEP studies in Innovation, Research and Education Peer review of impact? Options and challenges Liv Langfeldt RCN 14 April 2008.

Measuring Research Impact Using Bibliometrics Constance Wiebrands Manager, Library Services.

INTRODUCTION TO BIBLIOMETRICS 1. History Terminology Uses 2.

Role of librarians in improving the research impact and academic profiling of Indian universities J. K. Vijayakumar Ph. D Manager, Collections & Information.

Jian Wang Assistant Professor Science Based Business Program LIACS, Leiden University

Bibliometric Analysis of Water Research

Presentation transcript:

3. Challenges of bibliometrics DATA: many problems linked to the collection of data FIELDS – DISCIPLINES : various classifications, not satisfactory INDICATORS : which ones are valid/needed to quantify research performance? MAPPING : to assess potential rather than past performances (metrics) workshop

3. Challenges FIELDS – SCIENTIFIC DISCIPLINES Various classification (OCDE, national, ISI, Scopus, …) Problems of coverage Academic structure might be quite different from the classification of the database Context : a research assessment exercice (ULB; school of economy and business) Comparison between bibliometrics and peer-review approaches to measure publications in economics Issue 1: retrieval of all publications relevant to economics from the ISI-Thomson database (Web of Science) Issue 2 : matching the knowledge-based sorting with the academic delimitation of disciplines in this university

Agricultural economics & policy - Fisheries - Geosciences - Water resources - Engineering (environmental) Area studies - Demography - Gerontology - Nursing - Psychology (3 sub-domains) - Ethnic studies - Family studies - Computer science (3 sub-domains) - Communication - Education & educational research And also... What does ISI database show about the sciencific production in economics of a Belgian university (ULB) ? Retrieval of ULB publications ( ) for all ISI categories related to economics (JEL classification) For each category, manual sorting of publications 4 categories with 100% articles in economics 26 categories with at least 10% articles in economics 20 other relevant categories with less than 10% articles

3. Challenges of scientometrics INDICATORS: choice is critical! Repeatable, based on transparent methodology (allow the institutions to reproduce the methodology in-house) Capable of identifying comparable levels of research quantity and/or quality across disciplines Quantitative (scientific production) vs qualitative (citations) Impact on researchers behaviour Main drawbacks no control for the size of the institution databases not complete for SHS Australian case study (Harzing, in Economics & Bysiness, 2005) Indicators used by the government for funding : till 1993, competitive research grants; since1994: # publications Consequence : high publication quantity (# papers) but slow publication quality (citations per paper)

3. Challenges of scientometrics INDICATORS Which indicator is a valid and robust measure of research quality for a discipline? FNRS study of co-publications ( ) h-index : measure scientific productivity and apparent scientific impact m-index : measure the quality of the h « best » papers

3. Challenges Indicators (a tool for improvment) To feed research policy and improve performances Size versus production Production versus quality Univ E Univ C Univ K Univ G Univ D Size Number of papers Univ E Univ C Univ K Univ F Univ G Univ H Univ I Univ J Univ A Univ D Univ B Number of papers Number of citations per paper PHYSICS & ASTRONOMY

6. Pending questions Research evaluation is usefull But Needs to be carefully handled And Raises several questions Impact of evaluation on research activity, strategy (researchers, universities) Impact of evaluation on research funding (disciplines?) Risk to miss emergence of teams, themes (bibliometrics assess the past) Choice of methodology (peer review? bottom-up?) How to feed the research part of the (probably unavoidable) rankings with indicators developed as a support of research policy Possible role of Eurohorcs