EuroCRIS Platform Meeting - Vienna 2-3 October 1998 CRIS as a source for tracking science publication patterns Fulvio Naldi - Carlo Di Mento Italian National Research Council

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct CRIS: Databases on Scientific & Technological Outputs  Papers published on scientific journals  Books or chapters of books  Contribution to conferences  Grey literature  Other kinds of scientific/technological outputs

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Main purposes A way to describe current activity and available skill and knowledge Scientific evaluation of people and institutes Bibliographic research when traditional bibliographic databases fail

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Scientific evaluation Outputs are usually classified according some rough indicators: National/International journals (distribution, editorial board, …) Type of paper: scientific article, review, letter, note, … National/International Conference (participants, speakers, scientific committee, …) Availability of the Proceedings Type of contribution: invited, full / short communication, … Number of co-authors ……

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Other "cold" indicators Diffusion Inclusion in SCI Impact Factor (IF) of the Journals Number of Citations, Immediacy Index, Self-Citing, Self-Cited, …... Mainly based on the Science Citation Index ® (SCI) of the Institute for Scientific Information (ISI) - Philadelphia - US

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Inclusion in SCI

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Inclusion in SCI Reliable but no selective (poor significance)

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct SCI includes: CELL NATURE SCIENCE Inclusion in SCI

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct SCI includes: CELL NATURE SCIENCE and also: JOURNAL OF THE AMERICAN HELICOPTER SOCIETY ORIENTAL INSECTS USDA FOREST SERVICE PACIFIC NORTHWEST RESEARCH STATION RESEARCH PAPER USDA FOREST SERVICE SOUTHERN FOREST EXPERIMENT STATION RESEARCH PAPER Inclusion in SCI

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Impact Factor (IF) -Geographical and disciplinary coverage -Technical applicability -Political opportunity Widely adopted with cautions and several criticisms on: total number of citations in the last 2 years IF = number of articles of last year

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Impact Factor Geographical Coverage There is a very poor coverage of the scientific literature in the under-developing countries (cfr. The European Report on Science and Technology Indicators - EC 1994)

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Impact Factor Disciplinary Coverage

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Impact Factor Technical applicability IF values depend on the Journals selected by ISI IF values of journals in different disciplines are not directly comparable Journals with low IF values, mainly if published in US, do not have international relevance High relevance, very specialised (low diffusion) journals may have low IF values Multidisciplinary journals rank first in less populated disciplines

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Impact Factor A possible approach 1. Group the journals of each scientific field (Mathematics, Physics, Chemistry, Biology, Medicine, etc.) and/or of each discipline (175 in ISI schema) in a small number (e.g. 4) of "Relevance Groups" according their IF value 2. Count the number of items in each "Relevance Group" and use these numbers for disciplinary and geographical comparisons

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Impact Factor Political opportunity New fields and borderline researches may be penalised Researchers are less motivated to submit their papers to new journals The selection of the journals: -influences IF values -has important economical implication

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Conclusions - 1 Publication-based evaluation of individual scientists and of research groups (Labs, Institutes, University, Councils, etc.) requires a way to rank scientific journals according their relevance in the scientific community.

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Conclusions - 2 There are some doubts on the general applicability of Impact Factor for any evaluation purpose. Emerging media (Web) are not yet included in SCI

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Proposal Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Proposal Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators If doubts are confirmed...

Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct Proposal Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators If doubts are confirmed... Set up a group of experts (CRIS, Scientometry, Bibliometry, Science Policy, etc.) to implement and maintain a "Michelin Guide" of the Scientific Journals according both “cold”, heuristic and empirical indicators.