1. EuroCRIS Platform Meeting - Vienna 2-3 October 1998 CRIS as a source for tracking science publication patterns Fulvio Naldi - Carlo Di Mento Italian National Research Council

2. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 CRIS: Databases on Scientific & Technological Outputs  Papers published on scientific journals  Books or chapters of books  Contribution to conferences  Grey literature  Other kinds of scientific/technological outputs

3. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Main purposes A way to describe current activity and available skill and knowledge Scientific evaluation of people and institutes Bibliographic research when traditional bibliographic databases fail

4. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Scientific evaluation Outputs are usually classified according some rough indicators: National/International journals (distribution, editorial board, …) Type of paper: scientific article, review, letter, note, … National/International Conference (participants, speakers, scientific committee, …) Availability of the Proceedings Type of contribution: invited, full / short communication, … Number of co-authors ……

5. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Other "cold" indicators Diffusion Inclusion in SCI Impact Factor (IF) of the Journals Number of Citations, Immediacy Index, Self-Citing, Self-Cited, …... Mainly based on the Science Citation Index ® (SCI) of the Institute for Scientific Information (ISI) - Philadelphia - US

6. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Inclusion in SCI

7. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Inclusion in SCI Reliable but no selective (poor significance)

8. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 SCI includes: CELL NATURE SCIENCE Inclusion in SCI

9. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 SCI includes: CELL NATURE SCIENCE and also: JOURNAL OF THE AMERICAN HELICOPTER SOCIETY ORIENTAL INSECTS USDA FOREST SERVICE PACIFIC NORTHWEST RESEARCH STATION RESEARCH PAPER USDA FOREST SERVICE SOUTHERN FOREST EXPERIMENT STATION RESEARCH PAPER Inclusion in SCI

10. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Impact Factor (IF) -Geographical and disciplinary coverage -Technical applicability -Political opportunity Widely adopted with cautions and several criticisms on: total number of citations in the last 2 years IF = number of articles of last year

11. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Impact Factor Geographical Coverage There is a very poor coverage of the scientific literature in the under-developing countries (cfr. The European Report on Science and Technology Indicators - EC 1994)

12. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Impact Factor Disciplinary Coverage

13. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Impact Factor Technical applicability IF values depend on the Journals selected by ISI IF values of journals in different disciplines are not directly comparable Journals with low IF values, mainly if published in US, do not have international relevance High relevance, very specialised (low diffusion) journals may have low IF values Multidisciplinary journals rank first in less populated disciplines

14. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Impact Factor A possible approach 1. Group the journals of each scientific field (Mathematics, Physics, Chemistry, Biology, Medicine, etc.) and/or of each discipline (175 in ISI schema) in a small number (e.g. 4) of "Relevance Groups" according their IF value 2. Count the number of items in each "Relevance Group" and use these numbers for disciplinary and geographical comparisons

15. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Impact Factor Political opportunity New fields and borderline researches may be penalised Researchers are less motivated to submit their papers to new journals The selection of the journals: -influences IF values -has important economical implication

16. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Conclusions - 1 Publication-based evaluation of individual scientists and of research groups (Labs, Institutes, University, Councils, etc.) requires a way to rank scientific journals according their relevance in the scientific community.

17. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Conclusions - 2 There are some doubts on the general applicability of Impact Factor for any evaluation purpose. Emerging media (Web) are not yet included in SCI

18. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Proposal Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators

19. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Proposal Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators If doubts are confirmed...

20. Naldi F., Di Mento C. - CRIS as a source... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998 Proposal Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators If doubts are confirmed... Set up a group of experts (CRIS, Scientometry, Bibliometry, Science Policy, etc.) to implement and maintain a "Michelin Guide" of the Scientific Journals according both “cold”, heuristic and empirical indicators.