Use of bibliometry for the evaluation of researchers and teams in medicine and biology A practical view from a School of Medicine Dean Patrick Berche Universitary School of Medicine Paris Descartes Colloque de l’Académie des sciences "Évolution des publications scientifiques - Le regard des chercheurs" des mai 2007

- High diversity of approaches and methodologies, as illustrated by the existence of many disciplines from basic biology to clinical research at the bedside - Artisanal aspects of this research, contrasting with the necessity of heavy technological platforms and critical mass of researchers required to perform competitive research - Very high number of scientific journals (>4600) Main features of Research in Biology & Medicine

1.Importance of basic Research for breakthroughs in medicine: a continuum from cognitive research to clinical research at the bedside 2. Close relationship between Pedagogy and Research : no good pedagogy without research 3. Difficulties to program Research : is it possible to plan it? Many discoveries in medicine and biology have never been planed, neither financially supported by institutions, at least at the beginning. Examples: HIV, Helicobacter pylori, RNA silencing… At the most, the institution could define large and blurred fields, related to medical Research Strategy to develop Research in a School of Medicine 4. Only matter individuals ! EVALUATION Originality, creativity, curiosity, capacity of wondering and questionning with respect to banal or unusual observations, capacity to interact and to create favorable scientific environment.

1.To allow ideally a complete liberty for researchers, without administrative constraints, without predefined scientific frames and priorities… 2. To favor a good scientific environment - Create critical mass composed of many teams at the same location, with convivial places for meeting!! - Reinforce big technological platforms - Encourage interactions with other scientific fields (physics, chemistry, mathematics…). What is the role of the institution to boost competitive research ? 3. To identify, evaluate and recruit the best researchers : It is the main role and responsability of institutions: essential, difficult and dangerous task The major objective is to improve knowledge to better treat and cure patients.

- Creativity, originality and innovative potential of a researcher - Aptitude to interfere with others teams to progress in their work - Capacity to attract young scientists and to transmit skill and competence What do we want to evaluate ? Who do we want to evaluate ? - Young scientists and medical doctors - Confirmed scientists and medical doctors: achievements and competitivity of their research teams - The national and international rank of Research of our Medical School Hard to evaluate with bibliometric analysis

This evaluation is mainly qualitative: - Originality of previous works and new project - Is it a risky Research? - Choice of the Research laboratory chosen for the PhD thesis - Assessment of the real contribution of the scientist in the publications related to the PhD thesis - Quality of publications estimated by reading the original articles - Oral presentation of works and answers to questions Evaluation of young scientists The bibliometric analysis must be considered caustiously at this stage, often mainly related to the quality of Research laboratory

Evaluation of senior scientists and associated Research teams (1) A quantitative bibliometric analysis - The global productivity of a researcher and of a team, especially the paucity or absence of significant publications over the last 5 years-period. - The number of articles in journals with high impact factors and in the best journals of the discipline. - A comparison with recently recruited scientists in the same discipline. (2) A qualitative bibliometric analysis - Proportion of articles associated with a leadind signature rank of scientist (1 rst, 2 rd, last or corresponding author). - Proportion and rank in the articles reporting original works, compared to reviews, letters, answers… - Citation index in articles > 5 years, but only for articles publishing original contributions where the scientist signs in useful position. - Proportion of joint-authorship articles - The repetition (‘’psittacisme’’) of articles according to the title and abstracts. The bibliometric analysis allows to globally assess the impact of a senior scientist and his associated team.

Evaluation of senior scientists and associated research teams Bibliometric analysis The bibliometric analysis must be cautiously used, without forgetting the pitfalls and limits of this approach. This analysis is only complementary of the qualitative analysis of original works, looking for the clue and creative ideas of the Research developed in publications, and to the true contribution of scientist in the published works and breakthroughs. Other important elements for evaluation - Number of Research grants (industrials, National Public Agencies, international grants [EEC]…) - Collaborations: quality of national and international teams - Number of licenses and creation of start-up

How to estimate the scientific production of an institution as Paris Descartes Medical School? The estimation of the scientific level of institutions can be only based on publications of original works in useful position in major journals.

Nature, Cell, Science, Nature Genetics, Nature Immunology, Nature Medicine, New England Journal of Medicine Articles in 7 top biological journals IF >25 Useful position of French authors: 1 rst, second and last position 75 from Paris Descartes Medical School

, New England Journal of Medicine, Lancet, Nature Medicine, JAMA Lancet, Nature Medicine, JAMA Publications in 4 top clinical journals IF Useful position of French authors: 1 rst, second and last position 51 from Paris Descartes Medical School

1.The bibliometric analysis must be performed by discipline, since the impact factors of international journals are related to the extent of the scientific communauty implicated in the field. 2. The signature leadind rank of scientist is crucial : is he a major contributor of the work or project? Is he associated as co-author (which only reflects the capacity to interact, and not the real contribution, including publications in major journals) ? 3. The importance of original works in publications must be clearly estimated. General reviews in major journals only indicate notoriety and recognition, not originality and creativity. 4.The IF of journals are excellent indicators, stable in time and based on the quality of the selection performed by independent international experts. However, major breakthroughs might be published in obscure journals of low IF. 5. The citation index does not distinguish between original works, technical notes, letters, and reviews, neither takes care of the position of authors, auto-quotations… It is an historical indicator requiring several years before significant citations of articles appear. It is useful to measure notoriety of a senior scientist. Conclusion