1. Bibliometric Analysis in Social Sciences and Humanities: paradigm shift and mutual challenges João Costa Investigação e Ensino na NOVA June 29, 2012

2. OUTLINE 1.Bibliometrics in SSH: a paradigm shift and the response of the area. 2.The challenge taken by institutions and individuals (the case of FCSH). 3.The shortcomings of bibliometrics. 4.The challenge for SSH. 5.The challenge for bibliometrics.

3. A Paradigm Shift Common (and old) assumptions for evaluating science in SSH: -(Double-blinded) peer review as a standard; -Books and book chapters -Books and book chapters are as important as journal articles; monograph series -Major journals edit monograph series for longer articles; books -Major conferences edit books with selected papers; -Citation in any type of format signals impact; -Impact compared within areas and domains.  BOOKS are a major signal of academic achievement.  CITATION signals impact.

4. A Paradigm Shift New assumptions/concepts: -Impact factors; -Journals with higher relevance than books or book chapters; -No clear place for “cultural” products; -Comparability across different areas.  BOOKS lost relevance.  MARKERS OF IMPACT suddenly changed.  SSH suddenly perceived as areas with little impact.

5. The SSH response -Perception that adherence to paradigm shift is condition for access to funding; -Investment on diversification of publication types.  LEIDEN STUDY witnesses the growth of SSH in the relevant publication types.

6. The SSH response – the case of FCSH Rapid increase and investment in publications indexed in CI (source: Converis)

7. The SSH response – the case of FCSH -Institutional stimulus for investment in indexed publications: Individual productivity prizes; Productivity prizes to I&D centers; Membership assessment based on CI publications in certain I&D centers. Clear assumption of challenge.

8. The shortcoming of bibliometrics Leiden Study:

9. The shortcoming of bibliometrics Coverage: FCSH productivity in 2011 (including: articles in peer-reviewed journals, books as author/editor, book chapters, peer reviewed conference proceedings, journal issue) Total: 1170 Indexed: 55 (4,7%)

10. The shortcoming of bibliometrics Coverage and weakness in assessing impact (R. Santos): Citation ind. No. References No. ArticlesTotalNo. WoK% WoK SCI2008 1076 66382,2 AHCI+SSCI300 12 683 4 35334,3 AHCI1507 1113 54649,9 SSCI1505 57280714,5 Quadro 1. Numbers of cited references and proportion indexed in WoK. Source: Random sample of articles in WoS published in 2011.

11. The shortcoming of bibliometrics Hiding relevant work: Three examples: INET-ID: production of materials supporting the promotion of FADO to World Intangible Heritage by UNESCO.

12. The shortcoming of bibliometrics Hiding relevant work: Three examples: IELT: Collection of oral patrimony.

13. The shortcoming of bibliometrics Hiding relevant work: Three examples: CHAM: Underwater Archaeology

14. The shortcoming of bibliometrics Coverage: highly cited authors Book citations in 2007 (source: Times Higher Education with Thomson) Anthony Giddens (sociology)1303 Noam Chomsky (linguistics)812 Judith Butler (philosophy)960 Ulrich Beck (sociology)733 Jurgen Habermas (philosophy)1049

15. The shortcoming of bibliometrics Coverage: highly cited authors Occurrences and citations in WoS Anthony Giddens (sociology)3 / 1 Noam Chomsky (linguistics)19/79 Judith Butler (philosophy)21/ 57 Ulrich Beck (sociology)0 / 0 Jurgen Habermas (philosophy)3/0

16. The challenge for SSH -Precise identification of markers of research performance. -Clear adherence to evaluation of research (peer-reviewing, etc.) -Ranking of scientific products.

17. The challenge for bibliometry -Development of specific tools for different research areas; -Identification of correspondences across areas (e.g. what’s the SSH correspondent of a short article in Chemistry in terms of impact/work volume?)

18. The challenge for bibliometry -Inclusion of domain-specific products (e.g. patrimony collections, materials for language policy, education materials, research-based cultural products, etc.)

19. The challenge for evaluation -Make sure that evaluation takes into account the actual productivity instead of focusing on imposing a new typology of research products.

20. Conclusion: mutual challenges -More impact, wider coverage and stronger international profile for the SSH will come about if bibliometrics is taken seriously, and if the specificity of SSH is taken seriously by bibliometrics.