2 Journals in the arts and humanities: their role and evaluation Professor Geoffrey Crossick Warden Goldsmiths, University of London

3

4 Distinctive place of journals in arts & humanities research  Diversity of output monographs, edited collections edited texts, journal articles practice outputs  No clear hierarchy of esteem amongst as well as within each challenge for RAE and promotion panels esp problems for practice outputs

5 Why do we want to know about journals? Explore the terms  Use of journals to indicate: activity and productivity impact - relevance and use quality - peer judgment

6 Measurements and proxies Activity & ProductivityImpactQuality Quantitative Tangible & Qualitative methods & intangible methods & data dataevidence & data Increasing use of proxy metrics to infer impact and quality Especially in context of changes to RAE

7 Why is it so hard to rank journals or use citation data?  Rank according to what criteria?  Impact factors very difficult  Citation behaviour is very different not cumulative cf science old work remains highly cited [See ISI list] critical discourses as mode of research citation not clear sign of quality/influence the culture of the footnote  Diverse outputs - not just paper-based  Print output different arts & humanities  Lower % paper outputs in ISI-Journals

8 Highest citations in ISI humanities journals Karl Marx 2 Lenin 3 Shakespeare 4 Aristotle 5 The Bible 6 Plato 7 Freud 8 Chomsky 9 Hegel 10 Cicero

9 RAE submissions 1996 & 2001 Humanities, Languages and Arts  Journal articles? % % sciences 90% 96% engineering 57%78% social sciences 42% 54%  Books? % % sciences 6% 3% engineering 8% 5% social sciences32% 28%  Book chapters % %  Other %2001 – 9%

10 Journal publication & RAE quality Journal articles as % outputs to RAE2001 by discipline UoA3b3a455* Law Asian studies History Art & design Music

11 ISI journals & the arts & humanities  Publication in ISI journals often small % overall outputs RAE 2001  Philosophy highest at 52%  Library & information management 40%  Most other subjects in 20%-29% range  English & French just 21%  Below 20% in Italian, Theology, Art & Design and Middle Eastern & African Studies

12 Current projects on quality & journals  Issue of evaluating quality very current  So too is assessing standing of journals  But they’re not the same thing at all  And decreasingly so arts & humanities  Reflect on these:  RAE and metrics  European Science Foundation’s ERIH  Humanities Indicators Project

13 RAE metrics  Metrics-driven RAE  ‘Neither citations nor RC income’  AHRC expert group  outputs but no proxies for their quality  Post-2008: STEM cf rest of disciplines  ‘ robust indicators’ being sought – for STEM  primarily HESA and bibliometric data  arts & humanities 2013+? earlier impact  consultation non-science  national bibliometrics consortium

14 European Reference Index for the Humanities  European Science Foundation “ The ERIH lists will help to identify excellence in Humanities scholarship & should prove useful for the aggregate bench-marking of national research systems…in determining the international standing of the research activity in a given field in a country. However, as they stand, the lists are not a bibliometric tool”

15 Methods & goals of ERIH  Expert Panels in 15 disciplinary areas  ESF member lists, iterative consultation  Categorisation (lists emerging this year)  A = high-ranking international level  B = standard international level  C = important local/regional level  Resistance to hierarchy: why?  Say above all to strengthen peer review  How, if at all, will it be used?

16 Humanities Indicators Project  American Academy of Arts & Sciences  many variables on state of humanities  Publications element:  Focus is monograph publications data no quality indicators sought  No plans to look at journal publishing  Making humanities count: the importance of data

17 Arts & humanities journals: the challenge of bibliometrics  Is there a challenge?  Does anyone want to do bibliometrics with arts & humanities journals?  Little interest UK or elsewhere  In many ways for good reasons set out here  Yet looming is RAE post-2013…. can we build ‘robust measures’ without them? can it be done with them?