The Impact of Business and Management Research The ABS Journal Quality Guide: Version 4 Aidan Kelly, Charles Harvey, Huw Morris, Michael Rowlinson

Outline Rating research quality –Reasons –Methods Types of journal quality list The ABS 2010 Methodology –Rating scale –The list Changes since 2009 Recent developments

Rating Research Quality: Reasons Staff selection and development Promotion, pay and related decisions Assessing departments for development Funding by government and agencies Library purchase decisions Developing understanding of the field

Rating Research Quality: Methods Author Peer comment Individual citation Individual readership Peer rating Article Peer review Article citation Article readership Online rating feedback Journal Journal impact factor Journal quality list Co-citation

Types of Journal Quality List Institutional lists. based on the views of researchers within a department (e.g. Aston, 2010; Cranfield, 2008; ESSEC, UTD, ). Peer surveys. Assessments of peers in a field or sub-field (e.g. McKercher, 2005; Neil, 2006; Jamal, Smith and Watson, 2008; Peters, Daniels, Hodgkinson and Haslam, 2009 ). Citation studies. Judgements made on the basis of the number of times an average article in a journal is cited by the authors of articles in related journals (e.g. Institute of Scientific Information Scopus). Derived lists. Extrapolated from ratings awarded in assessment or audit activities (e.g. Geary, Marriot and Rowlinson, 2004). Hybrid lists. A combination of two or more of the methods listed above (c.f. ABS, 2009).

The ABS Methodology 2010: The ABS Guide

The ABS Methodology Started with a list of all journals from which two or more articles were submitted to the RAE Collection of ISI JCR data including Journal Impact Factor for 2008 and Five-Year mean Journal Impact Factor. 3.With RAE 2008 profile and sub-profile data, calculation of mean for each journal in RAE 2008 of the Grade Point Average, for outputs and overall, of institutions citing journal in submissions. 4.Comparison with institutional lists e.g. Aston (2008), Cranfield (2009), Kent (2007). 5.Initial editorial panel review and specification of ratings.

The ABS Methodology Calculation of World Elite Rating by reference to journal ratings from 10 leading international business schools. 7.Second editorial panel review and consideration in several cases of the following factors: i. Years and frequency of publication ii.Status of editor and editorial board iii.Quality of articles in three recent issue. 8. Final review by editors. 9.ABS Research Conference release 10.Consultation with the business and management community online and request for feedback.

The ABS Methodology 2010: World Elite Rating 1.Financial Times list (2009) 2.University of Queensland (2007) 3.Australian Business Deans Council (2008) 4.Monash University, Melbourne (2007) 5.ESSEC Business School Paris (2005) 6.Centre National de la Recherche Scientifique (2008) 7.Wirtschaftsuniversität Wien (2008) 8.Dutch Business Administration academics (1999) 9.University of Groningen School of Management (undated) 10.University of Texas Dallas (undated)

The ABS Methodology 2010: Outcomes 4* World elite 22 (2.7%) 4Top journal in its field 71 (8.7%) 3 Highly regarded journal230 (28%) 2Well regarded journal295 (35.9%) 1 Recognised journal203 (24.7%)

Proportion of Journals Grade ABS 2008 ABS 2009 ABS Total

RAE 2008 Grade Mean cites in BMS Mean GPA outputs Mean GPA score Total

The ABS Methodology 2010: Journal Citation Impact Factor

Use of Standardisation Grade Impact 2008 Five Year Mean

Pearson ‘r’ Correlations Kent 2007 Aston 2008 World Elite Count RAE 2008 Outputs GPA Impact Factor 2008 Five Year Mean Impact ABS ABS

Reduction in the number of journals Greater attention to field centrality Recognition of World elite status Reduction in the ratings awarded to several journals Journals put on warning Encouragement to journal editors to apply for Thomson ISI listing and possibly other citation databases Summary of Changes since 2009

Recent developments: which may affect the development of the guide in future HEFCE suggest move to bibliometrics (Dec 2006) HEFCE bibliometrics consultation (March 2007) HEFCE suggests individual citation (Sept 2007) CNRS produce journal rating lists (Sept 2007) UUK suggest citation is a bad idea (Oct 2007) Australian Deans produce journal ranking list (Nov 2007) HEFCE REF Consultation (2009) HEFCE follow RCUK to encourage greater focus on impact (2009) EFMD decides not to formally adopt a journal list (2009) Conservatives suggest REF will be delayed until 2015 (2010) QR funding changes announced (2010) AACSB consider research measures that are not wholly focused on journals (2010)

Issues to consider for next time? Next ABS Journal Guide scheduled for Degree of mechanisation versus peer review in ratings. How to deal with practitioner focused outputs. Inclusion of Elsevier Scopus and other citation databases – should we continue to privilege ISI Thomson. What account to be taken of circulation and readership in assessing journals and other outputs. Will guides and lists be superseded by measures of individual citation and individual ratings.