Author-perceived Quality Characteristics of Science, Technology and Medical (STM) Journals Dr. John J. Regazzi Selenay Aytac, MBA Scholarly Communication Lab College of Information and Computer Science Long Island University, NY METRO Science Technology Medical Librarians Special Interest Group October 12, 2007

LIU, CICS Scholarly Communication Lab, October 12, Scope of the Research  The aim of this research study is to explore author-perceived quality characteristics of STM (Science, Technology and Medicine) journals. One of the most, if not the most, important issue today facing scholarly communication is what constitutes quality in the publishing and dissemination of research findings.

LIU, CICS Scholarly Communication Lab, October 12, Research Questions (1) What quality attributes are most important to authors for STM journals? (2) What are some of the underlying differences among researchers that might account for certain journals to be called “prestigious” while others are considered less valuable? (3) What are the key factors considered in submitting articles to STM journals by potential authors? (4) Which attributes might be more important in the changing electronic publishing and web- based market?

LIU, CICS Scholarly Communication Lab, October 12, Literature Review  16 attributes studied and linked to the literature: 1. Online tools (Tibbitts, 2006; Rowlands and Nicholas, 2005; Mabe, 2003) 2. Society Pub. (Chressanthis and Chressanthis, 1993; Gorman and Calvert, 2001; Mabe, 2003) 3. Editorial board (Franke, 1990; Nisonger, 2002; Tibbitts, 2006; Rowlands and Nicholas, 2005; Mabe, 2003) 4. Previous experience(Mabe, 2003) 5. Impact factor (Garfield, 1955; Yue and Wilson, 2007; Saha, 2003; Tibbitts, 2006; Rowlands and Nicholas, 2005; Mabe, 03) 6. Reputation (Frank, 1994; Tibbitts, 2006; Rowlands and Nicholas, 2005; Mabe, 2003) 7. Rejection rate (Bjork and Holmstorm, 2006); Rowlands et al 2004) 8. Time to publication(Bjork and Holmstorm, 2006; Gleser, 1986; Tibbitts, 2006; Rowlands and Nicholas, 2005; Mabe, 2003) 9. Price (Bjork and Holmstorm, 2006; Chressanthis and Chressanthis, 1993; Rowlands et al, 2004)) 10. Publisher (Mabe, 2003) 11. Readership (Franke and et all., 1990; Tibbitts, 2006; Rowlands and Nicholas, 2005) 12. Colleague recs. (Mabe, 2003) 13. Journal online (King and et all, 2006) 14. Copyright (Grimby, 2005; Rowlands et al, 2004)) 15. Open access (Schroter, 2005; Regazzi and Caliguiri, 2006; Rowlands et al, 2004)) 16. Design (Gorman and Calvert, 2001; Joseph, 2006; Erdman, 2006; Mabe, 2003)

LIU, CICS Scholarly Communication Lab, October 12, Methodology  Data collected from the volunteer full-time faculty of Long Island University.  Subjects were invited by mail. Letters of invitation to the research study were mailed to the faculty members of two colleges: (1) College of Information and Computer Science (CICS), and (2) School of Health Profession and Nursing Science (SHS). These 2 schools were selected represent much of the publishing interest of ALPSP members  Pre-test for the survey instruments was undertaken among 3 subjects prior to the experiments

LIU, CICS Scholarly Communication Lab, October 12, Methodology 3 Research Methods used: (1) Survey: ranking importance of 16 attributes (2) Focus groups and (3) Semi structured face-to-face interviews conducted for this study.

LIU, CICS Scholarly Communication Lab, October 12, Data Collection  13 Surveys completed  2 Focus Group(s) CICS Focus Group with 7 subjects SHS Focus Group with 6 subjects  5 Face-to-face semi-structured interview(s) 2 CICS Subjects 3 SHS Subjects

LIU, CICS Scholarly Communication Lab, October 12, Survey: Prior to the each Focus Group  Each subjects first asked for the key demographics such as: (1) academic title, (2) tenured or not, (3) number of publications in last 7 years, and (4) gender.  And the subjects have been asked to circle the number that represents how they would rate the importance of the pre-defined 16 journal attributes (5 point Likert scale was used to measure the perception)

LIU, CICS Scholarly Communication Lab, October 12, Survey: 16 Pre-defined Quality Attributes were Ranked 1. Availability of Online Manuscript Tools 2. Journal Published by a Society or Not-for-profit 3. Having an Editorial Board to Oversee the Journal 4. Previous Experience with the Journal 5. The Impact Factor of the Journal 6. The Reputation of the Journal 7. The Rejection Rate of the Journal 8. The Estimated Length of time to Article Publication 9. The Price of the Journal 10. The Specific Publisher of the Journal 11. The Readership of the Journal 12. Recommendations form Colleagues about the Journal 13. The Availability of the Journal Online 14. Copyright Restrictions on you 15. Open Access, Public Access and Web posting options and policies of the Journal 16. Design: quality of images, typesetting, etc.

LIU, CICS Scholarly Communication Lab, October 12, Focus Groups  Focus groups were divided into 3 broad areas of discussion: a) the most important attributes b)the least important attributes c) other attributes that the group wish to discuss

LIU, CICS Scholarly Communication Lab, October 12, Face-to-Face Interviews  5 in-depth interviews were conducted the following days to discuss these aforementioned quality attributes with the volunteer subjects from the focus group  Each interviewee was asked to describe the process from the research to article submission and when and why a journal is considered.

LIU, CICS Scholarly Communication Lab, October 12, Findings: CICS Survey

LIU, CICS Scholarly Communication Lab, October 12, CICS – Ranking by Importance

LIU, CICS Scholarly Communication Lab, October 12, Findings: SHS Survey

LIU, CICS Scholarly Communication Lab, October 12, SHS – Ranking by Importance

LIU, CICS Scholarly Communication Lab, October 12, Findings: Rankings by Group and Attributes

LIU, CICS Scholarly Communication Lab, October 12, Statistical Analysis  Are there a group differences? Overall by Discipline (CICS vs. SHS) Gender Tenure (Tenured vs. not Tenured) (Reliability of Survey Instrument: Highly reliable - Chronbach’s alpha =.8)

LIU, CICS Scholarly Communication Lab, October 12, Statistical Analysis– Overall Differences by Discipline Online Manuscript Tools CICSMean 2.71 SHSMean 4.33 Rejection rate CICS Mean 3.29 SHSMean 4.00 “Online Manuscript Tools and Rejection Rate Groups are significantly different at the p=.013 and p=.042 levels respectively.”

LIU, CICS Scholarly Communication Lab, October 12, Statistical Analysis– Overall Differences by Gender Online Manuscript Tools MaleMean 2.60 FemaleMean 4.00 Recommendations Male Mean 3.60 FemaleMean 4.63 Society Publisher MaleMean 2.40 FemaleMean 4.13 “Online Manuscript Tools and Recommendations and Society Publisher are significantly different at the p=.047 and p=.023, and p=.005 levels respectively.”

LIU, CICS Scholarly Communication Lab, October 12, Statistical Analysis– Overall Differences by Tenure vs. Non-Tenure Copyright TenuredMean 4.30 Non-TenuredMean 3.00 “Copyright restrictions is significantly different at p=.023 level”

LIU, CICS Scholarly Communication Lab, October 12, Content Analysis-Focus Groups-Coding  We had 16 pre-defined set of concepts for these experiments, however, some new attributes emerged during the focus groups and interviews as well  Each script was coded for implicit and explicit terms for conceptual analysis

LIU, CICS Scholarly Communication Lab, October 12, Focus Group-Coded Data

LIU, CICS Scholarly Communication Lab, October 12, Analysis: by Attribute Importance and Awareness

LIU, CICS Scholarly Communication Lab, October 12, Interviews Key Questions:  When do you start thinking about journals?  How many journals do you consider submitting your article?  How do you think about journals at different points of research process?

LIU, CICS Scholarly Communication Lab, October 12, Interviews – Journal Identification by Research Stage Research Idea Literature Review Data Collection Write Up Submit Article 2-3 Journals(1) 1 Journal(2) 1-2 Journals(3) 1 Journal(4) 2 Journals(5) The 5 subjects interviewed identified the stage of research they begin to consider a journal, and the number of journals they are considering at that point.

LIU, CICS Scholarly Communication Lab, October 12, Conclusion 1. Group differences by discipline (Computer Science vs. Allied Health) – significant differences on the perceived value of online tools and rejection rate, with SHS faculty rating these attributes significantly higher than CICS faculty. 2. Gender differences– significant differences for recommendations from colleagues, online manuscript tools, and society as publisher, with female faculty rating these attributes higher than male faculty. 3. Tenure vs. non-tenured differences– significant differences on copyright, with tenured faculty rating the importance of this attribute significantly higher than non tenured faculty. 4. The leading perceived attributes were: (a) the reputation of the journal, (b) the estimated length of time to article publication, and (c) the readership of the journal, as highly leading; others were identified as important as “emerging” issues/attributes: (d) recommendations from colleagues about the journal, (e) copyright restrictions, and (f) open access.

LIU, CICS Scholarly Communication Lab, October 12, Implications for the Future  Monitoring these attributes as more articles become available on the Web, through search engines, and with the emerging new functionality of Web 2.0.  Further studies: Larger sample size Different disciplines Deeper analysis of differences of group, gender, and publications Role of ‘social responsibility’ in the development of journal brand equity