1. Research Integrity in ASHA: Scientific Publication Practices 2006 ORI Research on Research Integrity Conference University of South Florida Safety Harbor Resort and Spa - Tampa, FL December 1-3, 2006 Janis Costello Ingham, Ph.D. University of California – Santa Barbara Sharon E. Moss, Ph.D. American Speech-Language-Hearing Association

2. Co-Presenters Dean C. Garstecki, Northwestern University Jennifer Horner, Medical University of South Carolina Joanne Jessen, ASHA Charissa R. Lansing, University of Illinois, Urbana - Champaign James H. McCartney, California State University Fred D. Minifie, University of Washington (retired) Randall R. Robey, University of Virginia Sarah Slater, ASHA This research was supported by the Research on Research Integrity Program, an ORI/NIH collaboration, grant # RO1 N544534-0151

3. Session Outline Purpose of Study Methodology Results Conclusions Questions???

4. Purpose of Study The purpose of this research survey was to -- sample CSD academic and research community importance and adequacy of research integrity topics publications in ASHA’s scientific journals

5. Methodology Instrument Development Pilot Testing Fielding/Survey Population

6. Methodology: INSTRUMENT DEVELOPMENT

8. Methodology: Instrument Development

10. Methodology: PILOT TESTING Cognitive interviews Field tested

11. Methodology: FIELDING/SURVEY POPULATION Editor, Associate Editor, Reviewer Publications Board Member Board of Ethics Member Contributing Scientist / Author General Readership Graduate Student

12. Results: RANKINGS OF IMPORTANCE The survey Q: In regard to scientific integrity in research publications in general, how important is ______ ? Scale: 1-10 (1 = Not At All Important and 10 = Critically Important) or, Do Not Know

13. RESULTS: The TOP TEN ( derived from Group 1: Editors, Associate Editors, Pub Board) 1 – Fabrication of data 2 – Falsification of data 3 – Separation of advertising sales from scholarly content decisions 4 - Plagiarism 5 – Criteria for accepting advertising

14. The TOP TEN, continued 6 – Maintenance of retracted papers in electronic archives 7 – Publication of retractions and errata 8 – Confidentiality of the peer review process 9 – Disclosure to Editor of author conflicts of interest 10 – Author declaration to Editor of adherence to HIPAA requirements

15. The TOP TEN, continued There was high agreement in ranking of items between Group 1 (Editors, Associate Editors, Publication Board members) and Group 2 (reviewers, authors, and Board of Ethics members) (See handout, blue items) Rank order correlation was 0.94. Mean highest rankings were 9.71, 9.74 and 9.70, respectively

16. The TOP TEN, continued Within the Top Ten, readers judged three items to be considerably less important than the other judge groups: Criteria for accepting advertising Confidentiality of the peer review process Disclosure to the Editor of author conflicts of interest

17. RESULTS: The “BOTTOM TEN” 56 – Authors’ ID during the review process 55 – Requirement that copyright be transferred to ASHA 54 – Authors’ responsibility to report errors for correction 53 – Explicit statement of humane care and Treatment of animals in the published MS 52 – Evaluation of reviewers

18. The BOTTOM TEN, continued 51 – Reviewers’ evaluation of MSs previously reviewed for another journal 50 – Data sharing for meta-analyses 49 – Publication of previously published data 48 – Reviewers’ ID during the review process 47 – Authors’ declaration to Editor of adherence to IRB requirements

19. The BOTTOM TEN, continued There was good agreement in ranking of items among the three groups. Mean lowest rankings were: 4.66, 6.11 and 7.12, for Groups 1, 2, and 3, respectively.

20. THE “READER” GROUP Throughout the data, where incongruent ratings occurred, they were typically generated by the Reader group (18 items out of 56 total). (See red items on the handout.) Rank order correlations: Group 1 vs. Group 3 = 0.86 Group 2 vs. Group 3 = 0.83

21. ASHA’s Publication Policy Documents Publication Manual of the American Psychological Association ASHA’s Editor’s Handbook ASHA’s Information for Reviewers ASHA’s Instructions for Authors ASHA’s Reviewer Agreed Letter ASHA’s Code of Ethics

22. Adequacy of Coverage of Publication Ethics Issues in ASHA’s Policy Documents Expert judges identified all sections judged to be potentially relevant to the survey’s 56 items from all six policy documents Panel of 6 expert judges rated each identified item from each document 0 = item not relevant to the given topic 1 = only tangentially relevant 2 = partially relevant, but not completely adequate 3 = adequate coverage/discussion of the topic 4 or more rankings of “3” = topic adequately covered

23. Adequacy of Coverage of Publication Ethics Issues in ASHA’s Policy Documents Eight of the Top Ten items were judged to be discussed adequately in at least one of ASHA’s policy documents. (See handout.) However, of the total 56 items, 28 (50%) were judged not to be adequately discussed, or were completely ignored. (See handout.) Adequate coverage of specific items is distributed across five documents, sometimes singly; sometimes redundantly. (See handout.)

24. Conclusions Develop a comprehensive document. Make recommendations to Publications Board and Board of Ethics. Develop additional policies. Educate the research and reader membership. Use these findings as a baseline for the future.

25. Questions