1. 1 Foundations for RCR Talked presented at the Conference on: Mentoring and Supervision for the Responsible Conduct of Research Washington University School of Medicine July 24, 2006 Nicholas H. Steneck, PhD © Nicholas H. Steneck For personal use; may not be quoted or used without permission of the author

2. 2 Two approaches to RCR education  Formal ~ set curriculum / learning experience Courses, seminars, retreats, workshops Web-base programs  Informal ~ learn from others, on the job Journal clubs and laboratory meetings Mentor/PI sessions  Conference focus: Informal RCR education  Objectives: Key areas that should be covered Different approaches to RCR instruction for mentors Major challenges

3. 3 1990s, shift to formal RCR education  Key events: 1989, IOM Report 1990, NIH Training Grant requirement  On Being a Scientist (2nd edition) In the past, young scientists learned the ethics of research largely through informal means–by working with senior scientists and watching how they dealt with ethical questions. That tradition is still vitally important. But science has become so complex and so closely intertwined with society's needs that a more formal introduction to research ethics and the responsibilities that these commitments imply is also needed-an introduction that can supplement the informal lessons provided by research supervisors and mentors. (Preface )

4. 4 Core areas  Developed over time: 1994 Training Grant Requirement  Conflict of Interest  Responsible authorship  Policies for handling misconduct  Data management  Human & animal subjects 2000 HHS RCR requirement  9 areas  Other areas could be added: Grant and financial management Laboratory safety & special rules (biological materials…) Workplace rules Nine Core Areas 1. Data acquisition, management, sharing, and ownership 2. Mentor/trainee responsibilities 3. Publication practices and responsible authorship 4. Peer review 5. Collaborative science 6. Human subjects 7. Research involving animals 8. Research misconduct 9. Conflict of interest and commitment Nine Core Areas 1. Data acquisition, management, sharing, and ownership 2. Mentor/trainee responsibilities 3. Publication practices and responsible authorship 4. Peer review 5. Collaborative science 6. Human subjects 7. Research involving animals 8. Research misconduct 9. Conflict of interest and commitment

5. 5 Informal difficult to characterize  Wide range of informal instruction & settings: Mentor (mentoring) & supervisor (supervising) Research director / thesis advisor / major professor Administrative staff (IRB staff, department administrators…) Colleagues & fellow students  Mentor/thesis advisor may not be major influence Graduate students learn from postdocs Undergraduates learn from graduate students Coleagues may be more important than thesis advisors  Little is known (empirically) about how RCR is shaped by informal “instruction” Will focus on mentors, but there are other influences

6. 6  Grows in importance over time  Plays greatest role in shaping RCR  Poorly planned, organized, and assessed  In some cases, may do as much harm as good Hypotheses ~ informal RCR instruction earlylate importance informal formal

7. 7 I. What should be taught? Key areas? QRP ~ 10% - 50% High or highest standards ?? Could be as high as 1 in 100 Research behavior  Option #1. Professional Standards Focus on what researchers do not appear to know

8. 8 Research misconduct (FFP)  More common than generally assumed  Definitions & procedures not complicated  Accepting responsibility is a problem Reporting suspected misconduct Protecting whistleblowers Granting due process  Challenge Mentors in ideal position to promote RCR re. FFP Mentors are sometimes poor role models

9. 9 Misconduct studies  JM Ranstam, Control Clin Trials (2000) Survey, 442 biostatisticians, 37% response 51% knew about fraud in medical research  26% involved FF  31% directly involved in projects with misconduct Estimates of rate,.69% –>.80% (.25% standard)  Geggie, J Med Ethics (2001) Survey, 305 new medical consultants, 64% response  55.7% observed misconduct (FF lower)  5.7% committed misconduct in the past  18% would commit in future  17% had research ethics training

10. 10 Misconduct studies continued…  Gardner, Contemporary Clinical Trials (2005) Sample ~ authors (549) who reported results of pharmaceutical clinical trials (64% response) 1% reported target article misrepresented the research 5% reported fabrication in study they had participated in 17% knew personally of fabrication in a study  What should be taught; key areas?  Definition of research misconduct!  How to report research misconduct and to whom!  There are no justifications for engaging in misconduct!  Research misconduct is not tolerated in MY research group!  Misconduct is everyone’s responsibility

11. 11 Questionable Research Practices  Definition Less than responsible conduct Falls short of good research practice At odds with “commonly accepted practices”  Examples Misrepresentation of credential Conflicts of interest that impact results/actions Undeserving/improper authorship Sloppy and bias publication practices  What can/should mentors teach?

12. 12 Authorship & publication  Relevant authorship & publication rules  Practical experience learning how to deal with: Who should be an author? One publication or several? Necessary background research? What information is essential? Elements of a complete, accurate citation? Abstract? What constitutes a conflict of interest?  QRPs common (rampant) in research publications  Mentors… May not provide enough practical publication experience Mentors can be poor role models and set poor examples

13. 13 Citation errors  Inaccurate information/claims in notes Grouped as major and minor (or) Citational vs. quotational Rates range from 10% to 30%  Recent findings: FieldAllMajor Otolaryngology (2000)37.511.9 Primary Care/AIDS (2003)3% / 8% Manuel Therapy (2004)20 to 59% citational  Rate appears high & constant

14. 14 Data management & interpretation  Relevant data management & interpretation rules  Practical experience learning how to deal with: Necessary elements of a rigorous design? How data should be collected and stored? Who owns the data and with whom can it be shared? Appropriate and inappropriate statistical analyses? How to handle data that do not fit? What conclusions do the data support? Not support?  Data in publications is often biases and/or unreliable  Mentors… May not provide enough practical experience discussing data Mentors can be poor role models and set poor examples

15. 15 Data practices that concern researchers  Al-Marsouki, Contemporary Clinical Trials 26(2005) Design% Failure to use random allocation92 Failure to specify in the protocol the main outcome measure88 Inadequate allocation concealment84 Different follow-up schedules in arms80 Conduct% Tampering with treatment packs so as to un-blind allocation95 Selective withdrawals on basis of knowledge of allocation92 Data falsification92 Data fabrication92 = Federal definition of misconduct

16. 16 Al-Marsouki, continued…. Analysis% Altering analysis methods until finding a significant result100 Use of battery of methods of comparison to get the right answer100 Altering results in knowledge of allocation100 Excluding patients to exaggerate effects or remove adverse events99 Use of primary outcome measure that was not pre-specified96 Reporting% Failure to report unfavorable results100 Selective reporting of positive results or omission of adverse events96 Selective reporting based on p-values92 Report of subgroup without reference to wide study92

17. 17 Al-Marsouki, Impact & likely to occur Over-interpretation of “ significant" findings in small trials83 Selective reporting based on p-values80 Selective reporting of outcomes in the abstract76 Subgroup analyses done without interaction tests75 Negative or detrimental studies not published68 Putting undue stress on results from subgroup analysis68 Inappropriate subgroup analyses64 Selective reporting of (i) subgroups (ii) outcomes (iii) time points64 Selective reporting of positive results/omission of adverse events data60 Failure to report results or long delay in reporting60 Post-hoc analysis not admitted59 Giving incomplete information about analyses with non significant results56 Analysis conducted by the sponsor of the trial54 Over-interpretation of “ significant" findings in small trials83 Selective reporting based on p-values80 Selective reporting of outcomes in the abstract76 Subgroup analyses done without interaction tests75 Negative or detrimental studies not published68 Putting undue stress on results from subgroup analysis68 Inappropriate subgroup analyses64 Selective reporting of (i) subgroups (ii) outcomes (iii) time points64 Selective reporting of positive results/omission of adverse events data60 Failure to report results or long delay in reporting60 Post-hoc analysis not admitted59 Giving incomplete information about analyses with non significant results56 Analysis conducted by the sponsor of the trial54

18. 18 Impact of COI on results  Bekelman (2003), JAMA Meta-analysis of 37 COI studies (1,000s of trials) Positive correlation (3.60 OR), industry sponsorship & positive outcomes  Lexchin (2003), BMJ Meta-analysis of 30 COI studies Positive correlation (4.05 OR), industry sponsorship & positive outcomes  Friedman (2004) 398 publications, NEJM and JAMA Correlation (2.35-2.64 OR), industry/positive outcomes

19. 19 What needs to be taught? Ten Top Behaviors (Martinson, Nature, June 05)AllMidEarly 1. Falsifying or ‘cooking’ research data0.30.20.5 2. Ignoring major aspects of human-subject requirements0.3 0.4 3. Not properly disclosing involvement in firms whose products are based on one‘s own research 0.30.40.3 4. Relationships with students, research subjects or clients that may be interpreted as questionable 1.41.31.4 5. Using another’s ideas without obtaining permission or giving due credit1.41.71.0 6. Unauthorized use of confidential information1.72.40.8 7. Failing to present data that contradict one’s own previous research6.06.55.3 8. Circumventing certain minor aspects of human-subject requirements7.69.06.0 9. Overlooking others' use of flawed data or questionable interpretation12.512.212.8 10. Changing the design, methodology or results of a study in response to pressure from a funding source 15.520.69.5 = Federal definition of misconduct

20. 20 Martinson, continued…. Other behaviors AllMidEarly 11. Publishing the same data or results in two or more publications 4.75.93.4 12. Inappropriately assigning authorship credit10.012.37.4 13. Withholding details of methodology or results in papers or proposals 10.812.48.9 14. Using inadequate or inappropriate research designs13.514.612.2 15. Dropping observations or data points from analyses based on a gut feeling that they were inaccurate 15.314.316.5 16. Inadequate record keeping related to research projects27.527.727.3  Why do researchers engage in these behaviors? Do not know they are wrong? Have been “taught” that there are justifications for bending the rules?

21. 21 Other key areas for RCR education?  Option #2. Principles and professional responsibility  Essential foundation for RCR Strong sense of professional responsibility Awareness of social responsibility Moral reasoning skills  Mentors ideally situated to develop all three Know and interact with trainees on a personal level Can lead by example – role model Generally oversee trainees entry into professional practice

22. 22 II. Teaching methods  Options open to mentors Role model; set a good example Include RCR in:  Regular and informal meetings with trainees  Laboratory meetings / journal clubs  Seminars and advanced courses  Lab postings & information updates Other ways to influence:  Encourage trainees to attend RCR lectures & programs  Encourage & support attendance at professional meetings  With permission, involve in reviews & other professional activities  Are mentors up to delivering the task at hand?

23. 23 III. Major challenges  Mentors are not trained to be RCR mentors 10-20% of researchers have some RCR education Graduate education focused on research not teaching  Mentoring seldom is planned & organized No lesson plan or course of study No common content or curriculum  Mentoring skills are not evaluated / rewarded Hiring & promotion based on research not teaching  Outcomes of mentoring are not assessed Evaluate research skills, not professional knowledge Professional knowledge of trainees is not assessed

24. 24 How can RCR mentoring be improved?  Research institutions should rigorously assess mentoring skills and reward accordingly  NIH & NSF should encourage (require?) training programs to introduce RCR across the curriculum and as part of mentoring  RCR educators should pay more attention to mentoring Currently focus on courses, seminars, web pages and other formal instruction Need better resources to help mentors provide RCR education  More information on the strengths and weaknesses of the role mentoring plays in foster RCR

25. 25