1. MUSC Biomedical Trainee Retreat on the Responsible Conduct of Research
“Data Management, Record Keeping, and Experimental Rigor”
Ed Krug
BE101
12/07/16

2. Which of these 3 issues has been the greatest frustration for you in the following – give an example:
maintaining your own research records
troubleshooting problems in your own experiments/studies
reproducing the work of others (either from your lab or from the literature)

3. Studies show a very low reproducibility for articles published in scientific journals
The biotech company Amgen had a team of about 100 scientists trying to reproduce the findings of 53 “landmark” articles in cancer research published by reputable labs in top journals. Only 6 of the 53 studies were reproduced. [Nature 483: (2012)]
Scientists at the pharmaceutical company, Bayer, examined 67 target-validation projects in oncology, women’s health, and cardiovascular medicine.  Published results were reproduced in only 14 out of 67 projects.[Nature Reviews Drug Discovery 10:712 (2011)]
However, lack of reproducibility does not necessarily mean results are false.

4. According to the NIH … (RFA-GM-15-006)
“The lack of reproducibility is not due primarily to intentional fabrication or falsification of data. Rather, in many cases there is a lack of awareness or adherence to sufficiently high standards in the planning and execution of scientific experiments, and in transparency in the reporting of science.  Examples include inherently weak experimental designs and over-interpretation of statistically marginal differences and variability of materials, differing brands or even lots of reagents, differing or drifting strains of organisms and cells in culture, or other variables that have not been adequately controlled.”

5. According to the NIH … (RFA-GM-15-006)
“Graduate students were often significantly dependent on the mentor or the mentor's lab for the training received, and postdoctoral fellows were primarily dependent on the mentor or mentor's lab at all institutions.  Rather than being learned in prescribed curricula, training in good laboratory practices that influence data reproducibility appears to be largely passed down from generation to generation of working scientists, with substantial variation from laboratory to laboratory.”

7. Suggested Best Practices to Enhance Rigor and Reproducibility
Steward and Balice-Gordon. Neuron 84: (2014)

8. Suggested Best Practices to Enhance Rigor and Reproducibility
Steward and Balice-Gordon. Neuron 84: (2014)

9. Essential Characteristics of a Research Notebook
Bound pages
Number every page - index in front
Table of Contents
Abbreviations list
Relevant sections - title, purpose, materials & methods, results, conclusions, notes
Purpose and M&M written before starting the experiment
Includes reagent details, e.g. mfr, lot number, location, etc.
Cross-referenced to computer files, images, etc.
Cross-referenced with collaborator’s notebook
Sufficient clarity that your mentor and another trainee could understand and replicate what you have done!
Notebooks are university property!
stays with PI – 3 years past acceptance of final financial statement (6 years from end of grant to be safe)

10. “Data” is essentially all that would be required to replicate your study
1) Physical records
lab notebook
instrument printouts
images
computer analysis
lab meeting discussion
probes and cells
2) Analysis of primary data
data selection
statistical analysis
calculations
normalization
3) Presentation of results
graphs
tables
pictures
Figures
4) Publication
posters
seminars
manuscripts
funding requests

11. Give a copy of this inventory to your mentor and any collaborators.
I strongly suggest a listing of all notebook and page numbers corresponding to any data used for a poster, manuscript or funding application!
Give a copy of this inventory to your mentor and any collaborators.

12. NIH Data Sharing Policy
“The NIH expects and supports the timely release and sharing of final research data from NIH-supported studies for use by other researchers.” NOT-OD
“Definition of Research Data: Recorded factual material commonly accepted in the scientific community as necessary to validate research findings. It does not include preliminary analyses; drafts of scientific papers; plans for future research; peer reviews; communications with colleagues; physical objects (e.g., laboratory samples, audio or video tapes); trade secrets; commercial information; materials necessary to be held confidential by a researcher until publication in a peer-reviewed journal; information that is protected under the law (e.g., intellectual property); personnel and medical files and similar files, the disclosure of which would constitute an unwarranted invasion of personal privacy; or information that could be used to identify a particular person in a research study.”

13. Image Processing Issues
“Data may be excluded from the experimental results only if you have a sound reason to do so!”
Mother Nature
indicate justification in your notebook for any observations excluded for an experiment

14. Availability of image processing tools has led to increase in falsification of image data
Gilbert. Nature doi: /news

15. Be transparent in including image processing details in reporting your research
The Journal of Cell Biology:
25% of accepted papers have at least 1 figure with undocumented processing (applied to both controls and experimental images)
revokes the acceptance of about 1% of its papers due to inappropriate image manipulation (differential modifications with intention to deceive)
Image manipulation precludes alternative interpretation of the data in the future.
Rossner, M. (2006). The Scientist 20:24-25.

16. Brightness -Contrast Adjustment
Journals routinely scan all images in a manuscript
As photographed
Brightness -Contrast Adjustment

17. Manipulated pixels can be easily detected

18. Was this manipulation necessary?
Rossner, M. and Yamada, K. (2004). J. Cell Biol. 166:11-15.

19. All data have signatures – even gels

20. “Innovative approaches” to data presentation are not allowed!

21. Sometimes image manipulations are not so easily detected
As photographed
Find the changes (3)

22. Nature, Instructions to Authors
What CAN I do?
Nature, Instructions to Authors
List all image acquisition and processing tools and software
Document key image-gathering setting and manipulations in the Supplemental materials
Clearly demarcate borders between images collected at different times
Avoid use of touch up tools or deliberately obscuring parts of an image
Processing is acceptable only if applied across the entire image
Be prepared to deliver the original, unprocessed images to the editor

23. Brightness and contrast adjustments must be applied equally across all images being compared
Rossner, M. (2006). The Scientist 20:24-25.

24. You can only adjust brightness and contrast to the point where you do not lose data
Rossner, M. and Yamada, K. (2004). J. Cell Biol. 166:11-15.