Ethical Conduct of Research Ran Libeskind-Hadas Harvey Mudd College Four handouts!

Why talk about ethics? Our credibility as scientists and engineers depends on applying high ethical standards False or unreliable results set science back Funding sources rightfully demand that work be conducted ethically

Why talk about ethics? Our credibility as scientists and engineers depends on applying high ethical standards False or unreliable results set science back Funding sources rightfully demand that work be conducted ethically

Why talk about ethics? Our credibility as scientists and engineers depends on applying high ethical standards False or unreliable results set science back Funding sources rightfully demand that work be conducted ethically

Why talk about ethics? Our credibility as scientists and engineers depends on applying high ethical standards False or unreliable results set science back Funding sources rightfully demand that work be conducted ethically And agencies such as the Federal Office of Research Integrity investigate and prosecute violations…

Ethical issues at all levels… Fraud Bad judgment Errors

Fraud

Ethical issues at all levels… Fraud Bad judgment Errors O. Uplavici, Predbezna Zprava, 1887 cited for over 50 years 1939: “Dr. O. Uplavici ( )” by Clifford Dobell Jaroslav Hlava

What are the issues? Conflicts of interest Experimental Design and Conduct – Appropriate use of human subjects – Subjectivity and perception bias Data Integrity and Analysis – Acquisition – Management – Appropriate use of statistical tools Publication – Fabrication – Falsification – Selective reporting – Plagiarism – Authorship – Citation … and more

What are the issues? Conflicts of interest Experimental Design and Conduct – Appropriate use of human subjects – Subjectivity and perception bias Data Integrity and Analysis – Acquisition – Management – Appropriate use of statistical tools Publication – Fabrication – Falsification – Selective reporting – Plagiarism – Authorship – Citation … and more

What are the issues? Conflicts of interest Experimental Design and Conduct – Appropriate use of human subjects – Subjectivity and perception bias Data Integrity and Analysis – Acquisition – Management – Appropriate use of statistical tools Publication – Fabrication – Falsification – Selective reporting – Plagiarism – Authorship – Citation … and more

What are the issues? Conflicts of interest Experimental Design and Conduct – Appropriate use of human subjects – Subjectivity and perception bias Data Integrity and Analysis – Acquisition – Management – Appropriate use of statistical tools Publication – Fabrication – Falsification – Selective reporting – Plagiarism – Authorship – Citation … and more

Examples of Conflicts of Interest (COI) You have research funding from a company and they would like your results to show X (or not show X). You are asked to recommend which instrument to purchase and you own stock in one of the companies. You are involved in a hiring decision and a friend of yours has applied for the position. Your employer owns some equipment that you would like to use to do some work for your start-up. You have been given a paper to review that in some way reduces the value of your prior work.

Examples of Conflicts of Interest (COI) You have research funding from a company and they would like your results to show X (or not show X). You are asked to recommend which instrument to purchase and you own stock in one of the companies. You are involved in a hiring decision and a friend of yours has applied for the position. Your employer owns some equipment that you would like to use to do some work for your start-up. You have been given a paper to review that in some way reduces the value of your prior work.

Examples of Conflicts of Interest (COI) You have research funding from a company and they would like your results to show X (or not show X). You are asked to recommend which instrument to purchase and you own stock in one of the companies. You are involved in a hiring decision and a friend of yours has applied for the position. Your employer owns some equipment that you would like to use to do some work for your start-up. You have been given a paper to review that in some way reduces the value of your prior work.

Examples of Conflicts of Interest (COI) You have research funding from a company and they would like your results to show X (or not show X). You are asked to recommend which instrument to purchase and you own stock in one of the companies. You are involved in a hiring decision and a friend of yours has applied for the position. Your employer owns some equipment that you would like to use to do some work for your start-up. You have been given a paper to review that in some way reduces the value of your prior work.

Examples of Conflicts of Interest (COI) You have research funding from a company and they would like your results to show X (or not show X). You are asked to recommend which instrument to purchase and you own stock in one of the companies. You are involved in a hiring decision and a friend of yours has applied for the position. Your employer owns some equipment that you would like to use to do some work for your start-up. You have been given a paper to review that in some way reduces the value of your prior work. Ran’s current situation

COI Safeguards Most institutions have explicit COI policies and disclosure forms. Declaration of competing interests Please complete a declaration of competing interests, considering the following questions: - Have you in the past five years received reimbursements, fees, funding, or salary from an organisation that may in any way gain or lose financially from the publication of this manuscript, either now or in the future? - Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this manuscript, either now or in the future? - Do you hold or are you currently applying for any patents relating to the content of the manuscript? Have you received reimbursements, fees, funding, or salary from an organization that holds or has applied for patents relating to the content of the manuscript? - Do you have any other financial competing interests? - Do you have any non-financial competing interests in relation to this paper? If you can answer no to all of the above, write 'I declare that I have no competing interests' below. If your reply is yes to any, please give details below.

Experimental Design & Conduct Your experiment involves human subjects. Your experimental design is complicated and not fully documented. Your experimental design has the possibility of human subjectivity and bias.

Safeguards Human Subjects: Any study that involves human subjects must be reviewed by an IRB (“Institutional Review Board”) prior to commencing the study. Experimental design is ideally peer reviewed in published findings (but even then, subtle details are difficult to document and often missed).

Data Integrity and Analysis Data is not recorded reliably. Multiple contributors to dataset using ambiguous standards. Sensitive data is not secured or authenticated. Which statistical tests to use is ambiguous. Questionable protocols for throwing out “outliers”.

Data Integrity and Analysis Data is not recorded reliably. Multiple contributors to dataset using ambiguous standards. Sensitive data is not secured or authenticated. Which statistical tests to use is ambiguous. Questionable protocols for throwing out “outliers”.

Data Integrity and Analysis Data is not recorded reliably. Multiple contributors to dataset using ambiguous standards. Sensitive data is not secured or authenticated. Which statistical tests to use is ambiguous. Questionable protocols for throwing out “outliers”.

Data Integrity and Analysis Data is not recorded reliably. Multiple contributors to dataset using ambiguous standards. Sensitive data is not secured or authenticated. Which statistical tests to use is ambiguous. Questionable protocols for throwing out “outliers”.

Data Integrity and Analysis Data is not recorded reliably. Multiple contributors to dataset using ambiguous standards. Sensitive data is not secured or authenticated. Which statistical tests to use is ambiguous. Questionable protocols for throwing out “outliers”. True Story

Safeguards Some large institutions have auditors who perform routine or reactive audits.

Publication Some closely-related work was done earlier by others and is not properly cited. Some of this work was already published elsewhere. Selective reporting: Some of the data or studies that don’t support the conclusions are omitted. Some co-authors did not contribute to the work. Citations are copied from earlier papers and not verified.

Publication Some closely-related work was done earlier by others and is not properly cited. Some of this work was already published elsewhere. Selective reporting: Some of the data or studies that don’t support the conclusions are omitted. Some co-authors did not contribute to the work. Citations are copied from earlier papers and not verified.

Publication Some closely-related work was done earlier by others and is not properly cited. Some of this work was already published elsewhere. Selective reporting: Some of the data or studies that don’t support the conclusions are omitted. Some co-authors did not contribute to the work. Citations are copied from earlier papers and not verified.

Publication Some closely-related work was done earlier by others and is not properly cited. Some of this work was already published elsewhere. Selective reporting: Some of the data or studies that don’t support the conclusions are omitted. Some co-authors did not contribute to the work. Citations are copied from earlier papers and not verified.

Publication Some closely-related work was done earlier by others and is not properly cited. Some of this work was already published elsewhere. Selective reporting: Some of the data or studies that don’t support the conclusions are omitted. Some co-authors did not contribute to the work. Citations are copied from earlier papers and not verified.

Safeguards Journals and conf. proceedings generally have clear policies about what may be submitted. Peer review will catch some errors. Some journals require a statement on how each author contributed. Errors found by others can be published.

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

Some Tests… – Harm test: Do the benefits outweigh the harms, short term and long term? – Reversibility test: Would I still think this choice is good if I traded places? – Common practice test: What if everyone behaved in this way? – Legality test: Would this choice violate a law or a policy of my employer? – Colleague test: What would professional colleagues say? – Wise relative test: What would my wise old aunt or uncle do? – Mirror test: Would I feel proud of myself when I look into the mirror? – Publicity test: How would this choice look on the front page of a newspaper? When in doubt, talk to (several) people that you trust

What are the issues? Conflicts of interest Experimental Design and Conduct – Use of human subjects – Subjectivity and perception bias Data Integrity – Acquisition – Management Publication – Fabrication – Falsification – Selective reporting – Appropriate use of statistical tools – Plagiarism – Authorship – Citation … and more “The Truth Wears Off” Case Study 1: Reviewing a Manuscript Case Study 2: Grant Review Case Study 3: Kinky Data

Research misconduct Handling of data

Case Study 1: Reviewing a Manuscript Researchers are frequently asked to review manuscripts for journals as a professional courtesy Manuscripts under review are considered confidential

Questions Is Prof. Dvizer’s behavior here ethical? Why or why not? Are there some things that Prof. Dvizer could have done differently in order to mitigate your concerns? Should Sue have declined to review the manuscript? Why or why not?

Here’s what happened next… Sue read Litik’s paper and gave a thoughtful and constructive review Litik’s paper was rejected by the editor, based on strong criticism from other reviewers on a part of the paper not relevant to Sue’s work and not in Sue’s area of expertise or interest Sue now plans to submit a manuscript based on some of the ideas from Litik’s paper and her own work but had no way to cite Litik’s work What should Sue do?

Case Study 2: Reviewing a Grant Proposal Established researchers are asked to review grant proposals (NSF, NIH, HHMI, etc.) A grant can make a large impact on a research program, and a particularly large impact on a junior faculty member’s career

Questions What’s your advice to Polly? Is there any additional information that you feel that you’d like to have before being absolutely certain about the right way to proceed? Are there any slight changes in the circumstances under which your advice would be different?

Case Study 3: Kinky Data (adapted from a real case by Dr. Adam Johnson) Integrity and presentation of data

Questions What’s your advice to Prof. Lemma? Can you imagine any slight differences in the scenario which would result in very different advice?

“Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It’s a little bit off, because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of the electron, after Millikan. If you plot them as a function of time, you find that one is a little bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher. Why didn’t they discover that the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan’s value they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that.” - Richard Feynman, Caltech Commencement address 1974 Selective Reporting and Bias

“The Truth Wears Off” (John Lehrer, New Yorker Magazine, December 13, 2010)

Discussion Topics What’s wrong? – The scientific method? – Experimental design? – Human bias? – The use of statistics?

More Resources On dealing with fraud and whistle-blowing: – Google “NIH the lab” On bias, experimental design, and misuse of statistics – “Of Beauty, Sex and Power” by Andrew Gelman and David Weakliem, American Scientist, 2009 On a variety of research ethics topics… – Google “engineering ethics UIUC”