Causality, Reasoning in Research, and Why Science is Hard Sources: D. Jensen. “Research Methods for Empirical Computer Science.” William M.K. Trochim. “Research Methods Knowledgebase”

More on Causality What is causality?

What’s Important About Causality? Explanation Association provides prediction, but not explanation Identifying causal mechanisms may uncover underlying reasons for relationships Control Understanding causality allows us to predict the effects of actions without performing them Allows more efficient exploration of the space of possible solutions

Conditions for Causal Inference

Problems with Association

Are Feathers Associated with Flight? Do they have a casual relationship with the ability to fly?

Related Fallacies Common (Questionable) Cause Fallacy Post Hoc Fallacy This fallacy has the following general structure: A and B are regularly associated (but no third, common cause is looked for). Therefore A is the cause of B. Called “Confusing Cause and Effect” fallacy, if in fact, there is not common cause for A and B Post Hoc Fallacy A Post Hoc is a fallacy with the following form: A occurs before B.

Eliminating Common Causes

Control

Randomization

Modeling

Reasoning Methodologies in Research

Types of Reasoning in Research

Deductive vs. Inductive Methodologies

What is Abduction?

Examples of Abductive Reasoning A Medical Diagnosis Given a specific set of symptoms, what is the diagnosis that would best explain most of them? Jury Deliberations in a Criminal Case Jurors must consider whether the prosecution or the defense has the best explanation to cover all of the evidence No certainty about the verdict, since there may exist additional evidence that was not admitted in the case Jurors make the best guess based on what they know

“… when you have eliminated the impossible, whatever remains, however improbable, must be the truth.” - Sherlock Holmes

Abductive Reasoning in Science Abduction selects from among the hypotheses being considered, the one that best explains the evidence Note that this requires that we consider multiple alternative hypotheses Abductive Reasoning is closely related to the statistical method of Maximum Likelihood Estimation Possible threats to validity Small hypothesis spaces Small amounts of evidence to explain

Challenges in Abductive Reasoning Creating hypothesis spaces likely to contain the “true” hypothesis Approach: create large hypothesis spaces Knowing when more valid hypotheses are missing from the hypothesis space Approach: constantly evaluate and revise the hypothesis space (multiple working hypotheses) Creating good sets of evidence to explain Approach: seek diverse and independent evidence with which to evaluate hypotheses

Why use multiple working hypotheses? Objectivity: Helps to separate you from your hypotheses; shift from personal investment in hypotheses to testing the hypotheses Focus: Reinforces a focus in falsification rather than confirmation Efficiency: Allows experiments to be designed to distinguish among competing hypotheses rather than evaluating a single one Harmony: Limits the potential for professional conflict and rejection because all hypotheses are considered and evaluated

“Strong Inference” John R. Platt, Science, October 1964 “Strong Inference - Certain systematic methods of scientific thinking may produce much more rapid progress than others.” Not all science/research is created equal Don’t confuse research activity with effective research Activity: building systems; proving theorems; conducting surveys; writing and publishing articles; giving talks; obtaining grants Research: improved predictions; better understanding of relationships; improved control of computational artifacts Many researchers are active; only a subset do effective research

Initial Questions for “Strong Inference”

Arguments and Fallacies Aside from general reasoning methodologies, one must ensure the validity of all arguments used in any research endeavor An argument Consists of one or more premises and a conclusion A premise is a statement (a sentence that is either true or false) that is offered in support of the claim being made, which is the conclusion (also a sentence that is either true or false) Modus Ponens (and Modus Tollens) A fallacy Generally, an error in reasoning (differs from a factual error), An "argument" in which a logically invalid inference is made (deductive) or the premises given for the conclusion do not provide the needed degree of support. (inductive)

Common Fallacies Ad Hominem Appeal to Authority Appeal to Belief Appeal to Common Practice Appeal to Popularity Begging the Question Biased Sample Hasty Generalization Ignoring A Common Cause Burden of Proof Straw Man See: http://en.wikipedia.org/wiki/Fallacy http://www.nizkor.org/features/fallacies/

Why is Science Hard? [Notes]