The Scientific Method

Common Mistakes in Applying the Scientific Method The scientist prefers one outcome over another (bias) Ignoring or ruling out data that do not support the hypothesis; tendency to find something “wrong” with evidence that does not support the hypothesis (not treating all data in the same way) Failure to estimate quantitatively systematic errors Faulty interpretation of statistical data

Bad conclusions can be drawn when there is… Selection bias: a distortion of evidence or data that arises from the way the data are collected Reversed burden of proof: burden of proof should rest on those making a claim, not on the critic Assertion that claims which have not been proven false must be true (and vice versa)

Correlation vs Causation Correlation implies causation is a logical fallacy by which two events that occur together are claimed to be cause and effect. It is dangerous to: ignoring the possibility that the correlation is coincidence or that both correlated events have another common cause draw conclusions about causation from statistical correlations. If you only have A and B, a correlation between them does not let you infer A causes B, or vice versa.

The Simpsons (Season 7, “Much Apu about Nothing”) Homer: Not a bear in sight. The “Bear Patrol” must be working like a charm! Lisa: That’s specious reasoning, Dad. Homer: Thank you, dear. Lisa: By your logic I could claim that this rock keeps tigers away. Homer: Oh, how does it work? Lisa: It doesn’t work. Homer: Uh-huh. Lisa: It’s just a stupid rock. But I don’t see any tigers around, do you? Homer: Lisa, I want to buy your rock.

Case Study: Willie Soon Associated with the George C. Marshall and Fraser Institutes Has received funding from the American Petroleum Institute (this is acknowledged in his main paper) While it doesn’t prevent him from doing good science, this calls into question his ability to be unbiased

Claim: “[T]he 20th century is not unusually warm or extreme.” This was not based on a quantitative analysis. The authors considered anomalous conditions to be warm, cold, wet or dry relative the 20th century. The relationship between wet/cold and warm/dry is not 1:1. Authors do not properly distinguish between local and global temperature changes No attempt was made to estimate the errors in using proxy data No way to resolve short time scales

Claim: CO2 has nothing to do with global climate change The authors can only responsibly claim one point in their paper They have a model with two free parameters They can fit these parameters to global temperature change data In their model, their best fit includes a higher contribution from the sun than green house gases Just because the data fits a model does not imply the model is true. (One would have to test all possible models in order to prove that any one model was true.) A convincing argument would entail performing a simulation that produced output that matched the data without fitting *Acknowledged funding sources in paper: Electric Power Research Institute, Mobil Foundation, Texaco Foundation and American Petroleum Institute

Some useful references about Correlation and Causation… http://www.onpedia.com/encyclopedia/Correlation-implies-causation-(logical-fallacy) http://www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml http://teacher.pas.rochester.edu/phy_labs/appendixe/appendixe.html http://en.wikipedia.org/wiki/Pseudoscience http://www.venganza.org/ Willie Soon Papers and Rebuttal http://adsabs.harvard.edu/full/1996ApJ...472..891S http://www.cfa.harvard.edu/~wsoon/1000yrclimatehistory-d/Jan30-ClimateResearchpaper.pdf http://www.atmos.washington.edu/~davidc/ATMS211/articles_optional/Mann_on_Soon2003.pdf