1 CONFIRMATION BIAS

2 Confirmation Bias: A Ubiquitous Phenomenon in Many Guises –Raymond Nicholson, TUFTS –Review of General Psychology –50+ pages –non-technical –7 full pages of references Francis Bacon (1620) –“…great and pernicious predetermination…” Poyla (1954) –“…scientific thought as distinguished from everyday thought.”

3 BIAS Treatment and Evaluation of Evidence Building a Case **no expectation of impartiality Impartial Evaluation Motivated Bias Unmotivated Bias Philosophy Logic Statistics Psychology Hempel (1945) Observed: A white shoe Concluded: All ravens are black. Logic: All observed non-black things are non-ravens (the Contrapositive) Silly? All these insurgents are Muslim…

4 Incarnations of Confirmation Bias Evidence interpretation Evidence censoring Restriction of alternative hypotheses Seeking confirmatory evidence Reification – over-estimating the salience of a taxonomy Illusory Correlation Primacy – becoming biased by the beginning of a stream of evidence, early hypothesis building

5 Wasson(1966) Four cards presented. Cards have a number on one side and a letter on the other. Confirm or deny: All vowels have even numbers on the other side. AB47 Most subjects (~90%) flip “A” and/or “4” People seek to confirm the hypothesis, they do not seek countervailing evidence.

6 Taylor and Smyth (1860’s) Egyptologists and Numerologists, they discover “too strange” relationships in Pyramid dimensions. –(2 x base)/height =  –base/casing stone = 365 –height * 10 9 = distance from earth to sun –density of the earth –period of the procession of the earth’s axis –mean temp of the earth’s surface Conclusion: Space aliens directed the construction of the Pyramids

7 Einstein and the Expanding Universe Universe assumed static until mid-50’s Expanding universe model could have been discovered by any post-Newton astronomer (S. Hawking, 1988) Einstein actually had evidence of expansion as a by-product of General Relativity, but covered it with his now-infamous Cosmological Constant. Called this the biggest mistake of his career.

8 Type 1 Error recall P[type 1] = P[reject Ho|Ho] =  Typical 1-tailed test runs on  = In 40 independent tests, you EXPECT one false rejection Sequential testing is a great temptation in simulation analysis Ability to make STDERR arbitrarily small means that meaningful indifference zone needs to be determined a priori.

9 Confirmation Bias Our leaders and customers have it Our families and neighbors have it We have it, but we can protect ourselves