1. "Classical" Inference

2. Two simple inference scenarios Question 1: Are we in world A or world B?

3. Possible worlds: World A World B Xnumberadded [-.5,.5]38 [-1, 1]6830 [-1.5, 1.5]8719 [-2, 2]958 [-2.5, 2.5]994 (- ∞, ∞)1001 Xnumberadded [4, 6]38 [3, 7]6830 [2, 8]8719 [1, 9]958 [0, 10]994 (- ∞, ∞)1001

5. Jerzy Neyman and Egon Pearson

6. Correct acceptance of H 0 pr(D= H 0 | T=H 0 ) = (1 –  ) Type I Error pr(D= H 1 | T=H 0 ) =  [aka size] Type II Error pr(D= H 0 | T=H 1 ) =  Correct acceptance of H 1 pr(D= H 1 | T=H 1 ) = (1 –  ) [aka power] D : Decision in favor of: H 1 : Alternative Hypothesis H 0 : Null Hypothesis T : The Truth of the matter: H 1 : Alternative Hypothesis

7. Definition. A subset C of the sample space is a best critical region of size α for testing the hypothesis H 0 against the hypothesis H 1 if and for every subset A of the sample space, whenever: we also have:

8. Neyman-Pearson Theorem: Suppose that for for some k > 0: 1. 2. 3. Then C is a best critical region of size α for the test of H 0 vs. H 1.

9. 9 When the null and alternative hypotheses are both Normal, the relation between the power of a statistical test (1 –  ) and  is given by the formula  is the cdf of N(0,1), and q  is the quantile determined by .  fixes the type I error probability, but increasing n reduces the type II error probability

10. Question 2: Does the evidence suggest our world is not like World A?

11. World A Xnumberadded [-.5,.5]38 [-1, 1]6830 [-1.5, 1.5]8719 [-2, 2]958 [-2.5, 2.5]994 (- ∞, ∞)1001

12. Sir Ronald Aymler Fisher

13. Fisherian theory Significance tests: their disjunctive logic, and p-values as evidence: ``[This very low p-value] is amply low enough to exclude at a high level of significance any theory involving a random distribution….. The force with which such a conclusion is supported is logically that of the simple disjunction: Either an exceptionally rare chance has occurred, or the theory of random distribution is not true.'' (Fisher 1959, 39)

14. Fisherian theory ``The meaning of `H' is rejected at level α' is `Either an event of probability α has occurred, or H is false', and our disposition to disbelieve H arises from our disposition to disbelieve in events of small probability.'' (Barnard 1967, 32)

15. Fisherian theory: Distinctive features Notice that the actual data x is used to define the event whose significance is evaluated. Also based on H 0 and H 1 Can only reject H 0, evidence cannot allow one to accept H 0. Many other theories besides H 0 could also explain the data.

16. Common philosophical simplification: Hypothesis space given qualitatively; H 0 vs. –H 0, Murderer was Professor Plum, Colonel Mustard, Miss Scarlett, or Mrs. Peacock More typical situation: Very strong structural assumptions Hypothesis space given by unknown numeric `parameters' Test uses: a transformation of the raw data, a probability distribution for this transformation (≠ the original distribution of interest)

17. Three Commonly Used Facts Assume is a collection of independent and identically distributed (i.i.d.) random variables. Assume also that the X i s share a mean of μ and a standard deviation of σ.

18. Three Commonly Used Facts For the mean estimator : 1. 2.

19. Three Commonly Used Facts The Central Limit Theorem. If {X 1,…, X n } are i.i.d. random variables from a distribution with mean  and variance  2, then: 3. Equivalently:

20. Examples Data: January 2012 CPS Sample: PhD’s, working full time, age 28- 34 H 0 : mean income is 75k

21. 21996.00 89999.52 119999.9 40999.92 67600.00 68640.00 96999.76 77296.96 65000.00 71999.72 100100.0 45999.72 149999.7 19968.00 10140.00 37999.52 74999.60 69992.00 31740.80 65000.00 57512.00 87984.00 35999.60 38939.68 99999.64 74999.60 149999.7 47996.00 62920.00 54999.88 104000.0

22. Hyp. Value Probability H 0 -1.024022 0.3138

23. Comments The background conditions (e.g., the i.i.d. condition behind the sample) are a clear example of `Quine-Duhem’ conditions. When background conditions are met, ``large samples’’ don’t make inferences ``more certain’’ Multiple tests Monitoring or ``peeking'‘ at data, etc.

24. Point estimates and Confidence Intervals

25. Many desiderata of an estimator: Consistent Maximum Likelihood Unbiased Sufficient Minimum variance Minimum MSE (mean squared error) (most) efficient

26. By CLT: approximately: Thus: By algebra: So:

27. Interpreting confidence intervals The only probabilistic component that determines what occurs is. Everything else are constants. Simulations, examples Question: Why ``center’’ the interval?

28. Confidence Intervals $68,898.16 ± $12,152.85 ``C.I. = mean ± m.o.e’’ = ($56,745.32, $81,051.01)

29. Using similar logic, but different computing formulae, one can extend these methods to address further questions e.g., for standard deviations, equality of means across groups, etc.

30. Equality of Means: BAs SexCountMeanStd. Dev. 122363619.5431370.01 220951395.4325530.66 All43257705.5629306.13 ValueProbability 4.4249430.0000

31. Equality of Means: PhDs SexCountMeanStd. Dev. 12166452.7136139.78 21173566.7629555.10 All3268898.1633707.49 ValueProbability -0.5607450.5791