Hypothesis Testing and Statistical Significance

 Estimators and Correlation  Hypothesis Testing and Statistical Significance  labels and other questions 2

3 General definition, continuous and discrete variables: For discrete variables:

4  What is an estimator?  Often a trade-off between bias and variance

5 (Typeset equations courtesy Variance defined: Population variance: (have all obs 1…N) Two estimators of population variance:

6 vs. (Typeset equations courtesy

7

 Standard Deviation  Spread of a list  Single variables have SD 8  Standard Error  Spread of a chance process  Sampling Distributions have SE Graphics: Wikipedia

9

 Remember that a z-score tells us where a score is located within a distribution– specifically, how many standard deviation units the score is above or below the mean. 10

 For example, if we find a particular difference that is x standard errors wide, how confident are we that the difference is not just due to chance?  So… we can use z-scores on a normal curve to interpret how likely a given outcome is (how likely is it due to chance?) 11

 Example, you have a variable x with mean of 500 and S.D. of 15. How common is a score of 525?  Z = /15 = 1.67  If we look up the z-statistic of 1.67 in a z-score table, we find that the proportion of scores less than our value is  Or, a score of 525 exceeds.9525 of the population. (p <.05) 12

 z is a test statistic  More generally: z = observed – expected SE  Z tells us how many standard errors an observed value is from its expected value. 13

 A confidence interval is a range of scores above and below the mean.  The interval is in standard errors  It is the interval where we expect our value to be  A confidence coefficient is the likelihood that a given interval has the true value of the parameter  Sample value = true population value + error 14

 One-tailed  Directional Hypothesis  Probability at one end of the curve  Two-tailed  Non-directional Hypothesis  Probability is both ends of the curve 15

 Null Hypothesis:  H 0 : μ 1 = μ c ▪ μ 1 is the intervention population mean ▪ μ c is the control population mean 16 Alternative Hypotheses:  H 1 : μ 1 < μ c  H 1 : μ 1 > μ c

 Null Hypothesis:  H 0 : μ 1 = μ c ▪ μ 1 is the intervention population mean ▪ μ c is the control population mean 17 Alternative Hypothesis:  H 1 : μ 1 ≠ μ c

 Do Berkeley students read more or less than 8 hours a week?  H 0 : μ = 8 The mean for Berkeley students is equal to 8  H 1 : μ ≠ 8 The mean for Berkeley students is not equal to 8 18

 Do Berkeley students read more than 8 hours a week (the average for students across the country)?  H 0 : μ = 8 There is no difference between Berkeley students and other students  H 1 : μ > 8 The mean for Berkeley students is higher than the mean for all students 19

 A p-value is the observed significance level (more on this in a moment)  A test statistic depends on the data, as does p.  This chance assumes that the null hypothesis is correct. Thus, the smaller the chance (p-value), the morel likely that the null can be rejected.  The choice of a test statistic (e.g., z, t, F, Χ 2 ) depends on the model and they hypothesis being considered  The basic process is exactly the same, however. 20

 When p value >.10 → the observed difference is “not significant”  When p value ≤.10 → the observed difference is “marginally significant” or “borderline significant”  When p value ≤.05 → the observed difference is “significant”  When p value ≤.01 → the observed difference is “highly significant” 21

 We cannot hypothesize the null  As odd as it may seem at first, we reject or do not reject the null; a traditional hypothesis test tests against the null.  We never use the word proof with hypothesis testing and statistics, we reject or accept.  Prove has a specific meaning in mathematics and philosophy, but the term is misleading in statistics. 22

 Type I Error: falsely rejecting a null hypothesis (false positive)  Type II Error: Failing to reject the null hypothesis when it is false (false negative) 23

(The auto data) 24