1. Hypothesis Testing

2. Distribution of Estimator To see the impact of the sample on estimates, try different samples Plot histogram of answers –Is it “normal” Key point: even if we have the correct model we could get an answer that is way off just because we are unlucky in the sample. –Sampling error How do we know if we have been unlucky? How can we minimise the chances of bad luck?

3. Hypothesis Testing Non-trivial because of sampling distribution –See examples Is the evidence consistent with the hypothesis being true? General approach –what would the distribution look like if the (null) hypothesis was true? –Where on the distribution is our estimate? –How likely is our estimate to occur when the hypothesis is true? –How likely is the hypothesis to be true?

4. A Criminal Trial Metaphor of a criminal trial We have the hypothesis that the accused is innocent We ask if the evidence is consistent with the hypothesis being true If not we can reject the hypothesis If yes we fail to reject –Note: don’t “accept” the hypothesis

5. The Distribution of  Typically we will not observe the distribution, we will have only one estimate We could create the distribution –Lots of small samples: but loose consistency Classical approach is to assume that the distribution is “normal” –i.e. use stylized distribution in place of the histograms

6. Normality The assumption of normality is made for convenience. Is it reasonable? Two ways in which it could be true –Data could be normal –Central limit theorem applies Data is normal if residual is normal –Bell shape is reasonable –Normal is more restrictive: approximation? –Since OLS estimator is a linear function of data it will also have a normal distribution –This is a small sample property i.e. applies for all sizes

7. CLT Central limit theorem states that even if the residuals are not normal OLS will be Process of applying the OLS formula creates a new random variable (the estimator) that has a normal distribution. –See example This is an asymptotic property i.e. for large samples Implications for Classical hypothesis testing –Data must be normal –Or you must have lots of it (100+ obs)

8. Distribution of  OLS OLS is unbiased so the distribution is centered on the true value Its variance depends on the variance of the residuals –Estimate produced by stata Using either the CLT or the normality of u,  OLS is normally distributed

10. Mechanism for Hypothesis Test 1.State the Hypothesis we want to test H 0 :  MPC = 0.7 H 1 :  MPC ≠ 0.7 2.Calculate the distribution of  OLS assuming that H 0 is true. 3.Find our estimate on the distribution 4.What is the probability that our estimate would have come from this distribution? 5.Does this lead us to believe the null hypothesis?

12. Any estimate is possibly consistent with any hypothesis –Always the possibility of a real fluke even with no mistakes –But some are clearly more likely than others We can measure the probability of an estimate occurring if we know the distribution of the estimator Clearly we are unlikely to get an estimate of 0.72 if the true value is 0.7 –But it is possible! –P( b OLS ≥0.72|  MPC =0.7)=0.000001 –Usually use 5%,10% or 1% as the threshold value

13. Another Example Use the individual consumption data 1.State the Hypothesis we want to test H 0 :  MPC = 0.81 H 1 :  MPC ≠ 0.81 1.Calculate the distribution of  OLS assuming that H 0 is true. 2.Find our estimate on the distribution 3.What is the probability that our estimate would have come from this distribution? 4.Does this lead us to believe the null hypothesis?

15. Again any estimate could be consistent with this hypothesis What is the probability of a fluke now? Clearly fairly likely to get an estimate of 0.801 if the true value is 0.81 Probability is much greater than 5% –So we “cannot reject” the hypothesis –P( b OLS ≤0.801|  MPC =0.81)=0.11 How did I get this “p-value”?

16. Some Comments Criminal Trial metaphor: The null hypothesis (innocence) will only be overturned if there is overwhelming evidence What constitutes “overwhelming” – Not with regard to the size of the difference between values for  (compare the two examples). –it is the difference in probability –i.e. the distance on distribution

17. Test Statistics Clearly some duplication between our two tests even though they were on different data So we can systematize things a little better We make use of property of normal distributions

18. So now we only ever have to deal with one distribution, the “standard” normal Note also how the construction of Z explicitly removes the issue of scale Test procedure –State Hypothesis –Calculate Z assuming H 0 is true Now we can compare the calculated values of Z with the standard normal distribution

19. Our Two Examples Aggregate Consumption: 1.State the Hypothesis we want to test H 0 :  MPC = 0.7 H 1 :  MPC ≠ 0.7 1.Calculate the test statistic assuming that H 0 is true. –Z=(0.7216591 -0.7)/(0.0022551)=9.6 2.Find our estimate on the distribution –Either find the test statistic on the standard normal distribution –Or compare with one of the traditional threshold (“critical”) values: 2.58(1%), 1.96 (5%), 1.64(10%) 3.|Z|>all the critical values and Prob (Z>9.6)=0 4.So we reject the null hypothesis

21. Comment –We will reject the idea that  MPC = 0.7 if there is overwhelming evidence that  MPC is bigger or smaller –The evidence is our estimate (0.72) –Is this big enough? –Remove the scale from the problem by calculating the test statistic: Z=9.6 –Is this big enough? –Traditionally 1.96 would be the “critical value” because of 5% probability of Z>1.96 as fluke –“beyond reasonable doubt” –Free to decide for ourselves (p-value)

22. Individual Consumption 1.State the Hypothesis we want to test H 0 :  MPC = 0.81 H 1 :  MPC ≠ 0.81 1.Calculate Z assuming that H 0 is true –Z=(0.8012171-0.81)/0.0074599=-1.17 2. Compare with critical values –|z|<1.96 –Prob (Z<-1.17)=0.12 3.Cannot Reject the Null Hypothesis

24. Comment –We will reject the idea that  MPC = 0.81 if there is overwhelming evidence that  MPC is bigger or smaller –The evidence is our estimate (0.80) –Is this big enough? –Remove the scale from the problem by calculating the test statistic: Z=-1.17 –Is this big enough? –Traditionally 1.96 would be the “critical value” because of 5% of Z<-1.96 as fluke –p-value of 0.12

25. Issues in Hypothesis Testing Test of significance “t-test” Rule of thumb Significance level

26. Test of Significance A test of H 0 :  = 0 is given the special name of “test of significance” Test statistic is simple Z=(  OLS –  se(  OLS )=  OLS /se(  OLS ) Which is calculated by most statistical software Simple eyeball test of significance Variable is or is not “statistically significant” Not the same as economically significant

27. t-test Strictly speaking the Z test is only valid when , the variance of u is known as it is used to calculate se(  )  will almost never be known and will have to be estimated When it is estimated the distribution of the estimator (and therefore the test statistic) is no longer normal Has a t-distribution with N-K degrees of freedom Fortunately t≈Z when N-K is large

28. Rule of Thumb Easy to “learn off” test procedure –Form the test statistic –Reject hypothesis if test statistic>2 in absolute terms –Useful for “eyeball” tests of significance Stata command “test” does the whole procedure automatically –But does “F-test” –This is the square of t-test

29. Significance Level Probability of a fluke When we choose a critical value we choose a significance level also: –2.58(1%), 1.96 (5%), 1.64(10%) If we reject the null because |Z|>1.96, we say we reject the null at the 5% significance level. We acknowledge that there is a 5% chance that Z>1.96 even though the null is true This is type 1 error: Rejecting a true Null –Criminal trial: Convicting the innocent

30. The test is set up make this as low as possible –i.e. reject only if overwhelming evidence Why not make it zero? Cant because would never reject any null –Criminal: always acquit This matters because setting up a hypothesis is setting up a procedure that is deliberately biased against rejecting Make sure that is what you want for your null