Hypothesis testing Week 10 Lecture 2

Functions of inferential statistics To estimate a population parameter from a random sample If you draw two random samples of the same size from a population, it is very likely that the two means you get will be different Standard error of the mean is the standard deviation of all sample means: To test hypotheses data are from sample The probability that our result is due to chance alone If you get data from whole population, no hypothesis testing is required. Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

Two types of hypotheses Research hypothesis (alternative hypothesis) A prediction of the relation between variables Null hypothesis (H0) There will be no relation between the variables Any relation observed are due to chance alone Examples: The higher the annual income, the greater the Internet usage There is no relation between annual income and Internet usage It is the null hypothesis that is tested We look at the probability that our result is due to chance alone. Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

Result of hypothesis testing Reject Null hypothesis Indirectly accept the research hypothesis Research hypothesis is supported Fail to reject null hypothesis Research hypothesis is not supported How do we determine whether or not to reject the null hypothesis Compute some statistics reflecting the difference Find out the probability that any difference is due to chance Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

ISYS3015 Analytical Methods for IS professionals Probability The likelihood of something happening Denote by p Numerical value ranging from 1 to 0 Probability of discrete variables Probability of continuous variables Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

Statistical significance How far must an outcome be away from the expected? At what level of probability we believe a result is more likely due to a real difference (caused by experiment) than to chance Level of significance (significance level) a =0.05, 0.01, 0.1, 0.001 Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

Making errors in hypothesis testing Type I error Reject the null hypothesis when it is true Type II error Fail to reject the null hypothesis when it is false Significance level --  (0.05) The probability of rejecting null hypothesis when it is true Power – 1- (0.80) The ability to reject a false null hypothesis Actual situation: null hypothesis is Conclusion True False Fail to reject H0 Correct decision Type II error Probability: 1-  Probability:  Reject H0 Type I error Probability:  Probability: 1- Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

One-tailed and two tailed hypotheses One-tailed hypothesis The direction of the relation is predicted in the alternative hypothesis Example People with high education are more interested in politics than people with low education H1: m1 > m2 Two-tailed hypothesis No prediction about the direction of the relationship is made There is a difference in interest of politics between people with high education and people with low education H1: m1 ≠ m2 Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

Drawing conclusion from hypothesis test p-value The p-value indicates the probability that one would obtained a test statistic which is more extreme than the observed one when the null hypothesis is true. The possibility that any observed difference is due to chance. if p-value < a, reject null hypothesis Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

ISYS3015 Analytical Methods for IS professionals One-tailed test Upper tail test Critical region (reject region) locates on the upper tail The area indicates the maximum probability that you can reject a null hypothesis (a) The corresponding value of its boundary is the critical value Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

ISYS3015 Analytical Methods for IS professionals Two tailed test Two critical regions locate on two tails Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

One and two-tailed tests Easier to reject a null hypothesis in a one-tailed than in a two-tailed test if the test statistic falls in the expected direction One tailed-test can not handle the situation when the test statistic falls in the “wrong” tail Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

p-value in one and two-tailed test p-value from a two-tailed test is double the value from a one-tailed test p = P (z >= zobservedH0 is true ) (upper-tailed test) p = 2P (z >= | zobserved |H0 is rue) (two-tailed test) Some test returns either one or two-tailed test results, some returns both Find the p-value suitable for your hypothesis Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

Simplest hypothesis testing statistic t-test Assesses whether the means of two groups are statistically different from each other Sample size is small Dependent variable is interval or ratio scale Independent variable has two levels Approximately normal distribution of the measure in the two groups is assumed Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

ISYS3015 Analytical Methods for IS professionals t-score t-score difference between means/standard error of the difference Give both one- and two-tailed p-value Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney

ISYS3015 Analytical Methods for IS professionals t-test case Scenario You want to measure the acquisition of mathematical skills by distance learning and traditional classroom learning. The study involves the comparison of 20 students, ten taught in classroom and ten taught by distance learning program. The final test scores were collected as dependent variable. Write down your null hypothesis Write a two-tailed hypothesis and a one-tailed hypothesis Friday, May 21, 2004 ISYS3015 Analytical Methods for IS professionals School of IT, University of Sydney