1. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 1 Hypothesis testing Statistical Tests Sometimes you have to make a decision about a characteristic of a population. For example you claim a new drug is better treating a disease then a current one. You will observe the change of a symptom of the disease under the new drug and under the standard. And you hope there is a difference between the drugs which is not only due to chance. The opposite of your claim will be the null hypothesis, which means the observed difference is only due to unexplained 'chance' (no effect). If you can reject the null hypothesis, you will accept the alternative hypothesis: there is a non-chance difference between the drugs (effect). Accepting the alternative hypothesis (your claim) means that there must be a strong evidence against the null hypothesis.

2. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 2 Example 1 For some branches of industry, it is important to check whether the mean of body height of adults has changed. If you want to be sure you must sample all German adults and compute the mean of their body height. A more practicable method is to choose a representative sample of n adults and compute the mean of their body heights. Thesis: The mean of body height of German adults is 173 cm (from former investigations). You have to decide whether this is still valid or not. Statistical tests for unknown parameters This sample mean is an estimation of the unknown mean (expectation) of the body height of all German adults. Because it does not reflect the whole information of the population there is a risk in your decision.

3. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 3 Null hypothesis:  = 173 against Alternative hypothesis:   173 Sample of sample mean estimates the unknown expectation: Difference between sample mean and reference value Is this sample mean consistent with the Null hypotheses? Or is it so unlikely that the Null hypothesis should be rejected? For this decision we would accept an error probability of 0.05. Up to which value k is this difference randomly, when will the sample mean be inconsistent with the Null hypothesis (difference too large)? Statistical tests for unknown parameters  unknown expectation of the whole population  0 =173 reference value

4. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 4 In order to infer from the sample mean to the expectation of the population, we use the distribution of the point estimator Statistical tests for unknown parameters Using for example a sample size of n = 100, Then we know the distribution of Suppose the random variable X (body height) follows a normal distribution with known standard deviation  = 10 and unknown .

5. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 5 If the Null hypothesis is true,  = 173, and consequently Statistical tests for unknown parameters Because of it is very unlikely to D taking values of this region if H 0 is true, we reject it in this cases. We find this critical region as and hence This decision has an error probability of 0.05, since also with p = 0.05 D can take values of this critical region C even though the Null hypotheses is true. Using this distribution, we determine a region C for rejecting the H 0 by

6. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 6 Decision in example 1 From the sample, we get d = 2. Accepting an error probability of 0.05, the critical region is Statistical tests for unknown parameters Because of d = 2 belongs to C (it exceeds the critical value of k = 1.96) we reject the null hypothesis. The mean of the body height of German adults is no longer 173 cm. The confidence level is 0.95 (or: the risk is 0.05).

7. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 7 Statistical tests for unknown parameters In practice you do not compute the difference d, you make your decision using the following statistic T Under the Null hypothesis, with EX =  0, and also T ~ N(0,1). If the sample value of T is out of this range, the null hypothesis is rejected. The error probability is then 0.05, because of H 0 true, T is out of this range with p= 0.05 too. Thus with probability 1 - 

8. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 8 In example 1,  = 10, n = 100, Because the sample value T = 2 lies in the critical range, the null hypothesis is rejected. Risk  = 0.05 Under the null hypothesis, T ~ N(0, 1), consequently with p = 0.95 T lies in the interval The null hypothesis is rejected if T 1.96 it is equivalent to |T| > 1.96 (critical range). Statistical tests for unknown parameters

9. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 9 Testing scheme Comparison of the unknown mean  under normal distribution with respect to a reference value (  is assumed to be known ) Statistical tests for unknown parameters

10. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 10 Critical range for risk  Ablehnung Under Null Hypothesis, T lies in this interval with probability 1-  1 -  Density of statistic T under Null hypothesis Rejection Statistical tests for unknown parameters

11. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 11 Kinds of error Interpretation in case of H 0 : there is no difference, no effect Error of 1. kind: Rejection of a true H 0, false alarm you detect a not existing difference Error of 2. kind: Accepting an incorrect H 0, missing to sound alarm you overlook an existing difference Statistical tests for unknown parameters

12. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 12 One-sided and two-sided tests Two-sided (two-tailed) test Null hypothesis:  =  0 Alternative hypothesis:    0 One-sided (one-tailed) tests Null hypothesis    0 Alternative hypothesis:  >  0 or Null hypothesis:    0 Alternative hypothesis:  <  0 Statistical tests for unknown parameters

13. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 13 Errors of 1. und 2. kind Error of 1. kind: α Density of T under Null hypothesis Critical range for one-sided test: Statistic Statistical tests for unknown parameters

14. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 14 Errors of 1. und 2. kind Error of 1. kind: α Statistic Density of T under Null hypothesis Critical range by one-sided test Statistical tests for unknown parameters Density of T for one-sided alternative

15. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 15 Error of 2. kind: β Critical range by one-sided test Errors of 1. and 2. kind Error of 1. kind: α Statistic Density of T for one-sided alternative Density of T under Null hypothesis Statistical tests for unknown parameters

16. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 16 Interpretation  The smaller , the bigger  will be.  The probability  for the rejection of a false Null hypothesis can be calculated with respect to any alternative reference value  1 when the sample size n is given. Only increasing n can minimize  for a given  !  The smaller the difference  0 -  1, the bigger  will be (overlapping). Statistical tests for unknown parameters

17. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 17 Minimum of sample size for guaranteeing maximal errors , , (  ² known) L denotes the practically relevant difference in mean Comparison with 0  (one sample) Two-sided test: 2 2 2 12/1 0 )(     L zz nn One-sided Test: 2 2 2 11 0 )(     L zz nn Statistical tests for unknown parameters

18. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 18 Testing hypothesis on the mean of a normal distribution (1) One-sample tests Null hypothesis Alternative hypothesis Statistic Critical range 00 :  H 01 :  H 2/1   zT 00 :  H 01 :  H   1 zT 00 :  H 01 :  H n X T / 0    ~ N(0, 1)   1 zT Statistical tests for unknown parameters

19. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 19 Null hypothesis Alternative hypothesis Statistic Critical range 00 :  H 01 :  H 2/1,1   n tT 00 :  H 01 :  H   1,1n tT 00 :  H 01 :  H ns X T / 0   ~ 1  n t   1,1n tT Testing hypothesis on the mean of a normal distribution (2) One-sample tests Statistical tests for unknown parameters

20. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 20 Statistical tests for unknown parameters Note Testing with a software system results in a p-value (often called significance), which reports the probability you will observe the given sample ore a more extreme one assuming the Null hypothesis were true. For p-value less then the risk  you reject the Null hypothesis, if you test against a two-sided alternative hypothesis.. In case of one-sided testing take one half of p-value to compare with . ►

21. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 21 One-sample test: The expectation of a population is compared with a given reference value  0. Two-sample tests can have a paired design (dependent samples) or an unpaired design (independent samples). Two-sample test: The expectations  1 and  2 of two populations are compared. unpaired: the samples are obtained in unrelated (disjoint) groups (for example healthy and ill, or female and male) paired: each data point in one sample is matched to a unique data point in the second sample (for example pre test/post test design observing twice the same subjects or objects) Statistical tests for unknown parameters

22. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 22 Notations Sample size: Sample means: Sample variances: Pooled variance Statistical tests for unknown parameters

23. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 23 Null hypothesis Alternative hypothesis Statistic Critical range 0: 0  D H  0: 1  D H  2/1,1   n tT 0: 0  D H  0: 1  D H    1,1n tT 0: 0  D H  0: 1  D H  n s d T D  ~ 1  n t   1,1n tT Testing hypothesis on the mean of a normal distribution (3) Two-sample tests Statistical tests for unknown parameters

24. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 24 Null hypothesis Alternative hypothesisStatistic Critical range yx H  : 0 yx H  : 1 2/1,2   yx nn tT yx H  : 0 yx H  : 1   1,2 yx nn tT yx H  : 0 yx H  : 1 yx yx g nn nn s YX T    ~ 2  yx nn t   1,2 yx nn tT Testing hypothesis about the mean of a normal distribution (4) Two-sample tests Statistical tests for unknown parameters

25. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 25 Null hypothesisAlternative hypothesis Statistic Critical range yx H  : 0 yx H  : 1 2/1,   f tT yx H  : 0 yx H  : 1   1,f tT yx H  : 0 yx H  : 1 y y x x n s n s YXT 2 2 /)(  ~ 1  f t   1,f tT Testing hypothesis about the mean of a normal distribution (5) Two-sample tests Statistical tests for unknown parameters

26. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 26 Comparison X  to Y  (Two-sample test) Two-sided test 2 2 2 12/1 0 )( 2     L zz nn One-sided test 2 2 2 11 0 )( 2     L zz nn Minimum of sample size for guaranteeing maximal errors , , (  ² known) L denotes the practically relevant difference between the means Statistical tests for unknown parameters

27. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 27 Example: Two-sample test for unpaired samples Is there a difference in hemoglobin values for healthy children and those suffering a certain illness? Normal distribution with equal variances is provided, error probability 0.05 data for healthy children: data for ill children: Statistical tests for unknown parameters Null hypothesis: Risk:

28. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 28 Example (continued) Because of 2,63 > 2,086, the Null hypothesis is rejected, which means the illness changes the mean of hemoglobin level significantly. The result of the sample can be generalized for the whole population with a confidence of 95% yx yx g nn nn s YX T    Statistic: Criteria of rejection: Statistical tests for unknown parameters

29. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 29 One-sample tests Testing hypothesis about the variance of a normal distribution (1) Statistical tests for unknown parameters Null hypothesis Alternative hypothesisStatisticCritical range oder

30. WS 2007/08Prof. Dr. J. Schütze, FB GW KI 30 Two-sample tests This test is used to decide between the unpaired T-test and the Welch-test in order to compare means. Testing hypothesis about the variance of a normal distribution (1) Statistical tests for unknown parameters ►