1. Logistic regression

2. Recall the simple linear regression model: y =  0 +  1 x +  where we are trying to predict a continuous dependent variable y from a continuous independent variable x. This model can be extended to Multiple linear regression model: y =  0 +  1 x 1 +  2 x 2 + … + +  p x p +  Here we are trying to predict a continuous dependent variable y from a several continuous dependent variables x 1, x 2, …, x p.

3. Now suppose the dependent variable y is binary. It takes on two values “Success” (1) or “Failure” (0) This is the situation in which Logistic Regression is used We are interested in predicting a y from a continuous dependent variable x.

4. Example We are interested how the success (y) of a new antibiotic cream is curing “acne problems” and how it depends on the amount (x) that is applied daily. The values of y are 1 (Success) or 0 (Failure). The values of x range over a continuum

5. The logisitic Regression Model Let p denote P[y = 1] = P[Success]. This quantity will increase with the value of x. The ratio: is called the odds ratio This quantity will also increase with the value of x, ranging from zero to infinity. The quantity: is called the log odds ratio

6. Example: odds ratio, log odds ratio Suppose a die is rolled: Success = “roll a six”, p = 1/6 The odds ratio The log odds ratio

7. The logisitic Regression Model i. e. : In terms of the odds ratio Assumes the log odds ratio is linearly related to x.

8. The logisitic Regression Model or Solving for p in terms x.

9. Interpretation of the parameter  0 (determines the intercept) p x

10. Interpretation of the parameter  1 (determines when p is 0.50 (along with  0 )) p x when

11. Also when is the rate of increase in p with respect to x when p = 0.50

12. Interpretation of the parameter  1 (determines slope when p is 0.50 ) p x

13. The data The data will for each case consist of 1.a value for x, the continuous independent variable 2.a value for y (1 or 0) (Success or Failure) Total of n = 250 cases

15. Estimation of the parameters The parameters are estimated by Maximum Likelihood estimation and require a statistical package such as SPSS

16. Using SPSS to perform Logistic regression Open the data file:

17. Choose from the menu: Analyze -> Regression -> Binary Logistic

18. The following dialogue box appears Select the dependent variable (y) and the independent variable (x) (covariate). Press OK.

19. Here is the output The Estimates and their S.E.

20. The parameter Estimates

21. Interpretation of the parameter  0 (determines the intercept) Interpretation of the parameter  1 (determines when p is 0.50 (along with  0 ))

22. Another interpretation of the parameter  1 is the rate of increase in p with respect to x when p = 0.50

23. Nonparametric Statistical Methods

24. Definition When the data is generated from process (model) that is known except for finite number of unknown parameters the model is called a parametric model. Otherwise, the model is called a non- parametric model Statistical techniques that assume a non- parametric model are called non-parametric.

25. Example – Parametric model Normal distribution – known except for the two parameters  and .  

26. Example – Non parametric model No assumptions are made about the distribution could be normal, skewed bimodal etc 0 0

27. The sign test A nonparametric test for the central location of a distribution

28. We want to test: H 0 : median =  0 H A : median   0 against (or against a one-sided alternative)

29. The Sign test: S = the number of observations that exceed  0 Comment: If H 0 : median =  0 is true we would expect 50% of the observations to be above  0, and 50% of the observations to be below  0, 1.The test statistic:

30. 50% median =  0 If H 0 is true then S will have a binomial distribution with p = 0.50, n = sample size.

31. median If H 0 is not true then S will still have a binomial distribution. However p will not be equal to 0.50. 00 p  0 > median p < 0.50

32. median 00 p  0 < median p > 0.50 p = the probability that an observation is greater than  0.

33. n = 10 Summarizing: If H 0 is true then S will have a binomial distribution with p = 0.50, n = sample size.

34. n = 10 The critical and acceptance region: Choose the critical region so that  is close to 0.05 or 0.01. e. g. If critical region is {0,1,9,10} then  =.0010 +.0098 +.0098 +.0010 =.0216

35. n = 10 e. g. If critical region is {0,1,2,8,9,10} then  =.0010 +.0098 +.0439+.0439+.0098 +.0010 =.1094

36. If n is large we can use the Normal approximation to the Binomial. Namely S has a Binomial distribution with p = ½ and n = sample size. Hence for large n, S has approximately a Normal distribution with mean and standard deviation

37. Hence for large n,use as the test statistic (in place of S) Choose the critical region for z from the Standard Normal distribution. i.e. Reject H 0 if z z  /2 two tailed ( a one tailed test can also be set up.

38. Nonparametric Confidence Intervals

39. Now arrange the data x 1, x 2, x 3, … x n in increasing order Assume that the data, x 1, x 2, x 3, … x n is a sample from an unknown distribution. Hence x (1) < x (2) < x (3) < … < x (n) x (1) = the smallest observation x (2) = the 2 nd smallest observation x (n) = the largest observation

40. Consider the k th smallest observation and the k th largest observation in the data x 1, x 2, x 3, … x n Hence x (k) and x (n – k + 1) P[x (k) < median < x (n – k + 1) ] = P[at least k observations lie below the median and at least k observations lie above the median ] If at least k observations lie below the median than x (k) < median If at least k observations lie above the median than median < x (n – k + 1)

41. Thus P[x (k) < median < x (n – k + 1) ] = P[at least k observations lie below the median and at least k observations lie above the median ] = P[The number of observations below the median is at least k and at most n-k] = P[k  S  n-k] S has a binomial distribution with n = the sample size and p =1/2. where S = the number of observations below the median

42. Hence P[x (k) < median < x (n – k + 1) ] = p(k) + p(k + 1) + … + p(n-k) = P = P[k  S  n-k] where p(i)’s are binomial probabilities with n = the sample size and p =1/2. This means that x (k) to x (n – k + 1) is a (1 – P)100% confidence interval for the median

43. Summarizing where P = p(k) + p(k + 1) + … + p(n-k) and p(i)’s are binomial probabilities with n = the sample size and p =1/2. x (k) to x (n – k + 1) is a (1 – P)100% confidence interval for the median

44. n = 10 and k =2 Example: P = p(2) + p(3) + p(4) + p(5) + p(6) + p(7) + p(8)=.9784 Binomial probabilities Hence x (2) to x (9) is a 97.84% confidence interval for the median

45. Example Suppose that we are interested in determining if a new drug is effective in reducing cholesterol. Hence we administer the drug to n = 10 patients with high cholesterol and measure the reduction.

46. The data

47. The data arranged in order x (2) = -3 to x (9) =15 is a 97.84% confidence interval for the median

48. Example In the previous example to repeat the study with n = 20 patients with high cholesterol.

49. The data

50. The binomial distribution with n = 20, p = 0.5 Note: p(6) + p(7) + p(8) + p(9) + p(10) + p(11) + p(12) + p(13) + p(14) = 0.037 + 0.0739 + 0.1201 + 0.1602 + 0.1762 + 0.1602 + 0.1201 + 0.0739 + 0.037 = 0.9586 Hence x (6) to x (15) is a 95.86% confidence interval for the median reduction in cholesterol

51. The data arranged in order x (6) = -1 to x (15) = 9 is a 95.86% confidence interval for the median

52. For large values of n one can use the normal approximation to the Binomial to find the value of k so that x (k) to x (n – k + 1) is a 95% confidence interval for the median. i.e. we want to find k so that

54. Next day we will consider: 1.The Wilcoxon signed rank test The Wilcoxon signed rank test is an alternative to the Sign test, a test for the central location of a single population and 2.The Wilcoxon rank sum test The Wilcoxon rank sum test is a nonparametric test for comparing the central location of two populations