1. Modeling Wim Buysse RUFORUM 1 December 2006 Research Methods Group

2. Part 1. General Linear Models Research Methods Group

3. General Linear Models Dataset from Research Methods Group

4. General Linear Models Dataset from p. 89 - 95 Research Methods Group

5. General Linear Models Effects of three levels of sorbic acid (Sorbic) and six levels of water activity (Water) on survival of Salmonella typhimurium (Density) Water density = log(density/ml) Research Methods Group

6. ANOVA approach Research Methods Group General Linear Models

7. Results Research Methods Group General Linear Models

8. The same data, but each treatment is presented as a ‘dummy variable’. (Warning: for educational purposes only.) Research Methods Group General Linear Models

9. Regression with a first independent variable. Research Methods Group General Linear Models

10. We add a second independent variable. Research Methods Group General Linear Models

11. We add a third one. Research Methods Group General Linear Models

12. We add a fourth one. Research Methods Group General Linear Models

13. We continue to construct the model. Research Methods Group General Linear Models

14. Finally, the results. Research Methods Group General Linear Models

15. Comparison of the two approaches. Research Methods Group General Linear Models

16. Comparison of the two approaches: -They give the same results (in terms of SS.) -The approach to choose depends on what you want to know. -The regression approach still works when the ANOVA approach is not possible anymore (for instance when there are missing values). Research Methods Group General Linear Models

17. Example: modelling approach with normally distributed data. Protocol and dataset. Research Methods Group

18. Example: modelling approach with normally distributed data. Data: Screening of suitable species for three-year fallow file = Fallow N.xls Protocol: p. 13 Research Methods Group

19. The analysis approach is written down in chapter 19 of ‘Good statistical practice for natural resources research’ Research Methods Group Example: modelling approach with normally distributed data.

20. Modelling approach: general 5 steps: 1.(Visual) exploration to discover trends and relationships 2.Choose a possible model: The trend you see Knowledge of the experimental design Biological/scientific knowledge of the process 3.Fitting = estimation of parameters 4.Check = assessing the ‘fit’ 5.Interpretation to answer the objectives. Research Methods Group

21. Expanding the model ANOVA and regression Same calculations Data = pattern + noise = systematic component + random component Same assumptions Systematic components are additive Variability of the groups is similar The random component is (rather) normally distributed. The random variability of “y” around the systematic component is not affected by this systematic component. Research Methods Group

22. GENERAL LINEAR MODELS Research Methods Group

23. GENERAL LINEAR MODELS Research Methods Group

24. GENERAL LINEAR MODELS Research Methods Group Data = pattern + noise Pattern: is explained by a linear combination of the independent variables (Data ≈ N(m,v) and the variance is rather constant across the different groups) Noise: N(0,1) and the variance is rather constant across the different groups

25. Expanding the model If the data are not normally distributed or if the variance of the different groups is not similar: Possible approach = transformation of the data = « linearising » the model Problems: -You don’t work anymore on a scale that has a biological meaning. -Retransforming the standard errors back to the original scale is not possible anymore. Research Methods Group

26. Better solution: GENERAL LINEAR MODELS => GENERALIZED LINEAR MODELS Research Methods Group Less restrictions; two essential differences: 1.Data can be distributed according to the family of exponential distributions = Normal, Binomial, Poisson, Gamma, Negative binomial 2.Link function: the link between E(Y) and the independent variables is not longer a linear combination of the independent variables. It is also possible that the linear combination of the independent variables is a function of can also be a linear combination of a function of E(Y). (We don’t transform the dependent variables but include the transformation into the model). Expanding the model

27. Research Methods Group Also: - The systematic component (linear combination of independent variables) can include both continuous and categorical variables and even polynomials But still: -The variance is constant across the different groups (or has become constant because of the transformation through the link function) Expanding the model Better solution: GENERAL LINEAR MODELS => GENERALIZED LINEAR MODELS

28. Generalised linear models Research Methods Group Statistical theory is more difficult, but the menus in GenStat and the way you can interpret the output is very similar to what we know from ANOVA and regression.

29. Research Methods Group = =

30. Example 1. Logistic regression Example: cardio-vascular disease according to age Research Methods Group age and chd.xls

31. Example: same data but according to age group Research Methods Group Example 1. Logistic regression

32. Example: the linear regression is not an appropriate model and the predictions at the extremes will not be correct Research Methods Group Example 1. Logistic regression

33. Example: test χ 2 test: limited information Research Methods Group Example 1. Logistic regression

34. Bernoulli process: an (independent) event that can have two possible outcomes (1 – 0, success- failure, …); with a given probability of succes Tossing a coin: head or tail; p = 0,5 Throwing 6 with a dice (success) compared to throwing any other number; p = 1/6 Conducting a survey: is the head of the household male or female?; calculate p from the proportion found in the collected data Screening of cardio-vascular diseases. p disease = 43 out of 100 individuals = 0.43 Research Methods Group Example 1. Logistic regression

35. In GenStat Research Methods Group Example 1. Logistic regression

36. Logistic function Research Methods Group Example 1. Logistic regression

37. Logistic function Sigmoid form Linear in the middle The probability is restricted between 0 et 1 Small values: flatten towards 0; large values: flatten towards 1 Research Methods Group Example 1. Logistic regression

38. GenStat output Similar, but ‘deviance’ instead of ‘variance’ and test χ 2 instead of F Research Methods Group Example 1. Logistic regression

39. GenStat output model Research Methods Group Logit(CHD) = -5,31 + 0,1109 AGE Example 1. Logistic regression

40. Research Methods Group Logit(CHD) = -5,31 + 0,1109 AGE Example 1. Logistic regression

41. Research Methods Group Example 1. Logistic regression

42. Binomial distribution: when we repeat the Bernoulli process, the order of success or failure can change Example: head of household in a survey Research Methods Group Example 1. Logistic regression

43. Calculation of probabilities if success = female headed household with p = 0,2 Research Methods Group Example 1. Logistic regression

44. Calculated probabilities for obtaining success Research Methods Group We can now construct a frequency distribution of obtaining success Probability = long-run frequency = frequency when very many data = binomial distribution Example 1. Logistic regression

45. Binomial distribution Counts of a categorical variable Example: experiment of survival of trees from different provenances File: survival trees.xls Research Methods Group Example 1. Logistic regression

46. Several approaches possible Research Methods Group 1 Example 1. Logistic regression

47. Several approaches possible Research Methods Group 1 Example 1. Logistic regression

48. Research Methods Group 2 Example 1. Logistic regression Several approaches possible

49. Research Methods Group 2 Example 1. Logistic regression Several approaches possible

50. Research Methods Group 3 Example 1. Logistic regression Several approaches possible

51. Research Methods Group 3 Example 1. Logistic regression Several approaches possible

52. The Bernoulli distribution is a special case of the binomial distribution There exist ‘families of distributions’. Research Methods Group Example 1. Logistic regression

53. There is of course a difference in the variability that is explained. Research Methods Group 1 2 3 Example 1. Logistic regression

54. Example 2. Modelling counts We used logistic regression to analyse counts. Bernoulli distribution: distribution of success of events that follow a Bernoulli process (1 or 0, yes or no) Binomial distribution: distribution of possible (and independent) combinations of Bernoulli events So, more like analysis of proportions. Next: Poisson distribution: distribution of counts of Bernoulli events Research Methods Group

55. Poisson distribution: distribution of counts of Bernoulli events BUT: p is very small n is very big p*n < 5 Events happen randomly and independent of each other. Research Methods Group Example 2. Modelling counts

56. Poisson distribution = distribution of rare events Number of civil airplane crashes (when there is no war) in the whole world during several years. Number of infected seeds in seed lots that are certified by a controlling agency. Number of individuals of a rare tree species in a square kilometre in the same Agro Ecological Zone. Research Methods Group Example 2. Modelling counts

57. THUS The distribution that best describes counts is not automatically a Poisson distribution. It depends of the context. Research Methods Group Example 2. Modelling counts

58. Some mathematical statistics Research Methods Group The proportion mean/variance must be 1. = Poisson index In GenStat: (s 2 -m)/m Example 2. Modelling counts

59. We briefly have seen already other counts: χ 2 test Research Methods Group χ 2 test: is there evidence of an association between two discrete variables H 0 : no association H 1 : association Example 2. Modelling counts

60. We could use another kind of probability to calculate the test statistic Research Methods Group Example 2. Modelling counts

61. But now we look at the table in another way. If we consider the counts in the table as a variable, we could construct a frequency distribution. Research Methods Group Example 2. Modelling counts

62. Long run frequency distribution = probability distribution We just expanded the binomial distribution into the multinomial distribution. Binomial distribution: Independent observations p success = everywhere the same. The probability that an individual observation falls into a specific cell of the table is the same for all cells. Multinomial observation: + The number of total observations is fixed. Research Methods Group Example 2. Modelling counts

63. If the total number of observations was not fixed => Poisson distribution BUT Thanks to a lot of difficult statistical theory: we can also use the Poisson distribution even if the total number of observation is not fixed. Research Methods Group Example 2. Modelling counts

64. CONCLUSION Even though the context is important to decide whether we can use the Poisson distribution to analyse counts (‘distribution of rare events’) Generally: Analysis of ‘multiway contingency tables’ => Poisson distribution + logarithm link = LOGLINEAR MODELING Research Methods Group Example 2. Modelling counts

65. Analysis of counts = Often we can use the Poisson distribution But not always Research Methods Group Example 2. Modelling counts

66. Example 2. Loglinear modelling Research Methods Group =

67. Adding interactions Example 2. Loglinear modelling

68. Research Methods Group = χ 2 test Loglinear modelling Example 2. Loglinear modelling

69. Research Methods Group Modelling of complex datasets: Adding or dropping terms and interactions in the model and changing their order Good model (‘good fit’ ) when the ‘residual deviance’ becomes almost equal to the number of degrees of freedom (or ‘mean deviance’ = 0) At that moment we can assume that the remaining residual variability is caused by the random variability (noise) Adding too many terms: ‘residual deviance’ => 0 Example 2. Loglinear modelling

70. Research Methods Group Example: lambs.xls Example 2. Loglinear modelling