1. Mann-Whitney U = Wilcoxon rank sum is the non-parametric test equivalent to t-test ssd<-read.table("dimorphism.txt",header=T) attach(ssd) names(ssd) [1] "binomial" "sex" "svl" Is there a difference in body size (SVL) between males and females of true lizards (Lacertidae)? male<-svl[sex=="male"] female<-svl[sex=="female"] wilcox.test(male,female,paired=FALSE) Wilcoxon rank sum test with continuity correction data: male and female W = 15396, p-value = 0.1493 alternative hypothesis: true location shift is not equal to 0 Non-parametric tests in R

2. ssd<-read.table("dimorphism.txt",header=T) attach(ssd) names(ssd) [1] "binomial" "sex" "svl" male<-svl[sex=="male"] female<-svl[sex=="female"] wilcox.test(male,female,paired=TRUE) Wilcoxon rank sum test with continuity correction data: male and female V = 8822.5, p-value = 1.773e-06 alternative hypothesis: true location shift is not equal to 0 Wilcoxon two-sample (=Wilcoxon signed-rank) test non-parametric equivalent for paired t-test Is there a difference in body size (SVL) between males and females of true lizards (Lacertidae) when you compare between sexes of the same species? Non-parametric tests in R

3. names(ssd) [1] "binomial" "sex" "svl" wilcox.test(male,female,paired=TRUE) Wilcoxon rank sum test with continuity correction data: male and female V = 8822.5, p-value = 1.773e-06 alternative hypothesis: true location shift is not equal to 0 Is there a difference in body size (SVL) between males and females of true lizards (Lacertidae) when you compare between sexes of the same species? Wilcoxon two-sample (=Wilcoxon signed-rank) test non-parametric equivalent for paired t-test Here we have a problem: we want the test to be according to the species name but we didn’t declared it any where Non-parametric tests in R

4. We will use recast: this will transform the data to a matrix (something you will usually try to avoid in other statistical programs library(reshape2) sex<-recast(ssd,binomial~sex,measure.var = "svl") names(sex) [1] "binomial" "female" "male" Notice that there is a change in names compared to the pervious slide Wilcoxon two-sample (=Wilcoxon signed-rank) test non-parametric equivalent for paired t-test Is there a difference in body size (SVL) between males and females of true lizards (Lacertidae) when you compare between sexes of the same species? Non-parametric tests in R

5. sex<-recast(ssd,binomial~sex,measure.var = "svl") names(sex) sex wilcox.test(sex$female,sex$male,paired=TRUE) [1] "binomial" "female" "male" Wilcoxon signed rank test with continuity correction data: sex$female and sex$male V = 3423.5, p-value = 1.773e-06 Non-parametric tests in R Wilcoxon two-sample (=Wilcoxon signed-rank) test non-parametric equivalent for paired t-test We will use recast: this will transform the data to a matrix (something you will usually try to avoid in other statistical programs Is there a difference in body size (SVL) between males and females of true lizards (Lacertidae) when you compare between sexes of the same species?

6. sex<-recast(ssd,binomial~sex,measure.var = "svl") names(sex) sex wilcox.test(sex$female,sex$male,paired=TRUE) p.s. this is also a way to do paired t-test t.test(sex$female,sex$male,paired = T) [1] "binomial" "female" "male" Non-parametric tests in R Wilcoxon two-sample (=Wilcoxon signed-rank) test non-parametric equivalent for paired t-test We will use recast: this will transform the data to a matrix (something you will usually try to avoid in other statistical programs Is there a difference in body size (SVL) between males and females of true lizards (Lacertidae) when you compare between sexes of the same species?

7. Kruskal-Wallis the equivalent to one-way ANOVA Kruskal-Wallis rank sum test data: clutch by type Kruskal-Wallis chi-squared = 7.1639, df = 2, p-value = 0.02782 The code for this test is very similar to ANOVA test but instead of aov you write kruskal.test kruskal.test(clutch~type) island<-read.csv("island_type_final2.csv",header=T) Attach(island) Non-parametric tests in R

8. Spearman test and Kendall’s-tau test are equivalent to correlation test cor.test(clutch, mass, method="spearman") #or cor.test(clutch, mass, method="kendall") The code here is a variation of the common correlation test, by adding the definition of a non-parametric test in the ‘method’ argument Instead of writing: cor.test(clutch,mass)we will write: Spearman's rank correlation rho data: clutch and mass S = 1907900, p-value < 2.2e-16; rho 0.5402 Kendall's rank correlation tau data: clutch and mass z = 9.747, p-value < 2.2e- 16; tau 0.4059 We will get respectively: Non-parametric tests in R

9. Generalized linear models (GLM) We will use GLM when our response variable is not continuous (counts, proportions, binary etc.) – or when the parametric test assumptions (normal distribution, equality of variance) are not met GLM is structured from three parts 1. linear predictor; 2. link function; 3. error distribution The first is the parameter value, the second refers to transformation (for example “identity” when there is no transformation and “log” for logarithmic transformation) and the third refers to the distribution of the residuals – for example gama, binomial and normal distribution A unique case where link=identity and error=normal the GLM is a linear model

10. Generalized linear models (GLM) Structured from three parts: 1. linear predictor; 2. link function; 3. error distribution modelX<-glm(clutch~log10(age)+asin(lat),family=Gamma) Log linkarcsin linkGamma errors The first is the parameter value, the second refers to transformation (for example “identity” when there is no transformation and “log” for logarithmic transformation) and the third refers to the distribution of the residuals – for example gama, binomial and normal distributions A unique case where link=identity and error=normal the GLM is a linear model

11. Non-linear models Sometimes it is clear that the relationship between the predictor and the response is not linear We can test models that know how to deal with this type of data structure. For example: add quadratic equation to the model Response = a(predictor) 2 +b(predictor)+c and we can test if the quadratic model is better than the linear using AIC or anova() More explanation in the model selection part of the presentation model2<-lm(y~x+I(x^2))

12. Non-linear models We can test breaking point models that have different linear equations for different values of the predictor Y = A 1.x + K 1 for x < breakpoint Y = A 2.x + K 2 for x > breakpoint Losos & Schluter 2000. Analysis of an evolutionary species-area relationship. Nature 408: 847-850. Sometimes it is clear that the relationship between the predictor and the response is not linear

13. Multiple predictors Life is complicated, what can we do. Sometimes what is interest us is affected by more than one variable Smith, R. J. 1999. Statistics of sexual size dimorphism. Journal of Human Evolution 36: 423-459. The heartbeat of lizards, for example, is affected by their body size and environmental temperature, and also from the time and speed their were moving lately

14. Smith, R. J. 1999. Statistics of sexual size dimorphism. Journal of Human Evolution 36: 423-459. We can explain the variable that interests us (heartbeat) if we have data on the predicting variables The assumption is that when we put all three of them to the equation we see the effect of each one when the other two help constant Multiple predictors Life is complicated, what can we do. Sometimes what is interest us is affected by more than one variable The heartbeat of lizards, for example, is affected by their body size and environmental temperature, and also from the time and speed their were moving lately

15. Smith, R. J. 1999. Statistics of sexual size dimorphism. Journal of Human Evolution 36: 423-459. The assumption is correct when we don’t have high correlation between the predictors Multiple predictors We can explain the variable that interests us (heartbeat) if we have data on the predicting variables The assumption is that when we put all three of them to the equation we see the effect of each one when the other two help constant

16. Which test should we choose? If we have a few predictor variable (lets say 4) and they are all categorical the test for them will be ANOVA (lets say 4-way ANOVA) If we have a few predictor variables (lets say seven) and they are all continuous the test for them will be Multiple Regression

17. How do we write a test with a few explanatory variables? We use the ‘+’ between the predictors. model<-lm(Grade~days_studied+professor_age+prayer_number+reconstruction, data=marks) summary(model) lm(y~a+b+c) For example a test that tries to predict the grades of a course based on how much we studied, the age of the lecturer, how much we prayed and whether there are test from previous years

18. If we have a few predictor variables (at least 2) – part of them (at least one) are categorical and part of them (at least one) are continuous the test for them will be ANCOVA (analysis of co-variance) Which test should we choose?

19. How will it look graphically? ANOVA Regression ANCOVA For example I measured the length of three teeth of the common fox, males and females, through their distribution range pFMSDFSS 0.00417468.9304435.21 Intercept 0.00142.3103.81 sex 0.0020704.915098.9230197.7 tooth 0.722761659.8 Error Two significant variables: a difference between the teeth and the sexes Upper carnassial Bottom carnassial Upper canine ניב שן שסע תחתונ ה שן שסע עליונה Vulpes vulpes

20. ANOVA Regression ANCOVA For example: the length of the teeth in the common fox as a function of latitude Pt Std. ErrorEstimate > 0.000125.250.3789.554Intercept > 0.00015.730.0080.044Latitude We can see evidence for Bergman’s rule But it is easy to notice that it’s a horrible model: the canines are smaller than both of the carnassial R-squared: 0.015, F = 32.83, 1 & 2161 DF; p < 0.0001 *Regression line is a model for the relationship between the predictor and the response How will it look graphically?

21. Graphical ANCOVA It is easy to understand it graphically: in the example there is a single variable on the X and a response variable with two levels – continuous and categorical (dashed and full lines) Null hypothesis a. response Continuous predictor response b. Continuous predictor Categorical significant, Cont. not significant d. response Continuous predictor Both significant Continuous predictor c. response And thanks to Daniel for the plots Categorical not Significant, Cont. significant

22. ANOVA Regression ANCOVA Example: Teeth length in common fox, as a function of latitude (X axis) and sex (color) and which tooth it is (shapes) pFMSSSDffactor > 0.0001191.799 1sex > 0.000127758.7914332.728665.32tooth > 0.0001844.69436.1 1Latitude 0.51114.22158Residuals All the variables are significant This models explains 96.3% of the variation We have here 6 regression lines How will it look graphically?

23. Reading ANCOVA results in R Response = intercept + a for level 1 of the 1 st categorical predictor variable or + b for level 2 of the 1 st + c for level 1 of the 2 nd categorical predictor or d for level 2… +k*(value of the continuous predictor variable) + error For example, if we go back to foxes pt Std. ErrorEstimatefactor >0.000155.920.0784.342 Intercept (tooth c) >0.0001224.210.0388.485tooth_m >0.0001174.840.0386.617tooth_p >0.000112.560.0310.391sex_male >0.000129.060.0010.043Latitude According to the model tooth P length of a male in Tel Aviv 12.726 = 4.342+6.617+0.391+32*0.043 Example without interactions Tooth / sex Latitude

24. When the predictor is categorical, R compares all the factors to the intercept of the first factor in the alphabet Here the predictor is island type and “Continental” is the first in the alphabet So mean clutch size* on continental islands is 0.33 and on land bridge islands is 0.33149+0.12399 = 0.45548 The difference between continental and Land bridge is significant t=2.309, p=0.021 Estimate Std. Errort valuePr(>|t|) (Intercept) 0.331490.0298411.11<2e-16*** typeLand_bridge 0.123990.053692.3090.0216* typeOceanic 0.021840.037770.5780.5635 * After logarithmic transformation, no one lays third of an egg island<-read.csv("island_type_final2.csv",header=T) levels(type) [1] "Continental" "Land_bridge" "Oceanic" model<-lm(clutch~type, data=island) summary(model) How to read lm results in R

25. When the predictor is continuous, R reports its slope with its SE and, t and p values for it Here the predictors that explain clutch size are island area and latitude So clutch size increases in 0.0059 (the units are log 10 eggs) with the increase in each latitudinal degree and decreases in 0.0016 units with the increase in island area (but the decrease is not significant t=0.098, p=0.92) The effect of latitude is significant (t = 3.259, p=0.00126) island<-read.csv("island_type_final2.csv",header=T) model3<-lm(clutch~area+lat,data=island) summary(model3) Estimate Std. Errort valuePr(>|t|) (Intercept) 0.2614370.0612644.2672.68E-05*** area -0.00160.01633-0.0980.92196 lat 0.0058540.0017963.2590.00125**

26. How to read lm results in R In log 10 mean clutch size of a lizard on New Caledonia (latitude 21, log island area 4.27 sq km, we will ignore it for a second that area was not significant) will be: Intercept+slope*area+slope*latitude 0.261-0.0016 (slope) *4.27 (area) +0.00585 (slope) *21 latitude = 0.377 Estimate Std. Errort valuePr(>|t|) (Intercept) 0.2614370.0612644.2672.68E-05*** area -0.00160.01633-0.0980.92196 lat 0.0058540.0017963.2590.00125** Or 2.38 eggs (10 in the power of 0.377) When the predictor is continuous, R reports its slope with its SE and, t and p values for it

27. How to read lm results in R In ANCOVA we have both categorical and continuous predictors, R reports intercept for the first and slopes for the second, with their SE, t and p values accordingly model4<-lm(brood~mass+lat+type,data=island) summary(model4) EstimateStd. Errort valuePr(>|t|) (Intercept) 1.1770110.1342188.7695.62E-13*** mass -0.24020.035362-6.7932.66E-09*** lat -0.018640.003677-5.0683.00E-06*** typeLand_bridge -0.21750.108051-2.0130.0479* typeOceanic -0.065510.095145-0.6890.4933 Here we predict the number of broods in a year with mass, latitude and 3 categories of island types (continental, Land bridge and Oceanic)

28. How to read lm results in R ANCOVA: categorical and continuous predictors Residual standard error: 0.2771 on 72 degrees of freedom (242 observations deleted due to missingness) Multiple R-squared: 0.478, Adjusted R-squared: 0.449 F-statistic: 16.48 on 4 and 72 DF, p-value: 1.25e-09 We can see here the R 2 values, df, F value etc. of the model Notice that R ignored empty (=NA) cells model4<-lm(brood~mass+lat+type,data=island) summary(model4) EstimateStd. Errort valuePr(>|t|) (Intercept) 1.1770110.1342188.7695.62E-13*** mass -0.24020.035362-6.7932.66E-09*** lat -0.018640.003677-5.0683.00E-06*** typeLand_bridge -0.21750.108051-2.0130.0479* typeOceanic -0.065510.095145-0.6890.4933

29. How to read lm results in R model4<-lm(brood~mass+lat+type,data=island) summary(model4) Because alphabetically continental<Land_bridge<Oceanic our intercept is for the first category: species on continental/ So the number of yearly broods of species on continental islands is significantly larger than of species on Land bridge islands and not significantly larger than of species on oceanic islands (notice: the difference is negative) In addition number of yearly broods decreases with the mass and with the increase in latitude (negative slope: higher brood frequency to small lizards on tropical islands EstimateStd. Errort valuePr(>|t|) (Intercept) 1.1770110.1342188.7695.62E-13*** mass -0.24020.035362-6.7932.66E-09*** lat -0.018640.003677-5.0683.00E-06*** typeLand_bridge -0.21750.108051-2.0130.0479* typeOceanic -0.065510.095145-0.6890.4933

30. In the categorical variables we have a problem: R calculates only the difference between each category and the first category in the alphabetic order. Here a comparison between land bridge islands and oceanic islands to continental islands. But R doesn’t report the difference between land bridge islands and oceanic islands. Moreover, it doesn’t give use SE and the difference from zero for both of these categories, just the difference from continental islands and the SE for this test (not the SE of the category it self) model4<-lm(brood~mass+lat+type,data=island) summary(model4) EstimateStd. Errort valuePr(>|t|) (Intercept) 1.1770110.1342188.7695.62E-13*** mass -0.24020.035362-6.7932.66E-09*** lat -0.018640.003677-5.0683.00E-06*** typeLand_bridge -0.21750.108051-2.0130.0479* typeOceanic -0.065510.095145-0.6890.4933 relevel

31. We can outsmart R, we can tell it what will be the first category that he will be comparing the rest to using the function relevel : Or relevel (2) model4<-lm(brood~mass+lat+type,data=island) summary(model4) model4a<-lm(brood~mass+lat+relevel(type, " Land_bridge " ),data=island) summary(model4a) model4b<-lm(brood~mass+lat+relevel(type, " Oceanic " ),data=island) summary(model4b)

32. relevel (3) Notice that the general model parameters stayed the same Residual standard error: 0.2771 on 72 degrees of freedom (242 observations deleted due to missingness) Multiple R-squared: 0.478, Adjusted R-squared: 0.449 F-statistic: 16.48 on 4 and 72 DF, p-value: 1.25e-09 model4a<-lm(brood~mass+lat+relevel(type, " Land_bridge " ),data=island) summary(model4a) Estimate Std. ErrorT valuetPr(>|t|) (Intercept) 0.959510.1489516.4421.16E-08*** mass -0.24020.035362-6.7932.66E-09*** lat -0.018640.003677-5.0683.00E-06*** relevel(type,Land_bridge)Continental 0.2175010.1080512.0130.0479* relevel(type,Land_bridge)Oceanic 0.1519890.0800721.8980.0617. We can outsmart R, we can tell it what will be the first category that he will be comparing the rest to using the function relevel :

33. PredictorResponsetestIn R CategoricalSuccess/failureBinomial**binom.test CategoricalCountsChi-square/Gchisq.test CategoricalcontinuousANOVA*aov continuous Regression/correlationlm continuousCategorical/countsChi-square/ANOVAlm Categorical, multiple predictors continuousMulti-way ANOVAaov continuous, multiple predictors continuousMultiple regressionlm Both categorical & continuous predictors continuousANCOVAlm If the assumptions of the parametric models (equality of variance, normal distribution of the residuals) are met: *t-test if there are only 2 categories ** or logistic regression: https://www.youtube.com/watch?v=EocjYP5h0cE Choosing the right model

34. Interactions It is easy to understand it graphically: in the example there is a categorical variable with two levels (dashed and full lines) and a continuous variable (on the X axis) Null hypothesis a. response Continuous predictor response c. categorical significant, continuous not e. response Continuous predictor b. response Continuous predictor Both significant, no interaction d. response Continuous predictor Both significant with interaction f. response Continuous predictor Continuous significant, categorical not, there is interaction Continuous significant, categorical not, there is no interaction

35. Interactions in R We use ‘+’ between the predictor variables. For interaction we will use ‘:’. If the predictor variable has interaction and main effect we will use ‘*’. For example the model that tries to predict grade of a course according to how much we studied, age of the lecturer, how much we prayed and if there are test from previous years Here we asked for two interactions: between prays and past year tests, and between lecturer age and prayer lm(y~a*b) lm(y~a+b+c+a:b) model<- lm(Grade~days_studied+professor_age+prayer_number*reconstruction_exist +professor_age:prayer_number, data=grades)

36. Important: The basic assumption of multi predictor tests is that there is no correlation between the predictors High correlation between two predictors is called multi-co- linearity and can be expressed by the tolerance (1-R2) or by Variance Inflation Factors (VIF=1/tolerance) If there is a strong multi-co-linearity then the model is not stable and parameter estimation might be incorrect Don’t add to your model predictors with high correlation among them

37. Always, Always, Always the more predictors we’ll add the more variance will be explained The ratio is monotonous – and trivial: in the worst case the parameter estimate of additional variables will be zero (for example number of species = 22.5 + 87*latitude + 0*number of mandates of religious parties in the same area But the parameter estimate will never be exactly zero – it will just be very small – lets say a species is added for every 5120 mandates added to Shas, or a species is subtracted for every 974 mandates that are added to the Bait Hayehudi Our R 2 raises from 0.45 to 0.45007 – is it worth it? Model selection

38. With every statistical question we can explain 100% of the variance by have the same number of variable as the sample size Example? What is your height? Model selection But, what the predictive ability of this model gives us for the next datum? http://en.wikipedia.org/wiki/Overfitting

39. The more predictor variables we add more of the variance will be explained Our goal as scientists is to explain the maximal number of phenomena with the minimal number of predictor variables Have you heard about Occam's razor? If we have many predictors we very much like to know if it’s worth to complicate our life for them Model selection

40. We can test which predictors in the model are significant Model selection We will start with a very complicated model and we’ll remove each time the predictor (or the interaction) that has the highest p-value – until all p-values are lower than 0.05 (or any other threshold). The final model will be MAM = minimum adequate model Example: we are trying to explain the clutch size of different lizard species (response variable: clutch size) with data for body mass, their environmental temperature, elevation, and the number of broods Backwards (stepwise) elimination 1. Using the p value

41. Estimatesetp (Intercept) 1.5850.6282.5230.012* mass 4.0121.5072.6620.008** temp 0.0020.0330.0560.956 elevation 0.0000.0010.2400.810 broods -0.2340.098-2.3780.018* We can just test which variables in the model are significant Model selection Lets start with the most complicated model: clu<-read.table(“eggs.txt”,header=T) model1<-lm(clutch~mass+temp+elevation+broods, data=clu) Example: we are trying to explain the clutch size of different lizard species (response variable: clutch size) with data for body mass, their environmental temperature, elevation, and the number of broods

42. Estimatesetp (Intercept) 1.60980.43323.71600.0002*** mass 4.01041.50582.66300.0079** elevation 0.00040.00140.24500.8068 broods -0.23170.0917-2.52800.0117* Model selection We will remove the temperature and run the model again: model2<-lm(clutch~mass+elevation+broods) We can just test which variables in the model are significant Example: we are trying to explain the clutch size of different lizard species (response variable: clutch size) with data for body mass, their environmental temperature, elevation, and the number of broods

43. Estimatesetp (Intercept) 1.56880.39933.92900.0001*** mass 4.37350.256617.04200.0000*** broods -0.23310.0914-2.55000.0110* Model selection Clutch size is affected by body mass and the number of yearly broods, and that’s it model3<-lm(clutch~mass+broods) All the variables are significant STOP! model3 = MAM We can just test which variables in the model are significant We will remove the elevation and run the model again:

44. forward addition Model selection We can start with the simplest model, and add a new variables each time, and leave it in the model if its p- value is lower than 0.05 (or any other threshold) model1a<-lm(clutch~mass) model2a<- lm(clutch~mass+broods) model3a<- lm(clutch~mass+broods+elevation) When we get to a model with non-significant predictors (model3a in our example) we will stop and choose the previous model (model2a in our example) as MAM We can just test which variables in the model are significant

45. forward addition Model selection Notice: not all the possible variation among the predictors (and their interactions) are tested in forward addition and backward elimination, it is possible that the best combination was not tested On the other hand the number of models increases in the power of the number of predictors we use in the model, so its not practical to try all the possible combinations unless you have a strong some computer and some time We can just test which variables in the model are significant We can start with the simplest model, and add a new variables each time, and leave it in the model if its p- value is lower than 0.05 (or any other threshold)

46. Akaike Information Criterion Comparing two models based on two parameters: how “good” is the model (the accuracy in which the reality in it is described) compared to its complexity (how many parameters we estimated) AIC = 2k-2ln(L) K is the number of parameters and L is the maximum likelihood of the model (without getting into details in this case it expresses the residual sum of squares – the smaller it is the better the model. We can also write [AIC = 2k+n[ln(RSS)) Model selection Alternative way for model selection http://en.wikipedia.org/wiki/Residual_sum_of_squares Hirotsugu Akaike http://en.wikipedia.org/wiki/Akaike_information_criterion The lower the AIC the better the model

47. Akaike Information Criterion AIC = 2k-2ln(L) Model selection Hirotsugu Akaike AIC rewards descriptive accuracy via the maximum likelihood (High L), and penalizes lack of parsimony according to the number of free parameters (high K) Notice that the model support will be stronger with the decrease in the AIC In R model comparison based on AIC is very simple AIC(model1,model2,model3)

48. Lets go back to the lizards Model selection model1<-lm(clutch~mass+temp+elevation+broods) model2<-lm(clutch~mass+elevation+broods) model3<-lm(clutch~mass+broods) AIC(model1,model2,model3) dfAIC model164430.458 model254428.461 model344426.521 Here we can see that model 3 is the best (it has the lowest AIC score) Example: we are trying to explain the clutch size of different lizard species (response variable: clutch size) with data for body mass, their environmental temperature, elevation, and the number of broods

49. AIC doesn’t allow us to test how good is one model just to compare between two models that are based on the same data The AIC score is meaningless by itself : we can’t compare AIC score of two models that ask different questions or based on different data Akaike Information Criterion Moreover, rule of thumb is that you can’t decide which model is better if their AIC difference is lower than 2 AIC(model1,model2,model3)

50. Akaike Information Criterion dfAIC∆AIC model344426.5210 model254428.4611.94 model164430.4583.937 We will arrange the models based on their AIC scores from the lowest (the best) to the highest and calculate for each the difference between each AIC score and the lowest AIC score – to get the ΔAIC of each model. We can’t say that model 3 is better than model 2 because the difference between their AIC score is lower than 2 AIC(model1,model2,model3) Rule of thumb is that you can’t decide which model is better if their AIC difference is lower than 2

51. Akaike Information Criterion Notice: rule of thumb does not apply to the model with the best model nested within it AIC(model1,model2,model3) AIC = 2k-2ln(L) This is because if we add a parameter to the best model it will never increase the AIC score in more than 2 In nested models we can say that a simpler model is as good as a more complicated model with ΔAIC of 2 or lower – but not that a more complicated model is as good as nested model with ΔAIC of 2 or lower Arnold 2010. Uninformative parameters and model selection using Akaike's information criterion. Journal of Wildlife Management, 74: 1175-1178. Rule of thumb is that you can’t decide which model is better if their AIC difference is lower than 2

52. Model selection: AIC and other animals There is no reason, and it is wrong, to calculate the AIC score for a single test (this is similar to saying that a basketball team sinked in a specific game 79 points – it has no value if we don’t know how many point the opponent team had) The model with the lowest AIC can defiantly have variables with p-values higher than 0.05 (AIC is relatively permissive for the parameters it allows) AIC and p-values come from different statistical philosophies and you shouldn’t mix them* *but see Johnson 2014. Revised standards for statistical evidence. PNAS 110: 19175-19176, who suggest the philosophies can be reconciled – and that p values <<0.05 should be used See also a lively debate about p values and AIC in Ecology (95 #3, 2014: www.esajournals.org/toc/ecol/95/3 )

53. 1.AICc BIC: -2*ln L + k*ln(n) Correction to AIC for models with small sample size (חבילת R http://cran.r-project.org/web/packages/AICcmodavg/AICcmodavg.pdf)http://cran.r-project.org/web/packages/AICcmodavg/AICcmodavg.pdf 2. AIC weights “Akaike weights are used in model averaging. They represent the relative likelihood of a model. To calculate them, for each model first calculate the relative likelihood of the model, which is just exp(-0.5 * ∆AIC score for that model). The Akaike weight for a model is this value divided by the sum of these values across all models.” † † http://www.brianomeara.info/tutorials/aichttp://www.brianomeara.info/tutorials/aic Aho et al. 2014. Model selection for ecologists: the worldviews of AIC and BIC. Ecology, 95: 631-636. †† Wagenmakers & Farrell 2004 3. Baysian Information Criterion, BIC Less permissive 1 than AIC to large number of parameters †† AICc = AIC+(2k*[k+1]/[n-k-1]) 1. Notice : k is multiplied by sample size! In the AIC sample size is not incorporated Model selection: variation on the AIC theme

54. Remember for each research: "No statistical procedure can substitute for serious thinking about alternative evolutionary scenarios and their credibility" Westoby, Leishman & Lord 1995. On misinterpreting 'phylogenetic correction. J. of Ecology 83: 531-534.

55. YNET : 23.07.2014 מספר הרקטות שנורו לישראל, כמה יורטו ע"י כיפת ברזל (וכמה לא יורטו): לפני ואחרי הכניסה הקרקעית

56. תרגיל בית לשימוש בשעת אזעקות: מדלו ב-R האם הכניסה הקרקעית הורידה את ירי הרקטות לעבר ישראל? האם היא הורידה את הירי המדוייק (=רקטות ש"כיפת ברזל" טרחה ליירט) או את הפרופורציה שלו מכל הירי? האם הזמן עד כה פעל לטובתנו?