1. Practical statistics for Neuroscience miniprojects Steven Kiddle Slides & data : http://bit.ly/1Jaor2r

2. We are unlikely to finish all the slides Keep them, they may be helpful for your miniproject

3. Lecture outline Taught component – How to present statistics – Hypothesis testing – Normal distribution Practical component – Plots – Statistical tests – Multiple testing corrections

4. Taught component

5. Why are statistics important? Help to make science more repeatable and objective Help you to interpret your results Help you to assess the level of evidence you have supporting a hypothesis A vital skill for a scientific career!

6. How to report statistics Always report: (1)Statistical software you used (2)Statistical tests you used (3)Significance level you used (4)Sample size I checked these in 17 randomly chosen neuroscience project posters

7. How not to report statistics! I found that: (1)16/17 didn’t report the statistical software used (2)11/17 didn’t report the statistical tests used (3)9/17 didn’t report the significance level used (4)2/17 didn’t report the sample size!

8. Commonly used analysis methods Plotting: -Box plots -Line plots Hypothesis testing – T-test – ANOVA – Chi-squared

9. Hypothesis testing Two types of hypothesis – Null hypothesis (H 0 ) Usually that there are no differences between groups or that two variables are unrelated – Example : (H 0 ) Smoking and lung cancer are unrelated – Alternative hypothesis (H 1 ) There are differences between groups, or that two variables are related – Example : (H 1 ) Smoking and lung cancer are associated

12. Significance levels You accept the alternative hypothesis if the chance of your data being generated under the null hypothesis (the ‘p-value’) is beneath a pre-specified significance level α – Typically α = 0.05 You should state the significance threshold you use in your report

14. Multiple hypothesis testing I Suppose you have a significance threshold of α = 0.05 Suppose that you measure 100 variables that are NOT related to a disease You perform 100 hypothesis tests to compare your variables to disease state H 0 : Variable is not affected by disease state H 1 : Variable is affected by disease state For how many variables do you expect to reject the null hypothesis (H 0 ) even though its true?

15. Multiple hypothesis testing II α = 0.05 means that if the null hypothesis (H 0 ) is true, we would expect to reject it 5% of the time So if H 0 is true and we did 100 tests, we would expect to reject H 0 5 times by chance alone That is bad, these findings will not replicate How do we stop it? Multiple testing corrections

16. Bonferroni correction – If we want α = 0.05, instead use α = 0.05/n where n is the number of tests you want to use – So for 100 tests, we would use α = 0.0005, and would only have 5% chance of any test rejecting the null hypothesis Benjamini-Hochberg correction – Popular alternative

18. Normal distribution

19. Tests that rely on assumptions of normality T-tests ANOVA / linear models

20. How to check if you data is normally distributed Histograms Statistical tests Can apply to data But better to apply to residuals of the models – For t-test, that means looking at the groups separately – For ANOVA, that means extracting residuals from the model

21. What do you do if your data is not normally distributed? If sample size is really small – Nothing you can do – use test anyway If data is skewed – Transform data (e.g. log? square root?) Use non-parametric tests – Mann-whitney U instead of T-test – Spearman’s Rank Correlation Last resort - Remove outliers? – Systematically and preferably only if you know what causes them

22. How to present plots Label both axes – Large enough to read Show units If using stars (*) for significance levels, explain what *, **, *** means Lunnon et al., (2012) Journal of Alzheimer’s Disease

23. How to present statistics I Say what statistical software used, e.g. – SPSS, STATA, R, MATLAB, etc Say what the sample size is Say what statistical test is being performed – T-test, ANOVA, chi-squared, etc Say what significance level you are using for the study – Think, is it appropriate given my sample size and number of hypotheses being tested?

24. How to present statistics II Report p-value – And/or multiple testing corrected p-value E.g. Q-values for Benjamini-Hochberg Report coefficient ( β ), and ideally it’s standard error for each reported statistic – This can be more informative than a p-value, especially for small datasets

25. How to present statistics III A more complete guide, tailored to SPSS and specific tests is given at: http://statistics-help-for-students.com/

26. Be cautious in your interpretations Correlation does not equal causation! Can you hypothesise a mechanism by which causation could occur?

27. Why does correlation not equal causation? It looks like the variables are correlated when they are not – How this happens? By chance, especially when multiple testing is performed but not corrected for Variables are truly correlated but there is either: – Reverse causation – Confounding by other variables

28. Confounding

30. Statistical software Excel – Point and click, quite limited SPSS – Point and click, a little limited STATA – Command line R, MATLAB, etc – Command line, very useful, steep learning curve

31. http://www.r-project.org/ http://www.rstudio.com/

32. R introduction

33. Practical component - SPSS Data is faked to show large differences, real data will not be so clear cut

34. Outline Data Tests for normality Plots T-test ANOVA Chi-squared Non-parametric tests

35. Data Create folder in ‘My Documents’ Download data and save in your new folder: Slides & data : http://bit.ly/1Jaor2rhttp://bit.ly/1Jaor2r Open zip folder Double click on ‘neuroscience_example.sav’ to open SPSS

36. Introduction to the data 5 variables – a, b, c, d, e 2 are binary – a, b 3 are continuous – c, d, e

37. Normality checks I Need to check data is normally distributed when we want to apply – T-test – ANOVA – Linear regression

38. Normality checks II Let’s see if the variable ‘d’ is normally distributed

39. Normality checks III Can see that the data has two peaks Rejects the null hypothesis that the data is normally distributed

40. Normality checks III Now we take into account variable ‘a’, we find that ‘d’ is normally distributed when we take into account ‘a’

41. Plots Histograms (shown in normality check) – Show distribution of a continuous variable Boxplots – Show the distribution of a continuous variable between groups Line plot/scatter plot – Shows the relationship between two continuous variables

42. Generating a boxplot I

43. Generating a boxplot II

44. Generating a boxplot III

45. Generating a boxplot IV Double click on plot to label axis

46. Labelling a plot I Double click to change label

47. How to present plots Label both axes – Large enough to read Show units If using stars (*) for significance levels, explain what *, **, *** means Lunnon et al., (2012) Journal of Alzheimer’s Disease

48. Labelling a plot II

49. Saving plots Rename document and save in your folder You can now open the document and extract the plot as an image

50. Boxplot exercises Make a few more boxplots comparing binary variables to continuous variables Try adding labels Try saving Try to interpret the boxplot – Do you see differences between the groups

51. Generating a line plot I

52. Generating a line plot II

53. T-test Compares a binary variable (yes/no) to a continuous variable Example null hypothesis – Mean height is the same across males and females Example alternative hypothesis – Mean height is different between males and females

54. Performing a t-test I

55. Performing a t-test II Descriptive statistics

56. Performing a t-test II Levene test null hypothesis : the variance (and standard deviation) of the two groups are the same. At the significance level 0.05 we can reject the null hypothesis. Therefore we should use the second row (‘Equal variances not assumed’).

57. Performing a t-test II T-test null hypothesis : the mean of the two groups are the same At the significance level 0.05 we can reject the null hypothesis (p-value is less than 0.001). I.e. the data supports the fact that the variable ‘d’ is different between the two groups.

58. How to report findings (option A) A two-sample t-test assuming unequal variances performed in SPSS showed differences in variable ‘d’ between groups 0 (N = 19) and 1 (N = 31) in variable ‘a’ at the 5% significance level (mean difference = 1.6, standard error = 0.89, p-value < 0.001 ).

59. How to report findings (option B) Materials and methods: – Statistical analysis Statistical analysis was performed in SPSS 20. Group differences were analysed using two sample t-test assuming unequal variances. A significance level of 5% was applied to all hypothesis tests. Results: – Variable ‘d’ was found to differ between groups 0 (N = 19) and 1 (N = 31) in variable ‘a’ (mean difference = 1.6, standard error = 0.89, p-value < 0.001 ).

60. How that might look Materials and methods: – Statistical analysis Statistical analysis was performed in SPSS 20. Group differences were analysed using two sample t-test assuming unequal variances. A significance level of 5% was applied to all hypothesis tests. Results: – Change in blood glucose levels differed between males (N = 19) and females (N = 31) (mean difference = 1.6 ng/ml, standard error = 0.89, p-value < 0.001 ).

61. How to present statistics A more complete guide, tailored to SPSS and specific tests is given at: http://statistics-help-for-students.com/

62. One-way ANOVA Extension of t-test idea Compares a binary variable (yes/no) to a several variables, continuous or nominal (including binary) Example null hypothesis – Mean height is the same across males and females, regardless of age Example alternative hypotheses – Mean height is different between males and females – Mean height differs across ages

63. Performing ANOVA I

64. Performing ANOVA II Which variables differ between variable ‘a’ group 0 and 1? How would you perform a Bonferroni multiple testing correction? How would you report these findings? (clue: http://statistics-help-for-students.com/ )http://statistics-help-for-students.com/

65. Chi-squared test Compares multiple nominal variables Example null hypothesis – Lung cancer and smoking are unrelated Example alternative hypotheses – Smokers are more likely to have lung cancer

66. Performing chi-squared I

67. Performing chi-squared II How would you report these findings? (clue: http://statistics-help-for-students.com/ )http://statistics-help-for-students.com/

68. Non-parametric tests If a continuous variable is not normally distributed, using parametric tests may give you misleading results – T-test and ANOVA are parametric tests Solution, use non-parametric tests – Such as Mann-Whitney U Spearman’s Rank Correlation

69. Mann-Whitney U A non-parametric equivalent of a t-test

70. Performing Mann-Whitney U I

71. Performing Mann-Whitney U II How would you report these findings? (clue: http://statistics-help-for-students.com/ )http://statistics-help-for-students.com/

72. Spearman’s Rank Correlation A non-parametric equivalent of correlation, or a one-way ANOVA between a binary variable and a continuous variable

73. Performing Spearman’s Rank Correlation I

74. Performing Spearman’s Rank Correlation II How would you report these findings? (clue: http://statistics-help-for-students.com/ )http://statistics-help-for-students.com/