1. Statistics (and SoTL Projects) Jim Smith Michigan State University Lyman Briggs College It’s as much who you know, as what you know! n “Lies, Damned Lies, and Statistics” n A Case Study in Statistics n Use of a Stats Analytical Framework n Statistics 101 n Some Basics n Which Statistical Test Should You Use in Your SoTL Work? n Some Examples

2. Learning Goals n By the end of this session, scholars will (have): Understand the need for and the role(s) of statistical tests in science education research; Be able to explain some basic tenets of statistics; Explored some literature examples of the application of statistics to SoTL work; and Begin to formulate a statistical framework for their Biology Scholars research project.

3. Statistics and SoTL Projects n Did student learning increase as a result of my cell model assignment? n Do representations in comic book form improve student understanding of photosynthetic processes? n Are there differences in the preparedness and motivation levels of students who W/D/F in Introductory Biology? n Does increased metacognition decrease the W/D/F rate in Introductory Biology? Statistics are important components of strategies to answer these questions!

4. With respect to Statistics: n Where would you place yourself on the following scale? 1.From my perspective, there’s “Lies, Damned Lies, and Statistics” 2.I agree with the above statement somewhat, but not strongly. 3.I am halfway between “Damned Lies” and “Comfort”. 4.I agree with the below statement, but not strongly. 5.I am completely comfortable with statistics as they apply to SoTL work.

5. The Problem – Young and Fry (2008) n “While the results are reported to be statistically significant, I am not familiar enough with the Spearman’s Rho nonparametric correlation analysis to know whether the data are truly convincing. ” n ASM Bio Scholar Summer 2012

6. Young and Fry (2008)

7. Framework for Analysis of Statistical Inference Identify Variables of Interest  Formulate Hypotheses  Design Sampling Strategy  Collect Raw Data  Perform Statistical Test(s)   Inference (Level 1) Inference (Level 2+) Tasks Identify the appropriate elements in the article of interest and map them onto the boxes Decide if the statistical test was appropriately chosen and carried out Decide if the claims (inferences) are supportable at each level Tie the framework to your project Smith, Valles & Zeleke, in prep.

8. An Application – Young and Fry (2008) n What is the MAI, and how was it used to measure aspects of metacognition?

9. An Application – Young and Fry (2008) n What is the difference between metacognitive knowledge and metacognitive regulation?

10. Framework for Analysis of Statistical Inference Identify Variables of Interest  Formulate Hypotheses  Design Sampling Strategy  Collect Raw Data  Perform Statistical Test(s)   Inference (Level 1) Inference (Level 2+) Tasks Identify the appropriate elements in the article of interest and map them onto the boxes Decide if the statistical test was appropriately chosen and carried out Decide if the claims (inferences) are supportable at each level Tie the framework to your project Smith, Valles & Zeleke, in prep.

11. ASM BSP S12: STATS 101 What do I really need to know about Statistics?

12. ASM BSP S12: STATS 101 What do I really need to know about Statistics? If you are going to do SoTL research, you really do need a basic working knowledge of statistical inference

13. Fun, but no help for today

14. Extremely helpful!

15. What the difference between: n Experimental (or Research) Hypothesis and n Statistical Hypothesis

16. What the difference between: n Experimental (or Research) Hypothesis H A : Metacognition improves student learning and n Statistical Hypothesis

17. What the difference between: n Experimental (or Research) Hypothesis and n Statistical Hypothesis n H A : Course grade is correlated with MAI total

18. What the difference between: n Experimental (or Research) Hypothesis and n Statistical Hypothesis n H 0 : No correlation of Course grade with MAI total

19. The Problem – Young and Fry (2008) n The data allow us to reject the hypothesis that there is no correlation between Course Grade and MAI total. What is the difference between significance level and effect size?

20. What the difference between: n Experimental (or Research) Hypothesis H A : Metacognition improves student learning and n Statistical Hypothesis

21. What the difference between: n Discrete Numbers and n Continuous Numbers

22. With respect to Statistics: n Where would you place yourself on the following scale? 1.From my perspective, there’s “Lies, Damned Lies, and Statistics” 2.I agree with the above statement somewhat, but not strongly. 3.I am halfway between “Damned Lies” and “Comfort”. 4.I agree with the below statement, but not strongly. 5.I am completely comfortable with statistics as they apply to SoTL work.

23. What the difference between: n Discrete Numbers Categorical; membership known without error and n Continuous Numbers

24. What the difference between: n Discrete Numbers and n Continuous Numbers Not known without error; have a distribution

25. Overall: Mean = 74.70 (74.70%) +/- 11.45 SD TH: Mean = 24.47 (81.57%) +/- 3.91 SD ER:Mean = 28.96 (72.40%) +/- 5.52 SD MC: Mean = 21.27 (70.90%) +/- 4.45 SD Discrete or Continuous?

26. What the difference between: n Non-parametric Statistics For discrete data; for data that violate parametric assumptions and n Parametric Statistics

27. What the difference between: n Non-parametric Statistics e.g., Chi-squared tests, Mann-Whitney U test, Wilcoxon Rank Sums test and n Parametric Statistics

28. What the difference between: n Non-parametric Statistics and n Parametric Statistics

29. What the difference between: n Non-parametric Statistics and n Parametric Statistics e.g., t-test, ANOVA, Pearson’s r

30. Assumptions of Parametric Statistics (for continuous data) n Observations are unbiased and independent n Data are normally distributed n Equal variances across groups

31. Statistical Decision Tree From: Robert Gerwien, “A Painless Guide to Statistics: Online Resources for Biology”, http://abacus.bates.edu/~ganderso/bi ology/resources/statistics.html

32. Statistical Decision Tree Was the use of Spearman’s Rank Correlation test appropriate? Work with your group Yes/No, and why Young and Fry: Revisited

33. Young and Fry: Was theirs the appropriate statistical test? REM - You will find disagreement in the ranks (including among manuscript reviewers)

34. Test Time: What kind of data? Test? n Encouragingly, using t test analysis, we found that in four of our courses (both BSCI 223 courses, BSCI 424, and BSCI 422) there was significant improvement on the concept inventory scores from presurvey to postsurvey. From Marbach-Ad et al. 2009

35. Test Time: What kind of data? Test? From Drew & Triplett 2008

36. Test Time: What kind of data? Test? From Steck et al. 2012

37. Test Time: What kind of data? Test? From Steck et al. 2012

38. Test Time: What kind of data? Test? From Steck et al. 2012 “I’m baffled…”

39. You may (probably) need help!! n Become friends/colleagues with a really smart biologist or social scientist who uses a statistical approach in their own work A lot of these folks have strong experimental design skills Ecologists and Evolutionary Biologists seem to be particularly strong n On-campus statistical consulting service?