Statistics (and SoTL Projects) Jim Smith Michigan State University Lyman Briggs College It’s as much who you know, as what you know! n “Lies, Damned Lies, and Statistics” n A Case Study in Statistics n Use of a Stats Analytical Framework n Statistics 101 n Some Basics n Which Statistical Test Should You Use in Your SoTL Work? n Some Examples

Learning Goals n By the end of this session, scholars will (have): Understand the need for and the role(s) of statistical tests in science education research; Be able to explain some basic tenets of statistics; Explored some literature examples of the application of statistics to SoTL work; and Begin to formulate a statistical framework for their Biology Scholars research project.

Statistics and SoTL Projects n Did student learning increase as a result of my cell model assignment? n Do representations in comic book form improve student understanding of photosynthetic processes? n Are there differences in the preparedness and motivation levels of students who W/D/F in Introductory Biology? n Does increased metacognition decrease the W/D/F rate in Introductory Biology? Statistics are important components of strategies to answer these questions!

With respect to Statistics: n Where would you place yourself on the following scale? 1.From my perspective, there’s “Lies, Damned Lies, and Statistics” 2.I agree with the above statement somewhat, but not strongly. 3.I am halfway between “Damned Lies” and “Comfort”. 4.I agree with the below statement, but not strongly. 5.I am completely comfortable with statistics as they apply to SoTL work.

The Problem – Young and Fry (2008) n “While the results are reported to be statistically significant, I am not familiar enough with the Spearman’s Rho nonparametric correlation analysis to know whether the data are truly convincing. ” n ASM Bio Scholar Summer 2012

Young and Fry (2008)

Framework for Analysis of Statistical Inference Identify Variables of Interest  Formulate Hypotheses  Design Sampling Strategy  Collect Raw Data  Perform Statistical Test(s)   Inference (Level 1) Inference (Level 2+) Tasks Identify the appropriate elements in the article of interest and map them onto the boxes Decide if the statistical test was appropriately chosen and carried out Decide if the claims (inferences) are supportable at each level Tie the framework to your project Smith, Valles & Zeleke, in prep.

An Application – Young and Fry (2008) n What is the MAI, and how was it used to measure aspects of metacognition?

An Application – Young and Fry (2008) n What is the difference between metacognitive knowledge and metacognitive regulation?

Framework for Analysis of Statistical Inference Identify Variables of Interest  Formulate Hypotheses  Design Sampling Strategy  Collect Raw Data  Perform Statistical Test(s)   Inference (Level 1) Inference (Level 2+) Tasks Identify the appropriate elements in the article of interest and map them onto the boxes Decide if the statistical test was appropriately chosen and carried out Decide if the claims (inferences) are supportable at each level Tie the framework to your project Smith, Valles & Zeleke, in prep.

ASM BSP S12: STATS 101 What do I really need to know about Statistics?

ASM BSP S12: STATS 101 What do I really need to know about Statistics? If you are going to do SoTL research, you really do need a basic working knowledge of statistical inference

Fun, but no help for today

Extremely helpful!

What the difference between: n Experimental (or Research) Hypothesis and n Statistical Hypothesis

What the difference between: n Experimental (or Research) Hypothesis H A : Metacognition improves student learning and n Statistical Hypothesis

What the difference between: n Experimental (or Research) Hypothesis and n Statistical Hypothesis n H A : Course grade is correlated with MAI total

What the difference between: n Experimental (or Research) Hypothesis and n Statistical Hypothesis n H 0 : No correlation of Course grade with MAI total

The Problem – Young and Fry (2008) n The data allow us to reject the hypothesis that there is no correlation between Course Grade and MAI total. What is the difference between significance level and effect size?

What the difference between: n Experimental (or Research) Hypothesis H A : Metacognition improves student learning and n Statistical Hypothesis

What the difference between: n Discrete Numbers and n Continuous Numbers

With respect to Statistics: n Where would you place yourself on the following scale? 1.From my perspective, there’s “Lies, Damned Lies, and Statistics” 2.I agree with the above statement somewhat, but not strongly. 3.I am halfway between “Damned Lies” and “Comfort”. 4.I agree with the below statement, but not strongly. 5.I am completely comfortable with statistics as they apply to SoTL work.

What the difference between: n Discrete Numbers Categorical; membership known without error and n Continuous Numbers

What the difference between: n Discrete Numbers and n Continuous Numbers Not known without error; have a distribution

Overall: Mean = (74.70%) +/ SD TH: Mean = (81.57%) +/ SD ER:Mean = (72.40%) +/ SD MC: Mean = (70.90%) +/ SD Discrete or Continuous?

What the difference between: n Non-parametric Statistics For discrete data; for data that violate parametric assumptions and n Parametric Statistics

What the difference between: n Non-parametric Statistics e.g., Chi-squared tests, Mann-Whitney U test, Wilcoxon Rank Sums test and n Parametric Statistics

What the difference between: n Non-parametric Statistics and n Parametric Statistics

What the difference between: n Non-parametric Statistics and n Parametric Statistics e.g., t-test, ANOVA, Pearson’s r

Assumptions of Parametric Statistics (for continuous data) n Observations are unbiased and independent n Data are normally distributed n Equal variances across groups

Statistical Decision Tree From: Robert Gerwien, “A Painless Guide to Statistics: Online Resources for Biology”, ology/resources/statistics.html

Statistical Decision Tree Was the use of Spearman’s Rank Correlation test appropriate? Work with your group Yes/No, and why Young and Fry: Revisited

Young and Fry: Was theirs the appropriate statistical test? REM - You will find disagreement in the ranks (including among manuscript reviewers)

Test Time: What kind of data? Test? n Encouragingly, using t test analysis, we found that in four of our courses (both BSCI 223 courses, BSCI 424, and BSCI 422) there was significant improvement on the concept inventory scores from presurvey to postsurvey. From Marbach-Ad et al. 2009

Test Time: What kind of data? Test? From Drew & Triplett 2008

Test Time: What kind of data? Test? From Steck et al. 2012

Test Time: What kind of data? Test? From Steck et al. 2012

Test Time: What kind of data? Test? From Steck et al “I’m baffled…”

You may (probably) need help!! n Become friends/colleagues with a really smart biologist or social scientist who uses a statistical approach in their own work A lot of these folks have strong experimental design skills Ecologists and Evolutionary Biologists seem to be particularly strong n On-campus statistical consulting service?