1. © 2013 Lloyd P. Rieber Statistics in Education for Mere Mortals Inferential Statistics Lloyd P. Rieber Professor of Learning, Design, & Technology The University of Georgia …an open, online course

2. © 2013 Lloyd P. Rieber Huh? Example of reporting a test of a statistical hypothesis: Percentage means and standard deviations are contained in Table 1. A significant main effect was found on the test of learning outcomes, F(1, 97) = 9.88, p <.05, MS error = 190.51. Participants given the educational game scored significantly higher (mean=91.5%) than participants who were not given the game (mean=71.2%).

3. © 2013 Lloyd P. Rieber Running an Olympic Marathon: No Significant Difference? 26 miles, 385 yards Times of top 2 women runners at 2012 Olympics in London: – 1. Tiki Gelana, Ethiopia, 2:23:07 – 2. Priscah Jeptoo, Kenya, 2:23:12 Is a difference of 5 seconds statistically significant?

4. © 2013 Lloyd P. Rieber Total votes cast for Bush or Gore in 2000: No Significant Difference?

5. © 2013 Lloyd P. Rieber A statistically significant difference is not necessarily an important difference

6. © 2013 Lloyd P. Rieber Ready to buy? I invented a new way to teach ________. When I compared my invention to the traditional approach, students scored 5% higher on the test. Are you convinced it is a better approach?

7. © 2013 Lloyd P. Rieber Convinced yet? TrialScore of My Invention Score of Traditional Approach 18580 28682

8. © 2013 Lloyd P. Rieber Convinced yet? TrialScore of My Invention Score of Traditional Approach 18580 28682 38479 48581 58385

9. © 2013 Lloyd P. Rieber Hypothesis Testing An Example of Inferential Statistics Hypothesis testing basically answers the question: If we repeat the experiment how many times, out of 100, would my invention have to produce a higher score for you to be convinced it is truly a better approach?

10. © 2013 Lloyd P. Rieber Hypothesis Testing An Example of Inferential Statistics The hypothesis that there is no difference is called the null hypothesis. p is the probability that the experimental results show a difference, when in fact there is no difference. So, we hope that this probability is very small. How low is acceptable? General answer: No more than 5 times out of 100, or p <.05

11. © 2013 Lloyd P. Rieber Does My Invention Work? Let ’ s consider the research outcome possibilities…

12. © 2013 Lloyd P. Rieber RealityMy conclusion, based on experiment It truly does not work It works. It doesn’t work. It really works!It works. It doesn’t work. Type I Type II ERROR

13. © 2013 Lloyd P. Rieber Experimental Designs Experimental design is used to identify cause-and- effect relationships. The researcher considers many possible factors that might cause or influence a particular condition/phenomenon. The researcher controls for all influential factors except those having possible effects. The importance of random selection and assignment of participants.

14. © 2013 Lloyd P. Rieber Independent and Dependent Variables Variable: any quality or characteristic in a research investigation that has two or more possible values. Independent variable: a possible cause of something else (the manipulated variable) Dependent variable: a variable that is potentially influenced by the independent variable.

15. © 2013 Lloyd P. Rieber Comparing Means: Hypothesis Testing Comparing two means – The t statistic – t tests Comparing more than two means – The F statistic – Analysis of Variance

16. © 2013 Lloyd P. Rieber Two Typical Uses of t Tests in Education & Training Dependent (Correlated) t Test One-group pretest-posttest design You select one group of people at random for your evaluation. Procedure: Administer a pretest (observation 1); Group participates in treatment (i.e. activity, intervention, etc.); Administer a posttest (observation 2); Is there a difference between the pretest mean and posttest mean? GroupTime Group1Obs 1TrtObs 2

17. © 2013 Lloyd P. Rieber You select two groups of people at random for your evaluation. Procedure: You randomly assign the people to the two groups (with equal numbers in each group); Group 1 participates in treatment (i.e. activity, intervention, etc.); Group 2 does not; Administer a posttest to both groups (observation); Is there a difference between the means of the two groups? Two Typical Uses of t Tests in Education & Training Independent t Test Posttest-only control group design GroupTime Random Assignment Group1TrtObs Group2Obs

18. © 2013 Lloyd P. Rieber What If You Have More Than Two Groups?

19. © 2013 Lloyd P. Rieber Analysis of Variance F= Between Groups Variance Within Groups Variance

20. © 2013 Lloyd P. Rieber Sources of Error (think variability)

21. © 2013 Lloyd P. Rieber Remember the hanging chads? A good example of “error in measurement.”

22. © 2013 Lloyd P. Rieber Quick, answer this question… Bob has five cookies. Jim has four cookies. After Bob gives Jim two more cookies, how many does he have? 7 6 5 4 3 2 Bob Jim Answer choices

23. © 2013 Lloyd P. Rieber Degrees of Freedom

24. © 2013 Lloyd P. Rieber Understanding Degrees of Freedom I’m thinking of 5 numbers with a mean of 46, can you guess what they are? 1. __________ 2. __________ 3. __________ 4. __________ 5. __________

25. © 2013 Lloyd P. Rieber Understanding Degrees of Freedom I’m thinking of 5 numbers with a mean of 46, can you guess what they are? 1. 34 2. 44 3. 93 4. 18 5. __________ 41

26. © 2013 Lloyd P. Rieber F(1, 97) = 9.88, p <.05, MS error = 190.51 F= Between Groups Variance Within Groups Variance = MS treatment MS error SS treatment SS error df treatment df error = df treatment df error = = Number of groups - 1 N total - Number of groups

27. © 2013 Lloyd P. Rieber Girl Scout Cookie Sales Boxes of CookiesDeviation Scores Xx2x2 2818324 1111 1000 5-525 4-636 2-864 ∑X=60∑x=0∑x 2 =450 Example taken from Spatz, 1997.

28. © 2013 Lloyd P. Rieber Statistics in Education for Mere Mortals Inferential Statistics Lloyd P. Rieber Professor of Learning, Design, & Technology The University of Georgia …an open, online course