© 2013 Lloyd P. Rieber Statistics in Education for Mere Mortals Inferential Statistics Lloyd P. Rieber Professor of Learning, Design, & Technology The University of Georgia …an open, online course

© 2013 Lloyd P. Rieber Huh? Example of reporting a test of a statistical hypothesis: Percentage means and standard deviations are contained in Table 1. A significant main effect was found on the test of learning outcomes, F(1, 97) = 9.88, p <.05, MS error = Participants given the educational game scored significantly higher (mean=91.5%) than participants who were not given the game (mean=71.2%).

© 2013 Lloyd P. Rieber Running an Olympic Marathon: No Significant Difference? 26 miles, 385 yards Times of top 2 women runners at 2012 Olympics in London: – 1. Tiki Gelana, Ethiopia, 2:23:07 – 2. Priscah Jeptoo, Kenya, 2:23:12 Is a difference of 5 seconds statistically significant?

© 2013 Lloyd P. Rieber Total votes cast for Bush or Gore in 2000: No Significant Difference?

© 2013 Lloyd P. Rieber A statistically significant difference is not necessarily an important difference

© 2013 Lloyd P. Rieber Ready to buy? I invented a new way to teach ________. When I compared my invention to the traditional approach, students scored 5% higher on the test. Are you convinced it is a better approach?

© 2013 Lloyd P. Rieber Convinced yet? TrialScore of My Invention Score of Traditional Approach

© 2013 Lloyd P. Rieber Convinced yet? TrialScore of My Invention Score of Traditional Approach

© 2013 Lloyd P. Rieber Hypothesis Testing An Example of Inferential Statistics Hypothesis testing basically answers the question: If we repeat the experiment how many times, out of 100, would my invention have to produce a higher score for you to be convinced it is truly a better approach?

© 2013 Lloyd P. Rieber Hypothesis Testing An Example of Inferential Statistics The hypothesis that there is no difference is called the null hypothesis. p is the probability that the experimental results show a difference, when in fact there is no difference. So, we hope that this probability is very small. How low is acceptable? General answer: No more than 5 times out of 100, or p <.05

© 2013 Lloyd P. Rieber Does My Invention Work? Let ’ s consider the research outcome possibilities…

© 2013 Lloyd P. Rieber RealityMy conclusion, based on experiment It truly does not work It works. It doesn’t work. It really works!It works. It doesn’t work. Type I Type II ERROR

© 2013 Lloyd P. Rieber Experimental Designs Experimental design is used to identify cause-and- effect relationships. The researcher considers many possible factors that might cause or influence a particular condition/phenomenon. The researcher controls for all influential factors except those having possible effects. The importance of random selection and assignment of participants.

© 2013 Lloyd P. Rieber Independent and Dependent Variables Variable: any quality or characteristic in a research investigation that has two or more possible values. Independent variable: a possible cause of something else (the manipulated variable) Dependent variable: a variable that is potentially influenced by the independent variable.

© 2013 Lloyd P. Rieber Comparing Means: Hypothesis Testing Comparing two means – The t statistic – t tests Comparing more than two means – The F statistic – Analysis of Variance

© 2013 Lloyd P. Rieber Two Typical Uses of t Tests in Education & Training Dependent (Correlated) t Test One-group pretest-posttest design You select one group of people at random for your evaluation. Procedure: Administer a pretest (observation 1); Group participates in treatment (i.e. activity, intervention, etc.); Administer a posttest (observation 2); Is there a difference between the pretest mean and posttest mean? GroupTime Group1Obs 1TrtObs 2

© 2013 Lloyd P. Rieber You select two groups of people at random for your evaluation. Procedure: You randomly assign the people to the two groups (with equal numbers in each group); Group 1 participates in treatment (i.e. activity, intervention, etc.); Group 2 does not; Administer a posttest to both groups (observation); Is there a difference between the means of the two groups? Two Typical Uses of t Tests in Education & Training Independent t Test Posttest-only control group design GroupTime Random Assignment Group1TrtObs Group2Obs

© 2013 Lloyd P. Rieber What If You Have More Than Two Groups?

© 2013 Lloyd P. Rieber Analysis of Variance F= Between Groups Variance Within Groups Variance

© 2013 Lloyd P. Rieber Sources of Error (think variability)

© 2013 Lloyd P. Rieber Remember the hanging chads? A good example of “error in measurement.”

© 2013 Lloyd P. Rieber Quick, answer this question… Bob has five cookies. Jim has four cookies. After Bob gives Jim two more cookies, how many does he have? Bob Jim Answer choices

© 2013 Lloyd P. Rieber Degrees of Freedom

© 2013 Lloyd P. Rieber Understanding Degrees of Freedom I’m thinking of 5 numbers with a mean of 46, can you guess what they are? 1. __________ 2. __________ 3. __________ 4. __________ 5. __________

© 2013 Lloyd P. Rieber Understanding Degrees of Freedom I’m thinking of 5 numbers with a mean of 46, can you guess what they are? __________ 41

© 2013 Lloyd P. Rieber F(1, 97) = 9.88, p <.05, MS error = F= Between Groups Variance Within Groups Variance = MS treatment MS error SS treatment SS error df treatment df error = df treatment df error = = Number of groups - 1 N total - Number of groups

© 2013 Lloyd P. Rieber Girl Scout Cookie Sales Boxes of CookiesDeviation Scores Xx2x ∑X=60∑x=0∑x 2 =450 Example taken from Spatz, 1997.

© 2013 Lloyd P. Rieber Statistics in Education for Mere Mortals Inferential Statistics Lloyd P. Rieber Professor of Learning, Design, & Technology The University of Georgia …an open, online course