Inferential Statistics Assoc. Prof. Dr. Şehnaz Şahinkarakaş

P Value There are many ways to make general inferences: Considering the sampling error Distribution of sample means Standard error of the mean (SEM) Confidence interval, etc Our focus will be on the most common inferential tests used in Social Sciences!

Practical versus Statistical Significance A result is statistically significant if it is unlikely to occur by chance! It is known as the significance level or p-value. In educational research, having .05 level of significance is accepted statistically significant (p=.05). However, while interpreting the results, you should also consider its practical importance as well!

Statistical significance If the findings hold true 95% of the time then p = 0.05 If the findings hold true 99% of the time then p = 0.01 If the findings hold true 99.9% of the time then p = 0.001

Inferential Statistics Descriptive statistics is used simply to describe what's going on in the data. Inferential statistics helps us reach conclusions that extend beyond the immediate data alone; i.e., to make inferences from our data to more general conditions E.g. to infer from the sample data what the population might think to make judgments of the probability of an observed difference between groups: is it a dependable one or one that might have happened by chance in this study?

Analyzing Categorical Data (X2) (non-parametric) The Chi-Square Test (X2) Used to analyze the categorical data. E.g. How many male and female teachers favor the new curriculum? If they do not differ significantly, you cannot generalize your findings (even if one of them is found to be more than the other) If there is a significant difference, you can safely say males (or females) favor the new curriculum.

To see the significance here, we need chi-square

Grading portfolios together with the teacher was very useful 44 Questions (Section B) Yes No X2 1 Grading portfolios together with the teacher was very useful 44 41.08 ** 2 It is a good idea that an outsider, besides the teacher, grades the portfolios 23 22 0.02 n.s. 3 It was good that all my essays were graded 32 13 8.02 * 4 It was good that I had my grades from my final drafts 45 -- *p<.01 **p<.001 n.s.= statistically not significant

Statistics for Experimental Studies

Inference Techniques for two variables One of the most common inferential test is t-test: to compare the average performance of two groups on a single measure to see if there is a difference. E.g. You might want to know whether attitudes of eighth-grade boys and girls towards learning English differ. You need to compare the average performance between these two groups using the t-test.

Types of t-tests Independent Samples Related/ Samples also called dependent means test Interval measures/ parametric Independent samples t-test* Paired samples t-test** Ordinal/ non-parametric Mann-Whitney U-Test Wilcoxon test * 2 experimental conditions and different participants were assigned to each condition ** 2 experimental conditions and the same participants took part in both conditions of the experiments

Independent sample t-test (parametric data for two independent groups) It is used to compare the mean scores of two different, independent groups. E.g. Mean scores on the achievement test of class A is 80 and class B is 85. If the t-test score is at or below .05 significance level, it is possible to say that the difference between these score is statistically significant.

Paired sample t-test (parametric data for the same group at two intervals) It is used to compare the mean scores of the same group before and after a treatment: it shows whether the change in the mean scores is significantly significant. PS. To calculate the t-test for correlated means, you need to pair the scores for each individual!

Sample t-test Table N Mean sd t p Group A 30 9.39 1.95 1.2 .625 Group B 33 9.63 1.72 There is no statistical significance in the means of Groups A and Group B.

N Mean sd t P Group A 82 69.36 8.05 3.26 .002 Group B 80 73.42 7.78 There is a statistically significant difference in the attitudes of the two groups towards reading books in English for fun (t=3.26, p<.05). Group B has a more positive attitude (X=73.42) towards reading books for fun than Group A (X=69.36).

Mann Whitney U Test (non-parametric data for two independent groups) Used when you do not assume that the dependent variable is a normally distributed interval variable . You need only assume that the variable is at least ordinal.  Can also be used with small number of participants (around 20) The Mann-Whitney test combines and ranks the data from sample 1 and sample 2 and calculates a statistic on the difference between the sum of the ranks of sample 1 and sample 2. *Used instead of Independent Sample T-Test

Wilcoxon (Signed Rank) Its use is similar to Mann Whitney U Test The Wilcoxon test for paired data ranks the absolute values of the differences between the paired data in sample 1 and sample 2 and calculates a statistic on the number of negative and positive differences (differences are calculated as sample 2 - sample 1). *Used instead of Pair Sample T-Test

Sample Whitney U/Wilcoxon Table Table 1 indicates that Group A has the highest mean rank, so they are more successful than Group B. This result is statistically significant (U=110; p=.01).

Inference Techniques for three or more variables ANOVA is used for 3 or more variables in parametric data. It is used when we want to see the differences between the means of more than two groups: a more general form of the t-test. (e.g. Effect of reading for fun on the attitudes of Group A, B, & C) Variation both within and between each of the groups is analyzed statistically and is represented as the F value.

Sample SPSS Outcome

Sample ANOVA Table (APA) E.g. Following is a table that shows means on test anxiety scales of 6th, 7th, and 8th graders. We need to demonstrate the results of ANOVA using two tables; so the following table is not enough N Mean sd Scheffe 6th graders 52 70.28 8.42 7th graders 60 80.38 9.77 8th graders 50 82.80 9.33

N Mean sd Scheffe 6th graders 52 70.28 8.42 7th graders 60 80.38 9.77 Table 1. Descriptive Results N Mean sd Scheffe 6th graders 52 70.28 8.42 7th graders 60 80.38 9.77 8th graders 50 82.80 9.33 Table 2. ANOVA Results Sum of squares df Mean squares F P Between groups 4583.14 2 2291.57 26.94 .000 Within groups 13520.00 159 85.037 Total 18104.00 161 Analysis of data shows that there is a statistically significant difference in the test anxiety levels of the groups (F=26.94). In other words, students’ test anxiety level shows difference at different grade levels. (The higher the grade level, the higher the anxiety level) According to the Scheffe results, 7th (X=80.38) and 8th (X=82.80) graders’ anxiety level is higher than that of the 6th (X=20.28) graders.

Kruskal-Wallis (non-parametric data for three or more independent groups) Similar to ANOVA But it is used for non-parametric data

Sample Kruskal Wallis As Table 1 indicates, participants’ grades show difference according to the method they have received (x2 = 9.26; p<.05). This shows that the difference is statistically significant. When the mean rank is taken into consideration, the highest effect was with Method C, followed by Method B and A.