Analyzing Significant Differences between Means Dr. K. A. Korb University of Jos

Outline Types of Statistics Purpose of Inferential Statistics Null hypotheses Interpreting Inferential Statistics Reporting Inferential Statistics Dr. K. A. Korb University of Jos

Types of Statistics Chi-Square: The purpose of a chi-square is to test whether frequency counts are distributed differently for different samples ONLY use the Chi-Square if you have categorical data (i.e. teaching at a private or governmental school) Do NOT use if you have continuous scores (i.e. Likert Scale, grades, attendance) Dr. K. A. Korb University of Jos

Types of Statistics ANOVA, ANCOVA, t-test: Determine whether average scores differ significantly Use t-test if you only have 2 groups Use an ANOVA if you have more than 2 groups. If you get a significant result, follow up with Tukey’s HSD to determine whether groups are significantly different from each other. Use an ANCOVA if you have pre- and post-test data These three statistics are also called Inferential Statistics. Dr. K. A. Korb University of Jos

Types of Statistics Correlation, Regression: Determines the strength and direction of a relationship between two variables Use if you want to compare two continuous variables (i.e., grades and classroom attendance) Dr. K. A. Korb University of Jos

No Yes Yes No Yes No Yes No No Yes Dr. K. A. Korb University of Jos Do you have categorical data only? No Yes Do you have independent and dependent variables? Chi-Square (χ2) Do you have more than 2 independent variables? Yes No Do you have more than 2 variables? Yes Do you have pre- and post-tests? No Yes No No Yes ANOVA: Analysis of Variance ANCOVA: Analysis of Covariance t-test Regression Correlation

Purpose of Inferential Statistics In educational research, we can never sample the entire population that we want to generalize our results to. Instead, we choose a sample of the population Then we want to make inferences about the population based on the results of our study based on the sample. The purpose of inferential statistics is to determine whether the findings from our sample can generalize to the entire population or that our findings were simply the result of chance. Dr. K. A. Korb University of Jos

Purpose of Inferential Statistics Imagine a room full of socks. You want to determine whether there are more white socks than green socks in the room. However, there are too many socks to count, so you want to take a sample of socks and draw a conclusion about whether there are more white socks based on your sample. The purpose of inferential statistics is to determine whether the colors chosen in your sample likely reflects the entire room or if your results were due to chance. Dr. K. A. Korb University of Jos

Purpose of Inferential Statistics What factors will determine whether the sample of socks adequately represents the entire room? First, the size of our sample. If we only pick two socks, they would very likely not represent the entire room. The larger our sample is, the more representative our sample will be of the entire room and the more accurately our conclusions will be for the entire room. This is why when conducting experiments, the larger the sample is, the better. With large samples, our results will more likely reflect the entire population. Dr. K. A. Korb University of Jos

Purpose of Inferential Statistics What factors will determine whether the sample of socks adequately represents the entire room? Second, the size of the difference between white and green socks in the entire room will affect our results. If there are only two more white socks in the entire room, then we likely will not be able to determine this difference in our sample. If there are thousands more white socks in the entire room, we should find this in the sample. Therefore, when conducting studies, try to make your treatment very effective. Very effective treatments result in a large change in your dependent variable and enable you to find a significant difference in your sample. Dr. K. A. Korb University of Jos

Purpose of Inferential Statistics To summarize, you will most likely find significant results in your study if you: Have a large sample size Have an effective treatment that results in a large change in your dependent variable Dr. K. A. Korb University of Jos

Purpose of Inferential Statistics For an education example, you want to determine if there is a difference between boys and girls in Nigeria on their science ability. You cannot sample all of the boys and girls in Nigeria. Instead, draw a sample of boys and girls and test their science ability. Then you will use inferential statistics to determine whether any mean difference between boys’ and girls’ science ability scores can generalize to all Nigerian students or if your results are due to chance. Dr. K. A. Korb University of Jos

Purpose of Inferential Statistics In this study, you will most likely find significant results if: You have a large sample of boys and girls A large difference in science ability truly exists between boys and girls Dr. K. A. Korb University of Jos

Null hypotheses Null hypotheses: Predicting that no average difference between the groups will be found You actually want to prove that differences do exist between groups. Because you do not want to say that differences exist when they really do not, you begin with the assumption that your results ARE due to chance. Then if your results are truly significant – there is little likelihood that your results are due to chance – you can confidently say that differences do exist between the groups. Dr. K. A. Korb University of Jos

Null hypotheses When writing your null hypotheses, be sure to include: Significant differences in what? This will be your dependent variable Significant differences between what two groups? This will be your independent variable There are NO significant differences in science ability (in what) between boys and girls (what two groups). Dr. K. A. Korb University of Jos

Interpreting Inferential Statistics t-tests: Use if you are comparing two groups One treatment and one control group Boys and girls Pre- and Post-test for one group of people ANOVA: Use if you are comparing more than two groups Two treatments and one control group Children in Primary 2, 3, and 4 High, medium, and low socioeconomic students ANCOVA: Use if you are comparing pre- and post-tests for more than one group Pre- and Post-tests for treatment and control groups Pre- and Post-tests for boys and girls Dr. K. A. Korb University of Jos

Interpreting Inferential Statistics When you calculate inferential statistics, you will get three important numbers Either t (for a t-test) or F (for ANOVA and ANCOVA): A number that tells you how large the difference is between your two groups Degrees of freedom (df): One number (for a t-test) or two numbers (for ANOVA or ANCOVA) that depends on your sample size. Using tables, the t or F is then compared to the df to determine a probability Probability (p): This tells you whether your results are due to chance or not. Dr. K. A. Korb University of Jos

Interpreting Inferential Statistics In general, statisticians recommend that if the p value is less than .05, your results are significant. The significance level that you set prior to conducting your statistics, p < .05, is the alpha (α) If your calculated p value is less than your alpha, then your null hypothesis is rejected. Significant p-values indicate that your results are NOT due to chance and thus represent a meaningful difference in the population. You can therefore conclude that a difference does exist between your two groups in the population. Dr. K. A. Korb University of Jos

Reporting Inferential Statistics For a t-test: Girls performed significantly better than boys in their science ability tests (t(43) = 4.61, p <.001). 43 = degrees of freedom 4.61 = t value .001 = p value Dr. K. A. Korb University of Jos

Reporting Inferential Statistics Dr. K. A. Korb University of Jos Reporting Inferential Statistics This chart of average science ability scores clearly shows that girls have more science ability than boys. The chart can easily be constructed in Excel.

Reporting Inferential Statistics For either ANOVA or ANCOVA: Significant differences were found in science achievement between high, medium, and low socioeconomic status students (F(2,136) = 31.86, p < .001). 2, 136 = Degrees of Freedom. ANOVAs and ANCOVAs will always have 2 numbers in the degrees of freedom. In this example, 2 is a function of the number of groups you are comparing and 136 is a function of the sample size. 31.86 = F value .001 = p value Dr. K. A. Korb University of Jos

Reporting Inferential Statistics This chart shows that science ability increases as SES increases. Dr. K. A. Korb University of Jos

Reporting Inferential Statistics This chart that science ability increases as SES increases and also that girls tend to do better than girls. Dr. K. A. Korb University of Jos