1. Statistics for Education Research Lecture 5 Tests on Two Means: Two Independent Samples Independent-Sample t Tests Instructor: Dr. Tung-hsien He the@tea.ntue.edu.tw

2. Independent-Sample t Test: Independent-Sample t Test: 1. Independent-Samples t tests compare means of two groups (two independent samples). 2. The two means come from a single dependent variable. 3. Use G*Power to decide sample sizes. 4. Ideally, participants should be randomly selected and randomly assigned to the two groups, so that any difference in response is due to the treatment (or lack of treatment) and not to other factors.

3. 5. The two groups should have the same number of participants so that the robustness of homogeneity can be ensured. 6. Check the two assumptions behind this type of test, i.e., normality and homogeneity, before using it. 7. Check normality by performing K-S test (Kolmogorov-Smirnov test). 8. Check homogeneity by performing Levene’s test (nonsignificant is desired), if the two groups’ sizes are different. If they have equal ns, don’t need to run Levene’s test.

4. 9. If the two assumptions are violated, either use nonparametric tests (e.g., 2-sample K-S test or Mann-Whitney U, but the sample size must be small < or = 20) or use data transformations (e.g., square root). 10. Condition: Testing Differences in Two Means of Experimental Group and Control Group on a Single Dependent Variable.

5. Bonferroni Adjustment Technique: Bonferroni Adjustment Technique: a. Condition: Multiple dependent variables need to analyzed by performing univariate analysis techniques like paired-sample t test or independent-sample t tests. b. Reason for Using Bonferroni Adjustment Technique: If any univariate test is performed repeatedly to analyze different dependent variables, the chance to make Type I error will increase (i.e.,  will be inflated). As a result, Ho become easier to be rejected when it should not be at the original  level.

6. c. Example: Independent-Sample t tests are used twice to analyze reading achievement and scores on written papers, respectively. d. Procedures of Bonferroni Adjustment Technique: Dividing the original  by the number of times tests are performed. Then, after t tests are performed, compare p-values to the values of the divided  to see if p-values are higher or smaller than the  /n. In our case,  should be 0.025 (i.e., 0.05/2). e. Decision: If p-values are smaller than 0.025, then it is significant. If p-values are larger than 0.025, then it is nonsignificant.

7. Example: Example: a. Scenario: In a study that focuses on the effects of Phonics instruction on reading comprehension, the researcher randomly select 40 subjects from the population. Then the researcher randomly and evenly assigns the 40 subjects into the experimental and control groups. Then phonics instruction is given to the experiment group, whereas the control group does not receive any treatment. After 4 weeks, the researcher tests the two groups’ reading comprehension by using a test.

8. a. Scenario: This test has 6 items and each item occupies 5 points (maximum score is 30). The researcher wants to know whether the instruction will make significant effects on the experiment group’s reading comprehension. b. Conditions: Two independent samples with two means on a single dependent variable. c. Goal: To see whether the difference in the experiment group mean and control group mean is so huge (significant) that it is very Unlikely that sampling error will cause this difference.

9. d. X control = sampling error + reading proficiency (constant: 常數 ) X experiment =sampling error + reading proficiency (constant: 常數 ) + effects of instruction X experiment =sampling error + reading proficiency (constant: 常數 ) + effects of instruction e. Between-Group Homogeneity: Because the two groups are randomly selected, the constant will be assumed to be same. f. Hypothesis Testing: Ho: X control = X experiment Ha: X control ≠ X experiment

10. g. One-tailed,  =.05, n1=20, n2 = 20, n = 40 h. Proper Stat Technique: Independent-Sample t Test i. See “SPSS Procedures” for detailed info. j. Report effect sizes (post hoc) so that “practical significance” can be more clarified given that “statistical significance” is found.

11. 3P 3P PP: Some researchers support that bilingualism prompts bilingual children to reflect on and manipulate structural features of a language, including its phonemes, words, and syntactic structures (metalinguistic awareness), whereas others do not.

12. IP: However, (a) very few studies ever tackled whether foreign language learning on word-awareness skills among Mandarin-English marginal preschool bilinguals could contribute to a better metalinguistic ability compared to that of monolinguals, and (b) previous studies do not manipulate variables like different degrees of bilingualism, socio-economic status (gender & age, amounts of L2 exposure), and task’s difficulties. SP: Marginal bilinguals may perform better on certain tasks.

13. Criticisms: Criticisms: 1. Is there a clear link between PP and IP? 2. Is it reasonable to claim that young EFL learners can be deemed as marginal bilinguals? If it is reasonable, does the author provide the key studies and researchers that can justify this categorization? Literature Review Literature Review 1. Studies that highlighted bilinguals’ advantages in metalinguistic awareness. 2. Studies that failed to find these advantages

14. 3. Criticisms on previous studies’ being unable to find advantages (See p. 134) Criticisms: Criticisms: 1. Does the author clearly specify that investigating word awareness among these three kinds of metalinguistic awareness is significant? (See 1 st paragraph under Subheading of LITERATURE REVIEW on p. 132)

15. 2. Is there any evidence (like previous studies) that can justify the author’s criticisms? In other words, are these criticisms strongly supported by previous research? (p. 134) 3. Is there any research that highlights EFL can been seen as a type of bilinguals? Is it important to have evidence to justify this category? Method & Results Method & Results 1. Participants: a. two samples that constituted 60 preschoolers;

16. b. Screening activities 2. Tasks: a. Word Segmentation Task b. Word Size Judgment Task c. Name Manipulation Task 3. Procedures: 4. Analysis: a. Independent T-test b. Paired-Sample T-test

17. Criticisms: Criticisms: 1. Are participants randomly selected and assigned? 2. Is the format of T-test tables appropriate? (Table 1, p. 138: What seems missing? Descriptive statistics of both groups) 3. Is the result of paired-sample t-tests reported? 4. Are the results of t-tests properly reported? (e.g.: t(58)=-1.220, p>.05, p. 138)

18. Discussions & Conclusion Discussions & Conclusion 1. Discussions on non-significant results by referring to qualitative analyses 2. Discussions on significant results by referring to qualitative analyses 3. Limitations 4. Implications Criticisms: Criticisms: 1. Is it appropriate to discuss non-significant results?

19. 2. Is it appropriate to cite limitations in discussion sections? 3. Is it appropriate to cite qualitative data to augment quantitative results? 4. Is it appropriate to conclude that metalinguistic benefits exist for EFL preschoolers as the marginal bilingual group merely outperformed the monolingual group on one (out of three) task? (11 th line from the top, p. 143)

20. 5. Is it appropriate to conclude that “limited exposure to a second language can enhance one aspect of word awareness” when participants are not randomly selected and English input may come from other sources in addition to English instructions?