1. Adnan Ajsic Northern Arizona University
Effects of Corpus-based Instruction on L2 Recognition and Recall of Signal Markers
Adnan Ajsic
Northern Arizona University

2. Overview Introduction Research Questions Methods Results Discussion
Participants
Materials
Instruction
Assessment
Results
Discussion
Tentative conclusions
Limitations and future directions
References

3. Introduction
Signal markers “are expressions such as first, finally, and however, or phrases such as in conclusion, on the other hand, and as a result. Other kinds of words such as subordinators (when, although), coordinators (and, but), adjectives (another, additional), and prepositions (because of, in spite of) can also serve as [signal markers]. [They] are like traffic signs; […] they tell your reader when you are giving a similar idea (similarly, and, in addition), an opposite idea (on the other hand, but, in contrast), an example (for example), a result (therefore), [etc.]” (Oshima & Hogue, 2006, p. 25).

4. Introduction, cont’d Action research project (Grabe & Stoller, 2011)
Discourse structure: lexical signaling (Grabe, 2009)
Linking adverbials (many SMs) are a distinctive characteristic of academic prose (Biber et al., 1999)
SM-related terminology: relational markers (Degand & Sanders, 2002); transition signals (Oshima & Hogue, 2006); signal words and phrases (Grabe & Stoller, 2011)

5. Introduction, cont’d
SMs can be n-grams (cf. text-organizers, Biber et al., 1999; links, Carter & McCarthy, 2006; text-oriented n-grams, Hyland, 2008)
Functional orientation toward vocabulary instruction (i.e., both single- and multiword units; cf. Paquot, 2007, 2008)
Relational markers (Degand & Sanders, 2002); formulaic sequences (Jones & Haywood, 2004); EAP and exemplification markers (Gilquin et al., 2007, Paquot, 2008)

6. Research Questions
RQ1: How do ESL students score on (a) a general vocabulary test, and (b) tests of ability to recognize and recall SMs before the treatment?
RQ2a,c: Are there differences in the effects of the different treatment conditions (corpus-based online, corpus-based offline, traditional) on the experimental groups’ ability to recognize (c: taught) SMs after the treatment?
RQ2b,d: Are there differences in the effects of the different treatment conditions (corpus-based online, corpus-based offline, traditional) on the experimental groups’ ability to recall (d: taught) SMs after the treatment?

7. Methods: Participants
Participants: 48 IEP Level 5 ESL students (in 4 Reading Lab sections)
Proficiency levels: advanced intermediate (but IEP placement test scores: 10-21/30 on reading and /30 on writing )
L1s: Arabic (28 or 58.3%), Chinese (20 or 41.7%)

8. Methods: Materials
Specialized pedagogic corpus (32,419 word tokens/4,076 word types) compiled from composition course readings (chapter 6 from Roen, Glau & Maid, 2010, and newspaper and journal articles about social media and young people)
Class handouts: concordance-based exercises (corpus-based online and corpus-based offline) and dictionary-based exercises (traditional)
AntConc (Lawrence, 2011; corpus-based online)
(corpus-based online)

9. Methods: Instruction (SMs taught)
also although during too while and so on as well/as for example for instance such as
Week 1 (also, too)
Week 2 (and so on, as well/as)
Week 3 (for example, for instance, such as), and Week 4 (although, during, while) (cf. Römer, 2011)
Römer, 2011 suggests teaching targets should be grouped according to

10. Methods: Assessment
Pre-/post-test T1: recognition (Ss asked to underline SMs in a 369/380-word excerpt of text about youth and social media)
Readability: Lexile 1450L/1390L, FKGL 14.7/14.7
Pre-/post-test T2: recall (Ss asked to fill in blanks using provided SMs in a 391/366-word excerpt of text about youth and social media)
Readability: Lexile 1340L/1400L, FKGL 13.5/14.0

11. Results (all SMs)
General vocabulary test: Ss asked to underline unfamiliar words in three 100-word excerpts from class readings (M=10.24, 3.41%) (cf. Schmitt, Jiang, & Grabe, 2011)
Readability: Lexile 1440L, FKGL 15.1
F (3, 53) = 1.892, p = .142
Treatment N M SD
CorpusOnline 14
9.71
1.98
CorpusOffline
11.36
3.63
Traditional 15
8.40
3.78
Control
9.57
3.67
Total 57
9.74
3.44
Check for differences in vocab knowledge by L1!

12. Results (all SMs), cont’d
T1: Recognition (pre- and post-test, all SMs)
Pre-test
Post-test
Treatment L1 n M SD
CorpusOnline
Arabic 5
3.20
3.27
9.60
1.95
Chinese
6.00
5.79
6.20
4.32
Total 10
4.60
4.67
7.90
3.64
CorpusOffline 9
5.78
5.33
6.78
3.77
4.40
5.60
4.50 14
5.29
5.25
6.57
3.88
Traditional 7
6.71
4.15
7.29
1.98 6
2.17
1.72
7.67
3.01 13
4.62
3.93
7.46
2.40
Control
4.46
10.29
3.82 4
7.75
5.91
2.95 11
6.18
4.90
8.73
4.00 28
5.43
4.43
8.29
3.35 20
4.80
4.94
6.60
3.53 48
5.17
4.61
7.58
3.45
Max. score (pre- and post-test): 42.

13. Results (all SMs), cont’d
T1: Recognition (between-subjects, all SMs)
Treatment: F (3, 40) = .425, p > .05
L1: F (1, 40) = 1.049, p > .05
Treatment  L1: F (3, 40) = .132, p > .05
T1: Recognition (within-subjects, all SMs)
Time: F (1, 40) = , p = .001, η2 = .26
Time  Treatment: F (3, 40) = .602, p > .05
Time  L1: F (1, 40) = 2.082, p > .05
Time  Treatment  L1: F (3, 40) = 5.175, p = .004, η2 = .28

14. Results (all SMs), cont’d
T1: Recognition (between-subjects, all SMs) Treatment × Time L1 × Time

15. Results (all SMs), cont’d
T1: Recognition (within-subjects, all SMs)
Treatment × Time (L1 Arabic) Treatment × Time (L1 Chinese)

16. Results (all SMs), cont’d
T2: Recall (pre- and post-test, all SMs)
Pre-test
Post-test
Treatment L1 n M SD
CorpusOnline
Arabic 5
20.40
4.56
3.60
1.14
Chinese
22.20
2.78
7.80
4.15
Total 10
21.30
3.68
5.70
3.62
CorpusOffline 9
21.11
1.62
7.89
4.14
21.60
1.67
9.20
3.83 14
21.29
1.59
8.36
3.93
Traditional 7
23.57
2.37
6.57
2.57 6
22.33
2.07
8.50
4.23 13
23.00
2.24
7.46
3.43
Control
20.00
3.27
4.00
2.31 4
22.75
1.50
7.75
2.06 11
21.00
3.00
5.36
2.84 28
21.32
3.07
5.82
3.36 20
1.96
8.35
3.53 48
21.69
2.68
6.88
Why are pre- and post-test not comparable?
Max. score (pre- and post-test): 28.

17. Results (all SMs), cont’d
T2: Recall (between-subjects, all SMs)
Pre-test
Treatment: F (3, 40) = 1.179, p > .05
L1: F (1, 40) = 1.504, p > .05
Treatment  L1: F (3, 40) = 1.271, p > .05
Post-test
Treatment: F (3, 40) = 1.925, p > .05
L1: F (1, 40) = 7.987, p = .007, η2 = .17
Treatment  L1: F (3, 40) = .497, p > .05
Is there a between subjects post-test for all SMs here (i.e., one not focusing on SMs taught)?

18. Results (all SMs), cont’d
T2: Recall (between-subjects, all SMs)
Treatment × L1 (pre-test) Treatment × L1 (post-test)

19. Results (SMs taught) T1: Recognition – SMs taught (post-test only)
Treatment L1 n M SD
CorpusOnline
Arabic 5
6.20
1.10
Chinese
3.60
2.30
Total 10
4.90
2.18
CorpusOffline 9
5.56
2.88
4.60 14
5.21
2.81
Traditional 7
6.29
2.29 6
4.67
2.07 13
5.54
2.26
Control
6.00
1.83 4
4.00
1.41 11
5.27
1.90 28
5.96
2.15 20
4.25
2.12 48
5.25
2.28

20. Results (SMs taught), cont’d
T1: Recognition – SMs taught (post-test only)
Treatment: F (3, 40) = .149, p > .05
L1: F (1, 40) = 7.134, p = .011, η2 = .15
Treatment  L1: F (3, 40) = .264, p > .05

21. Results (SMs taught), cont’d
T1: Recognition – SMs taught (post-test only) Treatment × L1

22. Results (SMs taught), cont’d
T2: Recall – SMs taught (post-test only)
Post-test
Treatment L1 n M SD
CorpusOnline
Arabic 5
1.20
1.30
Chinese
3.00
1.58
Total 10
2.10
1.66
CorpusOffline 9
2.89
2.32
2.80
1.48 14
2.86
1.99
Traditional 7
2.14
1.77 6
2.53 13
2.54
2.11
Control
1.14
1.35 4
3.75
2.06 11
2.09
2.02 28
1.96
1.88 20
3.10
1.86 48
2.44
1.93

23. Results (SMs taught), cont’d
T2: Recall – SMs taught (post-test only)
Treatment: F (3, 40) = .299, p > .05
L1: F (1, 40) = 5.302, p = .027, η2 = .12
Treatment  L1: F (3, 40) = 1.087, p > .05

24. Results (SMs taught), cont’d
T2: Recall – SMs taught (post-test only) Treatment × L1

25. Discussion (all SMs)
RQ1: How do ESL students score on (a) a general vocabulary test, and (b) tests of ability to recognize and recall SMs before the treatment?
(a) students knew 96.59% of vocabulary, no significant differences between groups (comparable)
(b) generally low, no significant differences between groups (comparable)

26. Discussion (all SMs), cont’d
RQ2a: Are there differences in the effects of the different treatment conditions (corpus-based online, corpus-based offline, traditional) on the experimental groups’ ability to recognize SMs after the treatment?
Task 1: Recognition (pre- and post-test)
Significant effect of instruction overall (T1<T2)
Significant differences between L1s and treatments (aptitude-treatment interaction)
L1 Arabic (CorpusOnline), L1 Chinese (Traditional)
L1 Arabic students did better overall (aptitude-task interaction?)

27. Discussion (all SMs), cont’d
RQ2b: Are there differences in the effects of the different treatment conditions (corpus-based online, corpus-based offline, traditional) on the experimental groups’ ability to recall SMs after the treatment?
Task 2: Recall (pre- and post-test)
Significant differences between L1s
L1 Chinese students did better overall (aptitude-task interaction?)

28. Discussion (SMs taught)
RQ2c: Are there differences in the effects of the different treatment conditions (corpus-based online, corpus-based offline, traditional) on the experimental groups’ ability to recognize TAUGHT SMs after the treatment?
Task 1: Recognition (post-test only)
No significant differences between treatments
Significant differences between L1s
L1 Arabic students did better overall (aptitude-task interaction?)

29. Discussion (SMs taught), cont’d
RQ2d: Are there differences in the effects of the different treatment conditions (corpus-based online, corpus-based offline, traditional) on the experimental groups’ ability to recall TAUGHT SMs after the treatment?
Task 2: Recall (post-test only)
No significant differences between treatments
Significant differences between L1s
L1 Chinese students did better overall (aptitude-task interaction?)

30. Conclusion
Although all types of instruction appear to be effective for the teaching of SMs overall (cf. Grabe, 2009, on instruction effects), (if possible) they should be matched with students’ aptitudes, i.e. corpus-based SMs instruction may not be universally effective
Students with different L1 backgrounds also seem to interact with tasks differently (L1 Arabic students better at recognition, L1 Chinese better at recall), so aptitude-task interaction should be taken into account also (inductive vs. deductive?)
There may exist intervening extraneous variables which are unaccounted for here (e.g., listening comprehension, academic prep/amount of study outside of class)

31. Limitations and future directions
Readability level of assessment texts
Recall task not directly comparable across time
Sample size
Future directions:
Pre-teaching of SMs
Shorter, higher-readability assessment tasks
Comparison between single- and multi-word items
Use of SMs in writing

32. References
Alali, F. A., & Schmitt, N. (2012). Teaching formulaic sequences: The same as or different from teaching single words? TESOL Journal, 3(2), Degand, L., & Sanders, T. (2002). The impact of relational markers on expository text comprehension in L1 and L2. Reading and Writing: An Interdisciplinary Journal, 15, 739–757. Gilquin, G., Granger, S., & Paquot, M. (2007). Learner corpora: The missing link in EAP pedagogy. Journal of English for Academic Purposes, 6, Grabe, W. (2009). Reading in a second language: Moving from theory to practice. Cambridge: CUP. Grabe, W., & Stoller, F. (2011). Teaching and researching reading. Harlow: Longman. Jones, M., & Haywood, S. (2004). Facilitating the acquisition of formulaic sequences: An exploratory study in an EAP context. In N. Schmitt (Ed.), Formulaic sequences: Acquisition, processing, and use (pp. 269–292). Amsterdam, the Netherlands: John Benjamins.

33. References cont’d
Oshima, A., & Hogue, A. (2006). Writing academic English. Pearson Longman. Paquot, M. (2007). Towards a productively-oriented academic word list. In J. Walinski, K. Kredens, & S. Gozdz-Roszkowski (Eds.), Corpora and ICT in language studies. PALC Lodz studies in LANGUAGE 13 (pp. 127–140). Frankfurt am main: Peter Lang. Paquot, M. (2008). Exemplification in learner writing: A cross-linguistic perspective. In F. Meunier & S. Granger (Eds.), Phraseology in Foreign Language Learning and Teaching. Amsterdam: John Benjamins Publishing. Roen, D., Glau, G. R. and Maid, B. M. (2010). The McGraw-Hill guide: Writing for college, writing for life. New York: McGraw-Hill. Römer, U. (2011). Corpus research applications in second language teaching. Annual Review of Applied Linguistics, 31, Schmitt, N., Jiang, X. and Grabe, W. (2011). The percentage of words known in a text and reading comprehension. Modern Language Journal, 95, 26–43.

34. Questions? Comments? adnan.ajsic@nau.edu
Thank you!
Questions? Comments?

35. Results (all SMs), by L1 and treatment
L1 Arabic Time: F (1, 24) = , p = .000, η2 = .44 Time  Treatment: F (3, 24) = 3.563, p = .029, η2 = .31 CorpusOnline: F (1, 4) = 9.846, p = .035, η2 = .71 Control: F (1, 6) = , p = .019, η2 = .63 L1 Chinese Time: F (1, 16) = 1.884, p > .05 Time  Treatment: F (3, 16) = 2.254, p > .05 Traditional: F (1, 5) = , p = .003, η2 = .84
Add the breakdown by treatment here!