CRELLA University of Bedfordshire May 2012 Parvaneh Tavakoli Effects of Task Design on Native and Non-native Task Performance

 Tasks as units of Teaching and Testing  Task Design in Language Teaching Sequencing and syllabus design Cognitive processes and SLA L2 developmental processes  Task Design in Language Testing Are language testing tasks of equal difficulty (Pollitt, 1991) Defining task difficulty remains a challenge (Iwashita, et al, 2001) A hierarchy of task difficulty yet to be established (Bachman, 2002)  What can task difficulty help with? Selecting & categorizing tasks for test purposes Providing a more reliable assessment Improving the validity of the interpretations and uses made on test results

 Study 1: Effects of degree of task structure on L2 spoken performance  Study 2: Effects of storyline complexity and structure on L2 & L1 spoken performance  Study 3: Effects of storyline complexity on L2 written performance (work in progress)

 Task structure & Planning Conditions  6 oral narrative tasks (picture stories) were used with 80 Iranian EFL learners under testing conditions  Degree of task structure was operationalized through tightness of the structure presented in the picture stories Structure was defined based on knowledge structures (Mohan, 1991; Hoey, 1983) Tight structure (fixed sequence of events) Loose structure (arbitrary sequence of events)  Findings: Task structure proved to have an impact on L2 performance: the more structured tasks elicited more accurate and more fluent language There was no systematic impact on syntactic complexity or lexical diversity Planning improved performance in all different measures across the two LP levels

 Storyline Complexity foreground only (simple) foreground and background (complex)  Structure loose structure tight structure  Context of language learning learning English in Tehran learning English in London  Native speakers & L2 learners

 Does task design have an impact on L2 learners’ performance? Accuracy, fluency, syntactic complexity, lexical diversity  Does task design have an impact on native speakers’ performance? Fluency, syntactic complexity & lexical diversity

Phase 1 Learners in Tehran (N=60) Phase 2 Learners in London (N=40) Phase 3 Native speakers (N=40)

 Independednt variables structure: within-participant storyline complexity: between-participant  Narratives Oral narratives typically used in EFL settings They are popular in language testing, e.g. previously used by ETS; currently used by different local, national and international institutions  Setting in one-to-one sessions with the researcher

 Tehran 60 female Iranian L2 learners of English in a language school in Tehran aged between 18-34; Intermediate level (B1-B2)  London 40 male/female L2 learners of English in a college in London aged between 19 and 47, different L1s, Intermediate level (B1-B2)  London 40 male/female native speakers of English first year undergraduate students of English program/psychology in a university in London aged between 19 and 60

 Data transcribed and coded  Task accomplishment  Fluency repair, speed and breakdown fluency Mid-clause vs end-clause pauses  Syntactic complexity subordination and MLU  Accuracy percentage of error-free clauses  Lexical Diversity corrected measure of D calculated by “vocd” qualitative analysis of lexical selection  A range of statistical analyses Factor analysis, MANOVAs, ANOVAs, T-tests

 A complex storyline encourages L2 performance of more syntactic complexity L2 learners both in Tehran and in London It affects native speakers’ performance  Presence of structure encourages L2 performance of higher accuracy and fluency L2 learners both in Tehran and in London It doesn’t affect native speakers’ performance

 Means of subordination in native speakers’ performance  Picnic vs. Football 1.71 vs F=2.56, p<.01*, η 2=.150  Walkman vs. Journey 1.86 vs F=2.62, p<.01*, η 2=.150

 Means of length of utterance in native speakers’ performance  Picnic vs. Football vs F=2.64, p<.01*, η 2=.160  Walkman vs. Journey10.73 vs F=1.87, p<.07ns, η 2=.08

Narratives Phase 1 Phase 2 Phase 3 FPDifferenceFootball1.28(.16)1.41(.13)1.5(.30) * 3 vs. 1 Journey1.24(.17)1.34(.15)1.54(.37) * 3 vs. 1 & 2 Picnic1.59(.38)1.71(.28)1.71(.22) Walkman1.36(.27)1.51(.21)1.86(.39) *

Narratives Phase 1 Phase 2 Phase 3 FPDifferenceFootball7.62(4.41)9.67(5.24)1.89(1.79) * 3 vs. 1 & 2 Journey9.48(5.03)10.55(5.58)2.92(2.73) * Picnic7.83(6.50)8.13(3.81)2.06(2.78) * Walkman8.06(4.68)9.89(4.97)2.58(2.20) *

Patterns of mid-clause pausing in the three groups

Patterns of end-clause pausing in the three groups

Narratives Phase 1 Phase 2 FPFootball42%(.16)43%(.12) Journey31%(.17)38%(.12) Picnic41%(.21)47%(.09) Walkman30%(.16)36%(.11)

Narratives Phase 1 Phase 2 Phase 3 FPDifferFootball28.75(11.26)38.37(11.18)40.12(7.89) * 1 vs. 2 & 3 Journey25.82(9.49)36.11(11)38.75(9.77) * Picnic27.76(5.89)36.59(9.46)39.90(8.89) * Walkman33.62(6.40)43.37(12.43)45.67(10.15) * “D” across the Three Groups

 Task design as a source of variance in L1 & l2 performance (construct-irrelevant variance)  task variability may well introduce error into the assessment of the oral ability  Studying task design and its effects on performance can provide insight into some systematic and predictable variation  These findings are in contrast with Elder et al., 2002 & Iwashita et al., 2001  Reliability: a statement of the accuracy, consistency and fairness of a measuring instrument (Banerjee, 2000)

 Researching task design: speaking From Ellis 1987 up to now  Researching task design: writing Kuiken & Vedder (2007) Ong & Zhang (2010) Kormos (2011, 2011)  How do they relate to one another? Does task difficulty affect speaking and writing production processes in the same way? Can a single model/index of task difficulty account for both modes of production?

 Written narratives: the same picture stories  Participants: 40 intermediate learners in London  20 minutes to complete their writing (no strategic planning, no drafts, no dictionaries)  Research questions: What are the effects of storyline complexity on syntactic complexity of L2 learners’ narrative writing? Are the effects of storyline complexity on L2 written performance comparable to those of oral performance?

 Syntactic complexity in written and oral L2 performance

 Bachman, L. (2002). Some reflections on task-based language performance assessment. Language Testing, 19(4),  Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.  Banerjee, J. (2000). Reliability. Routeledge Encyclopedia of language teaching and learning. London: Routeledge,  Elder, C., Iwashita, N., & McNamara, T. (2002). Estimating the difficulty of oral proficiency tasks: What does the test-taker have to offer. Language Testing, 19(4),  Fulcher, G. (2003). Testing second language speaking. London: Longman.  Fulcher, G., & Marquez Reiter, R. (2003). Task difficulty in speaking tests. Language Testing, 20(3),  Hoey, M. (1983). On the surface of discourse. London: George Allan and Unwin.  Iwashita, N., McNamara, T., & Elder, C. (2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51(3),  Mohan, B. A. (1991). LEP students and the integration of language and content: Knowledge structures and tasks. In C. Simich-Dudgeon (Ed.), Proceedings of the first symposium on limited English proficient students' issues. Washington, DC: Office of Bilingual Education and Minority Language Affairs.  Pollitt, A. (1991). Giving students a sporting chance: Assessment by counting and judging. In J. C. Alderson & B. North (Eds.). Language testing in the 1990s (pp ). London: Modern English Publications in Association with the British Council.