Investigating the ‘parallelness’ of speaking narrative tasks Chihiro INOUE PhD student at Lancaster University TBLT2009 1

Starting Point Language tests: ‘comparable’ tasks are vital = statistical equivalence of score distribution (i.e. scores  calculate M, SD) …What kind of language do the tasks actually elicit? Need for in-depth analyses of performance to include more evidence of ‘parallelness’ (e.g. various aspects of performance, functions, syntactic structures, idea units, perceived difficulty, etc.) 2

From my pilot studies… 3 Two seemingly ‘similar’ narrative tasks differed in performance measures, functions, syntactic structures, idea units, and perceived difficulty.  because of the differences in relationships among characters, prominence of characters, resulting damages [Since the picture was from a commercial test, it cannot be put on the web.] [Since the picture was from a commercial test, it cannot be put on the web.]

So, for the main study… How ‘similar’ can two picture sequence be?  It was decided to select picture sequences that are as similar as possible in relationships among the characters, prominence of characters, and similar storylines.  Two pictures were taken from Hill (1960). One sequence was modified so as to make the number of pictures the same as the other one. (Piloted to ensure it didn’t make understanding the sequence difficult.) 4

Task A (Balloon) 5

Task B (Baby) 6

Data Recordings of the two tasks performed by 45 students at a university in Tokyo, Japan Interviewed by myself in one-to-one sessions Instructed to talk a lot including as much detail as possible 2 minutes’ planning, no time limit for narration No note-taking, no dictionary use 7

Participants 45 students at a university in Tokyo, Japan At 3 proficiency levels (CEFR B1, B2, C1) judged by the Oxford Quick Placement Test (i.e. multiple-choice format, testing vocabulary, grammar, and reading ) 8

Research Question Are the two tasks ‘parallel’ in terms of the measures of fluency, accuracy, syntactic complexity, lexical complexity, and perceived difficulty?  paired sample t-test for each measure 9

Performance Measures Used No. of words Speech rate % of Junior High Textbook Vocab % of Senior High Textbook Vocab D-value % of error free clauses Errors per 100 words Errors per AS unit Average length of AS-units Average no. of subordinate clauses per AS-unit Robinson (2001)’s task difficulty questionnaire 10

Results: Overall (1) 11

Results: Overall (2) 12

Results: QPT Levels (1) 13

Results: QPT Levels (2) 14

Summary Overall: Task A (Balloon) seems to be ‘more difficult’. - less fluent, more erroneous, less syntactically complex, perceived ‘more difficult’  However, the effect sizes are small = these two are parallel? QPT levels: statistical significance (or large effect size) is seldom observed in the measures, and the patterns are not consistent.  ‘Parallelness’ depends on proficiency 15

Ways to go - Analyse the remaining other half of participants (45 more to go for the main study) - A lot of individual differences found (especially among B1s and C1s) = Levels (ratings) for the narrative performance; not by QPT levels = Compare performance on the two tasks by individuals more closely Any suggestions or comments are very welcome! 16

References Hill, L. A. (1960). Picture Composition Book. London: Longman. Robinson, P. (2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 23,