Acknowledgement This research was conducted with approval of the British Council’s Assessment Research Group (ARG) Any opinions, findings or conclusions expressed in this presentation are those of the presenter and do not necessarily reflect the views of the ARG or the British Council wider network

Output register parallelism in direct and semi-direct speaking tests Ethan Quaid The Sino-British College / USST The Second Annual Conference and Learning and Teaching Forum, Shanghai, 2017.

Concurrent validity – correlations r = >.80 Reliability – construct irrelevant facets Practicality – cost, administration

Interchangeability and disparity …there are few situations in the world in which what the learner says has absolutely no effect on what he hears next. (Underhill, 1987, p.35) …face-to-face talk is to be regarded as the unmarked form of interaction, and communicating by telephone or speaking into a microphone as marked forms. (Van Lier, 1989, p.493)

Summary of findings to date

Summary of findings to date Research question: Can register parallelism be shown in test-taker output in direct and semi-direct speaking tests?

Aptis General Speaking sub-test Computer-based: pre-recorded and scripted task features Online practice test version

Counterbalanced study design

Analyses and results Discourse rhetorical structure direct interrogatives and declarative instructional prompts Discourse markers and particles [Elena: OPI, Part 3, Item 1, Response] 82 you can see a lot of people, .hhh (.) you know? [Elena: CB-OPI, Part 2, Item 2, Response] 52 (*er*) it is free admission so you can go >whenever< you want [Chloe: OPI, Part 3, Item 3, Response] 92 so (.2) um: basketball is more difficult. it’s (.8) ok?% [Emily: CB-OPI, Part 1, Item 1, Response] 10 ok. (.) (*there a:re four*) members in my family. (.) .hhh! my

Analyses and results Lexical density values

Analyses and results Syntactic Complexity AS-Units (measurement) A single speaker’s utterance consisting of an independent clause, or sub-clausal unit, together with any subordinate clause(s) associated with either. (Foster, Tonkyn & Wigglesworth, 2000, p.365)

Analyses and results Syntactic complexity continued…

Analyses and results Syntactic complexity continued…

Self-report statements Computer-based test ‘something is pushing me to speak without any stops’ (Elena) ‘There is more pressure to speak accurately’ (Chloe) Face-to-face test ‘you feel like you’re talking to someone’ (Emily) ‘feels like interaction between two people’ (Elena)

Conclusion Slight register shift Contextualisation Lexical density non-consequential or even a positive for the Password computer-based speaking test Contextualisation Lexical density Syntactic complexity

Reference Choi, I. (2014). The comparability of direct and semi-direct oral proficiency interviews in a foreign language context: A case study with advanced Korean learners of English. Language Research. 50.2, 545-568. Foster, P., Tonkyn, A. & Wigglesworth, G. (2000). Measuring spoken language: A unit for all reasons. Applied Linguistics. 21.3, 354- 375. Kiddle, T. & Kormos, J. (2011). The effect of mode of response on a semi-direct test of oral proficiency. Language Assessment. Quarterly. 8.4, 342-360. O’Loughlin, K. (2001). The equivalence of direct and semi-direct speaking tests. Studies in Language Testing 13. United Kingdom: Cambridge University Press. Shohamy, E. (1994). The validity of direct versus semi-direct oral tests. Language Testing. 11.2, 99-123. doi: 10.1177/026553229401100202 Underhill, N. (1987). Testing spoken language: A handbook of oral testing techniques. United Kingdom: Cambridge University Press. Van Lier, L. (1989). Reeling, writhing, drawling, stretching and fainting in coils: Oral proficiency interviews as conversations. TESOL Quarterly. 23, 489-508. doi:10.2307/3586922 Zhou, Y. J. (2008). A comparison of speech samples of monologic tasks in speaking tests between computer- delivered and face-to-face modes. Japan Language Testing Association Journal. 11, 189–208.

Lecturer in English for Academic Purposes The Sino-British College Thank you! Ethan Quaid Lecturer in English for Academic Purposes The Sino-British College ethan.quaid@sbc-usst.edu.cn