DEVELOPMENT OF AN ONLINE READING TEST DEVELOPMENT OF AN ONLINE READING TEST Associate Professor Dr. Alisa Vanijdee Dean, School of Liberal Arts Sukhothai Thammathirat Open University, Thailand

OBJECTIVES (1) develop an online English reading test; (2) validate the quality of the test; (3) study the reading performance-level of the test-takers; (4) analyse common weaknesses found in the test results, and (5) propose guidelines to improve the reading performance of test-takers.

INFORMANTS 31 academics and graduate students

RESEARCH INSTRUMENTS (1) 3 sets of English reading tests; (2) test evaluation forms, and (3) a test-specifications crosscheck form.

RESULTS (1) 3 sets of online reading tests posted on the website of Sukhothai Thammathirat Open University;

RESULTS (2) the tests are qualified as specified: the tests were designed based on the reading process and the reading evaluation process, with the reading performance level based on CEFR level;

the tests were modified according to the IOC scores of lower than.50 and the experts and pilot-test takers suggestions together with the trial test scores of the pilot-test takers

the reliability of the 3 sets was analysed by Kuder Richardson 20 formula, which yielded.86,.88, and.61 respectively; the difficulty and discrimination was accepted at.20 < = p <=.80 — the rest were modified;

RESULTS (3) the reading performance level of the 3 sets of tests: markslevelNumber of test takers Version 1Version 2Version C 1 (effective operational proficiency) B 2 (independent user: vantage) B 1(independent user: threshold) Needs improvement 000 Total number 31

RESULTS (4) the test-takers weaknesses included insufficient understanding of complex structure and lack of vocabulary; (5) the guidelines for reading improvement focus on reading strategies practice at the cognitive, metacognitive, and socio-affective level.

DEVELOPMENT PROCESS Theoretical Framework - Reading process framework (Khalifa & Weir, 2009: 62) - designed based on the mikroskills from Alderson (2000) and - test usefulness by Bachman & Palmer (1996)

Test specifications for Reading CEFR Unaldi’s (2009) comprehensive framework Text difficulty -vocabulary profile of ‘lexitutor’ -Flesch-Kincaid Grade Level

Test specifications for grammar selected from Grammar City & Guilds ESOL

Test specifications for Vocabulary AWL list

TEST SPECIFICATIONS designed based on the mikroskills from Alderson (2000) and test usefulness by Bachman & Palmer (1996). Unaldi (2009) detailed specification

Unaldi’s detailed specifications CONTEXTUAL Response Method: Gap filling Text length: up to 500 words

Genre: everyday materials such as letters and s(e.g., enquiries, orders, letters of confirmation etc.), public information leaflets brochures, short official documents straightforward instructions for equipment expository and informative newspaper/magazine articles on familiar subjects personal letters with description of events, feelings and wishes simple informational sources (e.g., junior encyclopaedias, leaflets and brochures)

Rhetorical task: narrative, descriptive, instructive, expository Pattern of exposition: define, describe, elaborate, illustrate, compare and contrast, classify Explicitness of text structure: explicit

Structural resources Words/sentence: Ave Flesch-Kincaid Grade Level: Ave: 8 The complexity of sentence structure: mostly simple sentences (but some use of subordinate clauses in PET) Cohesion: explicit

Lexical resources 95% K1-20 words = K % K2 8.7% K3 2.3% AWL words: 2.5%

Nature of information (abstract/concrete): concrete Content knowledge: not required: everyday situations encountered in work, school, leisure etc., (personal feelings, opinions and experiences, hobbies and leisure… familiar topics in expository texts

COGNITIVE Type of reading: - understand the main points and/or relevant points though not necessarily in detail (description of events, feelings and wishes, significant and clearly signalled reasoning, and argumentation) -identify unfamiliar words from the context -extrapolate the meaning of occasional unknown words from the context and deduce sentence meaning

Text level: word, sentence and across sentences

SAMPLES OF TEST SPECIFICATIONS ANALYSIS  See word file

Components of the Test  Grammar  Vocabulary AWL words  Reading  Test on CD

processed through lexitutor software ( The analysis revealed frequencies of vocabulary, types and levels (K1, K2, K3, AWL); types, tokens, families, tokens per family, types per family). Flesch-Kincaid Grade Level was also taken into consideration when adapting reading texts to correspond the specifications in B1 B2 C1. The level of difficulty of the Vocabulary

 Thank you.