1. DEVELOPMENT OF AN ONLINE READING TEST DEVELOPMENT OF AN ONLINE READING TEST Associate Professor Dr. Alisa Vanijdee Dean, School of Liberal Arts Sukhothai Thammathirat Open University, Thailand Alisa.Van@stou.ac.th;avanijdee@hotmail.com

2. OBJECTIVES (1) develop an online English reading test; (2) validate the quality of the test; (3) study the reading performance-level of the test-takers; (4) analyse common weaknesses found in the test results, and (5) propose guidelines to improve the reading performance of test-takers.

3. INFORMANTS 31 academics and graduate students

4. RESEARCH INSTRUMENTS (1) 3 sets of English reading tests; (2) test evaluation forms, and (3) a test-specifications crosscheck form.

5. RESULTS (1) 3 sets of online reading tests posted on the website of Sukhothai Thammathirat Open University;

6. RESULTS (2) the tests are qualified as specified: the tests were designed based on the reading process and the reading evaluation process, with the reading performance level based on CEFR level;

7. the tests were modified according to the IOC scores of lower than.50 and the experts and pilot-test takers suggestions together with the trial test scores of the pilot-test takers

8. the reliability of the 3 sets was analysed by Kuder Richardson 20 formula, which yielded.86,.88, and.61 respectively; the difficulty and discrimination was accepted at.20 < = p <=.80 — the rest were modified;

9. RESULTS (3) the reading performance level of the 3 sets of tests: markslevelNumber of test takers Version 1Version 2Version 3 76-100 C 1 (effective operational proficiency) 332 51-75 B 2 (independent user: vantage) 121411 26-50 B 1(independent user: threshold) 161418 0-25 Needs improvement 000 Total number 31

10. RESULTS (4) the test-takers weaknesses included insufficient understanding of complex structure and lack of vocabulary; (5) the guidelines for reading improvement focus on reading strategies practice at the cognitive, metacognitive, and socio-affective level.

11. DEVELOPMENT PROCESS Theoretical Framework - Reading process framework (Khalifa & Weir, 2009: 62) - designed based on the mikroskills from Alderson (2000) and - test usefulness by Bachman & Palmer (1996)

12. Test specifications for Reading CEFR Unaldi’s (2009) comprehensive framework Text difficulty -vocabulary profile of ‘lexitutor’ -Flesch-Kincaid Grade Level

13. Test specifications for grammar selected from Grammar City & Guilds ESOL

14. Test specifications for Vocabulary AWL list

15. TEST SPECIFICATIONS designed based on the mikroskills from Alderson (2000) and test usefulness by Bachman & Palmer (1996). Unaldi (2009) detailed specification

16. Unaldi’s detailed specifications CONTEXTUAL Response Method: Gap filling Text length: up to 500 words

17. Genre: everyday materials such as letters and emails(e.g., enquiries, orders, letters of confirmation etc.), public information leaflets brochures, short official documents straightforward instructions for equipment expository and informative newspaper/magazine articles on familiar subjects personal letters with description of events, feelings and wishes simple informational sources (e.g., junior encyclopaedias, leaflets and brochures)

18. Rhetorical task: narrative, descriptive, instructive, expository Pattern of exposition: define, describe, elaborate, illustrate, compare and contrast, classify Explicitness of text structure: explicit

19. Structural resources Words/sentence: Ave Flesch-Kincaid Grade Level: Ave: 8 The complexity of sentence structure: mostly simple sentences (but some use of subordinate clauses in PET) Cohesion: explicit

20. Lexical resources 95% K1-20 words = K1 84.7 % K2 8.7% K3 2.3% AWL words: 2.5%

21. Nature of information (abstract/concrete): concrete Content knowledge: not required: everyday situations encountered in work, school, leisure etc., (personal feelings, opinions and experiences, hobbies and leisure… familiar topics in expository texts

22. COGNITIVE Type of reading: - understand the main points and/or relevant points though not necessarily in detail (description of events, feelings and wishes, significant and clearly signalled reasoning, and argumentation) -identify unfamiliar words from the context -extrapolate the meaning of occasional unknown words from the context and deduce sentence meaning

23. Text level: word, sentence and across sentences

24. SAMPLES OF TEST SPECIFICATIONS ANALYSIS  See word file

25. Components of the Test  Grammar  Vocabulary AWL words  Reading  Test on CD

26. processed through lexitutor software (http://www.lextutor.ca/vp/). The analysis revealed frequencies of vocabulary, types and levels (K1, K2, K3, AWL); types, tokens, families, tokens per family, types per family).http://www.lextutor.ca/vp/ Flesch-Kincaid Grade Level was also taken into consideration when adapting reading texts to correspond the specifications in B1 B2 C1. The level of difficulty of the Vocabulary

28.  Thank you.