1. Comparing two vocabulary tests and TOEIC
Masaya Kanzaki
Kanda University of International Studies
12:45-1:10, Room MR35

2. “In English, there is a relatively close relationship between how many words you know, as measured on the standard vocabulary tests, and how well you perform on reading tests, listening tests and other formal tests of your English ability.”
Nation & Meara (2002). Vocabulary. In N. Schmitt (Ed.), An introduction to applied linguistics.

3. Vocabulary Levels Test
1 business 2 clock part of a house 3 horse animal with four legs 4 pencil something used for writing 5 shoe 6 wall

4. Vocabulary Size Test
1. see: They saw it. a. cut b. waited for c. looked at d. started

5. Purpose
To compare scores on the Vocabulary Levels Test (VLT), the Vocabulary Size Test (VST), and the TOEIC and see how closely they are related

6. Participants
140 university students who enrolled in 5 sections of a TOEIC preparatory course in April , 2011
International Communication majors
99 second year, 19 third year, and 22 fourth year students
102 female and 38 male

7. Materials
4 levels of the VLT Version 1 (2000, 3000, 5000, and Academic Vocabulary, 30 questions per level, 120 questions in total)
5 levels of the VST (from 1st to 5th 1000, 10 questions per level, 50 questions in total)
TOEIC (100 listening questions: score range 5-495, 100 reading questions: score range 5-495, Total score range )

8. Procedures – VLT Done in class on Day 2 (April 13-15)
40 minutes for 4 levels (120 questions)
A separate answer sheet was used.
An answer key was provided and students scored the test themselves.
The answer sheets were collected.
133 students took the test but 6 of them did not complete it. (N = 127)

9. Procedures – VST Done in class on Day 3 (April 18-20)
25 minutes for 5 levels (50 questions)
A separate answer sheet was used.
An answer key was provided and students scored the test themselves.
The answer sheets were collected.
127 students took the test but 5 of them did not complete it. (N = 122)

10. Procedures – TOEIC
Institutional Program (IP test) on campus on May 27, 2011, 5 p.m.-7:30 p.m.
Students had to pay 3,700 yen to take the test.
2 hours, 200 questions
The results were provided by the TOEIC organizer, IIBC, two weeks after the test.
62 students took the test. (N = 62)

11. Results

12. Descriptive Statistics (N = 127)
VLT
Descriptive Statistics (N = 127)
VLT 2000
VLT 3000
VLT 5000
VLT AV
VLT Total
Mean
25.00
18.61
16.43
19.28
79.32
Median 25 19 16 20 79
Mode 18 SD
3.46
4.83
4.01
4.54
13.70
Variance
11.95
23.33
16.12
20.57
187.79
Range 22 23
Min 12 7 4 36
Max 30 29 27
115

13. VLT
Histogram (N = 127)

14. VLT Correlations (N = 127) VLT 2000 VLT 3000 VLT 5000 VLT AV VLT Total
.49*
.35*
.54*
.71*
.57*
.65*
.86*
.60*
.78*
.87*
* p < .01

15. Descriptive Statistics (N = 122)
VST
Descriptive Statistics (N = 122)
VST 1000
VST 2000
VST 3000
VST 4000
VST 5000
VST Total
Mean
8.99
6.20
6.86
5.54
4.63
32.22
Median 9 6 7 5 32
Mode 4 33 SD
0.90
1.30
1.27
1.23
1.42
3.50
Variance
0.82
1.70
1.63
1.52
2.02
12.22
Range 18
Min 3 1 23
Max 10 8 41

16. VST
Histogram (N = 122)

17. VST Correlations (N = 122) 1 .18 .07 -.07 .08 .36* .20 .30* .34* .74*
VST Total 1
.18
.07
-.07
.08
.36*
.20
.30*
.34*
.74*
-.01
.49*
.22*
.53*
.66*
* p < .01

18. Descriptive Statistics (N = 62)
TOEIC
Descriptive Statistics (N = 62)
TOEIC L
TOEIC R
TOEIC Total
Mean
350.24
246.61
596.85
Median
352.5
250
595
Mode
405
255
590 SD
47.70
55.37
92.34
Variance
Range
230
280
495
Min
225
130
370
Max
455
410
865

19. TOEIC
Histogram (N = 62)

20. TOEIC Correlations (N = 62) TOEIC L TOEIC R TOEIC Total 1 .60* .88*
.91*
* p < .01

21. VLT vs. VST Correlations (N = 109) VLT 2000 VLT 3000 VLT 5000 VLT AV
VLT Total
VST 1000
.11
.33*
.38*
.34*
.37*
VST 2000
.35*
.28*
.27*
VST 3000
.06
.13
.22
.16
.18
VST 4000
.24*
.21
.30*
.32*
VST 5000
.31*
.17
VST Total
.41*
.46*
.43*
.50*
.57*
* p < .01

22. VLT vs. VST

23. VLT vs. TOEIC Correlations (N = 59) TOEIC L TOEIC R TOEIC Total
.27
.50*
.44*
VLT 3000
.31*
.74*
.61*
VLT 5000
.42*
.66*
.62*
VLT AV
.47*
.69*
VLT Total
.45*
.80*
.72*
* p < .01

24. VLT vs. TOEIC

25. VST vs. TOEIC Correlations (N = 54) TOEIC L TOEIC R TOEIC Total
.11
.31
.25
VST 2000
.27
.30
.33*
VST 3000
.17
.04
.12
VST 4000
.15
.29
.26
VST 5000
.28
.36*
VST Total
.38*
.48*
.49*
* p < .01

26. VST vs. TOEIC

27. VLT & VST vs. TOEIC Correlations (N = 51) 1 .52* .98* .36* .79* .68*
VLT Total
VST Total
VLT&VST
TOEIC L
TOEIC R
TOEIC LR 1
.52*
.98*
.36*
.79*
.68*
.67*
.32
.48*
.46*
.38*
.69*
.85*
.89*
* p < .01

28. VLT & VST vs. TOEIC

29. Why does the VLT correlate better with the TOEIC?
The number of questions
Test-toughness
Anything else?

30. References
Beglar, D. (2010). A Rasch-based validation of the Vocabulary Size Test. Language Testing, 27 (1), 101–118. Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the vocabulary levels test. Language Testing, 18 (1), 55–88.