1. Student Use of CALL Software and its Effect on Learners Alan Bessette Bessette@poole.ac.jp Poole Gakuin University GloCALL - 2007

2. Learning Activities Authentic communication with other learners or native speakers through CMC activities Authentic tasks such as web searches, web quests and simulation games Perhaps most common: language practice

3. CALL for language practice Provides learner with immediate feedback Provides learners with autonomy over their learning

4. Benefits to learners Opportunity to notice is necessary for the conversion of input to intake. Schmidt (1994) Control can increase motivation and help facilitate language learning.

5. Research Questions Preliminary Research Question: Does learner performance improve? Main research questions: 1.Is there a correlation between learner improvement and how often learners repeated exercises? 2.Do learners assign importance to the software features that give them control over their learning?

6. Methods - Participants Two sets of students 1 st year  48 students  3 classes 2 nd year  52 students  3 classes

7. Methods - Classes Longman English Interactive (LEI)  1 st year: LEI 1 Modules B3 to C5  2 nd year: LEI 2 Modules B3 to C5 Classes met twice a week for 90 minutes  Once in computer lab  Use LEI for about 60 minutes

8. Methods - LEI Main features  Four level four skill software program  Three modules with 5 units each  Listening with video support (2), grammar, vocabulary, speaking, pronunciation, and reading activities  Unit quizzes and module tests

9. Methods – LEI Features that support learning  Glossary, grammar reference and video grammar coach  Immediate feedback: Software indicates correct answers and students can redo activities.  Students control number of times they watch and listen to listening activities.

10. Methods - Instruments Pre- and post-tests Module C test LEI reflection questionnaires End-of-study questionnaire

11. Pre- and Post-tests Given in Oct 2005 and February 2006, respectively Equivalency not validated statistically, but similar in every way Post-test is 30% of students final grade.

12. Module C test Overlap in what was covered Given at the end of the semester Module test is 10% of final grade.

13. LEI Reflection Questionnaires Adapted from Jamieson, Chapelle and Priess (2005) Given four times; about once every other week

14. LEI Reflection Questionnaires Students were asked to  Indicate level and which module they did  Rate how difficult an activity was  Indicate how many times they repeated an activity  Indicate how many mistakes they made

15. LEI Reflection Questionnaires Main purpose  Measure how the number of mistakes correlates with the number of times activities were repeated  Measure correlation between number of times activities were repeated and test performance.

16. End-of-study Questionnaire Usability and appropriateness of level Effectiveness of and level of interest for each activity Benefits of using LEI How LEI helped students improve

17. Results Study began with robust numbers  48 in 1 st year classes  52 in 2 nd year classes Due to absences for the pre- or post-tests and incomplete LEI reflection questionnaire data  19 in 1 st year classes  26 in 2 nd year classes

18. Test Results Pre-Post-Module C 1 st year Mean (n = 19) 64.976.879.1 2 nd year Mean (n = 26) 61.168.777.9

19. Test Results: Paired t-test Improvement between pre- and post-tests 1 st year: Significant improvement  Post-test was 11.9 points higher  t(18) = -4.752 and p < 0.0002 2 nd year: Significant improvement  Post-test was 7.6 points higher  t(25) = -4.058 and p < 0.0004

20. Test Results Post-test and Module C correlation 1 st year: positive, but not strong  r(14) = 0.331 and p<0.254 2 nd year: positive, but not strong  r(14) = 0.331 and p<0.254

21. 1 st Year LEI Reflection Questionnaires (n = 19)DifficultyTimes listened# of mistakesTimes attempted 1 st listening2.62.11.72.0 Grammar3.71.81.9 Vocabulary2.91.7 Reading3.41.71.9 2 nd listening2.82.11.81.9

22. 2nd Year LEI Reflection Questionnaires (n = 26)DifficultyTimes listened# of mistakesTimes attempted 1 st listening2.62.93.52.3 Grammar3.62.72.4 Vocabulary3.02.42.3 Reading3.52.72.5 2 nd listening2.8 2.42.5

23. Reflection Questionnaire Results Grammar and reading activities easier than listening activities  Vocabulary in between Not much variation in average number of mistakes and average number of times repeated  Slightly more time spent on listening

24. 1 st year correlation results Post-Module CRepeatsMistakes Post-1.00 Module C0.731.00 Repeats-0.42-0.401.00 Mistakes-0.62-0.460.901.00

25. 2nd year correlation results Post-Module CRepeatsMistakes Post-1.00 Module C0.331.00 Repeats-0.40-0.561.00 Mistakes-0.44-0.220.971.00

26. Correlation Summary Post-test and Module C  1 st year: strong positive correlation  0.73  2 nd year: positive correlation  0.33 Number of mistakes and repeats  1 st year: strong positive correlation  0.90  2 nd year: strong positive correlation  0.97

27. Correlation Summary Hypothesized correlations between post-test scores and repeats Negative and weak  1 st year: -0.42  2 nd year: -0.40

28. End-of-study Questionnaire Benefits of using LEI  Understand grammar: 24 (13, 11)  Improved listening: 35 (19, 16)  Improved speaking: 20 (10, 10)  Increased vocabulary: 20 (12, 8)  Made English fun: 25 (17, 8) Improved listening: 85%

29. End-of-study Questionnaire How LEI helped students improve  Listen many times: 33 (17, 16)  Repeat exercises: 28 (18, 10)  Study at own pace: 29 (17, 12)  Get correct answers: 12 (9, 3)  Grammar explanations: 13 (7, 6)  Increased motivation: 13 (9, 4)  At own level: 13 (9, 4)  Interesting: 19 (14, 5)  Lots of practice: 18 (10, 8)  Video clips: 24 (13, 11)

30. End-of-study Questionnaire Results positive Being able to control how they use LEI seems to be very important for students  Being able to listen many times  Being able to repeat exercises  Being able to study at their own pace

31. Preliminary Research Question Does learner performance improve? Pre- and post-test scores showed improvement for both 1 st year and 2 nd year students  1 st year improvement was greater  LEI 2 is probably more difficult Cannot conclude that LEI is responsible for improvement  Too many other factors

32. 1 st Research Question Is there a correlation between learner improvement and how often learners repeated exercises? No correlations between post-test and number of times activities were repeated Strong correlation between number of mistakes and repetition of activities  Students are using software as expected.

33. 2 nd Research Question Do learners assign importance to the software features that give them control over their learning? End-of-study questionnaire results strongly support this hypothesis.  Being able to listen many times: 87%  Being able to repeat exercises: 74%  Being able to study at their own pace: 76%

34. Conclusions Major problem was the small sample sizes  Difficult to control for absences and incomplete data Need a more accurate method of obtaining student use of software  Software to record student behavior expensive, but better than student reports.

35. Selected References  Jamieson, Joan, Carol A. Chapelle and Sherry Preiss. 2005. “CALL Evaluation by Developers, a Teacher, and Students.” CALICO Journal Volume 23 (1): 93-138.  Schmidt, Richard. 1994. “Deconstructing consciousness in search of useful definitions for applied linguistics.” AILA Review. 11: 11-26  Taguchi, Nobuko and Keiko Schneider. 2004 “Longman English Interactive.” CALICO Journal. 6. p. 23.  LEI website  http://www.pearsonlongman.com/ae/multimedia/pdf/LEI1AE_scope_seq.pdf http://www.pearsonlongman.com/ae/multimedia/pdf/LEI1AE_scope_seq.pdf  http://www.pearsonlongman.com/ae/multimedia/pdf/LEI2AE_scope_seq.pdf http://www.pearsonlongman.com/ae/multimedia/pdf/LEI2AE_scope_seq.pdf