The effectiveness of virtual conversations in language learning among Malaysian students By Harmi Izzuan Bin Baharum Alexei Tretiakov Anuradha Mathrani

Overview Background of the problem Statement of the problem Research Questions Research Design & Methodology Data Analysis and Results Discussion Implications

Background of the problem English in Malaysian Society Malaysian society is culturally diverse, involving Malay, Chinese, and Indians The official language is Malay, but English is highly important in business Most Malays are not sufficiently proficient in English Many Malays reside in rural areas with poor access to learning opportunities Dominance of Malay in civil service where many Malays are employed Negative attitude towards learning English among Malays due to colonial history Most adult Malays are shy and difficult to engage due to cultural reasons

Background of the problem Application of MUVE in education and language learning: Suitable scaffolding environment (Roed, 2003), promote interactional and transactional strategies (Peterson, 2005), non-threating environment (Rankin et al, 2006), focusing on new vocabulary ( Peterson, 2006), useful and interesting language learning platform (2009), authentic environment for interaction (Liou, 2012) role-play (Dalgarno et al, 2016) spatial contexts (Berger et al, 2016). Research studies are still limited both in the application of MUVE in education and language teaching with respect to measuring: - Effectiveness My research investigates quantitatively effectiveness of 3D MUVE in facilitating scenario-based approach. Research focuses on one important practical problem – teaching oral business English communication skills to Malay learners.

English teaching paradigms and Computer assisted language learning Malaysia New Approach - 21st Century Integrative CALL Constructivist Task-based Content-based ESP Business English Socio-cognitive Situated learning Scenario-based learning Multimedia & Internet Multi-user immersive 3D environment Authentic discourse Malay business settings Agency Interaction Negotiation 1970s-1980s: Structural CALL 1980s-1990s: Communicative CALL English-teaching paradigm Grammar-translation and audio-lingual Communicative language teaching View of language Structural ( a formal structural system) Cognitive ( a mentally constructed system) Materials / Technology Textbooks, teaching modules, self-access learning materials ( cassettes and video tapes E-mail & Chat tool Principal use of computers Drill and practice Dialogue-based software with fixed response Communicative exercises Principal objective Accuracy And fluency (adapted from Warschauer, 2000)

Statement of the Problem Due to the limitations of the learning opportunities attitude & personality traits: *Malay learners needs assistance to improve their oral English skills. *Malay learners need new approach within their cultural context to encourage them to use the language.

Research Questions Is online three dimensional multi-user virtual environment effective in facilitating scenario-based approach of oral business English to Malay learners?

Research Design & Methodology Design a scenario-based business English teaching course in a MUVE environment For experiment, create in immersive MUVE environment For control, create also a conventional (classroom) version of the course Design and execute an experiment to evaluate the effectiveness of MUVE in facilitating SBE A real experiment involving control to provide a valid comparison

Participants Malay learners University students English for Professional Communication course 144 from 180 students volunteered in the study To ensure validity, use a representative sample (e.g., a class including both urban and rural learners)

Effectiveness of the learning system Two groups: A and B, belonging to the same English course. Subjects are allocated to groups A and B at random. Repeated measures design – ensures similar experience for all subjects to minimize ethical concerns. Measure learning gains/outcome Group A Group B Evaluation 1 (E 1) Study stage 1 ( S1) Use the system Do not use the system Evaluation 2 (E 2) Study stage 2 (S 2) Final evaluation ( E 3)

Research Instruments Effectiveness Oral assessment scale Measure learning gains. 2 raters.

Oral assessment scale To determine learning gains An analytic scale-focus on the specific language features Existing scale by Oliveira (2004) based on Bachman and Palmer’s language testing model

Oral assessment scale Appendix 4: Proficiency Rating Scale Student Number   4 3 2 1 Organizing ideas Presents information nearly everything and replies appropriately, with little need for adjustment or repetition Presents information fairly well and usually replies appropriately, needing some repetition and/or adjustment Presents little information and needs constant adjustment and repetitions Presents very little information ; needs constant adjustment and repetitions Introductions Presenting the house from outside Presenting the house from inside Parting Vocabulary Produces the vocabulary required to carry out the task successfully Sometimes lacks the right words, but manages to accomplish the task reasonably well Lacks many of the right words, but manages to accomplish the task Lacks most of the vocabulary required, which seriously harms his/her performance in the task 5 Names of rooms 6 Price of the house 7 Nature of room (large, small etc.) 8 Size of the house Appendix 4: Proficiency Rating Scale

  4 3 2 1 Speech shows very few problems with word order and agreement Speech shows few problems with word order and agreement Speech shows several problems with word order and agreement Speech shows constant problems with word order and agreement 9 Accuracy Student hardly ever halts or hesitates Student is occasionally hesitant but manages to accomplish the task Student frequently halts and hesitates, compromising the accomplishment of the task Student halts and hesitates, seriously compromising of the task. 10 Fluency

Data Analysis & Findings (Effectiveness) Classroom Teaching Second Life Teaching Pair Pre-test score Post-test score Difference Mean 48.32 52.971 4.64 49.147 53.412 4.26

Data Analysis (Effectiveness) Results of the T-test of the classroom teaching Paired Differences t df P value Mean Std Deviation 95% Confidence interval of the Difference 4.647059 5.87 -2.917575 -4.5473 33 3.478e-05

Data Interpretation Effectiveness Finding shows that there is a significance difference between the pre-test scores and the post-test scores of the classroom teaching. The increase at 4.64 is proven to be significant. Therefore, the classroom teaching has been successful in increasing the oral proficiency scores of the students significantly.

Data Analysis Effectiveness Results of the T-test of the Second Life teaching Paired Differences t df P value Mean Std Deviation 95% Confidence Interval of the Difference 4.264706 5.82 -2.583987 -4.2942 33 7.245e-05

Data Interpretation Effectiveness Finding shows that that there is a significance difference between the pre-test scores and the post-test scores of Second Life teaching. The increase at 4.26 is proven to be significant. Therefore, the Second Life teaching has also been successful in increasing the oral proficiency scores of the students significantly.

Classroom Teaching VS Second Life Teaching The results of the T-test between Classroom Teaching and Second Life Teaching Unpaired t-test t df P value Mean 95% confidence Interval of the difference Classroom Teaching 4. 647 -2.613380 0.2683 66 0.6054 Second Life Teaching 4.264 2.741157

Data Interpretation Effectiveness Use T-Test comparing classroom teaching and Second Life teaching Based on the analysis, the p-value is 0.6054. Based on the P –value, there is no significance difference between classroom teaching and Second life teaching. However, the means shows that Classroom teaching is slightly better than Second Life teaching in terms of improving oral communication skills among the students.

Discussion of Findings (Effectiveness) Scenario-based approach in terms of reproducing realistic social interaction scenario support the its use in teaching of oral business English communication skills. Scenario-based teaching can be executed in MUVE and it is as effective as in the classroom environment.

Implications for practice Scenario-based teaching on MUVE can be used for teaching Scenario-based teaching on MUVE can be used in distance learning

Conclusion Empirical evaluation of effectiveness The research provided quantitative evidence to confirm the effectiveness of MUVE as an environment for scenario-based business English learning. More broadly, the research provided, based on a quantitative empirical evaluation, support the increasingly held view that MUVE offer a platform for effective teaching and learning.