1. Evaluation of a Virtual Reality Game for Education Maria Virvou, Constantinos Manos, George Katsionis, Kalliopi Tourtoglou Department of Informatics University of Piraeus Piraeus 18534, Greece mvirvou@unipi.grmvirvou@unipi.gr; kman@singular.gr;kman@singular.gr gkatsion@singular.grgkatsion@singular.gr; ktourtog@internet.grktourtog@internet.gr

2. Evaluation of educational software Evaluation is a crucial part of the design process of educational software that has to be iterative to a large extend. The educational material must be tried out on students and refined and then tried again and the cycle continued for as long as necessary (Jones et al.,1993)

3. The Multi-Tutor Game A virtual reality game that teaches students several subjects such as mathematics, history, geography and spelling. It aims at increasing student's engagement by providing a popular and motivating VR environment.

4. The Multi-Tutor Game History is taught in a virtual world of lands with castles and warriors. Biology is taught in a virtual water world. Spelling is taught in a virtual world of woods. Mathematics is taught in a virtual world of planets of the outer space. Each domain is taught in a different VR world

5. Images…

6. The Multi-Tutor Game The player tries to reach the "land of knowledge" and find the treasure which is hidden there. To achieve this, the player has to obtain a good score, which is accumulated by all four domains. The game aims at motivating students to have a good standard in lessons which are different from each other.

7. The Multi-Tutor Game Common student modelling component for all four domains. Error diagnosis: The student receives points for his/her total score depending on the severity of the error. Domain independent student features are common for all four domains (e.g carelessness/ carefulness of the student)

8. Evaluation One important aspect of the evaluation is the reason why educational software is adopted in the first place, i.e. what the underlying rationale is (Jones et al.,1993) In the case of the Multi-Tutor Game, the objective was: To make educational software more engaging and motivating than other forms of software while retaining and even improving the underlying reasoning mechanisms.

9. Parts of the evaluation One part: Comparison between The Multi-Tutor Game and educational software with a conventional user interface but with the same underlying reasoning mechanisms. Second part: Finding out the extend to which the Multi-Tutor Game could be used by children at their leisure time.

10. Classroom experiment participants 16 school children of 11-12 years old. 4 human teachers of history, biology, spelling and mathematics respectively. 2 computer assistants.

11. Classroom experiment A class of 16 school children were divided into two groups: of 8 children. The first group were given the the Multi-Tutor Game to work with. The second group were given educational software with a simple interface but with the same underlying mechanisms.

12. Classroom experiment results After having interacted with the software, the players of the Multi-Tutor Game remembered the correct answers to a higher extend than the other group. This showed that the game had achieved its aim of being at least as effective as conventional educational software in the learning outcomes (in fact: slightly better)

13. Classroom experiment results Time: On average, the students who had used the Multi-Tutor Game had spent more time with the system. a) More to explore the game. b) More time to read lessons. This showed that the Multi-Tutor Game was indeed more engaging.

14. Classroom experiment results Interviews: The players of the Multi-Tutor Game were fascinated by the idea of a game in the classroom. They were more enthusiastic about the software they had used than the other group of students.

15. Lab experiment Assessment of the entertaining effects of the Multi-Tutor Game. 20 children of 11-12 years old from the same school and class. No human teachers present. 3 computer assistants.

16. Lab experiment results Since the game was not a school assignment, children focused on the game environment. 73% pointed out that the game would be better as a game if it was more adventurous with more sounds and more virtual objects.

17. Lab experiment results 46% commented on the educational aspect of the game and said they had liked it. 35% made no comment. 19% said that they were annoyed by the fact that the game reminded them of the school syllabus. However, most of them remembered what they had learned from the game in the domain of lessons.

18. Conclusions Children would be quite happy to work with a computer game which represents a more amusing teaching fashion than that of conventional educational software. The educational benefits of the game are at least as good as those of conventional educational software.

19. Future work Children are quite familiar with commercial games and therefore they have high expectations from the game environment. The virtual environment of the Multi-Tutor Game is planned to be enhanced in the near future.