A MBI L EARN Ambient Intelligent Multimodal Learning Environment for Children Confirmation Viva June 2009 Jennifer Hyndman, B.Sc. (Hons.) Supervisors: Dr. Tom Lunney, Prof. Paul Mc Kevitt School of Computing and Intelligent systems Faculty of Computing and Engineering University of Ulster, Magee. 1

O VERVIEW Research Objectives Literature Review Research Motivation Project Proposal Evaluation and testing Software Analysis & Prospective Tools Relation to other work Potential Contributions Research Plan Thesis Outline Publications Conclusion 2

R ESEARCH O BJECTIVES Investigate potential of an educational game integrated with virtual learning environment Positive impact on children at a primary level Means to merge multiple modalities Mobile Devices e.g. Smart Phones (Nokia) /Personal Digital Assistant (PDA) Interface Design /Usability Role of serious games AmbiLearn / TreasureLearn Educational game e.g. Treasure hunt / puzzels Test and evaluate AmbiLearn/TreasureLearn Demonstrate educational potential 3

L ITERATURE R EVIEW Multimodal Interfaces Natural communication methods Bolt’s Media room (Bolt, 1980) SmartKom (Wahlster et al. 2001) Mobile interfaces Speech and pen Complementary modalities (Cohen, 1992) Multimodal interfaces for Children Collaborative environments KidsRoom (Bobick et al. 1998) TICLE (Scarlatos, 2002) 4

Embodied Agents Persona Effect (Lester et al. 1997) Gandalf (Thorisson, 1997) Speech, gaze, facial expressions, gestures and projected screen Steve (Johnson et al. 1997) Speech, gaze, gesture, demonstration Hans Christian Anderson (Corradini et al. 2005) Speech, gesture 5

Learning Environments Technology enhanced learning M-learning, e-learning, web-based learning Games in education Pedagogical features Feedback/Motivation (Rosas et al. 2003) Augmented reality Enhancing real world experiences ReadIt (Weevers et al. 2004) Virtual reality Rich sensory experience Submerge into learning context (Crumpton et al. 1997) 6

Virtual learning environments Information – anytime/anywhere Unclear pedagogical effects (Dillenbourg, 2002) Monitor and assess Human Computer Interaction Child computer interaction (ChiCi group, 2007) Child centred design Interaction styles (Strommen et al. 1996) Dexterity/speech/reading/background/interaction (Bruckman et al. 2007) 7

R ESEARCH M OTIVATION 8

P ROJECT P ROPOSAL Multimodal communication collaborative learning mobile handheld device AmbiLearn architecture Multimodal mobile system Ubiquitous learning environment TreasureLearn application Treasure hunt / puzzles Designed for children Collaborative learning / problem solving 9

A RCHITECTURE OF A MBI L EARN 10

T REASURE L EARN A CTIVITY 11

E VALUATION /T ESTING OF A MBI L EARN /T REASURE L EARN Pre-test / Post-test Fun Toolkit (Read & MacFarlane, 2006) 12

S OFTWARE A NALYSIS & P ROSPECTIVE T OOLS Spoken dialogue systems WebSphere Voice Toolkit (IBM, 2009) Microsoft Speech (Microsoft, 2008) Embodied Agents X-Face (Balci, 2004) COLLAGEN (Cassell, 2001) Development kits Java ME – mobile handheld devices DevKit Pro – Nintendo homebrew Apple SDK – iPod/iTouch/iPhone 13

R ELATION TO OTHER WORK 14

P OTENTIAL C ONTRIBUTIONS Improve upon existing multimodal learning environments Supporting collaborative learning indirectly through the use of game Provide user logging module – future analysis Contribute to knowledge Multimodal mobile games in education Multimodal interfaces on mobile devices Collaborative learning with mobile devices Advance the area in terms of providing a flexible and rich learning experience for children 15

R ESEARCH P LAN Activities Submissions Deliverables 16

T HESIS O UTLINE 0. Abstract / Acknowledgements / Declaration Chapter 1. Introduction Chapter 2. Literature Review Chapter 3. Ambient Intelligent Multimodal Learning Environment for Children Chapter 4. AmbiLearn and TreasureLearn Implementation Chapter 5. Evaluation of AmbiLearn and TreasureLearn Chapter 6. Conclusion Appendices / References 17

P UBLICATIONS Hyndman, J., Lunney, T. And Mc Kevitt, P. (2009) AmbiLearn: Ambient Intelligent Multimodal Learning Environment for Children. In: Proceedings of the 10 th Annual PostGraduate Symposium on The Convergence of Telecommunications, Networking & Broadcasting, PGNet, Liverpool, 22 nd – 23 rd June. Liverpool UK : Liverpool John Moores University

C ONCLUSION 19 Focus on multimodal learning environments Mobile Multimodal interfaces Serious games Design and develop AmbiLearn architecture /TreasureLearn architecture Demonstrate and test educational potential AmbiLearn and TreasureLearn Evaluate AmbiLearn/TreasureLearn Improve upon existing multimodal learning environments Supporting collaborative learning indirectly through the use of game Provide user logging module – future analysis

T HANK YOU ! Q UESTIONS ? 20 I never try to teach my students anything, I only try to create an environment in which they can learn. – Albert Einstein