1. Animated Pedagogical Agents: Historical Figure Applications Bob Heller, Athabasca University bobh@athabascau.ca Mike Procter, Athabasca University mikeprocter@shaw.ca Presented at the International Workshop on Intelligent and Adaptive Web-based Education System (IAWES 2007) held in conjunction with the 15 th annual International Conference on Computers in Education, Hiroshima, Japan, November 6, 2007

2. Acknowledgements Daniel Ha – chat log analysis Lisa Jewell – chat log analysis, content developer, test chatter Meggie Yang – AIML builder Dean Mah – Web implementation Billy Cheung – Graphics, test chatter Julianna Charchun – Chat log analysis Jude Onuh – AIML programmer Funding from the Athabasca University Mission Critical Research Fund

3. Outline  APA Review  Historical Figure Conversational Agents Freudbot Piagetbot  Animated Historical Figures Freudbot  Concluding Coments

4. Animated Pedagogical Agents Definitions programs deployable on the internet with an embodied representation and interactive capabilities through one or more modalities APAs have an Artificial Intelligence backend that allows the designer to simulate communicative agent behaviour while guiding human-agent interactions towards pedagogical goals and objectives

5. Animated Pedagogical Agents Johnson, Rickel, & Lester (2000) Advantages of APA increase the “bandwidth of communication” increase computer capacity to engage and motivate The Persona effect

6. Animated Pedagogical Agents: Persona Effect Dehn & Van Mulken’s (2000) literature review Major Findings 1. User’s subjective experience yes 2. User’s objective behaviour ? 3. Performance outcomes?

7. Animated Pedagogical Agents: Persona Effect Research limitations variability in animation/visual presentation dependent measures (esp. social presence) experimental control constrained application space

8. Animated Pedagogical Agents: Persona Effect Historical Figure Applications Stronger test of the engagement function Expanded set of social roles Opportunity to use narrative Tighter synthesis between content and persona

9. Historical Figure Conversational Agents Freudbot Developed using AIML Design Principles  Narrative Theory  Psycholinguistic Theory with emphasis on pragmatics Content development  Test chatters  How much ALICE?  Pilot study

10. Heller, Procter, Mah, Jewell, & Cheung (2005) Proof of concept Instructional set Historical Figure Conversational Agents: Freudbot

11. Method 53 participants (81% women) recruited through the web Random assignment to instructional set condition Focused on self-report measures of chat experience and analysis of conversational log Historical Figure Conversational Agents: Freudbot

12. Findings Neutral evaluation overall Those who would chat again (64%) had significantly higher ratings than those who would not Freudbot experience  “ It was pretty cool the way it felt like I was actually interacting with Freud... he's deceased though, yeah, but the picture, the fast answers... made me pay attention to the answers alot more than if I had been simply reading a text written by someone else. Plus it was cool to feel like I could voice my own opinion with the most well-known psychoanalyst of all time.” Historical Figure Conversational Agents: Freudbot

13. Findings No effect of instructional set Historical Figure and Quizbot applications were highest rated Expansion was strongly supported Need for improvement Historical Figure Conversational Agents: Freudbot

14. Heller & Procter (2007)  Developed Piagetbot based on Jean Piaget  Leverage development costs of Freudbot  Focused on learning outcomes under two different interfaces: Text-based vs. Chat- based Historical Figure Conversational Agents: Piagetbot

15. Method 59 participants (85% women) recruited through the web Random assignment to interface condition (text vs. chat) Focus on self report measures and learning outcomes Historical Figure Conversational Agents: Piagetbot

16. Text Interface Historical Figure Conversational Agents: Piagetbot

17. Chat Interface Historical Figure Conversational Agents: Piagetbot

18. Findings  Logical interface associated with better learning outcomes.  Logical interface was associated with higher ratings  Piagetbot and Freudbot were equally rated  Those who would chat again (60%) had significantly higher ratings than those who would not Historical Figure Conversational Agents: Piagetbot

19. Animated Historical Figures Animating the conversational agents Strong test of the engagement function of APAs Lower threshold of believability Avoids many of the visual variables Haptek Character Software – People Putty text to speech, lip sync, facial expressions, personality parameters

20. Objectives Effects of animation (no image, static image, animated image) Effects of targeting Role of learning styles Animated Historical Figures: Freudbot

21. Method 88 participants (81% women) recruited through the web Random assignment to interface condition (no image, static image, animated image) Animated Historical Figures: Freudbot

22. Preliminary Findings  The no image condition associated with significantly higher ratings than the two other conditions that did not differ from each other  Those who would chat again (64%) had significantly higher ratings than those who would not overall and within conditions Animated Historical Figures: Freudbot

23. Concluding Comments  Animation needs to better integrated with communicative acts  Room for improvement in conversational performance  Across all 3 studies, appox 2/3 will chat again

24. Thank you

25. Animated Pedagogical Agents IntelliMedia Centre for Intelligent Systems – U of North Carolina http://research.csc.ncsu.edu/intellimedia/ Herman the Bug Cosmos Whizlow

26. Animated Pedagogical Agents Centre for Advanced Research on Technology in Education (CARTE) – USC http://www.isi.edu/isd/carte/ STEVE ADELE

27. Animated Pedagogical Agents Teachable Agents – Vanderbilt U http://www.teachableagents.org/overview.html Autotutor – U of Memphis http://www.autotutor.org/ Research on Innovative Technologies for Learning – Florida State U http://ritl.fsu.edu/_Website/index.asp - Pedagogical Agent Learning Systems

28. Animated Pedagogical Agents Embodied Conversational Agents - Northwestern U Media Lab – MIT http://agents.media.mit.edu/ http://agents.media.mit.edu/

29. Conversational Agents  AKA Chatbots, Chatterboxes, Talk bots  SmarterChild  ALICE – Artificial Linguistic Internet Computer Entity http://www.alicebot.org/

30. Conversational Agents

31.  ELIZA (1966) “Its name was chosen to emphasize that it may be incrementally improved by its users, since its language abilities may be continually improved by a "teacher". Like the ELIZA of Pygmalion fame, it can be made to appear even more civilized, the relation of appearance to reality, however, remaining in the domain of the playwright.” Weizenbaum, 1966, p. 2  Like ELIZA, ALICE relies on simple pattern matching and iterative/incremental adjustments  Artificial Intelligence Markup Language (AIML)

32. Conversational Agents  ALICE/AIML Strengths Open source and XML compliant Numerous AIML parsers Numerous front-end options Loebner contest winner Large user group http://www.pandorabots.com http://www.pandorabots.com ALICE’s brain is free Easy language to learn

33. Conversational Agents  ALICE/AIML Weaknesses Content generation Maintenance (AKA Targeting) Bending the will of ALICE Hard language to master

34. Historical Figure Application Emile http://www.hud.ac.uk/hhs/research/emile/emileframeset.htm Shakespeare http://www.pandorabots.com/pandora/talk?botid=c6937cfb3e354738

35. Discussion  Historical figures vs. coaches  Course Assistant Expanding the scope  Quiz pal  English as Second Language Teacher  Speech Rehab Therapist

36. Animated Pedagogical Agents Gulz and Haake (2006) Focus on motivation & engagement as the promise of APAs & the importance visual form and look APA engagement can directly influence behaviour and cognition APAs must be believable & lifelike (like synthetic agents in entertainment applications) to maintain/strengthen the illusion of social exchange.

37. Animated Pedagogical Agents Limited range of application ‘Constrained’ by backend systems Historical Figure Applications - greater potential to maximize engagement