1. Cristina Conati Department of Computer Science University of British Columbia Intelligent Tutoring Systems: New Challenges and Directions

2. # Students Learning gains Classroom Students Classroom Students Students with Human Tutor Students with Human Tutor 22 2 sigma effect (Bloom, 1984): Average student’s achievement with a personal human tutor better than 98% of classroom students Bloom, B. "The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring." Educational Researcher 13 (6):4–16. Preamble

4. Intelligent Tutoring Systems (ITS) u Interdisciplinary field Cognitive Science ITS  Aiming to  Create computer-based tools that support individual learners  By autonomously and intelligently adapting to their specific needs Education Artificial Intelligence Human-Computer Interaction

5. Outline u Background and definitions u Achievements and new research directions u Sample Projects

6. Precursors of ITS  Computer-Assisted Instruction (CAI) systems Cognitive Science ITS Education Artificial Intelligence Human-Computer Interaction CAI

7. Curriculum Computer Answer Present Problem Get Student Answer Compare Answers If correct If incorrect Remediation Present Feedback  Computer-Assisted Instruction (CAI) systems Precursors of ITS Curriculum Computer Answer Present Problem Get Student Answer Compare Answers If correct If incorrect Remediation Present Feedback Shute and Potska 1996, Handbook of Research on Educational Communications and Technology

8. CAI systems (cont.)  All branching in the program to be pre-defined Sequencing of topics and exercises All relevant student answers All feedback actions  Student’s solution process is not taken into account, only final answers No information on the reasons for the student behavior  Fine for drill-and-practice in simple domains such as basic math operations Unmanageable for more complex domains and pedagogy (e.g., support problem solving in physics)

9. N

10. Good Human Tutors..  Can provide more flexible and comprehensive support to learning  Recognize a large variety of student’s behaviors  Diagnose student’s understanding (and other relevant states)  Provide adequate tailored interventions at different stages of the interaction

11. Intelligent Tutoring Systems (ITS) Cognitive Science Education Human-Computer Interaction Artificial Intelligence - Represent knowledge and processes relevant for effective tutoring - Reason to select effective tutorial actions - Learn from experience CAIITS

12. Pedagogical model -Teaching strategies -Remediation - curriculum computer solution (solution step) Student solution (solution step) Communication model Domain Model Concepts Principles.. Tutor Solution Generator Interface Student Modeler Student model -knowledge, Goals, Beliefs… Ideal ITS Pedagogical model -Teaching strategies -Remediation - curriculum computer solution (solution step) Student solution (solution step) Communication model Domain Model Concepts Principles.. Tutor Solution Generator Interface Student Modeler Tutorial Action -Select activity Student model -knowledge, Goals, Beliefs… -Hints -Feedback -Corrections -Etc.

13. Achievements u In the last 20 years, there have been many successful initiatives in devising Intelligent Tutoring Systems (Woolf 2009, Building Intelligent Interactive Tutors, Morgan Kaufman) –Including CanergieLearning, a company that commercializes ITS for Math in hundreds of high schools in the USA

14. # Students Learning Improvements Classroom Students Classroom Students Human Tutor ~1  22 How much learning improvement? ITS Most sophisticated CAI: 0.5  (Dodds & Fletcher, 2004)

15. However… u Mainly ITS that provide individualized support to problem solving through tutor-lead interaction (coached problem solving)coached problem solving –Well defined problem solutions => guidance on problem solving steps –Clear definition of correctness => basis for feedback

16. Coached problem solving Example: Andes, tutoring system for Newtonian physics Conati, Gertner and VanLehn 2002, J. of User-Modeling and User-adapted Interaction

17. Beyond Coached Problem Solving u Coached problem solving is a very important component of learning u Other forms of instruction, however, can help learners acquire the target skills and abilities –At different stages of the learning process –For learners with specific needs and preferences

18. Key Trends in ITS Affective Tutors -Understand and react to learners emotions Collaborative Learning Environments - Adaptive scaffolding of effective group-based learning ITS Adaptive Open Learning Environments -Support learning via free exploration of virtual worlds, interactive simulations and educational games Meta-Cognitive Tutors Scaffold acquisition of learning and reasoning (meta-cognitive) skills

19. Challenges u Activities more open-ended and less well-defined than pure problem solving –No clear definition of correct/successful behavior u Higher-level user states (meta-cognitive, affective) –difficult to assess unobtrusively from interaction events u High level of uncertainty

20. Key Trends in ITS Affective Tutors -Understand and react to learners emotions Collaborative Learning Environments - Adaptive scaffolding of effective group-based learning ITS Adaptive Open Learning Environments -Support learning via free interaction with virtual worlds, interactive simulations and educational games Meta-Cognitive Tutors Scaffold acquisition of learning and reasoning (meta-cognitive) skills

21. Our Approach u Student models based on formal methods for probabilistic reasoning –Bayesian networks and extensions u Decision theoretic approach to tutorial action selection u Increase the bandwidth through innovative input devices: –e.g. eye-tracking and physiological sensors

22. Outline u Background and definitions u Achievements and Trends u Sample Projects

23. Affective Tutors -Understand and react to learners emotions ITS Adaptive Open Learning Environments -Support learning via free exploratin of virtual worlds, interactive simulations and educational games Meta-Cognitive Tutors Scaffold acquisition of learning and reasoning (meta-cognitive) skills Decision theoretic support for Analogical Problem Solving

24. u Students find it helpful to refer to examples in the early stages of problem solving (e.g. Reed and Bolstad, 1991)examples u But the effectiveness of APS is mediated by two meta- cognitive skills (Vanlehn 1998) Adaptive Support for Analogical Problem solving Analogical Problem Solving (APS)

25. 25 Sample Physics Example a) Problem Windowb) Example Window

26. Relevant Meta-Cognitive Skills 1. Min-Analogy 2. Self-explanation: tendency to elaborate and clarify to oneself given instructional material (Chi et al, ’89) discover fill Knowledge Gaps minimize copying from examples can be used to learn new domain principles

27. Impact of Student Characteristics u Unfortunately, some students lack these skills (e.g. Vanlehn 1999) –Maximize copying –Don’t learn new domain principles via SE u Furthermore, lack of domain expertise leads to selection of inappropriate examples (e.g. Novick 1988)

28. Impact of Problem/Example Similarity u These effects are mediated by student existing knowledge and meta-cognitive skills, e.g. Student with tendency for min-analogy and SE may still benefit from a very similar example LowProblem-Solving Success & Learning High XX Problem-Solving Success & Learning √ X

29. Adaptive support for Example Studying The EA (Example Analogy) Coach (Muldner and Conati, User Modeling 2005) u Suggests examples that aid both problem solving success and learning - By supporting min-analogy and SE u Provides interface scaffolding to help learners use them effectively (demo)demo

30. EA-Coach Architecture Coach Student Model Interface Knowledge Base Problem Specification Example Pool Problem Pool Solver Expected Utility Calculation Example-Selection Mechanism Solution Graph Dynamic Bayesian network evolution of student knowledge and studying behaviour given student interface actions how intermediate solution steps derive from physics rules and previous steps

31. u Done for every example known by the EA-Coach Select example with Maximum Expected Utility Decision Theoretic Approach to Example Selection Prediction of learning & problem solving success Example Solution Simulation of problem solving Multi-attribute Utility Function gives expected utility of example based on these predictions Steps Similarity Probabilistic Student Model -Knowledge of physics rules -Tendency for SE and Min-analogy Problem Solution

32. Evaluation (Muldner and Conati, IJCAI 2007) u 16 participants solved problems with the EA-Coach u Within subjects design u Conditions Adaptive: examples selected via the decision-theoretic mechanism Static: example pre-selected for each problem as the most similar in the available pool  standard approach currently used in ITS (e.g., Weber 1996)

33. Results u Learning Adaptive condition generated significantly more self- explanations and fewer copy episodes: better learning u At no cost for problem solving success No significant differences in problem solving success between conditions The adaptive condition on average made more errors and took longer to solve problems Bi-product of learning

34. Discussion u Encouraging evidence that our proposed decision- theoretic approach to example selection triggers the desired behaviors u Future work More proactive adaptive interventions Eye-tracking instead of masking interface to capture student reasoning

35. Eye Tracking and Self-explanation u We have already investigated the value of eye-tracking information to monitor student self-explanation u Domain: interactive simulations for understanding mathematical functions

36. Sample Activity

37. User Model (Bunt, Muldner and Conati, ITS2004; Merten and Conati, Knowledge Based Systems 2007) Learning User Model (Dynamic Bayesian Network)  Number and coverage of student actions  Self-explanation of action outcomes  Time between actions  Gaze Shifts in Plot Unit Gaze Shifts in Plot Unit Interaction Behavior Interface actions Input from eye-tracker

39. Results on Accuracy

40. Discussion u Evidence that eye-tracking can support real-time modeling of user reasoning processes u Future work Data mining of eye-tracking + interface actions to uncover other relevant attention patters ( Amershi and Conati, IUI 2007 ) Devise and test adaptive interventions to trigger self- explanation and exploration

41. Affective Tutors -Understand and react to learners emotions Adaptive Open Learning Environments -Support learning via free exploration of virtual worlds, interactive simulations and educational games Meta-Cognitive Tutors Scaffold acquisition of learning and reasoning (meta-cognitive) skills ITS Modeling Student Affect in Educational Games

42. Educational Games u Educational systems designed to teach via game-like activities u Usually generate high level of emotional engagement and motivation. u Still little evidence on pedagogical effectiveness (e.g. Vogel 2004, Van Eck 2007) –Often possible to play well without reasoning about the target instructional domain

43. Example: The Prime Climb Educational Game Designed by EGEMS group at UBC to teach number factorization to students in 6 th and 7 th grade (11 and 12 year old)

44. Crucial to model student affect in addition to learning Our Solution Emotionally Intelligent Pedagogical Agents that u Monitor how students learn from a game u Generate tailored interventions to help students learn… u …while maintaining a high level of student emotional engagement

45. Initial Prime Climb Pedagogical Agent (Conati and Zhao, Intelligent User Interfaces 2004) u Answers to students’ help requests u Provides unsolicited hints –Both after correct and incorrect moves u Based only on a probabilistic model of student’s knowledge Press help if you need help

46. Hints at Incremental Level of Detail

47. What else do we need? u Prime Climb, with the model of student knowledge and the agent, generated better learning than the basic game (Conati and Zhao IUI 2004). u But by taking affect into account, the agent could do even better Decide what to do when the student is upset Decide how to use student’s positive states to further improve learning

48. Possible actions + predicted effect on learning and affect Long Term Goal Model of Student Knowledge Select actions to optimize balance between learning and emotional engagement Model of Student Emotional State A decision-theoretic Emotionally Intelligent Agent for Prime Climb

49. How to Assess Emotions? u Emotions can be assessed by –Reasoning about possible causes (i.e. the interface keeps interrupting the user, so she is probably frustrated) –Looking at the user’s reactions

50. But Things are not Always that Easy u The mapping between emotions, their causes and their effects can be highly ambiguous –Diffent people react differently to similar events –How much emotions are shown may depend on culture, personality, current circumstances u Very hard to build models of user affect

51. Challenge u Difficulty of modeling affect in edu-games enhanced by the fact that players often experience –Multiple emotions –Possibly overlapping –Rapidly changing u For instance: –Happy with a successful move but upset with the agent who tells them to reflect about it –Ashamed immediately after because of a bad fall

52. Previous Approaches u Reduce uncertainty by modeling –one relevant emotion in a restricted situation (e.g., Healy and Picard, 2000; Hudlicka and McNeese, 2002) –only intensity and valence of emotional arousal (e.g. Prendinger 2005) u Model longer term, mutually exclusive emotions like boredom, frustration, flow (D’Mello et al 2008, Arroyo et al 2009) u Not sufficient to react promptly to the more instantaneous, possibly overlapping emotions observed in Prime Climb

53. Our solution u Base emotion assessment on information on both causes and effects of emotional reaction u Probabilistically combined in a Dynamic Bayesian Network

54. Player Reactions The Prime Climb Affective Model Predictive Assessment Emotional State Game-based Causes Based on the OCC Theory of Emotions (Ortony Clore and Collins, 1998) Diagnistic Assessment

55. OCC Theory Action: OUTCOME Joy/Distress Admiration/Reproach Pride/Shame Joy/Distress Goals Action OUTCOME Goals e.g., Have Fun Avoid Falling Defines 22 different emotions are the result of evaluating (appraising) the current circumstances with respect of one’s goals

56. The Predictive Part of the Model Predictive Assessment Emotional State Game-based Causes Infers player goals at runtime from personality traits and interaction events Have Fun Learn Math Succeed by Myself Want Help Beat Partner Has information to assess which game states satisfy/dissatisfy the goals 6 of the 22 emotions in the OCC theory Joy/Regret toward the game Admiration/Reproach toward the agent Pride/Shame toward oneself Conati and MacLaren 2009, J. of User Modeling and User-Adaptive Interaction

57. DDN for the OCC Theory Goals Satisfied Goals Personality Interaction Patterns Joy/ Regret titi User Action Outcome Pride/ Shame Agent Action Outcome Goals Satisfied Goals Personality t i+1 Joy/ Regret Admiration/ Reproach u All the relations in the model have been defined via empirical studies and observations (Conati and MacLaren 2005)

58. u User study with 66 students (10 and 11 year of age) (Conati and Maclaren UMUAI 2009) –Pretty good results in capturing emotions towards the game (69.5% accuracy) –Not so good for emotions towards the agents, specifically in recognizing regret (47% accuracy) u Due to the model’s inability to capture goal shifts from Succeed-by-myself to Want-help at difficult times –Because goals are modeled as static Evaluation of the Diagnostic Model

59. Player Reactions Adding Diagnostic Information (Conati and Mclaren, User Modeling 2009) Predictive Assessment Emotional State Game-based Causes Diagnostic Assessment

60. Diagnostic Assessment u Long term goal: integrate multiple detectors: physiological sensors, face and intonation recognition u Current focus: Electromyogram (EMG) –Applied on the forehead detects activity in the corrugator muscle –greater activity is a reliable indicator of negative affect (e.g., Cacioppo 1993) »emotions expressed on demand

61. EMG signal Signal peak in correspondence of a frown

62. EMG Signal The Prime Climb Affective Model Predictive Assessment Emotional State Game-based Causes Infers player goals at runtime (e.g., Have Fun, Learn Math, Avoid Falling…) Has information to assess which game states satisfy/dissatisfy the goals Diagnostic Assessment Includes 6 of the 22 emotions in the OCC theory Joy/Regret toward the game Admiration/Reproach toward the agent Pride/Shame toward oneself Relevant probabilities learned from data from ad-hoc study User BehaviorsOverall Valence

63. Evaluation u Tested with 41 students (from 6 th and 7 th grade) –Periodically reported their emotions during game playing –Model predictions compared against these self-reports

64. Emotional Reports

65. Evaluation u Good results in presence of strong, equally-valenced emotions –Adding EMG significantly improves accuracy u Weaker results in presence of subtler or conflicting emotions u Poorer performance in the presence of contrasting emotions

67. Comparison of Predictive and Complete Model Emotion Accuracy Predictive ModelCombined Model Joy 74.80 79.10 p < 0.05 Distress 53.48 56.7 J/D Combined 64.14 67.9 P = 0.067 Admiration 83.49 81.18 Reproach 39.11 63.02 P < 0.05 A/R Combined 61.3 73.10 P < 0.05 u Encouraging evidence that EMG can help model assessment of individual emotions with clear overall valence

68. Lots of Exciting Future Work u Add more diagnostic elements to improve model accuracy (e.g., more expression recognition, speech/intonation patterns) u Integrate model of affect and model of learning u Create emotionally intelligent agent that takes into account both student affect and learning to decide how to act u Include longer term emotions (boredom, happiness, frustration) u Prove that it works better than agent with no affect!

69. Conclusions u AI has the potential of having a huge impact on society by affecting how people learn and train –Bring benefits of one-to-one tutoring to all u Successful ITS have already been deployed to support problem solving activities u The benefits of AI in education and training can go further, e.g. –Support for life-long learning via promoting meta-cognition –Innovative personalized activities: learning via exploration, learning via game playing –Affect-sensitive tutors u Continuous dialogue with educators and cognitive scientists crucial to do this right

70. Thanks to… Andrea Bunt Heather Maclaren Cristina Merten Kasia Muldner David Ternes u And to you all for your kind attention

71. Evaluation of the Combined Model: Mild/Mixed Valence Data Points Mild ValenceMixed Valence u No improvement over predictive model u One EMG on forehead not adequate for assessing mild emotions –Need to integrate it with other sensors, i.e. heart-rate monitor, EMG on zygomatic muscle

72. Mixed Valence Unless there is strong evidence coming from the causal component Overall Valence Valence Detector Joy/Distress Admiration/ Reproach Evidence from any valence detector forces Valence node to be positive or negative Causal prediction Positive/Negative Valence forces J/D and A/R to have the same valence. Pos/Neg Need to improve goal recognition