Learning Companion Project Emotions and Their External Manifestations in the Learning Process by Selene Mota Leila Toplic

Learning Companion - Goal: facilitating student’s efforts at learning - Tasks: Learning Companion watches and responds to student’s affective state - “watching especially for signs of frustration and boredom that may precede quitting, for signs of curiosity or interest that tend to indicate active exploration, and for signs of enjoyment and mastery, which might indicate a successful learning experience.”

Disappointment, Puzzlement, Negative Affect Positive Un Learning Constructive Learning Frustration, Discard, Misconception Hopefulness, Fresh Research Disappointment, Puzzlement, Confusion Awe, Satisfaction, Curiosity IV III I II

Objective of our research To identify facial expressions and posture changes, which indicate that a person is on task or off task.

5 MIT graduate students, 4 men and one woman Subjects 5 MIT graduate students, 4 men and one woman

System ? Participant VCR Camera PIC Serial Port 1 Computer 1 Galvactivator Information Recorder ? Serial Port 2 Participant Computer 2 Pressure Sensor on Chair

Procedure: experiment 1) Introduction: Each participant was given a brief introduction to the research and its rules 2) Test: Participants were asked to play the game for 15 minutes. Data collected during the experiment: video footage of the participants’ facial expressions, skin conductivity and posture changes.

Procedure: post-experiment 3) Post-experiment interview to gather the data on participants’ emotional experiences during the experiment. Sample questions asked: - Did you like playing this game? Why? - Do you play computer games in general? - How did you learn to play this game? - Was the game interesting? Confusing? Boring? - Did the software provide enough support?

Procedure: post-experiment 4) Coding: Participants were asked to watch the video of their experiment and label their facial expressions they had displayed during the experiments with the following four labels: frustration, interest, confusion, and boredom. 5) Coding: Two teachers were asked to watch the same video segments and label participants’ facial expressions.

Data to be compared: - post-experiment interviews - participants’ coding (facial expressions & posture) - teachers’ coding (facial expressions & posture) - SCR - posture

Time Line Skin 15 r.p.s aprox. Video Capture 30 fps Information Capture 1 fps Sensor Chair 4 fps

Players Labeling

Skin Conductivity

Skin Conductivity

Skin Conductivity

Player 3

610-650 sec

Confusion Frustration Boredom Interest

Frustration Confusion Boredom Interest

Observations and Recommendations: Conduct the next testing with children (age 11+) Change labeling on the coding window in ISIS It is very important to focus both on participants' facial expressions and posture changes because some peoples bodily movements to express their emotions All participants indicate that after succeeding in their tasks (i.e. solving a puzzle) and experiencing satisfaction for that for a short period of time, they go into the relaxation state right after that and before moving onto the next task Curiosity doesn't belong only to the first quadrant. Curiosity might mean two things: 1) being curious about the things encountered during the same research; 2) being curious before (and after) starting a fresh research about the things we might encounter in this new episode Proposing new graph where: interest is present at all times during the game and it can be either positive or negative, high or low. Boredom is in a shape of a half-ring placed right outside of the negative portion of the circle (II & III quadrant).

Discussion