1. Learning Companion Project
Emotions and Their External Manifestations in the Learning Process by
Selene Mota
Leila Toplic

2. Learning Companion - Goal: facilitating student’s efforts at learning
- Tasks: Learning Companion watches and responds to student’s affective state - “watching especially for signs of frustration and boredom that may precede quitting, for signs of curiosity or interest that tend to indicate active exploration, and for signs of enjoyment and mastery, which might indicate a successful learning experience.”

3. Disappointment, Puzzlement,
Negative
Affect
Positive
Un Learning
Constructive Learning
Frustration,
Discard,
Misconception
Hopefulness,
Fresh Research
Disappointment, Puzzlement,
Confusion
Awe,
Satisfaction,
Curiosity IV
III I II

4. Objective of our research
To identify facial expressions and posture changes, which indicate that a person is on task or off task.

5. 5 MIT graduate students, 4 men and one woman
Subjects
5 MIT graduate students, 4 men and one woman

6. System ? Participant VCR Camera PIC Serial Port 1 Computer 1
Galvactivator
Information Recorder ?
Serial Port 2
Participant
Computer 2
Pressure Sensor on Chair

7. Procedure: experiment
1) Introduction: Each participant was given a brief introduction to the research and its rules
2) Test: Participants were asked to play the game for 15 minutes. Data collected during the experiment: video footage of the participants’ facial expressions, skin conductivity and posture changes.

8. Procedure: post-experiment
3) Post-experiment interview to gather the data on
participants’ emotional experiences during the
experiment.
Sample questions asked:
- Did you like playing this game? Why?
- Do you play computer games in general?
- How did you learn to play this game?
- Was the game interesting? Confusing? Boring?
- Did the software provide enough support?

9. Procedure: post-experiment
4) Coding: Participants were asked to watch the video of their experiment and label their facial
expressions they had displayed during the
experiments with the following four labels:
frustration, interest, confusion, and boredom.
5) Coding: Two teachers were asked to watch the same video segments and label participants’ facial expressions.

10. Data to be compared: - post-experiment interviews
- participants’ coding (facial expressions & posture)
- teachers’ coding (facial expressions & posture)
- SCR
- posture

11. Time Line Skin 15 r.p.s aprox. Video Capture 30 fps
Information Capture fps
Sensor Chair fps

12. Players Labeling

13. Skin Conductivity

14. Skin Conductivity

15. Skin Conductivity

16. Player 3

18. sec

23. Confusion
Frustration
Boredom
Interest

24. Frustration
Confusion
Boredom
Interest

25. Observations and Recommendations:
Conduct the next testing with children (age 11+)
Change labeling on the coding window in ISIS
It is very important to focus both on participants' facial expressions and posture changes because some peoples bodily movements to express their emotions
All participants indicate that after succeeding in their tasks (i.e. solving a puzzle) and experiencing satisfaction for that for a short period of time, they go into the relaxation state right after that and before moving onto the next task
Curiosity doesn't belong only to the first quadrant. Curiosity might mean two things: 1) being curious about the things encountered during the same research; 2) being curious before (and after) starting a fresh research about the things we might encounter in this new episode
Proposing new graph where: interest is present at all times during the game and it can be either positive or negative, high or low. Boredom is in a shape of a half-ring placed right outside of the negative portion of the circle (II & III quadrant).

26. Discussion