LaVonda Brown Georgia Institute of Technology LearnLab Workshop 2012 Carnegie Melon University

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Purpose  Motivation  Background  Design  Hypotheses  Results  Conclusion  Future Work 2

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  To develop a reliable, non-invasive method of monitoring academic engagement within the domain of computer-based education (CBE) 3

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Socially Interactive Robot Tutor (SIRT) ◦ Real-time monitoring of actions to determine level of engagement and understanding ◦ Adapt to educational needs through instructional scaffolding ◦ Engage children by forming a personal relationship  Essential for longevity 4  Platform

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Teachers/tutors are able to observe the student’s engagement in real-time and employ strategies to reengage the student  They are able to determine engagement by following behavioral cues from students  This improves attention, involvement and motivation to learn  Behavioral engagement is often related to the academic achievement of a student 5

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Scales/Surveys ◦ Used to evaluate motivation once the student has completed a system  Electroencephalography (EEG) signal measurements ◦ Able to identify subtle shifts in alertness, attention, and workload in real time  Eye Gaze and Head Pose ◦ Able to determine six user states in an e-learning environment: attentive, full of interest, frustrated/struggling to read, distracted, tired/sleepy, and not paying attention 6

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Eye gaze and head pose will be the baseline  Will use this to develop a novel model of student engagement based on mouse and keyboard events.  Two tests of high and low difficulty  Three event processes will be monitored ◦ Total Time  slow, average, or fast ◦ Response Validity  correct or incorrect ◦ Proper Function Execution  on-task or off-task 7

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Hypothesis 1. The student is engaged if his or her series of events (or combination of events) are classified as: ◦ On-task and correct (regardless of speed) ◦ On-task, slow or average, and incorrect 8

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Eye gaze and head pose will not be an accurate measure of user state/engagement for the high difficulty test.  The use of pencil and paper will create false negatives since eye gaze will be directed towards the paper instead of the computer screen. 9

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Various combinations of the event processes can determine the following about the engaged student: ◦ If the student is on-task and has a series of fast responses with a series of correct answers, the student needs questions of higher difficulty. ◦ If the student is on-task and has a series of slow and/or average responses with a series of correct answers, the student does a great deal of thinking and understands the material. ◦ If the student is on-task and has a series of slow and/or average responses with a series of incorrect answers, the student lacks understanding and needs questions of lesser difficulty 10

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 11

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 12

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 13

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 14

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 15 This chart shows the how often we received each combination of events throughout both tests (S=slow, A=average, F=fast, C=correct, I=incorrect, O=on-task, O’=off-task). The combinations OIF, O’CS, O’CA, O’CF, O’IS, O’IA, O’IF do not occur during this study.

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 16

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  In order for a student to be engaged, he or she must be on-task or choose events that successfully execute functions needed to navigate through the assessment.  However, if a student is on-task, but answers incorrectly, and at a fast pace (OIF), he or she is classified as being disengaged.  If a student is classified as being off-task, we can automatically classify this student as being disengaged (regardless of speed and/or response). 17

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation  Add a survey to the end of the experiment to better determine understanding, engagement, and difficulty. ◦ If the student is on-task and has a series of fast responses with a series of correct answers (OCF), the student may need questions of higher difficulty. ◦ If the student is on-task and has a series of slow responses with a series of correct answers (OCS), the student may understand the material and require more time to think. ◦ If the student is on-task and has a series of slow responses with a series of incorrect answers (OIS), the student may lack understanding and need questions of lesser difficulty.  This additional information will be used in the future to better integrate instructional scaffolding and adaptation with the device. 18

Int’l Conference on Robotics and Automation 2010 Anchorage, Alaska HumAnS Lab. Presentation 19