The role of eye tracking in usability evaluation of LMS in ODL context Mr Sam Ssemugabi Ms Jabulisiwe Mabila (Professor Helene Gelderblom) College of Science Engineering and Technology University of South Africa School of Computing

1.Introduction - Eye Tracking and Usability - LMS & ODL Technology has been a part of our everyday environment for generations. It empowers us and frustrates us, it simplifies and complicates our life. It separates us and brings us closer together. But even though we interact with technology everyday, we easily forget that technology products are made by people, and that someone, somewhere should get the credit when technology works well for us or get the blame when it doesn’t. (Garrett James, 2003: 7)

The role of eye tracking in usability evaluation of LMS in ODL context 1.Introduction - Eye Tracking and Usability - LMS & ODL 2.Objective of the study 3.Tasks 4.Results 5.Conclusion 1.Introduction 2.Problem statement 3.Methodology 4.Tasks 5.Results 6.Conclusion

Eye tracking is the technique whereby an individual’s eye movements are measured so that the researcher knows both where a person is looking at, at any given time and the sequence in which the eyes are shifting from one location to another. (Poole and Ball, 2006) The role of eye tracking in usability evaluation of LMS in ODL context

Eye tracking metrics Heat maps Fixations Saccades Eye gaze duration Area of interest scan path (AOI) Nielson, 2006

Usability defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. (ISO , 1998) The role of eye tracking in usability evaluation of LMS in ODL context

The ODL Context Diversity of student profile in ODL institutions Young and old Employed and unemployed Rural areas and urban areas Diverse student profile - widely varying ages, experience, backgrounds - language, location

Learning Management System (LMS) A LMS software application includes programs for e-learning, administration, documentation, tracking, and reporting, training programs and online events (Ellis, 2009).

Usability assessment of HCI Include: Expert evaluation Heuristic evaluation Walkthrough Observation Retrospective self report (questionnaires, think aloud) (Holzinger, 2005) Other techniques – Eye tracking

Why? 1.The role of eye tracking 2.Usability assessment LMS 3.ODL context – diversity of student population The role of eye tracking in usability evaluation of LMS in ODL context

Participants First year Unisa students Registered for End User Computing (EUC131T) Who?

Tobii 1750 eye tracker Unisa, Pretoria School of Computing Usability Laboratory Instruments Tobii Eye Tracker 1750 Video recording Questionnaire Observations Methodology

) Task - Objective Navigation Reading Text input

The Tasks Task 1: Starting from myUnisa homepage, go to myUnisa (use icon or drop down menu) Task 2: Find ‘Claim myLife ’ on myUnisa screen Task 3: Set-up myLife account (Input of student number; Surname, names, date of birth, ID number or Passport/Foreign ID (for foreign students); Click on ‘acknowledge guidelines’ Task 4: Join myUnisa Task 5: Activate your myUnisa password

Discussion of Results Task 1 Claim myLife account Task completion – 67% Assistance needed – 75% Errors made – 92 % Task 2: Join myUnisa Task completion – 83% Assistance needed – 42% Errors made – 58%

Interpretation

Interpretation Problem 1: Terminology Difference between “Claim myLife account” and “Join myUnisa” not clear “Registration” was selected by two users which gives a completely different screen to “join myUnisa” Problem 2: Process 2 separate processes, passwords mixed up Problem3: Instructions Transition difficulties, moving from “Claim myLife” to “Join myUnisa”. Participants did not knowing what to do next Problem 4: Varying computer literacy levels

The Tasks Time to complete tasks Participants who used computers regularly or had studied computing required an average of 25 minutes to complete the tasks Participants without computing experience or who had not studied computing previously required on average 54 minutes to complete the tasks

Conclusion 1.Diversity in age, experience, background, and location of ODL students necessitate rigorous usability evaluation 2.Recommendations can be made regarding the interface design based on eye tracking data, and 3.The interpretation of the information collected through eye tracking should be used together with other usability evaluation methods – observation, heuristics, questionnaire