The role of eye tracking in usability evaluation of LMS in ODL context Mr Sam Ssemugabi Ms Jabulisiwe Mabila (Professor Helene Gelderblom) College of Science Engineering and Technology University of South Africa School of Computing

Technology has been a part of our everyday environment Introduction - Eye Tracking and Usability simplifies and complicates our life. It separates us and - LMS & ODL brings us closer together. But even though we interact with technology everyday, we easily forget that technology products are made by people, and that someone, somewhere should get the credit when technology works well for us or get the blame when it doesn’t.

(Garrett James, 2003: 7)

The role of eye tracking in usability evaluation of LMS in ODL context Introduction - Eye Tracking and Usability - LMS & ODL 2. Problem statement 2.

Objective of the study 3. Methodology 3.

Tasks 4. Tasks 4.

Results 5. Results 5.

Conclusion 6. Conclusion

The role of eye tracking in usability evaluation of LMS in ODL context Eye tracking is the technique whereby an individual’s eye movements are measured so that the researcher knows both where a person is looking at, at any given time and the sequence in which the eyes are shifting from one location to another .

(Poole and Ball, 2006) www.company.com

Eye tracking metrics

•

Heat maps

•

Fixations

•

Saccades

•

Eye gaze duration

•

Area of interest scan path (AOI)

•

www.company.com

Nielson, 2006

The role of eye tracking in usability evaluation of LMS in ODL context Usability defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness , in a specified efficiency and context of use.

satisfaction (ISO 9241-11, 1998) www.company.com

The ODL Context Diversity of student profile in ODL institutions

•

Young and old

•

Employed and unemployed • Rural areas and urban areas Diverse student profile - widely varying ages, experience, backgrounds - language, location

Learning Management System (LMS) A LMS software application includes programs for e-learning, administration, documentation, tracking, and reporting, training programs and online events (Ellis, 2009).

Usability assessment of HCI

• •

Include: Expert evaluation Heuristic evaluation

• •

Walkthrough Observation

• •

Retrospective self report (questionnaires, think aloud) (Holzinger, 2005) Other techniques – Eye tracking

Why?

The role of eye tracking in usability evaluation of LMS in ODL context 1. The role of eye tracking 2. Usability assessment LMS 3. ODL context – diversity of student population

Who?

Participants

• •

First year Unisa students Registered for End User Computing (EUC131T)

Methodology

• • • •

Instruments Tobii Eye Tracker 1750 Video recording Questionnaire Observations www.company.com

Tobii 1750 eye tracker Unisa , Pretoria School of Computing Usability Laboratory

) Task - Objective

• • •

Navigation Reading Text input

www.unisa.ac.za

www.company.com

https://my.unisa.ac.za/portal/

The Tasks Task 1: Starting from myUnisa homepage, go to myUnisa (use icon or drop down menu) Task 2: Find ‘Claim myLife e-mail’ on myUnisa screen Task 3: Set-up myLife account (Input of student number; Surname, names, date of birth, ID number or Passport/Foreign ID (for foreign students); Click on ‘acknowledge guidelines’ Task 4: Join myUnisa Task 5: Activate your myUnisa password www.company.com

Discussion of Results Task 1 Claim myLife e-mail account

• • •

Task completion – 67% Assistance needed – 75% Errors made – 92 % Task 2: Join myUnisa

• • •

Task completion – 83% Assistance needed – 42% Errors made – 58% www.company.com

Interpretation www.company.com

Interpretation Problem 1: Terminology Difference between “Claim myLife e-mail account” and “Join myUnisa” not clear “Registration” was selected by two users which gives a completely different screen to “join myUnisa ” Problem 2: Process 2 separate processes, passwords mixed up Problem3: Instructions Transition difficulties, moving from “Claim myLife ” to “Join myUnisa”. Participants did not knowing what to do next Problem 4: Varying computer literacy levels www.company.com

The Tasks Time to complete tasks

• •

Participants who used computers regularly or had studied computing required an average of 25 minutes to complete the tasks Participants without computing experience or who had not studied computing previously required on average 54 minutes to complete the tasks www.company.com

Conclusion 1.

Diversity in age, experience, background, and location of ODL students necessitate rigorous usability evaluation 2.

Recommendations can be made regarding the interface design based on eye tracking data, and 3.

The interpretation of the information collected through eye tracking should be used together with other usability evaluation methods – observation, heuristics, questionnaire www.company.com