The role of eye tracking in usability evaluation of LMS in ODL context

Mr Sam Ssemugabi Ms Jabulisiwe Mabila

(Professor Helene Gelderblom)College of Science Engineering and Technology

University of South AfricaSchool of Computing

1. Introduction - Eye Tracking and Usability

- LMS & ODL

Technology has been a part of our everyday environment for generations. It empowers us and frustrates us, it simplifies and complicates our life. It separates us andbrings us closer together. But even though we interact

withtechnology everyday, we easily forget that technology products are made by people, and that someone, somewhere should get the credit when technology works well for us or get the blame when it doesn’t.

(Garrett James, 2003: 7)

The role of eye tracking in usability evaluation of LMS in ODL context

1. Introduction - Eye Tracking and Usability

- LMS & ODL

2. Objective of the study

3. Tasks

4. Results

5. Conclusion

1. Introduction 2. Problem statement

3. Methodology

4. Tasks

5. Results

6. Conclusion

www.company.com

Eye tracking is the technique whereby an individual’s eye movements are measured so that the researcher knows both where a person is looking at, at any given time and the sequence in which the eyes are shifting from one location to another . (Poole and Ball, 2006)

The role of eye tracking in usability evaluation of LMS in ODL context

www.company.com

Eye tracking metrics

•Heat maps •Fixations

•Saccades

•Eye gaze duration

•Area of interest scan path (AOI)

•Nielson, 2006

www.company.com

Usability defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. (ISO 9241-11, 1998)

The role of eye tracking in usability evaluation of LMS in ODL context

The ODL Context

Diversity of student profile in ODL institutions

• Young and old

• Employed and unemployed

• Rural areas and urban areas

Diverse student profile - widely varying ages, experience, backgrounds - language, location

Learning Management System (LMS)

A LMS software application includes programs for e-learning, administration, documentation, tracking, and reporting, training programs and online events (Ellis, 2009).

Usability assessment of HCI

Include:

• Expert evaluation

• Heuristic evaluation

• Walkthrough

• Observation

• Retrospective self report (questionnaires, think aloud) (Holzinger, 2005)• Other techniques – Eye tracking

Why?

1. The role of eye tracking

2. Usability assessment LMS

3. ODL context – diversity of student population

The role of eye tracking in usability evaluation of LMS in ODL context

Participants

• First year Unisa students

• Registered for End User Computing (EUC131T)

Who?

www.company.com

Tobii 1750 eye tracker Unisa , Pretoria

School of Computing Usability Laboratory

Instruments

• Tobii Eye Tracker 1750

• Video recording

• Questionnaire

• Observations

Methodology

www.company.com

)

Task - Objective• Navigation

• Reading

• Text input

www.unisa.ac.za

https://my.unisa.ac.za/portal/

www.company.com

The Tasks

Task 1: Starting from myUnisa homepage, go to myUnisa (use icon or drop down menu)

Task 2: Find ‘Claim myLife e-mail’ on myUnisa screen

Task 3: Set-up myLife account (Input of student number; Surname, names, date of birth, ID number or Passport/Foreign ID (for foreign students); Click on ‘acknowledge guidelines’

Task 4: Join myUnisa

Task 5: Activate your myUnisa password

www.company.com

Discussion of Results

Task 1 Claim myLife e-mail account

• Task completion – 67%• Assistance needed – 75%• Errors made – 92 %

Task 2: Join myUnisa

• Task completion – 83%• Assistance needed – 42%• Errors made – 58%

www.company.com

Interpretation

www.company.com

Interpretation

Problem 1: Terminology

Difference between “Claim myLife e-mail account” and “Join myUnisa” not clear “Registration” was selected by two users which gives a completely different screen to “join myUnisa”

Problem 2: Process

2 separate processes, passwords mixed up

Problem3: Instructions

Transition difficulties, moving from “Claim myLife” to “Join myUnisa”. Participants did not knowing what to do next

Problem 4: Varying computer literacy levels

www.company.com

The Tasks

Time to complete tasks

•Participants who used computers regularly or had studied computing required an average of 25 minutes to complete the tasks

•Participants without computing experience or who had not studied computing previously required on average 54 minutes to complete the tasks

www.company.com

Conclusion

1. Diversity in age, experience, background, and location of ODL students necessitate rigorous usability evaluation

2. Recommendations can be made regarding the interface design based on eye tracking data, and

3. The interpretation of the information collected through eye tracking should be used together with other usability evaluation methods – observation, heuristics, questionnaire