Designing and Evaluating a

Contextual Mobile Application

to Support Situated Learning

Abeer Alnuaim, PhD

Abalnuaim@ksu.edu.sa

Testing

Effectiveness of the App (HCI) In-Context Evaluation (DUE)

Development Methodology

Four Phases

HCI Teaching

Motivation Aim

HCI Teaching

Human-computer Interaction studies the way people interact with computers in a particular context and

evaluates the extent to which these computer based

systems are, or are not, designed for successful interaction.

Teaching interaction design is a challenging task.

HCI Teaching

Applying empathic design strategies when designing

aids in developing a product that pleases the user

(McDonagh and Thomas, 2010) .

Thus, immersing students into real would environments

to gather requirements could generate empathy and

thus designing a product that is related to the users’

needs.

Motivation & Aim• This research emerged from seeking to identify ways

of getting Human-Computer Interaction Design students into real world environments.

• Mobile learning applications can provide contextual information that could help students stay focussed on the purpose and outcome of the activity, rather than being distracted by the process.

• Mobile Learning is all about personalisation.

• Aim: To investigate the process of designing a mobile learning application in a contextual mobile learning model.

A contextual mobile learning model

HCI’s Students’ Coursework

It involved designing a GUI for a touch-screen based kiosk to be installed in the university’s cafeteria to offer support to students and staff, helping them make the right meal choices.

The assignment was structured as a group project involving three or four students, where the initial work consisted of requirements gathering and analysis to produce a set of artefacts such as a PACT analysis personas and scenarios, and a set of functional and non-functional requirements. The assignment deliverable was an in-class presentation of their work.

Development Methodology

The development process has been identified in

phases. These Phases were derived according to the

User-Centred Design Process (UCD) in the field of HCI.

In UCD. The phases for this study were as follows:

Phase One: Requirements Gathering

Phase One Key Findings:• Students lose focus on the purpose of tasks when away

from classroom. They may get distracted by their

surroundings and miss out key elements.

• Some students have been found to struggle in analysing

their findings and specifically in using their findings to

develop new ideas.

• Students care about their privacy and would not easily

compromise it.

• Android smartphones are poplar among students.

Phase 3 &4 : Design and Evaluation

• Designing a contextual mobile learning application

requires consideration of a number of issues. These

include students’ different learning styles and

preferences, the location’s characteristics and its

physical and psychological effect on the user, as well as

the appropriateness of the location-specific content.

• An iterative design approach was followed.

Detailed activities within the

iterative development process

Iteration 5

Evaluating the effectivness

Vavoula and Sharples’s (2009) three-level framework for

evaluating mobile learning was used.

Participants• There were 55 students enrolled in the HCI module,

seven females and 48 males.

• Students were allowed to form self-selecting groups of 3

or 4.

• This resulted in 17 groups; however, only 16 groups

presented their work.

• 3 groups borrowed the university’s HTC desire phones,

• 10 groups used their own phones

• While 3 groups neither borrowed or used their own.

HCI’s Deployment Methodology

Presentation Results

Most groups did very well in their assignment.

The average class mark for this cohort for the

elements that were supported by use of the app is

77.32%, which is above the cohort average mark

for the assignment as a whole which is 66.15%.

Questionnaire

• 23 students filled in and returned the paper

questionnaire.

• The mean SUS score based on all the

responses was 69; this is above the average

SUS score.

Pedagogical UsabilityStatements

N

Mean

Scores

The app helped me in my

observation

23 4.17

The app gave me hints on

what to look for

22 4.36

The app helped me

organise my ideas

22 4.18

It was helpful to have a

space for note taking

23 2.43

The app helped our group

members to share ideas

and notes

23 3.74

The Forum (Blog) within

the app was useful

18 3.00

It was useful to track my

progress through Profile

21 3.71

The app helped me

develop ideas for PACT

22 4.36

Findings• overall engagement with the observational

requirements activity, as well as the quality of the

students’ insights and requirements emerging as a

consequence of their observations supported by

the use of the sLearn, indicates that the app has

had an effective impact on student learning.

• The approach used in immersing the student in real

environments, can help them view the situation

from the perspective of the user, hence helping

them to generate empathy.

FindingsChallenges in designing and evaluating mobile

learning in this context:

• group dynamics

• ownership of mobile devices

• willingness and motivation to engage or try

something new

• intrinsic ability

• the novelty value of the app, and the usability of

the interface versus the helpfulness of the content.

In-Context Evaluation

To accurately assess the use and value of this mobile

learning app for higher education students, it was

necessary to conduct in-situ evaluations in the

environment of its intended users.

Moreover, it is critical to understand what might

influence the usability and the user experience of this

app in such a busy environment.

Participants• MSc students enrolled in the ‘Designing the User

Experience’ Module

• They are familiar with concepts of HCI.

• 7 out of 30 students participated in the evaluation,

five males and two females.

ObservationsResults

• Hesitation or confusion in general (At the

beginning).

• Confused whether to save the notes first or post

then save.

• Did not know how to remove the keyboard.

• Conscious that staff will notice that he/she is not

here for buying food.

InterviewTheme One: General usability

1. Instructions: more explicit instructions should be

available to students at the beginning.

2. Redesign: all participants prefer that under each

prompt there should be a text box to write their

notes and observations.

Theme Two: The Context

Personality and self-consciousness: 28.57% of the

participants did not feel comfortable looking around

at people and writing on the phone.

DiscussionsThe results of the individual evaluations were

considered as a whole, then they were grouped into

three main overlapping categories relating to:

(1) Design and GUI of the app,

(2) Usability, User experience and Students’

Perspectives, and

(3) Designing and deploying a Blended Learning

Model.

Guidelines for implementing a mobile application

for situated learning activities in HE

Guideline

Accessible from the learner’s mobile

device- Multi platform compatible

Suitable Contextual Content

Provide Independent Choices

Multimodal Interaction

Collaborative facility

Clear Instructions

What is next?

References• Alnuaim, A. (2015) Designing and evaluating a

contextual mobile learning application to support situated learning. PhD, University of the West of England.

• MCDONAGH, D. and THOMAS, J., 2010. Disability relevant design: Empathic design strategies supporting more effective new product design outcomes. The Design Journal, 13(2), pp. 180-198.

• VAVOULA, G. and SHARPLES, M., 2009. Meeting the Challenges in Evaluating Mobile Learning: A 3-Level Evaluation Framework. International Journal of Mobile and Blended Learning, 1(2), pp. 54-75, Copyright, IGI Global. Reprinted with permission of the publisher.