Design of metadata surrogates in search result interfaces of

learning object repositories:

Linear versus clustered metadata design

Panos Balatsoukas

Anne Morris

Ann O’Brien

Contents

Definition of user-centred metadata Evolution of metadata surrogate design Aim and Objectives of the usability test The META-LOR 1 prototype Methodology Results Conclusions – Recommendations

Future Research

Metadata definitions

‘Data about data’

“Structured data about an object that supports functions associated with the designated object” – (Greenberg, 2005)

Learning object metadata: metadata used for the efficient description of learning objects and the effective support of educational-learning functions related to the described learning objects.

User-centred metadata

ContentPresentation

Learner

Relevance Usability

Technologies

Evolution of metadata presentation and content

Copied from: Marchionini et al (1993)

http://hcil.cs.umd.edu/trs/93-10/93-10.html

Design of metadata surrogates Metadata elements providing access or arranging access to

the resource should follow content related elements such as title, abstract, subject heading or keywords.

Users prefer content related metadata for finding and identifying sources and technical and physical metadata for selecting and obtaining access to the resource.

There is a debate among researchers as to whether metadata surrogates should be displayed in list, tabular, dynamic or category-based format in search result interfaces.

It is suggested that abstracts should contain contextualised information relevant to users’ search query.

Metadata surrogates should not include only topical or subject related information.

Design of learning object metadata in search result interfaces

The need to include a description/abstract of the contents of the learning object in the metadata surrogate;

The use of user-centred metadata terminology and vocabularies; and

The use of clustered rather than linear and information cluttered learning object metadata surrogates.

Aims and objectives

To examine users’ interaction with two different learning object metadata surrogates: 1. a linear metadata surrogate interface, and 2. a clustered metadata surrogate interface.

The objectives of this study were: To investigate the time needed by learners to identify a

relevant learning object, using both interfaces; To study the impact of task complexity on users’

interaction with both interfaces; and To examine learners’ subjective satisfaction for both

interfaces.

Linear metadata surrogate

Pop up box

Pop up box

Clustered metadatasurrogate

Metadata categories

Methodology 1

Usability participants’ profile: 12 postgraduate students in Information and

Computer Science Task List analysis and scenarios:

3 tasks with varying degrees of complexity (Low, Medium and High complexity)

Error rate and Time Observation (Think Aloud protocol)

Background and post test questionnaires Post test interviews

Methodology 2

The three Tasks:

Results of the usability test

Differences in Time

participants performed the three tasks slightly faster using the clustered metadata surrogate interface.

Mean time of 314 secs in the Linear.

Mean time of 301 in the Clustered

Task complexity and Interface

There were no significant differences observed between task complexity and metadata interface design

Subjective satisfaction Subjects were

significantly more satisfied with the clustered metadata surrogate interface.

Mean overall satisfaction for clustered metadata surrogate = 7.8.

Mean overall satisfaction for the linear metadata surrogate interface = 6.3

Qualitative results (1)

Participants (n=10) liked the way metadata was presented in the clustered metadata surrogate interface:

Plausibility and engagement Structure and organisation of information Two participants preferred the linear interface

(prior familiarisation; not meaningful metadata clustering).

Qualitative results (2)

Subjects liked the use of most of the general and technical category metadata (e.g. title, subject, description, format, identifier)

Few of the educational related metadata were perceived as useful (e.g. Audience, interactivity level, difficulty)

Qualitative results (3) Subjects did not like the inclusion of many metadata

elements and lengthy metadata surrogates.

Some participants (n=4) would like to select the metadata elements to be displayed in the surrogate.

Other metadata elements: Relation metadata People’s comments The time it takes for a learning object to be

downloaded/accessed Accessibility needs Information about the quality of learning objects

Conclusions - Recommendations

The provision for alternative displays of metadata surrogates, for example, both in linear and clustered forms.

The design of adaptive interfaces that present the content and format of metadata surrogates according to learners’ needs.

The use of pop up boxes for documenting and presenting the meaning of learning object metadata elements to users.

Need to extend the LOM standard with new metadata elements, such as, ‘the time it takes for a learning object to be downloaded’, ‘accessibility needs’ information, as well as ‘information about the ‘quality’ of a learning object.

Research in progress… Usability assessment of three learning object repositories

(MERLOT, ARIADNE Knowledge Pool and JORUM/UK).

Survey of students’ perceptions of the importance of learning object metadata elements.

User study on the criteria students employ to judge the relevance of learning objects.

Development of Heuristic evaluation checklist for the evaluation of metadata surrogates in search and search result interfaces.

Development of guidelines and recommendations for the design of learning object metadata schemas and Learning Object Repositories.