Embed Size (px)
Citation preview
http://userlab.open.ac.uk/
http://userlab.open.ac.uk/
Andrew Brasher
Andrew Brasher, Patrick McAndrew
Userlab, IET, Open University
Human-Generated Learning Object Metadata
2
Summary
• Examine factors affecting quality of human generated metadata
• How can ontologies and systems which exploit them help?
• Conclusions
3
Quality of human generated metadata
• Often neither complete nor consistent
• Factors which influence the quality of human produced metadata :– Motivation of the producers– Accuracy– Consistency
Currier, S., et al (2004) Chan, L.M (1989)
4
Socio-cognitive engineering
Sharples et al. (2002)
5
In this talk…….
• An ontology is a formal explicit specification of a shared conceptualisation (Gruber 1993)
but….
• Issues with langauge dependence (Guarino 1998)
6
An aim of metadata
• To deliver right resources, to right people, in the right place, at the right time to aid these people achieve their learning goals
Content
Collaborative environment
People
7
Implications
• To deliver appropriate learning objects implies descriptions of – Temporal, spatial context– Social context (including descriptions of people)
– Learning objects– Devices– Delivery infrastructure
• and processes to select and deliver the right resources
8
For generation of LO metadata:
• Intrinsic
• Extrinsic
2 sources:contained within the resource itself, are a necessary part of the resource itself. E.g.: format of a resource, and title of a resource.
not contained within the resource itself. E.g.: personal or community views about the expected use of the resource.
9
Generation of metadata
MetadataSource Process
Intrinsic sources. E.g.: format of a resource, and title of a resource.
Extrinsic sources. E.g.: personal or community views about the expected use of the resource.
Metadata?Process?
10
In practice, there are two types of sources which require human intervention at point of production to create metadata descriptors:• extrinsic sources
and
• most intrinsic sources within non-textual resources.
Human intervention
e.g. expected use
e.g. sound, movies, multimedia
11
1) Interactivity Type; 2) Learning Resource Type; 3) Interactivity Level; 4) Semantic Density; 5) Intended End User Role;6) Context; 7) Typical Age Range; 8) Difficulty; 9) Typical Learning Time; 10) Description; 11) Language.
Most intrinsic sources within non-textual resources
Extrinsic sources e.g. expected use
Educational category of the IEEE Learning Object Metadata
1) Interactivity Type; 2) Learning Resource Type; 3) Interactivity Level; 4) Semantic Density; 5) Intended End User Role;6) Context; 7) Typical Age Range; 8) Difficulty; 9) Typical Learning Time; 10) Description; 11) Language.
2 sources requiring human generation of metadata
12
Example
• Difficulty“How hard it is to work with or through this learning object for the typical intended target audience.”
“NOTE—The “typical target audience” can be characterized by data elements5.6:Educational.Context and5.7:Educational.TypicalAgeRange.”
very easyeasymediumdifficultvery difficult
IEEE, (2002).
13
Example: “Learning in the connected economy”
14
Example: “Learning in the connected economy”
15
How to create difficulty meatadata?
• By authors using structured vocabularies
• Task analysis
16
Metadata creation
Kabel et al., 2003 VDEX, 2004
17
Example LO
18
Example
19
Categorisation
“Typical learning time” / hours “Difficulty”
< 1 Very easy
1 <= “Typical learning time” < 3 Easy
3 <= “Typical learning time” < 4 Medium
“Typical learning time” > 4 Difficult
20
Multiple Contexts
• Previous idea works for a single context
• What about multiple contexts?– Who?– How could it be exploited?
21
Goal
• To enable a system to compare the difficulty of LO’s on the same topic– designed for different ‘target audiences’
• Looking for a general ‘solution’
22
context2 (museum)
Metadata museum
Metadata degree
context1 (Art History degree)
easyeasy
Target audience
Target audience
relationship
LO degreeLO museum
23
• Art history degree course team create difficulty metadata
• Museum guide team create difficulty metadata
• Relationship metadata created by ‘managers’
Metadata CreationAutomatic generation?Task analysis,
endemic motivation
Task analysis, endemic motivation
24
context2 (museum)
Metadata museum
Metadata degree
context1 (Art History degree)
easyeasy
Target audience
Target audience
relationship
LO degreeLO museum
ContextRelationship classThe ContextRelationship class describes relationships between assignments of difficulty in different contexts. An example of how data contained within an instance should be interpreted is: "For students in the Museum context, the assignments of difficulty made for students in the Art History degree context are more difficult."
25
Ontology• Assignments of difficulty made in this context context1 are perceived as “more difficult” than assignments of difficulty made in this context context2
• Assignments of difficulty made in this context context1 are perceived as “less difficult” than assignments of difficulty made in this context context2
• Assignments of difficulty made in this context context1 are perceived as “as difficult” as assignments of difficulty made in this context context2
26
Multiple contexts
context2 (museum)
context1 (Art History degree)
easyeasy
Possibility of disagreements?
Possibility of refinements?
“More difficult than”
“Less difficult than” “Less difficult than”context3 (Secondary school)
easy
27
Conclusions
• Consider and exploit endemic motivation in the system design
• Single context: LOM schema sufficient(?)
• Multiple contexts: more complex
28
Future plans
• More complex representation needed– Ontology / OWL
• Other domains– User Profiles
• Task modelMetadata creation task model- Ontologies are one of the many tools that can be used
• Forget this: it’s never going to work!
29
