Computer Standards & Interfaces 31 (2009) 1028–1035

Contents lists available at ScienceDirect

Computer Standards & Interfaces

j ourna l homepage: www.e lsev ie r.com/ locate /cs i

A relation metadata extension for SCORM Content Aggregation Model☆

Eric Jui-Lin Lu a,⁎, Chin-Ju Hsieh b

a Department of Management Information Systems, National Chung Hsing University, 250 Kuo-Kuang Rd., Taichung, 402 Taiwan, ROCb Department of Information Management, Chaoyang University of Technology, 168 Gifeng E. Rd., Wufeng, Taichung County, 413 Taiwan, ROC

☆ This research was partially supported by the Nationaunder contract no.: NSC94-2213-E-005-037.⁎ Corresponding author. Fax: +886 4 22857173.

E-mail address: jllu@nchu.edu.tw (E.J.-L. Lu).

0920-5489/$ – see front matter © 2008 Elsevier B.V. Adoi:10.1016/j.csi.2008.09.036

a b s t r a c t

a r t i c l e i n f o

Article history:

To increase the interchang Received 3 April 2008Received in revised form 23 July 2008Accepted 28 September 2008Available online 24 November 2008

Keywords:CAMSCORMOntologyMetadatae-Learning

eability and reusability of learning objects, Advanced Distributed LearningInitiative suggested a set of metadata in SCORM Content Aggregation Model to describe learning objects andexpress relationships between learning objects. However, the suggested relations defined in the metadata ofthe SCORM CAM are limited. To resolve the problem, new relations were proposed by researchers.Unfortunately, some of the relations are redundant and even inappropriate. In addition, the usability of theserelations has never been formally studied. Therefore, in this paper, we summarized and analyzed existingrelations, removed duplicated relations, and developed a new relation metadata extension for SCORM CAM.Also, we surveyed 145 students in attempt to understand whether or not the proposed relations can increasetheir learning effectiveness. The results of the survey showed that learners agreed that the proposed relationsare helpful.

© 2008 Elsevier B.V. All rights reserved.

1. Introduction

Due to the emergence and flourishing of the Internet, thedevelopment of e-learning systems has become an important researchtopic in both academia and industries. Therefore, many learning sys-tems and learning objects(LOs) were developed. One major problemof these LOs is that they cannot be reused among different learningsystems. To resolve the problem, Advanced Distributed LearningInitiative (ADL) [23] developed a reference model called SharableContent Object Reference Model (SCORM) [25].

There are two kinds of LOs defined in SCORM. One is asset, and theother is SCO [19]. Assets are digital media such as text, images, sound,assessment objects, or any other piece of data. Each SCO is composedof assets or other SCOs. Metadata is utilized to describe details of LOsto increase reusability and interoperability.

The metadata defined in SCORM Content Aggregation Model(CAM) is based on IEEE Learning Object Metadata [12]. All metadatafor LOs are classified into nine categories, and one of the categoriesis “RELATION”. A relation in the “RELATION” category is mainly usedto describe a LO and express relationships between LOs. When usedskillfully, a relation is a very useful metadata that can enhance learn-ing effectiveness as well as increase the reusability of LOs. Forexample, as shown in Fig. 1, LOA describes how bubble sort works. Atthe bottom of LOA, there is a figure illustrating how bubble sort worksin steps. With the relations proposed in this paper, one can define the

l Science Council, Taiwan, ROC,

ll rights reserved.

figure as an learning object of type “Illustration”. If the figure is storedin a repository, it can also be easily searched and reused by otherlearners and authors. Additionally, the application of relations can befurther extended. If the author of LOA wishes to provide moreillustrations to help learners, she can easily provide links to otherillustrations such as LOB and LOC. LOs such as LOB and LOC can becreated by the author or other authors as long as they can be accessed.Also, these LOs can be searched and reused if they are stored inrepositories.

As defined in the metadata of SCORM CAM, there are twelvesuggested relations as shown in Table 1 for “RELATION” category.However, these suggested relations can only describe structure-oriented relationships and cannot express semantic relationshipsbetween LOs [20]. Therefore, many relations were proposed [6,8,9,14–17,20–22] in the past. These relations were developedmainly based ontwo major theories. One is instructional design theory (IDT), and theother is rhetorical structure theory (RST). Although these relationscould express semantic relationships between LOs, they were limitedas follows.

First of all, some of these relations are redundant. For example,both “Example” and “Illustration” relations were not only defined inIDT, but also defined in RST. Secondly, there are a few inappropriaterelations. For example, the “Policy” relation defined in IDT describes aset of predefined principles of actions. However, even its creatoradmitted that the definition of policywas not crystal clear [24]. Finally,although these relations can express semantic relationships betweenLOs, whether or not they can help learners have not been formallystudied yet.

In this paper, we summarized and analyzed existing relations,removed duplicated relations, and developed a “RELATION” metadata

Fig. 1. An example application.

1029E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

extension for SCORM CAM. The design of the proposed relationswas based on the following principles: conform to SCORM as muchas possible and can be implemented as easy as possible. To studywhether or not the proposed relations is really helpful for learners,we created learning materials similar to Fig. 1 and used question-naires to survey 145 students. Based on our survey results, the answeris positive. In addition, we discovered an interesting fact that six outof 15 proposed relations are preferable by graduate students thanthat of undergraduate students. Thus, it would be very helpful forauthors to provide relations for different types of learners. It is notedthat, because both the learning materials used in the questionnairesand the surveyed students were in the domain of informationtechnology (IT), the proposed relations might be limited to the ITdomain.

The rest of the paper is organized as follows. First, we brieflydiscussed the relationships between LOs in Section 2. Then wedescribed relations proposed in the past. We also discussed andremoved duplicated relations and proposed a new relation metadataextension for SCORM CAM in Section 3. In Section 4, we described howthe survey was conducted and presented the survey results. Finally,we draw our conclusions and future work in Section 5.

2. Literature review

The relationships between learning resources are classifiedinto three categories: the relationships between learning topics, therelationships between learning topics and learning objects, and therelationships between learning objects.

Table 1The suggested relations in RELATION category.

ispartOf haspart isversionof hasversionisformatof hasformat references isreferencesbyisbasedon isbasisfor requires isrequiredBy

2.1. The relationships between learning topics

The main purpose of describing relationships between learningtopics is to express ordinal structure in learning such as learningsequence and learning navigation. For example, to learn multi-plication, one shall learn addition first. Also, one has to know sub-traction before doing division. Due to its importance, manyresearchers [1,2,7,18] invested a vast amount of efforts in investigatingthe relationships between learning topics in the past. Recently, ADLdeveloped a model called SCORM Sequencing and Navigation toaddress this issue. To avoid redundant work, we focused on the othertwo types of relationships in the rest of the paper.

2.2. The relationships between learning topics and learning objects

The main theory for the relationships between learning topicsand learning objects is instructional design theory (IDT) [3] whichencourages teachers to search for related learning resource and ex-ploit them to satisfy all possible learning needs and tasks to completethe established goal. Based on IDT, Ullrich [8,21,22] constructed aninstructional ontology which is shown in Fig. 2. The instructionalontology includes two kinds of instructional objects. One is funda-mental learning objects which are used to convey the central idea oflearning topics. The other is auxiliary learning objects which are usedto further describe the concepts of fundamental learning objects.According to the definition of the instructional ontology, fundamentallearning objects can be used to express the relationships between thelearning topics and learning objects, and auxiliary learning objects canbe used to express the relationships between learning objects. Thedetails of relations defined in the instructional ontology would bedescribed in Section 3.1.

2.3. The relationships between learning objects

In addition to auxiliary learning objects, some researchersexplored the relationships between learning objects using rhetorical

Fig. 2. The instructional ontology.

1030 E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

structure theory (RST) [4]. In the past, RST is an approach to analyzethe rhetorical structure of article contents. Later, researchers [6,9,14–17] extended the concept to express the relationships betweenlearning objects and proposed a rhetorical–didactic relation. Thedetails of relations defined in the rhetorical–didactic relationwould bedescribed in Section 3.2.

3. The design of a relation metadata extension

In this section, we will design a new relation metadata extensionfor SCORM CAM. The proposed relations are based on previous works:instructional ontology (InO) [20–22,8] and rhetorical–didactic rela-tion (RDR) [6,9,14–17]. Therefore, we will discuss these relations first.In total, there are thirty-three relations.

3.1. The instructional ontology

The instructional ontology includes twenty-three relations, andthey are described as follows:

Definition A definition is used to describe the meaning of a word,a phrase, a symbol, or a proper noun which appears inLOs. For example, e-learning by definition is to engage inlearning activities by utilizing information technology andthe Internet.

Fact A fact is an event that happened. For instance, “Tim Berners-Lee is the creator of HTML” is a fact.

Law A law is either a general principle that can be found innatural phenomena or statements that have been proven tobe true. The “Law” class also consists of two sub-classes:“Law of Nature” and “Theorem”. For example, Moore's Lawdescribes that the number of transistors on an integratedcircuit doubles approximately every eighteen months, butthe price reduces by one half [11].

Law of Nature A learning object of type “Law of Nature” describesgeneral rules observed in nature. For example, Newton'sLaws of Motion is a law of nature in Physics.

Theorem A theorem is a concept that has been shown to be true.Bayes theorem, for instance, is a theorem.

Process The “Process” class includes two sub-classes: “Policy” and“Procedure”. A process is a flowof events that describes howa task can be accomplished in steps. For example, softwaredevelopment life cycle (SDLC) describes the steps howinformation systems can be developed.

Policy A policy describes a set of predefined principles of actions. Itis usually composed of informal suggestions or guidelines

for specific activities. For example, interview is an approachfor system analysts to obtain system requirements. A goodpolicy for interview is to confirm meeting time with theinterviewee in advance and make sure that she compre-hends the to-be-discussed subjects.

Procedure A procedure is a sequence of steps that can be followed toaccomplish a goal. An algorithm of, for example, bubble sortis a procedure.

Interactivity An learning object of type “Interactivity” is some kindof activities that allow learners to practice or develop a skillinteractively. The “Interactivity” class consists of four sub-classes: “Exploration”, “Real World Problem”, “Invitation”,and “Exercise”. They were also defined in the “EDUCATION”category of SCORM CAM.

Illustration An illustration is to illustrate a concept or parts of a con-cept of a learning object. For example, LOB and LOC shown inFig. 1 are two illustration objects for the concept of bubblesort.

Example An example is an auxiliary learning object that is used tofurther explain parts or thewhole of a fundamental learningobject.

Counterexample In the instructional ontology, a counterexample isnot an example of a fundamental learning object, but it isoften mistakenly thought of as one. For example, a paral-lelogram is often mistakenly treated as a rectangle. There-fore, parallelogram can be used as an counterexample for alearning object that describes rectangle.

Evidence An evidence is a learning object that supports the claimsmade for a law or any learning object of its subclasses.The “Evidence” class includes two sub-classes: “Proof” and“Demonstration”. For instance, the time complexity of bub-ble sort is O(n2), and the time complexity of quick sort is O(nlogn). Thus, this is an evidence that quick sort is fasterthan bubble sort.

Proof A proof is an evidence that is derived formally or mathema-tically to support a law.

Demonstration A demonstration is used to demonstrate, in generalthrough experiments, that a law holds under a certaincondition. For example, Galileo's experiment showed thatfalling objects of different weights landed at the same time.

Explanation A learning object of type “Explanation” provides extrainformation for a fundamental learning object so as to highlightits important properties. The “Explanation” class includes threesub-classes: “Introduction”, “Conclusion”, and “Remark”.

Introduction A learning object of type “Introduction” provides thebird's-eye view of a fundamental learning object so that

1031E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

learners have a rough ideawhatwill be covered in a learningresource.

Conclusion A learning object of type “Conclusion” summarizes keypoints covered in a fundamental learning object.

Remark A remark provides extra but inessential information for afundamental learning object. For instance, when a learningtopic is about entity relationship diagram (ERD), an exampleremark can be “ERD is similar to the class diagram in UnifiedModel Language (UML). ERD is mainly used in structuredanalysis and design, while UML is used in object-orientedanalysis and design.”

3.2. The rhetorical–didactic relation

The rhetorical–didactic relation includes ten relations, and they areexplained as follows:

Example and Illustration The definitions of “Example” and “Illustration”are identical to the “Example” and “Illustration”, respectively,defined in instructional ontology.

Instance If a learning topic LTA is sorting, it is then said that bubblesort is an instance of LTA.

Restriction A restriction describes caseswhere a certain theory fails. Forexample, in 1640s, Fermat stated that all Fermat numberswere prime numbers, and the formula for calculating Fermatnumbers was:

Fn = 22n + 1 ð1Þ

However, Euler found that F5 was not a prime number in 1732.

Amplify/Extension An amplify or extension object is a learning object

that is extended from another learning object. For example,semantic web was grown out of the traditional web.

Continues A learning object of type “Continues” describes thesequence relationship between two learning objects whereone is performed after the other. For example, LOA representsdata before sorting, and LOB represents the data after sorting.Then, LOB continues LOA.

Deepen/Intensification A deepen object provides information for

another learning object in depth. For example, LOA describeshow greatest common divisor (GCD) is obtained, and LOB

describes in details the reasons why the process describedin LOA can obtain GCD. Then, LOB deepens LOA.

Opposition An opposition describes a statement proposed by aspecialist that is in contradiction with another statementmade by another specialist. For example, when designingXML documents, some experts suggested avoid using attri-butes to reduce processing time [13]. Still, some expertsstated that it could shorten processing time by usingattributes [10].

Alternative A learning object of type “Alternative” describes a thingthat has been explained in another learning object but indifferent format. For example, LOA describes bubble sort intext, and LOB describes bubble sort in animation. Then, LOB

is an alternative to LOA.

3.3. The proposed relations

In this paper, we designed a new set of relations for learnersin the domain of information technology. The proposed relationswere based on both the instructional ontology and the rhetorical–didactic relations. The design of the relations adhered to the followingprinciples:

• The proposed relations can be served as an extension to themetadata of SCORM CAM.

• Although the number of proposed relations should be as small aspossible, the proposed relations should be able to clearly express notonly the relationships between learning topics and learning objects,but also the relationships between learning objects.

• It is required that the proposed relations can be used in practice andimplemented easily into any existing SCORM-conformed learningmanagement system (LMS). For example, as shown in Fig. 1, itshould be easy to denote the figure at the bottom of LOA as an“Illustration” object. Furthermore, if the author of LOA wishes toprovide more illustrations to help learners, a LMS should be able toallow the author to search and retrieve more illustrations, such asLOB and LOC, from repositories.

Based on the previous discussions, the following relations worthfurther investigation:

Interactivity, Exploration, Real World Problem, Invitation, and Exercise Asstated earlier, the “Interactivity” class and its subclasses havealready been defined in the “EDUCATION” category. To pre-vent duplication, they were not defined in the proposedrelations.

Illustration The meanings of “Illustration” and “Example” are some-what overlapped and sometimes cannot be clearly distin-guished one from the others. According to Oxford AdvancedLearner's Dictionary, an illustration is either “a drawing orpicture in a book, magazine, etc. especially one that explainssomething”, “the process of illustrating something”, or “astory, an event or an example that clearly shows the truthabout something”. If the second definition were used for“Illustration”, one can use the relation “Process” to describelearning objects of type “Illustration”. If the third definitionwere used for “Illustration”, one can use the relation“Example” to describe learning objects of type “Illustration”.Therefore, in the proposed relations, an illustration is de-fined and only defined as “a drawing or picture that explainssomething” to avoid ambiguity.

Amplify/Extension A learning object of type “Amplify/Extension”

connects two related but different learning topics. Becausethe relationships between learning topics are not consid-ered, it is excluded from the proposed relations.

Explanation The relation “Explanation” is somewhat confusing.According to Oxford Advanced Learner's Dictionary, expla-nation is either “a statement, fact, or situation that tells youwhy something happened” or “a statement or piece ofwriting that tells you how something works or makessomething easier to understand”. Thus, an example objectcan be used to explain something. So does a process object.We argue that it is extremely difficult for authors to tellwhether or not a learning object is an explanation object.Consequently, we followed InO's approach and defined the“Explanation” class as the parent class of “Introduction”,“Conclusion”, and “Remark”. In addition, we defined the“Explanation” class as a abstract class.

An abstract class is represented as a dashed node in Fig. 3. Thistype of classes cannot be instantiated and thus cannot be assignedto learning objects. The existence of the abstract classes is to expressrelationships between related classes or subclasses. In addition to“Explanation”, “Fundamental” and “Auxiliary” classes are alsoabstract. In other words, there will be no learning objects of type“Fundamental” or “Auxiliary”.

Fact In the field of information technology, facts are generallyused to describe events, such as who created HTML or when

Fig. 3. The relationships between the proposed relations.

1032 E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

the first computer was constructed, that happened inhistory. Therefore, this type of learning objects can be easilyreplaced with learning objects of type “Remark”.

Law of Nature A learning object of type “Law of Nature” describes alaw in natural science. Thus, this type of learning objects israrely used in the field of information technology.

Policy Based on the definition, a policy is similar to a guideline.Ullrich, the creator of the instructional ontology, alsosaid that the definition of policy was not crystal clear andthus suggested one could make changes if necessary [24].Therefore, we renamed “Policy” to “Guideline”.

Counterexample/Restriction In the instructional ontology, a counter-

example is usually misunderstood as an example of somelearning object. We argue that misleading examples (i.e.counterexamples) can confuse learners. Also, the definitionof “Counterexample” is not the counterexample that wecommonly use to refute a statement by example. Thus, thedefinition for “Counterexample” in the instructional ontol-ogy is not adopted in the proposed relations. Interestingly, alearning object of type “Restriction” in RDR represents acounterexample that we commonly use. As a result, in theproposed relations, the term “Restriction”was removed andthe term “Counterexample”was re-defined to represent theunsuccessful or exceptional situation of a learning object.

Instance Based on its original definition in RDR, an instance is anexample of a learning object or a learning topic. If it is anexample of a learning object, we can simply use the existing

Fig. 4. The number of students that considered a relati

“Example” relation. Otherwise, if it is an example of alearning topic, the learning resource itself is also a learningtopic. As stated earlier, the relationships between learningtopics were defined elsewhere, and thus “Instance” was notconsidered in the proposed relations to prevent duplication.

Continues A sequence of learning objects of type “Continues” can beexpressed with either a “Process” or a “Procedure” relation.Similarly, a learning object of type “Process” or “Procedure” canalso be expressedwith a sequence of “Continues”objects. In theproposed relations, “Continues”was not included for brevity.

Deepen/Intensification A learning object of type “Deepen” is to explain

on is either ‘

another learning object in depth. However, it is sometimesdifficult for authors to distinguish whether or not a learningobject is a “Deepen” object of another learning object. Forexample, an author can use either an illustration, an theorem,or texts to explain a learning object in depth. Should this objectbe defined as of type “Illustration”, “Theorem”, or “Deepen”? Inconsequence, we argued that the relation “Deepen”was of nopractical use and not included the relations.

Opposition A learning object of type “Opposition” can be either alearning object or a learning topic. If it is a learning topic, itthen describes the relationships between two learningtopics which is not considered in the paper. If it is a learningobject, then it is either a proven theory or simply a state-ment without proof. For the former case, one can use therelation “Theorem” to describe the object. For the latter case,one can treat it as an opinion and then use the relation

no-opinion’, ‘useful’, or ‘highly useful’.

Fig. 5. The number of students that considered a relation is ‘useful’ or ‘highly useful’.

1033E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

“Remark” to describe the object. As a result, “Opposition”was not considered in the proposed relations.

Alternative When LOA is an alternative to LOB, it means that both LOA

and LOB explain the same thing but in different format.However, there is a data element called “format” defined inthe “TECHNICAL” category. Thus, “Alternative” is a dupli-cated relation.

From the above discussions, we proposed a relation metadataextension for SCORMCAM.Outof 33 relations,18 relationswere removed.Additionally, “Illustration” and “Counterexample” were re-defined, and“Policy”was renamed to “Guideline”. Theproposed15 relations are “Law”,“Theorem”, “Process”, “Procedure”, “Definition”, “Guideline”, “Introduc-tion”, “Remark”, “Conclusion”, “Illustration”, “Counterexample”, “Exam-ple”, “Evidence”, “Demonstration”, and “Proof”. The relationshipsbetween the proposed relations are depicted in Fig. 3. All instructionalobjects are of type either “Fundamental” or “Auxiliary”. Because it is of nouse todefineany learningobject as “InstructionalObject”, “Fundamental”,or “Auxiliary”, they are defined as abstract classes.Moreover, as discussedpreviously, there are two other abstract classes — “Examples” and“Explanation”. They are included to express structure relationshipsbetween classes and their corresponding subclasses.

4. Analysis and discussions

To find out whether or not the proposed relations could really helplearners and to explore whether or not learners had different pre-ferences on relations, we used questionnaires to survey 145 studentsat the Department of Information Management, Chaoyang Universityof Technology, Taiwan. In these surveyed students, there were 42graduate students and 103 undergraduate students. Out of 145questionnaires, 137 of them were valid.

There were fifteen questions (one for each relation) in thequestionnaires. Each question was rated on a discrete scale from 5to 1, with corresponding verbal descriptions ranging from ‘highlyuseful’ through ‘useful’, ‘no-opinion’, ‘useless’, to ‘highly useless’;

Table 2The T-tests for graduate and undergraduate students.

(A) No Difference

Definition Guideline Theorem Remar

Level of significance 0.05 0.05 0.05 0.05T test 0.5046865 0.21950921 0.05015336 0.0555

(B) Difference

Process Procedure Law

Level of significance 0.05 0.05 0.05T test 0.006629289 0.007848081 0.000756355

respectively. Moreover, the survey was conducted as follows: Firstly,we prepared five learning materials, and each of them includedseveral relations. For each relation object, we provided links to morerelations of the same type. For the example as shown in Fig. 1, moreillustrations were provided for the illustration object at the bottom ofLOA. Secondly, we taught students how to use the learning materials,and let students read through them and answer each question.

The number of students who selected ‘no-opinion’, ‘useful’, and‘highly useful’ on each proposed relation was firstly calculated. It isfound that, as shown in Fig. 4, all proposed relations are consideredhelpful in learning. To investigate this issue further, the number ofstudentswho selected either ‘useful’ or ‘highly useful’ on each proposedrelation was also calculated. It was found that, as shown in Fig. 5, morethan half of the students (that is, more than 69 students) considered 12out of 15 relations are either ‘useful’ or ‘highly useful’. Based on theexperimental results, it is suggested that all proposed relations shouldbe included in the relation metadata extension. Also, when designingschema for the relation metadata extension, it is suggested that“Conclusion”, “Remark”, and “Evidence” can be defined as optional.

ANOVA analysis was used to figure out whether or not under-graduate and graduate students had different preferences on relations.It was assumed that the level of significance was 0.05. Based on theexperimental results summarized in Table 2, it was found that, among15 relations, 7 of themwere significantly different.

We continued our analysis by calculating two ratios for each of the7 relations. One ratio is the number of undergraduate students whoconsidered a relation is either ‘useful’ or ‘highly useful’ over the totalnumber of undergraduate students. The other ratio is calculated forgraduate students. The calculated ratios for each of the 7 relations arepictured in Fig. 6. From Fig. 6, it is shown that, the ratios of “Process”,“Procedure”, “Law”, “Conclusion”, “Evidence”, and “Demonstration”for graduate students are higher than the corresponding ratios forundergraduate students. It is interesting to note that, although lessthan 30% of undergraduate students considered both “Evidence” and“Conclusion” were important, more than 60% of graduate studentsconsidered both “Evidence” and “Conclusion” were helpful.

k Illustration Example Counterexample Proof

0.05 0.05 0.05 0.055037 0.1303536 0.84000124 0.280607373 0.27707975

Conclusion Introduction Evidence Demonstration

0.05 0.05 0.05 0.050.000961491 0.010272104 9.38764E−07 0.000294603

Fig. 6. The ratios of graduate and undergraduate students that considered a relation iseither ‘useful’ or ‘highly useful’.

1034 E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

Based on the experimental results, it is suggested that, whendesigning LOs, some relations can be omitted for certain types of learnersto decrease tedious work in editing metadata if necessary. For example,when designing LOs for undergraduate students, authors can choose notto define learning objects of types “Evidence” and “Conclusion”. On thecontrary, when designing LOs for graduate students, authors can choosenot to define learning objects of type “Introduction”.

Finally, we summarized what we had found based on the survey:

• Based on Fig. 4, all proposed relations were considered helpful.• Based on Fig. 5, out of the 15 relations, only three of them (i.e.“Remark”, “Counterexample”, and “Evidence”) were considered lessuseful. To reduce tedious work in editing metadata, it is suggestedthat learning objects of type “Remark”, “Counterexample”, and“Evidence” do not have to be defined.

• Based on Table 2, graduate and undergraduate students have differentpreferences on 7 relations, and they are “Demonstration”. “Evidence”,“Introduction”, “Conclusion”, “Law”, “Procedure”, and “Process”. Addi-tionally, based on Fig. 6, more than 60% of graduate students consideredboth “Evidence” and “Conclusion” were helpful, while more than 70%undergraduate students did not think so. Therefore, it is suggested thatdifferent sets of relations can be used for different type of learners. Forexample, when creating learningmaterials for undergraduate students,learning objects of type “Evidence” and “Conclusion” can be omitted.

5. Conclusions and future works

In this paper, we discussed the relationships between learningtopics and learning objects as well as between learning objects. Afterdiscussing existing relations, we removed duplicated and inappropri-ate relations, redefined a few relations, and proposed a new relationmetadata extension for SCORM CAM. Furthermore, we investigatedthe usefulness of the proposed relations. From the experimentalresults, it is shown that the proposed relations are helpful for learnersin learning. Also, different types of learners have different preferenceson relations.

Still, there are a few issues worth further investigation. The ultimategoal of designing relations for learning resources is to increase re-usability of LOs and provide a mean for learners to located the requiredLOs easily. Therefore, it is interesting to design a common metadataset [5] for e-learning. We believed that it would be practical and helpfulto design a set of relations for each domain. And then, from bottom up,a common metadata set can be developed. Additionally, to makeauthoring easier, it will be helpful to develop an authoring system thatsupports the proposed relations.

References

[1] A.J. Canas, T. Reichherzer, G. Hill, N. Suri, R. Carff, T. Mitrovich, D. Eberle, Knowledgeofmodeling and creation of EI-Tech: performance support and training systems forelectronic technicians, Expert System with Application 25 (4) (2003).

[2] A.J. Ganas, G. Hill, and J. Lott, Support for constructing knowledge models inCmapTools. Technical Report IHMC CmapTools 93-02, Institute for Human andMachine Cognition, 2003. availabe at http://cmap.ihmc.us/Publications/White-Papers/Support for Constructing Knowledge Models in CmapTools.pdf.

[3] C.M. Reigeluth, (Hrsg), Instructional Design Thoeries and Models: A NewParadigm of Instructional Ontology, vol. 2, Lawrence Erlbaum Associates, 1999.

[4] William C. Mann, Sandra A. Thompson, Rhetorical structure theory: toward afunctional theory of text organization, Text 8 (3) (1988) 243–281.

[5] Yang Ding, A review of ontologies with the semantic web in view, Journal ofInformation Science 27 (6) (2001) 377–384.

[6] Stehpan Fischer, Course and exercise sequencing using metadata in adaptivehypermedia learning systems, ACM Journal of Education Resource in Computing 1(1) (2001).

[7] Gwo-Jen Hwang, A concept map mode for developing intelligent tutoring system,Computer & Education 40 (3) (2003) 217–235.

[8] Philipp Karger, Carsten Ullrich, Erica Melis, Integrating learning object repositoriesusing a mediator architecture, Proceedings of the First European Conference onTechnology Enhanced Learning, 2006, pp. 185–197.

[9] Alexander Loser, Christian Grune, Marcus Hoffmann, A didactic model, definitionof learning objects and selection of metadata for an online curriculum, Proceedingof the International Workshop of Interactive Computer Aided Learning, 2002.

[10] EricJui-Lin Lu, Bo-Chan Wu, Po-Yun Chuang, An empirical study of XML datamanagement in business information systems, The Journal of Systems and Software79 (7) (2006) 984–1000.

[11] Gordon Moore, Cramming more components onto integrated circuits, ElectronicsMagazine 4 (1965).

[12] Learning Technology StandardsCommittee of the IEEE. Draft standard learningobjectmetadata (LOM). Technical report, IEEE, 2006. available athttp://ltsc.ieee.org/wg12/files/LOM 1484 12 1 v1 Final Draft.pdf.

[13] Cover. R. SGML/XML: Using element and attributes. Technical report, 2006.availabe at http://www.oasis-open.org/cover/elementsAndAttrs.html.

[14] Adbulmotaleb EL Sddik, Stefan Fischer, Ralt Steinmetz, Reusable multimediacontent in web-based learning systems, IEEE Multimedia 8 (3) (2001) 30–38.

[15] Cornelia Seeberg, Adbulmotaleb El Saddik, Achim Steinacker, Klaus Reichenberger,Stephan Fischer, Ralf Steinmetz, From the users' needs to adaptive documents,Proceeding of the Integrated Design and Process Technology, 1999.

[16] Achim Steinacker, Anderas Faatz, Cornelia Seeberg, Lvica Rimac, Stefan Hormann,Abdulmotaleb El Saddik, Ralf Steinmetz, Medibook: combining semantic networksmetadata for learning resources to build aweb based learning system, Proceedingsof the EdMedia, 2001.

[17] Achim Steinacker, Cornelia Seberg, Stephan Fischer, Ralf Steinmetz, Multibook:metadata for web-based learning systems, Proceeding of the 2nd InternationalConference on New Learning Technologies, 1999.

[18] Jun-Ming Su, Shian-Shyong Tseng, Ching-Tai Chen, Wen-Nung Tsai, Adaptivelearning environment to meet pedagogical needs, Journal of Information Scienceand Engineering 20 (6) (2004) 1057–1077.

[19] ADL Technical Team. SCORM 2004 3rd Edition Content AggregationModel, version1.0. Technical report, 2006.

[20] Carsten Ullrich, The learning-resource-type is dead, long live the learning-resource-type, Learning Objects and Learning Designs 1 (1) (2001) 7–15.

[21] Carsten Ullrich, An instructional component for dynamic course generation anddelivery, Proceeding of Berliner XML Tage, 2003.

[22] Carsten Ullrich, Description of an instructional ontology and its application inwebservices for education, Proceeding of Workshop on Application of Semantic WebTechnologies for E-learning, 2004.

[23] The ADL Initiative, 2006. available at http://www.adlnet.org/.[24] Discuss forum for the ontology of instructional objects, 2006. cached at http://

forum.activemath.org/viewtopic.php, this discuss forum was closed.[25] Sharable content object reuseable model (SCORM), 2006. available at http://

www.adlnet.gov/scorm/index.cfm.

University of Missouri-Rolla, MO, USA, in 1996. During1997–2004, he was a professor of the Department of

Eric Jui-Lin Lu received his B.S. degree in TransportationEngineering and Management from National Chiao TungUniversity, Taiwan, ROC, in 1982; M.S. degree in ComputerInformation Systems from San Francisco State University, CA,USA, in 1990; and Ph.D. degree in Computer Science from

Information Management and had served as Director ofComputer Center and Head of Graduate Institute ofNetworking and Communication Engineering at ChaoyangUniversity of Technology, Taiwan, ROC. He is currently aprofessor of the Department of Management InformationSystems at National Chung Hsing University, Taiwan, ROC.

His current research interests include XML and Web technologies, distributedprocessing, and network security.

1035E.J.-L. Lu, C.-J. Hsieh / Computer Standards & Interfaces 31 (2009) 1028–1035

Chin-Ju Hsieh is currently a software engineer at DataSystems Corp. She received her M.S. degree in Information

Management from Chaoyang University of Technology,Taiwan in 2007. Her research interests include SemanticWeb and XML Technology.