A Trust Evaluation Model for E-Learning Systems

■ Research Paper

A Trust Evaluation Model for E-LearningSystems

Wenan Tan1,2*, Senbo Chen1, Jingxian Li1, Lingxia Li3, Tong Wang2

and Xiaoming Hu2

1 School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing,China2 School of Computer and Information, Shanghai Second Polytechnic University, Shanghai, China3Department of Information Technology and Decision Science, Old Dominion University, Norfolk, VA USA

In recent years, electronic learning (e-learning) has become an increasingly importantmethod in education. Under the environment of Internet of Things, it has been predictedthat e-learning systems will grow at an even greater pace. Therefore, it is crucial to find asimple and effective evaluation method to assess user trust in e-learning systems. In thispaper, we present an evaluation model based on user trust cloud and user capability fortrusted e-learning. A user trust cloud model is proposed to assess a user’s subjective trust,and a capability matrix method is introduced to assess a user’s objective trust. Theproposed model has been implemented for trust management in e-learning systems.Experimental results show that the proposed trust evaluation model is useful and applica-ble to the user trust assessment in complex e-learning systems. Copyright © 2014 JohnWiley & Sons, Ltd.

Keywords e-learning; user trust cloud; capability matrix; TC-UCEM; systems science

INTRODUCTION

Since the emergence of the research on generalsystems in the 1940s, systems science has pene-trated into many fields (Lin et al., 2013). Systemsscience has five different meanings: a science ofdescription of system, a science of design of gen-eral systems, a science of dealing with systems, ascience of system operations, and a science open

for the integration with other disciplines(Warfield, 2003). Research indicates that systemsscience can deal with the overwhelming com-plexity of a variety of systems, and so far, sys-tems science has contributed to many researchareas (Qian et al., 1993; Xu, 2000; Wan and Jones,2013).

With the development of global economy, in-formation technology has drawn a great deal ofattention (Wilamowski et al., 1999; Wilamowski,2010). In the information technology area, re-search on enterprise systems including servicesystems has become one of the most important

*Correspondence to: Wenan Tan, School of Computer Science andTechnology, Nanjing University of Aeronautics and Astronautics,Nanjing 210016, China.E-mail: [email protected]

Copyright © 2014 John Wiley & Sons, Ltd.

Systems Research and Behavioral ScienceSyst. Res 31, 353–365 (2014)Published online 25 March 2014 in Wiley Online Library(wileyonlinelibrary.com) DOI: 10.1002/sres.2283

research topics (Xu, 2011a, 2011b, 2013; Li, 2012;Tsai et al., 2012; Chen and Fang, 2013; De Vrieset al., 2013; Kataev et al., 2013; Wang et al., 2013;Xing et al., 2013; Zhou et al., 2013). Meanwhile, in-formatics has been studied in the framework ofsystems science. In service systems, systemcomponents such as service objects and serviceinteractions are directly related to the quality ofservice. So far, little research has been conductedon the trust evaluation in electronic learning(e-learning) systems. In this paper, the trustmanagement of e-learning systems is studied insystems perspectives.

E-service systems can provide more agileand powerful services using Web service tech-nologies, such as service-oriented architecture(SOA), and enterprise systems technologies.Many relevant concepts have been proposed,such as e-commerce, e-business, e-marketplace,e-government, and e-learning (Wang et al.,2012). Among these new concepts, e-commerceseeks to add revenue streams by using the Inter-net to build and enhance relationships withclients and partners and to improve efficiency.E-business is the application of information andcommunication technologies for supporting themanagement of all activities in the businesses.E-business has rapidly extended its scale andscope during the last two decades; it becomesone of the most challenging areas for industryand research communities (He and Xu, 2012;Viriyasitavat et al., 2012; Xu et al., 2012; Shanet al., 2013a; Xia et al., 2013). In addition toe-business, e-government is the transformationof public sector internal and external processesthrough net-enabled operations, informationtechnology and communications, to optimizegovernment service delivery (Chen et al., 2013;Shan et al., 2013b). It aims at providing electronicinformation and serving citizens and society.E-marketplace is the infrastructure of electronicmarket information systems for buyers andsellers to conduct business through electronictransactions (Guo et al., 2012a, 2012b). E-learningis very different from the aforementionedconcepts; it is essentially the computer and net-work-enabled transfer of skills and knowledge.E-learning comprises all forms of electronicallysupported learning and teaching, and it is an

effective means to implement the learning pro-cesses on the Internet (Uden et al., 2007). Manye-learning systems such as educational caselibrary(Kanfer et al., 2000), syllabus generatorsystem (Carchiolo et al., 2007), distance learningadministrative system (Romano et al., 2005) andstudent field experience learning tracking system(Costa and Silva, 2010) have been used to helpeducators improve their teaching and to help stu-dents achieve their learning objectives and needs.

In this study, we mainly focus on the trust re-search in e-learning systems and propose a trustevaluation method for the trust management ofe-learning systems. The well-established commu-nity of inquiry (CoI) framework emphasizes theimportance of building a trusting environmentand developing interpersonal relationships onstudents’ learning experience in e-learningenvironments. E-learning services have beenwidely adopted in practice and have been increas-ingly accepted as an effective learning model.E-learning provides many benefits includingconvenience, cost saving, flexibility, greater collab-oration, personalized learning, etc. So far, we havewitnessed many innovative developments ine-learning systems and applications. Recently, the re-search community has predicted that an e-learningecosystem will become the next generation learn-ing system (Chang and Guetl, 2007). As trust is akey part of any ecosystem, trust managementfor e-learning service providers and service re-ceivers has become indispensable in e-learningsystems (Iacono and Weisband, 1997). In addi-tion, trust management is also important for thesecurity of e-learning services. However, fewpieces of research have been conducted on thetrust assessment of e-learning systems. Oneobvious reason is that online learning is just inits infant stage, and a lot of methods and technol-ogies are still in the stage of exploration. In thispaper, we explore the trust management and pro-pose an evaluation model based on user trustcloud (UTC) and user capability in e-learningsystems.

In the last few years, researchers have pro-posed a few trust evaluation models. An explor-atory qualitative study was conducted onwhether the formation of swift trust in the firstfew weeks of an online course could help to

RESEARCH PAPER Syst. Res

Copyright © 2014 John Wiley & Sons, Ltd. Syst. Res 31, 353–365 (2014)DOI: 10.1002/sres

354 Wenan Tan et al.

explain the success of the online course (Li et al.,2002). Blaze et al. provided a comprehensive anal-ysis on trust management based on a simple lan-guage for specifying trusted actions and trustrelationships (Blaze et al., 1996). The role-basedtrust management framework was presented toestimate user trust levels to authorize corre-sponding permissions according to the roles.Abdul-Rahman et al. proposed a trust modelbased on the real-world social properties of trust,which was outlined for supporting trustmeasurement in virtual communities based onexperience, reputation, and distributed recom-mendations (Abdul-Rahman and Hailes, 2000).Jøsang presented a concept called subjective logicto assess trust values based on the triplet repre-sentation of trust (Jøsang, 1999). Researchersintroduced a formal method to represent a sub-jective UTC model for the computation of subjec-tive uncertainty such as randomness andfuzziness of subjective trust relationship.Considering the impact of the user capability

change on interactive actions, we extend our pre-vious work in this area and propose a trust eval-uation model by integrating the UTC and theuser capability (TC-UCEM) in e-learning sys-tems. In this study, UTC is proposed for user sub-jective trust, and user capability is presented foruser objective trust.The paper is structured as follows: In the sec-

tion on Related Work, related work is brieflyreviewed. In the section on Decision-making,the trust evaluation based on UTC and capabilitymatrix is presented. The section on Trust Evalua-tion Model Based on User Trust Cloud andBehaviour Capability Matrix describes the exper-imental procedures and the results. The sectionon Experiments and Results discusses the experi-mental results; finally, in the section on DataAnalysis and Discussion, the conclusion andfuture research directions are provided.

RELATED WORK

The proposed trust evaluation model is based onsubjective dimension and objective dimension. Inthis section, some related work is introduced. Theconcepts of e-learning system and trusted service

are discussed at first; then, we describe e-learningsystemsbasedon Internet of Things (IoT) and explainthe advantages IoT brings to the e-learning systems.

E-learning Systems and Trusted Service

E-service systems have received a great deal ofattention recently as the service providers candeliver e-service to end users promptly by inte-grating existing functionalities and externalready-to-use services as brand new services(Tan et al., 2012, 2013).

Because of the development of the Internettechnologies, more and more people are usinge-learning systems and applications for their edu-cational needs (Beer et al., 2005; Hill and Roldan,2005; Carchiolo et al., 2007). Many dynamice-learning platforms have been developed forreplacing the traditional passive e-learning plat-forms. Active e-learning employs a broad rangeof Internet technologies such as personalized on-line learning environments, simulation, Web 2.0,and learning management systems (LMSs) toachieve various educational goals. As learnershave diverse learning styles and learning needs,new learning platforms must be able to deal witha number of requirements in order to survive andsucceed.

An e-learning system can be regarded as an in-telligent system that integrates electronic learn-ing and network community (Barolli et al.,2006). For example, agents can be created in e-learning systems to automatically perform tasksand also collaborate with other agents in anetworked community environment. Chen andLo proposed an open learning community as anagent e-learning service system (Chen and Lo,2004). Their framework consists of four layers:interface, network community, agent, and docu-ments and databases. These four layers can inter-act and collaborate in an e-learning environment.Each layer has different functions to supportusers with some activities related to interactionand collaboration on the Web (Chen, 2009). Todevelop cost efficient and dependable learningservices for education purpose, Pasatcha et al.designed a distributed e-education system thatused SOA as a model for deploying, discovering,

Syst. Res RESEARCH PAPER


Trust Evaluation Model for E-Learning Systems 355

integrating, implementing, managing, and in-voking an e-education system (Pasatcha andSunat, 2008). To provide students with realisticaudio-visual contents, Lee et al. (2009) proposedan interactive e-learning system that uses thepattern recognition and augmented reality tech-nology. They also designed a mentoring systemfor self-studying that allows students to learn au-dio-visual contents interactively (Lee et al., 2009).Chen (2009) presented an integrated e-learningsystem that was open, intelligent and self-learning. The system was composed of sixmodules: synchronous learning module, asyn-chronous learning module, learning evaluationmodule, information service module, systemicservice module, and resource management mod-ule. In addition, the system was based on data or-ganization in data warehouse with self-learningas its primary goal (Chen, 2009). To help usersto operate the system easily, Nagata et al. (2009)proposed the gesture interface as an operationmethod for the e-learning service system; theyimplemented and evaluated some gestures fortheir object-based e-learning system. The systemwas made with Adobe Flash, and it was capableof recognizing two positions and motions ofuser’s fingers from a Universal Serial Bus camera(Nagata et al., 2009). As the Web services elimi-nate many interoperability issues for distributedcomponents running on different hardware andsoftware platforms, a new e-learning systembased on Web services has been proposed, inwhich the developed Web services includeAssessment, Course Management, Grading,Marking, Metadata, Registration, and ReportingWeb services (Su et al., 2007). As a powerful andexpressive non-textual medium that can captureand present information, instructional videosare extensively used in e-learning service sys-tems. Zhang et al. presented an interactive multi-media-based e-learning environment that enablesusers to interact with it to obtain knowledge inthe form of logically segmented video clips andproposed a natural language approach tocontent-based video indexing and retrieval toidentify appropriate video clips that can addressusers’ needs (Zhang and Nunamaker, 2004).

Because the next generation e-learning plat-forms will be based on SOA and cloud

computing, researchers have discussed the evo-lution of LMS and presented the core challengesto be addressed in order to achieve informationinteroperability in next generation e-learningplatforms (Dagger et al., 2007; Li et al., 2012; Taoet al., 2013). After investigating the problem ofadaptation and personalization in e-learning sys-tems, a new metric, quality of learning (QoL), hasbeen suggested for e-learning systems to evaluatethe learning process. The proposed system canprovide dynamic learning content and an adap-tive learning process for learners to enhance thequality of their learning (Liu and Yang, 2005).

While the previously mentioned studies haveinvestigated e-learning systems from differentperspectives, their main focuses have been onhow to construct the service platform to meetthe users’ requirements and how to use Webservice effectively. The issue on how to ensuree-learning service has not been investigated inthose studies. In this study, we carry out theresearch on the trust evaluation in e-learningsystems and present the concept of trusted ser-vice. The service system plays three roles: serviceproviders, service resources, and service systemconsumers. Although many researchers havestudied trust evaluation models for service re-sources and service providers, none of them hasstudied service consumers. As the trust of serviceconsumers may affect the quality of service andthe service system and eventually affect theimplementation of e-learning, we believe thatresearch on the trust of service consumers isneeded.

E-learning Systems Based on Internet of Things

The emergence of IoT brings more advantages toe-learning systems that help us to evaluate usertrust in e-learning systems. The connection ofphysical things to the Internet makes it possibleto assess remote sensor data and to control thephysical world from a distance. The mashup ofcaptured data with data retrieved from othersources, for example, with data that are con-tained in the Web, gives rise to new synergisticservices that go beyond the services that can beprovided by an isolated embedded system.




The advantages IoT brings to e-learning are asfollows:

1. Creating an ideal learning environment for e-learning system evaluation.

In e-learning mode, users’ learning envi-ronment is always changing. Creating a stableenvironment for learners is imperative and im-portant for users in e-learning systems. IoT,aiming at fulfilling the comprehensively inter-connected network for things to things, humanto things and human to human, can actively per-ceive the existence of things and obtain informa-tion about objects, timely eliminating adversefactors, to create an ideal learning environment(Li et al., 2013a, 2013b).

2. Providing individual services for users in e-learning systems.

Internet of Things provides individual servicesfor users according to the study situation. IoTmakes all the things digital and networking tothe fullest extent, leads the reality of objects toconnect in an intelligent way and makes the net-work service intelligent. It can dynamically per-ceive learners’ surroundings, support learners inmaking better use of all available resources tomeet individualized learning needs, and makethe individual developments possible.

3. Providing learning resources for e-learningsystems.

In the IoT environment, through radio-frequency identification and sensors, getting allkinds of information in the physical world is possi-ble. Various communication networks can expandthe extension of hardware and software applica-tion, realizing a greater degree of integration andthe sharing of resources, making learners to obtainhigh quality learning resources more likely.

4. Collecting enough information for a trust eval-uation model.

The countless sensors in IoT are seen as infor-mation sources; according to a certain frequency,it tracks periodically and monitors learners’ be-haviour and then collects the information. Eachcollection gets new data; this will be the basis ofthe trust evaluation.

DECISION-MAKING

An e-learning system is a typical application ofInternet technologies in the field of education.Such a system is usually deployed in an open net-work. To ensure the QoL, it is important for a ser-vice provider to identify service receivers basedon their credibility. The e-learning service systemperforms three basic functions: e-learning servicesystem platform, e-learning service providers,and e-learning service receivers (learners). Wecan consider the e-learning service users as trustobjects or trust subjects. The user trust in the pro-portion of the entire trust is based on the user’sexpected value. We allocate different trustweights to a user’s own trust based on the pro-portion of the user’s own trust value.

Because there are no perfect trust evaluationmechanisms available, we assume that futuree-learning systems will use a rating system. Tosimplify the discussion of our trust model, weassume the following:

(1) There are many subjects and objects ine-learning systems.

(2) An e-learning system should provide a ratingmechanism to evaluate objects.

(3) The rating mechanism of an e-learning sys-tem deals with two cases: The first case is tograde users; the second is to grade usersaccording to the accuracy of finishing the taskand the comments of other users.

(4) An e-learning system should provide somemechanisms to avoid illusive evaluation.

(5) Generally, we grade users on a scale rangingfrom 0 to 1.

User Trust Cloud

A cloud model can be used to describe trustrandomness and fuzziness. We propose a modelfor the UTC based on a cloud model, which canbe used to evaluate the user subject trustappropriately.

Definition 1: User trust degree (UTD) is anordered set of numbers in a normalized uni-versal set [0, 1], that is, UTD= [0, 1]. UTD is




composed of sequential or discrete numbersthat represent a trust object’s reputation;0 and 1 represent the lower and upper limitsof reputation, respectively.

Definition 2: User trust space (UTS) is an or-dered set of qualitative concepts that repre-sents the qualitative degree of trust.

Definition 3: User trust cloud is an e-learningtrust concept represented by the cloud modeland composed of many cloud drops. UTD= [0,1] is the universal set of UTC. For any e∈UTS,e is a qualitative trust concept of a user trustsystem (UTS), and any x∈UTD is an imple-ment of e. The certainty degree of x for e, thatis, μ(x)∈ [0, 1], is a random value with the sta-bilization tendency of

μ : UTD→ 0; 1½ �; x∈UTD; x→μ xð Þ (1)

The distribution of x within UTD is defined asUTC(x), and every x is called an e-learning UTCdrop.

According to definition 3, the cloud has someimportant characteristics as follows:

(1) The cloud can describe the trust subjectivitywith more approximation accuracy. Thecloud is the random variable x in the quanti-tative universal set of U in which x is not justa simple random variable in terms of proba-bility; for any x∈U, x has a certainty andthe certainty is also a random variable in-stead of a fixed number.

(2) The degree of certainty of the cloud drop canbe considered as the accuracy in which thedrop represents the concept for. The larger theprobability of the cloud drop is, the greaterthe degree of certainty of the cloud drop is.

(3) The cloud is a model that combines randomvariables in the probability theory and the de-gree of certainty in fuzzy mathematics.

User trust cloud can be described by Ex, Enand He. Ex is the user expected value, whichcan be calculated according to historic opera-tional records. En is the dispersion of user historicvalue, which indicates the stability of user study.

He is a measure of the dispersion on the usercloud drops, which can also be considered asthe entropy of En, and it is determined by therandomness and fuzziness of En.

Behaviour Capability

User capability evaluation is another importantaspect of the evaluation for users in e-learningsystems. For a service provider, ‘a user is trusted’means that the user is not only trusted but alsohas the ability to be trusted. Based on the previ-ously mentioned consideration, we developed acapability matrix that includes interactive objectsand interactive activities to describe the behav-iour capabilities.

Definition 4: User: Every e-learning participantis an independent user. User is the subject oflearning activities in the community. Relatedusers may complete learning tasks by commu-nicating with others. We assume that each usermay communicate with n users. Y is the sam-ple set of n users, that is, Y= {Y1,Y2, ⋯,Yn}.

Definition 5: The behaviour capability elementof each user can be understood as one of itsdescription at a specific point. A user has m di-mensional capability attributions. User capa-bility attributions are reflected throughinteractive actions with others. For example,object x has m dimensional capability attribu-tions; the sample set X can be described asX= {x1, x2, ⋯, xm}.

Definition 6: User weight and behaviour capa-bility weight: Different weights are allocatedto users according to frequencies of a user’s in-teractive activities. User weight set can be de-scribed as ω′

j ¼ ω1′;ω2

′⋯;ω′n

� �. Different

weights are assigned appropriately to capabil-ity attributions according to the importance ofinteraction capability in the e-learning system.The set of weights of behaviour capabilityattributions can be expressed as follows:ω ¼ ω1 ;ω2 ; ⋯;ωmf g.In a user capability evaluation model, each user

can interact with different users based on different




actions. For simplification, we assume that user Xinteracts m kinds of actions with n users.

TRUST EVALUATION MODEL BASED ONUSER TRUST CLOUD AND BEHAVIOURCAPABILITY MATRIX

In this section, we describe the user evaluationusing the UTC and the capability evaluationmodel respectively. We propose a reasonable userevaluation algorithm according to the expectedvalue along with entropy, as well as a usercapability algorithm according to the behaviourmatrix multiplication.

User Trust Evaluation Based on User TrustCloud

To evaluate user trust more accurately, it is im-portant to realize that assigning an appropriateweight to the user trust has a great impact ontrust evaluation of e-learning. We have previ-ously presented an idea that the trust weightsare assigned to user trust and user capability ran-domly (Iacono and Weisband, 1997). Withoutconsidering the complexity of user trust, that ideais not sufficient for solving complex problems inpractice. As an improvement to our previouswork, we propose to allocate weights to usertrust according to its expected value and use itto evaluate user comprehensive trust.Without losing the generality of the method,

we classify the levels of trust qualitatively and al-locate user trust weights according to theexpected values. The details are shown in Table 1.From Table 1, one can see that the user weight

is allocated with the highest value of 0.5 whenpeople are in a full trust, while the user weightis allocated with the lowest value of 0 when aperson is not trusted. The table indicates thatthe higher the expected value is, the more trustedthe user is.

Using xi, we compute three numerical valuesof the UTC, the trust values of Ex, En and Heare separately calculated by an equation as fol-lows:

Ex ¼ ∑n

i¼1pixi (2)

S2 ¼ 1N � 1

∑n

i¼1pi xi � Exð Þ2 (3)

En ¼ffiffiffiπ2

r∑n

i¼1pi xi � Exð Þ (4)

He ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiS2 � En2�� q

(5)

In Equation (2), xi is the historic record that re-flects the user operating status, we define xi= 0when a user has non-positive operations. In addi-

tion, ∑N

i¼1pi xi � Exj j represents the first-order abso-

lute central moment and s2 is sample variance ofxi in Equation (3). In Equation (4), pi representsaction i′s impact factor for all n actions by user x.

User trust evaluation formula can be expressedas follows:

ECtr Xð Þ ¼ 1� Enð ÞExþ 1σ

1�He2� �

(6)

where ECtr(X) represents the trust cloud evalua-tion of user x, and σ is trust hyperentropy factor.It depends on the expected value and entropy.Equation (4) illustrates that the user trust evalua-tion value is larger if the expected value is largerand entropy is smaller.

Table 1 User trust level and user trust weight

Expected value ofuser trust (Ex) Trust level Trust weight (α)

Ex = 0 Distrusted 00<Ex≤ 0.25 Little trusted 0.10.25<Ex≤ 0.5 Generally trusted 0.20.5<Ex≤ 0.75 Extremely trusted 0.30.75<Ex≤ 1 Completely trusted 0.4Ex= 1 Ultimately trusted 0.5




User Behaviour Capability Evaluation

The evaluation algorithm of user behaviourcapability trust is discussed in this section. Userx is an entity with an e-learning role and has di-mensional capability; user x has m cooperationactivities with n users. The user interactive actionmatrix can be given as

Xx←Yj ¼

y11 y12 ⋯ y1ny21 y22 ⋯ y2n

⋱ym1 ym2 ⋯ ymn

26664

37775 (7)

where x←Yj represents that object x has an inter-active action with object Yj, and Yj is the jth objectin n object set, while Yij represents the successfuldegree for which x interacts with Yj on the ith ca-pability attribution action.

Reasonable weights need to be allocated to dif-ferent actions. Each interactive action contributesto different dimensional capability according tothe attribution definition of the interactive actions.

For user x, the user trust value is obtainedthrough the weight matrix multiplied by the in-teractive action matrix. Firstly, the cross productof capability attribution weight matrix and theinteractive action matrix is calculated; m weightsare assigned appropriately to capability attribu-tions according to history records, which isdepicted as

EAtr X←Yið Þ ¼ ω1 ω2 ⋯ ωmð Þ

�

y11 y12 ⋯ y1n

y21 y22 ⋯ y2n

⋱

ym1 ym2 ⋯ ymn

2666664

3777775

(8)

EAtr X←Yið Þ ¼ Y11 Y12 ⋯ Y1nð Þ (9)

where EAtr(X←Yi) is one-dimensional matrix ofn user evaluation values according to m dimen-sional capability attributions in Equation (8). ωi

is the weight for the ith interactive action. The

weight ωi satisfies ∑m

i¼1ωi ¼ 1.

ω′j

� T¼ ω′

1 ω′2 ⋯ ω′

n

� �T ¼

ω′1

ω′2

⋮ω′n

0BBBB@

1CCCCA (10)

EAtr Xð Þ ¼ EAtr X←Yið Þ ω′� �T

(11)

where ω′j

� Tis the inverted matrix of (ω′

j ) in

Equation (10), the weight of the same capabilityattribution can be set differently according tothe users (interactive action objects), and ω′

j is al-located to users based on frequency of user inter-actions.

EAtr Xð Þ ¼ ω′1Y11 þ ω′

2Y12 þ ⋯þ ω′nY1n (12)

EAtr(x) is a comprehensive value of user capa-bility that we consider m dimension capabilityattributions of x and n interactive objects that xinteracts with. The innovation of this algorithmis that matrix multiplication is proposed to com-plete the user capability evaluation.

Comprehensive Evaluation Model

Here, we propose a comprehensive evaluationmodel based on user trust evaluation and usercapability evaluation, which reflects the reliabil-ity of user in the e-learning system. The inte-grated evaluation can help a service provider toselect appropriate service receivers. The compre-hensive model can be expressed as

LE Xð Þ ¼ αECtr Xð Þ þ 1� αð ÞEAtr Xð Þ (13)

In Equation (13), LE(X) represents the e-learningcomprehensive evaluation value; ECtr(X) is usertrust value, while EAtr(X) is the user capabilityevaluation that we have discussed previously. αis the weight allocated to user trust based on theexpected value of user.




Description of TC-UCEM

User trust cloud evaluation and user capabilityevaluation are two important aspects in success-ful e-learning trust evaluation. For service pro-viders, service receivers are selected based onan integrated evaluation in the e-learning system.The integrated evaluation model based on usertrust and user capability is called TC-UCEM.A trust evaluation model by integrating the

UTC and the user capability can be expressedby a five tuple:

TC-UCEM ¼ U;C;T;CA; Fð Þ (14)

where U represents the user object, C is UTC,which reflects UTD of user subject trust and thestability of user learning, and T describes usertrust value, which is calculated by UTC expectedvalue, entropy and hyperentropy. CA expressesuser the capability evaluation value, which canbe calculated by considering both the interactiveaction objects and interactive action capability at-tribution. F indicates the trust function for the de-termination of user trust and user capability trust.Both subjective user trust and objective capa-

bility trust are used in the TC-UCEM. The contri-bution of the model is that UTC is proposed toassess a user’s own trust, and capability matrixmultiplication is provided to evaluate user capa-bility with the consideration of capability attribu-tion and interactive objects. Appropriate weightsare assigned to user trust and capability trust.Based on user trust and capability trust, the in-

tegrated evaluation model avoids the problemswith trust measurement and overcomes theshortcomings of single capability estimation.The selection of service receivers and cooperativetask delegation can effectively increase reliabilityand flexibility. This model has the potential to re-duce risk and improve security in the implemen-tation of e-learning systems.

EXPERIMENTS AND RESULTS

The purpose of the experiment is to prove the use-fulness and effectiveness of the proposed model.

By applying the user cloud trust evaluation anduser capability trust evaluation algorithm, we col-lected the data from different platforms andconducted simulation experiments.

In the first experiment, direct interactions with100 users have been simulated; the interaction datarepresent the degree of successful operations. Usertrust value is calculated in two ways: UTC evalua-tion is calculated fromEquation (6), and traditionaltrust evaluation is calculated by expected value.The simulation results are shown in Figure 1.

In the second experiment, the capability inter-action of 100 interactive users on 10 different ca-pability attributions is simulated. Because mostof the websites do not provide a direct capabilityinteraction value, we conducted the experimentbased on the operation rating value of other ob-jects that the object interacts with. The user capa-bility evaluation value is calculated fromEquation (7) to Equation (11). The simulation re-sults have been shown in Figure 2.

The third simulated experiment is carried outbased on the first two experiments. The usercomprehensive evaluation value is calculatedfrom Equation (12) with the consideration of usertrust weight factor α. The simulation results havebeen shown in Figure 3.

DATA ANALYSIS AND DISCUSSION

Figure 1 has shown that the trend of the twocurves is inconsistent, while the curve of user

Figure 1 Curves of user trust evaluation value




cloud trust evaluation is relatively flat. The rea-son is that the UTC evaluation is determined byexpected value and entropy; the user trust valuedid not change significantly as the number of in-teraction changes. In addition, the maximumvalue in the curve of UTC evaluation is obtainedwhen the expected value is high and entropy islow. The curve of traditional trust evaluationchanges so rapidly that it is not able to reflectuser trust without considering the entropy andhyperentropy of the interaction data. Therefore,expected value could not accurately reflect thecomplexity of the user trust. This experiment re-flects that the UTC model is more realistic and ef-fective to express the stable trust evaluation ofuser in actual applications.

In Figure 2, the user capability trust evaluationvalue changes quickly when the number of be-haviour capability interaction changes. However,the value typically remains between 0.7 and 0.8.The curve illustrates that the level of user behav-iour capability fluctuates in a certain rangeinstead of a determined value. The reason is thatuser capability can be affected by different inter-active objects and different numbers of interac-tion. However, many pieces of research ontraditional trust models failed to discuss usercapability trust evaluations and neglected theobjective aspect of trust. Different from priorstudies, our experimental results show that theuser capability model is more accurate andmeaningful to practical situations.

Figure 3 shows that the range of the compre-hensive evaluation value is between 0.5 and 0.6,which is close to UTC evaluation value but isfar from user capability trust evaluation value.The reason is that user comprehensive trust is de-termined not only by expected value but also bythe stability of user evaluation value. As a result,it appears that user trust value is low, althoughuser capability evaluation value is high. The re-sults of the previously mentioned experimentsshow that the user comprehensive trust evalua-tion model is useful and reasonable for complexuser trust assessment.

SUMMARY AND FUTURE RESEARCH

The CoI framework suggested by Garrison et al.(1999) shows that trust is important to build aneffective learning community around e-learningsystems. However, not many prior studies havebeen conducted to investigate the trust evalua-tion and management in e-learning systems. Inthis paper, we propose a trust evaluation modelfor e-learning systems based on UTC and usercapability. The concept of UTC is proposed to as-sess user subjective trust; it expresses the ran-domness and fuzziness of user trust. Capabilitymatrix multiplication is introduced to calculateuser behaviour capability with the considerationof capability objects and interactive attributions.To evaluate compressive trust, appropriateweights for user subjective trusts are assigned

Figure 2 Curve of user capability trust evaluation

Figure 3 Curve of user comprehensive trust evaluation




according to user expected values. With the rapiddevelopment of service systems and the Internet,especially with a growing attention on e-learningsystems, we expect that our proposed trust eval-uation model will be very useful to support usertrust decision-making. Furthermore, there is stilla need for significant research in this area. Thereare still many unsolved issues including how todesign and assign behaviour capability weightsmore precisely, how to apply the proposed modelto other service platforms, and how to extend theapproach to other related fields. We will continueour research efforts to address these issues. In thefuture, we will conduct more e-learning researchon trust evaluation and its integration with newtechnology including Web 2.0, agent, SOA, datawarehouse and cloud computing (Xu et al., 2008;He et al., 2009; Duan and Xu, 2012; Ingvaldsenand Gulla, 2012; Fang et al., 2013; Hachani et al.,2013; Le et al., 2013; Li et al., 2013a, 2013b).

ACKNOWLEDGEMENTS

This work was supported in part by the NationalNatural Science Foundation of China underGrant No. 61272036 and Innovation Program ofShanghai Municipal Education Commission un-der Grant No. 11ZZ188.

REFERENCES

Abdul-Rahman A, Hailes S. 2000. Supporting trust invirtual communities. Proceedings of the 33rd AnnualHawaii International Conference on System Sciences,Jan 4-7, 2000.

Barolli L, Koyama A, Durresi A, De Marco G. 2006. Aweb-based e-learning system for increasing studyefficiency by stimulating learner’s motivation.Information Systems Frontiers 8(4): 297–306.

Beer M, Slack F, Armitt G. 2005. Collaboration andteamwork: Immersion and presence in an onlinelearning environment. Information Systems Frontiers7(1): 27–37.

Blaze M, Feigenbaum J, Lacy J. 1996. Decentralizedtrust management. Proceedings of 1996 IEEESymposium on Security and Privacy, Oakland, CA,pp. 164–173.

Carchiolo V, Longheu A, Malgeri M, Mangioni G. 2007.A model for a web-based learning system. Informa-tion Systems Frontiers 9(2-3): 267–282.

Chang V, Guetl C. 2007. E-Learning Ecosystem (ELES)-A Holistic Approach for the Development of moreEffective Learning Environment for Small-and-Medium Sized Enterprises (SMEs). Proceedings ofIn Inaugural IEEE-IES Digital EcoSystems andTechnologies Conference, Cairns, Feb 21-23, 2007, pp.420–425.

Chen G. 2009. The research on architecture of intelli-gent E-learning system. Proceedings of 4th Interna-tional Conference on Computer Science & Education,Nanning, July 25-28, 2009, pp. 1079–1081.

Chen X, Fang Y. 2013. Enterprise systems in financialsector-an application in precious metal trading fore-casting. Enterprise Information Systems 7(4): 558–568.

Chen C, Lo W. 2004. An open learning community foragent e-learning system. Proceedings of Society forInformation Technology & Teacher Education Interna-tional Conference, pp. 2839–2843.

Chen C, Zhu X, Ao J, Cai L. 2013. Governance mecha-nisms and new venture performance in China.Systems Research and Behavioral Science 30(3): 383–397.

Costa G, Silva N. 2010. Knowledge versus content ine-learning: A philosophical discussion. InformationSystems Frontiers 12(4): 399–413.

Dagger D, O’Connor A, Lawless S, Walsh E, Wade V.2007. Service-oriented e-learning platforms: Frommonolithic systems to flexible services. IEEE InternetComputing 11(3): 28–35.

De Vries M, Gerber A, van der Merwe A. 2013. Aframework for the identification of reusable pro-cesses. Enterprise Information Systems 7(4): 424–469.

Duan L, Xu L. 2012. Business Intelligence for Enter-prise Systems: a Survey. IEEE Transactions on Indus-trial Informatics 8(3): 679–687.

Fang S, Xu L, Pei H, Liu Y. 2013. An IntegratedApproach to Snowmelt Flood Forecasting in WaterResource Management. IEEE Transactions on Indus-trial Informatics 10(1): 548–558.

Garrison D, Anderson T, Archer W. 1999. Critical in-quiry in a text-based environment: Computer con-ferencing in higher education. The Internet andHigher Education 2(2-3): 87–105.

Guo J, Xu L, Xiao G, Gong Z. 2012a. ImprovingMultilingual Semantic Interoperation in Cross-Organizational Enterprise Systems through ConceptDisambiguation. IEEE Transactions on Industrial Infor-matics 8(3): 647–658.

Guo J, Xu L, Gong Z, Che C, Chaudhry S. 2012b. Se-mantic inference on heterogeneous e-marketplaceactivities. IEEE Transactions on Systems, Man andCybernetics, Part A: Systems andHumans 42(2): 316–330.

Hachani S, Gzara L, Verjus H. 2013. A service-orientedapproach for flexible process support within enter-prises: application on PLM systems. Enterprise Infor-mation Systems 7(1): 79–99.

He W, Xu L. 2012. Integration of Distributed EnterpriseApplications: A Survey. IEEE Transactions on Indus-trial Informatics 10(1): 35–42.




He W, Xu L, Means T, Wang P. 2009. Integrating Web2.0 with the Case-based Reasoning Cycle: A SystemsApproach. Systems Research and Behavioral Science26(6): 717–728.

Hill T, Roldan M. 2005. Toward third generationthreaded discussions for mobile learning: opportuni-ties and challenges for ubiquitous collaborative envi-ronments. Information Systems Frontiers 7(1): 55–70.

Iacono C, Weisband S. 1997. Developing trust in virtualteams. Proceedings of the Thirtieth Hawaii InternationalConference on System Sciences, Wailea, HI, Jan 7-10,1997, pp. 412–420.

Ingvaldsen J, Gulla J. 2012. Industrial application ofsemantic process mining. Enterprise InformationSystems 6(2): 139–163.

Jøsang A. 1999. An algebra for assessing trust in certi-fication chains. Proceedings of the Network and Distrib-uted Systems Security Symposium (NDSS’99), TheInternet Society.

Kanfer A, Haythornthwaite C, Bruce B, et al. 2000.Modeling Distributed Knowledge Processes in NextGeneration Multidisciplinary Alliances. InformationSystems Frontiers 2(3-4): 317–331.

Kataev M, Bulysheva L, Emelyanenko A, EmelyanenkoV. 2013. Enterprise systems in Russia: 1992-2012.Enterprise Information Systems 7(2): 169–186.

Le C, Gu X, Pan K, Dai F, Qi G. 2013. Public and expertcollaborative evaluation model and algorithm forenterprise knowledge. Enterprise Information Systems7(3): 375–393.

Lee S, Choi J, Park J. 2009. Interactive e-learning systemusing pattern recognition and augmented reality. IEEETransactions on Consumer Electronics 55(2): 883–890.

Li L. 2012. Effects of enterprise technology on supplychain collaboration: analysis of China-linked supplychain. Enterprise Information Systems 6(1): 55–77.

Li N, Mitchell J, Winsborough W. 2002. Design of arole-based trust-management framework. Proceed-ings of 2002 IEEE Symposium on Security and Privacy,pp. 114–130.

Li S, Xu L, Wang X, Wang J. 2012. Integration of hybridwireless networks in cloud services orientedenterprise information systems. Enterprise Informa-tion Systems 6(2): 165–187.

Li Q, Wang Z, Li W, Li J, Wang C, Du R. 2013a. Appli-cations integration in a hybrid cloud computing en-vironment: modeling and platform. EnterpriseInformation Systems 7(3): 237–271.

Li S, Xu L, Wang X. 2013b. Compressed Sensing Signaland Data Acquisition in Wireless Sensor Networksand Internet of Things. IEEE Transactions on Indus-trial Informatics 9(4): 2177–2186.

Lin Y, Duan X, Zhao C, Xu L. 2013. Systems ScienceMethodological Approaches. CRC Press: Taylor &Francis Group, London, New York.

Liu H, Yang M. 2005. QoL guaranteed adaptation andpersonalization in E-learning systems. IEEE Transac-tions on Education 48(4): 676–687.

Nagata K, Dounoue S, Koshi K, Miyoshi M, Kiyota K.2009. Implementation and Evaluation of the GestureInterface for Object Based e-Learning System. Pro-ceedings of Fourth International Conference on Innova-tive Computing, Information and Control (ICICIC),Kaohsiung, Dec 7-9, 2009, pp. 409–412.

Pasatcha P, Sunat K. 2008. The Educational SystemSupport Services Based on the Service OrientedArchitecture. Proceedings of International Symposiumon Communications and Information Technologies, Lao,Oct 21-23, 2008, pp. 215–218.

Qian X, Yu J, Dai R. 1993. A new discipline of science-The study of open complex giant system and itsmethodology. Systems Engineering and Electronics4(2): 2–12.

Romano N, Sharda R, Lucca J. 2005. Computer-supported collaborative learning requiringimmersive presence (CSCLIP): an introduction.Information Systems Frontiers 7(1): 5–12.

Shan S, Mao Z, Zhou R, Liu Z, Wu F. 2013a. Steamingmedia advertising: an empirical study. SystemsResearch and Behavioral Science 30(3): 398–411.

Shan S, Xin T, Wang L, Li Y, Li L. 2013b. Identifying in-fluential factors of knowledge sharing in emergencyevents: a virtual community perspective. SystemsResearch and Behavioral Science 30(3): 367–382.

Su M, Wong C, Soo C, Ooi C, Sow S. 2007. Service-Ori-ented E-Learning System. Proceedings of First IEEEInternational Symposium on Information Technologiesand Applications in Education, Kunming, Nov 23-25,2007, pp. 6–11.

Tan W, Wen X, Jiang C, Du Y, Hu X. 2012. An Evalua-tion Model Integrating User Trust and Capability forSelection of Cooperative Learning Partners. ChineseJournal of Electronics 21(1): 42–46.

Tan W, Xu W, Yang F, Xu L, Jiang C. 2013. A frame-work for service enterprise workflow simulationwith multi-agents cooperation. Enterprise InformationSystems 7(4): 523–542.

Tao F, LaiLi Y, Xu L, Zhang L. 2013. FC-PACO-RM: AParallel Method for Service Composition Optimal-Selection in Cloud Manufacturing System. IEEETransactions on Industrial Informatics 9(4): 2023–2033.

Tsai W, Lee K, Liu J, Lin S, Chou Y. 2012. The influenceof enterprise resource planning (ERP) systems’performance on earning management. EnterpriseInformation Systems 6(4): 491–517.

Uden L, Wangsa I, Damiani E. 2007. The future ofE-learning: E-learning ecosystem. Proceedings of Inau-gural IEEE-IES Digital EcoSystems and TechnologiesConference, Cairns, Feb 21-23, 2007, pp. 113–117.

Viriyasitavat W, Xu L, Martin A. 2012. SWSpec, ServiceWorkflow Requirements Specification Language:The Formal Requirements Specification in ServiceWorkflow Environments. IEEE Transactions on Indus-trial Informatics 8(3): 631–638.

Wan J, Jones J. 2013. Managing IT service managementimplementation complexity: from the perspective of




the Warfield version of systems science. EnterpriseInformation Systems 7(4): 490–522.

Wang S, Li L, Wang K, Jones J. 2012. e-business systemsintegration: a systems perspective. InformationTechnology and Management 13(4): 233–249.

Wang F, Ge B, Zhang L, Chen Y, Xin Y, Li X. 2013. Asystem framework of security management in enter-prise systems. Systems Research and Behavioral Science30(3): 287–299.

Warfield J. 2003. A proposal for systems science. Sys-tems Research and Behavioral Science 20(6): 507–520.

Wilamowski B. 2010. Challenges in Applications ofComputational Intelligence in Industrial Electronics.Proceedings of IEEE International Symposium on Indus-trial Electronics (IEEE ISIE 2010), Bari, Italy, July 4-7,2010, pp. 15–22.

Wilamowski B, Jaeger R, Kaynak O. 1999. Neuro-Fuzzy Architecture for CMOS Implementation. IEEETransaction on Industrial Electronics 46(6): 1132–1136.

Xia Y, Su W, Lau R, Liu Y. 2013. Discovering latentcommercial networks from online financialnews articles. Enterprise Information Systems 7(3):303–331.

Xing Y, Li L, Bi Z, Wilamowska-Korsak M, Zhang L.2013. Operations research (OR) in service industries:a comprehensive review. Systems Research and Behav-ioral Science 30(3): 300–353.

Xu L. 2000. The contribution of systems science to in-formation systems research. Systems Research and Be-havioral Science 17(2): 105–116.

Xu L. 2011a. Enterprise systems: state-of-the-art andfuture trends. IEEE Transactions on Industrial Infor-matics 7(4): 630–640.

Xu L. 2011b. Information architecture for supply chainquality management. International Journal of Produc-tion Research 49(1): 183–198.

Xu L. 2013. Introduction: systems science in industrialsectors. Systems Research and Behavioral Science30(3): 211–213.

Xu L, Liang N, Gao Q. 2008. An Integrated Approachfor Agricultural Ecosystem Management. IEEETransactions on Systems, Man, and Cybernetics Part C:Applications and Reviews 38(4): 590–599.

Xu L, Viriyasitavat W, Ruchikachorn P, Martin A. 2012.Using Propositional Logic for Requirements Verifica-tion of Service Workflow. IEEE Transactions on Indus-trial Informatics 8(3): 639–646.

Zhang D, Nunamaker J. 2004. A natural language ap-proach to content-based video indexing and re-trieval for interactive e-learning. IEEE Transactionson Multimedia 6(3): 450–458.

Zhou Z, Xiao Z, Liu Q, Ai Q. 2013. An analytical ap-proach to customer requirement information pro-cessing. Enterprise Information Systems 7(4): 543–557.




Documents

A Trust Evaluation Model for E-Learning Systems