Acceptance of TEL: Key Success Factors and Reasons of Failure

Acceptance of TEL: Key Success Factors and Reasons of Failure

PPE Summer School 2008, Ohrid, FYROM

Patrick Johnscher (TU Darmstadt)

Sebastian Kelle (Open University, Heerlen)

Steinn E. Sigurðarson (WU Vienna)

2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 2

Training & Learning

Training measures can be used simply because there is no alternative…

….but it should be about good design!


Good TEL Design

… is more than just a collection of PowerPoint slides…


So why don't we design what the users want?


Why is acceptance of TEL important?

What we just learned from Dilbert: TEL design should consider the users' requirements A system that meets expectations is more likely to gain acceptance and get used – at

least if there are learning alternatives

Common misperception: "If we build it, they will come!" E-Learning is not a field of dreams: just providing technology doesn't mean learners will

use the systems At the end of the day, TEL is no end in itself: there has to be a measurable learning

outcome, and a business case has to be made

Davis (1989): "Technology Acceptance" Information technology offers the potential for substantially improving white collar

performance But performance gains are often obstructed by users' unwillingness to accept and use

available systems Because of the persistence and importance of this problem, explaining user acceptance

has been a long-standing issue in MIS research

2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 62008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 6

A What-went-wrong Example: COVCELL

COVCELL Project – Effective language learning (http://www.covcell.org) Most important factor for language learning:

“learning through conversation” 88% of students 75.9% of teachers

So, interaction and communication is quite important?

COVCELL developed tools such as: User Presence and Chat Audio/Video Conferencing Collaborative Whiteboard Audio Recording

User acceptance as a basis for design requirement specifications


COVCELL: Percentage of users happy with…

14,3%

7,1%

28,6%

71,4%

0 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

…audio recording

…whiteboard

…audio/video conferencing

…user presence and chat

…using the wiki

…using the forum


COVCELL: User Ratings of…

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

…audio recording

…whiteboard

…audio/video chat

...user presence and chat

…wiki

…forum

Language LearningGeneral

Scale: 0-5

4.2 / 54.5 / 5

3.6 / 53.4 / 5

2.3 / 52.4 / 5

2.3 / 52.3 / 5

3.4 / 53.2 / 5

3.0 / 5

2.8 / 5


COVCELL: People didn't use…

71,4%

92,9%

78,6%

85,7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

…audio recording

…whiteboard

…audio/video conferencing

…user presence and chat

COVCELL: What went wrong?

Most users actually reported “I did not use it, but I would have liked to”

Most activities we mention here were “presence based”. The simultaneous presence of other users was required to engage in the activities, and use the tools

Users did not stay for long stretches of time in the system, and thus rarely were in the presence of other users


Some User Quotes

•I don't learning English with computers

•I did not like using computers because it decreased personalization with theteacher.

•I didn ´t like how many courses on line are badly organised and do not take the learner step by step through procedurs.

• I like being able to access material when it is conveinient for me.

• I liked: Being able to study even though I live far away from the Universityand when I was listening to lectures I could always stop and replay if I wasn'tsure of things I didn't like: Not being able to ask questions in class and notmeeting the other students.


A Few Questions…

What is "acceptance"? Let's try some definition…

Which factors influence acceptance? A closer look at acceptance models…

How can I measure acceptance? Using acceptance models in empirical research…


Acceptance – A Definition

"Acceptance of an innovation or technology describes the positive adoption decision of users – in contrast to the rejection of an innovation or technology." (Simon 2001, p. 87)

"Accordingly, the innovation-decision process is the process through which an individual or other decision-making unit passes

1. from first knowledge of an innovation,

2. to forming an attitude toward the innovation,

3. to a decision to adopt or reject,

4. to implementation of the new idea, and

5. to confirmation of this decision." (Rogers: Diffusion of Innovations, 2003, p. 161)


Acceptance – A Definition

Technology acceptance usually consists of two separate aspects: An attitude component, comprising affective and rational aspects A behavioral component, describing the adoption of an innovation in the

form of actual observable behavior (e.g. use of a system)

Since the behavioral component is usually easy to observe (and thus easy to measure), modeling and research is mostly focused on the attitude component of user acceptance:

Affective aspects, including motivational and emotional attributes Rational aspects, including cost-benefit relations as seen from the users'

individual perspective


Acceptance Models

Frequently cited models in the literature are: Technology Acceptance Model Technology Acceptance Model 2 The Unified Theory of Acceptance and Use of Technology

Basic concept underlying user acceptance models:


Technology Acceptance Model (TAM)

Developed by Davis (1989) Main idea: Perceived usefulness and perceived ease of use are

fundamental factors influencing the user acceptance as they influence the user’s attitude towards the system.


Technology Acceptance Model (TAM2)

Developed by Venkatesh and Davis (2000) Extension of the original TAM model to explain perceived usefulness and

usage intentions in terms of social influence process and cognitive instrumental processes.


Unified Theory of Acceptance and Use of Technology (UTAUT)

Developed by Venkatesh et al (2003)

Consolidation of eight prominent technology user acceptance models. The eight models reviewed are: Theory of Reasoned Action (TRA) Technology Acceptance Model (TAM) Motivational Model (MM) Theory of Planned Behavior (TPB) Combined TAM and TPB (C-TAM-TPB) Model of PC Utilization (MPCU) Innovation Diffusion Theory (IDT) Social Cognitive Theory (SCT)

The UTAUT states that four constructs play a significant role as direct determinants of user acceptance and usage behavior. They are: performance expectancy effort expectancy social influence facilitating conditions


Unified Theory of Acceptance and Use of Technology (UTAUT)


Using Acceptance Models in Empirical Research

Models are so-called "Structural Equation Models" (SEM) or "Causal Models" SEM encourages confirmatory rather than exploratory modeling; thus, it is suited

to theory testing rather than theory development. There are usually two main parts to SEM:

the structural model showing potential causal dependencies between endogenous and exogenous variables,

and the measurement model showing the relations between the latent variables and their indicators.

Benefits: User Acceptance Models like TAM, UTAUT and others are widely accepted and represent

state-of-the-art in IS research Models come with hypotheses "on board"; you don't have to formulate your own Scales and question items exist and have been tested in a number of previous studies

("tried and true") Drawbacks:

Relatively high demands on data quality Parameter estimation using covariance-based techniques can be considered advanced

statistics


Acceptance Model for TEL?


Influence Factors (Exogenous Variables): Draft Classification

Technological Organizational Didactical / Pedagogical Personal Cognitive Cultural Motivational …

Others?


Now over to you…

Form up in groups of 4-5 students

Think about factors influencing the acceptance of TEL – and reasons why TEL implementation fails (possibly also solutions presented at this summerschool?)

Try to cluster the success factors / reasons of failure into categories, e.g. "organizational", "technological"

Identify connections and dependencies between the separate factors: which factors directly influence acceptance; which influence each other (indirect influence on acceptance)?

Visualize your group results as a map on a flipchart (or laptop, e.g. with conzilla)


Literature

Venkatesh, V., Morris, M.G., Davis, F.D., and Davis, G.B.: "User Acceptance of Information Technology: Toward a Unified View," MIS Quarterly, 27, 2003, 425-478.

Venkatesh, V. and Davis, F.D.: "A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies," Management Science, 46, 2000, 186-204.

Davis, F. D., Bagozzi, R. P., and Warshaw, P. R.: "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models," Management Science, 35, 1989, 982-1003.

Goodhue, D. L.; Thompson, R. L. (1995): "Task-Technology Fit And Individual Performance", MIS Quarterly, Vol. 19, Issue 2, 213-237.

Rogers, E. M. (2003): Diffusion of innovations (5th ed.). New York: Free Press.

Nanayakkara, C. (2007): "A Model of User Acceptance of Learning Management Systems: a study within Tertiary Institutions in New Zealand", Educause Australasia 2007, http://www.caudit.edu.au/educauseaustralasia07/authors_papers/Nanayakkara-361.pdf.

Tai, L. (2008): Corporate E-Learning: An Inside View of IBM's Solutions. Oxford University Press, New York.


Construct Definitions

Attitude: Individual's positive or negative feeling about performing the target behavior (e.g., using a system).

Behavioral intention: The degree to which a person has formulated conscious plans to perform or not perform some specified future behavior.

Computer anxiety: The degree of an individual’s apprehension, or even fear, when she/he is faced with the possibility of using computers.

Computer playfulness: The degree of cognitive spontaneity in microcomputer interactions.

Computer self-efficacy: The degree to which an individual beliefs that he or she has the ability to perform specific task/job using computer.

Effort expectancy: The degree of ease associated with the use of the system. Facilitating conditions: The degree to which an individual believes that an

organizational and technical infrastructure exists to support use of the system.



Image: The degree to which use of an innovation is perceived to enhance one's status in one's social system.

Job relevance: Individual's perception regarding the degree to which the target system is relevant to his or her job.

Objective usability: A comparison of systems based on the actual level (rather than perceptions) of effort required to complete specific tasks.

Output quality: The degree to which an individual believes that the system performs his or her job tasks well.

Performance expectancy: The degree to which an individual believes that using the system will help him or her to attain gains in job performance.

Perceived ease of use: See the definition of effort expectancy. Perceived enjoyment: The extent to which the activity of using a specific system

is perceived to be enjoyable in it’s own right, aside from any performance consequences resulting from system use.

Perceived usefulness: See the definition of performance expectancy.



Perception of external control: See the definition of facilitating conditions.

Result demonstrability: Tangibility of the results of using the innovation. Social influence: The degree to which an individual perceives that

important others believe he or she should use the new system. Subjective norm: Person's perception that most people who are

important to him think he should or should not perform the behavior in question.

Voluntariness: The extent to which potential adopters perceive the adoption decision to be non-mandatory.


Business

Acceptance of TEL: Key Success Factors and Reasons of Failure