Information Systems Continuance

3 JUNE 2011

Prepared by Michael Ling Page 1

LITERATURE REVIEW

On

QUANTITATIVE RESEARCH METHODS

Limayem, M., S. G. Hirt, and C. M. K. Cheung (2007), “How Habits Limits the

Predictive Power of Intention: The Case of Information Systems

Continuance,” MIS Quarterly, Vol. 31, No. 4, 705-737.

Prepared by

Michael Ling

3 JUNE 2011


INTRODUCTION

Past research in continued usage of IS was limited to the study of initial IS

adoption, which was under the assumption that it was primarily driven by intention.

The authors recognized that this assumption had ignored the effect of frequently

performed behaviours on IS continuance.

This paper contributed to IS research by exploring the roles that IS Habit took

in the context of continued IS usage. It proposed that IS Habit had a moderating

effect on IS Continuance Intention to the extent that its effect on IS Continuance

Usage would diminish as the usage behaviour became more habitual.

Drawing from the habit literature, the IS Habit construct and its four

antecedents were developed: frequency of prior behaviour, satisfaction, stable

context and comprehensiveness of usage. PLS was employed as the research

method where three competing models were compared for the effect of IS habit on IS

Continuance Usage. The moderator model was found to possess the best

explanatory power.

3 JUNE 2011


SUMMARY

Data collection was divided into three rounds over a 4-week period to measure

university students’ usage of WWW. A total of 553 respondents answered the first

questionnaire, and 227 respondents participated in all three rounds. The first round

was to collect data for Perceived Usefulness, Confirmation, Satisfaction and IS

Continuance Intention; the second and third rounds were to measure IS Continuance

Usage. In particular, IS Continuance usage was measured by two items – frequency

of WWW usage (how often?) and intensity of usage (how many hours?).

The authors developed a six-item IS Habit scale. However, only the best

three items, which had composite reliability of 0.88, were used.

The data were analysed using PLS-Graph, which was selected for the

following reasons: (i) the formative nature of some of the measures and the non-

normality of the data; (ii) it was better suited to test moderation effects; (iii) it allowed

for small to medium-sized samples.

Regarding convergent validity, all reflective items had significant path loadings

at the 0.01 level, and acceptable levels of composite reliability (at 0.773 or above)

and average variance extracted (at 0.630 or above). The two formative items of IS

Continuance Usage had weights of 0.67 (t = 7.6) and 0.500 (t = 4.924).

Regarding discriminant validity, each construct shared greater variance with

its own block of measures than with other constructs that represented a different

block. The reflective measures fulfilled the criteria of cross-loadings.

A relatively large correlation (r = 0.751) was found between IS Continuance

Intention and IS Habit, which suggested that the measurements might have drawn

from the same construct. Nevertheless, the authors defended this point on

3 JUNE 2011


theoretical grounds and by citing similar empirical results from Towler and Shepherd

(1991-1992) and Trafimow (2000).

Regarding common method bias, LISREL were conducted on six indicators

(three from each of the IS Continuance Intention and IS Habit measures) and two

latent variables (IS Habit and IS Continuance Intention) and a method factor. The

findings showed the fit of the model did not improve significantly.

Regarding non-response bias, the demographics of respondents in the first

round, but not in the last, were compared to those who participated in all three

rounds. No significant differences were found.

Three models were tested to determine which one provided the best

explanatory power for IS Continuance Usage. A baseline model without

incorporating the IS habit construct (R2 = 0.180), a second model that modelled IS

habit as having a direct effect (R2 = 0.211) and a third model that modelled habit as a

moderator (R2 = 0.261). All path coefficients were reported significant at the 0.01

level. The hierarchical difference test showed that the interactions effect had an

effect size f of 0.063 which, according to the authors, represented a medium effect.

3 JUNE 2011


CRITIQUE

SEM was appropriate in this research as it allowed the specifications of the

relationships among the constructs and the measures underlying the constructs

concurrently, so that the measures of the construct and the hypothesized model

could be analysed simultaneously.

The selection of PLS-Graph, a component-based partial least squares

methodology, was appropriate compared to other covariance-based SEM (such as

LISREL) because PLS-Graph was better for theory development and predictive

applications.

The authors developed the antecedents of IS Habit: satisfaction, frequency of

past behavior, comprehensiveness of usage and stability of context. However,

stability of context was not used since “data are collected in only one context and we

therefore control for its impacts.” Nevertheless, the authors characterized stability of

context as “the presence of similar situational cues and goals across more or less

regularly occurring situations.” It was arguable that variations existed in universities,

just like any other social institutions, such as availability of facilities and examination

periods were likely to influence students’ usage of the WWW. As the research was

conducted over a period of four weeks, the probability that the respondents

experienced such unstable events could not be overlooked. The inclusion of the

stability construct might have increased the explanatory power of the model.

The authors defended the high correlation (r = 0.751) between IS Usage

Intention and IS Habit by citing references from theory and by making reference to

similar high correlation results previously found. Nevertheless, the high correlation

was a concern. The IS Habit measure was a new scale which, for all intents and

3 JUNE 2011


purposes, would be different from other habit measures previously used. Thus, it

was not convincing to support their correlation results with previous habit scales.

The authors could have run the model unconstrained and also constraining the

correlation between constructs to 1.0. If the two models differed significantly on a

chi-square difference test, then the two constructs would be different.

Common method variance was a type of spurious internal consistency which

occurred when the apparent correlations among indicators were due to a common

source. Since the data was based on self-reports, the correlation might be due to the

propensity of the subjects to answer similarly to multiple items even when there was

no true correlation of constructs. LISREL test concluded that common method

variance was not an issue.

Convergent validity could be assessed in several ways: (i) the correlations

among items which made up the scale – internal consistency validity; (ii) the

correlations of the given scale with measures of the same construct using scales

proposed by other researchers and, preferably, already accepted in the field –

criterion validity; (iii) the correlations of relationships involving the given scale across

samples or across methods. The results of Cronbach’s alpha and the average

variance explained (AVE) provided evidence for internal consistency construct

validity. The authors demonstrated criterion validity for Perceived Usefulness,

Confirmation, Satisfaction and IS Continuance Intention by referring to scales that

had been validated in prior research. The authors developed a six-item habit scale

and used the best three items in this research but fell short of providing detail for the

decision. It would be helpful if the new habit scale were to be compared against

3 JUNE 2011


previously developed habit scales. All the constructs were not tested for convergent

validity using cross samples or methods.

Discriminant validity referred to testing statistically whether two constructs

were different. Evidence was provided for discriminant validity, as below: (i) the item

loadings were higher for their corresponding constructs than for others; (ii) the square

root of the AVE for a given construct was greater than the correlations between it and

all other constructs.

The authors did not provide any reference to content or face validity. It was a

concern that whether the items measure the full domain implied by their label. The

indicators might exhibit construct validity, yet the label attached to the concept might

be inappropriate. Use of surveys or panels of content experts or focus groups were

methods in which content validity might be established.

Internal validity had not been adequately addressed by the authors. The

number of respondents participated in the three rounds was different – 553 in the first

round and 227 in all three rounds. It was not clear what sample size was used in the

model testing. The authors did not address the issue of mortality bias, which was an

obviously important issue here. For example, was there an attrition bias?

Another internal validity issue that had not been addressed was compensatory

rivalry. As the data collection took three weeks, the students might have promoted

competitive attitudes that could have biased the results.

The latent constructs that were associated with reflective measurement items

were Confirmation, Habit, IS Continuance Intention, Perceived Usefulness and

Satisfaction. The loadings of the reflective items were reported significant.

3 JUNE 2011


Overall Assessment

The choice of PLS-Graph was appropriate. The derivation of a new scale for

IS Habit was a significant contribution to IS research. The authors obtained the

highest R2 in the IS Habit moderated model against the baseline and the direct effect

models. Though the R2 value (0.261) of the moderating model was low, the

conclusion that the moderating model had the best explanatory power was correct.

The exclusion of the stability context antecedents in the IS Habit construct might

have reduced the variance explained by the model. The high correlation between IS

Usage Intention and IS Habit was a potential concern. Convergent validity,

discriminant validity and common method bias were largely in order. Content validity

and internal validity were not adequately addressed. On balance, there were more

strengths than weaknesses in the paper.

3 JUNE 2011


CONCLUSION

The key contribution of the paper rested on the scale development of the IS

Habit construct and the finding that there was moderating effect of IS Habit on IS

Continuance Intention and IS Continuance Usage.

The choice of the component-based PLS model, PLS-Graph, was appropriate

for the analysis. Three competing models were compared and the moderating model

was found to have the highest explanatory power. All loadings and weights of the

indicators were acceptable.

The research could have improved by addressing the concerns raised here.

In particular, further developed measurement scale of IS Habit; the inclusion of

stability in the model; consideration of interactions effect between Satisfaction and

Comprehensiveness of Usage and Frequency of Behavior.

Business

Information Systems Continuance