The Evaluation of Information Systems Success: A new ... Evaluation of Information... · Web viewInformation System Success (IS), Alignment, Fit, National culture, Business Strategy,

6th Global Conference on Business & Economics ISBN : 0-9742114-6-X

The Evaluation of Information Systems Success: a New Perspective

Hafid Agourram

King Fahd University of Petroleum & Minerals, Saudi ArabiaEmail: [email protected]

(966) 3 860 2678

Amine Nehari Talet

King Fahd University of Petroleum & Minerals, Saudi Arabia. Email: [email protected]

Mailing address: P O. Box 805King Fahd University of Petroleum & Minerals, Dhahran 31261Saudi Arabia.

Key words: Information System Success (IS), Alignment, Fit, National culture, Business Strategy, Information System Strategy

Acknowledgement: We would like to thank King Fahd University of Petroleum and Minerals and specifically the College of Industrial Management for their generous supports.

OCTOBER 15-17, 2006GUTMAN CONFERENCE CENTER, USA1

mailto:[email protected]

mailto:[email protected]


Abstract

Information system success has been a challenging concept for researchers for many

years. Research on the definition and the evaluation of information system (IS)

success was growing yet confusing. The social nature of IS success concept is a

major factor that may explain this confusion. From the global and multicultural

perspectives, a recent study showed that the meaning of IS success is not

homogenous across different national cultures. Organizations are more and more

attracted by doing business in different cultures and become global. This attraction is

also yielded by the successful technical capabilities of the new information

technologies that support the globalization strategy. The existing models of IS

success require a large amount of modifications and adaptations before they can be

applied in local and international organizations. This paper proposes a

comprehensive framework that is easy to use, integrates existing knowledge on the

alignment and IS success domains and can be applied to any local and national

context. The power of the framework is in its practicality and flexibility.

Introduction

The use of information technology and information systems in today’s organization

is growing in a phenomenon way. The advanced sophistication of the new

information technologies combined with their yielding benefits are major factors

that justify their massive use in almost every type of organization. Morton (1991)

reported that the rate at which new technology is introduced is increasing 20 to 30

percent annually and the numbers are promising to grow in the future. However, IS



investment decision makers are confused and frustrated because they are unable and

find it very difficult to justify their IT and IS investments by using existing tools of

IS performance and success. Researchers have developed many IS success models to

help managers measure the performance of their IS (DeLone and McLean, 1992,

2002; Seddon, 1997; Robey and Boudreau, 200.). The major problems with these

models are: First, they cannot be applied to all contexts and to all types of

information systems. Second, these models show how to evaluate IS success

independently from other organizational variables such as structure, strategy, and

processes. Lastly, the values of these measurements may sometimes lead to

confusion because they are not compared to a set of predefined expected values that

define success to the organization. For example, it may be that an organization

evaluated the success of its information system by using the impact of the system on

the decision making process dimension and found that this impact is average. Is that

a success or not? The answer is we don’t know because we luck a predefined level

of the impact of the IS on its users to which we can compare the actual value. It has

been largely documented that the organization and its information systems must fit

each other; that is, one cannot be separated from the other. In this paper, we use this

parallel philosophy between the organization and its IS to develop a very

comprehensive process framework that uses existing knowledge on the fit theory.

The framework can be used in any national and organizational culture and is very

practical for managers who wish to evaluate IS success. The following sections will

discuss existing IS success theory, the fit theory, our proposed framework, and the

benefits of the framework.



Information Systems success

Information system (IS) success and its determinants have long been considered

critical to the field of information systems. Despite many attempts to model success

(Delone and McLean, 1992, 2003), IS success definition and measure is still

problematic for many factors. The first factor is the mixture of the technical and

social aspects of an IS. IS success is then a perspective that emerges from the social

and technical interplay within organizations (Kanellis, Lycett and Paul, 1998).

Second, Alter (2000) argues that information technology and work practices are now

so intertwined that it is difficult to identify their respective contribution to success.

Other researchers link the difficulty of defining IS success to the methodological

aspects involved in measuring IS success «Specifying a dependent variable is

difficult because of the many theoretical and methodological issues involved in

measuring IS success» (Garrity and Sanders; 1998, p. 14). Seddon, Staples,

Patnayakuni, and Bowetell, (1999) argue that IS success is a fuzzy concept

contingent upon different stakeholders and different types of IT. In the practice

community, Markus and Tanis (2000) claimed that there is a fundamental gap in

both practical and academic thinking about information systems lack of consensus

and clarity concerning the meaning of success where information systems are

concerned.

The problem of IS definition and measurement becomes more difficult and more

complicated if we add the international dimension (Ishman, 1998; Garrity and

Sanders,1998, Pauleen et al., 2006). The international dimension includes cultural



terms such as values and assumptions which may be at the heart of the differing

perceptions and interpretations of IS success. Shing-Kao (1997) argues that

«Research has shown that people notice, interpret and retain information based on

their values, assumptions and expectations. Different assumptions and values lead to

different ways of looking at the same thing» (p. 13).

This international differing perception of the meaning of a phenomenon is an

important topic in International Management or Cross-cultural Management

disciplines. Do theories and concepts born in a specific culture apply or have the

same meaning in other cultures? Hofstede (1993), for example, after a large survey

on work-related values in sixty countries, concluded that management theories and

findings are not automatically transferable from one context to another. Shing-Kao

(1997), Kedia and Bhagat (1988), and Robichaux and Cooper (1998) add that the

majority of theories of management have a Western orientation.

Rosenzweig (1994) argues that a central concern in scientific research is external

validity. That is, the extent to which a theorized or observed relationship among

variables can be generalized to other settings. Rosenzweig (1994) claims that the

main question should not be, «Are scientific management theories that interest us

valid elsewhere? But how can we best understand management, as it exists around

the world? » (p.37).

Researchers agreed that the measurement of IS success is not an easy task. The

major problem with existing IS success model is that, first, they are very abstract

and don’t rend service to managers who are always searching for more practical

tools and techniques. Second, they are used independently from the organizational



and national contexts. The organizational context has been largely documented by

strategic alignment researchers such as Henderson, Venkatraman and Luftman and

others who built a large body of knowledge that stressed the necessity to align and to

fit IS and organizational strategies. On the other hand, the national context was

always ignored by existing IS success models. Agourram (2004) conducted a cross

cultural research that involves France, Germany, and Canada and found that IS

success is perceived and defined differently in these cultures. Therefore, concludes

the author, existing IS success models cannot be applied as they are in different

national contexts. The following section discusses the alignment concept.

The alignment theory

Alignment of business and information strategies referred to the extent to which

business strategies were enabled, supported, and stimulated by information

strategies. Evidence for alignment was sought in the use of information or

information technology, or both, which provided a comparative advantage to an

organization over its competitors. The importance of aligning the objectives and

strategies of an organization's information systems (IS) group with those of the

broader organization has been recognized for some time (Lederer and Mendelow,

1986; Henderson, 1990; Chan and Huff, 1993; Henderson and Venkatraman, 1996;

Luftman, 1996; Kearns and Lederer, 2000; Reich and Benbasat, 2000; Chan, 2002),

and is usually referred to as strategic alignment. The early research into strategic

alignment tended to be theoretical in nature (Henderson and Venkatraman, 1990)

providing the platform on which later work was developed. Based on these early



developments, a number of dimensions of strategic alignment have emerged. The

social dimension of strategic alignment has been defined as "the state in which

business and IT executives within an organizational unit understand and are

committed to the business and IT mission, objectives, and plans" (Reich and

Benbasat, 2000, p. 82). The intellectual dimension of strategic alignment is defined

as "the state in which a high-quality set of interrelated IT and business plans exist"

(Reich and Benbasat, 2000, p. 82). Put simply, research into the social dimension

tends to "... focus on the people involved in the creation of alignment" whilst the

intellectual dimension of alignment tends to "... concentrate on the content of plans

and planning methodologies" (Reich and Benbasat, 1996). This concentration has

naturally led to research at the CIO/CEO level, at which planning methodologies are

chosen and strategies developed. Furthermore the little research that has been

conducted on the social dimension of alignment has also been focused at this level

(Reich and Benbasat, 1996, 2000; Chan, 2002). There is evidence to suggest that

strategies developed at senior management level are often modified or even

sabotaged at lower levels of the organization during implementation (Davies, 1993;

Alien and Wilson, 2003; Nordstrom and Soderstrom, 2003). Chan and Huff (1992)

have also made the observation that strategy is developed at executive level but

essentially implemented at lower levels of an organization, with the consequence

that greater attention needs to be paid to the dynamics of alignment at lower levels

of the organization.

In studying strategic information systems (IS) alignment, it is important to examine

the associated processes over time rather than viewing alignment as an isolated



event. Organizations experience a series of interdependent changes in business

strategy and Information System strategy which impose to increase the alignment

between them. Alignment is a dynamic and emergent concept because of the

ongoing change in business and information strategies. As business strategy

changes, IS strategy must change in parallel. However, moving from one pattern of

alignment between business and IS strategies turns out to be a rather difficult

proposition. Consequently, organizations struggle to bring IS and business strategies

into alignment and sometimes go through potentially problematic trajectories, where

the business strategy changes but one (or more) of the IS strategy components fails

to change appropriately.

The strategic use of information systems has been a fundamental concern for every

organization. IS are used today to help organization implement and apply their

strategies. They are used as major tools that help organizations gain competitive

advantages over their competitors. Researchers such as Henderson and Venkatraman

(1993) and Luftman (1993, 1998) have always stressed the fact that the realization

of these competitive advantages is conditioned by the strategic use of IS. The

alignment of organizational strategy, objectives and goals with those of IS, enhance

the degree of the benefits these organizations can gain. Venkatraman (1993)

developed the alignment model which is defined in terms of four constructs:

business strategy, information technology strategy, organizational infrastructure and

processes, and information technology infrastructure and processes. The model

explains two fundamental characteristics of strategic management: strategic fit and

function integration.



Greater alignment or "fit" between an organization's business strategy and IS

strategy implies that the information systems are targeted on areas that are critical to

successful business performance. "Alignment between business and IS strategies

heightens managers' awareness and use of information systems, and it enables a firm

to better use IS to help realize its goals and objectives or obtain a competitive

advantage. Weak fit is problematic in two ways. First, it may create competing

motivations for different people in different parts of the organization as with the

tension between strategy and structure. Second, in the absence of a unifying theme

or logic arising from tight fit, organizational players may be confused as to the

appropriate decisions and behaviors in respect to achieving existing operational

performance targets." (Rudy Hirschheim, Rajiv Sabherwal 2001).

. The following section discusses our framework that uses Venkatraman theory.

The new IS success framework

Figure 1 shows our proposed framework. The framework is a process and not a

variant framework. The framework uses existing concepts from the alignment

theory. The framework is a solution to many IS success evaluation problems. Its

benefits will be discussed in the last section.


http://proquest.umi.com/pqdweb?index=2&did=97890138&SrchMode=1&sid=2&Fmt=4&VInst=PROD&VType=PQD&CSD=20021&RQT=590&VName=PQD&TS=1158209845&clientId=29075




The framework shows that IS success or IS performance is a process which is made of the

following phases:

1. In phase 1, the strategic orientation of the organization is established. This document

includes the directions and the plans as well as the goals of the organization. Parallel

to the strategic orientation of the organization, IS people establish the strategic

orientation of the information systems. These two processes are done at the same time

and the two must fit each other. (Jerry Luftman, 1993) has extensively reported

research work that discusses this first process.

2. Phase 2 includes two steps that are also executed in parallel. In the first step and after

having a clear idea on the direction of the organization, managers need to document

the expected outcomes of this direction; that is the expected performance of the

organization. At the same time, IS professionals would document the expected

performance of their information systems; that is IS success. IS expected performance

must be aligned with the expected organizational performance. Moreover, the

organizational expected performance determines the IS expected performance. If for

example management consider that the business processes of the organization must be

fluid and employ less people as one expected organizational performance element, this

would push IS professionals to consider maximum automation of business processes

as an element of the expected IS performance.

OCTOBER 15-17, 2006GUTMAN CONFERENCE CENTER, USA 11


3. In phase 3, business people implement organizational structures and procedures and

design the new processes to help realize the expected organizational performance. At

the same time, IS people implement the information systems that would help realize

the expected IS performance. The implementation components and logic must fit the

organizational infrastructures and procedures. In this step, and in case information

systems are acquired and not developed in house, the acquired system built-in

assumptions are taken into consideration in the IS implementation step.

4. Phase 4 does not execute right away. It would take few years before we start actions

in phase 4. In this phase, we compare the actual organizational performance with the

expected organizational performance, and we compare the actual IS performance with

the expected IS performance. Comparative research techniques can be used in this

phase. In this phase, we don’t need to search for organizational and IS performance

models and see it we can apply them to our organization. In this phase, the actual

organizational performance is the DEGREE to which the actual organizational

performance matches the expected organizational performance, and the actual IS

performance is the DEGREE to which the actual IS performance matches the expected

IS performance. If case the gaps between the expected and the actual performances is

large, control is sent back to the first phase.



Conclusions

Our framework yields to the following benefits:

It pushes people to think about the outcomes and the results before the actual

implementation of the systems. This is a major exercise that helps convince decision

makers to invest in information systems,

The model requires a continuous fit and alignment between the organization and the

information system which has been proved by research to be an excellent exercise, and

When it comes to evaluating the performance, we don’t need to borrow from

very theoretical model and see if we can apply them to the actual context. The variety

of information systems requires a dynamic IS evaluation system. Moreover, this phase

which used to be the hardest and the most researchable becomes the easiest. We just

compare what we have with what we expected. If the gap is large, actions can be easily

implemented to reduce it and hence ameliorate the actual IS success.



References

1. Alter, S. (2000). The Siamese Twin Problem: A Central Issue Ignored by Dimensions of Information Effectiveness. Communication of AIS. 2(20), 1-55.

2. Agourram, H, :The Impact of National Culture on The Meaning of Information Systems Success, D.B.A Dissertation, University of Sherbrooke, May 2004.

3. Alien, D. and Wilson, T., "Vertical trust/mistrust during information strategy formation", International Journal of Information Management, Vol. 23, pp. 223-37. 2003

4. Chan, Y.E., "Why haven't we mastered alignment? The importance of the informal organization structure", MIS Quarterly Executive, Vol. 1 No. 2, pp. 97-112. 2002

5. Chan, Y.E. and Huff, S.L., "Investigating information systems strategic alignment", Proceedings of the 14th International Conference on Information Systems, Orlando, Florida. 1993.

6. Davies, R., "Making strategy happen: common patterns of strategic success", European Management Journal, Vol. 11 No. 2, pp. 201-13. 1993.

7. DeLone, W.H and McLean, E.R. (1992). Information Systems Success: The Quest for The Dependent Variable. Information Systems Research, 3(1), 60-95.

8. DeLone, W.H and McLean, E.R (2003). The Delone and McLean model of Information systems success: A ten-year update. Journal of Information Systems Management, 19(4), 9-26.

9. Henderson, J.C and Venkatraman, N, Strategic alignment: Leveraging Information technology for transforming organizations, IBM Systems Journal, v32, no 1, 1993

10. Garrity, E.J. and Sanders, G.L. (1998). Introduction to Information Systems Success Measurement. In E.J. Garrity and G.L, Sanders (dir), Information Systems Success Measurement (p. 1-12). Idea Group Publishing.

11. Henderson, J.C. , :"Plugging into strategic partnerships: the critical IS connection", Skan Management Review, Vol. 3, pp. 7-18. 1990.

12. Henderson, J. C., & Venkatraman, N. (1994). Strategic alignment: a model for organizational transformation via information technology. In T. J. Allen & M. S. Scott Morton (Eds.), Information technology and the corporation of the 1990s (pp. 202–220). New York: Oxford University Press.

13. Henderson, J.C. and Venkatraman, N.,: "Aligning business and IT strategies", in Luftman, J.F. (Ed.), Competing in the Information Age: Strategic Alignment in Practice, Oxford University Press, New York, NY, pp. 21-42. 1996.

14. Hofstede, G. (1993). Cultures and organizations: software of the mind. London, McGraw-Hill.

15. Ishman, M. (1998). Measuring Information Success at the individual level in cross-cultural Environments. In E.J. Garrity and G.L, Sanders (dir), Information Systems Success Measurement. Idea Group Publishing.



16. Jerry N. Luftman, Paul R. Lewis, Scott H. Oldach: Transforming the Enterprise: The Alignment of Business and Information Technology Strategies. IBM Systems Journal 32(1): 198-221 (1993)

17. Kanellis, P., Lycett, M. and Paul, R.J. (1998). An Interpretive Approach to the Measurement of Information System Success: From Concept to Practical Application In E.J. Garrity and G.L, Sanders (dir), Information Systems Success Measurement (p. 133-151). Idea Group Publishing.

18. Kearns, G.S. and Lederer, A.L., :"The effect of strategic alignment on the use of IS-based resources for competitive advantage", Strategic Information Systems, Vol. 9, pp. 265-93. 2000

19. Kedia, B.L. and Bhagat, R.S. (1988). Cultural Constraints on Transfer of Technology Across Nations: Implications for Research in International and Comparative Management. Academy of Management Review, 13(4), 559-571.

20. Lederer, A.L. and Mendelow, A.L.: "Co-ordination of information systems plans with business plans", Journal of Management Information Systems, Vol. 6 No. 2, pp. 5-19. 1986

21. Luftman, J, Lewis, P.R, Oldach, S.H, Transforming the enterprise: the alignment of business and information strategies. IBM Systems Journal, v32, n1, 1993

22. Luftman, J., :"Applying the strategic alignment model", in Luftman, J. (Ed.), Competing in the Information Age: Strategic Alignment in Practice, Oxford University Press, New York, NY, pp. 43-69. 1996.

23. Luftman, J. Assessing IT/business alignment, Information Systems in Management. v20, i4, 2003.

24. Lyftman, J, Enablers and Inhibitors--- Despite Working on Different Levels, IT and Business Executives Must Communicate Better In Order For Companies to Experience the Benefits of Technology, Information week, Sept 14, n700, 1998.

25. Markus, M.L and Tanis, C. (2000). The Enterprise System Experience – From Adoption to Success. In R. Zmud (dir.), Framing the Domain of IT Management (p. 173-207). Pinnaflex.

26. Nordstrom, T. and Soderstrom, M., "Study of implementing an IT-impregnated corporate strategy", paper presented at the llth European Conference on Information Systems, Naples, 16-21 June. 2003

27. Pauleen, D., Evaristo, R., Davison, R., Ang, S., Alanis, M., and Klein (2006), "Cultural Bias in Information Systems Research and Practice: Are You Coming from the same place I am?" Communications of the AIS, 17, 354-372.

28. Reich, B.H. and Benbasat, I. , "Measuring the linkage between business and information technology objectives", MIS Quarterly, Vol. 20 No. 1, pp. 55-81. 1996

29. Reich, B.H. and Benbasat, I., "Factors that influence the social dimension of alignment between business and information technology objectives", MIS Quarterly, Vol. 24 No. 1, pp. 81-113. 2000.

30. Robichaux, B.P. and Cooper, R.B. (1998). GSS Participation: A Cultural Examination. Information and Management, 33, 287-300.


http://ftp.informatik.uni-trier.de/~ley/db/journals/ibmsj/ibmsj32.html#LuftmanLO93

http://ftp.informatik.uni-trier.de/~ley/db/indices/a-tree/o/Oldach:Scott_H=.html

http://ftp.informatik.uni-trier.de/~ley/db/indices/a-tree/l/Lewis:Paul_R=.html


31. Rosenzweig, P.M. (1994). When Can Management Science Research Be Generalized Internationally? Management Science, 40 (1), 28-39.

32. Rudy Hirschheim, Rajiv Sabherwal : Detours in the path toward strategic information systems alignment, California Management Review. Berkeley: Vol.44, Iss. 1; pg. 87, 23 pgs Fall 2001.

33. Seddon, P.B., Staples, S., Patnayakuni, R. and Bowetell, M. (1999). Dimensions of Information Success. Communication of the ACM. 2(20), 1-40.

34. Shing-Kao, L. (1997). A study of National Culture versus Corporate Culture in International Management. Dissertation abstract international, Nova South-eastern University.

35. Venkatraman, N., Henderson, J. C., & Oldach, S. : ‘Continuous Strategic Alignment: Exploiting Information Technology Capabilities for Competitive Success’, European Management Journal, vol. 11, no. 2, pp. 139 – 149. 1993


http://proquest.umi.com/pqdweb?RQT=572&VType=PQD&VName=PQD&VInst=PROD&pmid=27871&pcid=1556449&SrchMode=3

http://proquest.umi.com/pqdweb?RQT=318&pmid=27871&TS=1158209845&clientId=29075&VType=PQD&VName=PQD&VInst=PROD

http://proquest.umi.com/pqdweb?index=2&did=97890138&SrchMode=1&sid=2&Fmt=4&VInst=PROD&VType=PQD&CSD=20021&RQT=590&VName=PQD&TS=1158209845&clientId=29075

Documents

The Evaluation of Information Systems Success: A new ... Evaluation of Information... · Web viewInformation System Success (IS), Alignment, Fit, National culture, Business Strategy,