Enterprise COTS Implementation: Process, Stakeholder, and ... · CSF during implementation. Applying stakeholder theory and control theory can help develop new perspectives in CSF

Enterprise COTS Implementation: Process, Stakeholder,

and Control Perspective

by

Zafor Ahmed, PMP

A thesis submitted to the Faculty of Graduate and Postdoctoral

Affairs in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

in

Management

,

Carleton University

Ottawa, Ontario

©2018

Zafor Ahmed

Page 2 of 307

Abstract

Cutting the cost of information technology (IT) spending is an ongoing initiative not only in private

industry, but also in national governments in an effort to balance their budgets. One key dimension

of this cost reduction effort is in optimizing procurement policies where buying and reusing are

prioritized. In the case of IT systems, buying generally means purchasing and implementing

commercially available packaged solutions, commonly known as COTS solutions.

COTS implementation projects, often part of an organization’s strategic IT initiatives, require a

large investment of organizational resources, yet often fail to achieve desired benefits. Facing a

persistently high failure rate, identifying IS implementation-related critical factors appears to be

essential for both theory and practice. Although knowledge about critical success factors (CSF) is

quite mature, an absence of benefits realization suggests a lack of effective management of relevant

CSF during implementation. Applying stakeholder theory and control theory can help develop new

perspectives in CSF research as they center around the concepts of “whose interest matters” and

“how to govern them,” respectively. The current research investigates public sector enterprise

COTS implementation from multiple perspectives, such as implementation processes, engagement

of stakeholders, and control balancing, and how these factors relate to implementation-specific

CSF. This enhances our current understanding of implementation processes, stakeholder

engagement, and control balancing related to enterprise COTS implementations.

The current research draws from control theory, control balancing theory, stakeholder theory,

stakeholder agency theory, CSF literature, and IS-implementation literature to develop an

argument that asserts an integral relationship among processes, stakeholder engagement, control

balancing, and CSF. In addition to proposing a new and vital construct—stakeholder orientation—

micro-level and macro-level models for enterprise COTS implementation are identified. Control

theory and control balancing theory is then used to capture and validate the dynamic nature of

control configurations during IS implementation. Stakeholder theory and stakeholder agency

theory is further applied to capture the dynamic stakeholder orientation in a IS implementation

project. Finally, through analysis and synthesis, the current research establishes vital relationships

among processes, control configurations, stakeholder orientations, and CSF, where proper CSF

management leads to a successful COTS-implementation project outcome.

Page 3 of 307

ACKNOWLEGEMENTS The completion of this thesis is the culmination of many years of effort. While the work described

in this document is mine, particularly the shortcomings, I could not have accomplished any of it

without considerable support from a number of people and organizations.

I would like to take the opportunity to express my profound gratitude to Dr. Vinod Kumar and Dr.

Uma Kumar, my co-supervisors, for their continuous and dedicated support, advice, and help in

completing this research. Their insightful guidance and hospitality, throughout my PhD journey,

have been immense and immeasurable and I could not have imagined having a better mentor for

my PhD studies.

I also wish to thank all the other members of my Thesis Committee, in particular, Dr. Alan Cai

and Dr. Greg Sears for this major undertaking. I am very grateful and indebted to both for agreeing

to be part of the examining committee, and for providing valuable comments, advice, and

suggestions in significantly improving this research.

I wish to express my sincere gratitude to the Sprott School of Business, Carleton University, for

the benefits that I gained during my studies at the School. I also wish to thank the Administrative

Staff at the Sprott School of Business. In particular, I would like to express my sincere appreciation

to Melissa Doric for her dedicated support.

I am grateful to my parents for their support, love and encouragement. My parents always

reinforced the idea from the early years on that a solid education is something that can never be

taken away. One of the key motivator for me in pursuing a Ph. D. was my mother and for this, I

dedicate this thesis to my mother – Begum Selina Khandakar.

The responsibility for errors and omissions rests with the author

Zafor Ahmed

Page 4 of 307

Contents

1.0 Introduction .......................................................................................................................................... 8

1.1 Research motivation ........................................................................................................................ 9

1.2 Research objectives ........................................................................................................................ 15

1.3 Research questions ......................................................................................................................... 16

1.4 Case study approach ..................................................................................................................... 17

1.5 Organization of this thesis ............................................................................................................ 19

2.0 Literature review ................................................................................................................................ 21

2.1 Selection process for articles ......................................................................................................... 22

2.2 COTS and IS implementation ....................................................................................................... 23

2.2.1 An agile implementation........................................................................................................ 34

2.2.2 The agile manifesto and agile values ................................................................................... 35

2.3 Stakeholders and stakeholder engagement ................................................................................ 37

2.4 Control configuration and control balancing ............................................................................. 45

2.5 Critical success factors (CSF) ........................................................................................................ 49

3.0 COTS implementation framework .................................................................................................. 62

3.1 Enterprise COTS and its implementation ................................................................................... 63

3.2 Stakeholder engagement in COTS implementation .................................................................. 69

3.2.1 A Framework towards stakeholder engagement ............................................................... 71

3.2.2 Stakeholder sensitivity and stakeholder engagement ....................................................... 73

3.3 Control balancing in COTS implementation .............................................................................. 77

3.4 Mapping the research questions and objectives ........................................................................ 81

4.0 Research methodology ...................................................................................................................... 84

4.1 Overall approach ............................................................................................................................ 84

4.2 Research design .............................................................................................................................. 84

4.3 Research method ............................................................................................................................ 88

4.3.1 Review literature ..................................................................................................................... 89

4.3.2 Synthesize literature ............................................................................................................... 90

4.3.3 Map research questions .......................................................................................................... 91

4.3.4 Design interviews.................................................................................................................... 91

Page 5 of 307

4.3.5 Select sample ............................................................................................................................ 92

4.3.6 Conduct interviews ................................................................................................................. 96

4.3.7 Analyze data ............................................................................................................................ 97

4.4 Reliability and validity of the cases ........................................................................................... 104

4.4.1 Reliability ............................................................................................................................... 104

4.4.2 Validity ................................................................................................................................... 104

5.0 Case results and discussion ............................................................................................................ 106

5.1 Case A: Replacing an enterprise system for financial market operations ........................... 106

5.1.1 Implementation process for Case A ................................................................................... 109

5.1.2 Control configuration and balancing ................................................................................. 114

5.1.3 Stakeholder engagement and sensitivity ........................................................................... 120

5.2 Case B: Replacing a shared system for financial regulations ................................................ 125

5.2.1 Implementation process for Case B .................................................................................... 129



5.3 Case C: Financial billing systems replacement ........................................................................ 144

5.3.1 Implementation process for Case C .................................................................................... 147



5.4 Case D: Enterprise content management (ECM) ..................................................................... 164

5.4.1 Implementation process for Case D ................................................................................... 168



6.0 Cross-Case Analysis and Synthesis ............................................................................................... 186

6.1 Enterprise COTS Implementation Model ................................................................................. 186

6.2 Control configurations and control balancing ......................................................................... 211

6.3 Stakeholder orientations ............................................................................................................. 229

6.3.1 Responsibility dimension (normative orientation) .......................................................... 233

6.3.2 Paternalism dimension (normative orientation) ............................................................... 235

6.3.3 Neoclassical dimension (instrumental orientation) ......................................................... 237

6.3.4 Strategic dimension (instrumental orientation) ................................................................ 241

Page 6 of 307

6.4 Processes, control balancing, stakeholder orientation and CSF ............................................ 246

7.0 Conclusion ........................................................................................................................................ 259

7.1 Conclusion .................................................................................................................................... 259

7.2 Summary of answers to Research Questions ........................................................................... 260

7.2.1 What is an activity- and process-based model for enterprise COTS implementation?

.......................................................................................................................................................... 260

7.2.2 How can organizational processes and tools contribute to COTS implementation

success? ............................................................................................................................................ 262

7.2.3 What is the nature of control configuration in a multi-partner COTS implementation

and what are the factors responsible for the application of control balancing? ................... 264

7.2.4 How can CSF and other challenges be successfully managed through optimal control

balancing by the project? ............................................................................................................... 265

7.2.5 What is the nature of stakeholder orientation during a COTS implementation for a

public sector IS implementation? ................................................................................................. 266

7.2.6 How can CSFs and other challenges be successfully managed by the project through

optimal stakeholder engagement? ............................................................................................... 268

7.3 Implications for theory ................................................................................................................ 269

7.4 Implications for Practice.............................................................................................................. 272

7.5 Limitation of Current Research ............................................................................................ 275

7.5.1 Firms sampled and generalizability ............................................................................ 275

7.5.2 Interview method ........................................................................................................... 275

7.5.3 Granularity of the stakeholder orientation framework ............................................ 276

7.6 Future Research Directions ................................................................................................... 276

7.6.1 Agile implementation process of enterprise COTS implementation ...................... 276

7.6.2 Trigger factors for shifting stakeholder orientations for COTS implementation .. 277

7.6.3 Enacting Clan control and self control on the development team .......................... 277

7.6.4 Effectiveness of stakeholder orientation and balancing isomorphic pressure ...... 278

References ............................................................................................................................................... 280

Appendix A – Research Approval ....................................................................................................... 291

1. Ethics Committee Approval ..................................................................................................... 291

2. Organizational Approval .......................................................................................................... 292

Appendix B - Interview Guide ............................................................................................................. 292

1. Email to Potential Interview Candidates ................................................................................ 292

Page 7 of 307

2. Informed Consent ...................................................................................................................... 293

3. Interview Protocol ...................................................................................................................... 295

Appendix C – Informant List and Details .......................................................................................... 295

1. Case A Informant list: ................................................................................................................ 295

2. Case B Informant list: ................................................................................................................ 296

3. Case C Informant list: ................................................................................................................ 296

4. Case D Informant list: ................................................................................................................ 297

Appendix D – Strategic Investment Gating Process ......................................................................... 298

Appendix E ............................................................................................................................................. 302

Appendix F: Acronym Table ................................................................................................................ 302

Appendix G: Data Collection Instrument ........................................................................................... 303

Page 8 of 307

1.0 Introduction

Deployment of COTS (Commercial Off The Shelf) as enterprise IT (Information Technology)

have steadily increased over last two decades, ranging from multi-million dollar ERP

implementation to a stand-alone packaged software for individual users. This trend is apparent in

both the public and the private sectors where reduction of IT expenditure and speed of deployment

are two of the salient drivers of IT procurement decisions (Chidley, 2014). Cutting the cost of

information technology (IT) spending is an ongoing initiative not only in private industry but also

at the federal government level because governments in different countries are trying to cut costs

to balance their budget. For example, in the attempt to cut IT spending across various departments,

the Canadian government has implemented IT streamlining initiatives that include the

consolidation of data centers across departments and the reduction of various redundant systems

(Data Center Consolidation Initiatives, 2015). One of the key dimensions of this cost reduction

effort is optimizing the procurement policies where buying and reusing (as opposed to building)

are prioritized. In the case of computer software, buying generally means purchasing and

implementing commercially available packaged solutions, commonly known as COTS solutions.

Often these COTS implementation projects are part of an organization’s strategic IT initiatives

that require a large investment of organizational resources, yet they fail to achieve the desired

benefits. This should not be surprising as the failure rate of information systems (IS) development

and implementation projects are still disturbingly high. Depending on the source, this failure rate

can be found to be as high as 70% (Doherty, Ashurst, and Peppard, 2011). Although a substantial

amount of research has been conducted to identify technological and organizational factors that

should be managed properly to successfully complete the projects and deliver the expected

Page 9 of 307

benefits, most of this effort is focused on the domain of well-known COTS (Cecez-Kecmanovic,

Kautz, and Abrahall, 2014). Furthermore, given the persistence of such high failure rates, benefit-

realization from identifying these IS implementation related critical factors are largely absent in

practice (Cobb, 1996; Doherty, Ashurst, and Peppard, 2011). Since the literature and knowledge

on critical success factors (CSF) are quite mature, an absence of benefits realization can be

explained through a lack of effective management of those identified CSF. Although a significant

percentage of the CSF literature primarily attempts to identify specific CSF related to different IS

implementation or context, a closer look reveals a ‘people’ aspect related to most of the identified

CSF. Therefore, employing theoretical lenses of ‘stakeholder theory’ and ‘control theory’ can help

develop new perspectives in a CSF research as they center around the concepts of ‘whose interest

matters’ and ‘how to govern them’ respectively.

The purpose of this thesis is to investigate public sector enterprise COTS1 implementation from

multiple perspectives such as implementation processes, engagement of stakeholders, control

balancing, and how these factors relate to implementation specific critical success factors (CSF).

This will significantly contribute to enhancing our current understanding of implementation

processes, stakeholder engagement, and control balancing related to COTS implementations.

1.1 Research motivation

COTS implementation differs from traditional software development projects or enterprise

systems implementation projects in several key aspects, such as the required level of functional

1 Enterprise COTS serve the needs of an enterprise as opposed to a single department, user or specialized application

Page 10 of 307

and technical expertise available within an organization (Uçar and Bilgen, 2013), the level of

integration complexities, the degree of engagement by the vendor, and the degree of control over

the vendor and the project partners. Additionally, the actual implementation process is often found

to be different from the documented processes observed in the literature (Torchiano and Morisio,

2004; Li et al., 2009; Jadhav and Sonar, 2009). This gap between theory and practice has

effectively contributed to a higher risk level and uncertainty surrounding enterprise COTS

implementation ( Birkmeier and Overhage, 2009). Although a substantial body of IS literature has

focused on examining various aspects related to the well-known categories of COTS products,

such as ERP (Enterprise Resource Planning) and CRM (Customer Relationship Management), the

implementation of non-ERP COTS has received very little attention from researchers (Ayala et al.

2011) which is the focus of this research.

While analyzing different implementation models and comparing them to the generic definition of

COTS package implementation, it becomes apparent that there is a clear gap between the existing

models found in literature and the actual implementation process that is followed. For example,

legacy system replacement would require a critical task of meta-data conversion for the new

packaged solution (Lewis, and Wrage, 2004), which is not included in any of the models we

reviewed. Some of the models do point out the components of glueware2 or custom code; however,

meta-data conversion and data reconciliation can be much more complex compared to the

development of custom components required to support a COTS package. Additionally, some

implementation tasks may take place during a different stage than those shown in the reviewed

models from the literature. This variation can be attributed to the maturity level of a COTS product

2 Glueware is a software solution or platform that is designed to seamlessly integrate disparate and decentralized software solutions and systems with related resources (Techopedia).

Page 11 of 307

where more mature products like ERP and CRM demonstrate a consistent pattern, and less mature

products exhibit a higher variation. For example, the model derived by Morisio et al. (2002) for

non-ERP COTS implementation shows simultaneous activities of product identification,

evaluation, selection, and requirement analysis. Product selection and detailed requirement

analysis are two of the most critical activities of a COTS implementation project, and yet,

depending on the maturity level of the COTS product, they could be performed at different stages

of the project.

In light of above possibilities, an empirical investigation of enterprise COTS implementation

projects would significantly contribute to both theory and practice by helping to identify and

understand critical implementation phases, activities, processes, and tools that determines the

implementation success.

The identification of a micro-level or detailed enterprise COTS implementation model, capturing

tools, processes, and procedures, creates the opportunity for investigating several other related

aspects. For a multi-partner and multi-vendor implementation effort in the public sector, elements

of social responsibility and control can be of particular interest to both researchers and

practitioners. Generally, as the number of stakeholder increases in a project, control structure also

becomes complex. Accountability for a public-sector organization often includes the social

responsibility aspect. More specifically, the management of critical success factors (CSF) through

the adopting of appropriate control configurations and ensuring optimal stakeholder engagement

can be a significant contribution in relation to both communities. Even the research concentrating

on the individual aspects of Information Systems (IS) implementation – such as CSF identification,

implementation process, or application of various controls – is quite substantial, but largely non-

existent are investigations of CSF management by optimally engaging stakeholders and balancing

Page 12 of 307

control by applying different control configurations (Finney and Corbett, 2007). Enterprise COTS

implementation projects are often part of an organization’s strategic IT initiatives and require a

large investment of organizational resources, yet often fail due to the inadequate management of

CSF and other implementation-specific challenges. Regardless of the large body of existing

research concentrating on CSF, the sheer volume of unsuccessful implementation efforts clearly

indicates the need to adopt a different theoretical lens to research this issue. Therefore, utilizing

stakeholder theory and the theory of control balancing, to examine the implementation of

enterprise COTS and the management of CSF during the implementation process, holds much

promise.

Stakeholder engagement

A major aspect of the current thesis centers around the organizational stakeholder orientation

during an enterprise COTS or IS implementation. The concept of stakeholder engagement is

becoming increasingly popular among researchers and management practitioners of information

systems (IS) to explain the diverse outcomes associated with the implementation of new

technology. From an in-depth analysis of 400 strategic decisions, including technology

acquisitions and strategic reorganizations, Nutt (2002) reported an overwhelming 50 percent

failure ratio. He primarily classified “failure” as aborting an IS project, partial implementation, or

a failure to produce expected results. Interestingly, most of the failures were attributed to decision-

makers’ inability to integrate and engage relevant stakeholders. This finding is also in agreement

with the earlier literature stressing the significance of stakeholder interests (Bryson et al. 1990;

Bryson and Bromiley, 1993; Burby, 2003; Margerum, 2002). Bryson (2004) indicated that an

inability to synthesize information possessed by various stakeholders and the failure to address

their concerns while making decisions are major flaws in thinking and action that can lead to

Page 13 of 307

failure. On the other hand, evidence of positive outcomes through cautious stakeholder

engagement is also evident in the literature. Aside from supporting issue-legitimization and

facilitating closer alignment between organization and society, the positive impacts of stakeholder

opinions on decision-making and project implementation processes are well established in the

literature (Deelstra, Nooteboom, Kohlmann, Van den berg, and Innanen, 2003). Several other

studies have also identified stakeholder engagement as an effective organizational strategy and a

means to improve external stakeholder relations, often used in the public and private sectors (Kivit,

2011). Despite numerous studies indicating the strategic significance of stakeholder engagement,

research on enterprise COTS implementation appears to be largely missing this focus. Lorenzo

(2004), through a comprehensive review of the enterprise systems (ES) research, identified a high

risk of ES implementation failure at the project phase (Buckhout et. al. 1999; Scott 1999;

Davenport 1998), diffusion phase (Shepherd, 2001; James and Wolf, 2000; Gilbert, 1999) and the

benefit realization phase (Shepherd, 2001; Markus and Tanis, 2000; Davenport, 1998). In addition,

the categorization of existing ES implementation research indicates four salient trends: critical

success factors, measuring success, descriptive case studies, and long-term challenges (Lorenzo,

2004). Finney and Corbet (2007) have also raised similar concerns while examining ERP literature,

stating that much of it has focused on critical success factors with very limited or no regard to the

stakeholder perspective. An intimate understanding of various stakeholder groups would make it

possible to effectively address the challenges related to the project phase, the diffusion phase, and

the benefit realization phase, thus enhancing the overall probability of successful ES/IS

implementation (Finney and Corbet, 2007). Therefore, a necessity for understanding the nature of

stakeholder orientation and filling the existing knowledge gap in this area are two primary

motivations for adopting a stakeholder lens for the current thesis.

Page 14 of 307

Control balancing

Another salient aspect of the current thesis revolves around the concept of control balancing during

an enterprise COTS or IS implementation. Effectively controlling project and relationships is one

of the most critical factors for the success of an Information Systems (IS) implementation project

(Gregory, Beck, and Keil, 2013). Leimeister, Heusler, and Krcmar (2010) also voiced a similar

concern that the equilibrium of coordination and control mechanisms are absolutely essential for

an effective IS project governance. A review of the control literature indicates a dominance of

control types and their application in a vertical relationship structure. However, the concept of

control balancing is a much recent one, and its application in a horizontal relationship structure

has received very little attention (Gregory et al. 2013).

The concept of control involves a controller (the person exercising control) and a controllee (the

target of control). Numerous empirical studies have investigated control in an organizational or IS

project context (Kirsch, 1997, 2004). However, a majority of the studies related to control theory

has focused on the modes of control comprised of two broad categories: formal and informal

(Crisp, 2002). Formal control is typically documented and initiated by the management. Informal

control is usually comprised of undocumented measures initiated by the employee themselves

(Jaworski, 1988), and are commonly identified as self-control and clan control (Ouchi, 1980). A

less prevalent aspect of control theory is “control configuration” which is distinct from the concept

of control (Gregory et al. 2013). Gregory et al. (2013) defined control configuration as a

combination of control type, control degree, and control style. They have also conceptualized

control balancing as the act of making targeted adjustments to an existing control configuration.

Page 15 of 307

An enterprise COTS implementation project involving multiple clients and vendors possesses both

horizontal and vertical relationship structures. As the current thesis includes COTS

implementation cases possessing both horizontal and vertical relationship structures, findings from

our analysis of selected cases revealed valuable insight concerning the application of control

balancing in a multiple stakeholder context.

1.2 Research objectives

Enterprise COTS implementation is an inherently challenging task, and a public-sector context for

such an implementation can be a source of very rich information with much potential for

contribution to the understanding of the issue. Therefore, the objectives pertaining to the current

thesis are following:

1. To identify a micro-level or detailed model for enterprise COTS implementation by

focusing on implementation phases, processes, and tools. Additionally, a micro-level

analysis can also help reveal the valuable information flow patterns among the

implementation phases.

2. To identify a set of CSFs that are relevant for enterprise COTS implementation and

examines the management aspect of them through process, stakeholder engagement and

control balancing. Understanding and managing CSFs are critical aspects of any IS

implementation project. CSFs have been a dominant research theme in ERP

implementation research, and similar to ERP implementation, managing CSFs is equally

important for any non-ERP COTS implementation project.

Page 16 of 307

3. To develop a theoretically sound framework that can be used to identify an organization’s

stakeholder orientation during an enterprise COTS implementation project. Stakeholder

engagement has often been a blurred area in much package-software implementation

research (Finney and Corbett, 2007), despite the potential to explain many related

consequences. Therefore, developing a framework to identify an organization’s

stakeholder orientation during an enterprise COTS implementation would be a significant

contribution in this domain.

4. To validate the theory of control balancing (Gregory, Beck, and Keil, 2013) by examining

the dynamic relationships among the project partners from one phase to another phase of

the implementation process. The concept of control has been previously subject to

numerous empirical investigations in the context of IS implementation. However, control

balancing, a related concept, has not received sufficient attention. Understanding the nature

of control balancing, from the perspective of what triggers a targeted adjustment of control

portfolio, shows significant potential for contributing to the literature.

1.3 Research questions

Through this thesis, we intend to investigate multiple aspects of an enterprise COTS

implementation process. Key dimensions of the current research are related to the following

domains:

1. Implementation models, processes and tools related to COTS implementation projects;

2. The dynamic nature of control balancing and control configuration related to a COTS

implementation;

3. Identifying an organization’s stakeholder orientation during a COTS implementation; and

Page 17 of 307

4. Relationships among CSFs, control balancing and stakeholder engagement.

Our research questions are:

1. What is an activity-based and process-based model for enterprise COTS implementation?

2. How can organizational processes and tools contribute to COTS implementation

success?

3. What is the nature of control configuration in a multi-partner COTS

implementation and what are the factors responsible for the application of control

balancing?

4. How can CSFs and other challenges be successfully managed through optimal

control balancing by the project?

5. What is nature of stakeholder orientation during a COTS implementation for a public

sector IS implementation?

6. How can CSFs and other challenges be successfully managed through optimal

stakeholder engagement by the project?

1.4 Case study approach

The current thesis adopts a multiple-case-study research design to inquire about the

implementation of enterprise COTS packages. Usually, when the topic of research concerns

contemporary events and investigation requires no behavioural control, a case study method is

better suited for the investigation. According to Yin (1994), a case study is an empirical inquiry

that i) relies on multiple sources of evidence with data needing to converge, ii) benefits from the

Page 18 of 307

prior development of theoretical propositions to guide data collection and analysis, iii) investigates

contemporary phenomena in depth within its real-life context, and iv) shows no clear boundaries

between the phenomenon and the context.

The above definition reflects both the appropriateness and the urge of a case-study research for

investigating the proposed research questions concerning a contemporary phenomenon (enterprise

COTS implementation) where the investigation requires no behavioural control over the events.

In addition, a case-based research on IS implementation can produce a much richer description of

the events, and can then be used for further abstraction through other methods like grounded theory

research (Fernández and Lehmann, 2011).

Data for the current thesis have been collected through semi-structured interviews using a set of

open-ended questions that allowed the participants to describe the process of a COTS

implementation project in details, the nature of controls utilized during the project, the nature of

stakeholder engagement, and the critical success factors relevant to the project. A thorough

literature review has also been conducted to identify the state of the knowledge related to COTS

implementation. This served a twofold purpose: a) identification of knowledge gaps which helped

formulate the interview questions, and b) clarification regarding the current status of theory to

identify points of theoretical contribution or extension.

The unit of analysis for the current thesis is the implementation of an enterprise COTS, and the

total number of cases that will be analyzed through this research are four. We have interviewed 40

employees from various ranks involved with COTS implementation projects. In particular, the

projects we have selected for this research relate to the implementation of an enterprise COTS

Page 19 of 307

solution not belonging to the category of well known COTS (i.e. ERP or CRM). Additional

selection criteria included a public-sector focus, multiple clients, multiple vendors, failed

implementation, and successful implementation. The selected sample includes two cases of

successful implementation and two cases of failed COTS implementation. Although a failure is

difficult to define for a public-sector COTS implementation, we focused on the “post-

implementation client satisfaction” and “support cost” to create a distinction between successful

and failed projects. Although the use of the word “failed” may not be used at the organizational

level, primary intention of the current research is to establish that a lack of CSF management

through stakeholder engagement and control can lead to adverse outcome for a COTS

implementation.

1.5 Organization of this thesis

This thesis is divided into seven chapters.

The next chapter provides a review and synthesis of relevant literature. It starts with a review of

enterprise COTS implementation, enterprise systems (ES) implementation, and Information

Systems (IS) implementation literature. This includes both academic and practice-oriented

literature to identify the knowledge gaps with regards to the enterprise COTS implementation

process. Next, a review is undertaken of the critical success factors (CSF), stakeholder

engagement, and control literature. As much of the existing CSF research focus on identifying

context specific CSF for IS implementation, this review helps justify the need for a new theoretical

approach for studying CSFs, as well as facilitates the development of a stakeholder orientation

framework.

Page 20 of 307

Chapter 3 provides a synthesis of the preceding chapters, including the literature review, using

vital arguments to signify the existing knowledge gaps in the domain. It also elaborates the

concepts of control, stakeholder engagement, and stakeholder sensitivity. This elaboration and

synthesis pave the way to propose the current research objectives in a more tangible fashion.

Chapter 4 describes the utilized research methods in detail, including specific steps that are

followed. A discussion on the case reliability and case validity is also included in this chapter.

Chapter 5 presents the project overview and background for each COTS implementation case

selected for analysis. We also presented summarized primary data and results of initial coding in

this section.

Chapter 6 presents cross case analysis and a synthesis for all selected cases. It also presents the

results from our pattern coding of the primary data and discusses the theoretical integration of the

emerging themes that were identified through a grounded theory approach.

Chapter 7 provides a conclusion for the current research. In addition to presenting summarized

answers to each of our research questions, contributions to both theory and practice, limitations of

the current research, and future research directions are also discussed in this section.

Page 21 of 307

2.0 Literature review

This chapter reviews the relevant literature from enterprise COTS implementation, enterprise

systems (ES) implementation, and Information Systems (IS) implementation literature. In

addition, a review of critical success factors related to both ERP and non-ERP packaged software,

stakeholder engagement, and control literature has also been conducted.

Section 2.1 describes the article selection process for the current research. Specifically, this section

outlines the electronic sources and scholarly databases used for selecting peer-reviewed academic

articles. In addition, it presents the keywords used for locating articles.

Section 2.2 reviews the literature related to COTS and IS implementation. The primary goal of this

review is to identify the existing knowledge and models related to the implementation process and

the knowledge gaps when it comes to enterprise COTS. This also sets the ground for a micro-level

analysis and facilitates a connection to the concept of CSF by identifying key activities.

Section 2.3 reviews IS literature related to stakeholder theory. A guiding motivation for this review

is to examine the concept of “stakeholder orientation” and highlight the difference between

“consideration” and “engagement.” In addition, this section points out variations related to the

application of stakeholder theory, challenges related to the adoption of a stakeholder lens for COTS

implementation, and the possibility of multiple stakeholder orientations by the project team.

Section 2.4 delves into the concepts of control and control balancing associated with control

theory. The primary contribution of this review is to provide a justification for studying control

balancing in a multiparty COTS implementation context. This section also highlights the

distinction between hierarchical and vertical control in the absence of a direct controller-controllee

relationship.

Page 22 of 307

Section 2.5 reviews the literature on critical success factors (CSFs) related to IS implementation.

It also helps to narrow down the CSFs for COTS implementation by introducing categorization

schemes and the concept of relevance for the current study.

2.1 Selection process for articles

The subsequent sections of the chapter offer a review of the relevant literature. The first step

involves searches of electronic databases of journal articles using selected keywords and phrases

related to the current research topic. The most frequently cited academic papers from peer-

reviewed journals, as well as seminal articles related to IS implementation, CSF, stakeholder

theory, and control, were consulted. In addition, industry literature related to COTS

implementation was also explored to ensure that knowledge has been correctly identified from

academic journals.

The literature search utilized two primary sources of electronic databases – Business Source

Complete and Google Scholar. These two were chosen, as the goal of the current literature search

was to ensure article selection from leading business journals due to the managerial nature of the

research questions proposed.

Google Scholar is a database that “provides a way to broadly search for literature across many

disciplines and sources” (Cresswell, 2009: 31). More specifically, Google Scholar provides access

to a large selection of social science and engineering literature (Mingers, Marci and Petrovici,

2012). It is also considered to be a valuable complementary source because of its content as well

as the compilation of the contents (Pomerantz, 2006). As Chen states, “Google Scholar is able to

retrieve all scholarly publications from database and websites that are open to Google Scholar.

Subscription based abstracts and index that are not open to Google Scholar still have some unique

Page 23 of 307

contents, but these unique contents are primarily non-scholarly journal materials” (Chen, 2010:

226). Chen (2010) also emphasizes that Google Scholar is a reliable complement to traditional,

subscription-based database of scholarly publications. Quality of knowledge, reliability, accuracy

of citation counts, and wide coverage of the literature have also been reported by other researchers

(Mingers, Marci, and Petrovici, 2012; O’Leary, 2011).

Business Source Complete is a database that focuses on providing a comprehensive set of “peer-

reviewed, business related journals” (Business Source Complete, n.d.). Business Source Complete

provides “extremely comprehensive full-text access to certain key business and management

journals” (Bryman and Bell, 2003: 555).

The search of relevant literature was conducted through the use of keywords from two different

sets. The first set included keywords like IS implementation, COTS, COTS implementation,

Enterprise Systems etc., while the second set included keywords like implementation process,

phases, CSF, stakeholder, stakeholder engagement, stakeholder sensitivity, control configurations,

control balancing, control portfolio etc.

2.2 COTS and IS implementation

For the purpose of this research, two major categories of literature: peer-reviewed academic

journals and industry literature were considered. Innovation process models for information

systems, ERP implementation and COTS-specific implementation were examined to capture the

details of COTS-based implementation practices. In the following sections, we review some of the

significant models that can be considered valuable when it comes to implementing a COTS

solution.

Page 24 of 307

ERP literature was reviewed because most of the enterprise systems and COTS products share one

key characteristic: they both are packaged software. This close resemblance is apparent from some

of the adopted definitions of ERP system given by ERP researchers. For example, following is the

definition given by Peslak (2008):

An ERP system is an integrated commercial off-the-shelf (COTS) software package

that can perform all the major business functions of an organization. These functions

generally include all elements of the value chain from raw material purchases,

inventory management, production, goods shipments, invoicing, accounting, and

human resource management.

Non-ERP COTS often has additional custom components or glueware developed to make the

packaged components usable or functional for a certain organizational environment. Both

packaged software and custom-built software display great variations in terms of acquisition and

implementation methods. In parallel to the acquisition and implementation variations,

organizational IT competencies have also increased to accommodate such variations. This

acquisition process variation and enhanced organizational IT capabilities, over time, have been

stated by Daneshgar et al. (2013):

Over recent decades, the decision to select a software acquisition method has

been mainly relevant to large and medium size organisations with a large IT

investment. Given the wide-spread adoption of technology over that period, a

majority of large organisations are now more likely to possess sufficient levels

of IT competency to enable them to use a variety of methods for acquiring and/or

managing the acquisition process of their software systems including in-house

Page 25 of 307

development, outsourcing, and setting up offshore collaborators. Such

capabilities are normally related to various phases of the system development

life cycle, as well as support services.

Although in several other large-scale software development initiatives such as ‘game-

development’ heavily utilize COTS packages and ‘off-the-shelf’ components, the focus

of this research is purely from an enterprise perspective which limits our literature review

to systems that integrate various enterprise processes to give a holistic view of the

enterprise. The following sections offer a review of the most prominent models for

enterprise COTS implementation (including well-known COTS such as ERP, CRM),

their focus of analysis (micro versus macro), and their similarities and differences. In

addition, a more recent practice of IS implementation – an agile implementation – has

also been discussed in section 2.2.1 and 2.2.2

Umble et al. (2003) ERP Implementation Steps :

Umble et al. (2003), reviewing several earlier ERP implementation studies (Langenwalter, 2000,

Odenet al. 1993, Ptak, 1999, Ptak, 2000), compiled a list of 11 recommended steps for a successful

implementation. These include: 1) reviewing the pre-implementation process to date, 2) installing

and testing any new hardware, 3) installing the software and performing the computer room pilot,

4) attending system training, 5) training on the conference room pilot, 6) establishing security and

necessary permissions, 7) ensuring that all data bridges are sufficiently robust and the data are

sufficiently accurate, 8) documenting policies and procedures, 9) bringing the entire organization

on-line, either in a total cutover or in a phased approach, 10) celebrating, and 11) improving

Page 26 of 307

continuously. These steps identified by Umble et al. (2003) are indeed critical for a successful

packaged solution implementation, however, they consist of pre-implementation, during-

implementation and post-implementation activities. In addition to project activities, Umble et al.’s

(2003) recommended steps include policies and processes. Due to the broad focus, recommended

11 steps do not cover the implementation activities in sufficient detail.

Shin and Lee (1996) Non-ERP Implementation Process:

In the domain of non-ERP packaged software, Shin and Lee (1996) provided a more

comprehensive list of activities related to implementation processes. Reviewing earlier COTS

implementation research, Shin and Lee (1996) suggested 25 activities grouped into three phases

and seven sub-phases for packaged software acquisition and implementation.

Table 1: IS Project Phases

Phases Sub-Phases Activities

A. Project Formulation

phase

A-1. Project initiation

A-2. Requirement Analysis

1) Constitute Committee

2) Define Problems to Solve

3) Define the Scope and Goals of the Project

4) Planning Overall Process

5) Analyze Current Task Performing System

6) Define Users’ Information Requirement

7) Check Various Organizational Constraints

8) Integrate and Adjust Analysis Output

9) Information Source Selection and Searching

10) Make List of Alternative ASPs after Rough

Screening

B. ASP Selection Sub-

phase

And Acquisition Phase

B-1. Preparation

B-2. Selection

B-3. Acquisition

C. Installation and Post-

Implementation Use

phase

C-1. Installation

C-2. Post Implementation

Page 27 of 307

11) Develop Content and Format of RFP (Request

for Proposal)

12) Develop Evaluation Model

13) Collect and Analyze Vendor Proposal

14) In-depth Interview with Candidate Vendors

15) Benchmark Test and/or Arrange On-site

Demonstration

16) Make Final Decision and Authorization

17) Contractual Negotiation, Order and Take

Delivery

18) Develop Implementation Plan

19) ASP Modification

20) Internal Procedure Modification

21) Provide Training

22) Data Conversion and System Transition

23) Testing New System in the User’s Environment

24) System Operation and Maintenance

Implementation

25) Performance Evaluation

Shin and Lee’s (1996) proposed phases resemble the consolidated groups of actual phases that

most COTS implementation projects follow. Although the sub-phases align closely with most

commonly observed implementation phases, Shin and Lee (1996) did not indicate clear

demarcation points for the activities, which is essential for associating them to a phase and

identifying critical inter-phase relationships.

Markus and Tanis (2000) Enterprise System Model:

Page 28 of 307

Soh and Markus (1995) proposed an “IT investment to business value” model, where three linked

processes outline how IT investments result in organizational value. Based on this framework

Markus and Tanis (2000) proposed Enterprise System Experience Cycle, consisting of four phases:

project chartering, configuration and rollout, shakedown, and onward and upward, elaborated on

below.

The project chartering phase comprises decisions leading up to the funding of an enterprise system

or packaged software. Building a business case for adopting packaged software, selecting the

appropriate software package (ASP), identifying a project manager, and obtaining necessary

budget approval are typically the key activities performed during this phase. This phase includes

both “Project Formulation” and “ASP Selection,” as indicated by Shin and Lee (1996).

The configuration and rollout phase comprises activities intended to get the packaged solution

deployed for organizational units in a usable form. This corresponds to the Shin and Lee’s (1996)

phase C, “Installation and Post-Implementation use.” Key activities for this phase include

packaged software configuration, integration with existing environment, testing, data conversion

and validation, training, and rollout.

The shakedown phase aims to stabilize the system through eliminating any bugs. Key activities of

this phase include bug fixing, system performance tuning, retraining, and acquiring sufficient

support staff. Shin and Lee’s (1996) sub-phase “C-2” and activity 24 are similar to this phase

proposed by Markus and Tanis (2000).

The onward and upward phase concerns the maintenance of the packaged software, typically

though supporting users, ensuring optimal usage and facilitating upgrades. This phase starts from

Page 29 of 307

normal operation until the system is replaced with an upgrade or a different system. Although Shin

and Lee (1996) referred to this phase through activity 25, Markus and Tanis (2000) explicitly

associate this phase with benefit realization from organizational IT investments.

Both Shin and Lee (1996) and Markus and Tanis (2000) tried to capture the implementation and

post-implementation aspects of a COTS product from an enterprise perspective, but through

slightly different conceptualizations. Shin and Lee (1996) takes a top-down approach where major

implementation phases are hierarchically decomposed into lower level phases and tangible activity

items. Markus and Tanis (2000) also captured higher level implementation phases of an enterprise

system, however the focus of analysis is higher level benefit realization.

Morisio et al. (2002) model:

Morisio et al. (2002), through their empirical investigation, have identified four major process

phases for COTS-based implementation: requirements, design, coding, and integration. They have

also indicated a distinct implementation process for packaged software, which is quite different

from traditional IS implementation process. They reported that “COTS projects were obliged to

follow a process quite different from traditional projects, with more effort put into requirements,

test and integration, and less into design and code” (Morisio et al., 2002. This conclusion was

supported through the identification of three types of differences compared to the traditional

implementation process. Morisio et al. (2002) reported these differences under three categories: 1)

New activities, 2) Reduced activities, and 3) Modified activities. New activities include tasks like

product evaluations, product familiarization, and vendor interaction (both technical and non-

technical). These tasks also warrant new roles like a COTS evaluation team, and a team member

responsible for interacting with the vendor. Modified activities include the design and architecture

Page 30 of 307

phase, where the design focus shifted more on how to fit pieces together rather than analyzing the

internal workings of different modules. Reduced activities include tasks like coding, debugging,

unit testing, and code inspection.

Figure 1: COTS implementation phases and key activities (Source: Morisio et al.2002)

Whereas the derived model by Morisio et al. (2002) captures the core activities of a COTS

implementation at some level of detail, a comparison with other existing models indicates that

this model does not take into account some of the leading and trailing implementation activities,

many of which are critical for a successful COTS implementation. Additionally, this model can

potentially be enhanced by capturing the level of engagement by the different stakeholders for

various key COTS-specific activities.

In the next section, industry literature and knowledge, specific to COTS implementation, are

reviewed to identify COTS implementation models that are followed by practitioners.

Page 31 of 307

The industry literature indicates that COTS implementation models are developed for vendor-

specific products, which are based on implementation best practices and heavily dependent on the

features of the vendor-provided tools. Models based on EPIC (Evolutionary Process for Integrating

COTS-based systems) principles and RUP (Rational Unified Process) for COTS implementation

appears to be the most relevant model for the purpose of the present research. The EPIC

methodology, developed by Albert (2002), consists of four spheres of influence and iteratively

converging decisions over time through accumulating knowledge and increasing stakeholder buy-

in. Rational Unified Process (RUP) is a configuration based on EPIC principles, providing

guidance for COTS project implementation. Stages of the RUP model for a COTS implementation

are discussed below.

RUP Model:

RUP has four implementation stages for a COTS project: Inception, Elaboration, Construction and

Transition. The Inception phase corresponds to the project formulation (Shin and Lee, 1996) or

project chartering (Markus and Tanis 2000) phase in the most commonly found implementation

models in the academic literature. The primary task here is to establish feasibility through a

business case, identifying one or more candidate solutions and achieve concurrence among

affected stakeholders regarding the life-cycle objectives for the project.

Elaboration phase focuses on early implementation activities with an aim to achieve sufficient

stability of the architecture and requirements. The primary objectives of this phase are selecting

and acquiring components, mitigating risks, and developing a predictable cost and schedule for

implementation.

Page 32 of 307

Construction phase corresponds to the actual user acceptance testing or end-of-development stage

where a production-quality release ready for the user community is available. In earlier

implementation models reviewed, both “Installation and Post-Implementation use (Shin and Lee,

1996)” and “Configuration and Roll out (Markus and Tanis, 2000)” indicate these activities, which

essentially result in a production-ready version of the packaged software.

Transition phase of RUP is the final implementation stage of a COTS product where the objective

is to transition the solution to its users. This stage overlaps multiple stages of some of the existing

implementation models. For example, the Configuration and roll-out, and Shakedown phases

(Markus and Tanis, 2000) both jointly include activities indicated by the transition phase.

RUP phases for COTS implementation are product-specific, capturing many of the critical

implementation aspects of packaged software, but this may not be suitable for implementation

phases when it comes to public sector. This can be primarily attributed to the different business

model followed by government organizations compared to the business model of private industry.

A brief comparison of all the reviewed models are presented on Table 2:

Table 2: Implementation Model Comparison

Model Implementation

Target

Focus of Analysis Strengths Gaps

Umble et al.

(2003)

ERP

Implementation

A comprehensive

review of ERP

implementations and

identification of critical

activities

identification of critical

activities of

implementation

Missing significant

number of COTS

implementation specific

activities

Page 33 of 307

Simplified

categorization of

activities

Representative items

due to the

comprehensive nature of

the study

Identification of tools

Identification of leading

and trailing activities

Sequencing of activities

Information flow

Shin and

Lee (1996)

ERP

Implementation

takes a top-down

approach where major

implementation phases

are hierarchically

decomposed into lower

level phases and

tangible activity items.

Identification of

actionable items

Identification of

processes

Logical Process/phase

based grouping


Identification of leading

and trailing activities


Information flow

Markus and

Tanis

(2000)

Enterprise

Systems

Captures higher level

implementation phases

of an enterprise

system, however the

focus of analysis is

organizational benefit

realization

Higher level benefit

realization

Captures post

implementation aspect

of a COTS product

Multiple phases are

consolidated to keep the

model simple

Lack of focus on the

initiation and pre-

initiation activities


Information flow

Page 34 of 307

RUP and

EPIC

Model

Enterprise

COTS

Focuses on the

significance of iterative

convergence of

decisions and obtaining

stakeholder buy-in.

Very specific to

enterprise COTS

implementation projects

Supported by software

tools to control and

implement the flow and

phases

Vendor specific model

Information flow

Visibility of processes to

be able to adjust them

Morisio et

al. (2002)

Enterprise

COTS

Model developed

purely by analyzing

enterprise COTS

implementation and

primarily focuses on

the implementation

aspect

Captures several unique

aspects of enterprise

COTS implementation

does not take into

account some of the

leading and trailing

activities


2.2.1 An agile implementation

Considered from a flow perspective, all of the reviewed implementation models assume a waterfall

or sequential model for the activity flow. This is significantly different from the approach targeted

by the current thesis, which seeks to investigate an “agile” implementation model for enterprise

COTS.

Agile methods emerged as an alternative to plan-driven software development methods over a

decade ago (Dingsøyr et al. 2012; Agile Software Development, 2014). These are a group of

software development methods based on iterative and incremental development. Several

definitions of agile development are prevalent in literature. For example, Henderson-Sellers and

Serour (2005) defined agility as “the ability to adapt to different changes and to refine and fine-

tune development processes as needed.” Also, Lee and Xia (2010) define software development

Page 35 of 307

agility “as the software team’s capability to efficiently and effectively respond to and incorporate

user requirement changes during the project life cycle.” A more comprehensive definition of

agility is provided by Conboy (2009) as:

The continual readiness of an information systems development (ISD) method to

rapidly or inherently create change, proactively or reactively embrace change, and

learn from change while contributing to perceived customer value (economy,

quality, and simplicity), through its collective components and relationships with

its environment.

2.2.2 The agile manifesto and agile values

The articulation of the agile manifesto in 2001 has contributed to a significant change in the

software engineering field (Dingsøyr et al. 2012). This change is further observed through the

appearance of a host of methods adhering to the tenets of the manifesto, such as eXtreme

programming (XP), scrum, lean software development, feature-driven development (FDD), and

crystal methodologies.

The 12 principles of the agile manifesto (Agile Software Development, 2014) include: (1) Highest

priority to satisfy the customer through early and continuous delivery of valuable software, (2)

Welcome changing requirements, even late in development, (3) Deliver working software

frequently, from a couple of weeks to a couple of months, with a preference to the shorter

timescale, (4) Joint working group of business people and developers throughout the project, (5)

Build projects around motivated individuals (6) Utilize face-to-face conversation as the most

Page 36 of 307

efficient and effective method of conveying information to and within a development team, (7)

Working software is the primary measure of progress, (8) Agile processes promote sustainable

development, (9) Continuous attention to technical excellence and good design enhances agility,

(10) Simplicity – the art of maximizing the amount of work not done – is essential, (11) The best

architectures, requirements, and designs emerge from self-organizing teams, and (12) At regular

intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour

accordingly.

Besides these 12 principles outlined in the manifesto, the core values of agile development – a

focus on the individuals and their interactions, early delivery of working software, customer

collaboration and responsiveness to change (positively contributing to implementation time), and

considering the quality and cost of the software – are essential for today’s highly dynamic and

competitive business environments (Stavru, 2014). Due to the flexibility and adaptability offered

by agile methodology, both software development and IS implementation initiatives demonstrate

the application of agile methods. This is also reflected in the increasing volume of academic

literature in this domain. Dingsøyr et al. (2012) have identified a total of 1,551 research papers on

agile software development published between 2001 and 2010. Research in this domain has also

demonstrated substantial variations, ranging from simple adoption of agile methods (e.g., Boehm,

2002; Nerur et al. 2005) to aspects of team dynamics (Moe et al., 2009), the consequences of test-

driven development (Erdogmus et al. 2005; Janzen and Saiedian, 2005), adoption and post-

adoption issues (Cao et al. 2009; Mangalaraj et al. 2009), and the challenges of implementing agile

principles in distributed settings (Ramesh et al., 2006).

Page 37 of 307

Although the benefits and diverse effects of adopting an agile implementation approach are well

established in the literature (Misra, Kumar and Kumar, 2009; Misra, Kumar and Kumar, 2010a;

Misra, Kumar and Kumar, 2010b; Misra et al. 2012), they focus primarily on the in-house or

greenfield development of software. Adopting a similar approach for a packaged software

implementation is anticipated to introduce new challenges, processes and methodological changes

that have not been sufficiently identified by earlier studies examining COTS implementation.

Therefore, agile implementation of an enterprise COTS application holds much potential for

contributing to our understating of the implementation process in terms of activity flow,

information flow, and inter-phase relationships.

Most of the implementation activities, processes and tools employed in an enterprise COTS

implementation often demonstrate a visible connection with the people that are interested in or

involved with the implementation. Next section reviews IS literature related to stakeholder theory.

A guiding motivation for this review is to examine the concept of “stakeholder orientation” and

highlight the difference between “consideration” and “engagement.” A review of the stakeholder

and stakeholder engagement also helps clarify different aspects relating to ‘whose interest matters’

and ‘how does it matter’ in an enterprise COTS implementation.

2.3 Stakeholders and stakeholder engagement

Conceptual origin and representation

Originally rooted in strategic management literature, stakeholder theory has been receiving

increasing attention – both by the managers and academics – since the publication of Freeman’s

Page 38 of 307

(1984) landmark book Strategic Management: A Stakeholder Approach (Mitchell and Angel,

1997). Although Freeman intended to suggest that stakeholder relationship is a useful unit of

analysis when it comes to strategy or strategic management, he also attempted to clarify that the

concept of stakeholder emerged much earlier from a Stanford research, which defined it in 1963

as “those groups without whose support the organization would cease to exist” (Freeman, 2004).

Freeman later proposed a definition of stakeholder that is much broader than Stanford’s original

definition: “A stakeholder in an organization is (by definition) any group or individual who can

affect or is affected by the achievement of the organization’s objectives” (Freeman 1984).

The original stakeholder model presented by Freeman (1984) in relation to the organization was

quite simple. Placing the organization at the center and the all other significant stakeholders around

it, Freeman’s (1984) original model closely resembles a wheel. Figure 2 presents a version of the

Freeman’s (1984) original model, which shows seven stakeholders:

Figure 2: Stakeholder model presented by Freeman (1984)

Page 39 of 307

Besides the structural representation of the stakeholder model, another aspect related to the

stakeholder theory is the nature of application or utilization of the concept. Both of these aspects

are dominant in stakeholder literature and deserve sufficient consideration when adopting a

stakeholder lens.

A significant contribution towards the understanding of diverse applications or different features

of stakeholder theory is the work of Donaldson and Preston (1995), which aims to clarify the nature

and the purpose of stakeholder theory. Donaldson and Preston (1995) identifies three different uses

of stakeholder theory, each of which have a distinct value proposition: (1) descriptive, (2)

instrumental, and (3) normative.

Donaldson and Preston (1995) proposes a descriptive application of stakeholder theory as the

“theory is used to describe, and sometimes to explain, specific corporate characteristics and

behaviours.” Instrumental application refers to the usage of stakeholder theory “to identify the

connections, or lack of connections, between stakeholder management and the achievement of

traditional corporate objectives (e.g., profitability, growth)”. Finally, a normative view of

stakeholder theory was identified as intended to “interpret the function of the corporation,

including the identification of moral or philosophical guidelines for the operation and management

of corporations”. These three aspects of stakeholder theory are described as nested layers, with the

normative aspect at the core, instrumental in the middle, and the descriptive aspect on outside, as

shown in Figure 3:

Page 40 of 307

Figure 3: Three aspects of Stakeholder Theory (Donaldson and Preston, 1995)

Defining Stakeholders

Besides the scope variations naturally appearing from diverse definitions, Friedman and Miles

(2006) identified indiscriminate usage of the term “stakeholder” over the last two decades. It is

frequently used by private industries, public-sector organizations, businesses and the media, but

often without any clear definition or understanding of the term itself (Mainardes, Raposo, and

Alves, 2011). While confusion in non-academic circles is not surprising due to a lack of knowledge

concerning stakeholder theory, a consensus around the definition of the term also seems difficult

to find among academic researchers. The diversity surrounding the definition is made clear by

looking at the works of Bryson (2004), Buchholz and Rosenthal (2005), Pesqueux and Damak-

Ayadi (2005), Friedman and Miles (2006), and Beach (2008), which combined contain a total of

66 different concepts of the term “stakeholder” (Mainardes, Raposo, and Alves 2011).

This large array of stakeholder definitions ranges from a very wide scope to a narrow scope. Most

of the definitions observed in the literature drift between these two extremes. The existence of

considerable variations regarding the definition of a stakeholder is clearly depicted through the

Page 41 of 307

work of Mitchell et al. (1997), which compiles a list of different views regarding stakeholders

observed in the literature. This is presented in Table 3.

Table 3: Who Is a Stakeholder? A Chronology

Source Stake

Stanford memo, 1963 "those groups without whose support the organization

would cease to exist" (cited in Freeman & Reed, 1983,

and Freeman, 1984)

Rhenman, 1964: -- "are depending on the firm in order to achieve their

personal goals and on whom the firm is depending for

its existence" (cited in Nasi, 1995)

Ahlstedt and Jahnukainen, 1971 "driven by their own interests and goals are participants

in a firm, and thus depending on it and whom for its sake

the firm is depending"

Freeman and Reed, 1983: 91 Wide: "can affect the achievement of an organization's

objectives or who is affected by the achievement of an

organization's objectives" Narrow: "on which the

organization is dependent for its continued survival"

Freeman, 1984: 46 "can affect or is affected by the achievement of the

organization's objectives"

Freeman and Gilbert, 1987: 397 "can affect or is affected by a business"

Cornell and Shapiro, 1987: 5 "claimants" who have "contracts"

Evan and Freeman, 1988: 75-76 "have a stake in or claim on the firm"

Evan and Freeman, 1988: 79 "benefit from or are harmed by, and whose rights are

violated or respected by, corporate actions"

Bowie, 1988: 112, n. 2 "without whose support the organization would cease to

exist"

Alkhafaji, 1989: 36 "groups to whom the corporation is responsible"

Carroll, 1989: 57 "asserts to have one or more of these kinds of stakes"-

"ranging from an interest to a right (legal or moral) to

ownership or legal title to the company's assets or

property"

Freeman and Evan, 1990 contract holders

Thompson et al., in 1991: 209 in "relationship with an organization"

Page 42 of 307

Savage et al., 1991: 61 "have an interest in the actions of an organization and ...

the ability to influence it"

Hill and Jones, 1992: 133 "constituents who have a legitimate claim on the firm ...

established through the existence of an exchange

relationship" who supply "the firm with critical

resources (contributions) and in exchange each expects

its interests to be satisfied (by inducements)"

Brenner, 1993: 205 "having some legitimate, non-trivial relationship with an

organization [such as] exchange transactions, action

impacts, and moral responsibilities"

Carroll, 1993: 60 "asserts to have one or more of the kinds of stakes in

business"-may be affected or affect ...

Freeman, 1994: 415 participants in "the human process of joint value

creation"

Wicks et al., 1994: 483 "interact with and give meaning and definition to the

corporation"

Brenner, 1995: 76, n. 1 "are or which could impact or be impacted by the

firm/organization"

Langtry, 1994: 433 the firm is significantly responsible for their well-being,

or they hold a moral or legal claim on the firm

Starik, 1994: 90 'can and are making their actual stakes known"-"are or

might be influenced by, or are or potentially are

influencers of, some organization"

Clarkson, 1994: 5 "bear some form of risk as a result of having invested

some form of capital, human or financial, something of

value, in a firm" or "are placed at risk as a result of a

firm's activities"

Clarkson, 1995: 106 "have, or claim, ownership, rights, or interests in a

corporation and its activities"

Nasi, 1995: 19 "interact with the firm and thus make its operation

possible"

Donaldson and Preston, 1995: 85 "persons or groups with legitimate interests in

procedural and/or substantive aspects of corporate

activity"

(Adopted from Mitchel et al., 1997)

Page 43 of 307

Classifying Stakeholders

A direct consequence of the variations surrounding the definition of “stakeholder” is the

emergence of diverse methods and approaches for identifying relevant stakeholders. The most

frequently appearing stakeholder classification schemes focus on: primary or secondary

stakeholders; owners and non-owners of a firm; owners of capital or owners of less tangible assets;

actors or those acted upon; those existing in a voluntary or involuntary relationship with the firm;

rights-holders, contractors, or moral claimants; resource providers to or dependents of a firm; risk-

takers or influencers; and legal principals to whom agent-managers bear a fiduciary duty (Mitchell

et al., 1997).

Among the research inspired by these wide variations, and aiming to develop a comprehensive

stakeholder classification scheme, the work of Mitchell et al. (1997) is particularly noteworthy.

They have proposed a distinct and practical way to classify stakeholders by focusing on three

different attributes: power, legitimacy, and urgency. Power indicates the possession of an ability

to make someone do something that would not be possible without the possession of that ability.

The power of the stakeholder over the organization may be coercive (strength or threat), normative

(legislative, the media) or utilitarian (holding resources or information) (Mainards, Raposo, and

Alves, 2011). Legitimacy is a perception regarding the actions of an entity where actions are seen

to be acceptable according to the norms of the society or organization. Urgency indicates the

immediate need for an action. It is also defined by the Merriam-Webster Dictionary as “calling for

immediate attention” or “pressing.” Considering the presence or absence of these three factors,

Mitchell et al. (1997) have proposed eight different categories of stakeholders.

Page 44 of 307

Despite the close relationship with context of an organization, where the attributes of power,

legitimacy and urgency are constantly in play, Mitchell et al.’s (1997) classification scheme is

heavily focused on the nature or type of the stakeholder, rather than answering the question “who

is a stakeholder?” This leaves us with room for exploring other classification schemes and

frameworks for analyzing stakeholders, depending on the context.

Identifying Stakeholders

Regardless of the wide variations surrounding who a stakeholder is and their categorizations, the

critical role played by stakeholders in IS implementation remains uncontested in the literature. The

body of literature on stakeholder theory is quite extensive, and this theory has been used by the

researchers to investigate organizational ambiance, strategic management, ethical concerns,

business planning processes, e-government, project management, environment management, as

well as the successful implementation of information and communication technologies and large

information systems (Mishra and Mishra 2013). Besides a great amount of recent evidence, earlier

investigations by Lyytinen and Hirschheim (1988, 1987) also demonstrated that the failure of IS

implementation is contingent upon the level of satisfaction of different stakeholder groups.

Although these findings are excellent indications of the suitability of the stakeholder lens while

considering an enterprise COTS implementation, there is a lack of attention in the area of

organizational orientation towards stakeholder engagement and stakeholder sensitivity in the

context of public-sector COTS implementation.

Despite the wide applicability of stakeholder theory in different contexts, current review of the

literature indicates that the focus is primarily centered around stakeholder categorization and

stakeholder management in general. We believe that during a large-scale commercial off-the-shelf

Page 45 of 307

(COTS) software implementation, maintaining the appropriate level of stakeholder engagement

and stakeholder sensitivity is of outmost importance for achieving success. This is also made

obvious by relating back to Mitchell et al.’s (1997) categorization of stakeholders, as it is possible

for one stakeholder group to move to a different stakeholder group by attaining necessary

attributes. But such a transition could possibly jeopardize a large-scale project if the number of

dangerous stakeholder increases. Beside the necessity of maintaining proper sensitivity, the point

of engagement and the nature of engagement are still largely unexplored in this context. A joint-

effort COTS implementation involving multiple government organizations and multiple vendors

has a much broader scope in terms of affected parties and people, compared to an internal COTS

implementation by a single organization. Since this type of implementation crosses a single

organizational border, it can also mean a loose association among stakeholders (Boonstra, 2008),

which makes their proper identification and engagement during various implementation phases

absolutely critical for success. Therefore, an investigation of stakeholder orientations during COTS

implementation can not only can help us understand the aspects of sensitivity and engagement, but

also help us identify the manner in which a stakeholder approach is used during a project.

2.4 Control configuration and control balancing

Both the consideration and engagement of all key stakeholders are essential for a successful

enterprise COTS implementation. Although the key tenet of stakeholder theory emphasizes the

“consideration” aspect, ensuring proper “engagement” calls for a different approach. The requisite

for optimal stakeholder engagement is apparent considering the many characteristics of a multi-

partner COTS implementation. Aside from being a complex task and embedded with numerous

interdependencies, enterprise COTS implementation requires eliciting input from multiple

stakeholders – often with non-overlapping knowledge (Maruping et al., 2009). Along with the

Page 46 of 307

benefits of complementary knowledge and skills offered by various stakeholders, engagement also

brings the challenge of divergent priorities, goals, motivations, and power relations, which requires

proper control and management to ensure a positive outcome (Kirsch et al. 2002; Clark et al. 1997).

Strong support for the presence of these non-technical challenges is also offered through several

earlier studies where IS project derailment had little relationship with technical challenges or

technology (Markus and Benjamin, 1996; Hartwick and Barki 1994; Robey et al. 1989).

Kirsch (1996) defined managerial control for IS projects as “attempts to ensure that individuals

working on organizational projects act according to an agreed-upon strategy to achieve desired

objectives”. The dyadic nature of control – between a controller (the person exercising control)

and a controllee (the target of control) – has inspired numerous empirical studies relating control

to organizational or IS project contexts through diverse perspectives (Kirsch, 1997, 2004).

Additionally, literature related to control theory has focused on the modes of control comprised of

two broad categories: formal and informal (Crisp, 2002). Formal control is typically documented

and initiated by the management, commonly comprised of outcome control and behaviour control

(Ouchi, 1979; Eisenhardt, 1985). Informal control is usually comprised of undocumented measures

initiated by the employee themselves (Jaworski, 1988), and are commonly identified as self-

control and clan control (Ouchi, 1980). Although the simultaneous application of multiple control

types – belonging to both formal and informal control categories – has been observed in IS

development efforts, the choice of any control type is subject to the presence of certain antecedents,

such as outcome measurability, behaviour observability, knowledge of appropriate behaviour, task

characteristics, a desire to exercise self-control, etc. (Kirsch, 1997). In a more recent investigation,

focusing on formal controls alone, Heumann et al. (2014) have identified four antecedents for

Page 47 of 307

control choices: task complexity, legitimacy concerns, performance considerations, and

performance/efficiency concerns, which are in agreement with the findings of Kirsch (1997).

Whereas antecedents are indeed a critical factor in determining the choice of control mechanism,

the circumstances leading to a change of an existing control are equally important to maintaining

the effectiveness of the controls. In addition, the concept of control effectiveness is significant, as

an applied control does not always generate the expected behaviour change. This is evident in the

paradoxical pattern identified by Tiwana and Keil (2009), which leads to the distinction between

attempted control and realized control. An analogous distinction has also been observed by

Heumann et al. (2014), who suggest that control propagation across multiple levels of hierarchy

can have different effects. This brings forth the notion of “controls recognized” by the controllee.

Considering the exclusive focus on one specific controller-controllee dyad in most prior studies,

Heumann et al.’s (2014) examination of control propagation through multiple hierarchies indeed

offers a novel contribution to IS control literature. However, research related to the simultaneous

examination of both vertical and horizontal control relations among multiple project partners of an

IS implementation is still in its infancy. Additionally, much of the control theory literature

concentrates on a single organizational context, and does not include scenarios of applying control

among non-hierarchical project partners or in client-vendor relationships. This is significantly

different compared to a single organizational context with a hierarchical relationship, as some of

the traditional control modes like behavioural control, clan control or self-control are difficult to

enact by the controller. Acknowledging this challenge in the context of a large offshore IS

implementation project, Gregory et al. (2013) adopted a different conceptualization of the control

Page 48 of 307

type by defining what a “control configuration” is. They have also conceptualized control

balancing as the act of making targeted adjustments to the control configuration, which consists of

control type, control degree, and control style.

Enterprise COTS implementations, similar to most ISD projects, face a high degree of requirement

volatility and technical complexity. With a large group of stakeholders, ensuring goal congruence

and alignment with project objectives requires the continual adoption of an appropriate control

configuration for stakeholders. The very nature of the context calls for an effective application of

the control balancing concept. Aside from the identification of control relationships among key

stakeholder groups, identifying the conditions triggering a change in this configuration is essential

for understanding control balancing. In an earlier attempt to identify the reasons for control change

during project implementation, Mahring (2002) suggested a changing relationship between

vendors and clients as one possibility. Choudhury and Sabherwal (2003) identified vendor

performance as one such reason, which can also be considered as an antecedent for the condition

identified by Mahring (2002). Considering the evolving nature of IS implementation, Kirsh (2004)

categorized these conditions into three major categories: (1) project context, containing task

characteristics, task interdependencies and project performance; (2) stakeholder context,

containing changes in knowledge and skills, the nature of relationship, the lack of common goals,

role expectation changes; and (3) global context, comprising priority differences, geographic

differences, time zone differences, and cultural differences. Although the global context can be

considered to be a more stable antecedent, a majority of the conditions can be translated to

stakeholder perceptions or relationships among them. The control balancing triggers identified by

Gregory et al. (2013) also support such conceptualizations, including shared understanding and

Page 49 of 307

unfulfilled expectations. Support for these conditions also found in earlier research focusing on

control (for example, Narayanaswamy, 2014) defined control loss as the extent to which people,

processes, and resources do not progress as expected, which resonates with the unfulfilled

expectations identified by Gregory et al. (2013). Other factors that can be classified as “risk

perception” are also hinted at as possible conditions for adjusting control mechanisms for

mitigating exchange hazards and uncertainty between exchanging partners in scenarios of

opportunism and bounded rationality (Williamson, 1985; 1991). Considering different orientations

of control balancing, as well as examining such concept vis-à-vis stakeholder engagement and

CSF, can thus offer tremendous benefits for multi-partner COTS implementation.

2.5 Critical success factors (CSF)

In parallel to processes or implementation phases, COTS implementation should also address the

critical success factors (CSF) to ensure an overall implementation success (Boynton and Zmud,

1984). The CSF in IS research is a composite concept that represents both sides of an IS

implementation coin, where one side is implementation success and the other is implementation

failure. Both success and failure are two of the oldest research traditions in IS research, as the first

ICIS (International Conference on Information Systems) conference in 1980 highlighted the

following questions: What is IS success, and what determines IS success (Petter et al. 2013). In

the context of designing management information systems, Rockart (1979; Bullen and Rockart,

1981) elaborated that there are a few factors which are fundamental for the success of a company,

and their identification is a way to ensure proper consideration is given to them. The identification

of CSFs has been widely investigated in IS research. More specifically, CSF-centric research

related to ERP implementation and adoption is extensive (Kini and Basaviah, 2013; Xu et al. 2002;

Page 50 of 307

Soh et al. 2000; Ribbers and Schoo, 2002; Scheer and Habermann, 2000; Esteves-Sousa and

Pastor-Collado, 2000; Bingi et al. 1999; Al-Mashari et al. 2003; Hong and Kim, 2002; Somers and

Nelson, 2001; Umble et al. 2003).

An early attempt to systematically study the key enabling factors of a strategic information system

(SIS) implementation is the work of Runge (1985), who used a multiple-case-study approach to

study 35 systems in Britain. Runge (1985) identified five factors as key enablers for successful

SIS implementation: (1) the presence of a product champion, (2) a high level of customer

involvement in the development, (3) an existing system as a base to build upon, (4) a high degree

of attention to marketing, and (5) the circumvention of existing IS planning and prioritization

procedures. In light of our current understanding of IS implementation, some of the factors

identified here may seem questionable, for example “the circumvention of existing IS planning

and prioritization procedures.” However, the focus on “strategic” and “external utilization” of the

systems is probably responsible for the identified enabling factors to differ from other early studies

on IS implementation.

Using a mail-in survey of IS executives, King et al. (1989) investigated significant facilitators

behind an organization’s creation of strategic systems. They identified several positive forces,

including: technical support within the firm, existing IT leadership position, and pressure from

competition. On the opposite side, they identified a set of inhibitors, including: the importance of

other priorities, the difficulty of assessing tangible contribution, the lack of appropriate planning,

and the lack of top management support.

Page 51 of 307

Although King et al. (1989) focused on both failure and success, the factors identified through

their research are far from comprehensive in nature. A much wider scope and comprehensive

consideration of success factors related to IS implementation were demonstrated in the study

conducted by Rockart and DeLong (1988). One of their critical contributions was to pave the way

for IS implementation researchers to reach agreement regarding significant factors, or CSFs,

related to IS implementation. Observing 30 cases of EIS (Enterprise Information Systems)

implementation, Rockart and DeLong (1988) identified eight areas as the most important for IS

implementation success: (1) committed executive sponsor, (2) committed operating sponsor, (3)

clear link to business objectives, (4) management of organizational resistance, (5) management of

data, (6) appropriate technology, (7) appropriate information systems staff, and (8) management

of system evolution and speed. A later study conducted by Fitzgerald (1990) in the UK also

confirmed that most of these factors are relevant to IS implementation success; however, he also

reported that “a clear link of business objectives” was largely missing, and that “organizational

resistance” was insignificant in the UK context.

Although the areas of significance identified by Rockart and DeLong (1988) have been confirmed

by several other studies (Paller and Laska, 1990; Bird, 1991; Watson et al., 1991; Whymark, 1991;

Rainer and Watson, 1995; Turban, 1995; McBride, 1997) in addition to Fitzgerald (1990), two

noticeable features of the identified factors are their strong tie to “during-implementation” phases

and the broadness of scope. Besides the scope issue, earlier studies on IS implementation success

had another key characteristic in common: the lack of focus on studying a common system. This

is another possible source of variation responsible for the divergent findings from different

researchers. The seminal work by DeLone and McLean (1992) can be seen as a much-needed

Page 52 of 307

response to this challenge, which attempted to generalize the concept of IS implementation

success. Reviewing the research published during the period 1981–87, DeLone and McLean

(1992) represent IS success as a dependent variable relating to six independent variables: (1)

system quality, (2) information quality, (3) use, (4) user satisfaction, (5) individual impact, and (6)

organizational impact. DeLone and McLean (1992) thus proposed a framework to define IS

implementation success through these six interrelated variables. The original DeLone and McLean

model of IS implementation success, a one of the most cited frameworks in IS research (Lowry et

al., 2007), has helped a significant number of subsequent IS researchers to investigate, modify, or

extend the concept of IS implementation success (Dwivedi et al., 2015).

In understanding IS success, the DeLone and McLean model has proven to be a useful framework

(Petter et al., 2013). Its wide applicability resulted from the focus on the broader dimensions of

the IS implementation. Focusing on the broader dimensions of IS implementation, Leavitt (1965)

proposed four major categories (tasks, people, technology, and structure) to explain the

interrelationship between IS and other aspects of the working environment.

Both the DeLone and McLean (1992) and Leavitt (1965) models explaining IS implementation

success consist of different dimensions that include several distinct yet correlated set of variables

or factors, which collectively contribute to each dimension. Concentrating on this second aspect,

Petter et al. (2013) attempted to take a closer look at each dimension, identifying 43 determinants

impacting IS implementation success. The distinction between the research conducted by Petter et

al. (2013) and all other existing studies is the issue of “breadth versus depth.” Despite the focus on

depth of the issue, Petter et al. (2013) organized these factors into five broader categories to align

Page 53 of 307

with the organizational change model of Leavett (1965). In addition, they identified 15 success

factors that have consistently been found to influence IS success: enjoyment, trust, user

expectations, extrinsic motivation, IT infrastructure, task compatibility, task difficulty, attitudes

toward technology, organizational role, user involvement, relationship with developers, domain

expert knowledge, management support, management processes, and organizational competence.

While both aspects of the literature – focusing on the breadth and depth of implementation

activities – are essential for understanding the phenomenon of IS implementation success, a focus

on depth clearly helps to associate identified factors with organizations actors or driving forces.

Such association may provide a tremendous benefit for organizational leaders in adopting proper

policies to effectively manipulate those underlying variables and achieve the desired benefits.

A parallel stream of research to IS implementation success is IS implementation failure, and such

research has been prominent over the last four decades (Dwivedi et al., 2015). The primary reason

for such interest can be attributed to the alarmingly high rate of IS development and

implementation project failure, which according to some sources is in the area of 70 percent

(Doherty et al., 2011). IS failure is typically defined as the mirror image of IS success (Cecez,

2015). Lyytinen and Hirschheim (1987) classified four different types of failure: (1) an IS

correspondence failure, when a system does not meet the predefined set of objectives (a system is

either abandoned or scaled down); (2) an IS process failure that denotes unsatisfactory

development or performance, and does not deliver a functioning system or runs over time and over

budget; (3) an IS interaction failure, defined as user dissatisfaction with, or poor usage of, a

Page 54 of 307

system; and (4) an IS expectation failure, indicating the inability of the information system to

fulfill stakeholders’ needs, expectations, and interests.

Studying a failed IS implementation is clearly a different perspective from the analysis of a

successful implementation as the primary motivation behind such studies is often the desire to

learn from failure (Dwivedi et al., 2015). Contributions from this stream of research are also quite

substantial, as many different causes and consequences of IS failure have been identified by

various researchers (Avison and Wilson, 2002; Barker and Frolick, 2003; Beynon-Davies, 1995,

1999; Bussen and Myers, 1997; Fitzgerald and Russo, 2005; McGrath, 2002; Nelson, 2007; Pan

et al., 2008; Scott and Vessey, 2000). Although the factors identified through IS implementation

failure studies are extensive, similar to IS success research, studies focusing on IS failure can be

divided based on their focus on either breadth or depth. For example, Nelson (2007) analyzed 99

IS implementation projects and identified 36 causes of IS failure. These causes are then further

categorized under four major dimensions: process, people, product, and technology. Several other

researchers have taken a similar approach to identifying various causes of IS implementation

failure (Al-Ahmad et al., 2009; Barclay, 2008; Dwivedi et al., 2013b; Kappelman et al., 2006;

Schmidt et al., 2001; Wallace et al., 2004; Yeo, 2002). Indeed, the broader categories or higher-

level abstractions identified through failure research are often identical to those identified through

IS success research. However, despite the similarity between major dimensions, the perspectives

of both IS success and failure are equally important to IS literature, as certain conditions or

variables may appear as salient in one perspective but not on the other.

Page 55 of 307

Early IS implementation research focusing on success and failure demonstrates considerable

differences in terms of context when compared to their more recent counterparts. Earlier research

often studied diverse forms of IS implementations, which appears to be changing in recent research

where the focus is shifting towards the implementation of common types of strategic information

systems like ERP (Enterprise Resource Planning) and CRM (Customer Relationship

Management). Most of these systems also belong to the category of packaged software or COTS,

which requires additional implementation considerations compared to a greenfield development.

Regardless of the nature of IS implementation, many success and failure factors identified in earlier

research are still largely applicable to the more recent context of enterprise IS implementation

(Dwivedi et al., 2015).

Enterprise COTS implementation, which appears to have a close resemblance with ERP

implementation, is very likely to share a sizable number of CSF identified in existing ERP

literature. It may contain additional CSFs and challenges resulting from the lack of specialization

on all business functions by the chosen COTS product. CSF have also been used to address key

implementation issues through numerous ERP implementation studies (Holland and Light, 1999;

Reel, 1999; Teltumbde, 1999; Wu and Chang, 1999; Kumar, Maheshwari, and Kumar, 2002a;

Kumar, Maheshwari, and Kumar, 2002b; Kumar, Maheshwari, and Kumar, 2003; Gupta et al.

2016). Researchers have categorized them into broader categories to identify distinct conceptual

domains and facilitate policy formulation. Cantu (1999) has proposed five dimensions of CSFs,

with a total of 22 attributes. The CSF implementation framework identified by Cantu (1999) is

presented in Table 4:

Page 56 of 307

Table 4: CSF implementation framework

Critical success factor CSF attributes

Management/organization Commitment

Education

Involvement

Project team selection

Training

Roles and responsibility

Process Alignment

Documentation

Integration

Process redesign

Technology Hardware

Software

Systems management

Interface

Data Master files

Transactional files

Data structure

Maintenance and integrity

People Education

Training

Skills development

Knowledge management

(Source: Cantu, 1999)

Page 57 of 307

Using a different perspective, a policy-related categorization of CSFs has also been observed in

the literature. Finney and Corbett (2007), through their comprehensive compilation and analysis

of ERP implementations, identified 26 CSFs belonging to either a strategic or tactical group. This

categorization is presented in Table 5. In a more recent attempt, Tarhini et al. (2015) have

systemically reviewed 35 ERP implementation research studies focusing on CSFs, and identified

a group of 51 factors, which are in agreement with the findings of Finney and Corbett (2007).

However, Tarhini et al. (2015) presented a shorter list of CSFs, based on their salience in the

literature, consisting of: top management support and commitment, training and education, project

management, clear vision and objectives of the ERP system, careful change management, and

interdepartmental communication.

Table 5: Critical Success Factors

Strategic Tactical

Visioning and planning,

Build a business case,

Project champion,

Implementation strategy and

timeframe,

Vanilla ERP,

Project management,

Change management,

Managing cultural change

Balanced team,

Project team: the best and brightest,

Communication plan,

Empowered decision makers,

Team morale and motivation,

Project cost planning and

management,

BPR and software configuration,

Legacy system consideration,

IT infrastructure

Client consultation,

Selection of ERP,

Consultant selection and

relationship,

Training and job redesign,

Troubleshooting/crises

management,

Data conversion and integrity,

System testing,

Post-implementation evaluation

(Source: Finney and Corbett, 2007)

Page 58 of 307

A review of the ERP-focused CSF literature indicates the existence of different categorizations of

the CSF attributes (Cantu 1999; Finney and Corbett, 2007), depending on the purpose of

investigation; however, they often agree in terms of actual attributes. Regardless of the

considerable similarities between COTS and ERP implementations, there are some notable

differences when it comes to implementation approach, customization, migration, and the post go-

live support model. As a result, all the CSF identified through ERP implementation research may

not be applicable for a COTS implementation. In addition, managing the CSF is another key

consideration for any enterprise COTS or IS implementation. Despite a substantial body of IS

research focusing on CSF, our review of literature indicate that it is often not possible to manage

all pertinent CSFs equally well. This is attributed to two primary reasons: (1) each CSF addressed

during implementation (i.e. the amount of time spent on each CSF) has an associated cost that

contributes to the overall project cost, and (2) missing processes or capabilities to address each

CSF during implementation (Sun et al., 2005). A closer look at each CSF attribute reveals why

managing them individually leads to an increased project cost. However, this can be minimized by

taking a process- or policy-based perspective that every project can utilize without significant

additional expenses resulting from CSF management. However, adoption of such policy related

approach call for an understanding that most of CSF are part of one of the three broader categories:

people, process, and/or environment. Table 6 attempts to map 26 CSF identified by Finney and

Corbett (2007) into one or more of these broader categories.

Page 59 of 307

Table 6: CSF and category mapping

CSF People Process Environment

Visioning and planning X

Build a business case X

Project champion

X

Implementation strategy and timeframe

X X X

Vanilla ERP

X X

Project management

X X

Change management X X X

Managing cultural change

X X X

Balanced team X X

Project team: the best and brightest, X

Communication plan X

Empowered decision makers X

Team morale and motivation X

Project cost planning and management X X

BPR and software configuration X

Page 60 of 307

Legacy system consideration X

IT infrastructure X

Client consultation, X X

Selection of ERP X X

Consultant selection and relationship X X

Training and job redesign, X X

Troubleshooting/crises management, X X

Data conversion and integrity X X

System testing X X

Post-implementation evaluation

X X

Table 6 indicates that all of the CSF identified by Finney and Corbett (2007) are related to the

people, process, and/or environment categories. Leveraging the Stakeholder engagement could be

an effective approach to better manage CSF during enterprise COTS implementation, as most CSF

can be related to an individual, group or project team, both within the organization and outside of

the organization. Another complementary aspect that can facilitate the management of CSF is

balancing of control configurations. Control balancing has a direct impact on stakeholder

engagement. Similarly, environment specific CSF can be directly influenced by the

implementation processes.

Page 61 of 307

Research concentrating on individual aspects of ERP/Information Systems (IS) implementation,

such as CSF identification, implementation processes, and application of various controls, are quite

substantial, but the investigation of the CSF management through optimally engaging stakeholders

and balancing of control through applying different control configurations is largely non-existent

in the literature (Finney and Corbett, 2007). Therefore, examining the management of CSF through

the lens of stakeholder theory and control balancing will significantly enhance our understanding

of the enterprise COTS implementation.

Page 62 of 307

3.0 COTS implementation framework

COTS implementation differs from traditional software development or enterprise system

implementation projects on several key dimensions, such as the required level of functional and

technical expertise available within an organization, the level of integration complexity, the degree

of engagement by vendors, and client control over vendors and partners. For example, in an in-

house software development effort, integration might not appear as a significant challenge due to

explicit consideration of the organizational context and operating environment during the design

and architecture phase of the development. However, COTS products are developed with a generic

audience in mind. This often presents an integration challenge for the implementing organization

of a COTS product. Regardless of such substantial difference between a traditional software

development and a COTS implementation, the implementation process for non-ERP family COTS

has not been well examined in the literature. In addition to a host organization and a COTS vendor,

adding multiple partners and third-party consulting firms to the list of stakeholders makes the

implementation process even more complicated from a stakeholder engagement, and control-

balancing perspectives.

Enterprise COTS implementation projects are part of an organization’s strategic IT initiatives and

require large investments of organizational resources, yet often fail due to inadequate management

of the critical success factors (CSF) and implementation-specific challenges. The existing

knowledge gap surrounding the COTS implementation process, the opportunity for better

understanding the successful management of CSF, and balancing control at different project stages

– in the context of enterprise COTS implementation – collectively act as the primary motivational

forces behind the current investigation. In the following sections, we lay the foundation for the

current research by introducing and explaining related concepts, as well as constructs.

Page 63 of 307

3.1 Enterprise COTS and its implementation

There are several definitions of COTS products available in both the literature and industry. For

the purpose of this research, we have adopted the definition presented by Brownsword et al. (2000)

who define COTS as: “sold, leased, or licensed to the general public, offered by a vendor trying to

profit from it, supported and evolved by the vendor, who retains the intellectual property rights;

available in multiple, identical copies; and used without source code modification”. Key marketing

features of COTS are: cost savings, ease of integration and extension, reliability and capability

(Newcomb, 2007). However, COTS adoption and implementation to replace a legacy system or to

support existing business processes require a very different tactic compared to either custom-

developed in-house solution or outsourced software development, because the COTS vendor has

developed the software with a general audience in mind. Newcomb (2007) argues that the unique

business rules and logic required by most legacy systems are often beyond the functionalities

offered by packaged solutions readily available on the market, and a COTS purchase is a riskier

solution compared to an in-house development. COTS implementations could range from a very

simple out-of-the-box implementation with no project involvement to a very complex multi-

million-dollar project such as an enterprise document management system using IBM’s P83.

Nature of COTS implementation

A COTS package solution is the integrated assembly of one or more of the following (Albert and

Brownsword, 2002a): i) COTS packages or COTS package components, ii) legacy systems (piece

of the systems being replaced), iii) reuse libraries and other reuse sources (e.g., freeware,

shareware), iv) appropriate linkage to the broader organization’s architecture with which the

3 P8 is an enterprise electronic document management system sold by IBM

Page 64 of 307

solution must interface, v) changes to the end-user business processes necessary to match the

processes provided in the COTS packages, vi) any required custom code (including wrappers and

glueware).

There are numerous COTS or packaged solutions on the market for various purposes, and they all

come with different approaches to implementation. Even the simplest level (out-of-the-box) COTS

implementation may require organizational process changes and vendor relationship management

activities. For example, out-of-the-box solutions like Microsoft Office or VMware requires large

organizations to manage licences properly, and various Integrated Development Environments

(IDEs) or source control solutions (Visual Source Safe or VSS). In addition, they require the

organization to change the way source code is developed and managed, which may trigger internal

process changes.

Figure 4: COTS customization process

The customization of COTS products ranges from “out-of-the-box” to “extended,” based on the

degree of behavioural change necessary to make the product capable of supporting organizational

business needs. Figure 4 outlines how a vanilla COTS product is customized for an organization

Page 65 of 307

through configuration change, integration, and extension. “Out-of-the-box” COTS usually refers

to products that are stand-alone and can be deployed with the vendor-provided default

configuration. COTS products in this category are generally not very expensive and used for user

productivity. Some examples are Notepad++, Beyond Compare for file comparison, Log MX for

log parsing, etc.

For enterprise-level COTS products, the customization process generally involves both

configuration and integration changes. The configuration of COTS usually refers to the aspect of

the product that defines how the COTS package should behave in certain context. The ability to

configure COTS products gives the user an opportunity to incorporate some of the unique business

requirements of an organization. For example, COTS configuration could include how an

application should handle user authentication (LDAP vs. local database), how the application

should be accessed, the duration of an inactive session, various security parameters, the file system

location on the network etc. Values for these configuration parameters are usually different for

different organizations, and can be configured through a graphical user interface (GUI) or a

configuration file specific to that COTS product. The integration of COTS, on the other hand,

refers to product compatibility and communication with other existing systems within the

organization. For example, SQL connectivity for the backend database of Microsoft’s dynamic

client relationship management (CRM), or accessing CRM database information from an outlook

email client, will impose integration requirements on the COTS product.

Page 66 of 307

Implementation Time

Figure 5: Relationship of task complexity and required time based on COTS customization style.

The extension of COTS requires a significant amount of development tasks and can result in the

addition of a completely new module for the purchased software, or some smaller custom

components commonly referred as “glueware,” to facilitate interaction with down-stream

organizational systems. As we move towards this end of the COTS customization spectrum, the

level of complexity and risk introduced by configuration changes, and the integration and

extension requirements, call for full project management activities and a dedicated team of people.

Figure 5 depicts this relationship for different COTS customization styles.

Target Implementation Style

Given the different customization styles, an Enterprise COTS implementation often involves

configuration changes, integration with existing systems, and extensions to interface with legacy

systems. Considering the complexity of this implementation process, restricting the investigation

focus to a public-sector implementation following an agile development approach further enhances

Integrated&

Extended

Integrated

Configured

Out of the Box

Complexity

Page 67 of 307

the possibility of contribution towards existing knowledge in this context. The review of IS

implementation literature in chapter 2 indicates a general tendency towards phase-specific

investigations that follow a horizontal line of query. A few researchers, however, took a mixed

approach by combining both horizontal and vertical analysis in their investigations. Due to the

context and selected implementation methodology, the current research aims to focus on both

depth and breadth by identifying low-level activities and major phases of an enterprise COTS

implementation. To further clarify the contribution of the current research and present a basis for

moving the analysis forward, an initial assessment of the target organization’s existing COTS

implementation practices has been conducted. This helped identify a general model that is

followed for most COTS implementations. Using this as a base model, the analysis of the primary

data will be conducted to identify the actual implementation process followed by project teams for

all selected cases.

Historic Implementation Practices within the Target Organization

Target organization for the current research maintains a policy requiring all project-related

documents as well as knowledge documents be stored and tracked using the in-house electronic

document management system (IBM’s electronic document management systems, EDM P8). This

practice is strictly enforced within the organization, which makes the EDM P8 a valuable data

source to begin with. In order to identify the base model or default model of COTS implementation,

we analyzed the electronic document repository (EDM P8) for 22 projects, and identified a basic

IT implementation model that is followed within the organization. This gave us a starting point for

the present research. Tables 7 and 8 provide a summary of the projects that were studied to identify

this model:

Page 68 of 307

Table 7: Analyzed project summary

Project

Type

Budget Timeline Impact

Number of

Projects

Percentage of

Projects

Type I 100K Up to 3 months Work unit /

Department

6 27.3

Type II 100K to 500K 3 months to 1

year

Cross Dept. /

Functional

8 36.4

Type III 500K to $1M 1 year to 2 year Cross-function /

Org. wide

5 22.7

Type IV 1M or Above 2 years or more External 3 13.6

Table 8: A Sample of COTS based projects analyzed

COTS Product / Project COTS Description/ Usage

EDM P8 IBM’s electronic document management system used

for enterprise document management

Microsoft Dynamic CRM Customer Relationship Management

Vizor Used for Regulatory Reporting System

Dollar Universe Enterprise Job Scheduling

Sirsi Library catalogue management

Markit’s Cadis Used for Market Data Management

Planview Enterprise reporting of work hours and resource/work

allocation

A.K.A Museum Artifact management and tracking

Maitre D'assystance Enterprise Incident and Service request/ ticketing

system

Based on the repository analysis, a general IS implementation model has been identified for the

host organization. This model consists of seven phases: initiation, requirement and planning,

analysis and conceptual design, design and architecture, development, delivery, and close out. This

initial model is shown in Figure 6.

Figure 6 : COTS implementation phases

Page 69 of 307

A closer look at Figure 6 reveals several major discrepancies between the initially identified COTS

implementation model and our proposed research topic. The current thesis aims to investigate a

micro-level implementation model for an enterprise COTS following an agile development

practice. However, the identified model indicates a sequential or waterfall implementation model,

comprising of higher level project phases. Although the notion of major stakeholders has been

incorporated into Figure 6, due to a higher level abstraction it is not possible to capture the

stakeholder engagement aspect here. Additionally, the model is also incapable of incorporating the

rich information related to information flows, vital processes and CSF. The primary and secondary

data analysis will particularly focus on these aspects while enhancing this preliminary model of

COTS implementation. Furthermore, to fulfill one of the primary objectives of this research,

implementation related processes will be analyzed to determine their relationship with the CSFs.

3.2 Stakeholder engagement in COTS implementation

While the significance of stakeholder engagement is well established in the literature,

organizational stakeholder orientation is not. Stakeholder theory is dominated by its normative

core; as Jones and Wicks (1999) write, “the interests of all stakeholders have intrinsic value, and

no set of interests is assumed to dominate the others”. This is closely aligned with Freeman’s

(1984) original call for managerial attention to all stakeholder interests as a vital success factor,

and which bear responsibility for the implications of organizational actions. However, as

organizational or managerial decisions have implications for the stakeholders of an organization,

stakeholders likewise can affect the organization. This reciprocal relationship, which often forms

the core notion of strategic stakeholder management, has been largely neglected in existing

stakeholder theory literature (Fassin, 2012).

Page 70 of 307

In a public-sector COTS implementation, stakeholder taxonomy is often significantly different

from a private or profit-oriented organization. Besides primary stakeholder groups like employees,

customers, partners, vendors and government, there are also indirect stakeholders, like civic

society and pressure groups who defend the interest of specific stakeholder groups, which thus

gives rise of the notions of reciprocity, loyalty, and fairness towards the organization by

stakeholders. Besides the possibility of an instrumental approach adopted by organizations, the

ethical dimensions of stakeholder theory could very well be abandoned by pressure groups and

stake watchers when the strategy formulation is guided by a political resource perspective (Fassin,

2012).

Besides the complexity of stakeholder taxonomy, reciprocity concerns clearly highlight the need

for a systematic understanding of organization stakeholder orientation during COTS

implementation initiatives by public-sector organizations. Earlier research on stakeholder

orientations defines the concept in terms of allocated resources (Berman et al. 1999), such as time

dedicated to certain activities. During COTS or IS implementation, the allocated time for certain

tasks clearly has implications for all stakeholders, as a positive outcome for one group may not be

perceived as such by all other stakeholders. Time allocation is only one aspect of stakeholder

orientation, and other aspects of resource allocation – such as amount, basis, and target – should

also be considered as “attributes” of organizational stakeholder orientation, which together

represent the notion of “stakeholder sensitivity.” This forms a core dimension of an organization’s

stakeholder orientation.

A joint-effort COTS implementation involving multiple government organizations and multiple

vendors has a much broader scope in terms of affected parties and people, compared to an internal

IS implementation by a single organization. Since this type of implementation crosses

Page 71 of 307

organizational borders, it can also imply a loose association among stakeholders (Boonstra, 2008),

which makes the proper identification and engagement of stakeholders during various

implementation phases absolutely critical for success. Considering the scope of investigation and

the breadth of the possible stakeholders, the current research chose to focus only on the primary

stakeholders who were directly engaged or affected by the implementation process of the COTS

implementation project. This approach is also consistent with Boddy and Buschanan’s (1986)

definition of organizational information system stakeholders: “All those who have a practical

concern for the effective application of new technologies, and who are in a position to take or to

influence decisions about why and how they are used”.

3.2.1 A Framework towards stakeholder engagement

Although Donaldson and Preston (1995) proposed that the core of stakeholder theory lies in its

normative dimension, from a COTS or IS implementation success perspective, instrumental and

descriptive versions of stakeholder theory appear to be more dominant in the literature, due to the

nature of the context. For the current research, we have also aligned ourselves with the descriptive

approach (Donaldson & Preston, 1995) to identify the salient and key stakeholders, and identify

the engagement processes pertaining to a COTS/IS implementation.

As indicated in the previous section, the identification of legitimate stakeholders is a challenging

task in many cases because of the extensive variations among different stakeholder classification

schemes that exist in the literature. Because of temporal restrictions on our study’s scope and the

exclusive focus on the COTS implementation process, we found a question-based stakeholder

Page 72 of 307

identification scheme most suitable for the purpose. A question-based framework, such as those

presented by Pouloudi and Whitley (1997) and Cavaye (1995), usually asks relevant questions

concerning the IS and its nature to identify the key stakeholders. Table 9 shows the adopted

question-based framework for identifying major stakeholder groups for the selected COTS

implementation cases.

Table 9: Stakeholder identification

Relevant Question Identified Stakeholders* Category**

Who are the initiators of the

system?

Who are the sponsors of the

system?

Who have to adopt the system and

make it work?

Who are the intended users?

Who will receive the output of the

COTS/IS?

Who are the intended developers

and operators of the COTS/IS?

Who will be impacted and affected

by the system?

Who will win or lose by using the

COTS/IS?

Page 73 of 307

* Indicates the major stakeholder groups such as client, vendor, internal support unit, project sponsor, project team member and so on. ** Indicates classification for the identified stakeholder such as internal, external, primary or secondary stakeholders.

3.2.2 Stakeholder sensitivity and stakeholder engagement

Stakeholder management is one of the dominant themes in stakeholder research and yet

stakeholder engagement – an aspect of stakeholder management – is often taken for granted or

ignored. Achterkamp and Vos (2008), and Brown and Jones (1998) have found that project failure

cannot be always attributed to ineffective project management practices; rather, inappropriate

social interactions among project stakeholders often can cause projects to fail. Some authors have

oscillated between involvement and engagement by using the attribute of reciprocity or mutual

benefit. Macleod (2012) and others have pointed to care and commitment as the determinant of

engagement levels. In an attempt to clarify the term “engagement,” Pushor (2007) explains: “In

comparison to involvement, engagement comes from en, meaning ‘make,’ and gage, meaning

‘pledge’ – to make a pledge (Harper, 2002), to make a moral commitment”.

Stakeholder engagement can be depicted as the organizational effort to involve relevant

stakeholders in a positive manner through exchange and cooperative relationships. Stakeholder

literature from business ethics, social accounting, and human resource management indicates that

stakeholder engagement typically relates to themes such as responsibility, managerialism, and

social control and construction (Greenwood, 2007). All three are closely related to the power and

authority possessed by different groups. Moreover, Pushor and Ruitenberg (2005) have argued that

flattening the command structure of the organization by sharing power and authority among

involved parties can lead to higher engagement driven by mutual benefits. Missonier and Loufrani-

Page 74 of 307

Fedida (2014) have proposed a conceptual approach towards stakeholder engagement anchored in

Actor-Network Theory, which acknowledges the dynamic and emergent nature of the relationships

among various stakeholders in a project. In addition, Missonier and Loufrani-Fedida (2014)

identified three aspects that can effectively contribute towards a solid understanding of stakeholder

engagement: (1) problematization – framing the issues of interest and identifying how they affect

the actors, (2) interessement and enrolment – assigning roles to stakeholders and the acceptance

of the assigned role by the stakeholders themselves, and (3) mobilization – reaching agreement

among stakeholders in terms of actions. These findings, however, are in agreement with Pushor’s

(2007) argument, as all three processes combined elucidate the mutual benefits for all stakeholders,

which in turn drives engagement level. Complementary to stakeholder engagement is the concept

of stakeholder sensitivity, by which an organization endorses the normative core of stakeholder

theory. Mitchell et al.’s (1997) conceptualization of stakeholder salience – based on power,

urgency and legitimacy – is one of the most significant works contributing to and understanding

of sensitivity. By stakeholder sensitivity, we denote an organization’s attitude towards the

definition of stakeholder-scope in terms of size, which of course is not a static construct.

A micro-level examination of stakeholder engagement, by focusing on a COTS implementation

project, could reveal an instrumental application (Donaldson and Preston, 1995) of stakeholder

orientation, in addition to the more prevalent normative applications. For example, stakeholder

engagement in IS implementation projects is often used as an instrument for advancing project

goals rather than aligning stakeholder interests. Therefore, it is hoped that the development of such

a framework to identify proper stakeholder orientations for a COTS implementation, by analyzing

Page 75 of 307

successful projects, can provide a way to more accurately define the applications of a stakeholder

approach.

Considering typical phases and commonly observed activities belonging to each phase of an

enterprise COTS implementation, a distinct combination of engagement and sensitivity appears to

be a practical phenomenon. Inspired by this observation, we propose a four-quadrant model

ranging from a high engagement-sensitivity level to low engagement-sensitivity, as presented in

Figure 7.

B A C D

Figure 7: Stakeholder sensitivity and stakeholder engagement

However, it must be emphasized that the themes related to each quadrant of this model will be

determined through a grounded research approach, where the analysis of primary and secondary

case data is the source of all distinct themes, if any. This new model has been proposed specifically

High Stakeholder Sensitivity

Low Stakeholder Sensitivity

High Stakeholder

Engagement Low Stakeholder

Engagement

Page 76 of 307

to help plot the interactions between the key variables of “stakeholder engagement” and

“stakeholder sensitivity.”

Stakeholder engagement, as stated earlier, can be depicted as the organizational effort to involve

relevant stakeholders in a positive manner through exchange and cooperative relationships.

Stakeholder sensitivity, on the other hand, is manifested through an organization’s attitude towards

the stakeholder-scope in terms of size and the level of influence on organizational decisions.

The x-axis of the model is labelled as “stakeholder engagement.” As indicated above, engaging

stakeholder groups in an initiative or task may require a very different set of routines than simply

considering them. In an enterprise COTS implementation project, this engagement often calls for

task performance, processes of consultation, communication, dialogue, and exchange. The

intensity of the engagement can be measured using the frequency of activities related to these

categories.

The y-axis of the model is labelled stakeholder sensitivity. Stakeholder sensitivity can be seen as

a proxy for the responsible treatment of stakeholders, or stakeholder agency in Greenwood’s

(2007) model. This construct largely indicates the breadth of the various stakeholder groups that

are considered at any given stage of the COTS implementation process and their perceived

influence on the outcome of that stage. Similar to the level of engagement, stakeholder sensitivity

can also vary considerably from one stage to another of an COTS implementation project.

Maintaining an optimal level of sensitivity versus engagement is extremely critical for COTS

projects, to prevent a certain category of stakeholder from moving to a different category; for

example, a “dormant” stakeholder moving to the category of “dangerous” stakeholder and

jeopardizing the success of an IS project. This dimension reflects the organizational attitude

Page 77 of 307

influenced by ethical and moral values. A “low” sensitive view may include only contractually

related stakeholders, such as clients, vendors, owners, etc., while a “high” sensitive view might

consider all the stakeholders impacted by the system; this can even include the general population,

who may eventually be impacted by the COTS system in some distant future.

Inspired by the concept of “optimal trust” (Wicks et al., 1999), we also suggest the possibility of

an optimal level for both dimensions at each quadrant. However, this concept of “optimal level”

can be intensely dependent on the context of engagement. Particularly for COTS implementations,

a certain level of stakeholder sensitivity deemed optimal for one stage could act as a source of

confusion and chaos for another stage. Therefore, in the proposed engagement-sensitivity model,

the dotted lines dividing each for the four quadrants are indicative of a balanced combination.

Considering the presence of stakeholder engagement and stakeholder sensitivity, we argue that a

different combination level of these two constructs will lead to distinct stakeholder orientations

for each of the implementation phases of the project. A qualitative analysis of the primary data

will help identify these distinct stakeholder orientations for the current COTS implementation

cases and their relationship with the CSF.

3.3 Control balancing in COTS implementation

Control balancing, a lightly investigated aspect within control literature, is one of the key focus of

the current research. Control balancing in COTS or IS implementation implies a targeted

adjustment of the control configuration. The following sections further elaborate the underlying

constructs of a control configuration to facilitate the understanding of control balancing in a COTS

implementation.

Page 78 of 307

A micro-level analysis of enterprise COTS implementation is expected to reveal several key

activities for each implementation phase. The completion of these tasks within a specified time

and with an expected quality is a difficult endeavour that requires the application of well-planned

control configuration. The examination of the control configuration for a multi-party enterprise

COTS implementation is thus valuable because of its underlying relationship with stakeholder

groups. Since each of the activities at different phases of the implementation process is driven by

one or more individuals, various types of controls are applied by the project to align individual

behaviours with organizational objectives (Kirsch, 1996). As identified in the literature review

section, earlier control research related to IS implementation largely focused on control

antecedents (Kirsch et al., 2002; Kirsch, 1996, 1997), the effects of control (Nidumolu and

Subramani, 2003; Henderson and Lee, 1992; Maruping et al., 2009), and the dynamics related to

control choices (Kirsch, 2004; Mähring, 2002; Choudhury and Sabherwal, 2003).

The multiple clients and multiple vendors involved in a COTS implementation are expected to

bring significant changes to the control landscape of the project. This has been indicated by

Heumann et al. (2014), who state that:

existing studies almost exclusively focus on one specific control dyad (e.g.,

Henderson and Lee, 1992; Kirsch, 1996, 1997; Kirsch et al., 2002, 2010), and those

studies that look at multiple dyads seem to ignore the hierarchical position of

controllers (Soh et al., 2011). As a consequence of this lack of multi-level analyses,

we do not know whether control actions…

Page 79 of 307

This hierarchical structure is a vital theme for the current thesis, as the solution integrators are

often engaged in a contractual relationship with the COTS vendor. Additionally, a horizontal

collaboration structure among clients also existed at the same time. To build upon our earlier

argument related to the existence of distinct stakeholder orientations, we also like to investigate

the possibility of distinct control configurations. In order to facilitate such an examination, we rely

on the theory of control balancing (Gregory et al., 2013) for this context.

Gregory et al. (2013) define control configuration utilizing three distinct dimensions related to

control: control types, control degree, and control styles. Control types are typically related to

attaining certain goals and can be classified into three different categories: a) Procedural controls

that are applied to improve efficiency and effectiveness, such as tracking project milestones, daily

status reports, definition of roles and responsibilities etc.; b) Social controls that are applied to

develop a shared understanding among project partners through social and team-building

activities, informal information exchange, intercultural workshops, etc.; and c) Hybrid controls

that combine both procedural and social controls in order to achieve both goals simultaneously.

Hybrid controls may include common understanding workshops, site visits, and reflection-in-

action sessions.

The second dimension, control degree, indicates the measurement or intensity of the applied

control. Control degree can vary from tight to relaxed, where tight indicates a high volume or high

intensity of control, and relaxed indicates the opposite. The final dimension, control style, is related

to the concept of controller versus controllee roles and direction of the control application. Typical

control styles are either: a) Unilateral, where one party, typically the client, controls the other

party, typically the vendor: or b) Bilateral, where the client and the vendor use control mechanisms

jointly with an emphasis on mutual agreement of control selection and use.

Page 80 of 307

Based on the above definition of control configuration, Gregory et al. (2013) proposed three

distinct control configurations: a) authoritative, b) coordinated, and c) trust-based. The theory of

control balancing argues that a dynamic nature of control configuration for an IS implementation

project – which is driven by the level of shared understanding among the partners (Gregory et al.,

2013), and is the application of the control theory in our current context – demonstrates great

potential to validate and extend the concept of control balancing. As the current thesis proposes

the existence of dynamic stakeholder orientations during a COTS implementation project, it is also

possible to identify the existence of multiple control configurations through an examination of

different implementation phases.

We have utilized the control configurations developed by Gregory et al., 2013 (presented in Table

10) to facilitate the analysis of the primary data and identify the application of control balancing

in the current case.

Table 10: Control Configurations

Control

Configuration

Definition Meaning and Implication Typical Examples of

Control Mechanisms

Authoritative

control

• Procedural

control portfolio

• Tight control

degree

• Unilateral

control style

An approach to

control based on a

traditional

client–vendor

perspective

Client–vendor perspective:

• Means that exchange parties see

themselves as client and vendor with

clearly separated roles and

responsibilities

• Implies that the client specifies

requirements, the vendor delivers upon

them, and control selection and use are

driven by the client

• Results in situations in which the client

dominates the relationship from a

managerial perspective

• Status reviews

• Detailed examination of

deliverables

• Tracking of project

goals

• Definition of

client/vendor roles and

responsibilities

• Operational process

documents

Page 81 of 307

Coordinated

control

• Hybrid control

portfolio

• Tight control

degree

• Bilateral control

Style

An approach to

control based on a

coordination

perspective

Coordination perspective:


themselves as partners that need to

closely coordinate activities

• Implies that client and vendor work

toward coordinated, shared goals, search

jointly for problem solutions, and select

and use control mechanisms accordingly

• Results in situations in which neither

the client nor the vendor dominates

• Joint parallel testing

approach

• Site visits

• Workshops

• Reflection-in-action

sessions

• Coaching of team

members

• Joint communication

plan

• Lessons learned

sessions

• Feedback mechanisms

Trust-based

control

• Social control

portfolio

• Relaxed control

degree

• Bilateral control

Style

An approach to

control based on a

trust-based

perspective

Trust-based perspective:


themselves as part of the same team

based upon mutual trust and shared

understanding

• Implies that the vendor delivers without

the client getting deeply involved in the

process, and fewer resources are

dedicated to controlling the vendor

• Results in situations in which problems

are solved instantly, new ideas for

improvements are generated, and the

vendor takes over more responsibility in

the relationship

• Direct and pragmatic

coordination

• Brainstorming sessions

• Spontaneous

communication (e.g.,

facilitated through open

plan offices)

• Informal exchange of

ideas

3.4 Mapping the research questions and objectives

This research investigates an enterprise COTS implementation project by a government

organization from multiple perspectives. Both the review of the relevant literature and the

preliminary analysis of the organization practices indicate a significant knowledge gap and

Page 82 of 307

potential for contribution by identifying a micro-level COTS implementation model. Additionally,

a task-based analysis of implementation phases can help reveal critical information flows, inter-

phase relationships, processes and tools that can all be utilized to achieve different project

objectives.

Secondly, adding a CSF perspective to the current thesis will facilitate the study of multiple related

concepts, such as implementation processes, stakeholder orientation and control balancing. As the

review of existing CSF literature indicates, most CSF related to IS implementation can be mapped

to people, process or environment, and a micro-level implementation model will help identify the

process and tools that are directly connected to the management of various CSFs during the

implementation. Although CSFs (and their relationships to stakeholder engagement and control

balancing) are a major aspect of the current thesis, the identification of CSFs is not. Therefore, we

will be relying on the subset of CSFs identified through each project’s lessons learned activity as

selected candidates for examination through the current research. This list of CSFs, identified by

the implementing organization, will then be further enhanced by the CSF identified by existing

research in this domain.

Thirdly, CSFs belonging to the “people” category often appears to be a salient dimension in IS

implementation research. Our reliance on stakeholder theory and the theory of control balancing

draws attention to this particular aspect of existing IS implementation literature. More specifically,

the possibility of multiple stakeholder orientations and multiple control configurations can

potentially have a direct impact on various CSFs. This addresses the concern identified in our

fourth and sixth research questions, which focus on the management of CSFs through stakeholder

engagement and control balancing.

Page 83 of 307

Finally, a naturally related question to the aspect of control balancing – “what causes it?” – is

presented in our third research question. Gregory et al. (2013) identified the deterioration of

“shared understanding” as a possible trigger factor for the control configuration to change. For a

complex multi-case investigation like the present one, it is possible to have multiple trigger factors

responsible for the dynamic nature of control balancing. Therefore, investigating the possible

causes of control balancing can be seen as an extension of and contribution to the relevant theory.

Table 11: Situating Research Questions

Research Objectives Research Questions

1 To identify a micro-level model for

enterprise COTS implementation by

focusing on implementation phases,

processes, and tools.

What is an activity and process-based model

for enterprise COTS implementation?

2 To identify a set of CSFs that are relevant

for enterprise COTS implementation and

examines the management aspect of them

through process, stakeholder engagement

and control balancing.

How can organizational processes and tools contribute

to COTS implementation success?

How can CSFs and other challenges be successfully

managed through optimal stakeholder engagement by

the project?

How can CSFs and other challenges be

successfully managed through optimal


3 To develop a theoretically sound framework

that can be used to identify an

organization’s stakeholder orientation

during an enterprise COTS implementation

project.

What is nature of stakeholder orientation during a

COTS implementation for a public sector IS

implementation?

4 To validate the theory of control

balancing (Gregory, Beck, & Keil, 2013) by

examining the dynamic relationships among

the project partners from one phase to

another phase of the implementation

process.

What is the nature of control configuration in

a multi-partner COTS implementation and

what are the factors responsible for the

application of control balancing?

Page 84 of 307

4.0 Research methodology

Chapter 4 describes the research approach, design, and method that have been used to investigate

the research questions stated at the beginning of this thesis.

4.1 Overall approach

Current research is a grounded theory research based on multiple empirical cases. The basic

approach adopted for determining an agile COTS implementation and its relevance to stakeholder

orientation as well as control balancing is qualitative. We used interviews as the primary source of

data. A qualitative approach is selected to understand the meaning that “individuals …ascribe to

social or human problem” (Cresswell, 2009:4). In this research, identification of a micro-level or

detailed COTS implementation model and whether the critical success factors can be effectively

managed through adjusting stakeholder orientation and control configuration are the investigated

problems related to the successful implementation of an enterprise COTS. The meaning ascribed

by individuals is relevant in this case because the concept of success may be subjectively perceived

as well as objectively measurable.

4.2 Research design

The design for the current research uses multiple case studies to inquire about the actual process

of COTS implementation followed by a Canadian government organization. We have analyzed a

total of four cases through current research which includes two successful and two failed

implementations of an enterprise COTS product. A failed or ‘not so successful’ project can be

defined by looking at multiple criteria such as a lower return on investment (ROI), higher support

cost, user satisfaction, consideration for possible replacement etc. The inquiry also investigates the

Page 85 of 307

nature of stakeholder engagement and control balancing, and how they influenced the related CSFs

for the selected projects.

Over the past decades, case studies have gained considerable acceptance as a business research

method, particularly where holistic examination of complex phenomena in real-life settings is

required (Benbasat et al. 1987; Yin 1994). Due to its flexibility and individual variation, case study

has been consistently gaining popularity in IS research (Cavaye, 1996). In a case study research,

it is not only possible to combine several qualitative data collection methods such as interviews,

documentation, and observations but also include quantitative data such as questionnaires and time

series to provide richness and flexibility to the research process.

A case study research design can vary according to the type of case study one undertakes. Firstly,

akin to other research methods, the case study research can fulfill several purposes, the most typical

being exploration, description, and explanation (Post and Andrew, 1982; Babbie, 2001; Yin,

2003). Each of these three purposes is distinct and is employed to study different phenomenon. A

descriptive case study could provide us with a detailed description of a novel phenomenon, whilst

an explanatory case study could offer an aspect of sense-making by explaining why an event

occurred (Post and Andrew, 1982; Yin, 2003). The third type of case studies, an exploratory study,

usually employed to better understand a previously unexplored phenomenon (Babbie, 2001).

Considering the scope and the research questions posed earlier, both descriptive and exploratory

aspects are salient throughout the investigation approach taken for the current research.

Page 86 of 307

A case study design is appropriate because the phenomenon of interest, agile COTS

implementation in public sector, is a contemporary one that must be observed “within its real-life

context” (Dube and Pare, 2003:598). A case study design is also appropriate for the current

investigation as the research questions include ‘how’ and ‘why’ questions (Yin, 1994).

Additionally, such a complex phenomenon of enterprise COTS implementation is not suitable for

experimental inquiry simply because of the employee would perform such activities only as a part

of their day to day jobs not under controlled conditions.

“A case study is an empirical inquiry that investigates a contemporary phenomenon

within its real-life context, especially when the boundaries between phenomenon

and context are not clearly evident” (Yin, 1994:13).

Several earlier researchers acknowledged that immersion in rich data from multiple sources is a

strength of the case study method that makes it a valuable research method for developing

emergent theory, or for sharpening existing theory (Post and Andrew, 1982; Babbie, 2001; Voss

et al. 2002; Eisenhardt and Graebner, 2007; Siggelkow, 2007; Weick, 2007). The theory building

strengths of a case study research provide further motivation to use this method for our descriptive-

exploratory research which aims to refine and elaborate existing theories.

To investigate a phenomenon, a case study research can adopt either a single case or multiple cases

approach. (Yin, 2009; Eisenhardt, 1989). Single cases are usually the most common and simpler

approach for case study research and are often chosen to investigate rare, unusual or critical

circumstances (Yin, 2009).

For the current research, a multiple case study design is adopted because multiple cases are

required to adequately examine a COTS implementation process and develop a theory elaboration

Page 87 of 307

or extension. A multiple case study approach provides us with a broader access to the empirical

observations and an opportunity to collect sufficient data. Both of these aspects can support the

development of a valid theoretical model. The value of a multiple case approach, in highlighting

repeatable phenomena, has been previously argued by Leonard-Barton (1990) and Zivkovic

(2012). While developing robust and reliable theoretical constructs, a multi-case approach is also

considered more suitable over a single case approach by the wider research community (Bourgeois

and Eisenhardt, 1988; Yin, 1994; Dube and Pare, 2003).

A multi-case research increases the internal validity of identified constructs and proposed relations

through recursive pattern matching between theory and multiple sets of empirical evidence. This

is achieved through examining the same phenomenon across different contexts or environments

(Campbell, 1975; Bourgeois and Eisenhardt, 1988; Pare, 2004; Gibbert, Ruigrok, and Wicki,

2008).

In addition, a multiple case study research is often used to maintain external validity of a study

and ensure the generalizability of research findings by examining the same phenomenon in

different organizational context (Yin, 1981; 1994; 2003; Gibbert, Ruigrok and Wicki, 2008). Yin

(2009) states:

“Multiple cases resemble multiple experiments [... ] that means each case must be carefully

selected so that it (a) predict similar (a literal replication) or (b) predicts contrasting results but for

anticipatable reasons (a theoretical replication).”

Yin (2009) also suggests that a few cases (two to four) design would be ideal for literal replications,

whereas a few additional cases (four to six) can be used to pursue two different patterns of

theoretical replications. Since the current research is aiming more towards the literal replication of

Page 88 of 307

the findings, a sample size of four has been selected for this research. The primary reasoning for

selecting both successful and failed implementations is to eliminate any environmental and/or

organizational biases relating to the CSF.

4.3 Research method

Primary data was gathered through face-to-face semi-structured interviews. Informants selection

process took necessary precautions to ensure a representative population as a sample. To support

this, we categorized the stakeholder population into different role groups. Typical role groups

included project manager, project sponsor, end user, developer, solution architect, vendor, business

analyst, technical/enterprise architect etc. Interview participants included one or more individuals

from each role groups involved with the implementation of a COTS product. The focus of

interviews revolved around the implementation process, activity details, stakeholder

considerations, CSF, and control mechanisms.

Although a survey as a research instrument would be much more economical and capable of

reaching a large number of informants in comparison to an interview method (Cresswell, 2009),

we chose to utilize interviews because it provides the ability to explore an issue in depth.

Interviews also help maintain a balance by allowing interviewees to express their opinions,

concerns, point of views in their own words and enabling the researcher to guide the line of

questioning (Cresswell, 2009).

An observational method is also judged inappropriate because an observation can only be applied

during the implementation process and not upon the completion of the project. Since our data is

collected after the project go-live stage, to ensure a proper selection of both successful and failed

COTS implementations, using an observational method is not possible. In addition, an observation

Page 89 of 307

based approach is unlikely to allow the researcher understand the motivation behind using different

tools, processes and their interconnections with stakeholder orientation and the management of

CSF.

The key steps followed by this research are summarized in Table 12.

Table 12: Research Steps

Step Activity Expected Outcome

1 Review

Literature

Critical review of relevant literature which include: IS implementation, Critical

Success Factors, Stakeholder theory, and Control balancing. Provides guiding

theoretical base (Eisenhardt & Graebner, 2007; Layder, 1993, Yin, 1994)

2 Synthesize

Literature

Deductively related the existing theories and develop a tentative, a priori

theoretical link (Layder, 1993). Constructs and other tools identified from the

literature to facilitate data collection and analysis (Christensen, 2006).

3 Pose Research

Questions

Identification of key research questions, justified by the research gap and

literature synthesis (Layder, 1993; Yin, 1994).

4 Design

Interviews

Checklist to develop semi-structured interview questions, tentative interview

schedule, other process and tools to assist proper execution of a qualitative

interview.

5 Select Sample Identification of employees, vendors, partners and management who are

associated with the COTS implementation process related to the selected cases.

Focus on cases “chosen for the likelihood that they will offer theoretical insight”

(Eisenhardt & Grabner, 2007:27).

6 Conduct Interviews Audio records from the interviews, memos, contact summary sheet, memos,

transcriptions etc. They all are collectively the qualitative data set resulting from

the interview process.

7 Analyze Data Identification of process and multilevel codes from the data, identification of

anomalies and insight from the data that are not identified or explained by the

linked theories from step 2 ( Bryman & Bell, 2003; Christensen, 2006).

8 Refine Theoretical

Link

Based on the new insight or anomalies, modify the linked theories from step 2

(Carlile & Christensen 2005).

4.3.1 Review literature

Page 90 of 307

This thesis started with a review of the literature and identified possibilities of theory elaboration

as well as theory extension in chapter 2. The literature review provides “a rich theoretical

framework” (Yin, 1994:13) as the foundation for the investigation where the “existing theories

play a major part in the formulation of the new theory” (Layder 1993: 25).

The literature review follows a deductive process using existing theories and identifies potential

relationships between theories and phenomena (Weick, 1989). A review of the related literature

also provides the overall objectives for the research (Eisenhardt and Grabner, 2007) and guide the

interview design in subsequent stages of the research (Cresswell, 2009, Yin, 1994).

4.3.2 Synthesize literature

A link among stakeholder theory, control balancing theory and critical success factors is proposed

in Chapter 3. This link is derived through a review of existing CSF literature. Our review indicates

that most of the CSF identified through IS implementation research are related to people, process

or environment (Section 2.5 and Section 3.4) of an organization. Leveraging both the stakeholder

theory and the theory of control balancing, we acknowledge the existence of distinct groups of

CSF related to a COTS implementation. Subsequently, we reviewed existing literature to explain

how an enterprise COTS implementation can have multiple stakeholder orientations which, in turn,

can require multiple control configurations to successfully manage relevant CSF.

Constructs, processes and other tools were developed from the review of the literature in order to

facilitate the data analysis process (Christensen, 2006). These include:

1. Process related to IS and COTS implementation that will be used to guide the current study

of implementation process and contribute both horizontally and vertically to the existing

models in literature

Page 91 of 307

2. Definition of stakeholder engagement to contrast it with stakeholder sensitivity; a question-

based framework for identifying stakeholders.

3. A systematic way to categorize and study the critical success factors for IS implementation

process and relating them to stakeholders.

4. A framework to analyze control configurations in order to examine the control balancing

efforts during a COTS implementation.

4.3.3 Map research questions

The key research questions that will be investigated through current thesis are posed in Chapter 3

of this document. Section 3.1 delves deeper into the enterprise COTS implementation process to

identify our existing knowledge concerning implementation models, processes, and tools in this

context (research questions 1 & 2). Section 3.2 examines the engagement of stakeholders during a

COTS implementation process and sets the stage for probing organizational orientation towards

the stakeholders in such context (research questions 5 & 6). This section also proposes a potential

relation between the stakeholder orientation and CSF. Section 3.3 presents the concept of control

balancing to manage CSF during a COTS implementation (research question 4). In addition, this

section highlights the significance of factors influencing control configurations and the possibility

of multiple control configurations (research question 3).

4.3.4 Design interviews

Interview for the current research focused on four key areas directly related to six research

questions posed in Chapter 1 (section 1.3) and Chapter 3 (section 3.4) of this document. They are:

a) COTS implementation process and activities, b) management of stakeholders, c) CSF for the

implementation, and d) maintaining control throughout the project.

Page 92 of 307

Enterprise COTS implementation is a “situated activity” in Layder’s terminology where the firm

provides the “settings”. Potential interview questions are shown in Table 13 based on Layder

(1993).

Table 13: Potential interview questions

Situated Activity Checklist Potential Questions

“Who is doing what, to whom in this episode or strip of

interaction?”

What phases of enterprise COTS implementation the

project document followed?

What are the tools used to monitor implementation

progress of the project?

“What are the recurrent features of the behaviour and

interaction?”

How often a given control measure was used?

“What social functions do these patterns of behaviours

and forms of interaction serves? Are they intended or

unintended by the participants?”

Does an approach towards stakeholder align or

supports certain organizational mandate?

Was a given control able to minimize certain

unexpected outcome or risk?

“What forms of communication are being used?” How information was shared among the stakeholders?

How decisions and escalations were handled?

“What aspects of the setting are pertinent to the

analysis of particular episodes of activity? How do they

influence the actions?”

How decisions were implemented?

4.3.5 Select sample

For the current research, we investigated four different cases of COTS implementation. The

following criteria were used to select the organization, cases, and interview participants.

Since the current research aimed to investigate enterprise COTS implementation for the public

sector, our target organization must be a Canadian government department. Additional factors

such as multiple clients, and multiple vendors were also considered as selection criteria for the

projects. An overall project budget between 10 and 20 million dollars and a project duration

Page 93 of 307

between one and three years were used as guidelines to avoid selection of very small projects.

Enterprise COTS implies an interconnected and packaged software which is very different

compared to a stand-alone COTS product. This aspect was also considered while selecting the

sample cases for this research. A balanced selection of both successfully completed and failed

projects was also ensured. This resulted in the selection of two successful and two failed COTS

implementation projects for analysis. A failure for a completed project is often an extremely

difficult concept to define. This is because of the fact that a failure is often not apparent

immediately after the project’s completion. It only becomes visible after a COTS product has been

used for a few months. Therefore, we focused on “client dissatisfaction”, “lower than anticipated

ROI”, and “high support cost” to identify failed projects instead of relying on project terminations.

In addition, categorization for each selected project was reviewed by a business user of that COTS

product. The purpose of this review by an end-user was to confirm the validity of such

categorization and avoid any bias introduced by the researcher.

Initial interview targeted a total of 39 primary informants. Each interview lasted approximately an

hour and 20 minutes. This included a minimum of nine participants from each selected Case. Our

selection process considered people from different roles to construct a representative sample.

Participation to this face to face interview was voluntary and conformed to all ethical research

practices. For each selected case, number of interview participants and their roles are presented in

Table 14 through Table 17 below.

Page 94 of 307

Table 14: Case A Informants Summary

Type of Primary Data Description of the Informants Roles

Face to Face Semi-

structured Interview

Duration for each

Interview: 80 minutes

Role of Interviewee # of Interviews

with Project team

# of interviews

with vendor and

solution integrator

# of interviews

for Client/

Business Units

Project Manager 1 1

Business Lead 1

Business Analyst 1 1

Enterprise Architect 1

Data Architect 1

Solution Architect 1

Project team

member

2 1

Total # of

interviews

11

Table 15: Case B Informants Summary


Face to Face Semi-


Duration for each



with Project team

# of interviews

with vendor and

solution integrator

# of interviews

for Client/

Business Units

Project Manager 1 1

Business Lead 1



Data Architect 1

Solution Architect 1 1

Project team

member

1 2

Total # of

interviews

12

Page 95 of 307

Table 16: Case C Informants Summary


Face to Face Semi-


Duration for each



with Project team

# of interviews

with vendor and

solution integrator

# of interviews

for Client/

Business Units

Project Manager 1

Business Lead 1


Enterprise Architect

Data Architect 1

Solution Architect

Project team

member

3 1

Total # of

interviews

9

Table 17: Case D Informants Summary


Face to Face Semi-


Duration for each



with Project team

# of interviews

with vendor and

solution integrator

# of interviews

for Client/

Business Units

Project Manager 1

Business Lead 1

Business Analyst 1

Enterprise Architect

Data Architect

Solution Architect 1

Project team

member

3 1 1

Total # of

interviews

9

Page 96 of 307

In addition to primary interview data, we have also collected secondary data related to each

selected project. This was used to triangulate the research findings and enhance the validity of the

current research (Yin, 1994). Secondary sources of data are indicated in Table 18.

Tabel 18: Secondary Data Used for Triangulation

Types of Documents Purpose of Analysis

Meeting Minutes

Design Decision Documents

Project Planning documents

Test Cases and Plans

Analyze to determine the activities that took place on

certain phases, boundaries of the phases, links to other

phases, stakeholder considered and stakeholder

engaged, control configuration

4.3.6 Conduct interviews

Organizational approval and the approval of Carleton’s ethics committee have been obtained prior

to conducting the interviews for this research (please refer to appendix for approvals). The

participants were interviewed through a semi-structured interview:

“In semi-structured interviews the interviewer has a list of topics or questions that

that he or she wants to cover, although this list will be flexibly adhered to according

to the emergent demands of the interview situation. Semi-structured interviews are

designed to let interviewees respond in an open-ended way…..This individual’s

own interpretations and meaning are allowed to surface in the interview data”

(Layder, 1993:41).

Page 97 of 307

Interviews were conducted primarily to understand why certain process was followed, links

between phases, tools and processes that were utilized. Additionally, motivations for considering

‘who is a stakeholder’ and ‘what is a process’ are important and have been investigated. These

aspects directly relate to the investigation of organizational stakeholder orientation.

Interviews were conducted in person and were tape-recorded, with approval from the interviewer.

Interview data were used without identifying the interviewee. The interviewer took notes

immediately which served as ‘field notes’ and were used for guiding subsequent analysis of the

interview data.

4.3.7 Analyze data

With an aim to investigate the nature of control configuration, stakeholder orientation,

implementation processes and their collective relationship with CSF and generate a descriptive

and explanatory theory in this regard, we have resorted to a grounded theory method (Glaser &

Strauss, 1967; Martin & Turner, 1986; Turner, 1983). A grounded theory approach has been

extensively and effectively used for IS research in organizational context (Staehr, Shanks, &

Seddon, 2012), however, our primary motivations for adopting this approach are three:

First, is our ontological belief inclined more towards interpretivist approach as opposed to a realist

approach believing in an objective or external reality. This persuaded us to consider the perspective

of ‘social construction’ of reality by human actors in an IS implementation scenario (Berger and

Luckman, 1989). This led to our underlying epistemological belief that knowledge can be best

achieved by getting inside the world of those generating it (Orlikowski & Baroudi, 1991). As a

result, generative approach appears to be well suited for this purpose where actions and perceptions

of human stakeholders are investigated in a changing context of IS implementation. Therefore, a

Page 98 of 307

grounded theory approach is an ideal choice for our interpretivist approach as it "is an inductive,

theory discovery methodology that allows the researcher to develop a theoretical account of the

general features of a topic while simultaneously grounding the account in empirical observations

or data" (Martin and Turner, 1986, p. 141).

Second, COTS implementations investigated through this research is situated within an

organizational context. In addition, conceptualizing project as a ‘temporary organization’ adds

further complexity to the context. Although the role of context is not a primary premise for this

research, the contextual complexities introduced by the organization and the project must be

incorporated into an understanding of the phenomenon, rather than be simplified or ignored, to

produce accurate and useful results. This is also a major premise in of grounded theory research

(Martin & Turner, 1986; Pettigrew, 1990). Enterprise COTS implementation is a complex

phenomenon when considered from a process, stakeholder and control perspective and much of

the complexity often related to the context. As an isolated investigation of this phenomenon is not

very useful, we favour the use of a grounded theory method for the current research considering

the ability to incorporates the complexities of the organisational context (Orlikowski, 1993; Martin

and Turner, 1986; Pettigrew, 1990).

Lastly, grounded theory method originated with the aim to generate empirically grounded theory

(Glaser and Strauss, 1967), thus it is intended ‘to discover what is going on, rather than assuming

what should go on’ (Glaser, 1978:159) based on the systematic exploration of a phenomenon.

Therefore, supported by a rigorous, systematic and comprehensive approach to data collection and

analysis of multiple enterprise COTS implementations, our goal to identify useful theoretical

Page 99 of 307

conceptualisations, that explains the relationship between process, control, stakeholder orientation

and CSF, closely aligns with methodological purpose of a grounded theory.

To facilitate a rigorous, systematic and comprehensive analysis process, we have transcribed all

interview recordings. The transcribed interviews as well as the field notes taken during the

interview have been analyzed inductively (Bryman & Bell, 2003). As indicated earlier, the current

research follows a ‘Grounded theory’ approach for analyzing the qualitative data. Overall

grounded theory process followed by the current research is outlined in Table 12. Motivation to

use a grounded theory approach rises considering several aspects of an enterprise COTS

implementation. Identifying organizational stakeholder orientations and distinct control

configurations may help identify new constructs that drives these orientations or configurations.

However, these new constructs can only be identified through proper theoretical coding and theory

integration process for those codes.

Although we intend to respect the overall principle that grounded theory must explain the

behaviour being analyzed and fit the data collected from the case (Glaser and Strauss, 1967), we

primarily adopted the theoretical coding method used by Gregory, Beck and Keil (2013). Gregory

et al.’s (2013) theoretical coding and saturated category identification method is guided by

Glaser’s (1978, 1982) approach to the grounded theory research. Immediately after the problem-

formulation and case-study design phases, we performed an “open coding,” to identify various

indicators of stakeholder engagement and stakeholder sensitivity. In the next stage, we delimited

the coding process. The final three stages of the research process – theoretical coding, scaling up

and theoretical integration –focused primarily on establishing a theoretical connection between the

identified categories or orientations and the existing theoretical framework of the domain. Table

Page 100 of 307

19 outlines the details of the process that was followed while analyzing data and mapping them

into the core categories. This is different from the previously presented research steps in Table 12

as Table 19 focuses on the grounded theory process that was followed by the current research.

Table 19: Grounded Theory Research Process: Steps, Tasks and Outcomes

Research

Steps

Tasks Outcome

Problem

Formulation

Establishing the phenomenon in terms of

its practical relevance as a prerequisite to

produce grounded theory that has ‘grab’

(Glaser and Strauss 1967).

State what the problem is from a practice

and theory perspective and why it is

important (Van de Ven 2007).

Screening prior research to identify gaps

in the literature (Urquhart 2007).

Identified the enterprise COTS

implementation projects as a practically

relevant problem that many organizations

and project teams often struggle with.

Problem identified as COTS

implementation process, and managing of

CSF is often not well understood and

sources of struggle for the project team.

Identified gaps in the literature on COTS

implementation process, and management

of CSF through process, control

configurations, and stakeholder orientations

for enterprise COTS/IS implementations.

Multiple

case

study design

• Establishing engaged relationship

withpractitioners and negotiating access to

data (Pan and Tan 2011; Van de Ven 2007).

• Selecting a multiple candidate studies and

motivating the rationale for conducting a

multiple case study, e.g., the main criterion for

revelatory cases is “when an investigator has

an opportunity to observe and analyze a

phenomenon previously inaccessible to

scientific investigation” (Yin 2003, p. 42).

Obtained approval from Carleton’s ethics

committee to conduct an empirical study

containing human subject

Obtained approval from the leadership team

and reached an agreement with a large,

reputable government organization to

conduct a multiple case study of four

different enterprise COTS implementation

projects; Approval included access to both

primary data collection through face to face

interviews and secondary data collection

from organization’s electronic repositories.

Page 101 of 307

Selected “revelatory cases”: four instances

of enterprise COTS implementation

targeting four distinct operations areas of

the organization, which has been

inaccessible to scientific investigation

before (Choudhury and Sabherwal 2003, p.

313; Dibbern et al. 2008, p. 359).

Open coding

&

Data

collection

• Gathering rich primary and secondary data,

including intensive interviewing (Charmaz

2006).

• Coding the data and understanding what it is

about by going through interview transcripts

line by line, assigning conceptual labels to

data

segments, and identifying core categories

(Glaser 1978).

• Adhering to the principle of emergence of

grounded theory: categories should emerge

from the data in the sense that they must “fit”

(they must be readily, not forcibly, applicable

to and indicated by the data under study) and

“work” (they must be meaningfully relevant to

and be able to explain the behaviour under

study) (Glaser and Strauss 1967).

• Triangulating and comparing different slices

of data to find similarities and differences

(Charmaz 2006).

Conducted multiple in-person interviews,

and obtained project documentation such as

charters, gating decisions, project

management plan, meeting minutes,

escalations, contract related documents,

project completions reports and post-

deployment incident/problem records.

Generated more than 240 initial codes and

more than 400 pages of notes and analytical

memos (including spreadsheet

categorization notes/analysis).

Identified categories related to

implementation phases for selected projects

without applying existing practices

identified in literature

Identified categories related to the existence

of multiple and changing control categories

without applying existing concepts

identified in control literature that are

relevant the understanding of control

balancing decisions in COTS

implementation projects.

Identified categories related to the existence

of multiple and changing stakeholder

orientation without applying existing

concepts identified in stakeholder literature

that are relevant the understanding of

Page 102 of 307

stakeholder management in COTS

implementation projects.

Compared multiple perspectives, including

project team, business users, partners,

vendors, internal support teams, and the

leadership team and compared multiple

sources of data.

Compared the use of multiple processes and

tool from project’s intent perspective

Selective

coding

&

Data

collection

• Delimiting further coding to only those

concepts and variables that relate to the

emerged categories (Glaser 1978).

• Making constant comparisons between

instances of data labelled as a particular

category and other instances of data in the

same category to substantiate categories

(Urquhart et al. 2010).

• Further data collection guided by the

principle of theoretical sampling, i.e., deciding

on analytic grounds where to sample from

next (Glaser and Strauss 1967, p. 45).

Delimited further coding to a set of

tentative core categories which evolved into

control orientations and stakeholder

orientations

Followed the constant comparisons

technique of grounded theory research,

focusing on the development of categories

and concepts by constantly comparing data

to data (e.g., primary interview data to

secondary data such as project

documentation from electronic repository)

Theoretical

coding &

data

collection

• Analysis and specification of theoretical

relationships between core concepts and

categories (Bryant and Charmaz 2007, p. 25).

This theoretical coding (Glaser 1978), also

referred to as iterative conceptualization

(Urquhart et al. 2010), is aimed at increasing

the level of abstraction, relating categories to

each other, and clarifying which categories

may be properties of others.

Constructed detailed case narratives to

capture sequences of events and their

relationship at different project phases.

Validated the application of three different

control configurations with four distinct

intent which forms the four distinct control

orientations, influenced by three salient

trigger factors

Page 103 of 307

Combining control configurations with the

choice of processes, we have identified four

distinct stakeholder orientations

Scaling up • Engaging with other theories for theory

building: To raise the level of

conceptualization and scale up the emerging

theory, existing theories or concepts should be

used for comparisons (Urquhart 2007).

Thereby, meta theories and theoretical

categories with limited empirical content and

general scope are particularly suitable as

heuristic or sensitizing devices (Kelle 2007).

• Grouping higher level categories into

broader

themes with the goal of increasing the

generalizability of the theory and being able to

relate the theory to the broader literature

(Urquhart et al. 2010).

Engaged with literature on control

balancing (Gregory et al. 2014,) in ISD

context

Engaged with literature on stakeholder

management and engagement (Donanldson

and Preston, 1995)

Conceptualized control orientation and

stakeholder orientation as higher level

category

Defined it as making targeting adjust in

control configuration and stakeholder

orientations in terms of engagement level

and sensitivity level allows the project team

to manage CSF for the project.

Conceptualized four distinct orientations of

control configurations, and four distinct

stakeholder orientations

Theoretical

Integration

• Relating the theory to other theories in the

same or similar field by comparing the

substantive theory generated with other,

previously developed theories (Glaser 1978;

Urquhart et al. 2010).

Compared our core category of control

orientations with control balancing theory

(Gregory et al. 2014) and notion of “a

portfolio of control modes” (Kirsch 1997),

the literature on control dynamics

(Choudhury and Sabherwal 2003; Kirsch

2004)

Compared our core category of stakeholder

orientations with the stakeholder agency

theory (Hill & Jones, 1992) and analysis of

the application stakeholder theory’s

orientation (i.e. descriptive, instrumental,

Page 104 of 307

and normative) by Donaldson and Preston,

1995.

(Adapted from Gregory et al. 2013)

4.4 Reliability and validity of the cases

Reliability and validity of the current case study research was ensured in the following

manner.

4.4.1 Reliability

Reliability of this research is based on the use of a well-documented methodology and case study

protocol (Yin, 1994). A case database was created where all case interviews and field notes were

stored. This was to support a clear and documented chain of evidence (Yin, 1994). This database

supported derivation of the conclusion from our initial research questions. Audio transcription

were validated by a second person to ensure accuracy. Themes, categories and patterns derived

through qualitative analysis were constantly checked during the analysis to ensure the validity of

identified themes, categories and patterns.

4.4.2 Validity

Cresswell (2009) and Yin (1994) recommended several methods to ensure validity of a qualitative

research. We used the following strategies to ensure validity of current thesis.

Page 105 of 307

4.4.2.1 Triangulation

In addition to the primary data collected through interviews, we also collected secondary data for

triangulation. Themes, patterns, and categories were validated using secondary data which

primarily comprises project documentations, meeting-minutes, social event records, email

communications, and project office records.

4.4.2.2 Member Checking

A member checking was conducted after primary interview and analysis but before the final report

preparation. A second round of interviews with the initial participants was arranged. During the

second interview, identified themes, patterns and categories were presented to the informants for

further comments. They were asked to provide feedback to ensure the accuracy and correctness of

the analysis. To minimize any concern relating to biases resulting from a single coder, we have

consulted a subset of the informants to review all derived codes and the supporting raw data. This

additional validation during the data analysis process significantly enhanced the validity of the

primary and secondary codes for this research.

Page 106 of 307

5.0 Case results and discussion

In this chapter, we present and discuss the evidence we found and how it relates to linked

theories as presented in Chapters 2 and 3.

In Sections 5.1 through 5.4, we present the evidence from each case and discuss the results of

each case individually. We also highlight what was unique or exemplary in each case.

5.1 Case A: Replacing an enterprise system for financial market

operations

The project selected under Case A involves replacement of a critical legacy system with a COTS-

based alternative. The host organization classified the original system as ‘critical’ as the legacy

system operated on financial data that are essential for the organization’s money market

operations. The legacy system was initially deployed for production use in mid-2007. This system

allowed administration and pricing of securities held as collateral in other financial contract

processing systems of the organization as well as various financial operations of the organization

such as Securities Lending, Term Repo4, Special Purchase and Resell Agreement (SPRA) etc.

The primary goal of this project was to expand existing functional capabilities and increase the

efficiency of the original system. Expanding functional capabilities included the implementation

of several enhancements that were identified before and after the completion of the legacy system’s

deployment, incorporation of new high priority business requirements, decommission of some

4Under a term repurchase agreement, a bank will agree to buy securities from a dealer and then resell them a short time later at a specified price. The difference between the purchase and sale prices represents the interest paid for the agreement. Term repurchase agreements are used as a short-term cash-investment alternative (Investopedia).

Page 107 of 307

legacy features of the solution, and prepare the solution for the future inclusion of foreign market

operations. Through extensive discussions with the business community and contemplation by IT

solution architects, the project team was able to establish a long-term “roadmap” for this new

project. The roadmap provided an incremental growth of scope for the application and a schedule

that supported the attainment of organizational business objectives while ensuring all project

related technical initiatives are progressed in a logical sequence.

Financial Market Operations (FMO), the project selected for Case A, was initiated towards the end

of 2013. This project comprised of four separate releases. Each release delivered one of the four

domain-specific functionalities of the application over the 2013-2015 timeframe. Project team

followed an incremental approach of delivery where each release was building on the

functionalities delivered by the previous release. With a team comprised of 22 members, this

project decided to adopt a COTS-based system called “Cadis” to replace the legacy system. Cadis

is owned and supported by a globally leading data analytics company “IHS-Markit”. In addition

to internal support staffs and developers, the project team also included consultants from vendor

side (IHS-Markit) to assist with the design and development activities of the project.

Table 20 below shows a summarized profile of the FMO project, selected as Case A:

Table 20: Summarized Profile for Case A

Project Purpose Stakeholder

Composition

Project Cost

(Completion)

Project

Duration

Post deployment Remarks

Replace a legacy

system, used for

financial market

operations, with a

COTS based

product

Core Project team,

Extended project team,

support personnel,

consultants, Vendor,

Business users

10.5 Million

(CAD)

24

months

Project achieved the promised

benefits and experienced minimal

post deployment issues.

Page 108 of 307

FMO project was under taken with the organizational vision to achieve several major benefits

which included : (1) reduction of operational risk for the organization, (2) reduction of domestic

external pricing data providers from four to one thus reduced costs related to multiple data

providers and a simplified pricing processes, (3) improvement and simplification of business

processes, (4) automation of business processes, (5) simplification of the system architecture, (6)

improvement of hardware and software supportability, (7) improvement of the end-user experience

and addressing COTS products defects thus increasing business and IT effectiveness. Whereas, all

these factors provide quantifiable measures of the project success, for this research, we also

collected post-deployment incident information and user satisfaction data to assess the overall

project success.

We interviewed a total of 11 members for this Case. Each informant was directly involved with

the implementation of the project. The workgroup interviewed for Case A includes original project

team members, members from the customer-facing services team, and members of the user

community or business users. The customer-facing service team or internal support team provides

direct support to the business users by responding to questions about the product’s operation as

well as providing trouble-shooting and repair services. Some members of this COTS support team

were part of the core project team in 2013. In addition, several interview participants, belonging

to the project team and user community, were directly involved with the implementation of the

project, thus, were able to provide accurate information of the implementation process, stakeholder

engagement and control process.

Case A provided an early lesson about the interview process. First few informants of Case A asked

for additional clarifications regarding the ‘control’ mechanism. The informants thought that the

Page 109 of 307

control involves with the activities performed by the project controller and the project manager in

tracking time and budget of the project. We, therefore, modified the introduction to the interview

for all subsequent interviews by including the following statement:

“By ‘Control’ the questions refer to various processes utilized to ensure proper involvement and

engagement by different project participants. Controls might be formal such as meetings, reviews,

reminders or informal such as joint status review or discussion at social gathering”

5.1.1 Implementation process for Case A

Purpose of this initiative was to develop a solution that would address new domestic business

requirements, decommission existing legacy solutions, and enable the business unit for future

inclusion of foreign market operations. Through extensive discussions with the business

community and the IT Solution architects, an application “roadmap” was established which

provided a direction for incremental growth of scope. From an implementation perspective, Case

A followed an agile-like methodology by delivering the scope and features in incremental manner.

Although the prevalent agile terminologies and practices like “sprints” were not used by the project

team, project implementation practices followed by the core project team were clearly not a

waterfall method.

Activities related to the implementation of FMO project appears to be segmented, at the minimum,

into seven distinct areas or logically grouped activities.

Page 110 of 307

The initial set of activities included developing the project charter, setting up project controls,

conducting risk workshops, hosting project kickoff meetings, and obtaining steering committee’s

approval of the project charter.

The second set of activities comprised of developing preliminary project management plan and

obtaining approval for the plan. In addition, a key project artifact “Business Requirement

Document (BRD)” was finalized and approved at this stage. Other dominant activities appeared to

be related to the fulfilment of project team’s physical and logical requirements such as

provisioning of workstations, granting access to servers, offices, common spaces, implementing

co-location plan, and communicating reporting structure changes.

The third set of activities showed emphasis on the analysis of implementation process and

configuration of the COTS solution from a conceptual standpoint. Documents such as Technical

Conceptual Design (TCD), System Requirements Specifications (SRS) and System Use Cases

(SUC) were developed during this stage. Although the project management plan was progressively

elaborated throughout the lifecycle of the project, project team obtained a formal approval for the

completed project management plan at this point.

Next set of activities looked very closely at the solution design, solution configuration, and all

related components of the new COTS product. The conceptual design, produced by the previous

set of activities, was transformed into a concrete implementation plan at this phase. Activities at

this stage enabled the project to debate about feature or function-specific alternative designs.

Detailed functional and technical requirements were finalized and approved at this point.

Production and approval of detailed solution design were also completed in parallel to the

Page 111 of 307

requirements finalization. Other key activities involved development of test plans and delivery

plans.

Following the development of a detailed design and a requirements document, project team’s

activities appeared to be more hands-on and tangible in nature. Provisioning of development

environment and COTS application licences were completed at this stage and the project team was

fully equipped to start the application related coding and configuration activities. Activities such

as developing backend database components, glue-ware modules, extended features for the COTS,

new graphic user interfaces (GUI) were dominant at this stage. Due to a segmented nature of the

delivery, activities performed during this phase appeared to be repeating over the duration of

project’s lifecycle. Other activities such as preparing the user guide were initiated at this point.

Test case preparation, deployment of deliverable to the QA environment, execution of developer’s

unit testing, and assurance of code quality were also performed at this stage.

The sixth set of activities focused on ensuring the accuracy and validity of the project’s

deliverables, transition plan, and decommissioning of the legacy system. Project team in

conjunction with business users, reviewed the results of User Acceptance Testing (UAT). This was

a recurring activity as every module, delivered by the project team, went through an UAT as soon

as the module was identified as ‘release-ready’. Transition plan related items such as production

support model, end-user training plan and application’s disaster recovery posture were finalized

by the project team. The project also formally solicited the client or business sign-off for all project

deliverables and prepare the project for completion.

A final set of activities was a non-repeating set of activities. Due to the critical and complex nature

of the application, the project team agreed to support the operational or COTS support team

Page 112 of 307

regarding any system related issues for several months since the project’s go-live date. This special

arrangement between the project and COTS support team was finalized at this stage of the project.

Significant activities towards this end of the project included: seeking steering committee’s

approval to formally close the project, and reviewing the support agreements or SLA with the

vendor for accuracy. The Project team conducted multiple knowledge transfer sessions with the

business users and the IT operations team to build awareness of new processes. The Project team

also collected feedbacks from all project participants to ensure that COTS implementation related

best practices are correctly identified and captured for future use in similar context. Table 21 below

presents all salient activities performed by the project team in Case A. Respondent codes used in

Table 21 and in the subsequent tables are formed by combining three distinct pieces of information.

This helped us to uniquely identify each participant without disclosing their identity. For example,

the first letter in the respondent code “A01PT” indicates the Case association (i.e. Case A), the

two-digit numeric portion indicates the informant number (i.e. first person interviewed or 01), and

the tailing two letter code indicates the group affiliation (i.e. PT for project team, VR for vendor,

BU for business unit, and IT for internal IT support).

Table 21: Project Activities – Major Phases and flows for Case A

Project Activities Respondents Potential

Category

Develop a “Road Map”

Obtain Strategic Investment approval

A01PT, A02PT, A06BU Pre-Initiation

Establish project controls and project administration

Develop the Preliminary Project Charter

Conduct project Kick-Off meeting/presentation

Obtain approval of the Preliminary Project Charter

A01PT, A02PT, A03PT,

A04PT, A05VR, A06VR,

A07BU, A08BU, A10IT,

A11IT

Initiation

Develop preliminary Project Management Plan (PMP), as

appropriate

Obtain Approval of the preliminary PMP



Page 113 of 307

Generate estimates for project schedule, activities and resource

requirements

Arrange Project Team Logistics

Produce Business Use Cases/Business Process Modelling

Documentation

Enable and Oversee Production of the Business requirements

Document (BRD) ensure due diligence is done in the following

areas: ATIP, Audit, COOP, Procurement, SFS, and K&IS

Obtain Approval for Business Requirements Document (BRD),

including Business Use Cases

A07BU, A08BU, A09BU,

A10IT, A11IT

Requirement

analysis

Finalize Project Management Plan (PMP)

Obtain Final Approval of Project Management Plan (PMP)

Determine Solution Alternatives

Obtain Approval of the recommended solution

Engage and Begin Managing Chosen and Approved

Vendor/Service Provider

Produce Technical Conceptual Design (Solution Architecture

Document (SAD)

Obtain Stakeholder Approval of Technical Conceptual Design

(SAD)

Establish a Requirements Traceability Matrix

If needed, Complete the System Requirements Specifications

(SRS) and Produce System Use Cases


A05VR, A07BU, A08BU,

A10IT, A11IT

Conceptual

Design

Oversee Production of Detailed Requirements

Obtain Approval of Detailed Requirements

Oversee Production of Design Documents

Obtain Approval of Detailed Design

Produce Test Plan

Develop Test Cases


A05VR, A07BU, A08BU,

A10IT, A11IT

Design and

Architecture

Develop Programming Components

Plan and Perform Integrated Test of Built Solution in the Dev

Environment

Obtain Approval to Transfer built solution into the QA

Environment; execute transfer

Plan and Perform QA/Functional Testing


A10IT, A11IT

Development

Organize and Facilitate User Acceptance Testing (UAT) -

Release Candidate

Obtain User Acceptance Test (UAT) Approval of Solution

Obtain Client Approval to Transfer Solution into Production

Finalize Transition Plan

Obtain Approval sign-off of Support Agreement(s)


A05VR, A06VR, A07BU,

A08BU, A09BU, A11IT

Delivery

Page 114 of 307

Coordinate Delivery of User Training

Deploy Solution to Production

Monitor System Stability

Oversee Project Stand-Down

Produce Project Completion Report

Obtain Approval to Close the Project



A07BU, A08BU, A09BU,

A10IT, A11IT

Closing

In summary, implementation-related activities for the FMO project appear to be segmented, at

minimum, into seven logically distinct groups. Although the “pre-initiation” activities determined

the roadmap for a potential technical solution, approval of a project charter and a formal kick-off

meeting indicated a formal initiation phase for this project. Besides the initiation, FMO

implementation activities also indicated the existence of other logical groups, such as requirement

analysis, conceptual design, design and architecture, development, delivery, and a non-repeating

“close-out” phase.

5.1.2 Control configuration and balancing

When the FMO project initially started in September 2013, the project team and the business unit

already established a sound relationship with each other. This harmonized relationship resulted

from the cooperative search for an optimal solution to replace the legacy system. However, the

relationship between the core project team and the vendor exhibited significant gaps in terms of

understanding and expectations. This could be attributed to an absence of shared work history

between two parties. The vendor’s business was located in Europe and was never awarded any

contract by the organization. Similarly, the relationship between the core project team and

extended IT teams was also lacking any notable history of working together. As a result, project

Page 115 of 307

team placed significant emphasis on formal documentation such as contracts, roles and

responsibilities outline, meetings and vendor demonstrations. However, in the absence of a signed

contract or an agreement, formal status reporting was not exercised during this phase of the project.

Business user’s participations in project activities were more noticeable compared to those of the

extended IT support team as business users were frequently invited to project’s status update

meetings to ensure the accuracy and completeness of project’s scope.

Once the project team finalized the contract with the COTS vendor, a more formal and procedural

control was adopted to ensure a common understating and common perception regarding project’s

work scope. Project team introduced a mix of behavioural and outcome based controls such as

formal communication plans, regularly schedule status meetings, steering committee meetings etc.

and included the vendor within the target of these controls. As the project team was finalizing

several key documents like project management plan and business requirement document, it was

critical to keep the vendor well-informed and to obtain timely vendor-response to ensure the

accuracy of these documents. Although the project team initially maintained a trust-based

relationship with the COTS vendor as well as with internal IT support teams, lack of previous work

history, unique project context and exiting organizational context influenced the project

management team to adopt a frequent and active monitoring approach for the project’s progress.

This proactive outcome control was indicated by respondent A03PT who states:

“We laid out a high level plan for all parties involved with the project…our feeling

about the vendor was bit of mixed and the reason for this is simple…the product we

selected is mainly used for a different purpose – different form our intended

Page 116 of 307

use….also we never used this vendor before….. at the same quarter [name of the

organization] under took several large projects which made us concerned about

the level of internal [IT support teams] support we will get. On time delivery of

those application servers were on our critical path and we simply couldn’t wait till

due date on the service request. What if it’s not ready by the due date?”

At the same time, control mechanisms were planned, coordinated, and implemented jointly by the

project team and the client. This resulted in a bilateral or coordinated control style which helped

foster a cooperative relationship between the project team and the client very early in the project’s

lifecycle.

After the project team finalized the BRD and project management plan, the activity focus shifted

towards conceptual design and solution design of the COTS product. In addition to status meetings,

the vendor and the project team jointly conducted design discussion sessions. Besides technical

meetings, the project team conducted workshops focusing on project management methodology,

various process templates, and organizational release management practices. Informants also

reported regular socialization activities during and outside of the working hours. A trust-based

and less formal control mechanism were also apparent from the utilization of brainstorming

session, information exchange of ideas, and spontaneous communications. At the same time,

ensuring the timely configuration of all required technical environments such as Development,

Quality Assurance, Pre-Production and Production was a matter of concern for the project. These

tasks were performed by the internal infrastructure support team of the organization. The Project

Page 117 of 307

maintained a frequent status review and activity tracking relationship with all parties responsible

for the timely delivery of the project’s environments.

Next, the project team and the vendor started developing the planned solution. Although the mutual

commitment between vendor and the client appeared quite high at this phase, the control

mechanisms were not significantly relaxed. Project team maintained regularly schedule status

review meeting, milestone tracking, defects review sessions, inspection of documentation and

reports, testing deliverables. Additional social controls such as workshops, feedback gathering and

communication, lessons learned for each iteration were also observed at this point. Although a mix

of social and authoritative elements found in the exercised control configuration, both vendor and

the project team were tracking each other at the status review, defect monitoring and deliverable

review meetings. Primary reasons for such bi-directional control are two: (1) actual development

work for each iteration was divided between vendor provided consultants and project’s COTS

developers, and (2) development for some glueware modules were taking much longer than what

was planned. Additionally, a few low-quality deliverables also introduced a certain degree of

uncertainty on both sides. At this point, a bi-directional control configuration was indicated by the

interview participants. According to the informant A01PT:

Deadlines were a bit tight for the development and testing of the new Markit plugin

that was required for Matlab integration. This plugin is developed by Markit.

Although Markit was very much involved and working hard on this and responding

quickly a lot of back and forth was required between us and Markit and it took a

while before we got a working plugin. Also, it was new players on the Markit side

and new players on the [organization name] side (for both FMO and Matlab) so

Page 118 of 307

nobody was sure what/how to do things. In addition, the Matlab plugin required

was a lot more complicated then the existing Matlab plugins.

As the project team was preparing test cases and developing a technical user guide, timely

cooperation from business users was still quite necessary. Clients were often engaged through

brainstorming sessions, informal exchange of ideas, email communications and telephone

conversations. However, the client’s involvement frequency with project activities was much

reduced compared to the early phases involvement frequency and the response timeline was also

very relaxed. These engagement approaches indicate a trusting relationship between the project

team and the client. However, a more formal and outcome-based control, through regular status

and progress tracking, was maintained for the extended support team.

As the technical team finished developing the core solution, project team forged a solid

understanding with the COTS vendor. Towards the end of the project, both the project team and

the vendor demonstrated a heavy reliance on the informal exchange of ideas and communications.

During the development phase, tasks performance and meeting the commitments did not show

much deviations from the expectations held by the project team. This helped the project team

maintain the status quo in terms of procedural control for the current set of activities as well. Both

the vendor and the project team worked diligently to resolve any issues or concerns resulted from

the client acceptance testing. However, project actively prescribed and tracked the knowledge

transfer sessions and internal training sessions targeted for the COTS support team. This was a

Page 119 of 307

unilateral control approach by the project team to ensure the support readiness for the operational

team of the host organization.

As the project team was wrapping up all the activities, the relationship between the project team

and the client seemed slightly different as both parties were actively tracking and ensuring that all

commitments were fulfilled. Control activities that dominated between the client and the project

were feedback gathering, lessons-leaned, and deliverables review.

Table 22: Observed Control Configurations in Case A

Phase Vendor IT Support

Team

Business

Clients

Salient Factors Relating to

Control Decisions

Pre-initiation Trust Trust Trust No signed contract or agreement,

lack of knowledge about the

capability of selected COTS,

Industry foot print of the vendor

Initiation Authoritative Authoritative Trust Absence of shared History, Project

context; Organizational context (i.e.

PMO prescribed compliance

requirements)

Requirement

Gathering

Authoritative Authoritative Coordinated Need to improve shared

understanding, Clear understanding

on the future state

Conceptual

Design

Authoritative Authoritative Coordinated N/A

Design and

Architecture

Authoritative Trust Coordinated Confirmation of resource

availability

Development Coordinated Authoritative Trust Development of shared

understanding, Delays in delivery

Delivery Trust Authoritative Coordinated Improved understanding, Missing

information

Page 120 of 307

Closing Trust Coordinated Coordinated Improved understanding, fear of

unfulfilled commitments

In summary, the control portfolio for Case A demonstrated multiple distinct configurations over

the duration of the project’s lifecycle. Initially, the control portfolio for Case A was dominated by

a more trust-based approach due to the absence of a formal contract or agreement. An authoritative

approach was used to replace the trust-based control configuration due to the absence of a shared

history among project participants and a need to improve shared understanding. Although the level

of shared understating developed at a varying pace among different project partners, closing phases

of the project indicated the return of a trust-based control configuration, along with a coordinated

control configuration.

5.1.3 Stakeholder engagement and sensitivity

Before the FMO project was initiated, the program overseeing the project proceeded through

organization’s standard gate 1 and gate 2 approval processes. The purpose of these gates is to

review the appropriateness of the undertaking and ensure that there is an understanding of what is

to be done and why, making certain that the investment has a good direction and is aligned with

the organization’s long term strategy. After the business sponsor gained approval at the decision

gate 1, this investment proposal proceed to the gate 2 review. At this phase, only a selective set of

stakeholders were engaged with the activities. Roles involved at this stage were business sponsor,

PMO, IT program manger. Although the activities of this phase were quite significant, the project

team did not have any control over the inputs provided by the stakeholders.

Page 121 of 307

Once the project proposal received gate 2 approval confirming that the strategic investment is

sufficiently compelling and the work should continue, both the project management team and all

enabling departments were engaged with various project related activities. Activities such as

creating the project charter, setting up project controls, conducting risk workshops, holding project

kickoff meetings – all required inputs from broader groups of stakeholders. Due to the early stage

of the project and the adoption of a completely new technology (the COTS product), most of the

inputs and recommendations received by the project management team were taken quite seriously

and in an open manner. This indicated a very high sensitivity towards the project’s stakeholders.

Following the initiation of the project, the project management team focused more on creating the

project management plan and finalizing the business requirement document (BRD). At this point,

a much larger group of people from diverse background were involved with the project’s

requirements gathering activities. Considering a completely new implementation of a COTS

product, the project team tried to ensure a much broader focus on the scope during requirements

collections. Most stakeholders were engaged through brainstorming sessions and facilitated

workshops to provide their inputs. Some of the groups engaged by the project team included

business clients, extended support team, release management, IT Security, IT infrastructure, the

COTS vendor. Since the artifacts developed at this point will server both as a guide for and

commitment of the project team, stakeholder groups exerted significant influence at this phase of

the project. A high engagement level, in an inclusive manner, helped develop a shared

understanding among the key stakeholder groups of the project as stated by participant A02PT:

“Having the Technical Team Lead participate in the business requirements

gathering and in all working sessions with the business helped in the preparation

Page 122 of 307

of the Business Proposed Solution. Although the Business Requirements Document

traditionally comes first and then the Business Proposed Solution, doing the two in

parallel has benefits also - it helped in identifying/confirming some business

requirements and it helped the business understanding right away how their

requirements will be addressed in [Name of COTS System]”

The third and fourth set of activities, which involved development of a conceptual design and a

concrete implementation plan for the COTS modules, demonstrated a reduced stakeholder

engagement. Dominant theme for the activities on this phase indicated a more prescriptive

approach through refinement of initially gathered requirements. Leading stakeholders for the

activities were enterprise architect, solution architect, solution integrators, vendor’s solution

engineers, selected developers and enterprise data modeler. As the detailed functional and

technical requirements were finalized at this point, technical project lead and subset of the project

management team were heavily involved with the decision-making process at this phase.

Once the project team started implementing the design and technical components for the COTS

solution, activities progressed in much focused and self-directed manner. This implied a reduced

decision making activities and following a pre-established solution decisions. This approach

required a significantly less involvement from the extended stakeholder groups of the project.

Groups of roles driving the activities of this phase mostly comprised of solution developer, vendor-

provided consultants, technical team lead, database administrator, and technical team lead.

Developers from IT Support team were engaged to facilitate the technical knowledge transfer. A

Page 123 of 307

visible difference between this phase and the earlier phases is very minimal opportunity to

influence the approved solution design or add any new requirements.

As the development of deliverables was completed by the technical team and progressed through

the quality control process, the project team focused on convincing a wider audience that the

promised value has been delivered. In addition, the project also needed to ensure both external

partners and the internal IT support teams are ready for upcoming changes to business and

technical processes. To ensure this state of readiness, project team needed to communicate with a

large group of stakeholders starting from business sponsors to external recipients of data - located

outside of the organization. Groups of people engaged at this phase included: steering committee,

infrastructure support, IT Security, IT continuity of operations, operational support including

COTS support, middle-tier support, and database support, PMO, business users, up-stream systems

support and down-stream systems support teams. Besides obtaining approvals from key

stakeholders of the project, creating awareness and a positive mindset to accept the changes were

also the key motivating forces for such a large engagement. Despite a large group of people

engaged in this phase, stakeholder’s opportunity to influence the deliverables were very little as

most of the engagement techniques involved push communications such as presentations, emails,

and announcements. An increased level of task dependency and complexity of activities motivated

the project team to increase the engagement level as indicated by respondent A09BU:

“Deployment of NHA-MBS [module name] was a complex deployment involving

many groups as there were FMO, $U and Matlab changes. The steps to be followed

were carefully documented by the Project Manager, reviewed by the various

players involved and exercised in previous environments. This was very important

for a successful deployment to production.”

Page 124 of 307

As the project approached its completion of all major activities and obtained acceptance from the

user community, project team’s focus was mostly on ensuring operational readiness and learning

from mistakes. Although the stakeholders engaged at this stage was moderate, the opportunity to

influence project deliverables were negligible. Project team sought inputs from all key project

participants such as core project team, support team, IT infrastructure team, COTS vendor, and

business users. An apparent goal of the activities was to avoid similar mistakes in future.

High

Stakeholder

Engagement

Low

Stakeholder

Engagement

High

Stakeholder

Sensitivity

Low

Stakeholder

Sensitivity

Delivery & Transition

Initiation

Requirement Analysis & Collection

Pre Initiation

Development

Design & Architecture

Conceptual Design

Closing

Page 125 of 307

Figure 8: Stakeholder Orientation for Case A

In summary, the FMO project under Case A indicated a high stakeholder engagement during the

early lifecycle of the project. The requirements analysis phase also demonstrated a high level of

stakeholder sensitivity, as most of the inputs and recommendations received by the project

management team were included in the requirement documents. Significantly reduced stakeholder

engagement and stakeholder influence were observed during the mid-lifecycle of the project. This

helped the project team maintain its focus in their activities and minimized activity costs. However,

the engagement level gradually increased towards the end of the project’s lifecycle, although

sensitivity did not increase proportionally. Figure 8 above captures this dynamic nature of

stakeholder orientation for the entire project lifecycle for Case A where each project phase is

plotted on a different quadrant of the stakeholder orientation grid depending on the combined

engagement-sensitivity level of the phase.

5.2 Case B: Replacing a shared system for financial regulations

Case B involves a COTS implementation by the same organization where an external facing

enterprise system was replaced. The legacy system was used to collect, validate and maintain

financial data and financial returns filed by federally regulated Deposit Taking Institutions (DTIs)

of the country. The existing system was jointly owned by three different institutions of the

Canadian government (referred as Tri-agency from hereon). Previously, the legacy system had

undergone several enhancements since its deployment in 1998 to keep pace with the requirements

of the partner agencies, but eventually it became incapable of adapting to the increasing quantity

Page 126 of 307

and complexity of data collection and management needs. The system was also failing to comply

with the existing security, hardware, software and performance standards of the host organization.

Maintaining the status quo became an obviously inadequate option in the face of evolving business

drivers and business objectives of all three partner organizations. In this respect, several

dimensions appeared to be driving forces to consider a replacement of the existing system such as

(1) technical limitations of the existing solution prevented implementing any required

enhancement to support the collection of additional information, (2) the security standards used by

the legacy system were no longer considered adequate thus, elevating the level of risk for all

partner agencies that might result from a security breach exposing confidential/sensitive financial

information, (3) increasing dissatisfaction expressed by the external users of the system due to a

poor response time, (4) existing hardware and application components were going out of vendor-

support thus a significant increase in the maintenance cost and diverges from the enterprise

architecture direction, and (5) inadequate and inconsistent validation rules preventing a common

rule engine for the legacy system.

Following an in-depth analysis of the legacy system and the software market, the project team

recommended engaging Deloitte & Touche LLP, a leading solution integrator, to implement a

commercial-off-the-shelf (COTS) software solution, based on Vizor Software, that will meet the

Tri-agencies’ business requirements. This recommendation was accepted for implementation after

comparing it to the status quo as it involved significant expenditures to maintain the existing

system with inadequate functionality in addition to carrying a significant risk of system failure.

This project was initiated in May 2011. Over all project scope comprised of four distinct yet

integrated streams: core modules implementation, Partner Secure Remote Access (PSRA)

Page 127 of 307

configuration, Enterprise Application Integration (EAI) configuration, and Organization Change

Management (OCM). The core modules implementation consisted of two phases: (1) deposit

taking institution implementation which was delivered in the fourth quarter of 2013, and (2)

pension plan/insurance company implementation which was delivered in the April of 2014. PSRA,

EAI and OCM streams were planned to run concurrently with core module implementation. Vizor

based implementation of the core modules followed a pure agile approach with multiple sprints

delivering different scope of the project.

With a team comprising of 35 people, budget at completion (BAC) for this project was $19.5

million (CAD). Funding for the project was equally divided among three partner organizations,

although each partner agency was responsible for any expenses falling outside the project budget.

Cost estimates included a 15% contingency to allow for change requests and work involved in

customizing the solution. Since Vizor, the COTS vendor, was based in Dublin, Ireland,

development activities for this project were divided between two geographically separated

locations - Ottawa and Dublin.

Table 23 below shows a summary profile of the FRS project:

Table 23: Summarized Profile for Case B

Project

Name

Project

Purpose

Stakeholder

Composition

Project

Cost

(Completi

on)

Project

Duration

Post deployment

Remarks

Financial

Regulatory

System

Replace a

legacy

system with

a COTS

Core Project team,

Extended project team,

support personnel,

consultants, Vendor,

Solution Integrator,

Business users, Partner

19.5

Million

(CAD)

26 months

Project was able to

deliver all required

features for the core

module at completion;

however, the end users

including the support

Page 128 of 307

based

product

organizations, External

users/partners

personnel experience

several defects and

incidents with the

system.

In order to assess the influence of control configuration and stakeholder engagement on project’s

critical success factors, we augmented the list of CSF reported in IS literature with the CSF

identified from Case A and Case B. Additionally, to assess the level of success for this project, we

have resorted to the post-deployment incident records and user experience for both business and

IT users.

Through the project charter, this project promised to realize the following major benefits: (1) a

major extension of the Tri-agency’s ability to collect information and support future regulatory

changes of financial market, such as Basel III requirements thus, reducing operational and

reputational risk for the organization, (2) enhanced corporate effectiveness by reducing efforts

required to introduce or modify returns, creating capabilities to handle large multi-dimensional

returns, integrate the collection and validation of DTI, insurance and pension plan regulatory

filings into a single system, (3) enable simplified administration of filer accounts and

communication, (4) free up required resources for metadata management by introducing a self-

service model which includes contact management, password maintenance, and client

communication, (5) maximize the usage and availability of collected data for partner agencies, and

(5) reduced time for acceptance, validation and distribution of large regulatory information.

We interviewed a total of 12 members for this Case who were directly involved with the

implementation of the project. The workgroup interviewed for Case B included original project

Page 129 of 307

team members, members of the customer-facing services team, and members of the user

community or the business users, Tri-Agency partners, and the solution integration team.

5.2.1 Implementation process for Case B

The purpose of this initiative was to develop a COTS based solution that would address existing

shortcomings of the legacy system, help the host organization to comply with the international

financial regulations such as BASIL III, modernize existing application infrastructure from

‘capability and security’ perspective, and decommission the existing solution. Guided by the

organizational policy, the group leading this business initiative considered ‘buy before build’

approach while searching a replacement for the existing system. Due to the unique nature of the

application and a niche market, it was extremely difficult for the project team to identify an ideal

candidate solution early in the project’s life cycle. It was already anticipated that the project will

take over a year regardless of a ‘COTS’ or ’custom built’ approach. Considering a wide scope and

unique business requirements of this project, the host organization decided to handle the

procurement as a separate project. Unfortunately, the initial attempt to secure a vendor failed,

costing the organization nearly two million dollars and six months of project time. This served as

a source of a major ‘lessons learned’ for the project team. As a result of this negative experience,

several procurement documents (RFI and RFP) as well as solution requirements document were

modified to overcome problems encountered in earlier effort. In addition, the host organization

consulted with its peers outside of the country and identified one COTS based product used in a

similar context. These findings, coupled with a more flexible procurement approach, enabled the

project team conduct a successful procurement on its second attempt. The full procurement cycle

Page 130 of 307

included issuance of a revised RFP, proposals evaluation, vendor demonstrations, contract

negotiation and a contract award.

From an implementation perspective, the project selected under Case B followed a completely

agile methodology by delivering scope and features in nine distinct sprints over two separate

phases. Each sprint lasted for three to four weeks which is slightly longer than typical sprint

durations recommended by the agile methodology. Activities related to the implementation of Case

B appeared to be segmented, at the minimum, into five distinct areas or logical groups. However,

due to considerable efforts and investment during the RFP process of this project, much of the

work related to planning, requirements analysis, and conceptual design were already accomplished

during the procurement process.

The core project team was directly engaged in September 2012. At that point, the project

management team possessed an approved version of the Business Requirement Document (BRD),

Project management plan, and secured most of the project’s logistics. The project team primarily

focused on the conceptual solution and design that were developed during the vendor

demonstration and were not finalized. Influenced by the agile methodology, the overall conceptual

design was divided in logically segmented chunks that are consistent with overall design of the

solution. The requirements specification document and business use cases were revisited, finalized

and segmented. These changes also triggered a change for the initial project management plan.

Key artifacts produced by the project at this point were a requirement traceability matrix and an

acceptance plan for the deliverables. As the agile nature of the project required multiple and

frequent deployment to different development environments such as quality assurance, pre-

production and production. To address this challenge, project team initiated multiple “Request for

Page 131 of 307

Change (RFC)” for all planned sprint deployments. Approvals for the revised project management

plan, business use cases and the acceptance plan were also obtained at this point. Other critical

activities involved finalizing risk management plan, change management plan, quality

management plan, and escalation process.

Next set of activities involved developing various work packages that were to be delivered by the

first sprint of the project. An approved and validated design document was provided to vendor’s

development team located in Dublin, Ireland. This work included enhancements and modifications

of the COTS product. Integration and data migration activities related to sprint’s requirement were

implemented by the consultants, the solution integratory and organization’s IT support team

located in Ottawa. As the implementation of the approved design was proceeding as planned, the

design and architecture team started to finalize the design for the next sprint. Therefore, both

design and development related activities were taking place in parallel. Other key activities at this

stage involved the development of test and delivery plans for the sprint currently under

development.

An agile approach also influenced post development activities of the project. Although each sprint

developed a functional version of the COTS product with a subset of the functionalities, a full

transition of the final product and the decommissioning of the legacy system took place towards

the end of the project’s lifecycle. In each sprint, all functionalities developed went through a rigour

unit testing conducted by the developer in Dublin. Once a deployment packaged was shipped to

the host organization, the project team conducted usability, user acceptance and integration testing

for the package. Project obtained informal sign-off from business sponsors after each sprint. While

the development activities were progressing as per project management plan, architecture, design,

and development work for the upcoming sprint was also progressing in parallel. Transition

Page 132 of 307

activities such as – preparation of support documents, system’s security assessment, support

team’s training and developing a disaster recovery approach were finalized by the project team

Similar to those observed under case A, the final set of activities for this project was a non-

repeating set. Salient activities at this phase of the project’s lifecycle were collecting feedback

from different stakeholder groups, executing an administrative phase-closure by obtaining client

signoff for deliverables, closing all open procurement related contracts and updating necessary

project documentations.

Table 24: Project Activities – Major Phases and flows

Project Activities Respondents Activity

Group

Opportunity exploration

Develop a “Road Map”


Procurement of the candidate solution

B01PT, B02PT,

B06BU, B07BU,

B08BU, B09BU

Pre-Initiation


Establish Governance Reporting for the project

Hold Risk workshop(s)

Complete Project Risk/Type/Gating Assessment




Obtain Approval to Gate into Ph1.

B01PT, B02PT,

B03PT, B04PT,

B05VR, B06VR,

B07VR, B09BU,

B10BU, B11IT, B12IT

Initiation

Develop preliminary Project Management Plan (PMP), as appropriate



requirements



Documentation

B01PT, B03PT,

B04PT, B05VR,

B06VR, B07VR,

B08VR, B09BU,

B10BU, B11IT,

Requirement

analysis

Page 133 of 307




B12IT



Establish Project Design Guidelines and Modelling Standards



Engage and Begin Managing Chosen and Approved Vendor/Service

Provider


Document (SAD)

Obtain Stakeholder Approval of Technical Conceptual Design (SAD)


Acceptance Plan

Complete the System Requirements Specifications (SRS) and

Produce System Use Cases

Initiate Request for Change (RFC) for planned implementation of the

recommended solution into the Production environment

Devise the Project Delivery Strategy

Coordinate setup of the Development and Test Environments

B02PT, B03PT,

B06VR, B07VR,

B08VR, B10BU,

B11IT, B12IT

Conceptual

Design





Organize and Lead Tactical Planning for Development based on

Approved Detailed Design

Write the "To Be" (future state) Business Procedures

Update the Support and Operations Guide

Produce Test Plan

Develop Test Cases

B02PT, B03PT,

B06VR, B07VR,

B08VR, B10BU,

B11IT, B12IT

Design and

Architecture



Environment

Prepare User Guide

Prepare for QA Testing

Obtain Approval to Transfer built solution into the QA Environment;

execute transfer


Arrange for Pre-Prod and/or Prod Environment Set-up

Facilitate Client Preparation of UAT Test Cases

Define Ongoing Support Agreement(s)

B03PT, B06VR,

B07VR, B08VR,

B10BU, B11IT,

B12IT

Development

Page 134 of 307

Produce Training Material

Organize and Facilitate User Acceptance Testing (UAT) - Release

Candidate



Complete Release Management Plan and Build Form


Coordinate Final Support and Operations Guide



Complete Project Delivery Phase-end Activities

B01PT, B03PT,

B04PT, B05VR,

B07VR, B08VR,

B09BU, B10BU,

B12IT

Delivery




Obtain Approval of Project Completion Report

Complete Project Close-out phase-end/project-end Activities


Do Formal GO LIVE (Final Release)


B01PT, B02PT,

B03PT, B04PT,

B05VR, B06VR,

B07VR, B08VR,

B09BU, B10BU,

B11IT, B12IT

Close out

In summary, implementation-related activities for the FRS project appeared to be segmented, at

minimum, into five logically distinct groups. Pre-initiation or pre-project activities for FRS

consisted of significant efforts related to requirements collection and procurements. As a result,

only conceptual design, design and architecture, development, delivery, and closing phases were

performed by the core project team after a technical roadmap was established. Initiation and

requirement analysis were two additional phases for this project, although in this instance they

were performed as a separate project. Furthermore, implementation activities for the FRS project

followed an explicitly agile approach where conceptual design, design and architecture,

development, and delivery related tasks were executed in an iterative fashion until the entire

project scope was completed.

Page 135 of 307


From a control perspective, Case B demonstrated a significantly complex scenario. This control

complexity was a true reflection of the project’s stakeholder landscape as the project team was

controlling two external partner organizations, two distinct vendor teams, internal IT support team

and the internal business client.

In May 2011, a subset of the core project team in conjunction with business authorities from all

three partners were working towards a successful procurement. Due to the existence of a shared

system (Legacy application), all three partners had a shared work-history and some degree of

common understanding regarding business needs. Therefore, the activities between project team

and the business users involved site visits by partners, common understanding workshops, formal

and informal meetings, exchange of ideas and formal status reports. Announcements and progress

reports were often distributed to all partners through a ‘joint communication’ approach. Internal

IT teams were engaged by the project on an ad-hoc basis without any direct reporting relationship.

At the same time, Informal exchange of ideas between the project and internal IT teams were

prevalent and the support staff were invited to attend brainstorming sessions organized by the

project. As the contract was not awarded at this point yet, both vendors were engaged through a

mix of formal and informal approaches. Procurement documents and vendor demonstrations were

formal but the vendors were not part of the regular status meeting. Other actions of the project

team also indicated a quite formal and unilateral control style such as RFP specifications requiring

the COTS vendor to partner with a mature and well established solution integrator, provide a

formal project management structure on the vendor side, allowing the host organization to screen

Page 136 of 307

resumes and interview process for vendor provided resources, maintaining physical presence of

vendor personnel on client premises.

Soon after initiating the second phase of project, a contract award was made and time allocation

was confirmed for both the business users and internal IT support teams. This provided the

opportunity for everyone involved with the project to charge time against the project’s cost center.

Due to the involvement of two different vendors, early phases of the project relationship between

the project team and the vendors indicated very formal and frequent status reporting activities.

Most of the reporting from the vendor side involved confirmation of client’s requirements in

respect to the scope of the project. Project team decided to use a fixed cost contract and reduce

risk for the organization and its partners. Project’s choice of contract type motivated the vendor to

exert certain degree of control over the project team as both vendors tried to ensure that client

provided requirements are not significantly different compared to the specification on RFP

documents. On the other hand, the project team was trying to ensure that business needs are

adequately addressed and in a comprehensive manner. As a result, frequent communications and

review meetings were heavily utilized by both parities. Agile approach introduced daily stand up

meetings which is an integral part of agile methodology. Although primary purpose of daily stand

up was to identify and remove road blocks, it also showed the up to date status of all project

activities. IT infrastructure and extended support teams were not part of the daily stand up meeting

but weekly status meetings were used to monitor activities performed by other support teams.

Design of this COTS application required multiple application and database severs for each

environment (Development, QA, PreProduction and Production). Provisioning of these technical

Page 137 of 307

environments was slower than the expected turnaround time which caused the project management

team to adopt a close monitoring approach of project’s progress through weekly meetings.

Project on the case B developed a detailed ‘requirement traceability matrix (RTM)’ to keep track

of agreed scope of the project. A dedicated process called ‘design authority’ was established to

review and approve decisions concerning conceptual and actual design for each COTS module.

Dominated by brainstorming and facilitated workshops, design authority process introduced a fast-

paced decision making. Design decisions were also regularly tracked and reviewed by the project

management team. To facilitate the tracking of status, project introduced RACIN (Responsible,

Accountable, Concur, Inform and Notify) chart for various critical activities. As the conceptual

designs were produced and architectural decisions were made, project team and clients were

monitoring each other – forming a control loop. Project wanted the client to follow a process while

presenting a design related requirement (behaviour) and the client wanted assurance that certain

requirements are included in design decisions (outcome).

As the activities moved into implementation, project team maintained the existing status review

and daily stand ups for information exchange and health check. Development for the first few

sprints progressed according to the scheduled plan. Although the customization and modifications

of the COTS application were performed by the vendor’s technical team in Dublin, site-visits by

some of the vendor’s technical team members and on premise presence of vendor’s director

significantly reduced the feeling of uncertainty for the project team. Regardless of the vendor’s

effort to build trust with the project team, an increased concern was resulted from some defects in

deliverables and delayed delivery time of 4th and 5th sprints. This schedule slippage induced the

Page 138 of 307

project team to become cognizant in tracking the work progress and task outcome for vendor’s

technical team in Dublin.

Once the development work was in progress and sprint deliverables started to complete, project

team developed a solid understanding with both vendors. Each sprint included a demonstration of

deliverables with client to secure a formal acceptance and a ‘lessons learned’ session to collect

client’s feedback. Towards the end of the project, both the project team and vendors demonstrated

a heavy reliance on informal exchange of ideas and informal communications. Joint

communication and joint demonstration sessions were commonly observed between vendors and

the project team. Organization’s COTS support team participated in technical and business

knowledge transfer sessions offered by the vendor and core project team. Although several of the

knowledge transfer sessions were voluntary, project ensured that members of the internal COTS

support team are certified on the COTS product they will be supporting in near future.

Table 25: Observed Control Configurations in Case B


Team

Business

Clients

Salient Factors Relating to

Control Decisions

Pre-initiation Trust Trust Trust No signed contract or agreement,

lack of knowledge about the

capability of selected COTS,

Industry foot print of the vendor

Initiation Authoritative Authoritative Trust Absence of shared History, Project

context, Vendor capability,

Organizational context (i.e. PMO

prescribed compliance

requirements)

Requirement

Gathering

Coordinated Authoritative Coordinated Mutual agreement on requirements,

shared understanding, Clear

understanding on the future state,

missed deadlines

Page 139 of 307

Conceptual

Design

Authoritative Authoritative Coordinated Confirmation of resource

availability, Development of shared

understanding, protecting price

agreements

Design and

Architecture

Authoritative Authoritative Coordinated N/A

Development Authoritative Authoritative Trust Delays in delivery, reduction in

code quality, lack of physical

presence

Delivery Trust Authoritative Coordinated Improved understanding,

improvement of code quality, site

visits by vendor, Missing

requirement information

Closing Trust Coordinated Coordinated Improved understanding

In summary, the control portfolio for Case B demonstrated multiple distinct configurations over

the project’s lifecycle. Similar to Case A, the initial control portfolio was dominated by a more

trust-based approach due to the absence of a formal contract or agreement. As activities moved

towards the implementation and execution of the project’s scope, lack of mutual agreement on

requirements, lack of shared understanding, disagreements about the future state, and missed

deadlines encouraged the project team to adopt an authoritative portfolio. Additionally, to ensure

a timely information flow and validate business expectations, a coordinated control configuration

was observed between the project team and clients. As the project approached completion, a high

level of shared understanding among project participants allowed restoration of a trust-based

control portfolio.


Initiation of this project was subject to the successful satisfaction of all requirements set forth by

organization’s strategic investment gates which includes Gate 1: Opportunity proposal, Gate 2:

Page 140 of 307

Proposed Business Solution, and Gate 3: Business Case. These requirements were satisfied

through Pre-initiation activities of the project. As this case involved an external facing IT system

used by all existing financial institutions of Canada, preparing RFP and other procurement

documentations demonstrated a wide consideration of several stakeholder groups. Most of the

project’s pre-initiation and initiation activities were performed during the first phase of the project.

Primary objective of these activities was to secure a reliable and qualified COTS vendor. Besides

the immediate business users or application owners, external partners, external users, IT Support

teams, procurement department of the client’s organization, executive sponsors from all partner

organizations, business sponsors, PMO, IT Program manger were engaged with these activities.

Most of the requirements for the target COTS solution were collected and validated at this point

through inputs gathering, co-authoring of various project documents and providing approvals and

decisions. A high-level of engagement was also observed from the utilization of brainstorming

sessions, focus groups and facilitated workshops to capture stakeholder inputs. Fixed cost contract

also motivated the project team to be highly sensitive to any client recommendations and potential

requirements. Reason for this approach was to avoid potential change requests during the

implementation phase that may lead to cost increase. Several key artifacts were produced at this

stage which included: detailed business requirements, high-level implementation plan, RTM,

BRD, contracts and memorandum of understanding.

As the second phase of this project had access to a detailed requirements and RTM document,

project team were able to start design and architecture activities of the implementation plan.

Conceptual design activities painted a high-level picture from an enterprise wide integration of the

target application. All sprints for the project was also clearly identified at this point. Activities at

this conceptual planning stage as well as the design phase seemed more focused. Team

Page 141 of 307

management efforts, specific task assignments and user of team wiki were observed. Due to some

technical limitations of the selected COTS product and other down-stream internal systems,

change and improvement recommendations were not as flexible as it was in the earlier phases of

this project. Leading stakeholders for the activities were business leads from each partner

organizations, enterprise architect, solution architect, solution integrators, vendor’s solution

engineers, IT Security specialists, and enterprise data modeler.

For Case B, stakeholder engagement was slightly higher when the project entered in development

and implementation phases. Groups involved with the activities primarily included vendor’s

offshore development team, client’s internal development team, infrastructure support team.

Frequent input from the business users was needed to resolve decision conflicts as the offshore

development team did not have access to client’s production environment and production data.

This introduced some challenges during earlier sprints, but the project team ensured a minimal

change of scope during the development of an on-going sprint. COTS support team was formally

invited to technical knowledge transfer sessions to facilitate pre-production and production

deployment of sprint deliverable for the project.

Once a sprint was completed by the offshore development team in Dublin, it was delivered to client

for integration and deployment into organization’s internal servers. Each sprint included some

tasks that resembles the traditional delivery activities from a waterfall software implementation

model. Deployment was assisted by project’s solution integrator and vendor provided technical

support team. Although each sprint included a small subset of working functionalities, data

migration support provided by organization’s internal IT support teams was necessary for every

sprint. Following the deliver of a sprint, business communities and clients were engaged through

Page 142 of 307

sprint demonstration. Purpose of this demonstration focused on newly delivered functionalities

and business process changes. Sprint demonstrations were scheduled on a regular basis with the

business communities and project leads. COTS support team was invited to these meetings but

participation was optional. Targeted alerts for clients, information sessions, and presentations

indicated a more passive engagement by several stakeholder groups. Regardless of a passive

engagement approach, participations by different stakeholder groups increased at this point and

the project also introduced processes to capture recommendations and feedbacks from the user

communities. This was accomplished through the use of sprint reviews and lessons-learned

sessions. Information captured was analyzed and utilized to improve poorly performing processes

and relationship management approach on the next sprint of the project. Groups of people engaged

during sprint-deliver included – business leads, infrastructure support, IT Security, IT continuity

of operations, operational support including COTS support and database support, PMO, business

users, upstream systems support and downstream systems support teams.

Closing activities of this project showed a significantly higher stakeholder involvement compared

to that in the development phase. Project management team closely engaged members of other

internal projects which were impacted by the delivery of this project through coordination and

communication. Organization’s standard feedback survey was conducted to collect leant lessons

from all participating stakeholder. Partner organizations conducted their own lessons learned

activities independent of the host organization. Collected inputs and identified concerns were used

to adjust organization’s support agreement with the solution integrator and the vendor. A close

coordination with COTS support team was also visible as effective and efficient incident response

from the support team was critical to protect organization’s external reputation.

Page 143 of 307

In summary, the FRS project in Case B showed a very similar stakeholder orientation compared

to stakeholder orientation of Case A. Case B indicated an increasing trend of stakeholder

sensitivity, ranging from moderate to high, during initiation and requirements collection phases of

the project. At the same time, stakeholder engagement remained significantly high, which helped

the project develop a comprehensive list of stakeholder needs and expectations.

.

Figure 9: Stakeholder Orientation for Case B

High

Stakeholder

Engagement

Low

Stakeholder

Engagement

High

Stakeholder

Sensitivity

Low

Stakeholder

Sensitivity


Initiation


Pre Initiation

Development


Conceptual Design

Closing

Page 144 of 307

During the execution phase of the project, which included conceptual design, design and

architecture, and development, only groups with specialized skills were involved. Furthermore,

opportunities to influence project activities were tightly controlled. This helped the project team

maintain its focus on all execution-related activities of the project. Towards the end, the FRS

project showed a moderate to high engagement of stakeholders, coupled with little opportunity to

influence the project’s deliverables. Figure 9 above captures this dynamic nature of stakeholder

orientation for the entire project lifecycle for Case B where each project phase is plotted on a

different quadrant of the stakeholder orientation grid depending on the combined engagement-

sensitivity level of the phase

5.3 Case C: Financial billing systems replacement

Case C involves the replacement of a critical legacy system of the same organization with a COTS

based product.

Supported through a legacy system “Bank Note Distribution System (BNDS)”, the host

organization distributed bank notes to Financial Institutions (FIs) to replenish their inventory of

bank notes, push new bank notes into circulation, retrieve notes from when FI’s inventories grow

too large, and retrieve worn bank notes to retire from circulation. FIs are charged a fee by the host

organization for each transaction. These billable fees, in addition to the full amount of the notes

being exchanged, are ultimately posted as journal entries into the SAP financial system of the host

organization. The legacy BNDS billing solution used a mix of database sources, custom SAP code,

and multiple flavors of technologies. Over time, the existing solution had experienced several

issues, including billing and reporting inaccuracies discovered by the FIs. Several of the business

users of this application are located in the Montreal and Toronto Agency Operations Centers

Page 145 of 307

(AOC) of the organization. With the existing system, AOC users were unable to make any

adjustments to billing transactions before the transactions were posted to organization’s SAP

general ledger. Such incorrect transaction could only be resolved and rectified by organization’s

finance department by manually entering data due to the absence of a proper electronic interface

for business users. As a result, often financial institutions were receiving invoices from the host

organization that did not match with the actual transactions for a given period.

For case B, the solution proposed by the project team was a COTS based replacement of the legacy

system. With the proposed solution, billing data will reside within the existing databases but

standard billing functions will be introduced, and bill correction features will be available to

authorized users. The new COTS solution will also introduce a ‘Billing-report’ quality assurance

functionality prior to billing confirmation. The summary invoice, adjustment and charges details,

and settlement invoices will all originate from the same database source and will be produced by

the department’s authorized personnel to ensure correctness of the reporting before billing

confirmation is complete and reports are delivered to the FIs.

The project under case C was initiated in July 2012. The project incrementally developed and

delivered all key functionality of the new application during the 2012-2014 time frame. With a

team comprised of 15 members, the project decided to adopt a COTS based system called “Cadis”

to replace the legacy system. Cadis is owned and supported by a leading data analytics company

“IHS-Markit”. In addition to internal support staff and developers, members of the project team

included consultants from vendor side (IHS-Markit) and external consultants to assist with design

and development processes. The core project team, including the consultants, was located at host

organization’s head office in Ottawa.

Page 146 of 307

Table 26 below shows a summary profile of the FBSR project:

Table 26: Summarized Profile for Case C

Project

Name

Project

Purpose

Stakeholder

Composition

Project Cost

(Completion)

Project

Duration

Team

size

(Core)

Post deployment

Remarks

Financial

Billing

Systems

Replacement

(FBSR)

Replace a

legacy

system with

a COTS

based

product

Core Project

team, Extended

project team,

support

personnel,

consultants,

Vendor,

Business users

7.5 Million

(CAD)

18 months

15

Although project

was able to deliver

most of the required

functionalities, the

support team and

clients experienced

significant post

deployment issues

In order to identify the critical success factors and assess the level of success for this project, we

have utilized project’s lessons learned, business case and project charter. As a part of a larger

business process redesign program, FBSR project was launched to address transaction-fee related

billing issues which had adverse effects on both the SAP and Cognos data warehouse environments

of the organization.The primary benefits of this project aimed to (1) lower operational risk for the

organization by improving user interface and overall system usability, leading to a reduction or

elimination of manual data entry errors, (2) increase FI confidence in the organization’s processes

by providing the Financial Institution partners with reliable billing and supporting reports that will

enable them to reconcile their deposits, (3) improve business flexibility by centralizing billing

processing rules, fee amounts and detailed transactional information under one system.

We interviewed a total of 9 members for Case C who were directly involved with the

implementation of the FBSR project. The workgroup interviewed for Case B included original

Page 147 of 307

project team members, members of the customer-facing services team, and members of the user

community or the business users, Agency office users, and extended project team.

5.3.1 Implementation process for Case C

The primary intent of this initiative was to replace multiple legacy systems that were jointly used

for satisfying a single business need. The organization used the legacy system to control

distributions of bank notes to other financial institutions of the country. A joint review of the

application data and behaviour, by IT solution specialists and business users, indicated the

possibility of a COTS replacement for existing system. Based on this analysis, project team

decided to use a COTS product that is primarily used for processing financial data and data

warehousing solutions.

Case C followed a hybrid methodology for its implementation. Although an incremental

development approach was used for the technical implementation, a fully functional system was

delivered at the final phase of the project.

Activities related to the implementation of Case C appear to be segmented, at the minimum, into

seven distinct areas.

Initial activities included the developing of a project charter, setting up project controls, conducting

risk workshops, hosting project’s kickoff meeting, and seeking approval for the preliminary project

charter. This project was significantly different compared to typical COTS implementation project

as there was very little activities related to procuring the COTS solution prior to the project

initiation.

Page 148 of 307

In parallel to the project initiation activities, project management team was involved in the

development of the project management plan and the business requirement document. Approval

for these documents was sought after the project charter was approved by the business sponsor and

project steering committee. Interestingly, some of the preliminary activities such as

“familiarization with organizational project management processes and tools”, “reviewing and

understanding the basic project assignment” etc. were repeated by the project management team

at this phase. This could be attributed to a change in the project manager position of this project.

Initially assigned project manager had to leave this project due to some other organizational

priorities and the project-controller was requested to manage this project. In parallel to review the

project assignment, the new project manager also engaged in securing the project logistics. Due

to a change in existing relationship, most of the focus of the new project manager was on

negotiating with functional managers to secure necessary project human resources. As the solution

alternative analysis was completed and a candidate solution was selected, the project team started

to establish project design guidelines and modelling standards. At this stage, project team was able

to finalize solution requirements and reach an agreement with the COTS vendor in terms of work

effort and support requirements.

The third set of activities executed by the project team was more focused on the architectural model

of the future state. As the legacy system was used by multiple, geographically distributed agency

centers of the organization, a solid conceptual design and a data model to reduce process-related

conflicts were a priority for the project team. During this stage, the project team conducted multiple

rounds of technical and capability gap analysis. Results from gap-analysis triggered multiple

revisions of the conceptual design document for this project. Finally, the project team obtained

approval from business users and the business sponsor for the finalized conceptual design

Page 149 of 307

document. From a technical environment perspective, setting up the development and test

environments were completed at this point.

The next set of activates, conducted by the project team involved analysis of the actual functional

module of the COTS system in relation to the business requirements provided by the client.

Surprisingly, a significant number of the technical and security requirements were not the priority

for the project at this point. Most of the project team’s efforts were directed towards developing

an accurate data model, designing the user interfaces, designing automated flows to sanitize

incoming data, and integrating the new COTS solution with the organization’s financial system

(SAP). Besides getting an approval for the detailed solution design, the project team developed

test plans and delivery plans at this stage. Once the approval for detailed design was received,

project team prepared to proceed to the next phase of the project which included developing the

technical solution from a holistic perspective. Activities for this phase included the development

of backend database components, job scheduling solutions for process automation, graphic user

interface (GUI) design and legacy system’s data conversion for the target COTS system. As the

developers were working on implementing technical components of the solution, project team

finalized test cases to facilitate the quality assurance process. The project facilitated user

acceptance testing in an environment that is identical to the production environment of the

organization. Since the project was not following a segmented delivery pattern, business users

were able to execute their test-cases on a fully configured and integrated COTS application. Dry

run for the data migration and finalizing the audit requirements were completed at this point.

The sixth set of activities focused on ensuring the accuracy and validity of deliverables, transition

plan for the new COTS and decommissioning plan for the legacy system. Project team in

conjunction with the business users, reviewed the results of the user acceptance testing (UAT).

Page 150 of 307

This was also a repeating activity as every module, delivered by the project team, went through a

UAT as soon as it was tagged as ‘release-ready’. Transition plan such as production operations

support, support team’s training and disaster recovery methods was finalized by the project team.

The project formally solicited the client or business sign-off for all the project deliverables to

prepare the project for completion. Once the business approval was received, the organizational

release management and change management activities were initiated to migrate the solution to

the production environment. Although a formal go-live for the new COTS-based system was used

by the project, a parallel operation of old and new system was maintained. This parallel operation

was jointly supported by the project and the COTS support team to ensure a smoother transition

and address any unexpected behaviour of the new product.

The final set of activities for this project was seeking steering committee approval and business

approval to close the project. A feedback survey was conducted by the project team to comply

with the PMO requirements. Project management team also produced a project completion report

and engaged the COTS support team to monitor the stability of the new system.


Project Activities Respondents Activity

Group

Opportunity exploration


C01PT, C02PT,

C06BU, C07BU,

C08BU, C09BU

Pre-Initiation


Gain basic understanding of project assignment






C01PT, C02PT,

04PT, C05VR,

C06VR, C07BU,

C08BU, C09IT

Initiation

Page 151 of 307

Develop preliminary Project Management Plan (PMP)



requirements



Documentation




C01PT, C02PT,

C04PT, C05VR,

C07BU, C08BU

Requirement

analysis






Execute Procurement Process (for software, hardware, or services)

Engage and Begin Managing Chosen and Approved

Vendor/Service Provider


Document (SAD))

Obtain Stakeholder Approval of Technical Conceptual Design

(SAD)


If needed, Complete the System Requirements Specifications

(SRS) and Produce System Use Cases


C01PT, C02PT,

C03PT, C05VR,

C06VR, C07BU,

C08BU, C09IT

Conceptual

Design

Ensure diligence is done on: ATIP, Audit, COOP, Procurement,

SFS, and K&IS








Update the Support and Operations Guide

Produce the Project Preliminary Transition Plan

Produce the Project Delivery Plan

Produce Test Plan

Develop Test Cases

C01PT, C02PT,

C03PT, C04PT,

C05VR, C06VR,

C07BU, C08BU,

C09IT

Design and

Architecture



Environment

Prepare User Guide

C01PT, C02PT, ,

C04PT, C05VR,

C06VR, C07BU,

C09IT

Development

Page 152 of 307


Obtain Approval to Transfer built solution into the QA

Environment; execute transfer






Candidate



Complete Release Management Plan and Build Form


Prepare for Solution Transition into Production

Obtain Approval sign-off for the Support Agreement(s)




C01PT, C02PT,

C04PT, C05VR,

C07BU, C08BU,

C09IT

Delivery



Complete Project Close-out phase-end/project-end Activities


C01PT, C02PT,

C03PT, C04PT,

C05VR, C06VR,

C07BU, C08BU,

C09IT

Close out

In summary, implementation-related activities for the FBSR project appeared to be segmented, at

minimum, into seven logically distinct groups. Although FBSR was a COTS-based

implementation, procurement-related activities were almost non-existent. The primary reason for

such behaviour was the vendor’s choice. The COTS product selected for the FBSR project was

already in use within the host organization. This indicated an already existing work-relationship

with the vendor. Regardless, requirements analysis, conceptual design, design and architecture,

development, delivery, and closing related activities were distinctly identified and performed by

the project team.

Page 153 of 307


Replacement of existing business processes and a search for a suitable candidate solution for this

project started in mid-2012. Business analysis efforts for this project involved engaging multiple

agency offices and various internal teams of the organization to accurately capture existing

business processes and activities of the business unit. In the absence of a candidate solution and

fully staffed project team, early efforts of the project management team involved frequent face to

face meetings, over the phone conversations, direct communications, and brainstorming sessions

to acquire information, exchange ideas about potential solutions and explore possibilities of

modified business processes. Most of the efforts were directed towards communication with the

business users. Internal IT specialists were occasionally consulted to validate the feasibility of the

emerging future state in terms of a technical feasibility perspective. A potential vendor as well as

a candidate COTS solution were not identified at this stage. In mid-2013, the project appeared to

be more formal and organized in terms of administration and control. With an approved project

charter and a full-time project manager assigned, the project management team were able to decide

on a candidate COTS solution. The COTS product selected for this project was the same COTS

product that is used for the project discussed under Case A. At the initiation stage and in other

early phases of this project, the project team demonstrated a very relaxed control over the vendor.

Reason for such relaxed control over the vendor is attributed to an existing engagement of the

vendor with a higher priority project within the same organization. On the other side, an increase

in status review, prescriptive operational procedures, and planning meeting were also observed.

Both IT support team and clients were also involved in a less authoritative manner as this project

was sharing its technical resources with the project discussed under Case A.

Page 154 of 307

Soon after the initiation activities completed, project team started finalizing the requirements for

the selected COTS. Business and technical requirements were used to negotiate and finalize a fixed

priced contract with the vendor. Due to an existing relationship with the vendor, contract

negotiation was much simpler and faster than the organization’s typical contract negotiation

cycles. This also helped the project team to avoid a lot of procurement related meetings such as

bidder conference and vendor demonstration. As the project team consisted mostly of the members

of the organization’s functional teams and external consultants, project started to track the work

efforts and activity progress for internal IT support team and the vendor. This was accomplished

through weekly status-update meetings and status-update emails for assigned activities. As the

infrastructure and other project-logistics related requests were fulfilled by internal support teams,

organization’s standard SLA (Service level agreement) was leveraged to track progress and

completion of these activities. Despite the application of some forms of outcome-based controls,

project team’s overall relationship with both the business users and the internal support teams was

dominated by a positive outcome expectation.

Once the requirements were finalized, project team immersed into a high-level technical design

and low-level component designs of the COTS. Database schema design and data conversion

planning was also conducted along with the design activities. Most of the decisions made during

this phase were through design workshops and design decision meetings. Project controls for the

vendor turned into a more authoritative or unilateral at this phase. Regular status tracking and

other methodical approaches were observed. Project management team, at this point, relied heavily

on weekly status updates to assess how the project activities were progressing on the vendor side.

This change in control style also occurred soon after the team had a new project manager replacing

Page 155 of 307

the initial project manager. Primary data indicates a relationship between the adopted control style

and shift in leadership of the project, as respondent C03PT states:

“…as you re-call we had change of PM [Project Manager]… [name of new PM]

was very new to this and this was her first project to manage as a project manager.

So she relied a lot on the what we said at the weekly meeting [project status meeting]

but when we got to the QA[Quality Assurance], we realized the problem that $U

[glueware component for the solution] was not ready and several cadis modules

weren’t either…then we had no choice but to escalate and make some noise.”

Next, the project team started the development activities for technical components. A significant

portion of this development work was done by the project’s technical team with occasional

assistance from and design validation by the vendor. Interactions with the vendor during the

development were more balanced and bi-directional in nature as some of the unique needs of this

project required both the vendor and the project team to equally contribute to ensure a timely

completion of all activities. Assistance from the business users appeared more passive and

informal during the development phase. Any clarifications or change of requirements were

consulted with the client. Reason for such ad-hoc engagement by the client indicated a high degree

of confidence by the project team over the client in obtaining timely and reliable responses. The

configuration of the Production and the Pre-production environments was conducted by the

infrastructure support team of the host organization. Such environment configurations and other

internal support tasks were enforced through the SLA and IT service desk.

Page 156 of 307

As the development moved towards its completion, the project experienced several significant

setbacks. These negative outcomes included delayed development of technical components,

delayed delivery of glueware components and incomplete scheduling module to support process

automation. Participant C07BU provided a specific example of one of such setbacks:

“While the project team did a very good job with collecting the business requirements in

most areas, the requirements for the Cognos reports were left until the last minute and

were not finalized in time to allow the development and QA team to implement and test

them properly. It led to a lot of last-minute changes just before the deployment as well as

post-production.”

Negative outcomes and unfulfilled expectations during the development phase changed the

relationships between the project management team, COTS support team, and the vendor. Most of

these unfulfilled expectations resulted from lack of cooperation and engagement by the internal

support teams. This also resulted in multiple escalations by the project team to enforce cooperation

from the vendor and the support team. An increase in status tracking and meeting frequency was

observed in comparison to earlier phases of the project. As several product demonstrations and

joint parallel testing were conducted during this phase, project team maintained a close

coordination with the clients and agency offices. With a shared goal of timely delivery of a fully

functional COTS system, project team and the client demonstrated a high level of cooperation to

solve any identified problems and redesign all affected business processes.

The closing of the project showed a different relationship compared to the projects in Case A and

in Case B. Discovery of unexpected defects during integration and user acceptance testing

Page 157 of 307

motivated the project team to negotiate an extended support agreement with the vendor. All

deliverables were tracked for the completeness in an authoritative manner by the project team.

Cooperation between the project team and the business users remained unchanged during the

lessons learned activities where they saw themselves as equally responsible for the success of the

project.

Table 28: Observed Control Configurations in Case C


Team

Business

Clients

Salient Factors Related to

Control Decision

Pre-initiation N/A Trust Trust Absence of formal authority

Initiation Trust Trust Trust Existing contract with organization;

Project Context;

Organizational Context (PMO


requirements)

Requirement Trust Trust Trust Change in leadership

Conceptual &

Conceptual

Design

Authoritative Trust Trust Following Norm, Leadership

competencies

Design &

Architecture

Authoritative Trust Trust Following Norm, Leadership

competencies

Development Authoritative Authoritative Trust Missed Milestones, delays with

deliverables

Delivery Authoritative Authoritative Trust Disputes with vendor,

Disagreements, Competing internal

resources

Closing Authoritative Authoritative Trust Following Norm

Page 158 of 307

In summary, the control portfolio for Case C demonstrated multiple distinct control configurations

over the duration of the project’s lifecycle, which were significantly different compared to the

control configuration of projects in Case A and Case B. Although an existing work relationship

with the vendor and missing formal authority in the project management team led to a trust-based

control configuration, the project team chose to continue such a trusting relationship with the

vendor, internal support teams, and the client for subsequent project phases. This decision can be

attributed primarily to a lack of leadership strength and project management experience, as a junior

project manager replaced a more experienced project manager. However, the second half of the

project’s lifecycle was dominated by an authoritative control configuration, as the project

management team was unable to re-establish the desired level of shared understanding.

Additionally, the level of negative anticipation remained high among project team members.


Stakeholder engagement during the strategic initiative screening or project gating phases was

significantly lighter for this project compared to projects selected for Cases A and B. The initiative

justification provided by the business unit was quite convincing for the organization’s senior

leadership; however, a potential technical solution was not in sight. Pre-initiation activities of this

project demonstrated a heavy reliance on existing documentation and business processes to

develop an outline for the future state. Business lead, business analyst, and a temporary project

manager were primarily responsible for satisfying all requirements set forth by investment

proposal evaluation gates of the organization. Regardless of a little involvement by a wider group

of stakeholders, input collected by the smaller project team and several artifacts produced were

Page 159 of 307

quite significant for this project. Similar to previous two cases, organizational PMO played a

supportive role at this stage of the project by providing advice and guidance to the project team.

Once the project proposal received gate 2 approval confirming that the business investment is

sufficiently compelling for the organization and the project should be initiated, both the project

management team and a selective set of roles such as business analysis and technical solution

architect were engaged to support initiation activities. Involvement by various stakeholders

slightly increased at this point as the project manager started seeking inputs from various business

and technical teams to develop a comprehensive and feasible project management plan. Due to the

absence of a potential candidate solution, a large portion of the inputs collected was in the tentative

state. Once the project team was formed and a decision was made to adopt the COTS utilized by

the project under Case A, stakeholder engagement for this project increased further as the vendor

was included in planning discussions and in the decision-making process. In contrary to

organization’s usual project management practice, project team did not engage the COTS support

team and several IT support teams at this point. This decision had a significant negative impact on

the project’s work package identification, effort estimations and the work breakdown structure

development. Focus of this pre-design phase was largely on finalizing business requirements and

technical requirements which also included non-functional requirements such as scalability,

usability and availability of the technical solution. These activities were largely driven by the

solution architect and business analysts through a direct communication with end users of the

potential system and other technical teams. More commonly used engagement techniques such as

design workshops or intensive brainstorming sessions were not utilized by the project. This

resulted in a lower stakeholder engagement. Such reduced or low stakeholder engagement also

Page 160 of 307

had significant negative impact during the development and delivery phase of the project as

indicated by respondent C02PT:

“Testing that involves AIMS needs more than the time allotted during the project.

The project would benefit greatly from having appropriate testing resources

assigned to the team… It became clear right from the beginning of the testing cycle

that preparing and loading of the source transactions (mandatory prior to

execution any of the test cases) takes a significant period of time which was not

accounted for when the original estimates were done…if I recall correctly – the QA

team became involved soon before the development… As a result the QA process

was not as thorough as expected (especially related to the Cognos reports testing)

and took more time than it should. I also felt that the project could use an expertise

from the professional QA testers - the majority of the testing was done by the

Business Analysts which I believe is important but not sufficient.”

Immediately after the completion of requirements gathering and analysis activities, the project

succeeded in securing sufficient human resources to fully staff the core project team including the

project management team. As the project team started the execution of the third and the fourth set

of activities, which involved conceptual design of the COTS solution and a concrete

implementation plan of the COTS modules, stakeholder engagement remained roughly at the same

level with a slight increase. This can be attributed to a selective use of RACIN matrix to engage

the stakeholders. As the selection of a candidate solution was delayed during the initiation of this

project, the core project team was striving to fill-in a large amount of missing information at this

point. This required inputs from several internal and external stakeholder groups. Subsequently,

Page 161 of 307

these inputs and gathered information became part of some critical artifacts produced by the project

such as business requirement document, business user cases, design documents, and solution

architecture. Primary stakeholders for these activities included enterprise architect, solution

architect, solution integrators, vendor’s solution engineers, selected developers and enterprise data

modeler.

As the project completed design and architecting activities for the new system, three different

teams started hands-on development of the planned technical components which included vendor’s

technical team addressing product enhancements, project’s technical team developing user

interface, data models, and back-end solution components, and organization’s COTS support team

developing scheduling components and integration links for downstream systems. As the

development of the new solution progressed, each team identified several technical and logical

discrepancies. This required additional clarifications from the business users and occasional design

changes for the technical design. This resulted in a higher than usual engagement level by different

stakeholder groups at the development phase.

Multiple deadlines and milestones slippage occurred during the development cycle of this project.

This required the development window to extend beyond the planned duration. Once all technical

components were marked completed, the project started to migrate all developed components to

the production environment. Missed milestones and missed deadlines motivated the project

management team to take a firm stance towards any new recommendations and improvement

suggestions by the client or other stakeholder groups. To accommodate any ‘nice-to-have’ features

and non-critical requirements, project team proposed future enhancement releases for the system.

Deliverables and documentations of the project indicated a minimal use of RACIN matrix. Besides

Page 162 of 307

a low engagement level for most stakeholders, project also struggled to bring COTS support team

onboard for its knowledge transfer and product demonstration sessions. Regardless of any

substantial opportunity to influence the deliverables at this stage, project attempted to engage

several groups of stakeholders which included – steering committee members, infrastructure

support, IT Security, IT continuity of operations, operational support including COTS support and

database support, PMO, business users, users from the external agency offices and downstream

systems support teams.

Once the deliverables were reviewed by business users and by the project team, additional support

resources and support time were requested for this new application. This was primarily due to

some concerns from the client side on the stability of the new system. During the go-live and

project closing phase, multiple high priority incidents were reported by the client when some

accounting entries in the organization’s SAP general ledger did not balance. This also resulted in

several escalations as the COTS support team were unable to resolve the issues and assistance

from project’s technical team was necessary. This is indicated by several interview participants.

According to participant C06VR:

“Earlier involvement of the developers from [internal COTS support team’s name] team

would help with more smooth integration into the operational environment. Knowledge

transfer sessions are helpful but nothing can replace the real hands-on experience. I am

aware that this was part of the project plan from the very beginning however due to the

resource limitations in [portfolio name related to the project] it has not been done”.

Page 163 of 307

Closing of the project indicated a reduced engagement by different stakeholder groups. This is

evident from the project’s lessons learned and feedback sessions as well as the project closing

survey where participations were very low. The feedback provided by various stakeholder groups

for this project was significantly less compared to other similar sized projects of the organization.

Issues and any modification suggestions were added to backlog to avoid any further delay in the

project’s go-live.

Figure 10: Stakeholder orientation for Case C

High

Stakeholder

Engagement

Low

Stakeholder

Engagement

High

Stakeholder

Sensitivity

Low

Stakeholder

Sensitivity


Initiation


Pre Initiation

Development


Conceptual Design

Closing

Page 164 of 307

Figure 10 above captures this dynamic nature of stakeholder orientation for the entire project

lifecycle for Case C where each project phase is plotted on a different quadrant of the stakeholder

orientation grid depending on the combined engagement-sensitivity level of the phase.

In summary, the FBSR project in Case C showed a significantly different stakeholder orientation

when compared to stakeholder orientation of Case A and Case B. Whereas both Case A and Case

B indicated an increase in stakeholder engagement and stakeholder sensitivity for their early

lifecycle phases, Case C showed a decline in stakeholder engagement. This was evident from a

low stakeholder engagement during the “Requirement Analysis” phase of the project. Although

the execution-related activities of the project maintained a moderate to low stakeholder

engagement level. as well as a moderate stakeholder sensitivity level, the engagement gradually

increased toward the end of the project’s lifecycle. Additionally, sensitivity level also increased as

most of the client’s feedback appeared in the form of an enhancement or a change request.

5.4 Case D: Enterprise content management (ECM)

Case D involves a replacement effort by the organization of its existing “content management

solution” with a well-known COTS – SharePoint. The Enterprise Content Management (ECM)

program was under taken to enable the organization store, manage, share and find electronic

information with ease and efficiency. With a pressing need to create a single, trusted and reliable

source of content, seamless records management, and effective integration with the organization’s

existing productivity tools, ECM project was identified as a much-needed business investment.

ECM aimed to refine the information management, governance and security framework to support

organization’s effective management of information lifecycle, establishing a good understanding

of accountabilities and standardization of existing practices for all staff. This project, therefore,

Page 165 of 307

promised to provide greater efficiency and effectiveness in delivery of information management

services, reduce information risks related to legal compliance and organizational reputation,

remove risks associated with the support of the existing documents management system, and

deliver enhanced staff productivity.

A significantly differing aspect of this project, compared to earlier three projects, is that ECM was

built on the work of a previous ECM project, which ran from 2009 to 2012. The previous project

developed a solution based on the existing document management system of the organization, but

experienced significant challenges during implementation. As a result, the ECM project executed

between 2009 and 2012 was terminated before reaching its completion or closing phase.

Regardless of this failure, the motivation of the business owners of this application in replacing

the existing document management system was still very high. This high motivation can be

attributed to several factors such as the lack of ability by the legacy ECM solution to support

existing business needs, and assumption of a significant risk by the business by using an

unsupported technology. The existing document management system was built on IBM’s Filenet

3.5 environment. As a product, Filenet 3.5 was at end of its product lifecycle and was identified as

an unsupported product by the vendor- IBM. As a result, the organization did not have any suitable

alternatives to ensure its compliance with legal requirements for records retention and destruction

under the Library and Archives of Canada Act. In addition, the ability to find records on a reliable

basis that may be needed to respond to litigation, access and privacy requests, or public inquiries

was also compromised.

SharePoint based ECM project was initiated on February 2012 which followed an agile approach

of deployment. The core project activities lasted till December 2014. With a core project team

Page 166 of 307

comprising of 22 people, budget at completion (BAC) for this project was $11.5 million (CAD).

The project team consisted of organization’s internal ITS staff, external consultants and Microsoft

provided engineers working at the organization’s head office in Ottawa.

Table 29 below shows a summary profile of the ECM project:

Table 29: Summarized Profile for Case D

Project Name Project

Purpose

Stakeholder

Composition

Project Cost

(Completion)

Project

Duration

Post deployment Remarks

Enterprise

Content

Management

(ECM)

Replace a

legacy system

with a COTS

based product

Core Project team,

Extended project

team, support

personnel,

consultants,

Vendor, Business

users

11.5 Million

(CAD)

32 months

Project achieved some of the

promised benefits and

experienced moderate post

deployment issues;

However, due to significant

issues with integration and

handling changes- client

satisfaction was very poor.

After experiencing a significant financial and time loss from a failed attempt to upgrade the

existing ECM between 2009 to 2012, the organization decided to refine its objectives, and select

a new technical direction. The key business drivers and requirements of the project remained

largely unchanged. However, the new technical direction was expected to provide a broader

collaborative platform, which had emerged as an organizational need.

The ECM program was undertaken with a promise of providing the organization a more

disciplined approach towards managing electronic documents and email, and delivering a

foundational infrastructure (including people, processes and technology) for improving the

management of organization’s information assets. Specific benefits that were hoped to achieve

under this program included: (1) increased end user satisfaction with the content management tools

Page 167 of 307

as filing, finding, and sharing information would be more efficient, and searching for documents

and emails would be easy, fast, and consistent, (2) increased staff productivity resulting from a

faster access to reliable and trustworthy information, reduced time and effort spent on managing

paper-based documents, reduced need for physical space to store paper-based records, and

increased accuracy of responses to ATIP requests, (3) increased quality of content by providing a

clear understanding of what documents need to be stored, a proper guidance for disposition of

information that are no longer of any value, thus reducing the likelihood of losing vital information,

(4) reduced risk by enabling the organization to effectively meet its legislative, corporate and

contractual obligations, and (5) consolidated foundation for future evolution of the organization’s

collaborative tools and processes. The legacy ECM was missing significant features and

enhancements made within the industry over the past decade. Features such as wikis, blogs, and

shared calendars, and support for co-authoring were available by a majority of the software

vendors in ECM industry.

In addition to the project’s fulfilment of promised benefits, we also collected post deployment

incident statistics and user satisfaction data to assess the level of implementation success for this

project. Similar to preceding three cases, pre-established set of critical success factors that were

obtained from the IS literature is augmented with the factors identified through project’s lessons

learned activities. Project’s lessons learned identified several CSF for this project which included:

an increased focus on usability and initial priority on end-user’s concerns in order to gain strong

user adoption, implementing technology that is compatible with the organization’s current

technology landscape, staffing key positions using organization’s internal staff in order to ensure

accountability and control over the vision, scope and plan, and facilitating change by introducing

Page 168 of 307

new functionality in a gradual fashion through multiple “waves” of deployment over the entire

lifecycle of the project.

We interviewed a total of 9 members for Case D who were directly involved with the

implementation of the project. The workgroup interviewed for Case D included original project

team members, members of the customer-facing service team, and members of the user community

or the business users, and the COTS vendor providing the COTS software.

5.4.1 Implementation process for Case D

Case D involves an example of an organization-wide deployment of a COTS product. This

implementation effort of the organization was a second attempt to replace an existing COTS

product which was used for the same purpose.

Six months prior to the initiation of the ECM project, host organization started an evaluation

process through a Request for Expression of Interest (RFEOI). This RFEOI process engaged 10

different COTS vendors (HP, EMC, OpenText, IBM, Oracle, Microsoft, Autonomy, Hyland,

Minisis and Xerox). Project team used a range of criteria to assess each product, supported by the

information collected through the RFEOI industry sources (such as Forrester, AIM, Real Story

Group and Gartner), and advice from external procurement experts. This led to a shortlist of two

products, Microsoft’s SharePoint and OpenText. Both vendors were invited to participate in a

formal RFP process for this project. On the basis of technical demonstrations and written

submissions, it was found that both products offered very different strengths and presented

different challenges. As both products clearly satisfied most of the project’s requirements, price

Page 169 of 307

became a key differentiating factor between two vendors. Microsoft presented a lower costs, in

comparison with the estimates provided by OpenText, for the initial installation, software

acquisition and overall lifecycle costs including ongoing licensing and support costs of the COTS

product. In addition to the cost savings, Microsoft demonstrated a COTS-based product that

provided better collaboration, better integration with the organization’s existing office software

and enhanced usability, which overall were regarded as key elements in gaining user acceptance.

In addition to product’s capabilities, numerus examples of SharePoint-based ECM deployments,

within organizations that are similar to the host organization, was also an influencing factor.

Considering the product features, productivity factors, and existence of a rich knowledge-base

within the industry, Microsoft was chosen as the preferred vendor and awarded the contract for

this project.

ECM project followed an agile development and deployment approach. Reason for adopting a

multiple-delivery approach was the large scope of this project and identified project risks. Lessons

learned from an earlier failed attempt of the same initiative indicated ‘scope management’ as a

significant risk factor for the project. Each department of the organization owned a large volume

of corporate data in the form of various documents and communications records. Moreover, the

migration and conversion process for each department or functional group was not very different

compared to each other. This motivated the project team to follow a multi-phased delivery

approach where deployment of the new COTS product will be limited to a single functional unit

at a time.

Activities related to the implementation of Case D appears to be segmented, at the minimum, into

seven distinct areas.

Page 170 of 307

Although the project’s initiation phase was short and fast compared to previous three projects, a

significant amount of groundwork for the current project was completed prior to its initiation.

Failure of the initial replacement attempt motivated the project team and the program management

team to be extra vigilant in this second attempt. This resulted in an extensive focus on procurement

activities and vendor engagements.

Common project initiation activities such as setting up project controls, conducting risk

workshops, arranging a project kickoff meeting, and soliciting approval for the project charter

were observed at this phase. Preparation of several important project artifacts such as ‘project

charter’, ‘business requirement document’, and ‘project communication plan’ required

significantly less effort at this phase as these artifacts were previously developed during the earlier

attempt of the same strategic initiative. Having these critical artifacts in a ‘near-complete’ state

also enabled the project team to execute procurement related activities much earlier.

Once the project charter and the BRD were approved by the project’s steering committee, project

management team started finalizing a comprehensive project management plan. Processes required

to secure project’s required human resources were also executed at this time. As the new COTS

product, replacing organization’s existing ECM, will host data and documents that are considered

as corporate resources, project team ensured that due diligence is done in the areas of Access to

Information Privacy (ATIP), Audit, Continuity of Operations (COOP), Procurement, and

Knowledge and Information Services (K&IS). Unlike earlier three cases, project under Case D

was able to finalize most of the business and technical requirements at this stage of project.

However, several of the technical and functional requirements needed refinement and

Page 171 of 307

modifications as the second attempt of this project had selected a different COTS solution

compared to its predecessor.

Once the project team embarked upon the third set of logical activities, the project team actively

started to manage the selected vendor and the service provider for this project. Such an approach

by the project team was primarily due to a new technical direction resulting from the adoption of

a new COTS product. Additionally, whereas the legacy ECM application was a dedicated and

specialized tool for documents management, the new COTS (SharePoint) was not seen as a

specialized product in this context. SharePoint is commonly used for collaboration and sharing.

For this project, the team decided to modify existing business processes and customize the features

of a default SharePoint version to make it suitable as an enterprise content management system.

Although the delivery plan of this project consisted multiple phases, each phase focusing on one

functional area at a time, conceptual design was more encompassing and took a holistic view of

the entire solution. Technical gaps and functional capability gaps were identified by the design

team. Based on these gaps, appropriate mitigation strategies were also developed by the project

team. Other significant activities observed during this phase included finalizing the project

management plan, maintaining a close coordination with the infrastructure team to setup the

development and test environments for the project, and coordinating with the target business unit

for initial deployment to device a test strategy.

The fourth group of activities involved developing a detailed design and architecture for each

module of the COTS system. Low level or component level design activities performed during this

phase were not significantly different compared to the previous three Cases; however, focus of the

project team at this stage was a single functional group. Processes and activities under this logical

Page 172 of 307

group were repeated for each functional group as each business unit of the organization required

very similar content management features. Activities of the project team included developing test

cases and test plans, developing delivery plan and obtaining approval for the plan, redesigning

business processes, designing interface layouts for the end users, developing migration plan and

data conversion plan. This group of activities was partially repeated for each functional area under

the master delivery plan of the project.

Immediately after the approval of project’s technical design and the technical implementation

plan, project started implementing the design specifications developed on earlier phases. Major

segments of work packages at this stage included designing the interface of the new ECM system,

preparing existing data and meta data for migration to SharePoint’s SQL database, preserving the

data relationship integrity during the migration, and developing additional custom codes and

solution to facilitate data migration. Quality assurance and integration testing started soon after the

first build was deployed into the QA environment. At the same time, client was supported by the

project team to prepare for the UAT related activities for the recently completed build.

As the project moved into its sixth logical group of activities which involved transitioning the

validated product into production environment, the project team also started the conceptual design

activities for the next functional group. Once the first build was validated by the QA team, it was

migrated to an environment which is very similar to the production environment for further

validation by the business users. A few defects and unexpected system behaviour were identified

by the user community which were immediately addressed by the project team. Primary focus of

the project team at this point were on data migration, business process update, user education and

training, support team’s knowledge transfer, ensuring availability of records from the

Page 173 of 307

decommissioned system. Once the business approval was received for delivered modules, release

management and change management activities were initiated to migrate the solution to the

production environment.

As the new ECM became a system of record or a live system in the organization’s production

environment, project management team started to collect feedbacks and comments from project

team, support teams, and user community on their entire experience in respect to this project. Other

activities by the project team involved monitoring the new ECM’s stability, overseeing the

decommissioning of the legacy ECM, and preparation of the ‘project completion report’.


Project Activities Respondents Phase

Upgrade attempt for the existing system

Alternative analysis

D01PT, D02PT,

D06BU, D07BU,

D09IT

Pre-Initiation

Obtain relevant PMD Toolkit Material for the Project

Become familiar with the organization’s PM Processes and Tools


Gain basic understanding of project assignment






D01PT, D02PT,

D03PT, D04PT,

D05VR, D06VR,

D07BU, D08BU,

D09IT

Initiation

Develop preliminary Project Management Plan



requirements

Based upon estimates generated, update Planview [time allocation

tool] accordingly.

D01PT, D03PT,

D04PT, D05VR, ,

D07BU, D08BU,

D09IT

Requirement

analysis

Page 174 of 307



Documentation






Execute Procurement Process (for additional software, hardware, or

services)

Engage and Begin Managing Chosen and Approved Vendor/Service

Provider


Document (SAD)

Obtain Stakeholder Approval of Technical Conceptual Design (SAD)

Acceptance Plan

Complete the System Requirements Specifications (SRS) and

Produce System Use Cases

Initiate Request for Change (RFC) for planned implementation of the

recommended solution into the Production environment

Devise the Project Delivery Strategy


Develop Preliminary Support and Operations Guide

D01PT, D03PT,

D04PT, D05VR,

D06VR, D07BU,

D08BU, D09IT

Conceptual

Design

Ensure due diligence is done in the following areas:

ATIP, Audit, COOP, Procurement, SFS, and K&IS








Produce the Project Delivery Plan

Obtain Approval of the Project Delivery Plan

Produce Test Plan

Develop Test Cases

D01PT, D02PT,

D03PT, D05VR,

D06VR, D07BU,

D09IT

Design and

Architecture

Oversee the Solution Build

Develop Programming Components and data migration tools


Environment

Prepare User Guide

Finalize Implementation Preparation Planning and Due Diligence



D01PT, D02PT,

D04PT, D05VR,

D06VR, D09IT

Development

Page 175 of 307




Finalize Delivery Preparation Material


Candidate




Prepare for Solution Transition into Production





D01PT, D02PT,

D03PT, D04PT,

D05VR, D06VR,

D07BU, D08BU,

D09IT

Delivery

Provide alternative and support for missing documents



Obtain Approval of Project Completion Report


D01PT, D02PT,

D03PT, D04PT,

D05VR, D06VR,

D07BU, D08BU,

D09IT

Close out

In summary, implementation related activities for the ECM project appeared to be segmented, at

minimum, into seven logically distinct groups. Although the activities performed during initiation

and requirement analysis were much lighter in terms of workload compared to similar phases of

the other three Cases, a differentiating characteristic for this project was an earlier failed attempt

to upgrade the project. That earlier upgrade attempt of the ECM project performed a significant

requirement analysis that formed a major input for the conceptual design activities during the

project’s second attempt. Other distinct activity groups for this project included conceptual design,

design and architecture, development, delivery, and a non-repeating “close-out” phase.


During the initial upgrade attempt for the legacy ECM system, the initiative was primarily driven

by the vendor and the vendor provided technical consultants. With a long history of working with

Page 176 of 307

the same vendor and using the same product, most of the vendor provided recommendations and

suggestions were accepted in an uncontested manner by the organization. This was one of the

significant reasons that had tremendous adverse effect on the initial upgrade attempt that failed.

This failure of a strategic initiative and the significant cost resulting from it had significant

influence on the initial control configuration of this project. This influence is evident from a more

proactive control approach adopted by the program management as well as the project team

overseeing this project.

Six months prior to its initiation, the project engaged multiple well-known top tier COTS vendors

to determine a viable technical direction. During this pre-procurement phase, existing market

information collected by the project and technical proposals submitted by vendors were closely

scrutinized by the project team. Once a contract was awarded, the relationship between the project

team and the vendor remained at a very formal level which included frequent status tracking by

the project team as well as close monitoring of any vendor proposed change requests. This

indicated an approach taken by the project where the key interest lies on “what the vendor is doing”

as opposed to “how it is being done”. Relationship between the project team and the business users

and technical support teams was more mature and required less monitoring. Informant D07BU

stressed on a positive and open partnership between project team, procurement and the business

clients:

“There were many open, transparent, honest conversations about what to do. The

right people were at the table. This likely resulted from an openness and desire to

get this project right after the results from the previous ECM project.”

Page 177 of 307

Designing and architecting of the overall COTS solution and individual modules were performed

by the core project team. Besides SharePoint developer, this team also included organization’s

technical architect and subject matter experts from the vendor side. Project maintained the ‘status

quo’ in terms of control configuration for the early phases of the lifecycle. This continued trust

based relationship between the project team and the IT Support teams introduced some negative

outcomes for the project which included delayed delivery of project’s infrastructure components.

Project team member D01PT indicated this by describing the negative impact of such trusted

relationship:

“Too often project team members did not trust their own expertise and we relied

quite heavily on vendors to supplement with their SME’s [subject matter

expertise]…….I would say that before any external SME arrives on site that we

ensure that the appropriate software and environments are ready so they can

begin their work immediately. In our case because there were delays in signing

the SOW [statement of work] there was no time left between receiving the pre-

installation checklist to when the external SME arrived on site. Several days were

spent preparing the environments and securing the proper software.”

During the design phase, project - client interactions were significantly reduced; However, the

project team maintained a close coordination with the business to acquire any clarifications related

to its design decisions. Vendor was managed closely by the project team through conventional

status reports and encouraging them to follow the established design processes of the organization.

Page 178 of 307

Project team conducted its own review of the vendor provided design recommendations to ensure

accuracy and alignment of design components. Vendor also provided prescriptive guidelines for

the project team to follow for the data and meta data preparations. Despite an bi-lateral and tight

control style exercised by the project team, several instances of delayed design decisions were

observed at this phase. Primary data indicates a lack of understanding regarding responsibilities

and an absence of roles clarity were to blame for these delays.

Once the data preparation, data migration and COTS configuration activities begun, the project

team and the vendor moved into a relationship that reflected a higher level of mutual

understanding. Very frequent communications, informal discussion sessions and informal

exchange of ideas were used to coordinate the activities of this phase as opposed to outcome based

controls utilized in earlier phase. Advice provided by the vendor was promptly adopted to resolve

unexpected road blocks encountered by the project team. This behaviour of the project team was

primarily due to lack of expert knowledge regarding the new COTS product within the

organization and project team’s belief that vendor share the same urge to complete the project on

time with high accuracy of deliverables. Common practices like the manner of code repository

utilization and code migration methods were used by project and the vendor in a similar way. Both

the business users and IT Support teams were expected to aid the project as needed, however, no

additional control mechanisms were deployed to proactively control the cooperation.

As the project moved into the delivery stage, status review and validation activities related to

deliverables significantly increased. This increased level of scrutiny resulted from a few

unexpected defects that were discovered during the user acceptance testing. Project team revisited

Page 179 of 307

the contractual obligations by the vendor to ensure that contractual commitments are fully met and

pointed out the deliverable gaps in a more formal fashion by escalating the issue to the program

oversight committee of this project.

The client and the project team of this project exercised joint communications approach to update

the business users and project steering committee on the status of deliverables and the progress of

defect remediation planning. Besides, lessons learned and feedback collection, internal IT Support

team and COTS support team also utilized job shadowing with project team and informal coaching

activities to get a good grasp on the new COTS platform. A tight and unilateral control style on

the vendor was maintain by the project at this very end of the project’s lifecycle. This unilateral

control style was a direct result of the negative experience, related to the deliverable qualities, from

the preceding phase of the project.

Table 31: Observed Control Configurations in Case D


Team

Business

Clients

Salient Factors Related to

Control Decision

Pre-initiation Coordinated Trust Trust Absence of formal authority, strive

to reach an agreement

Initiation Authoritative Trust Trust Absence of previous contract,

Negative past experience, pre-

disposed trust condition, previous

cooperative settings,

Project Context,

Organizational Context (PMO


requirements)

Page 180 of 307

Requirement Trust Trust Trust Reduced ability to monitor

behaviour, lack of technical know

how. Positive assumption on the

validity of previous requirements

Conceptual

Design

Trust Authoritative Trust Missed delivery dates

Design and

Architecture

Coordinate Authoritative Trust Lack of clarity for roles and

responsibility; high task inter-

dependency

Development Trust Authoritative Trust Absence of technical expertise;

Perceived attitude of the other

actor

Delivery Authoritative Authoritative Authoritative Disputes with vendor,

Disagreements, Competing

internal resources

Closing Authoritative Authoritative Trust Following Norm

In summary, the control portfolio for Case D demonstrated multiple distinct control configurations

over the project’s lifecycle, which was significantly different compared to the control

configuration of the projects in Case A and Case B, but somewhat similar to Case C. Although

missing formal authority in the project management team led to a trust-based control configuration,

the project team chose to continue a trusting relationship with the vendor, internal support teams,

and the client in subsequent project phases. Such a decision can be attributed primarily to a lack

of leadership strength and project management experience, as a junior project manager replaced a

more experienced project manager. However, the second half of the project’s lifecycle was

dominated by an authoritative control configuration as the project management team was not able

to re-establish the desired level of shared understanding. Furthermore, the level of negative

anticipation remained high among the project team members regarding the future performance of

other project stakeholders.

Page 181 of 307


Initial attempt to update the existing ECM demonstrated significantly light stakeholder

engagement in general. Despite a large project budget, most communications were centered around

and project activities were performed by a small set of project team members under the direction

of the legacy system’s provider. The vendor of the existing ECM shared a long working

relationship with the host organization. This vendor also enjoyed the reputation of being one of

the top consulting firm globally. As a result, the project team relied heavily on the vendor’s

guidance and directions for the initial upgrade attempt of the ECM. This resulted in a much smaller

stakeholder footprint compared to other similar sized projects. Once the initial attempt to upgrade

the organization’s ECM failed, the project steering committee decided to replace the existing ECM

instead of upgrading it. This decision had noticeable impact on project’s stakeholder engagement

approach as the number of stakeholders needed to be involved with the new initiative significantly

increased.

During the Pre-initiation of the project, several vendors (10) and their COTS based products were

reviewed to determine a suitable alternative. Diverse groups of IT support team, business users

including the business sponsor of the project, procurement department, and strategic investment

oversight committee were also involved with the process. Regardless of the heavy involvement by

several stakeholder groups, most requirements for this initiative were derived from the work

performed under earlier upgrade effort and existing documentations related to the legacy ECM.

This resulted in a low engagement level by the stakeholder during the initiation phase of this

project. Processes and tools utilized by the project team in early stage of the project also limited

Page 182 of 307

stakeholder’s ability to provide effective inputs and project team’s ability to capture critical

feedbacks. Informant D08BU stated multiple such challenges during initial phase of the project:

“….while there was a good process for the group to share their ratings in the

consensus sessions, I think participants would have liked more opportunity to

discuss their comparison of functional RFP ratings within their own functional

team first…not to mention that RFP deliverable was too complex. It had a main

body and 12 appendices. It was too time-consuming to review for us and created

lots of potential for mistakes…..I think the demos were too long. It also resulted in

diminishing returns.”

This trend continued as requirements collection and validation were led by a small set of

individuals which included business analysts and vendor. Due to earlier efforts of requirements

gathering and existence of detailed documentations for legacy ECM, influence of stakeholders on

the project’s activities was also much lower compared to the stakeholder influence observed in

Case A and in Case B. Cost of such low engagement and low sensitivity were apparent from the

remarks of respondent D02PT:

“Our project definitely could have benefitted from a broader user representation

on the evaluation Team…..even one or two more people would have brought helpful

broader knowledge of the user needs to the table. As it was, it was a bit stretch for

the few individuals to represent the breadth of needs. As well, it would have been

useful to have more users participate in the demos”

Page 183 of 307

Once the requirements document and procurement process were finalized, the project activities

showed a similar trend of stakeholder engagement compared to the earlier cases. During the

conceptual design of this SharePoint based ECM and the implementation planning of the COTS

modules, was activities were mostly driven by the design and architecture group of the project.

Due to a completely new technical solution, data mapping considerations between legacy system

and the new COTS, and enterprise integration considerations for the new ECM triggered additional

changes to previously developed design documentations. Project team did not invite all groups of

stakeholder who were previously involved; however, due diligence was exercised by considering

the solution from a broader perspective such as enterprise integration and legislative requirements

(ATIP). Primary stakeholders for project activities at this stage included enterprise architect,

solution architect, solution integrators, vendor’s solution engineers, selected developers, and data

migration specialists.

Upon the completion of design and architecting activities for the new ECM, project team’s efforts

were centered around implementing the design, developing support modules, configuring the

COTS applications and preparing the existing data for the legacy system for migration. Activities

at this point were highly focused and performed by the project’s development team.

Responsibilities for performing these activities were limited to a small number of team members

which allowed very little or no room for deviation from the original implementation plan. A

reduced opportunity for feedback from other stakeholder groups such as different departments of

the organization indicated a low engagement by as well as sensitivity of stakeholders. During the

deployment of the customized COTS product, a similar pattern for the stakeholder orientation was

Page 184 of 307

observed. The number of participants slightly increased as the delivery of the configured system

required new activities such as client validation, joint workshop and walkthrough sessions. This

also required support from COTS support team, infrastructure support team and respective

business areas. Although an increased involvement from different stakeholder groups, the

opportunity to influence the delivered solution was minimal.

Figure 11: Stakeholder orientation for Case D

High

Stakeholder

Engagement

Low

Stakeholder

Engagement

High

Stakeholder

Sensitivity

Low

Stakeholder

Sensitivity


Initiation


Pre Initiation

Development


Conceptual Design

Closing

Page 185 of 307

Closing phase of this project contained multiple lessons learned sessions to collect accurate

feedback on project’s performance and management of challenges. Online questioners were

distributed to wider audience involved with various project activities at different stages of the

project. Although not everyone involved with the project provided the feedback, responses

received were quite substantial in terms of number and quality of information. Interestingly, the

problem areas identified through the ‘lessons learned’ activity seemed to be repeating from phase

to phase which indicated an inability of the project management team to incorporate or address

any identified concerns in a timely manner.

In summary, the ECM project in Case D showed a significantly different stakeholder orientation

compared to Case A and Case B. Whereas both Case A and Case B showed an increase in

stakeholder engagement and stakeholder sensitivity during their early lifecycle phases, Case D

showed a decline in stakeholder engagement. The early lifecycle of the ECM project showed a

moderate to high sensitivity level and a moderate to low engagement level. Although an earlier

effort of requirements gathering and existence of detailed documentations for the legacy ECM

were responsible for such a stakeholder orientation, the impact of this low stakeholder engagement

was quite detrimental for the ECM project. Similar to Case C, the stakeholder engagement level

gradually increased toward the end of this project’s lifecycle. Additionally, the sensitivity level

increased as most of the client’s feedback appeared in the form of an enhancement, defect repairs,

or change requests. Figure 11 above captures this dynamic nature of stakeholder orientation for

the entire project lifecycle for Case D where each project phase is plotted on a different quadrant

of the stakeholder orientation grid depending on the combined engagement-sensitivity level of the

phase.

Page 186 of 307

6.0 Cross-Case Analysis and Synthesis

In this chapter, we discuss the evidence from all four cases and how it relates to the linked theories

presented in Chapters 2 and 3. In Section 6.1, we discuss the results of the cases as a set and use

the cross-case results to answer research questions 1 and 2. In Section 6.2, we identify and discuss

control configurations and control balancing scenarios for each project phase. In Section 6.3, we

discuss the phase-specific stakeholder orientation for the projects. Finally, in Section 6.4 we

capture the relationships among control configurations, processes, stakeholder orientations, and

CSF for the project.

6.1 Enterprise COTS Implementation Model

In this section, we analyze all four cases from an implementation perspective of enterprise COTS.

In addition to the identification of a holistic COTS implementation model, we examined

implementation-related processes and tools that played significant roles in influencing one or more

critical success factors for our selected projects.

In analyzing COTS implementation for Case A, B, C, and D, we primarily focused on two

important aspects: (a) the end-to-end process followed by each project team in terms of high-level

project phases, and significant linkages between the phases; and (b) key project activities

and implementation-related tools and processes utilized during different phases.

Because our first two research questions align with the qualitative research type—“the discovery

of regularities” (Miles & Huberman 1994)—we adopted an event structure analysis approach to

identify critical elements and explore their relationships. Analyzing the primary case data for all

Page 187 of 307

four cases, we constructed an event matrix that arranged a series of concrete events in

chronological order. This event matrix was then further analyzed to group events into different

logical domains (presented in Figure 12), preserving the sequence and salience of events. This

analysis facilitated answering both research questions concerning COTS implementation

processes, as a process is essentially a string of coherently related events (Miles & Huberman

1994). Based on a temporal- and purpose-based categorization of all implementation activities for

all four projects, eight major categories appear as salient: (a) pre-initiation, (b) initiation, (c)

requirement and planning, (d) analysis and conceptual design, (e) design and architecture, (f)

development, (g) delivery, and (h) close-out.

In examining end-to-end implementation flow, we took a recommended project implementation

template from the organization’s electronic resource repository as an a priori model. This model

or general implementation guideline, sequential in nature, was then further refined based on our

developed-event matrix. This refined model, containing eight distinct groups of activities, is

presented as the new COTS implementation model (Figure 12).

Page 188 of 307

Figure 12. Agile COTS Implementation Model

The implementation flow, presented in Figure 12, captures the logical sequence followed by all

four projects with some minor variations. The host organization’s project management and

implementation practices strongly resembled a traditional waterfall style, and adoption of agile

methodology within the organization was still in its infancy. The nature of the COTS under

implementation, the project manager’s knowledge of agile practices, and the project management

office’s project implementation guidelines were also responsible variations in implementation

practices across all the cases. This is evident from the observations that a highly agile

implementation process was followed in Case B and near-waterfall implementation practices were

followed in Case C. However, after a synthesis of practices and sequences followed by all four

projects, the overall implementation model appears much closer to an agile than a waterfall model.

It is also evident that the developed COTS implementation model is not a pure agile

implementation model as requirement analysis and closing activities were often not repeated.

Whereas Figure 12 presents the implementation process from a macro level showing major phases

of the projects, Figure 13 presents our findings from a micro or activity level. This activity-level

analysis captures the inter-activity relationships and the stakeholder groups responsible for the

success of each key activity. Since the pre-initiation activities of a project are often not considered

part of the project, the remaining seven distinct project phases of the COTS implementation model

from Figure 12 are logically grouped into five broader categories. The resulting model, showing

stakeholder engagement at the activity level, is presented in Figure 13.

Page 189 of 307

Figure 13: COTS Implementation activities and stakeholder engagement

In the following sections, we describe the COTS implementation flow model presented in Figure

12. This discussion also focuses on COTS-specific key activities (Figure 13), significant processes,

tools, and procedures employed by different project teams to successfully overcome challenges

and manage CSF during various phases of the project.

Page 190 of 307

Pre-implementation Phase

The pre-implementation phase, although often not identified as part of a project’s formal lifecycle,

is the very early set of activities performed in preparation for a project. Our micro-level analysis

of each project’s activities indicates the existence of this phase where business users primarily

assume a leading role to turn an investment proposal into a COTS implementation project. An

organization’s management personnel usually initiate IT projects when they have an opportunity

to enhance business processes and increase operational efficiency (Ward & Daniel, 2002).

Additionally, compliance, public safety, and reputation are among the driving forces behind most

strategic IT initiatives in government organizations. Public organizations analyze these factors

when exploring business opportunities. For large projects, this phase comes before an IT project

begins. Informants from all four cases reported compliance, reputation, and efficiency gains as a

few of the key driving motivations behind each project. Activities performed during the pre-

implementation phase, such as business-case development and requests for proposals (RFP), play

an important role in COTS implementation success. Although the development of a BRD and RFP

are closely related to the core activities of a project, a large portion of our identified activities are

administrative in nature. For example, informants from each case reported to perform all or a

subset of the following tasks: contacting ITS portfolio management to confirm that the project is

in a portfolio schedule, obtain and review the business and IT proposed initiative summary sheet

(ISS) with its associated cost estimate spreadsheet, review the business case from the Investment

Governance Office (IGO), and prepare for project-gating requirements (refer to Appendix for gate

overview).

Page 191 of 307

Initiation

We have identified initiation as one of the earliest phases for each project’s lifecycle. Traditionally,

this phase is identified by the activities taking place immediately before or after the project

chartering and kick-off meeting. A similar trend was observed for all four selected cases, but

instances of overlapping with pre-initiation was observed in Case D. Regardless of the demarcation

points, the objective of this phase remained same for all cases. This included expressing and

understanding the general requirement of the technical initiative, type and goal of the project, size

and time horizon of the initiative, the project’s key stakeholders, general resource requirements,

organizational impacts, budget requirements, assumptions, risks, and risk mitigation strategy. For

our selected projects, overlapping of these activities with the pre-initiation activities resulted

primarily from an active role played by the program management who have already completed

some of these activities on behalf of the project (Case C).

Besides the activities performed during the business opportunity proposal and the development of

the business case, developing and managing the RFP process appeared as one of the most vital

activities for Cases A, B, and D. Because Case C decided to use the same COTS product and

vendor selected by Case A, this decision saved the project considerable time and effort with respect

to its RFP process.

A government RFP is usually significantly different compared to a private sector RFP and often

complicated due to policies and legal constraints (Elgazzaret al. 2005). However, the role of RFP

observed in the current study clearly emphasizes the importance of achieving the right level of

requirements, analysis of the market and product space, considering the challenges of a niche

market, and geographic restrictions. Although debates on the merit of RFP continue (Popp &

Page 192 of 307

Dallis, 2012), because government organizations are subject to principles of fairness and

transparency, they cannot avoid such process. This is primarily because an RFP process provides

an open and equal opportunity for every qualified vendor to respond to a project opportunity and

minimize any preferential treatment. Regardless, the Gartner Research Group has presented

research findings that indicate that an organization increases its chances of selecting the right

vendor and product by 40 percent if it executes the RFP process well (Karamouzis & Longwood,

2007). For example, Case B clearly affirmed the significance of the RFP. Its initial RFP was unable

to acquire a successful vendor and so the project team had to make a second attempt by modifying

the initial RFP requirements. This cost the project and the organization nearly 2 million dollars

(CAD). On the second attempt, the project team produced requirements to a greater detail and was

able to achieve three very important positive effects for the project: (a) selecting the right type of

vendor contract, (b) achieving accuracy in work scoping, and (c) feeding the requirement and

analysis phase with the bulk of the requirements. Informant B04PT, a senior business analyst, for

Case B made the following comment.

When you going out with an RFP … your RFP forms your requirement even though

their actual requirement and planning phase is happening at much later stage. …

Our RFP requirements were very detailed from a business perspective … what the

system needs to do … they were very detailed … extremely detailed. … From a RFP

perspective we didn’t have a bidder who didn’t know what we wanted to do and

[the winning vendor and solution integrator] had a really good idea about the

business processes.

Other significant process and tools identified during the initiation of an enterprise COTS

implementation belong to the following categories:

Page 193 of 307

RACI matrix/chart (Tool):

A RACI (Responsible, Accountable, Consulted, Informed) matrix—more formally known as a

linear responsibility matrix—lists the core deliverables vertically as row headers and the various

stakeholders responsible for those deliverables as column headers. RACI categories are:

1. Responsible: The person or role responsible for creating or developing the

product or performing the task.

2. Accountable: The person or role responsible for the quality of the product or task.

3. Consulted: This person or role consulted to provide information that is input into

the product or task.

4. Informed: Stakeholders are informed about the actions taken by the actors listed

under Responsible section.

The RACI matrix can be perceived as one of the simplest of stakeholder analysis and engagement

tools. In a complex IS implementation involving multiple partners and multiple vendors,

disagreement over requirements, refusal to take ownership of requirements, or missing

requirements are not unusual events. A common source of these issues is a lack of contributions

from all relevant parties. This is often caused by the assumption that merely inviting the key

stakeholders to meetings or asking questions in requirements sessions ensures proper

contributions. A major flaw in this assumption is that active engagement is not necessary for

stakeholder contributions. One of the ways that IT projects can ensure the active engagement of

all key stakeholders throughout the IS implementation is by defining the RACI matrix. A sample

RACI Matrix is presented in Table 32 to show the task distribution for relevant stakeholders.

Page 194 of 307

Table 32: RACI Matrix

Role Groups / Individuals

RESPONSIBLE

Responsible for producing the deliverable:

• Determining direction & approach with the Approver

• Seeking concurrence from those on the consulted list

• Obtaining documented approval from the Approver

ACCOUNTABLE

Accountable for:

• Providing early direction to author of the process about approach and objectives

of the deliverable

• Approving the deliverable with consideration of the comments provided by

individuals on the consulted list

CONSULTED

Accountable for:

• Commenting on the deliverable, seeking advice from others in their groups, if

appropriate

• Supporting and endorsing the final deliverable (once approved)

INFORMED

• Consulted in order to provide views, requirements and information that will end

up as content in the deliverable

Support for the effectiveness of a RACI matrix is evident in both Case A and Case B, where each

project team heavily utilized this tool for all major implementation activities, such as requirement

analysis, business-proposed solution development, design and architecture, build books, user

guides, disaster recovery, and business continuity planning. An improper or lack of RACI

utilization can have the opposite effects, as in Cases C and D, where both projects struggled to

maintain desired engagement level for their stakeholders, experienced delays in deliverables due

to role confusion and identified missed requirements at a later stage of the project. According to

informant A04PT, a business analyst in Case A:

RACI chart is great way to get the right people engaged for the right task. … There

will always be someone doing the actual work but C (Consulted) and I (Informed)

parts are very easy to miss in large projects. … This might have some serious

implications down the road. … For us, it was absolutely critical. … The informed

Page 195 of 307

and consult part helped avoid some major disagreement and this is simply because

engagement is difficult unless you have some skin in the game.

Requirement and Planning

A second logical set of activities for all Cases were related to collecting or finalizing requirements

and formulating an overall implementation plan. Activities performed during the initiation phase

appear to have had a direct and considerable influence on activities performed during this phase,

as high-level requirements and constraints were identified during the initiation of each project.

These high-level requirements and constraints served as foundational blocks for the activities of

this phase. The objective of this phase appears related to development and communication of the

project charter and project management plan to everyone involved to ensure a common

understanding of the project objectives, strategy, and approach. Planning aspects of this phase also

aimed to establish a clear understating regarding team involvement and project organization,

communication expectations, and general project milestones or target dates. This ensures that

efforts are focused toward project goals. Additionally, one objective of this phase is to express the

client's business requirements and ensure that related non-functional requirements for the COTS

solution are identified (e.g., security, facilities, or business continuity).

For all instances of the selected projects, the requirement and planning stage was a much deeper

dive into the initial business requirements gathered during the RFP process. In addition to

reviewing the functional requirements at a more detailed level, the project teams also refined and

identified the non-functional requirements specific to the organizational environment. Vendor(s)

and the solution integrator were directly involved with project teams for each Case. Most

Page 196 of 307

informants identified this interaction to be absolutely critical. The reason for such a high level of

emphasis is that often the vendor-related contract was not finalized at this point (e.g., Case B, Case

D), and this direct interaction with the vendor gave everyone involved with the project a final

opportunity to minimize any possibilities of future change requests. Additionally, business

requirements developed during the pre-IT implementation period were primarily focused on

functional requirements from a “present-day” organizational perspective. These requirements

contained a very rudimentary definition of non-functional requirements. Solidifying the non-

functional and technical requirements and identifying product enhancement needs were also

performed during this phase. According to informant B06VR, the vendor’s technical lead in Case

B:

There were lots of room for interpretations which either drives/increase the cost

coz we are going to take a higher risk premium as a vendor. As a supplier we are

gonna to say—if there is this much slugging here, we are gonna have to charge

more because it could go this way or that way. It also gives more room for dispute

and CRs.

From a vendor perspective, this phase carried a lot of significance for Cases A, B, and D, as the

project teams exposed their existing systems to the potential vendor to help solidify work scope

and finalize their RFP-related bids. The high-level requirements released during the RFP typically

help vendors formulate their own perspective on the requirements. For example, one specific

requirement could be delivered out of the box. A second might be the specific requirement could

be a minor configuration on the COTS product. And the third could be addressed through a minor

glueware, and so on. It was only during this phase that both the vendors and the solution integrators

had an opportunity to perform due diligence on the RFP requirements. More specifically, vendors

Page 197 of 307

were able to validate the requirements with project teams, get a good handle on the legacy system’s

data in terms of volume and cleanliness, and comprehend the structure and complexity around the

metadata of the legacy systems. All these parameters directly influenced the scope determination

and bid precision for the vendors. According to informant B05VR, a senior project manager of the

vendor’s team in Case B:

With a fixed priced bid, we put in a risk premium on that. We come in and go—

yeah … we think it’s gonna cost a million dollar[s] but these are the things that are

[a] little bit variable and if [we] have any problems, we are gonna bid a million

and a half. That’s to cover our risk. What that does, increase the price of our bid

and we may now lose the bid because we have gone too high if someone else didn’t

take that risk premium.

Requirement Traceability Matrix (Tool)

During the requirement and planning phase, one very important artifact that was produced by the

project teams in Cases A and B was the Requirement Traceability Matrix (RTM). An RTM’s

primary goals are to: (a) provide a record of each stated, committed, and agreed requirement; (b)

track progress and decisions during the project regarding how each requirement is being addressed;

and (c) provide auditable proof at the end of the project regarding how each requirement was

satisfied and how the scope of work was met and controlled.

Establishing an RTM was critical not only for determining scope but also for controlling the

change request process at later stages of the project. For example, in Case B, the project’s RTM

initially contained 183 requirements and was partially reviewed by the vendor. Since the project

Page 198 of 307

implementation followed an agile approach, the RTM established at this phase evolved further

during the subsequent phases of the project. However, deliverables were validated against the

requirements in the RTM to trace its origin and connect the deliverable that satisfied that

requirement. According to informant B07VR, a solution integrator in Case B:

From the vendor perspective and from the client perspective … everything that we

do for a project better trace back to a requirement otherwise we shouldn’t have

been doing it. There should never have been a user story that couldn’t be attached

to a requirement in the SOW [Statement of Work]. The only case [in which] that

could happen [is] if this is a new requirement which should have been a CR

[Change Request]. Theoretically, between the SOW and CRs, that should be the

scope of what was delivered because some of the CR might have removed the scope

from the SOW and you get credit.

DeepDive ™ Process

Analysis of early implementation phases of Cases A, B, and C indicates the existence of a key

process, DeepDive™. This process was initially developed by the IDEO group (a learning design

company) for rapid product development. However, due to the effectiveness and efficiency of this

process in problem solving and idea creation, DeepDive™ is now increasingly used for many types

of activities (RapidBI, 2007). These include complex requirements analysis and conceptualization

for information systems development.

Page 199 of 307

IDEO’s original process consisted of five steps: (a) understand and observe; (b) synthesize; (c)

visualize; (d) prototype, evaluate, and refine; and (e) implement. This process is shown in Figure

14:

Figure 14: IDEO’s original process for rapid product development (Moen, 2001).

Boynton and Fischer of the International Institute of Management Development (IMD) further

refined IDEO’s original process into a generalized innovative problem-solving method called

DeepDive™ which was acquired by Deloitte Consulting in 2006. According to Deloitte, “The

DeepDive™ is a combination of brainstorming, prototyping and feedback loops merged into an

approach that executives can use with teams to help develop solutions for specific business

challenges” (Deloitte, 2010).

In Case A and Case B, DeepDive™ was used in two stages of the projects: requirement and

planning, and analysis and conceptual design. Typical DeepDive™ sessions lasted from few hours

to a full day. This process appeared very effective in the areas of requirements clarification, scope

solidification, and component design clarification. According to informant A04PT, a business

analyst in Case A:

Page 200 of 307

So it was important from their [Vendor] perspective to nail down the specifications

what exactly we wanted contractually and it was important for us too … to make

sure that all our [client] requirements are captured and interpreted correctly. …

Way down in development during the sprints at times a lot of requirements we

would say No, we wanted it that way and they [Vendor] were very clear and able

to go back in terms of traceability and say, No, we did a deep dive back in 2012

and this was … this was the specification.

Although Case C and Case D did not specifically use DeepDive™, both projects used some

variation of this process in early phases of the project’s lifecycle. As this process was not identified

as one of the standard organizational processes promote by the PMO, its utilization in Case C and

Case D were on a limited scale. A lack of utilization of DeepDive™ can be directly linked to several

requirement-related disputes experienced by both projects.

Analysis and Conceptual Design

Soon after the completion of requirements gathering and finalization, the activities for each project

showed a similar trend, which involved looking at the overall COTS solution from a conceptual

standpoint within the organizational context. The primary objective of this phase is to determine

which options or alternatives best meet the requirements of the target business solution for the

existing organizational context. Therefore, if a project is decided to proceed, this phase continues

with the stakeholders' decision and agreement on the most appropriate option or alternative to

develop a conceptual design for the solution. For all selected cases, an analysis was performed on

Page 201 of 307

various options against specified criteria, and a recommendation was presented to stakeholders

with substantiating documentation.

For Cases A, B, C, and D, analysis and conceptual design focused on requirements elaboration and

clarification. For example, the client and vendor jointly discussed and decided how a particular

requirement would be handled by the future system. Additionally, activities in this phase were

significant from the perspective of maintaining scope and system’s functionality and requirement

traceability. According to informant A09BU, a business analyst for Case A:

What we had was high level requirements, it would have been broken down into

three or four more granular requirements, then they became user stories, i.e. how

to do something, and we had traceability in there [using RTM]. In the end, what we

reported back was: this requirement is met by the following user stories, and these

were the tests that we did to confirm that the user stories were valid, so by the sign

off of all those test against all those user stories confirms that we have delivered

those requirements.

Engaging the extended development and support team of the organization (including the COTS

support team and ITS operations support) was a key factor for a smoother development cycle and

transition to support phase. The dotted line from this phase, leading to the pre-closeout link (see

Figure 12), indicates this engagement and information flow. Informants from all four cases

affirmed the significance of this information flow by presenting both positive and negative

evidence. For organizations that are similar to the host organization—where information

technology services (ITS) has a dedicated team for each core function (e.g., network, server,

storage, database, security, applications) and follow a rigid change-management process—delayed

Page 202 of 307

initiation of this engagement is very likely to impact project deadlines and critical milestones. For

the selected cases under current analysis, Cases A and B were able to avoid all transition-related

roadblocks by engaging the support team very early in the project life cycle. However, negative

consequences, such as infrastructure unavailability (e.g., servers, firewall rules, network

connectivity) resulting from a delayed engagement, were reported by informants in Cases C and

D.

Design Authority (Process)

The design authority process was one of the most effective processes introduced by the project

under Case B. It was established soon after the discovery of some discrepancies between client

requirements and vendor implementation of those requirements. This process consisted of two key

elements: design authority group and a design approval process. The design authority group

primarily consisted of roles drawn from the business users, like solution architect, enterprise

architect, data architect, and subject matter experts. At the beginning of each sprint, this group

reviewed the design aspects of the requirements and discussed how they should be implemented.

As a result of this discussion, the vendor captured the design decisions and initiated an approval

process by sending the proposed design to the client. The design approval process was a very tight

document review process where the design document went through multiple feedback

modification cycles between client and vendor. In Case B, the design authority produced nearly

60 design decisions throughout the life cycle of the project. Design decisions were proven to be

valuable in three key areas: (a) detailing how the system should behave at a lower level, (b)

minimizing assumptions on the vendor’s side regarding components being developed, and (c)

Page 203 of 307

ensuring alignment between delivered functionalities and specifications to avoid unnecessary

change requests.

All of the project team members, from both the client side and vendor side, supported the

effectiveness and efficiency of the design authority process. They also expressed an intention to

employ this process in future projects. According to informant B01PT, a senior project manager

in Case B:

Most time this work falls on the development team almost with little bit of

involvement from the BAs [Business Analysts]. … I would say establishing a design

authority that includes the vendor, includes the architects and business is really

critical. … There were a number of times that we had issues, questions or things

that were not going as they should and we fell back on the document that was

produced as a design authority to make sure that we were delivering to the design.

Although the name Design Authority was not adopted formally, Case B’s project demonstrated

similar activities during this phase. This involved establishing formal design decisions and using

them as reference points. However, for Cases C and D, design decisions were much less formal,

which introduced some design changes and design conflicts at the late stages of a project’s

lifecycle. Looking across the Cases, the design authority process helped reduce the development

cost and time for both Case A and Case B, as well as helped foster an active engagement level by

both the IT and business stakeholders. Clarifying the design decisions ahead of every sprint

significantly reduced the volume of communications for the offshore team on the vendor side.

Reduction of requirements related to communications not only reduced the idle time on the

developer side, but also minimized deviation from the requirements. Design decisions produced

Page 204 of 307

by the design authority also acted as a safeguard against unexpected change requests.

Unanticipated change requests can be a substantial source of expense for a fixed-cost project. A

review of the disapproved change requests found in the projects change management log indicates

that both Case A and Case B were able to avoid several high-impact changes in many instances

throughout a project’s lifecycle.

All four projects selected for this study followed a variation of agile implementation, but Case A’s

and Case B’s processes closely resembled a traditional sprint-based agile approach. Adoption of

the agile philosophy also caused the analysis and conceptual design activities to repeat itself

throughout the project’s lifecycle. From this stage of the project, there were two possible choices

for the next phase, depending on the complexity of the required functionality: the design and

architecture phase followed by the development phase, or directly into the development phase. The

second route was usually followed for simple and standard requirements that did not require a

design approval by the project’s design authority.

Design and Architecture

A distinct set of activities related to technical design and architecting were observed for all four

projects. The objective of this phase is to determine the detailed design of the solution, including

how it will integrate with the existing environment and systems. This phase also includes the

procurement of additional hardware, software, equipment and other materials, and identifies any

new operating and maintenance needs for the target COTS system.

In the design and architecture phase, the primary task is to take the input from high-level

requirements and user stories, and convert them into a detailed design document. The design

Page 205 of 307

document outlines every detail of the target system behaviour related to the requirements.

Informants from Case B and C reported some examples of these detailed behaviour for the target

system, including how the user interface (UI) is going to work, what rules should trigger when a

certain button is clicked, how the web services should behave, or notification procedure for

scheduled job failure. Because almost all four projects divided development activities between the

in-house team (glueware development by the solution integrator) and the vendor-provided

technical team (COTS or product enhancement by the vendor), design decisions made during this

phase were critical for both the project team and the vendor.

For Cases A, B, and D, this phase appeared to be a repeated set of activity as each project team

considered the design of a single module at a time. Case C, on the other hand, attempted to design

the complete solution at this point, taking an encompassing approach. Regardless, periodic design

review and repeat of design workshops were reported by the informants in Case C for the duration

of the development cycle of the project.

Development

Following the design of an overall solution, the technical architecture group passed the approved

design decisions to the project’s development team, which included vendor-provided developers

and offshore development teams. The objective of this phase appeared to develop and test the

business solution, including the required environments. The project team covered several stages

of testing at this point. Salient categories of testing for Cases A, B, and D included unit testing,

system testing, functional, and integration testing. For Case C, testing was mostly limited to

developer’s unit testing of the COTS modules. All four projects also drafted a production

Page 206 of 307

implementation plan, which included production roll-out, user and technical training, raising

organizational awareness, and clarifying operating and maintenance responsibilities. User and/or

procedures guides were drafted and revised as required by the projects.

Both the informants and secondary data of Cases A, B, and D indicate that development was one

of the most critical and challenging phases of the project. First, this phase was the biggest work

stream for all four projects, and second, and more importantly, several critical activities progressed

in a parallel fashion. For example, metadata conversion and validation, product enhancement, and

data migration and reconciliation all happened in an iterative fashion.

Implementation practices in Cases A, B, and D indicate that all three projects followed an agile

approach of IS development that was significantly different from the traditional waterfall

approach. In addition, both Case A and Case B had a component of an offshore-development

model, as the vendor’s technical teams were located in Europe.

Agile methods emerged over a decade ago as an alternative to plan-driven software development

methods (Dingsøyr et al., 2012; Agile Software Development, 2014). It is a philosophy of software

development based on iterative and incremental development. Conboy (2009) formally defined

agile as “the continual readiness of an information systems development (ISD) method to rapidly

or inherently create change, proactively or reactively embrace change, and learn from change while

contributing to perceived customer value (economy, quality, and simplicity), through its collective

components and relationships with its environment.” Besides the twelve principles outlined in the

agile manifesto (Agile Alliance, 2015), the core values of agile development focus on individuals

and their interactions, early delivery of working software, customer collaboration, and

responsiveness to changes. These all contribute positively to implementation time, quality, and

Page 207 of 307

cost of software—all of which are essential for today’s highly dynamic and competitive business

environments (Stavru, 2014). Projects in Case A and Case B fully utilized the advantage of the

greater flexibility offered by the agile development and implementation method using result-

oriented planning. This led the project teams to develop with high accuracy the most important

and complex areas of functionality. This also provided the greatest business value and actively

engaged all stakeholders to help shape the new system. According to informant B06VR, the

technical lead of the COTS vendor in Case B:

Each sprint gives the vendor and the BA’s [Business Analysts] increasing ability to

understand how the business users think about the requirements. Therefore

experience gained from one sprint significantly increases the accuracy of

interpretation of the high level requirements at the detailed design phase and also

the correctness of the functionality delivered. … Each sprint also reduces the Risk

since we developed the most risky component early. …We can re-plan way earlier

and when it’s ok to re-plan; coz agile gives you the ability to react … scale down

the requirements or add more resources. If you do your detail design upfront, it’s

not worth the paper it’s written on because until you go to the development—you

don’t realize the problems with your design.

For Cases A, B and D, an agile implementation approach motivated the project team to split high-

level requirements into a fixed number of sprints during the requirement and planning phase. Each

sprint delivered a certain group of business functionalities for the project. During the sprints,

analysis and conceptual design, design and architecture, and development and delivery phases all

occurred in an iterative fashion.

Page 208 of 307

One of the more salient processes during the development phase, as well as the design and

architecture phase, was change control. Change control consists of a review and approval

committee, a set of guidelines, and escalation points. Strict enforcement of change control helped

Cases A and B avoid several changes that could have added significant expense to the project’s

fixed-cost budget, thus causing the project to exhaust all of its contingency funds.

Delivery

Due to the agile nature of the implementation process for all selected Cases, the delivery signifies

a different meaning compared to the delivery phase of a traditional waterfall model. The objective

of this phase is to ensure a smooth production roll-out of the solution, with a high state of readiness

in the user community. Furthermore, this phase was critical to the effective handover of the new

COTS system between the project team and the operations area(s). In Cases A and B, project teams

ensured that the implementation of the support model occurred prior to the solution deployment

activities related to delivery. For Cases A, B, and C, the project team remained intact for a settle-

in period to assist the user community and operations support teams as required. This was intended

to address any post deployment and go-live incidents with the new COTS product. Whereas the

delivery in a waterfall implementation model has a single time slot, that was not the case for the

selected projects under analysis. As a result, all four projects indicated post-deployment defect

remediation activities during the project’s lifecycle.

The regulatory system implemented under Case B and the financial market system implemented

under Case A both contained a delivery phase at the end of each sprint, when a certain set of

business functions were completely developed (as planned during the requirement and planning

Page 209 of 307

phase). However, the final data migration to the target production environment was performed as

a part of the “go-live” activities of the project. In Case C and Case D, a similar trend was observed,

but the scope of the deliverables was much larger than a typical sprint-based model. This made the

delivery phase for Case C and Case D resemble the delivery phase of a waterfall model of IS

implementation.

Business validation or audit was one of the key activities performed during the delivery phase of

each project. This included, primarily, user acceptance and integration testing. When testing in

delivery phase, there are usually a number of prevalent models that most COTS or IS

implementation projects utilize. Either the vendor tests the system and sends the results to the

client or the testing is done jointly in an integrated fashion by the vendor and client. For Case A,

B, and D, the second option was chosen. For all three projects, integrated testing teams comprised

of the vendor, project’s technical team, and the client, conducted testing, targeting both functional

and non-functional requirements. This approach is usually well suited for agile developments and

has proven to be very effective, including for the present cases. According to informant B03PT, a

senior COTS developer in Case B:

One good thing about this project was testing … there was unit testing that was

done by the vendor but the usability testing was done with business resources.

Solution integrator provided [the]test manager but not the actual testers. … We

said the business needs to give us the test resources for functional, regression and

system integration testing. … So we combined functional testing with the UAT

essentially. … They are testing the system as it going to be for them as opposed to

Page 210 of 307

we are testing it and saying it’s all good and they not accepting it. … This is a new

concept because typically you have testers and then you have business users after.

Close-Out

All four cases analyzed for the current research indicated a concluding set of activities. Activities

of this phase are quite similar across all cases, as “close-out” is not only mandated by the PMO’s

project management guideline but is also a standard best practice recommended by the Project

Management Institute (PMI). The primary objective of this group of tasks is to bring the project to

closure from an operational, administration, financial and human resources perspective. The

project close-out phase provides an opportunity to assess a project's success and results through

team and client satisfaction surveys and identification of lessons learned to be applied to future

projects. While recognition should be provided to individuals and the team throughout the project

life cycle, this is a good time to formally recognize the efforts of team members. Informants from

Cases A, B, and D indicated multiple close-out activities where feedback was collected to improve

next iteration, and recognition mechanisms were used to reward outstanding team performance.

In summary, implementation process-related activities for all four Cases demonstrated the

existence of seven logical groups: initiation, requirement and planning, analysis and conceptual

design, design and architecture, development, delivery, close-out. Although a pre-initiation group

was frequently observed for most of the Cases, this formed a major input for the initiation and

requirements phases of the projects. Furthermore, an agile aspect of the implementation was

supported by the repeating conceptual design, design and architecture, development, and delivery

Page 211 of 307

activities. Whereas a macro-level model (Figure 12) indicates the information flow between logical

project phases, a micro-level or activity-based model of COTS implementation (Figure 13)

captures the level of engagement and contribution by major project stakeholders. Activity analysis

of Cases A, B, C, and D also indicated existence of several implementation-related processes and

tools, such as deep-dive, design authority, RTM, and joint testing team. Projects utilizing these

tools and processes demonstrated a higher control and positive influence on their CSF compared

to those that did not utilize these processes and tools.

6.2 Control configurations and control balancing

From a control configuration perspective, our analysis of Cases A, B, C, and D identified several

distinct control configurations that are directly related to each project’s success or failure.

Furthermore, going above the simple functional purpose to a more abstract level of intended

purpose reveals a dynamic nature of control configurations and their relationship to the

management of CSF or challenges throughout the implementation of each project. Delving deeper

into the processes, tools, and control configurations, and adopting a categorization perspective, we

detected four discrete control orientations for each project: (a) strategic, (b) responsibility, (c)

harmony, and (d) persuasion. A summarized comparison of the four orientations is presented in

Table 33:

Table 33: Comparison of Control Orientations

Control

Orientations

Related Project Phases Control Configuration Objective

Strategic Pre-initiation

Initiation

Alignment of the project with organizational objectives

and strategic management of stakeholders

Responsibility

Requirement and planning

Analysis and Conceptual

Design

Aligning stakeholder’s expectations and project’s

objectives at a logical level

Page 212 of 307

Harmony


Development

Align project deliverables to organizational capability and

minimize negative impact on non-project stakeholders,

thus, Aligning stakeholder’s expectations and project’s

objectives at a physical level

Persuasion

Delivery

Close-out

Promoting project objectives related to user’s acceptance

of deliverables and reinforce the achievement of ‘strategic

values’ promised during the gating phase.

Our qualitative analysis of primary data from all four Cases indicates targeted adjustment of

control configurations at different phases of the project. This provides clear indications of control-

balancing decisions by the projects. An examination of Cases A, B, C, and D’s implementation

from a control balancing perspective indicates the influence of three distinct trigger factors: (a)

shared understanding among stakeholder groups, (b) negative anticipation by one party regarding

the future performance of the other, and (c) deviation of expectations or unfulfilled expectations.

These three factors can be further classified into two major categories: re-active control balancing

and pro-active control balancing.

Strategic

Strategic orientation was observed very early in each project’s lifecycle, during the opportunity

exploration and strategic initiative proposal phases of the project, which can also be considered

pre-project phases. At an early stage, engaging stakeholders selectively but optimally appeared to

be a rudimentary force that helped build project capabilities and much needed social capital to

tackle several CSF dimensions and other challenges throughout the project-lifecycle, such as top

management commitments, project champion, organization-wide project visibility, timely and

effective escalation, project management strength, and culture change management. Pre-defined

project implementation templates, the project “gating” process, and project management office

(PMO) oversight facilitated the establishment of a governance committee for each project.

Page 213 of 307

Although processes such as project gating and PMO oversight can be classified as organizational

procedural controls, collectively they enabled establishment of further control mechanisms such

as a memorandum of understanding (MOU). Specifically, in Cases A and B, formulation of a clear

project governance structure and MOU played critical facilitation roles during later stages of the

project, when escalation was necessary to handle a time-constraint situation or force project

partners into requirement compliance through a non-institutional process.

Table 34 Strategic Control Orientation

Control

Orienta-

tions

Project

Phases

Control Balancing Trigger Factors CSF and Challenges Saturated

Code/Consolidated

Code for trigger

Strategic

Pre-

initiation;

Initiation

No signed contract [A], lack of

knowledge about the capability of

selected COTS [A], Industry foot print

of the vendor [A, B]; Absence of

shared history [A, B], Project context

[A, B], Organizational context [A, B],

Absence of previous contract [D],

Absence of formal authority [C],

Existing contract with organization

[C], Negative past experience [D],

Pre-disposed trust condition [D],

Previous cooperative settings [D];

PMO prescribed compliance

requirements [A, B, C, D].

Visioning and

planning,

Building a persuasive

business case,

Project champion,

Client Consultation,

Leadership

commitment,

Culture and process

management

Contextual limitations

[leading to negative

anticipation];

Absence of past

performance and

history [ leading to low

shared understanding];

Need to strategic

resource, and

commitment, support [

Need for shared

understanding

development];

Pre-initiation is often executed as a separate project or preparatory activities for large budget and

complex projects, as was the case for the projects in Case B and Case D. Pre-initiation activities

directly lead to the initiation phase of the project where objectives include: expressing and

understanding the general requirement, type, and goal of the project, size and time horizon

determination, key stakeholders engagement, general resource requirements identification,

organizational impacts anticipation, budget, assumptions, risks, and risk-mitigation strategy

development.

Page 214 of 307

In the absence of explicit understanding of each other’s processes, partner’s capabilities, and

lower-level deliverables, a trust-based control configuration was allowed to dominate relationships

among stakeholders. A trust-based control enhanced dynamics for the project team and facilitated

augmentation of shared understanding among project participants during early phases, which, in

turn, aided due diligence on both the client and vendor sides. For all four Cases, application of

trust-based informal controls reflects the absence of a formal authority by one party over the other

through a binding contract or agreement. Whereas the ability to exert formal control or

administrative controls was often absent at these early stages of the project, the project team’s urge

to establish a common understanding or increase the level of shared expectations with all

stakeholders triggered a change in control configuration from a unidirectional to bi-directional

style. Such change of control configuration was observed in Case D. Alteration of control

configurations are also necessary when a negative anticipation regarding the performance of one

party is prevalent. Negative anticipations in the early life of a project could result from negative

past experiences, an absence of shared work history, pre-disposed trust conditions, project context,

and/or organizational context. These factors often require adopting a tighter and unidirectional or

authoritative control style to build necessary shared understanding and ensure proper synergy

among the stakeholders.

Regardless of the direction of the control configuration change, a common trait related to the

control portfolio shift was to address contextual challenges and strategic alignment of the project

with organizational priorities. Establishing project gates or strategic investment governance

mechanisms as critical control elements help ensure a project’s alignment with organizational

priorities. A four-stage gating control is imposed by the organization and the PMO-influenced

Page 215 of 307

informal and collaborative control choices of the projects. (Refer to Appendix D for a detailed

overview of the organization’s project gating process.) A collaborative and enabling control style,

such as trust-based control, was particularly important in building leadership commitments for

each project. This was indicated by the informant C01PT, a senior project manager in Case C:

Maintaining trust and reputation as central authorities were definitely the

driving forces, of course beside the increasing operational needs .… It [pre-

initiation and initiation] was vital for the success. Little knowledge of each

other’s internal processes [among the clients] or no history [between

clients and vendor], RFP [request for proposal], MOU [memorandum of

understanding] they all helped tremendously, especially getting the right

people at the table and close gaps [in knowledge and understanding].

Gating requirements had to be met to secure the funds but you also measure

the top level commitment here [Pre-initiation and initiation].

Since a project often executes in a dynamic organizational environment, contextual challenges

such as inter-project dependencies and competing organizational priorities, often demand

alteration of a project’s existing control configuration. Several informants in Cases C and D

reported such context-related challenges, where over-reliance on a trust-based control

configuration with the vendor and internal IT support teams resulted in several delays for multiple

deliverables. This eventually caused both Cases C and D to miss a few vital requirements and

project deadlines. Although a collaborative control configuration was initially optimal for all four

Page 216 of 307

projects, the project management teams in Cases A and B recognized the need to improve shared

understanding during the initiation phase of the project. Facing a similar organizational context

and a non-existent shared history with the vendor, projects in Cases C and D decided not to

introduce cognizant behavioural and outcome-based controls to enhance the level of shared

understanding among project participants. Therefore, divergent expectations, reduced sense of

urgency, and reduced comprehension of strategic significance were observed among various

stakeholder groups. This also impacted several strategic aspects of the projects in Cases C and D,

including maintaining executive leadership commitments and support, ensuring a fair share of

scarce resources (i.e., internal support), and process innovation and cultural manipulations to

integrate new business processes.

Responsibility

Responsibility orientation was more salient during the early implementation phases—e.g., project

kick-off, requirement and planning, and analysis and conceptual design—that immediately

followed the strategic pre-initiation phase of each project. A dominant theme emerged through

analysis of the performed activities and adopted processes of the project kick-off, requirement and

planning, and conceptual design phases. From a control perspective, this theme supports the notion

of “aligning expectations at a logical level.” Therefore, tools and processes employed at this stage

supported a more encompassing approach towards stakeholder considerations compared to the

actual engagement level, closely aligning to the normative aspect of stakeholder theory (Donaldson

& Preston, 1995), which suggests that all stakeholder interests have intrinsic value and should be

taken into consideration. Additionally, the control objective for the project teams reflected a need

Page 217 of 307

to map new COTS capabilities to the need of different stakeholder groups and existing business

processes. Such change often needs an introduction of behavioural control through the support

model.

Comparing the responsibility orientation across all four cases, there is a clear divergence between

Cases A and B and Cases C and D in terms of control configurations. In Cases A and B, the project

applied a unidirectional and frequent control on both the vendor and internal IT support teams.

Although the initial control configuration was very relaxed for both projects, conceptualization of

the future state and project team’s concern about a potential cost increase led to the adoption of a

cautious and proactive approach in terms of control. Controlling the vendor in an authoritative

manner helped both projects overcome several critical issues related to the procurement (i.e., RFP)

and off-shore development process. Support for such an authoritative approach is offered by earlier

control researchers like Choudhury and Sabherwal (2003). They have indicated informal controls

to be an inappropriate choice for an outsourced project and pointed out the difficulties in applying

a “clan control,” even for co-located internal teams.

Project teams were also able to exert a certain degree of control over the vendor-provided technical

team and the vendor’s project management practices. According to an informant from the vendor

side of Case B, the COTS technical lead, frequent use of outcome-based control has been

identified:

[Encompassing considerations for stakeholders] doesn’t mean RFP

requirements are invalid; rather it might the broader focus [by the client]

causing the slugging. … RTM [requirement traceability matrix] definitely

helped us solidify the cost estimates a lot, it also helped them [the project

Page 218 of 307

team] in regular progress tracking by simple putting a check mark or IP

[in-progress] beside the requirement.

Timely provisioning of project IT infrastructure components has been an ongoing challenge for

this organization. Because it was experienced and reported by earlier projects within the

organization, both projects in Cases A and Case B considered infrastructure provisioning-related

challenges very early in the project’s lifecycle. Consequently, both projects decided to adjust their

control style for internal IT support teams from an initial trust-based control to a more formal and

authoritative control style consisting of outcome-based and behavioural controls. This was

achieved by establishing a reporting protocol for completed deliverables and the infrastructure

support team following a project-prescribed provisioning sequence. Frequent status tracking for

IT support team’s activities helped ensure an on-time delivery of internal infrastructure

components. On the client side, both projects increased the control frequency. Interestingly,

business clients also monitored and inquired about several project parameters. Additionally,

business expected the project team to follow a prescribed protocol, Technical Proposed Solution

(TPS), to validate functional and non-functional requirements presented through Business

Proposed Solution (BPS). Production and validation of TPS required significant informal

interactions between the client and the project team, as well as with the vendor. However,

production of TPS required the project team to follow a process introduced by the client (i.e., a

form of behavioural control). For large TPS, the client also requested regular progress updates

from the project team. This shift from an informal control relationship to a coordinated or bi-

directional control, comprising of formal and social controls, was driven by the mutual willingness

of the project team and client to quickly reach an agreement in terms of functional requirements

Page 219 of 307

of the future system. The project team identified several gaps in understanding and diverging

opinions during brainstorming sessions and workshops with the business users. Furthermore,

formulation of requirements by the business users showed significant dependency on technical

experts from both the core project team and the vendor. These factors required both the project

team and clients to maintain close coordination in terms of cooperation and communications with

a common goal of formulating valid and consistent requirements for the target system.

In Cases A and B, the control configuration shifted from informal and relaxed to a more

authoritative and coordinated style, but control configurations for Cases C and D were quite

different during corresponding project phases. Both projects under Cases C and D displayed an

informal and relaxed control style throughout. Although an optimal level of shared understanding

between the project team and several other stakeholder groups was not present, no significant

milestones were missed by the project teams. A closer examination of this control attitude in Cases

C and D points to the influence of several contextual factors. These factors were responsible for

masking the presence of several trigger conditions related to control balancing decisions. For

instance, in Case C a new project manager with no significant project management experience was

assigned to the project during the requirement and planning phase. On the one hand, assessing

shared understanding among project participants requires the possession of emotional intelligence.

But on the other hand, emotional intelligence develops with the experience of the project manager.

A lack experience also encourages a project manager to follow a more traditional (i.e.,

authoritative) control style.

Other contextual factors, such as reduced ability to monitor an actor’s behaviour (e.g., the off-

shore development team), lack of technical know-how of the new COTS system, and incorrect

positive assumptions about the validity of previously collected requirements, motivated the project

Page 220 of 307

team in Case D to maintain a trust-based control configuration with the vendor, internal IT support

teams, and the client. Although a trust-based control portfolio, implemented through interactions

and collaborations, dominated both projects under Cases C and D, this choice of control

configuration was not motivated by legitimacy concerns or a desire to enable employees to deal

more effectively with inevitable contingencies (Adler & Borys, 1996). As a result, this enabling

control style failed to achieve its natural effects in Cases C and D. This was reflected in both Cases

by missed requirements and delayed deliverables at a later stage of the project’s lifecycle.

Additionally, informants from both projects indicated a lack of project management strength,

inadequate legacy systems considerations, inability of timely escalation of issues, and

infrastructure-related roadblocks—all of which introduced considerable delays for the project.

Table 35 presents a consolidated view of the control configurations for all four projects,

emphasizing the dominating control themes.

Table 35: Responsibility orientation

Project

Phases

Control Balancing Trigger Factors CSF and Challenges Saturated

Code/Consolidated

Code

Requirement

and planning,

Analysis and

Conceptual

Design

Need to improve shared understanding [A],

Clear understanding on the future state [A];

Agreement on requirements [B], shared

understanding [B], Clear understanding on the

future state [B], missed deadlines [B];

Confirmation of resource availability [B],

Development of shared understanding [B],

protecting price agreements [B], Following

Norm [C], Leadership competencies [C],

Reduced ability to monitor behaviour [D], lack

of technical know how [D], Positive assumption

on the validity of previous requirements [D],

Missed delivery dates [D].

Communication plan,

RFP Execution, Legacy

system considerations,

Project Management

Strength, Offshore

Development Model,

escalation process, IT

Infrastructure

Understanding

current state and

future state [need to

develop common

understanding]

Need to minimize

opportunistic

behaviour

[Negative

Anticipation]

Page 221 of 307

Harmony

As project implementation moved towards design and development activities, each project team

adjusted processes, tools, and control objectives to promote harmony among different stakeholder

groups and other internal processes. In terms of control configuration, the salient theme of this

phase appears to be the objective of aligning stakeholder needs with organizational and project

capabilities from a physical and logical resources perspective, and removing roadblocks for the

core project team.

Table 36: Harmony Stage

Project

Phases

Control Balancing Trigger Factors CSF and

Challenges

Saturated

Code/Consolidated Code

Design and

Architecture,

Development

Confirmation of resource availability [A],

Development of shared understanding [A, B],

Delays in delivery [A, B], Protecting price

agreements[B], Following Norm [C],

Leadership competencies [C], Missed

Milestones [C], delays with deliverables [C],

Lack of clarity for roles and responsibility

[D], High task inter-dependency [D], Absence

of technical expertise [D], Perceived attitude

of the other actor [D]

Low engagement with broader scope

Data conversion

and integration,

Training and job-

redesign,

Change

management,

Dedicated internal

support resource

Deviation of expectations

[Unfulfilled expectations]

Control configuration took a slightly different turn at this stage due to the offshore development

component and an agile delivery approach. All three projects in Cases A, B, and D demonstrated

a bi-directional or mutual control over the vendor compared to a more formal, unidirectional

control observed during earlier phases. Although some aspects of authoritative control were

present in the early harmony phase during design and architecture activities, this configuration

turned into a more coordinated style during the development stage. Reasons for such shift were

related to minimal negative anticipation (i.e., low unfulfilled expectations and higher level of

common understanding) by the project team on the vendor side. Other factors for such a control

Page 222 of 307

shift were directly related to the project and organizational contexts. All three cases had an offshore

component in their development cycles where a portion of the code or COTS-related product

enhancements were performed by the vendor’s technical team. On-site project teams were also

comprised of organizational employees and vendor provided consultants. To maintain the level of

shared understanding and overcome data conversion- and integration-related challenges, a less

authoritative approach was adopted by these projects. This helped maintain team dynamics,

individual motivations, empower team members, encourage frequent open communications, and

improve sense of belonging, as reported by the informants in Cases A and B. The immediate effects

of this approach could be observed in higher team cohesion and, occasionally, the vendor taking

on more responsibilities than anticipated. A combination of formal and social controls to maintain

shared understating is indicated by informant A10IT, a solution architect in Case A:

Discrepancies discovered [requirements provided by project partners] here [on

this phase] was a matter of worry [deviation of expectation]. Design authority did

a good job in resolving them. Daily status meetings and tracking meeting minutes

also helped accelerate the inputs [i.e., feedback from partners]. … [L]ots of things

were happening on the other side [COTS vendor in Europe], and we certainly

didn’t want to find any roadblocks at the last minute [i.e., negative expectations].

The dominant control configuration trend between all four projects and IT support teams appeared

more authoritative. Although an acceptable level of shared understanding was present between the

IT support team and the projects, past experience and multiple on-going projects competing for

the same resources compelled the project team to consider the possibilities of resource

Page 223 of 307

unavailability for their project. Dictated by the overarching goal in the harmony phase, at this point

unidirectional controls were applied on IT support teams to enforce desired behaviour. This

approach differentiated the harmony phase from the authoritative approach of the responsibility

phase as certain behavioural controls, targeted toward internal IT teams, helped develop norms for

the project. This eventually contributed to a high level of shared expectations and understanding

between the project teams and support teams. In Case C, the control relationship between the

project team and IT support teams shifted significantly from a trust orientation to an authoritative

style at the end of the harmony phase. This change in control configuration was influenced by the

unavailability of internal resources for the project, and so some of the critical scheduling

components for the project were not delivered as expected (i.e., unfulfilled expectations and

deviations of expectations). Although a deviation of expectations can certainly trigger a change of

control configuration, for Case C it was unable to achieve the desired effect of establishing mutual

understanding or expectation for future deliverables. This was primarily due to a late detection of

issues where the project was approaching the end of its lifecycle and not enough time was left for

the project to build the required shared understanding by switching to an authoritative control

configuration. Additionally, the project in Case C faced significant knowledge integration

challenges during the delivery and post-deployment stages of the project, as the project expected

the COTS support team to attend all knowledge transfer and technical sessions. However, the

COTS support team was occupied with other responsibilities and the knowledge transfer lagged

behind. Additionally, a mix of behaviour- and outcome-based control also helped reduce

uncertainties and negative anticipations related to the vendor’s offshore development teams. Both

outcome- and behaviour-based controls were applied to internal support teams due to some

unfulfilled expectations experienced by the projects. This significantly reduced the level of

Page 224 of 307

confidence of the project teams in obtaining internal support in a timely fashion, and therefore

increased negative anticipation on other party’s performance.

The control relationship between the project and the business units appeared much relaxed

compared to the responsibility-oriented control portfolio. A heavy reliance on interpersonal

processes by the project team and use of active communication channels between the project and

the client were prevalent in all four Cases. Although a similar trend of interactions between the

project and business units was observed during the strategic orientation of the projects, outcome

for this chosen control style was very different as the trust-based control provided a means for

integrating tacit knowledge during development, and placed little emphasis on codification. Such

informal control configuration also reflects the high level of shared understanding between the

project team and the client, resulting from a more-collaborative exchange of information.

Effectiveness of such enabling controls, with optimal shared understanding, can result in an

efficient exchange and cooperative relationship, thus minimizing both transaction costs and

required control cost between client and the project team. This was pointed out by informant

B12IT, an infrastructure support specialist in Case B:

Smoother sprints were possible here [in the development phase] only

because we engaged them [internal support teams, change management]

when we should have. MOU also helped get things moving on our end

[project level] as waiting for one email reply [decision /approval from

client] might slow down the entire development process. … This called for

much tighter co-operation between us [host] and them [solution integrator

and vendor].

Page 225 of 307

Persuasion

Persuasion, observed during the delivery and close-out phases, appears as the fourth and the final

salient category of control orientation for all four selected cases, when control configurations took

an instrumental turn (Donaldson & Preston, 1995). This suggests an application of control

configuration to promote project objectives, including facilitating the user’s acceptance of

deliverables, negotiating future enhancements and bug fixes, finalizing support agreements,

dispute resolutions and alternative dispute resolutions with vendors, and promoting process and

culture change within the organization. As it was critical for each project to reinforce the

achievement of strategic values promised during the phase, managing and validating business

expectations and establishing fulfilment of those expectations were the central theme for all

activities at this point. This change in the project team’s intention was reflected in a change of the

control configuration of each project.

Table 37: Control Configuration at Persuasion

Orientations Project

Phases

Control Balancing Trigger

Factors

CSF and Challenges Saturated

Code/Consolidated

Code

Page 226 of 307

Persuasion

Delivery,

Close-out

Improved understanding,

Fear of unfulfilled commitments;

Missing information

Disputes with vendor,

Disagreements, Competing

internal resources;

Following Norm

Managing culture

change,

Post-implementation

evaluation,

Troubleshooting and

crisis management

Expectation fulfillment

[unfulfillment];

Reduced [or increased]

negative anticipation;

Improved shared

understanding [or gaps

in shared

understanding];

Control configuration embraced a different but interesting look at this stage compared to earlier

stages, primarily due to the project team’s attempt to establish the values of delivered releases,

including the final customized COTS, to a diverse group of internal and external stakeholders over

a short timespan.

Control configuration between the project and vendor in Cases A and B versus Cases C and D

were at two opposing ends of the control spectrum. Cases A and B adopted a more informal, trust-

based control, whereas projects in Cases C and D embraced an authoritative or formal control

relationship in respect to vendors. This discrepancy between control choices was an apparent

manifestation of challenges confronted by each project during the delivery and close-out activities.

As the delivery activities largely involve the end-user experience for the new COTS and its

technical suitability from an organizational infrastructure and integration perspective, control

configurations targeting tactical versus strategic aspects may have significantly different outcomes

for the project. At this stage, both projects under Cases A and B shifted their control from

coordinated to a trust-based configuration. This was possibly due to the high level of shared

understanding between the project team and the vendor. The ease of integration of the new COTS

system with the organization’s infrastructure and delivery of most of the functional and non-

functional requirements according to the RTM also supported the notion of fulfilled expectations

Page 227 of 307

on both the project team and the vendor side. Furthermore, for both Cases A and B, COTS vendors

had an active and sustained on-site presence to facilitate client testing of the deliverables, thus

boosting the project team’s confidence for anticipated vendor support. A positive outlook for all

three control-balancing trigger factors persuaded the project team to adopt an enabling control

style to support the maintenance of shared understanding.

A completely opposite depiction was observed in Cases C and D, where the control between the

project and vendor shifted from a trust-based to an authoritative configuration. Whereas some

authoritative aspects of control were observed for all four cases at this stage, this was primarily

due to the efforts required to perform an administrative closure for the projects where the project

team conducted a contract audit for the vendors. Aside from executing an administrative closure,

informants from Cases C and D reported several instances of missed requirements, integration

conflicts, and deviation of design. These aspects led to a higher level of unfulfilled expectations,

negative anticipation regarding future performance of vendor, and reduced shared understanding

on different dimensions of the finished product. Consequently, both projects in Cases C and D

adopted a formal control style. Similarly, an authoritative approach, consisting of a status review,

deliverable inspections, and escalations, was employed with respect to the internal IT support

teams as both projects experienced delays related their technical environments, such as network

connectivity, implementing firewall rules, storage area network (SAN) provisioning, and

vulnerability assessment (VAS) by the IT security.

Project control configurations with respect to the business units also differed among the Cases at

this stage. Although Cases A and B demonstrated a high level of shared understanding between

the project and the client, a coordinated control was applied to strengthen the existing shared

understanding and affirm expectation fulfilment. An unswerving impact of such approach was

Page 228 of 307

observed from reduced resistance towards new business processes introduced by the redesigned

COTS product. According to informant B09BU, business lead from Case B:

Joint parallel testing did a lot of good for us [Tri-agency partners]. Not

only it helped us acknowledge the fulfillment of each other’s requirements,

it also helped save a lot of time by brining everyone on the same page [i.e.,

shared understanding].

On the contrary, an informal or trust-based control style between the project team and business

users resulted in significant negative implications for projects in Cases C and D. These included

multiple complaints from business users regarding redesigned business processes, incorrect

implementation of expected requirements, technical defects resulting in IT incidents, and post-go-

live product enhancement proposals. Although an early detection of unfulfilled expectations can

be mitigated through a mix of formal behavioural and outcome-based controls by improving shared

understanding among stakeholders, both projects in Cases C and D detected these control-

balancing indicators much later in the project’s life cycle, thus shifting their control configurations

at a later stage during user-acceptance testing. This significantly reduced the reaction time and

time required to re-establish an effective shared understanding between the project and the

business.

In summary, current analysis of Cases A, B, C, and D indicate the existence of several distinct

control configurations that are directly related to each project’s success or failure. We detected

four discrete control orientations for each project: (a) strategic, (b) responsibility, (c) harmony,

Page 229 of 307

and (d) persuasion. The control configuration objective of the strategic orientation strived to align

the project with organizational objectives and manage stakeholders from a strategic standpoint.

Responsibility orientation of a control configuration, observed during the requirement and

planning and the analysis and conceptual design phases, aimed to align stakeholder’s expectations

and project’s objectives at a logical level. The third salient control orientation, harmony, was

observed during the design and architecture and the development phases. Primary objectives of

this control orientation were to align project deliverables to organizational capability and minimize

the negative impact on non-project stakeholders, thus aligning stakeholder expectations and

project objectives at a physical level. The final category of the identified control orientations,

persuasion, was observed during the delivery and the close-out phases of the projects.

Characteristics of such control orientation attempted to promote the project’s objectives related to

the user’s acceptance of deliverables, and to reinforce the achievement of “strategic values”

promised during each project’s gating phase.

6.3 Stakeholder orientations

Our analysis of Cases A, B, C, and D from a stakeholder-orientation perspective identified several

distinct stakeholder sensitivity engagement combinations directly related to each project’s

performance, as well as post-deployment stability. However, moving beyond the more prevalent

descriptive approach of stakeholder theory to its instrumental and normative core reveals a

different depiction of project and organizational intent related to stakeholder engagement

(Donaldson & Preston, 1995). Additionally, a rather stimulating set of relationships between

stakeholder orientation and management of CSF or challenges emerged from our micro-level

Page 230 of 307

analysis of the implementation processes. Delving deeper into each project’s phase-specific

stakeholder orientation, by adopting a process and tools perspective, we detected four distinct

stakeholder orientations for each project: (a) responsibility, (b) paternalism, (c) neoclassical, and

(d) strategic. A high-level summary of the four orientations is presented in Table 38.

Table 38: Comparison of Stakeholder Orientations

Stakeholder

Orientation

Project

Phases

Engagement

Level

Sensitivity

Level

Project Objective of Orientation

Responsibility

Initiation;

Requirement

and Planning

High

High

(Normative Orientation)

Encompassing involvement and

consideration of all project stakeholders

Paternalism

Conceptual

Design

Design &

Architecture

Low to

Moderate

Moderate to

High

(Normative Orientation)

Encompassing consideration with reduced

involvement of project stakeholders

Neoclassical

Development

Low to

Moderate

Low to

Moderate

(Instrumental Orientation)

Reduced engagement and sensitivity to

protect project’s financial and scope

baselines

Strategic

Pre-initiation,

Initiation

Close-out

Moderate to

High

Low to

Moderate

(Instrumental Orientation)

Encompassing engagement and reduced

influence of stakeholders to promote

project’s strategic goals

Our qualitative analysis of primary data from all four Cases indicates the adoption of different

stakeholder orientations at different phases of the project. An examination of this aspect of Cases

A, B, C, and D’s implementations indicate the influence of four distinct motivational factors: (a)

performing due diligence for internal and external stakeholders, (b) aligning capabilities and

expectations, (c) minimizing cost, and (d) upholding organizational mandate. In addition to the

Table 38, showing a summary comparison of four stakeholder orientations, Table 39 below

presents the phase-specific indicators for each orientation category.

Page 231 of 307

Table 39: Phase specific orientation analysis and code derivation

Implementation

phases

Processes, Procedures

Tools, Policies and

Roles

Selected Codes/Indicators Saturated

Code / Core

Categories

Pre-Initiation Business Gating,

Medium term plans

Alignment with Medium term plan, Protecting

Reputation and reduce reputational risk, Building

trust, Reduce operational Risk, Increase

organizational effectiveness, Compliance with

Basel III and governmental requirements

Strategic

Initiation MOU, RFI/RFP,

Procurement, RTM,

External

Communication,

Demonstration, Vendor

management, Contract

negotiations, reference

Checking, escalation

channels

Protecting Canada’s economy, Protecting

financial institutions, Ensuring financial

wellbeing of the people of the country;

Government’s ability to ensure financial stability

Responsibility

Requirement

and Planning

DeepDive-I,

Prototyping,

RTM

Requirements clarification in terms of vendor

proposed solutions; Consider all parties the

system will directly impact; capture all concerns

and requirements

Responsibility

Analysis and

Conceptual

Design

Non-functional

Requirements,

DeepDive-II,

Mapping the requirements to the functionalities

of the new system; Determining the technical

capabilities and what they should be, Propose

refinements to original requirements captured

Paternalism

Design and

Architecture

Design Authority

process, Capacity

planning, Balanced

representation

Controlling the actual design process through a

design authority group and a design approval

process, Selected representation from business

and IT; Proposing capacity extensions not

captured by RTM

Paternalism

Development Change Control,

Escalation management,

Scope Control, product

enhancement

Monitor scope Creep and scope seep leading to a

change, Fixed cost off-shore development,

identify product enhancements to help deliver

RTM in full

Neoclassical

Delivery &

Transition

Integrated Testing,

Implementation co-

ordinator,

Joint test team to enhance client/partner’s buy-in

thus enhancing project deliver success

Strategic

Page 232 of 307

probability, maintaining internal integrity and

control structures

Close-out Organizational policy

integration, Knowledge

management,

Recognition, Enhancing

PMO Capabilities

Sense of achievement and appreciation leading to

employee dedication and partner relationships;

external communication for meeting deadline of

preannounced dates with commercial partners

Strategic

Based on the evidence from all four Cases, our proposed framework for stakeholder orientation

can be further enhanced to capture optimal engagement-sensitivity combinations for each phase of

an IS implementation project. Supported by empirical evidence and observed positive impacts of

each combination, each quadrant of the model in Figure 15 is further subdivided into two segments

to represent low-to-moderate and moderate-to-high aspects of engagement and sensitivity.

Figure 15: Phase-specific stakeholder orientation.

Page 233 of 307

Furthermore, the project phases are plotted to identify the most effective stakeholder orientation

for that phase. Supporting evidence for Figure 15 and the effectiveness of particular phase-specific

stakeholder orientations are discussed below in Sections 6.3.1 through 6.3.4.

6.3.1 Responsibility dimension (normative orientation)

The sub-segments labeled A and B in the first quadrant comprise the responsibility dimension of

the project’s stakeholder orientation, where the project maintains a strong engagement level as

well as a strong sensitivity level towards the stakeholders, which can be seen as a manifestation of

corporate responsibility. High sensitivity and engagement also allow the organization to place

stakeholder interests ahead of the organization’s own interests, reflecting the foundational core of

stakeholder theory (Evan & Freeman, 2004). Recognition by the organization that all stakeholder

interests have intrinsic worth is the primary driver for this orientation. Strong sensitivity may at

times appear to be valued above organizational interest, which needs adjustment through

stakeholder engagement to determine the optimal level.

For all four selected Cases, the initiation and requirement and planning phases demonstrated a

strong responsibility orientation, where processes and tools such as memorandum of understanding

(MOU), request for information/request for proposal (RFI/RFP), procurement, requirement

traceability matrix (RTM), external communications, vendor management, contract negotiations

(fixed versus variable rate), the buy-versus-build discussion, establishing escalation channels,

DeepDive™, prototyping, and RTM were all used. Although the tools and processes utilized by

Page 234 of 307

each project indicated this common theme of responsibility toward stakeholders, each project

demonstrated a differing level of engagement sensitivity at this stage. This discrepancy among the

projects resulted from the project’s context and team composition. For example, projects in Cases

A and B involved a completely new COTS product without any prior relationship with a vendor.

In contrast, Case C benefited from an existing vendor relationship and Case D was a second

attempt for the same initiative. Regardless, a transparent and comprehensive approach to identify

and communicate with those who have legitimate stakes in the business initiative was

demonstrated in each project. This closely aligns with the argument of moral and political

philosopher Jürgen Habermas, who argued that the moral legitimacy of stakeholder engagement

can only be ensured through an engagement process that is largely or entirely free of any strategic

motivation. Although such a high level of engagement, often driven by morality, required

considerable financial resources and time. A further manifestation of this moral dimension was

established through a high level of sensitivity by each project, which clearly opposed any ulterior

motives or deceitful manner. Such honest, open, and fair engagement with stakeholders led to a

stakeholder deliberation process (Legacy, 2010) in Cases A and B. As the deliberation process is

only achievable through a face-to-face dialogue between actors (Amin & Cohendet, 2004, p. 89;

Gertler, 2003, p. 86), substantial reliance on brainstorming sessions, focus groups, and facilitated

workshops with extended stakeholder groups helped alleviate several challenges, which was

consistent with the predictions by ethical

strategists (Noland & Phillips, 2010).

Although Cases C and D attempted to engage stakeholders through an inclusive approach (Legacy,

2010) to the planning process, they failed to achieve stakeholder deliberation. Consequences for

Page 235 of 307

such an inclusive approach without the necessary deliberation aspect are reflected through missed

system requirements, lack of organizational awareness of the new COTS and introduced processes,

resource unavailability from various support teams, and costly change requests from vendors who

were not included in the project scope.

A comparison and contrast in terms of stakeholder engagement and sensitivity level among all four

Cases is presented on Table 40.

Table 40: Responsibility Dimension

Cases Project

Phases

Engagement considerations and

sensitivity controls

Engageme

nt Level

Sensitivity

Level

CSF and Challenges

Case A

Initiation;

Requirement

and Analysis

A very broad set of stakeholders

were considered and a large portion

of them were engaged by the project

team to ensure proper

representation; Organizational

reputation and reputation of the

government was a driving force;

Although a large group of

stakeholders were considered,

organization maintained focus on

business objectives

High High Implementation strategy and

timeframe; Requirement

completeness; Project

management strength; Project

cost planning and management;

Balanced team; Optimal COTS

selection; Execution of RFP;

Team morale and motivation

Case B High High

Case C Moderate Moderate

Case D Low Moderate

6.3.2 Paternalism dimension (normative orientation)

The sub-segments labeled C and D of the second quadrant comprise the paternalism dimension of

the organization’s stakeholder orientation, which is manifested through the project’s strong

sensitivity and low-to-moderate engagement of stakeholders. This can be interpreted as the

organization working in the best interests of stakeholders (moderate-to-high sensitivity) without

necessarily engaging them (i.e., a paternalistic management approach). This is supported by the

information orientation versus communication orientation aspects found in general stakeholder

Page 236 of 307

theory (Deetz, 1995), indicating deliberate choices that managers can make (Eskerod, Huemann,

& Savage, 2015).

Although the organization’s ability to maintain stakeholder sensitivity without engaging them may

be questionable, this orientation still appears to be well aligned with the normative aspect of

stakeholder theory, resembling the traditional version of corporate social responsibility.

Furthermore, stakeholder agency theory (Hill & Jones, 1992), a modification of agency theory,

attempts to explain the suppression of strategic behaviour of the firm through incentive alignment

mechanisms and various structural forms that police the implicit and explicit contracts between

managers and stakeholders. Moving to a micro level, each project’s stakeholders are a part of the

nexus of implicit or explicit contract with the project team, where the project management team

assumes a central position in the nexus of such contracts. Such conceptualization also displays a

striking resemblance to agency theory, where principals appoint or hire agents to carry out

activities on their behalf. Additionally, both stakeholder-agent and principal-agent relationships

involve an implicit or explicit contract where reconciling divergent interests is the primary goal.

Regardless of such uncanny resemblances, project management team are never appointed by the

collective body of stakeholders, which sets the stakeholder-agent perspective apart from the

principal-agent relationship. Resorting to this theoretical lens of stakeholder agency, a project’s

paternalism orientation can be argued as being in alignment with the normative core of stakeholder

theory.

For Cases A, B, C, and D, the conceptual design and planning and design and architecture phases

demonstrated a strong paternalism orientation, as processes and tools like DeepDive™, design

Page 237 of 307

authority, capacity planning, balanced representation, and RTM dominated the implementation

flow.

Aspects of paternalism were demonstrated through various processes. One high visibility process

was design authority. Comprising of two key elements—design authority group and design

approval cycle—the design authority process was introduced early in the life cycle of the project.

In addition to minimizing discrepancies between client and vendor expectations, instrumental use

of this process by the client determined what gets implemented and how it gets implemented.

Table 41: Paternalism Dimension

Cases Project

Phases

Engagement considerations

and sensitivity controls

Engagement

Level

Sensitivity

Level

CSF and Challenges

Case A Conceptual

design &

planning;

Design &

architecture

Moderate stakeholder sensitivity

with a strictly controlled

engagement process guided by

the project team to maintain

focus; Design authority process

helped to materialize this

engagement process and

sensitivity control.

Moderate Moderate

Requirement Risk;

Requirement consistency;

Requirements stability;

Roles and responsibility

conflicts;

Case B Moderate Moderate

Case C Moderate Low

Case D Low Low

6.3.3 Neoclassical dimension (instrumental orientation)

The sub-segments labeled E and F of the third quadrant comprise the neoclassical dimension of an

organization’s stakeholder orientation, which is demonstrated through the project’s low sensitivity

and low-level engagement of the stakeholders. Maintaining a high stakeholder engagement and

sensitivity does not come without a cost in an IS implementation project, as valuable project time

and resources are required to address concerns that could significantly reduce the project’s

effectiveness, as well negatively impact the scope of the deliverables. To avoid disastrous

consequences, projects may adopt an neoclassical orientation, which is an economically based

Page 238 of 307

view where the firm treats the stakeholders as instrumental, minimizing any interest in building

stakeholder relationships. Although such an economic efficiency perspective aligns well with

Friedman’s theorem (Friedman, 1970) that the only social responsibility of the firm is to increase

its profits, Friedman’s assertions have been critiqued by those who argue business ethics should

not be a merely self-interested position (Chryssides & Kaler, 1993, pp. 231–33; Weiss, 1994, pp.

76–77; Mintzberg, 1995, pp. 205, 214–15; Hoffman, 2002, pp. 716, 718–19). However, to put

Friedman’s view on business ethics into perspective and minimize such diverging views, Wagner-

Tsukamoto (2005) introduced an economic revision of Friedman’s theorem by introducing

concepts of concepts of ethical capital and active moral agency (Wagner-Tsukamoto, 2007). In

light of Friedman’s theorem, Goodpaster (1991, p. 59) argues that an instrumental stakeholder

management is limited to fiduciary responsibility, supplemented by legal compliance, a similar

argument can be applied in a project context where the organization and PMO impose the

compliance requirements for the project. Regardless, Wagner-Tsukamoto’s (Wagner-Tsukamoto,

2007) introduction of ethical capital infuses an instrumental or neoclassical stakeholder

management approach with ethical concerns that go beyond a mere compliance approach. This

allows integration between the project’s stakeholder orientation, driven by performance or

efficiency gains and predominantly in a utilitarian, consequentialist tradition, with moral

judgment. Such ethical capital, although primarily residing at the organizational level, enables

projects to adopt a neoclassical approach of stakeholder orientation, where either cost savings or

cost overruns are incurred as a result of a project’s actions and are ultimately passed on to the

organization.

Page 239 of 307

All four projects analyzed under Cases A, B, C, and D demonstrated a neoclassical orientation

during the development phase of the project, when there were very few indications of building

relationships among stakeholders, or engaging the stakeholder in an encompassing manner.

Although Cases A, B, and C contained an offshore development aspect, the development phase

was the biggest work stream for all four, and so substantial in terms of project budget. A low

stakeholder engagement at this point not only helped projects attain much-needed cost efficiency

gains but also helped avoid design disruptions, often observed in high stakeholder engagement

situations. Therefore, besides the obvious financial gain or cost-efficiency motive of a project,

organizational ethical capital enabled projects to act ethically yet maintain low engagement and

sensitivity during the development phase. Such an economic interpretation of a project’s

stakeholder orientation is essential to justify an instrumental application of stakeholder theory

without sacrificing entirely the normative core of the theory. Both Cases A and B demonstrated a

very low engagement and sensitivity combination during the development cycle, which helped

manage several CSF and challenges, such as balancing out the early cost overruns, scope creep

resulting from unchecked customer requirements, and high utilization of project resources, thus

minimizing the negative effects of shared resources. This also supported a vital success factor for

an agile development process that involved empowerment of the project team in the decision-

making process.

Effective adoption of a neoclassical orientation needs to be supported through appropriate tools

and processes as ethical capital allows the project team to choose alternatives that might translate

into higher expenditures for the organization in the long run. Projects in Cases A, B, and C decided

to implement a COTS solution through an agile development approach and engage an offshore

vendor along with an experienced solution integrator, all reflecting the organization’s objective of

Page 240 of 307

economic efficiency and risk aversion. Although all four projects experienced both anticipated and

unanticipated changes during the development phase, effective use of MOU, project governance

structure, pre-established escalation channels, and change-control processes supported the

project’s neoclassical approach by keeping stakeholder engagement at a very low level.

The change-control process itself played a critical role for RRS implementation in Case B, as the

project was based on a fixed-price contract, and any broken links between user stories, statement

of work, and RTM requirements would have caused additional changes and likely increased the

overall project delivery cost for the organization. Similarly, in Case A and Case D, the escalation-

management process had an economic orientation with low stakeholder engagement. This process

appeared vital for the project’s implementation due to the nature of the host organization, which

had a strict IT environment and access-separation policies guided by the security policy of the

organization. Deploying any new or modified code and migrating data from one environment to

another was dependent on the availability of internal COTS support staff. A few times, escalations

were necessary to reduce standby costs of the project team. Regardless of a similar organizational

context for all four projects, Cases C and D portray a slightly different stakeholder orientation in

development phase compared to Cases A and B, where the former pair of projects maintained a

moderate level of stakeholder engagement. This can be interpreted as a ripple effect of maintaining

low stakeholder engagement and sensitivity during responsibility orientation. Both Cases C and D

sought business user and vendor consultation for missing requirements, requirement conflicts, and

design discrepancies during this phase, which compelled the projects not only to engage a larger

group of stakeholders, but also change initial design decisions (i.e., higher sensitivity).

Despite a justification for the necessity and incorporation of constructs such as ethical capital, a

neoclassical approach is still open to debate and criticism. The foundational principle for this group

Page 241 of 307

of critiques is supported by a belief that self-interested behaviour should not be viewed as ethical

behaviour and good must not be done for reasons of profit. Although active moral agency, coupled

with ethical capital, builds a convincing counter argument, this is further strengthened by Jones

and Wicks’ (1999) proposed “enacted environment” construct. The neoclassical orientation of a

project may cross the normative boundary of stakeholder theory, but Jones and Wicks’ (1999)

“convergent stakeholder theory” argues that an endangered management’s relationship with a firm

can be avoided through implementation of an enacted environment that allows corporate managers

to behave morally in a stakeholder context.

Table 42: Neoclassical Dimension

Cases Project

Phases

Engagement considerations

and sensitivity controls

Engagement

Level

Sensitivity

Level

CSF and Challenges

Case A

Development

Maintains a low engagement

by a small group of

stakeholders; often due to

economic reasons. Sensitivity

control was strictly enforced

with an intent to avoid

significant scope impact

resulting from changes

introduced during

development

Low Low Project cost performance;

Development of internal

technical capabilities;

Integration and internal

process considerations;

Empowering development

team;

Case B Low Low

Case C Moderate Moderate

Case D Moderate Low

6.3.4 Strategic dimension (instrumental orientation)

The sub-segments labeled G and H of the fourth quadrant comprise the strategic dimension of an

organization’s stakeholder orientation, where the project maintains a moderate-to-high

engagement level, coupled with a moderate-to-low sensitivity level, placing the project’s strategic

goals above stakeholder interests. As stakeholders are used for furthering the goals of the project

Page 242 of 307

under a strategic orientation, an instrumental approach to stakeholder theory is a more salient

explanation compared to a normative or descriptive one.

Since Freeman’s (1984) ground-breaking contribution to stakeholder theory, a heightened interest

in the concept of the stakeholder has propelled it to a central place in strategic management

discourse (see, e.g., Mitchell, Agle, & Wood, 1997; Freeman, Harrison, & Wicks, 2007; Freeman,

Harrison, Wicks, Parmar, & De Colle, 2010). The rationale articulated by Freeman emphasized

the significance of support generated by various stakeholder groups and thereby enhanced an

organization’s chances for survival (Freeman, 1984). As Turner (1990) suggested, a project is a

“vehicle (or agency) for organizing resources,” and a conceptualized project is “an agency

established by a parent organization (the principal) to achieve specific objectives” (p. 3). This

depiction of a project as a “temporary organization” (Turner, 2003) allows it to be examined

through several organizational-level theoretical perspectives. Therefore, Freeman’s (1984)

argument on stakeholder support for organizational survival is equally applicable. This view is

further augmented if infused with resource dependency theory (Pfeffer & Salancik, 1978) to argue

that a project is dependent on the resources offered by the stakeholders to fulfill its mission and

aims. So, stakeholder engagements are of strategic significance for a project due to the fact that a

project needs contributions from them that are both financial and non-financial in nature (Morris

& Hough, 1987; Aarseth, Rolstadås, & Andersen, 2011). This view is also supported by others

who hold that strategic engagement is considered to a means of establishing social networks

between organizations and their stakeholders, thus, generating to useful resources (e.g., Kim,

Brunner, & Fitch Hauser, 2005, 2006; Swanson, 2013).

Page 243 of 307

For all four cases, the pre-initiation, close-out, and delivery phases demonstrated a strong strategic

orientation, where processes, tools, and roles (e.g., business-gating process, integrated testing,

implementation coordinating, business walk-through of deliverables, establishing perceptual

congruence among stakeholders, administrative project closure, collaborative knowledge

management) were used.

The purpose of the gating process is to review the appropriateness of an IS implementation

initiative and ensure that there is an understanding of what is to be done and why, making certain

that the investment has a good direction and is aligned with an organization’s strategic objectives.

A common theme for Cases A, B, C, and D indicated: (a) reducing operational and reputational

risk; (b) increasing corporate efficiency; and (c) increasing corporate effectiveness, as some of the

key benefits that would be realized upon completion of the project. From a project perspective, the

business opportunity gating process enabled the project to clarify the value proposition and, more

importantly, secure key stakeholders’ commitments for the initiative that were vital for the

project’s survival and success.

Strategic engagement of stakeholders was also observed during the delivery and closing phases of

the projects. For Cases A, B, and D, an integrated testing approach utilized during the project’s

delivery phase helped obtain a business buy-in for the delivered solution by affirming the client’s

expectations and increasing their confidence. The role of implementation coordinator observed in

Case B, specifically during the project delivery and close-out phases, enabled the project to

synchronize with existing organization practices of fixed-window-based change management,

configuration and asset management, SLA-based incident management supporting IT resiliency,

and disaster-recovery management to ensure business continuity by engaging appropriate internal

and external stakeholder. The combined effects of these selective engagements can be tied to

Page 244 of 307

service level and IT resiliency enrichment, which are closely aligned with both the project’s and

the organization’s strategic objectives.

Absence of such strategic stakeholder orientation could have a direct and dire consequence for the

project. For example, in Cases C and D, both projects experienced several impediments from

internal change management and IT security teams during the delivery phases, which resulted in

unexpected delays and escalations for the project. Additionally, a low perceptual congruence

between the client and project team introduced unexpected debates and challenges concerning the

deliverable, which resulted in bloated backlog of future enhancements for the COTS solution. Such

detrimental consequences have already been predicted by resource dependency theory through

concepts such as stakeholder multiplicity and stakeholder inclusiveness. This is commonly

manifested in situations with a low degree of stakeholder inclusiveness, where non-considered

stakeholders may still be in a position to harm the project through a multiplicity effect or potential

negative influence of one group of stakeholders over the other (Eskerod et al., 2015).

Since the sensitivity level is moderate to low, and the engagement level in this quadrant is high, it

is possible that an organization might pursue its strategic objectives through a purely instrumental

behaviour at the expense of a normative aspect of the stakeholder theory. Regardless, a moderate-

to-high sensitivity for this orientation often indicates significant future commitments for projects.

Table 43: Strategic Dimension

Cases Project

Phases

Engagement considerations and

sensitivity controls

Engageme

nt Level

Sensitivity

Level

CSF and Challenges

Case A

Pre-Initiation;

A moderate sensitivity towards

stakeholders and high engagement of

internal stakeholders for strategic

High Low Visioning and planning;

Project champion; Project

Management Strength; Data Case B High Low

Page 245 of 307

Case C Delivery;

Close-out

reason; A balance between the

sensitivity and engagement is

maintained but an opportunity to act in

bad faith did exist.

Additionally, getting insight from

current implementation was essential

as it would be used for future strategic

decisions. Drive was to get a

comprehensive feedback from a small

group of legitimate stakeholders

Moderate Moderate conversion and integrity;

System integration and

acceptance testing; Training

and job redesign; Post-

implementation evaluation;

Troubleshooting /crises

management; Applying

lessons-learned for each

iteration

Case D Low Moderate

In summary, current analysis of Cases A, B, C, and D from a stakeholder-orientation perspective

identified several distinct stakeholder sensitivity-engagement combinations. Each of these

stakeholder orientations showed an integral relationship with each project’s performance. The

presence of these orientations during specific project phases generated a positive impact on various

CSF for Case A and Case B. On the contrary, an absence of these stakeholder orientations during

the same project phases generated a negative impact for Case C and Case D. Therefore, a rather

stimulating set of relationships between stakeholder orientation and management of CSF or

challenges emerged from a micro-level analysis of the implementation processes. Four distinct

stakeholder orientations were identified for each project: (a) responsibility, (b) paternalism, (c)

neoclassical, and (d) strategic.

Responsibility orientation was observed during early project phases during initiation and

requirements and planning. With high stakeholder engagement and high stakeholder sensitivity,

responsibility orientation aimed to ensure an encompassing involvement and consideration of all

project stakeholders. Paternalism orientation was observed immediately following the requirement

analysis phases of the projects, during the conceptual design and design and architecture phases.

With a low to moderate engagement level and a moderate to high sensitivity level, paternalism

Page 246 of 307

aimed to ensure an encompassing consideration of stakeholder needs with reduced involvement of

project stakeholders. Neoclassical orientation was observed during each project’s execution phase

when deliverables were produced or software modules were developed by the core project team.

With a low to moderate engagement level and a low to moderate sensitivity level, a neoclassical

orientation tried to reduced stakeholder engagement and sensitivity to protect the project’s

financial baseline and scope baselines. The fourth and final stakeholder orientation observed

during the very early lifecycle for each project was classified as “strategic orientation.” Mostly

observed during pre-initiation, initiation, and the close-out phase, a strategic stakeholder

orientation sought to ensure encompassing stakeholder engagement and reduced influence of

stakeholders to promote a project’s strategic goals.

6.4 Processes, control balancing, stakeholder orientation and CSF

In this section, we integrate theory and research by combining the processes, control

configurations, and stakeholder orientations with CSF, along with theoretical support for such

integration. Before moving on to a discussion of how the elements relate to each other and to

facilitate the theoretical argument for our model, we present phase-specific control configurations,

stakeholder orientations, and CSF in Table 44.

Table 44: Control Configurations, Stakeholder Orientations, and CSF

Project Phases Effective Control

Configuration

and Intent

Effective

Stakeholder orientation

Related CSF/CFF (Combined)

Pre-Initiation

Strategic (Trust based control to

support ‘good faith’

engagement

Strategic (High

Engagement and Moderate

to Low Sensitivity)

Visioning and planning,

Project champion,

Project Management Strength

Page 247 of 307

Initiation Strategic (Authoritative

Control) Responsibility

(High Engagement and

High Sensitivity)

Balanced team;

Optimal COTS selection;

Execution of RFP;

Empowered decision makers;

Team morale and motivation

Requirement and

Planning Responsibility

(Authoritative Control

and Coordinated Control)

Responsibility

(High Engagement and

High Sensitivity)

Requirement completeness;

Engaging internal IT support teams

(Both infrastructure and COTS

support)

Analysis and

Conceptual

Design

Responsibility



Paternalism (Moderate Engagement

and Moderate Sensitivity)

Requirement consistency;

Requirements stability

Design and

Architecture Responsibility



Paternalism (Moderate Engagement

and Moderate Sensitivity)

Roles and Responsibility

Clarification

Development Harmony

Neoclassical Low (Engagement and

Low Sensitivity)

Development of internal technical

capabilities

Integration and internal process

considerations (i.e. change

management, security clearance

etc.)

Delivery Persuasion

Strategic (High


to Low Sensitivity)

Data conversion and integrity,

System integration and acceptance

testing,

Training and job redesign,

Close-out Persuasion

Strategic (High


to Low Sensitivity)

Post-implementation evaluation;

Troubleshooting/crises

management,

Applying lessons-learned for each

iteration

Enterprise COTS implementation is a complex, intensive, and dynamic activity that requires close

coordination and cooperation among a diverse group of stakeholders. For large-scale

implementation, as in this study’s Cases, this often involves internal and external end-users,

Page 248 of 307

partners, public, government, sponsors, organizational IT leaders, external partners, vendors (in

the form of COTS providers and solution integrators), suppliers, internal IT support teams, and the

project team. Cognizant engagement and management of such a diverse group of stakeholders is

absolutely critical for any project’s success, for stakeholders not only exert influence on a project’s

operations, they also provide much-needed resources and support (Verbeke & Tung, 2013).

Whereas a resource-based view is useful for explaining the process of resource accumulation and

exploitation (Barney, 1986), the most effective and efficient access and usage of these resources

is demonstrated by a stakeholder approach through a deliberate stakeholder management (Verbeke

& Tung, 2013).

Although the significance of stakeholder engagement is well established in IS ethics and strategy

literature, the debate concerning criteria for morally acceptable engagement is still quite alive

(Swanson, 2013), as characterization of stakeholder engagement ultimately depends on

management style. Aside from the possibility of engaging stakeholders in a deceitful manner to

fulfill ulterior motives, stakeholder engagement is typically understood as “practices the

organization undertakes to involve stakeholders in a positive manner in organizational activities”

(Greenwood, 2007). Accepting this common understanding, the analysis here of stakeholder

orientations for different phases of each project indicates that the recommended method is an

integrative approach (Austin, 2003) that facilitates strategic collaboration between the parties. This

approach is also supported by previous researchers who also favoured an integrated and dynamic

framework to facilitate strategic stakeholder engagement (Ostrander, 2004; Schultz & Hatch,

2005).

Page 249 of 307

Such a strategic approach may appear in direct contradiction with the normative core of

stakeholder theory, as proponents of a normative orientation often argue that “good must not be

done for reasons of profits” (Friedman 1970/1993; Chryssides & Kaler, 1993, p. 231). Yet a

combination of both normative and instrumental approaches is necessary for enterprise IS

implementation projects, as demonstrated by Cases A, B, C, and D. Justification for such an

approach is also well founded in both IS and stakeholder literature. Although pressures from

stakeholders towards homogeneity via isomorphism is more relevant at the organizational level

(Verbeke & Tung, 2013), handling divergent interests of various stakeholder groups needs a

sensitive and cognizant management approach at the project level. Such divergent interests are

particularly problematic in the presence of power differentials, as agency theorists have

emphasized (Hill & Jones, 1992). Setting aside vendors’ obvious profit-maximizing motive, the

cornerstone of agency theory assumes a diverging interest between the principal and the agent that

requires the agent establish appropriate incentives and monitoring to limit opportunistic behaviour

(Demsetz, 1983; Fama, 1980; Fama & Jensen, 1983). Such divergence can best be explained

through a critical theory perspective of stakeholder management, where a pragmatic discourse

seeking effectiveness (i.e., vendor), an ethical-existential discourse seeking goodness (i.e., project

management team, users), and a moral discourse seeking correctness or justice (i.e., organizational

leadership) all coexist in a stakeholder-engagement scenario (Hill & Jones, 1992).

Regardless of general advice to assess stakeholder engagement from an opportunity-cost

perspective (Verbeke & Tung, 2013) and to introduce new constructs like ethical capital (Wagner-

Tsukamoto, 2007), both implicit and explicit contracts in a stakeholder-agent scenario require

governance structures and processes to enforce them (Hill & Jones, 1992). Furthermore, to

Page 250 of 307

effectively leverage resources embedded in a social network resulting from stakeholder

engagements, suitable control configurations are indispensable. In the context of ISD or ES

implementation, the critical role of control in integrating complementary stakeholder capabilities

has already been affirmed in control literature (Kirsh, 1997). The salience of this aspect is even

more evident in a COTS implementation involving an offshore team. As Clark et al. (1998) stated,

“the truly critical success factors associated with successful outsourcing are those associated with

vendor governance.” Therefore, as optimal stakeholder engagement is essential for the survival

and success of any project, keen sensitivity is also necessary to mitigate any threat from divergent

objectives and opportunistic stakeholder behaviours. This is evident from the cases analyzed for

this research, where distinct stakeholder orientation and control configurations were adopted to

achieve desired project outcomes.

As explained earlier, the application of stakeholder engagement in IS projects is often limited to

mere inclusion of stakeholders in the project’s initiation process, but without much consideration

of its implication. This often fails to achieve required stakeholder deliberation, inclusive dialogue,

and cross-examination of ideas during the planning process or subsequent stages of the project

(Cohen, 1989; Gutmann & Thompson, 2004; Smith, 2003; Young, 2000). Several processes

utilized in Cases A, B, and C, such as user stories, requirement workshops, process-design

workshops, brainstorming sessions, and focus groups, helped facilitate the stakeholder-

deliberation process. Moreover, the use of tools and project management processes, such as RTM-

specific requirements, risk ownership, response ownership, RACIN categorization of project

participants, communication-control plans, and escalation plans, enabled each project to maintain

a desired level of stakeholder engagement. Paralleling engagement management, Cases A and B

Page 251 of 307

used explicit processes to adjust sensitivity levels. A few such processes include design authority,

internal change-control meetings, and backlog prioritization process. These allowed the project

teams to determine activity focus and adjust the project scope to deliver maximum organizational

value with high efficiency. Validity of this claim is further supported by the challenges experienced

in Cases C and D, where incomplete and inconsistent requirements and client-project

disagreements resulted in significant user dissatisfaction. Primary data indicate that a low level of

stakeholder engagement and sensitivity were directly responsible for such undesired outcomes.

Several processes enabled the project teams to maintain high engagement with stakeholders and

still control sensitivity, but some processes adopted in Cases B and D also allowed the projects to,

at times, reduce engagement and achieve both cost and performance efficiency. For example, Case

D used a self-selection process to establish roles and responsibilities that significantly reduced

roles-and-responsibility-related challenges and empowered the team, thus improving team morale

and motivation. According to a project manager in Case D, the benefits of such a process were

evident:

Each member was asked if they wanted to be part of the team with an understanding of the

commitment that will be expected and the approach that will be used. … Each team member

understood they would be called upon to help each other out. … For example, when testing

was behind, respective team members from both the business and ITS jumped-in to keep

the project on track. Business resources were there for business knowledge, testing,

requirements, documentation, and training.

Page 252 of 307

Several other processes used in Cases A, B, and D, such as environment consolidation, were more

technical in nature and permitted projects to maintain low stakeholder engagement, yet still

achieved an efficiency gain. As informant D03PT, a senior developer from Case D states:

I think processes, procedures, plans all need to be worked and reworked throughout the

entire life of the project. In our [name of the project] project we started with a strategic

plan that produced numerous processes and procedures. We kept rebuilding, redesigning,

and reimagining these the entire life of the project. One repeatable process, that I recall,

was redesigned several times. In the beginning, it took two weeks to complete this process,

but by the end of the project it took 2 minutes! … with more accurate results, saving team

members well over 10 weeks of work as well as required support from five other teams to

do the same job.

Similar to processes, control configurations and targeted adjustments of control portfolios also

exerted a significant influence on a project’s stakeholder orientation and on CSF. Building upon

the earlier discussion of diverging stakeholder interests, the usability of a control perspective in

relation to stakeholder orientation is also supported by the primary data from our selected Cases.

Divergence of stakeholder interests is primarily driven by different utility functions of various

groups, where satisfying one group’s interests would reduce the resources available to satisfy other

groups’ interests, as postulated by stakeholder-agency theory (Hill & Jones, 1992). This was a

potential scenario in Cases A and B and an actual scenario in Cases C and D, where incorporating

late requirements from business users would have reduced the vendor profit and/or reduced the

project’s cost efficiency. Such inefficient resource allocation resulting from uncorrected

stakeholder divergence is also referred to as utility loss (Hill & Jones, 1992). Therefore, the

Page 253 of 307

significant role of incentive, monitoring, and enforcement aligning of diverging stakeholder

interests to minimize a project’s utility loss clearly indicates a close relationship between control

configuration, stakeholder orientation, and the challenges faced by the project team.

To substantiate our claim regarding the influence of appropriate control configuration on CSF and

stakeholder orientation, a user satisfaction and risk perspective of IS implementation success is

essential. A close correlation between these two dimensions is observed in DeLone and McLean’s

(1992) six categories of success measurements—system quality, information quality, use, user

satisfaction, individual impact, and organizational impact—and six dimensions of IS

implementations risks—team, organizational environment, requirements, planning and control,

user, and project complexity (Wallance, 1999).

Our analysis of CSF in all four Cases clearly aligns with earlier categorizations, where absence

could be perceived as risk and presence could be used as measurement criteria for implementation

success. Despite a striking alignment with existing knowledge base, a stakeholder perspective

introduces a direct or indirect people aspect behind most CSF, highlighting the implication of

control configurations. For example, in Cases A and B, a trust-based control followed by an

authoritative control during early phases of the projects helped establish projects and ground rules,

secure commitments from organizational leaders and users, clarify expectations from the project

team, and develop a shared understanding among partners. This, in turn, reduced several categories

of risk in both projects in the dimensions of team, user, organizational environment, and planning.

Although the effects of such authoritative control, containing mostly behavioural controls, aimed

to improve the engagement level and coordination among team members, the levels of shared

Page 254 of 307

understanding and expectations were still very low due to minimal opportunity of engagement

from the business users. Additionally, in the absence of a formal contract and time-allocation

process, a procedural control was not very effective during the project’s early phases. This

motivated the projects in Cases A and B to retain an authoritative approach by employing both

outcome-based and behavioural controls for the vendor and extended IT support team to ensure

required cooperation, commitment, and shared understanding. This had clear positive effects for

establishing a common understanding, forming project norms, and minimizing negative

anticipations regarding future performance. Yet, once the desired outcome was achieved, both

projects shifted to coordinated and trust-based control configuration toward the end of the project’s

life cycle, and negative changes in any of the control-balancing trigger factors led the project to

adopt an authoritative approach of control.

A trust-based control portfolio was observed in all four Cases, although not in a harmonized

fashion. Adoption decision for control styles was influenced by several different factors, both

generalizable and idiosyncratic to the project. Such control was reflected in the project’s reduced

communications, status meetings, progress updates, managerial reviews, audits, and other

administrative activities. Such a control was adopted in Cases A and B due to higher levels of

shared understanding, positive performance by the controlee (i.e., vendor and IT support teams),

and a matured level of self and clan controls. A trust-base control is particularly important in an

agile development environment with diffuse stakeholder groups (Hill & Jones, 1992), and where

clan formation is possible and empowerment of the team necessary for effective task executions.

In both Cases A and B, a trust-based control demonstrated a positive influence by motivating

project participants and reducing information asymmetry. This was reflected in behavioural

Page 255 of 307

changes such as the vendor assuming a higher level of responsibility than required and proactive

communications on the client side. According to informant A02PT, the technical lead for Case A:

Empowering people is something that always leads to surprising results. Once a team

realizes they are empowered and protected, creativity flourishes, tensions reduce, and an

amazing sense of ownership and pride occurs, usually resulting in some highly creative,

innovative solutions. Given the right people for the job, the right tools for the job and an

environment to be creative, some out of the box solutions were found that allowed us to

meet all of our client’s design requirements.

To the contrary, improper application of a trust-based control can severely harm the

implementation process and overall project objective. Projects in Cases C and D often applied

trust-based controls, while Cases A and B opted for an authoritative or coordinated configuration.

Although an inexperienced project manager and a lack of technical aptitude among the core project

team in respect to the new COTS product were the driving forces behind such decisions, a few of

the negative consequences of an incorrect control configuration included delayed delivery of work

packages, inconsistent and incomplete requirements, poor functional quality, and increased budget

due to unexpected changes. In Cases C and D, this further reduced shared understanding and

increased negative anticipation about future performance amongst the project’s stakeholders, thus

resulting in a high level of user dissatisfaction, unexpected process changes, and post-go-live

challenges and crisis management.

Besides two salient control configurations, a third category of control configuration, coordinated

control (Gregory et al., 2014), was also observed in all Cases. Although Henderson and Lee (1992)

Page 256 of 307

indicated that a joint application of both formal and informal control in IS implementation results

in higher performance, Kirsch (1997) advocated a portfolio of controls to address the challenge of

diffuse stakeholder groups with diverse knowledge and skills. Complementing Kirsch’s (1997)

argument for a control portfolio, Soh et al. (2011) introduced the concept of controllee-initiated

controls, as well as called for separate controls targeted at each stakeholder group to address

distinctive attributes of each element in a multi-stakeholder IS implementation. The coordinated

control configuration observed in Cases A, B, C, and D closely supports these earlier arguments.

This control primarily supported the concept of value co-production by stakeholders (Zeithaml,

1981; Parasuraman et al., 1985; Bettencourt et al., 2002), where poor performance by one

stakeholder group has negative consequences for others. While contingent upon a strong mutual

commitment (Gregory et al., 2014), a coordinated control style significantly reduced requirement

conflicts during early stages and disputes during audit and inspections in the late stages. Besides

informal meetings, Cases A, B, and C arranged site visits by offshore vendors that helped clarify

shared understating of technical issues and client work culture, and established a vendor-proposed

coordination process to overcome negative impacts of time zone differences between Canada and

Europe. A bi-directional control is also observed in Cases A and B, where the client initiated

regular discussion sessions with the project’s technical team and the vendor to redesign business

processes, validate proposed requirements, and understand the technical capabilities of the new

COTS solution. This eventually resulted in intense cooperation among the parties, as well as

enhanced sharing of knowledge among stakeholder groups. Besides the obvious impact of better

understanding and effective work coordination among stakeholders, a coordinated control

configuration significantly reduced both requirement risks and user risks, thus leading to better

Page 257 of 307

process design, effective training plans, and job redesign, as well as lower resistance in acceptable

deliverables by establishing perceptual congruence among stakeholders.

In summary, primary data and codes derived from the same suggest the direct influence of control

configuration and various processes on a project’s critical success factors. Furthermore, both

control configuration and various processes influenced stakeholder orientation within a project by

adjusting stakeholder engagement and sensitivity. A project’s stakeholder orientation, in turn, had

a direct moderating relationship with various CSF in the COTS implementation project. These

findings directly support our proposed relationships among control configurations, stakeholder

orientations, processes, and CSF, as presented in Figure 16.

Figure 16. Relationship between process, control configuration, stakeholder orientation and

CSF

Additionally, the effect of a control configuration on stakeholder orientation and CSF are largely

dependent on context-specific factors of the project. Such contextual factors also dictate the nature

of a control-configuration adjustment, leading to a dynamic control configuration scenario

throughout the project’s lifecycle. This aspect is depicted in Figure 17 below.

Processes

Stakeholder Orientation

Control Configuration

COTS Implementation

Outcome CSF

Page 258 of 307

Figure 17. The dynamic nature of a control configuration.

In summary, synthesizing earlier analysis of implementation processes, control configurations and

stakeholder orientations for selected Cases affirmed a close relationship among three vital

constructs (process, stakeholder orientation, and control configuration) of an enterprise COTS

implementation project and the project’s CSF. Although both implementation-related processes

and control configurations influenced the nature of stakeholder orientation for a COTS

implementation project, all three aspects of a project were found to exert considerable influence

on CSF for the project. Additionally, a dynamic nature of the control configuration indicated the

necessity for a control configuration shift based on the presence of three distinct trigger factors:

level of shared understanding, negative anticipation, and deviation of expectation.

Page 259 of 307

7.0 Conclusion

Chapter 7 contains five sections. In Section 7.1, we provide the study’s conclusions. In Section

7.2, we summarize the answers to our research questions. In 7.3 and 7.4, we discuss the theoretical

contributions and impacts for practice, respectively. Section 7.5 identifies various limitations of

this research. Finally, in Section 7.6, we provide suggestions for future research.

7.1 Conclusion

We set out to explore an existing IS phenomenon of enterprise COTS implementation through a

novel perspective of process, control, stakeholder engagement, and critical success factors. Our

objective was to identify the relationships among processes, control configuration, and stakeholder

engagement, and how they influence CSF for an enterprise COTS implementation.

Consequently, we drew from control theory, control-balancing theory, stakeholder theory,

stakeholder agency theory, IS-related CSF literature, and IS-implementation literature to develop

our argument. In doing so, we have proposed a new and vital construct, stakeholder orientation,

for COTS-implementation projects. Initially, we identified a micro-level model for enterprise

COTS implementation, which facilitated identification of key processes and their nature in an IS

project. Subsequently, we employed control theory and theory of control balancing to capture and

validate the dynamic nature of control configurations in an IS project. We also leveraged

stakeholder theory and stakeholder agency theory to capture the dynamic stakeholder orientation

in a IS implementation project. Finally, through analysis and synthesis, we established vital

relationships among processes, control configurations, stakeholder orientations, and CSF where

proper management of CSF leads to a successful COTS implementation project outcome.

Page 260 of 307

7.2 Summary of answers to Research Questions

In this section, we summarize the answers to the six research questions investigated through the

current research.

7.2.1 What is an activity- and process-based model for enterprise COTS

implementation?

Enterprise systems, in the form of commercial software packages that enable organization-wide

data- and business-process integration, has received considerable attention in IS literature. Even

early researchers like Markus and Tanis (2000) have affirmed the existence of wide range of

options for implementation and ongoing operations for such systems. A rich body of literature has

also investigated the traditional software development process. However, implementation

processes for COTS-based enterprise systems have been argued to be substantially different from

traditional software development (Morisio et al., 2002). Although intersecting with the research

on ERP implementation efforts, we have pointed out the possibility of considerable variations in

implementation processes for non-generic or well-known COTS. Additionally, most

implementation process research for well-known COTS (i.e., ERP, CRM) have primarily focused

on macro-level categorization of the process. Motivated by this, we have identified a macro-level

COTS implementation process (Figure 12) by analyzing four different enterprise COTS

implementations. Significance of this macro-level model is summarized by the Table 45 below.

Page 261 of 307

Table 45: Process-based (macro level) model for COTS implementation

Purpose Identified phases Characteristics of

the model

Significance or Contribution

Purpose of a macro level or

process level COTS

implementation model is to

capture the logical flow of

implementation activities,

logical process groups and

identify significant

processes influencing the

project CSF

Pre-initiation,

Initiation,

Requirement and

planning,

Analysis and

conceptual design,

Design and

architecture,

Development,

Delivery,

Close-out.

Logical Process

groups

Information flow

among the

process groups

Implementation

related processes

Well-defined Project

boundaries

Capturing the nature of the

implementation practices

Integration of processes and

tools with implementation

phases

Foundation for a activity

based or micro level

implementation model

Enabled by this macro-level model, we have further analyzed the responsible processes, tools and

information flow to develop a micro-level or activity based model for enterprise COTS

implementation (Figure 13). The significance of this micro-level model is summarized by the

Table 46 below.

Table 46: Activity-based (micro level) model for COTS implementation

Purpose Identified phases Characteristics of the

model

Significance or

Contribution

The purpose of a macro

level COTS

implementation model is

to identify the key

activities in relation to the

stakeholders involved;

This provides the

foundation for both

stakeholder orientation

and control configuration

decisions

Procurement &

RFI/RFP

Requirement & Planning


Development


Logical activity

grouping for each

phase

Stakeholder

engagement needs

Stakeholder

sensitivity control

Well-defined logical

phases

Depiction of

stakeholder

engagement

requirements and

points

Control directions

Page 262 of 307

Each of the four selected COTS products were intended for very different purposes, such as

administration of money market instruments, regulation of country-wide financial transactions,

control and administration of Canada’s currency circulation, and organization-wide knowledge

management. In addition to the identification of a generic implementation process for enterprise

COTS, a micro-level model also enabled us to identify the critical activities and project-specific

processes necessary to substantiate their value. The value aspect of identified processes and tools

were established by outlining their influence and relationship with each project’s stakeholder

engagement (Figure 16) and CSF.

7.2.2 How can organizational processes and tools contribute to COTS implementation

success?

COTS implementation success, well situated with the domain of IS implementation success, is no

longer an emerging research phenomenon in IS literature. Although this specific category of IS

has not received significant attention from researchers, practitioner involvement with COTS

implementation and IT investments under this category, from a process and tools perspective, is

rapidly growing. Motivated by this knowledge gap and industry trend, we aimed to investigate the

relationship between processes and tools and COTS implementation success. In answering this

question, we augmented the IS implementation-specific CSF from the literature with experiences

reported by informants in our analyzed Cases. We have found that the organizational tools and

processes introduced by various stakeholder groups have a direct relationship with the reported

CSF for each project. This relationship is summarized in Table 47 below.

Page 263 of 307

Table 47: Processes and tools influencing CSF

Name Category Impact CSF

RFP/RFI Process High engagement of stakeholder, and high

sensitivity towards stakeholder; Strategic alignment

of the project investment; Foundational process for

requirement and analysis.

Visioning and planning;

Build a business case;

Selection of appropriate

COTS product.

RACIN Tool Balancing the stakeholder engagement and

stakeholder sensitivity throughout the project

lifecycle; Stakeholder-role clarity; Facilitate

authoritative and co-ordinated controls

Project champion;

Change management;

Project management

RTM Tool Controlling stakeholder engagement and sensitivity

during early lifecycle; Managing stakeholder

expectation throughout the project’s

implementation; Validating deliverable scope

during the close-out phase

Client consultation;

Project management and

expectation management

Design

Authority

Process Preventing design changes and design conflicts;

Promoting high stakeholder engagement with

moderate sensitivity

Implementation strategy

and timeframe;

Empowered decision

makers; Team morale

and motivation; IT

infrastructure; Data

conversion and integrity

Deep Dive Process Generalized innovative problem-solving during

early phases of the projects; protecting integrity of

the requirements; minimizing conflicts

Legacy system

consideration; Client

consultation,

Joint Parallel

Testing

Process Minimizing scope validation delays and scope

related disagreement among the stakeholders;

enhancing shared understanding

Client consultation;

System testing

A positive influence on CSF led to a smoother COTS implementation process and an overall

implementation success. In parallel, we also captured scenarios where an inadequate management

of CSF resulted in client dissatisfactions, poor COTS quality or deliverables, and poor usability of

the delivered system.

Page 264 of 307

7.2.3 What is the nature of control configuration in a multi-partner COTS

implementation and what are the factors responsible for the application of control

balancing?

A seminal view of control traces its origin to Weber (1947), who defined control as a process of

creating and monitoring rules through hierarchical authority (Henderson & Lee, 1992).

Subsequent, researchers further contributed to the control literature by categorizing control in two

broader dimensions, formal and informal (Crisp, 2002). Both categories conceptualize control in

a dyadic sense through a controller and controllee relationship (Kirsch, 2004). Recent

contributions in this domain introduced the concept of control portfolio (Gregory et al., 2014) and

controlee-enacted controls in multi-stakeholder scenarios. Building on this perspective, we

attempted to investigate the nature of control configurations—more specifically, the dynamic

nature of control configuration and the underlying drivers for targeted control adjustment.

Our analysis of four COTS implementation projects indicates the existence of control

adjustment during various phases of the project. Based on evidence reported by all

informants, we identified four distinct control orientations for each project: (a) strategic,

(b) responsibility, (c) harmony, and (d) persuasion. Furthermore, we also identified three

distinct trigger factors: (a) shared understanding, (b) negative anticipation, and (c)

deviation of expectations. These three factors can be classified into two major categories:

(a) re-active control balancing, and (b) pro-active control balancing.

Page 265 of 307

7.2.4 How can CSF and other challenges be successfully managed through optimal


Empirical research in the IS domain has identified a large number of factors contributing to the

success or failure of enterprise systems implementations, including well-known COTS such as

ERP and CRM. This is evident from our review of CSF literature in Chapter 2. Although these

factors reside in a technical context, they are often unrelated to the technology itself (i.e., the

enterprise or COTS system being implemented). Rather, they often relate to politics, motivation,

and conflict resulting from multi-stakeholder interactions and relationships due to divergent

priorities and goals (Markus & Benjamin, 1996; Hartwick & Barki, 1994; Roby et al., 1989).

Despite the risk of introducing potential negative factors that may cause an IS implementation

project to fail, engaging all legitimate stakeholders in a meaningful way is indispensable if we

harness the complementary skills, tangible resources, and knowledge possessed by them.

Additionally, IS implementation success largely depends on the attitude and perception of the

primary stakeholder of the project. Therefore, a significant portion of the CSF and challenges in

IS implementation context are found to be related to people and processes involved with the

project.

Our findings from Cases A, B, C, and D, as elaborated at the end of Section 5.1.2, 5.2.2, 5.3.2,

5.4.2, and 6.2, indicate that changing control configurations of the project, depending on certain

existing factors, facilitate the development of shared understanding among stakeholders and

manage stakeholders’ perceptions. These, in turn, result in higher cooperation, diligent

contributions, reduced activity cost, innovative solutions, high quality and timely delivery, and

higher user acceptance. All these factors, in one way or another, were found to be related to the

salient CSF and challenges for the analyzed Cases.

Page 266 of 307

7.2.5 What is the nature of stakeholder orientation during a COTS implementation for a

public sector IS implementation?

As it is imperative that effective management of stakeholder engagement and relationship, so as

to elicit their cooperation and contribution, is critical for an IS project’s success and survival

(Kirsch et al., 2002; Clark et al., 1997), maintaining a static approach to stakeholder management

can also be detrimental. Reasons for such conjecture are largely driven by the fact that stakeholder

goals and priorities are often divergent (Markus & Benjamin, 1996; Hartwick & Barki, 1994; Roby

et al., 1989). Therefore, we proposed at the onset of this research the need for optimal stakeholder

engagement, coupled with a mechanism to regulate stakeholder influence on a project’s activities

and objectives. Driven by this motive, our inquiry on this aspect of stakeholder engagement found

a dynamic pattern where stakeholder engagement varied at different phases of the project. This

was supported by most informants in Cases A, B, C, and D. This pattern indicates significant

stakeholder engagement during early phases of a project, gradually subsiding during the design

and development phase, then increasing towards the end of the project’s life cycle. Pattern coding

of the primary data indicated several underlying factors responsible for such engagement

variations throughout the life cycle. In addition, processes and mechanisms were also observed to

regulate the effects of stakeholder engagement in a project’s objective. Combining the aspect of

stakeholder engagement with stakeholder sensitivity, we developed a new construct, stakeholder

orientation, to categorize the engagement variations for our project. This allowed us to identify

four distinct stakeholder orientation for each project: (a) strategic, (b) responsibility, (c)

paternalism, and (d) neoclassical. These four orientations were mapped in a grid, with each

occupying a dedicated quadrant. Furthermore, based on the stakeholder engagement framework

Page 267 of 307

developed by Greenwood (2007), we have indicated the possibility of sub-segments within each

quadrant of our model depending on the intensity of stakeholder engagement and sensitivity. This

is described in Table 48 below.

Table 48: Stakeholder Engagement and Stakeholder Sensitivity

Segments/

Quadrant

Title Stakeholder

Engagement

Stakeholder

Sensitivity

Relationship between

stakeholder engagement

and stakeholder

sensitivity

A

(Responsibility)

Ethical

altruism

Strong engagement of

stakeholders

Acts in the interest of

primary stakeholders;

Sensitivity is moderate

to high in the sense

that a few legitimate

stakeholders might be

left out

Increasing stakeholder

sensitivity with strong

engagement measure driven

by corporate social

responsibility; organization

should not totally sacrifice

own interests to help others'

interests

B

(Responsibility)

Anti-

capitalism

Moderate to high

engagement of

stakeholders as

determined by the

organization

Considers the interest

of all stakeholder

Including illegitimate

Participation of so many

(including illegitimate)

stakeholders that the

purpose of the firm may get

compromised

C

(Paternalism)

Limited

Paternalism

Low to moderate


determined by the

company


a broad group of

stakeholders

determined by the

organization

Acting in the perceived

interest of the stakeholders

with limited consultation

D

(Paternalism)

Strong

Paternalism

No or little stakeholder

Engagement


Legitimate stakeholder

as determined by the

organization

Acting in the perceived

interest of the stakeholders

without consultation to the

point of interference and

reduction of liberty.

E

(Neoclassic)

Market Low to moderate


due to economic

reasons

Does not act in the

interest of legitimate

stakeholder

Controlled and highly

focused engagement to

further the interests of the

owners. Organization and

stakeholders as economic

entities

F

(Neoclassic)

Illegal (outside

the boundary

of the law or

No stakeholder

engagement as

determined by agents in

control of the company

Does not act in the

interest of legitimate

stakeholder;

Organizations act in their or

principal’s interests either

illegally or outside moral

minimum norms. Could

Page 268 of 307

7.2.6 How can CSFs and other challenges be successfully managed by the project

through optimal stakeholder engagement?

As indicated in section 7.2.4, stakeholder engagement is essential for a project’s survival and

success. At the same time, the possibility of opportunistic stakeholder behaviour is a realistic threat

to a project’s objective. Additionally, traditional control mechanisms are often ineffective in a

complex and outsourced IS implementation. Therefore, a stakeholder orientation is necessary to

manage both the engagement and sensitivity of stakeholders. In addition to identification of four

distinct stakeholder orientations, we also provided theoretical justifications to support their

alignment with key tenets of stakeholder theory. Adoption of different stakeholder orientations at

different phases of the project demonstrates a profound impact on the CSF and implementation

challenges. For example, a responsibility approach during early project phases substantially

reduced requirement and user risks for Cases A and B. An opposing effect was observed for Cases

C and D, which did not demonstrate sufficient stakeholder engagement and sensitivity.

Appropriate stakeholder orientation also supported the concept of stakeholder integration (Plaza-

accepted

custom)

Treats stakeholders as

purely instrumental

include fraud, theft, and

abuse of human rights

G

(Strategic)

Reputation/

Legitimacy

Engaging small groups

of legitimate

stakeholders to further

shareholder interests.

Stakeholders are

selected primarily

guided by the strategic

objectives of the

organization with a

low to moderate

sensitivity

Narrow focus in terms of

stakeholder sensitivity and

moderate on engagement;

Engaging stakeholders

enhances strategic

alignment, reputation and

legitimacy with

stakeholders.

H

(Strategic)

Irresponsibility

(bad faith)

Excessive engagement

without accountability

or responsibility

towards stakeholders

Appears to act in the

interest of only

influential stakeholder

s

Engaging with stakeholders

under deceptive conditions,

acting ‘‘as if’’ the aim is to

meet stakeholders’

interests.

Page 269 of 307

U´beda, Burgos-Jime´nez, & Moreno, 2010) in all four Cases via integration of stakeholders’

demands, knowledge absorption, and improved performance. This, in turn, positively influenced

several other risk dimensions for the projects, such as complexity risk and planning and control

risk.

To summarize, our findings from Cases A, B, C, and D, as elaborated at the end of Section 5.1.3,

5.2.3, 5.3.3, 5.4.3, and 6.3, indicate that the choice of various tools and processes enables the

project team to control both stakeholder engagement and stakeholder sensitivity. Opposing

stakeholder orientation between Cases A and B, and Cases C and D can be correlated to the number

of challenges faced by each project. This gives rise to the concept of optimal stakeholder

engagement. Additionally, an optimal stakeholder engagement also helps ensure a positive

influence on pertinent CSF for the project as well as mitigate any challenges.

7.3 Implications for theory

The current research extends the knowledge of enterprise COTS and IS implementations in several

major ways.

First, the current research captures a successful implementation process for enterprise COTS, with

a micro-level or task-oriented focus. This task-oriented focus led to the discovery of several

beneficial processes, tools, and activities that can facilitate the further study of stakeholder

engagement and application of control theory in this context. A literature review related to this

context also indicates that a vertical analysis of an end-to-end COTS or IS implementation has not

received sufficient attention. Therefore, the implementation model identified through the current

research offers a significant contribution in the IS implementation domain.

Page 270 of 307

Second, the end goal of a public sector IS implementation is different compared to private sector

IS implementation. Due to divergent mission objectives and stakeholder groups, value

maximization often takes precedence over profit maximization for most government

organizations. The interpretation of value also widely varies between stakeholders (Flak, 2005).

Scholl (2001) has studied major e-government initiatives undertaken by public sector

organizations using stakeholder theory. However, the scope of Scholl’s (2001) research does not

allow for a detailed analysis of stakeholder theory in relation to the characteristics of public sector

organizations. Through a case-based grounded theory approach, the current research identified the

major stakeholder orientations in public sector IS implementation, including the manifestation of

three major aspects of stakeholder theory identified by Donaldson and Preston (1995): normative,

descriptive, and instrumental. Together, these can be considered theory extension efforts by the

current research.

Third, findings by Ballejos and Montagna (2011) clearly indicate the significance of the

stakeholder perspective when considering a multi-party joint IS implementation effort, which is

even more salient for government organizations due to a larger stakeholder group, as well as

different ethical considerations and value-maximization objectives (Flak, 2005). Ballejos and

Montagna (2011) have also found that engagement alone is not adequate for ensuring sufficient

stakeholder consideration in an IS design process. Rather, a level of consistency in stakeholder

representation is necessary to avoid underachievement of IS project goals due to poorly conceived

procedures. Yet, a majority of stakeholder-related research in the IS domain focuses on the concept

of “organizational responsibility towards the stakeholder,” which directly aligns with the

normative core of stakeholder theory. The current study complements this conventional approach

towards stakeholder research in a significant way, both by considering the reality that stakeholders

Page 271 of 307

can also impact the organization or project, and by focusing on the concept of stakeholder

orientation, which is manifested through a combination of stakeholder engagement and

stakeholder sensitivity. Furthermore, capturing the dynamic nature of stakeholder orientation in a

complex IS implementation project through a well-defined framework will facilitate understanding

related to the adoption of normative and instrumental aspects of stakeholder theory. This will also

help us understand Wicks’ (1999) convergent stakeholder theory, according to which the

instrumental usage of stakeholder theory does not require sacrificing the normative core. From this

perspective, the current research can be considered a significant theory elaboration effort.

Fourth, the current research analyzed the enterprise COTS implementation process through the

control-balancing theory proposed by Gregory, Beck, and Keil (2014), and identified the necessity

of control balancing for a multi-partner COTS implementation with a distributed team. Our

application of control-balancing theory is much more complex compared to that of Gregory et al.

(2014). This contributes to the generalizability of this relatively new theory. Furthermore, the

identification of underlying trigger factors that lead to control-balancing decisions can be

considered a substantial contribution to control-balancing theory.

Fifth, one of the major objectives of the current research aimed to identify stakeholder orientation,

which is a broader concept compared to stakeholder engagement. The development of a sound

theoretical framework to assess an organization’s stakeholder orientation can be considered a

significant contribution to stakeholder theory.

Finally, the current research offers a significant contribution to enterprise IS implementation by

linking implementation-specific processes, control configurations, and stakeholder orientations to

critical success factors for IS implementation project. Existing literature in this domain has

Page 272 of 307

examined control configurations, processes, stakeholder engagements, and CSF in isolation.

However, the effect of processes and control on stakeholder engagement and, in turn, on the CSF

has not yet been examined. Thus, our established links through the integration of all these

theoretical perspectives offers a holistic view into enterprise COTS implementation.

7.4 Implications for Practice

The knowledge and insights acquired through the analysis of multiple enterprise COTS

implementations will be appreciated by practitioners interested in similar implementations.

First, a task-and-activity-level analysis of the implementation phases facilitated the identification

of critical tasks, processes, and tools. Critical phases, information flow, tasks, tools, and processes

identified through our investigation can be utilized for most large-scale COTS implementation

projects to enhance their probability of success.

Second, the identification of CSF and their management are two important aspects of an enterprise

COTS and most IS implementations. Practitioners in all organizations are exceedingly interested

in both aspects, as most large-scale projects include a lessons-learnt exercise towards the end of a

project. Our examination of CSF management through stakeholder engagement and control

balancing will offer valuable insight for practitioners, enabling them to better manage CSFs during

a COTS implementation.

Third, the identification and conceptualization of stakeholder orientations related to COTS

implementation can help an organization, specifically project management teams, be cognizant of

Page 273 of 307

engagement versus sensitivity, which will help project management teams maintain an optimal

stakeholder orientation by selecting the right tools and processes.

Fourth, an appealing aspect of the current research for practitioners is the enumeration of control

dynamics for an enterprise COTS implementation, as controls are heavily utilized by most IS-

implementation projects. Insight into the factors that require the adjustment of control

configuration during implementation might be a source of further interest and benefit for

practitioners.

Fifth, the framework developed in this research can be of tremendous benefit for organizations in

evaluating their IS-implementation practices from an organizational stakeholder perspective. An

old but still accurate management adage—“You can’t manage what you don’t measure”—clearly

indicates the value of assessing stakeholder engagements in IS implementation using the

framework proposed in this paper, and validates the alignment of IT initiatives with organizational

stakeholder orientation. Through a grounded theory research approach, we identified four major

dimensions of organizational stakeholder orientation: (a) responsibility, (b) paternalism, (c)

neoclassical, and (d) strategic. These were further subdivided into two subsections each, resulting

in a total of eight different organizational orientations based on the level of stakeholder

engagement and sensitivity. The proposed framework shows the possibility of sacrificing the

normative aspect entirely in favour of an instrumental approach; however, due to the public-sector

context of the analyzed cases, each project demonstrated the characteristics of a convergent

stakeholder approach, whereby implementation phases have a well-defined normative core and

supporting instrumental processes to make them viable (Jones & Wick, 1999). Therefore, the

current research, in addition to answering the “how” aspect of stakeholder theory’s value

Page 274 of 307

proposition from a micro perspective, serves to elaborate on the convergent stakeholder theory

proposed by Wicks (1999).

Finally, the current research used an agile methodology to analyze IS implementation projects. In

modern times, agile and agility are recurring concepts that have engulfed, even baffled, both IT

professionals and organizational leaders. This is a consequence of never-ending efforts to find new

ways of combating ever-increasing threats from market volatility that generates turbulent business

environments, pressure for shorter delivery cycles, demand for greater productivity and efficiency,

and a rapidly shifting information technology landscape. Regardless of compelling motivations,

embarking on the agility bandwagon without a sound understanding of the concept may have dire

consequences for a company’s IT division and the organization as a whole. This is because of the

reciprocal relationship between IT and business, as established through strategic IT-business

alignment equations in the IT governance literature (Rahumi, Møller, & Hvam, 2016). As IT-

enabled processes and consumption of IT resources lead to business-value maximization through

strategic IT-business alignment (Peterson, 2004), adoption of innovative IT practices is often

influenced by organizational-level variables. Due to a virtually inseparable and often symmetric

relationship between IT and business models, successful agile transformation of IS implementation

practices is heavily influenced by an organization’s internal and external environment. Further

potential complications to agile transformation are tied to the traditional or prevalent practitioner’s

perspective that is unable to conceive of IT services and capabilities as concurrently stable and

dynamic. The COTS implementation process developed through the current research presents a

model for practitioners to capture how agile process can be integrated with traditional practices.

Page 275 of 307

7.5 Limitation of Current Research

7.5.1 Firms sampled and generalizability

We have used a single firm to collect data for this thesis which comprised of four distinct enterprise

COTS implementation cases. The firm selected for this research was of sufficient size and

complexity that it had designated IT organization with segregated support teams specializing on

different areas of IT such as COTS application support, network support, infrastructure support,

middle-tier support. This also indicated the existence of formalized practices (Weill & Ross, 2004).

Thus, this research may not be applicable to firms that do not have such a function or structure.

Moreover, this research selected an organization that belongs to the public sector. As the

stakeholder engagement approach is often different due to existing processes and the drive for

transparency, optimal stakeholder engagement levels identified through the current research may

not be fully supported by similar enterprise COTS implementation within a private organization.

For example, a private organization may demonstrate a higher level of instrumental approach

toward the stakeholder engagement influenced by the dominance of ‘stockholder’ theory.

7.5.2 Interview method

Akin to any other case study research, the current research also shows the possibility of suffering

from informant-bias due to the face to face interview method. Interview informants may have

sought to justify their actions and bias the data (Cresswell, 2009). Since the chief of ITS and the

senior leadership team have sanctioned the data collection, interview informants may have been

influenced (Kvale,1996). Additionally, project leadership team might have overstated the

Page 276 of 307

stakeholder engagement and project contributors might have overstated the level of management

control. We designed the interview protocol to minimize such risks through assurances of

anonymity and by conducting interviews with multiple individuals with different perspectives

within each project.

7.5.3 Granularity of the stakeholder orientation framework

A key contribution of the current research is the introduction of a stakeholder orientation

framework capturing ‘stakeholder engagement’ and ‘stakeholder sensitivity’. As the framework

models ‘engagement’ and ‘sensitivity’ using the X axis and the Y axis, resulting quadrants

represents an intersection between the engagement and sensitivity. Current research plots different

implementation phases at different quadrant of the framework based on the level of stakeholder

engagement and sensitivity combination. Although we have proposed the existence of an optimal

level for both construct within each quadrant, the current framework is not granular enough to

produce quantitative values corresponding to the indicated levels.

7.6 Future Research Directions

7.6.1 Agile implementation process of enterprise COTS implementation

The current research attempted to analyze control configuration and stakeholder orientation for

four large enterprise COTS implementation projects. All four selected projects were implemented

by the same organization. Although the selected projects tried to follow an agile implementation

approach, overall agile maturity level for the organization was still in the infancy. Therefore, all

Page 277 of 307

four projects under Case A, B, C, and D indicated the presence of waterfall or traditional project

management practices such as comprehensive upfront planning of the project’s scope. However,

an agile approach is evident during the execution phases of the selected Cases. Since the agile

methodology promotes a high level of team cohesion and frequent interactions among the team

members, the dynamic nature of stakeholder orientation and control configuration is an ideal topic

of investigation for future research in the context of a more mature agile environment.

7.6.2 Trigger factors for shifting stakeholder orientations for COTS implementation

The current research identified three salient factors as trigger conditions for changing control

configurations during a COTS implementation project. These factors are: level of shared

understanding, negative anticipation, and deviation of expectation. Current research found a

correlation between the increase or decrease of the level of these trigger factors and a change in

control configuration of the project. We anticipate the existence of additional trigger factors related

to a control configuration change during a COTS implementation. A longitudinal study would be

a more suitable approach to identify such trigger factors influencing control configurations that

were not captured by the current research.

7.6.3 Enacting Clan control and self control on the development team

The current research investigated the dynamic nature of control configuration during enterprise

COTS implementation. One of the three control configurations, the trust-based control

configuration, utilizes informal control mechanisms such as social controls. However, concepts

such as clan control or self-control were not explored or investigated through the current research.

This appears as an interesting dilemma for control researchers because the core project team in a

Page 278 of 307

COTS implementation project includes highly technical individuals who are typically

compensated for their unique skills and abilities (Ang and Slaughter 2001). Clearly, reward

structures that emphasize individual achievement represent an incentive misalignment from a team

perspective. Therefore, earlier researchers have argued that exercise of self-control can be

detrimental to the relationship between the use of agile methodology and project quality (Maruping,

Venkatesh, & Agarwal, 2009). Future research should investigate contingencies under which the use

of self control yields positive outcomes for COTS implementation project adopting an agile

methodology.

7.6.4 Effectiveness of stakeholder orientation and balancing isomorphic pressure

The stakeholder approach to strategic management examines the firm within a myriad of

relationships. Such diversity of relationships is also very much vivid in a large-scale enterprise

COTS implementation. Several researchers argued that devoting appropriate attention to all

legitimate stakeholders is important to achieve superior performance (Donaldson and Preston

1995; Freeman 1984). However, Verbeke and Tung (2013) argued that stakeholder engagement

leads to resource heterogeneity and competitive advantages in an early stage, but then at a later

stage contribute to inter-firm homogeneity through pressures favoring shared practices. Such

isomorphism or isomorphic pressure is contrary to project innovation. Future research should

investigate how optimal stakeholder orientation can help overcome such isomorphic pressure from

various stakeholder groups.

7.6.5 Validating the current research in the context of a shifting technological landscape

Page 279 of 307

IT infrastructures at present time are significantly shifting from an in-house model to a cloud-

based model. This shift is primarily driven by the rapid increase in the processing power of

computers, the speed of communication channels, and a constant decline of storage costs. In the

presence of these positive innovations, businesses are increasingly moving to cloud to stay

competitive and avoid IT infrastructure obsolesce. This is obvious by the recent adoption trend of

SAP HANA and other cloud-based ERP and enterprise COTS solutions. The cloud introduces

several factors in the domain of stakeholder engagement and control in addition to the differences

with the implementation process. Future research can further enhance the findings of this current

study by validating the stakeholder engagement framework as well as develop new control

dynamics suitable for such a context.

Page 280 of 307

References

Achterkamp, M. C. and Vos, J. F. J. (2008). Investigating the use of the stakeholder notion in project

management literature, a meta-analysis. International Journal of Project Management, 26, 749–757.

Agile software development (15 June 2014), In Wikipedia, the free encyclopedia, retrieved from

http://en.wikipedia.org/wiki/Agile_software_development on June 15th 2014

Albert, C. and Brownsword, L. (2001). Meeting the Challenges of Commercial-Off-The-Shelf (COTS)

Products: The Information Technology Solutions Evolution Process (ITSEP). Proceedings of the First

International Conference on COTS-Based Software Systems.

Albert, C. and Brownsword, L. (2002). Evolutionary Process for Integrating COTS-Based Systems

(EPIC): Building, Fielding, and Supporting Commercial-Off-the-Shelf (COTS): Based Solution,

Technical Report, CMU/SEI-2002-TR-005, ESC-TR-2002-005, November 2002.

Abts, C. (1997). COTS Software Integration Cost Modeling Study. University of Southern California,

Center for Software Engineering, June 1997

Ang, S., S. A. Slaughter. 2001. Work outcomes and job design for contract versus permanent information

systems professionals on software development teams. MIS Quarterly, 25(3) 321

Ayala,J. Ing, J., Perrault, E., Elliott, G., Letkemann, L., and Baynton, M. (2011). The Potential of Online

Learning in Addressing Challenges in Field Instructor Training, Scholarship in the Human Services. 13(1)

Birkmeier, D. Q. and Overhage, S. (2009). A Survey of service identification approaches: Classification

framework, state of the art and comparison. Enterprise Modelling and Information System Architecture.

4(2). 20-36

Beach, S. (2008). Sustainability of network governance: stakeholder influence. Proceedings of

Contemporary Issues in Public Management: The 12th Annual Conference of the International Research

Society for Public Management (IRSPM XII), Brisbane, 1-23.

Beaubouef, G. B. (2009). Maximize Your Investment: 10 Key Strategies for Effective Packaged Software

Implementations, Packt Publishing. Birmingham, United Kingdom

Berman, S. L., Wicks, A. C., Kotha, S., and Jones, T. M. (1999). Does stakeholder orientation matter?

The relationship between stakeholder management models and firm financial performance. Academy of

Management Journal, 42, 488-506.

Babbie, E. (2001). The Practice of Social Research, Belmont, California: Wadsworth Thompson

Learning.

Ballejos, L. C., and Montagna, J. M. (2011). Modeling stakeholders for information systems design

processes. Requirements Engineering, 16, 281–296

Brownsword, L. and Place, P. (2000). Lessons Learned Applying Commercial Off-the-Shelf Products.

Software Engineering Institute, Carnegie Mellon, retrieved June 5th 2014 from

http://www.sei.cmu.edu/reports/99tn015.pdf

http://en.wikipedia.org/wiki/Agile_software_development

http://www.sei.cmu.edu/reports/99tn015.pdf

Page 281 of 307

Brownsword, L. and Oberndorf, T. (2000). Developing new process for COTS based Systems. IEEE

Software, 17 (4), 48

Boddy, D., and Buchanan, D. A. (1986). Managing New Technology. Basil Blackwell, Oxford, UK.

Brown, A. D. and Jones, M. R. (1998). Doomed to failure: narratives of inevitability and conspiracy in a

failed IS project. Organizational Studies, 19, 73–88.

Bryson, J. M. (2004). What to do when Stakeholder matter, Public Management Review, 6(1), 21–53

Bryson, J., Bromiley, P. and Jung, Y. S. (1990). Influences on the Context and Process on Project

Planning Success. Journal of Planning Education and Research, 9(3), 183 –185.

Bryson, J. and Bromiley, P. (1993). Critical Factors Affecting the Planning and Implementation of Major

Projects. Strategic Management Journal, 14, 319 – 337.

Beaubouef, G. B. (2009). Maximize Your Investment: 10 Key Strategies for Effective Packaged Software

Implementations, Packt Publishing 2009

Benbasat, I., Goldstein, D. and Mead, M. (1987). The Case Research Strategy in Studies of Information

Systems. MIS Quarterly Vol. 11, 369-386.

Brown, A.D., Jones, M.R., 1998. Doomed to failure: narratives of inevitability and conspiracy in a failed

IS project. Organ. Stud. 19, 73–88.

Bingi, P., Sharma, M.K. and Godla, J. (1999), “Critical issues affecting an ERP implementation”,

Information Systems Management,16, 7.

Bullen, C. V. & Rockart, J. F. (1981). A primer on critical success factors. Cambridge, MA: Center

for Information Systems Research, MIT.

Boynton, A. C. & Zmud, R. W. (1984). An assessment of critical success factors. Sloan Management

Review, 25(3), 17-27.

Buchholz, R. and Rosenthal, S. (2005). Toward a contemporary conceptual framework for

stakeholder theory, Journal of Business Ethics, 58(1), 137-48.

Burby, R. (2003). Making Plans That Matter: Citizen Involvement and Government Action. Journal of

the American Planning Association, 69(1), 33 – 50.

Buckhout, S., Frey, E. and Nemec, J. (1999). Making ERP Succeed: Turning Fear into Promise,

Technology, 15.

Bryant, A. and Charmaz, K. (2007). The Sage Handbook of Grounded Theory, Thousand Oaks, CA: Sage

Publications

Bourgeois, L. J., and Eisenhardt, K. M. (1988) “Strategic Decision Process in High Velocity

Environments: Four Cases in the Microcomputer Industry”, Management Science (34)7, 816-835.

Page 282 of 307

Bryman, A., and Bell, E. (2003). Business Research Methods (1st ed., p. 608). Oxford: Oxford University

Press.

Campbell, D.T. (1975). Degrees of freedom and the case study. Comparative Political Studies, (8), 178-

193.

Cresswell, J. W. (2009). Research Design: Qualitative, Quantitative and Mixed Methods Approaches

(Third Edition, p. 260). Thousand Oaks, California: Sage.

Carlile, P. R., and Christensen, C. M. (2005). The Cycles of Theory Building in Management Research

the Cycles of Theory Building in Management Research. Harvard Business School Working Paper 05-

057, Retrieved from http://www.hbs.edu/research/pdf/05-057.pdf

Christensen, C. (2006). The ongoing process of building a theory of disruption. Journal o f Product

Innovation Management, (2004), 39-55. Retrieved from http://onlinelibrary.wiley.eom/doi/10.llll/j.1540-

5885.2005.00180.x/full

Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide through Qualitative Analysis,

Thousand Oaks, CA: Sage Publications.

Chemuturi, M. (2013). Mastering IT Project Management. Plantation, FL, J. Ross Publishing

Clegg, H. and Montgomery, S. (2006). How to write an RFP for information products. Information

Outlook, 10 (6), 23–31

Cecez-Kecmanovic, D., Kautz, K. and Abrahall, R. (2014). Reframing Success and Failure of Information

Systems: A Performative Perspective, MIS Quarterly, 38(2), 561-588

Chidley, A. (2014). Use COTS Parts to Cut Costs In Military And Aerospace Systems. Retrieved from:

http://electronicdesign.com/components/use-cots-parts-cut-costs-military-and-aerospace-systems

Cobb, M. (1996). Unfinished Voyages: A Follow-Up to the CHAOS Report, The Standish Group.

(http://www.umflint.edu/~weli/courses/bus381/assignment/vo.pdf)

Conboy, K. (2009). Agility from first principles: reconstructing the concept of agility in information

systems development, Information Systems Resource, 20, 329–354

Choudhury, V. and Sabherwal, R. (2003) Portfolios of Control in Outsourced Software Development

Projects. Information Systems Research. 14(3)

Data Center Consolidation Initiatives (2015, December 03) Retrieved from:

https://www.canada.ca/en/shared-services/corporate/data-centre-consolidation.html

Davis, B. (2013). Mastering Software Project Requirements. Plantation, FL, J. Ross Publishing

Davenport, T. (1998). Putting the enterprise into the enterprise system, Harvard Business Review, 76 (4),

121-131.

http://electronicdesign.com/components/use-cots-parts-cut-costs-military-and-aerospace-systems

https://www.canada.ca/en/shared-services/corporate/data-centre-consolidation.html

Page 283 of 307

Davenport, T. (2000). Mission Critical: Realizing the Promise of Enterprise Systems, Boston,

Massachusetts: Harvard Business School Press.

Deelstra, Y., Nooteboom, S. G., Kohlmann, H. R., van den Berg, J., and Innanen, S. (2003). Using

knowledge for decision-making purposes in the context of large projects in The Netherlands.

Environmental Impact Assessment Review, 23(5), 517–541.

Doherty, N. F., Ashurst, C., and Peppard, J. (2011). Factors Affecting the Successful Realization of

Benefits from System Development Projects: Findings from Three Case Studies, Journal of Information

Technology, 1-16.

Donaldson, T. and Preston, L. E. (1995). The Stakeholder Theory of the Corporation: Concepts,

Evidence, and Implications, The Academy of Management Review, 20 (1), 65-91

Deloitte (2010). What is DeepDive? Retrieved on June 17, 2014 from

http://www.deloitte.com/view/en_US/us/Services/consulting/a02b312c66702210VgnVCM100000ba42f0

0aRCRD.htm

DeLone, W.H., and McLean, E.R. (1992). Information Systems Success: The Quest for the Dependent

Variable, Information Systems Research 3 (1), 60-95.

Dingsøyr, T., Nerur, S., Balijepally, V., and Moe, N.B., (2012). A decade of agile methodologies: towards

explaining agile software development. Journal of Systems and Software 85 (6), 1213–1221,

Dube, L., and Pare, G. (2003). Rigor in Information Systems Positivist Case Research: Current Practices,

Trends, and Recommendations. MIS Quarterly, 27(4), 597- 635.

Dwivedi, Y. K., Wastell, D., Laumer, S., Henriksen, H. G., Myers, M, D., Bunker, D., Elbanna, A.,

Ravishankar, M., N., and Srivastava, S.C. (2015). Research on information systems failures and

successes: Status update and future directions. Information Systems Frontier, 17:143–157

Elgazzar, S., Kark, A., Putrycz, E. and Vigder, M. (2005). COTS Acquision: Getting a Good Contract,

4th international conference on COTS-Based SOftware Systems, Spain 2005

Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review,

16 (3), 620-627

Eisenhardt, K. M. (1991). Better stories and better constructs. Academy of Management Review, 14(4),

532-550.

Eisenhardt, K. M. and Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges.

Academy of Management Review, 50(1) 25-32.

Fassin, Y. (2012). Stakeholder Management, Reciprocity and Stakeholder Responsibility, Journal of

Business Ethics, 109, 83–96

Finney, S. and Corbett, M. (2007). ERP implementation: a compilation and analysis of critical success

factors. Business Process Management Journal, 13 (3), 329 - 347

Freeman, R. E. (1984). Strategic Management: A Stakeholder Approach. Boston: Pitman.

http://www.deloitte.com/view/en_US/us/Services/consulting/a02b312c66702210VgnVCM100000ba42f00aRCRD.htm

http://www.deloitte.com/view/en_US/us/Services/consulting/a02b312c66702210VgnVCM100000ba42f00aRCRD.htm

Page 284 of 307

Friedman, A. and Miles, S. (2006). Stakeholders: Theory and Practice, Oxford University Press,

Oxford.

Flak, L. S. and Rose, J. (2005). Stakeholder Governance: Adapting stakeholder theory to e-government.

Communications of the Association for Information Systems, 16, 662-664

Greenwood, M. (2007). Stakeholder Engagement: Beyond the Myth of Corporate Responsibility. Journal

of Business Ethics, 74 (4)

Gibbert, M., Ruigrok, W. and Wicki, B. (2008). What passes as a rigorous case study? Strategic

Management Journal, 29, 1465-1474.

Glaser, B. G., and Strauss, A. (1967). The Discovery of Grounded Theory: Strategies for Qualitative

Research, Chicago: Aldine Publishing Company

Glaser, B. G. (1978). Theoretical Sensitivity, Mill Valley, CA: The Sociology Press.

Gregory, R. W., Beck, R., and Keil, M. (2013). Control balancing in information systems development

offshoring projects. MIS Quarterly, 37(4), 1211-1232

Harper, D. (2001). Engage. In Online etymology dictionary. Retrieved on October 3, 2005 from:

http://www.etymonline.com/index.php?search=engage&searchmode=none

Henderson, J.C. and Lee, S. (1992). Managing I/S Design Teams: A control theories perspective,

Management Science 38(6), 757–777.

James, D. and Wolf, M. (2000). A Second Wind for ERP, The McKinsey Quarterly, 2, 100-107.

Jones, T. M., and Wicks, A. C. (1999). Convergent Stakeholder Theory. The Academy of Management

Review, 24(2), 206-221

Jadhav, A. S. and Sonar, R. M. (2009). Evaluating and selecting software packages: A review.

Information and Software Technology, 51, 555-653.

Karamouzis, F., and Longwood, J. (2007). Guidelines of an RFP process for standardized IT service

provider selections. Gartner Research Group

Kelle, U. (2007). The Development of Categories: Different Approaches in Grounded Theory, in The

Sage Handbook of Grounded Theory, A. Bryant and K. Charmaz (eds.). London: Sage Publications, 191-

213.

Kivits, R. (2011). Three component stakeholder analysis. International Journal of Multiple Research

Approaches, 5(3), 318–333.

Gupta, S. Misra, S. C., Singh, A., Kumar, V., and Kumar, U. (2016). Identification of Challenges and

their Ranking in the Implementation of Cloud ERP: A Comparative Study for SMEs and Large

Organizations, International Journal of Quality and Reliability Management, Accepted in June 2016. (in

Press 2016)

King W. R., Grover V. and Hufnagel E. H. (1989) 'Using Information and Information Technology for

Sustainable Competitive Advantage: Some Empirical Evidence' Information & Management 17(2), 87-93

http://www.etymonline.com/index.php?search=engage&searchmode=none

Page 285 of 307

Kirsch, L.J. (1996) The management of complex tasks in organizations: controlling the systems

development process. Organization Science, 7, 1–21.

Kirsch, L.J. (1997) Portfolios of control modes and IS project management. Information Systems

Research, 8, 215–239.

Kirsch, L.J. (2004) Deploying common systems globally: the dynamics of control. Information Systems

Research, 15, 374–395.

Kumar, V., Maheshwari, B. and Kumar, U. (2003). An Investigation of Critical management Issues in

ERP Implementation: Empirical Evidence from Canadian Organizations, Technovation - The

International Journal of Technological Innovation and Entrepreneurship, Vol. 23, 793-807.

Kumar, V., Maheshwari, B. and Kumar, U. (2002). Enterprise Resource Planning Systems Adoption

Process: A Survey of Canadian Organizations, International Journal of Production Research, 40 (3), 509-

523.

Kumar, V., Maheshwari, B. and Kumar, U. (2002). ERP Systems Implementation: Best Practices in

Canadian Government Organizations, Government Information Quarterly, 19 (1).

Kvale, S. (1996). Interviews - An Introduction to Qualitative Research Interviews (p. 326]. Thousand

Oaks, California: Sage.

Lorenzo, O. (2004). A Comprehensive Review of the Enterprise Systems Research, Working Paper,

retrieved April 14, 2014 from http://latienda.ie.edu/working_papers_economia/wp04-12.pdf

Letavec, C. (2014). Strategic Benefits Realization. Plantation, Florida: J. Ross Publishing

Lyytinen, K., and Hirschheim, R. (1987). Information Systems Failures - a Survey and Classification of

the Empirical Literature. In Oxford Surveys in Information Technology. Oxford: Oxford University

Press, 257-309.

Lyytinen, K. (1988). Stakeholders, IS failures and soft systems methodology: an assessment. Journal of

Applied Systems Analysis, 15, 61-81.

Leavitt, H.J. (1965) Applied organizational change in industry: structural, technological and

humanistic approaches. In J.G. March (ed.), Handbook of Organizations. Chicago: Rand-

McNally, 1965, pp. 1144–1170.

Layder, D. (1993). New Strategies in Social Research: An Introduction and Guide (First., p. 218). Oxford:

Polity Press.

Leonard-Barton, D. (1990). A Dual methodology for case studies: Synergistic use of a longitudinal single

site with replicated multiple sites, Organizational Science, 1 (3), 248-266.Margerum, R. (2002).

Collaborative Planning: Building Consensus and a Distinct Model of Practice. Journal of Planning

Education and Research, 21, 237 – 53.

Lewis, G. A. and Wrage, L. (2004). A Case Study in COTS Product Integration using XML. COTS-Based

Software Systems: Third International Conference, ICCBSS

http://latienda.ie.edu/working_papers_economia/wp04-12.pdf

Page 286 of 307

Maybury, M. T. (2006). MITRE TECHNICAL REPORT: Expert Finding Systems. Retrieved May 10th

2014 from http://infoautoclassification.org/public/articles/Maybury_MITRE-Technical-Report-Expert-

Finding-Systems.pdf

Markus, L. and Tanis, C. (2000) ‘The enterprise systems experience – from adoption to success’, In R.W.

Zmud (Ed.), Framing the Domains of IT Research: Glimpsing the Future Through the Past, Cincinnati,

OH: Pinnaflex Educational Resources, Inc.

Mainardes, E. W., Alves, H. and Raposo, M. (2011). Stakeholder theory: issues to resolve. Management

Decision, 49 (2), 226-252

Mahring, M. (2002). IT Project Governance, Pf. D. dissertation, Stockholm School of Economics,

Stockholm, Sweden

Mishra, A. and Mishra, D. (2013). Applications of Stakeholder Theory in Information Systems and

Technology. Inzinerine Ekonomika-Engineering Economics, 24(3), 254-266

Misra, S. C., Kumar, V., Kumar, U., Fantazy, K. and Akhter, M. (2012). Agile Software Development

Practices: Evolution, Principles, and Criticisms, International Journal of Quality and Reliability

Management (IJQRM), 29 (9), 972-980.

Misra, S.C., Kumar, V. and Kumar, U. (2010). Identifying Some Critical Changes Required in Adopting

Agile Practices in Traditional Software Development Projects, International Journal of Quality and

Reliability Management, 27(4)

Misra, S. C., Kumar, U. and Kumar, V. (2010). Modeling Critical Challenges Required for Adopting

Agile Software Development Practices in Projects Practicing Traditional Plan Driven Practices, Software

Quality Professional Journal, American Society for Quality Publication, 12(3), 20-32.

Misra, S. C., Kumar, V. and Kumar, U. (2009). Modeling Factors that will Influence Success of Projects

that want to Adopt Agile Software Development Practices, Journal of Systems and Software, 82 (9),

1869-90.

Mitchell, R. K., and Agle, B. R. (1997). Toward a Theory of Stakeholder Identification and Salience:

Defining the Principle of Who and What Really Counts. Academy of Management Review, 22(4), 853-886

Moen, R. (2001). A Review of the IDEO Process, Retrieved on June 17, 2014 from

http://www.rwjf.org/content/dam/web-assets/2001/10/a-review-of-the-ideo-process

Morisio, M., Seaman, C.B., Basili, V.R., Parra, A.T., Kraft, S.E. and Condon, S.E. (2002). COTS-based

software development: Processes and open issues. The Journal of Systems and Software, 61, 189–199

Maruping, L.M., Venkatesh, V. and Agarwal, R. (2009). A Control Theory Perspective on Agile

Methodology Use and Changing User Requirements, Information Systems Research 20(3), 377–399.

Narayanaswamy, R. Grover, V. and Henry, R. M. (2013). The Impact of Influence Tactics in Information

System Development Projects: A Control-Loss Perspective, Journal of Management Information Systems,

30(1), 191-226

http://infoautoclassification.org/public/articles/Maybury_MITRE-Technical-Report-Expert-Finding-Systems.pdf

http://infoautoclassification.org/public/articles/Maybury_MITRE-Technical-Report-Expert-Finding-Systems.pdf

http://www.rwjf.org/content/dam/web-assets/2001/10/a-review-of-the-ideo-process

Page 287 of 307

Newcomb, P. (2007). Think Outside the COTS. The Software Revolution, Inc., retrieved June 27th 2014

from http://www.tsri.com/files/Think%20Outside%20the%20COTS.pdf

Nidumolu, S.R. and Subramani, M.R. (2003). The Matrix of Control: Combining process and structure

approaches to managing software development, Journal of Management Information Systems 20(3), 159–

196.

Nutt, P. (2002). Why decisions fail: avoiding the blunders and traps that lead to debacles, Berrett-Koehler

Publishers

Oberndorf, P., Brownsword, L. and Sledge, C. (2000). An Activity Framework for COTS-Based Systems.

Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2000.

Oberndorf, T., Brownsword, L., Morris, E. J. and Sledge, C. A (1997). Workshop on COTS-Based

Systems, Software Engineering Institute, Carnegie Mellon University, retrieved June 10, 2014 from

http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=12791

Ouchi, W.G. (1978) The transmission of control through organizational hierarchy. Academy of

Management Journal, 21, 173–192.

Ouchi, W.G. (1979) A conceptual framework for the design of organizational control mechanisms.

Management Science, 25, 833–848

Pan, S. L., and Tan, B. (2011). Demystifying Case Research: A Structured-Pragmatic-Situational (SPS)

Approach to Conducting Case Studies, Information and Organization, 21(3), 161-176.

Petter, S., DeLone, W., & McLean, E. R. (2013). Information systems success: the quest for the

independent variables. Journal of Management Information Systems, 29(4), 7–62.

Péraire, C. and Pannone, R. (2005). The IBM Rational Unified Process for COTS-based projects: An

introduction. IBM, Retrieved May 20, 2014 from

https://www.ibm.com/developerworks/rational/library/aug05/peraire-pannone/

Popp, M. and Dallis, D. (2012). Planning and Implementing Resource Discovery Tools in Academic

Libraries, IGI Global

Pesqueux, Y. and Damak-Ayadi, S. (2005). Stakeholder theory in perspective, Corporate

Governance, 5 (2), 5-22.

Pouloudi, A., and Whitley, E. A. (1997). Stakeholder Identification in Interorganizational Systems:

Gaining Insights for Drug use Management Systems. European Journal of Information Systems, 6(1), 1-

14.

Post, J. E. and Andrew, P. N. (1982). Case research in corporation and society studies. In L. E. Preston

(Ed.) Research in Corporate Social Performance Policy: A Research Annual, (pp 1-34), Greenwich, CT:

Jai Press Inc.

Pushor, D. (2007). Parent Engagement: Creating a Shared World. Invited Research Paper-Ontario

Education Research Symposium

http://www.tsri.com/files/Think%20Outside%20the%20COTS.pdf

http://library.books24x7.com.proxy.library.carleton.ca/books.aspx?imprintid=109

http://library.books24x7.com.proxy.library.carleton.ca/books.aspx?imprintid=109

http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=12791

https://www.ibm.com/developerworks/rational/library/aug05/peraire-pannone/

Page 288 of 307

RapidBI (2007). Deep dive brainstorming technique, an innovative process for organizational

development, Retrieved on June 17, 2014 from

http://rapidbi.com/deepdivebrainstormingorganizationaldevelopment/

Rockart, J. F. (1979). Chief executives define their own data needs. Harvard Business Review, (March-

April).

Rockart, J. and De Long, D. (1988) Executive Support Systems, Dow Jones-Irwin, Homewood, Illinois.

Robertson, S. and Robertson, J. (2012). Mastering the Requirements Process: Getting Requirements

Right, Third Edition, Addison-Wesley Professional.

Runge, D. (1985). Telecommunications for Competitive Advantage unpublished PhD thesis, Templeton

College, Oxford.

Scholl, H. J. (2001). Applying Stakeholder Theory to e-Government: Benefits and Limits. Proceedings of

the 1st IFIP Conference on E-Commerce, E-Business, and E-Government (I3E 2001), Zurich,

Switzerland.

Scott, J. and Kaindl, L. (2000). Enhancing Functionality in an Enterprise Software Package, Information

and management, 37, 111-122

Shepherd, J. (2001). Enterprise Business Systems Must Be Upgraded and Maintained, AMR Research -

Executive View, (http://www.amrresearch.com/ exv/default.asp?i=57

Sykes, J.B. (Ed.). (1976). The concise Oxford English dictionary (6th

ed.). London: Oxford

University Press.

Software Development Lifecycle Phases (n.d.), University of Maryland. Retrieved July 3rd 2014 from

http://doit.maryland.gov/SDLC/COTS/Pages/Phase01Single.aspx

Stavru, S. (2014). A critical examination of recent industrial surveys on agile method usage; The Journal

of Systems and Software, 94, 87–97

Scheer, A-W. and Habermann, F. (2000), “Making ERP a success”, Association for Computing

Machinery. Communications of the ACM, 43, 57.

Sia, S.K. & Neo, B.S. (1997) Reengineering effectiveness and the redesign of organizational control:

a case study of the inland revenue authority of Singapore. Journal of Management Information

Systems, 14, 69–92.

Soh, C., Kien, S.S. and Tay-Yap, J. (2000), “Cultural fits and misfits: is ERP a universal solution?”,

Association for Computing Machinery. Communications of the ACM, 43, 47.

Somers, T.M. and Nelson, K.G. (2004), “A taxonomy of players and activities across the ERP

project life cycle”, Information & Management, 41, 257-78.

http://rapidbi.com/deepdivebrainstormingorganizationaldevelopment/

http://doit.maryland.gov/SDLC/COTS/Pages/Phase01Single.aspx

Page 289 of 307

Sun, A., Y., T., Yazdani, A., and Overend, J. D. (2005). Achievement assessment for enterprise resource

planning (ERP) system implementations based on critical success factors (CSFs). Int. J. Production

Economics, 98, 189–203

Torchiano, M. and Morisio, M. (2004). Overlooked aspects of COTS-based development, IEEE software

21 (2), 88-93

T. Ellis (1995). COTS Integration in Software Solutions – A Cost Model. Systems Engineering in the

Global Marketplace, NCOSE Symposium, St. Louis, MO

Tiwana, A., and Keil, M. (2009). “Control in Internal and Outsourced Software Projects,” Journal of

Management Information Systems, 26(3), 9-44.

Tarhini, A., Ammar, H., Tarhini, T. and Masa’deh, R. (2015). Analysis of the Critical Success Factors for

Enterprise Resource Planning Implementation from Stakeholders’ Perspective: A Systematic Review,

International Business Research, 8 (4)

Umble, E.J., Haft, R.R. and Umble, M.M. (2003), “Enterprise resource planning: implementation

procedures and critical success factors”, European Journal of Operational Research, 146, 241-57.

Urquhart, C. (2007). The Evolving Nature of Grounded Theory Method: The Case of the Information

Systems Discipline, in The Sage Handbook of Grounded Theory, A. Bryant and K. Charmaz (eds.),

London: Sage Publications, pp. 339-359.

Urquhart, C., Lehmann, H., and Myers, M. (2010). Putting the ‘Theory’ Back into Grounded Theory:

Guidelines for Grounded Theory Studies in Information Systems, Information Systems Journal, 20 (4),

357-381

Uçar, E and Bilgen, S. (2013). A Case-Based Model for Assessing the Effectiveness of Information

Systems Outsourcing, Journal of Information Technology Case and Application Research, 15 (3).

Van de Ven, A. H. (2007). Engaged Scholarship: A Guide for Organizational and Social Research, New

York: Oxford University Press

Verbeke, A. & Tung, V. (2013). The Future of Stakeholder Management Theory: A Temporal

Perspective. J Bus Ethics, 112:529–543

Ward, J. and Daniel, E. (2002). Benefits Management: How to Increase the Business Value of Your IT

Projects, John Wiley & Sons.

Wiegers, Karl E. (1996). Creating a Software Engineering Culture. New York: Dorset House Publishing.

Wiegers, Karl E. (2007). Practical Project Initiation: A Handbook with Tools. Microsoft Press

Weill, P., & Ross, J. W. (2004). IT Governance: How Top Performers Manage IT Decision Rights for

Superior Results (p. 269). Boston, Massachusetts: Harvard Business School Press.

Yin, R.K. (1981). The Case Study Crisis: Some Answers, Administrative Science Quarterly, (26) 1, 58-

65.

Page 290 of 307

Yin, R.K. (1984). Case study research: Design and methods, Newbury Park, CA: Sage.

Yin, R. K. (1994). Case study research: Design and methods (2nd ed.), Newbury Park, CA: Sage

Publications.

Yin, R.K. (2003). Case study research: Design and methods (3rd ed.), Thousand Oaks CA: Sage.

Zivkovic, J. (2012). Strengths and Weaknesses of Business Research Methodologies: Two Disparate Case

Studies, Business Studies Journal, (4) 2, 91-99.

Page 291 of 307

Appendix A – Research Approval

1. Ethics Committee Approval

Office of Research Ethics and Compliance

5110 Human Computer Interaction Bldg | 1125 Colonel By Drive

| Ottawa, Ontario K1S 5B6

613-520-2600 Ext: 2517

[email protected]

CERTIFICATION OF INSTITUTIONAL ETHICS CLEARANCE

The Carleton University Research Ethics Board-A (CUREB-A) has granted ethics clearance for the research

project described below and research may now proceed. CUREB-A is constituted and operates in compliance

with the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS2).

Ethics Protocol Clearance ID: Project # 107704

Project Team Members: Mr. Zafor Ahmed (Primary Investigator)

Vinod Kumar (Research Supervisor)

Project Title: Enterprise COTS Implementation: Process, Stakeholder, and Control Perspective [Zafor Ahmed]

Funding Source (If applicable):

Effective: November 13, 2017 Expires: November 30, 2018.

Restrictions:

This certification is subject to the following conditions:

1. Clearance is granted only for the research and purposes described in the application. 2. Any modification to the approved research must be submitted to CUREB-A via a Change to

Protocol Form. All changes must be cleared prior to the continuance of the research. 3. An Annual Status Report for the renewal of ethics clearance must be submitted and cleared by the

renewal date listed above. Failure to submit the Annual Status Report will result in the closure of the file.If funding is associated, funds will be frozen.

4. A closure request must be sent to CUREB-A when the research is complete or terminated. 5. Should any participant suffer adversely from their participation in the project you are required to

report the matter to CUREB-A. Failure to conduct the research in accordance with the principles of the Tri-Council Policy Statement: Ethical

Conduct for Research Involving Humans 2nd edition and the Carleton University Policies and Procedures for

the Ethical Conduct of Research may result in the suspension or termination of the research project.

https://maps.google.com/?q=1125+Colonel+By+Drive+%0D+%7C+Ottawa,+Ontario+K1S+5B6&entry=gmail&source=g

mailto:[email protected]



Page 292 of 307

Upon reasonable request, it is the policy of CUREB, for cleared protocols, to release the name of the PI, the title of the project, and the date of clearance and any renewal(s).

Please contact the Research Compliance Coordinators, at [email protected], if you have any questions or

require a clearance certificate with a signature.

CLEARED BY: Date: November 13, 2017

Andy Adler, PhD, Chair, CUREB-A

Bernadette Campbell, PhD, Vice-Chair, CUREB-A

2. Organizational Approval

No included in order to protect anonymity

Appendix B - Interview Guide

1. Email to Potential Interview Candidates

Subject: Invitation to participate in a research project on (Enterprise COTS Implementation:

Process, Stakeholder, and Control Perspective)

Dear <Name>,

My name is Zafor Ahmed and I am a doctoral student in the Sprott School of Business at Carleton

University. I am working on a research project under the supervision of Prof. Vinod Kumar and

Prof. Uma Kumar.

I am writing to you today to invite you to participate in a study entitled “Enterprise COTS

Implementation: Process, Stakeholder, and Control Perspective”. This study aims to investigate

four major projects implemented by the Bank of Canada from a stakeholder engagement and

control perspective.

This study involves one 60 minute interview that will take place in a mutually convenient, safe

location. With your consent, interviews will be audio-recorded. Once the recording has been

transcribed, the audio-recording will be destroyed. Additionally, care will be taken to protect your

identity at all time. This will be done by keeping all responses anonymous and allowing you to

request that certain responses not be included in the final project.

You will have the right to end your participation in the study at any time, for any reason, up until


Page 293 of 307

December 31st 2017. If you choose to withdraw, all the information you have provided will be

destroyed.

All research data, including audio-recordings and any notes will be encrypted. Any hard copies of

data (including any handwritten notes or USB keys) will be kept in a locked cabinet at the Bank

of Canada. Research data will only be accessible by the researcher and the research supervisor.

The ethics protocol for this project was reviewed by the Carleton University Research Ethics

Board, which provided clearance to carry out the research. (Clearance No. 107704)

CUREB-A:

If you have any ethical concerns with the study, please contact Dr. Andy Adler, Chair, Carleton

University Research Ethics Board-A (by phone at 613-520-2600 ext. 2517 or via email at

[email protected]).

If you would like to participate in this research project, or have any questions, please contact me

at [email protected]

Sincerely,

Zafor Ahmed, PMP, Ph.D. Candidate

Sprott School of Business

Carleton University

2. Informed Consent

LETTER OF INFORMED CONSENT: In-depth Interview

Title of research project: Enterprise COTS Implementation: Process, Stakeholder, and Control

Perspective

Date of ethics clearance: TBD


Page 294 of 307

Ethics Clearance for the Collection of Data: Expires: TBD

I understand that I have been invited to participate in a research study being carried out by Zafor

Ahmed, PhD Student, under the supervision of Prof. Uma Kumar, Sprott School of Business,

Carleton University and Prof. Vinod Kumar, Sprott School of Business, Carleton University.

I understand that the primary objective of this research study is to investigate the factors and

characteristics related to a successful enterprise COTS implementation in public sector. Also,

while participating in this research may not benefit me directly, I understand that the study will

advance my knowledge about the enterprise COTS implementation, Stakeholder engagement and

Control during COTS implementation in the Public Sector in general.

I understand that I am free to not answer questions, and to conclude the interview at any time. I

also understand that the interview will be audio recorded and may last up to 60 minutes.

I understand that I may withdraw from the study until December 31st 2017. Should I decide to

withdraw, the information I have provided will be destroyed and not used in the analysis or report

writing.

I understand that there may be some professional risk if my responses are critical of my employer.

To mitigate this risk, the researcher will avoid asking any question that will make me speak

negatively about my employer. I will retain anonymity in the final research paper and none of my

responses will be attributed to me. There will be no personally identifying pieces of information

collected for the purposes of this research and my department or agency will not be identified by

name or specific description.

I understand that the audio recordings will be stored electronically on an encrypted USB in a locked

drawer. These recordings will be transcribed and stored under a password-protected folder on a

secure computer. Recordings and transcriptions will be kept under lock and key in researcher’s

office. It will be ensured that the name of the respondent or the department or agency is not in the

transcription. The data will be used in dissemination of various academic journal papers and

conferences that come out of this research project. The recordings will be destroyed after

transcription, and transcriptions will be destroyed within one year of the completion of all data

collection. The data will be used only for this study.

This research project has been reviewed and has received ethics clearance by the Carleton

University Research Ethics Board. If you have any ethical concerns with the study, please contact

Dr. Andy Adler, Chair, Carleton University Research Ethics Board-A (by phone at 613-520-2600

ext. 2517 or via email at [email protected])

I have read and understand the above, and I have had my questions and concerns about the research

addressed: Yes No.

I agree for the interview to be audio-recorded: Yes No.

tel:(613)%20520-2600

tel:(613)%20520-2600


Page 295 of 307

Signature of Participant

Date Signed

Dr. Uma Kumar, Professor

Sprott School of Business Carleton University 1125 Colonel By Drive Ottawa, Ontario, K1S 5B6 [email protected] (613) 520-6601

Dr. Vinod Kumar, Professor Sprott School of Business Carleton University 1125 Colonel By Drive Ottawa, Ontario, K1S 5B6 [email protected] (613) 520-2379

Zafor Ahmed PhD Student Sprott School of Business Carleton University 1125 Colonel By Drive Ottawa, Ontario, K1S 5B6 [email protected] (447-5852613)

3. Interview Protocol

Appendix C – Informant List and Details

1. Case A Informant list:

Table 49: Case A Informant List

No Informant ID Role/Title Affiliation Member

Check

1 A01PT Senior Project Manager Host Organization/ Core Yes

2 A02PT Project Technical Lead Host Organization/ Core No

3 A03PT COTS Developer Host Organization/ Core No

4 A04PT Business Analyst Host Organization/ Core No

5 A05VR Project Manager COTS Vendor Yes

6 A06VR COTS Specialist/Consultant COTS Vendor No

7 A07BU Business Lead Host Organization/Business No

Page 296 of 307

8 A08BU Data Architect Host Organization/Business No

9 A09BU Business Analyst Host Organization/Business Yes

10 A10IT Solution Architect Host Organization/ ITS No

11 A11IT Enterprise Architect Host Organization/ ITS No

Total Number of Informants: 11

2. Case B Informant list:

Table 50: Case B Informant List


Check

1 B01PT Senior Project Manager Host Organization/ Core Yes

2 B02PT Project Technical Lead Host Organization/ Core No

3 B03PT COTS Developer Host Organization/ Core No

4 B04PT Business Analyst Host Organization/ Core No

5 B05VR Senior Project Manager COTS Vendor Yes

6 B06VR COTS Specialist/Consultant COTS Vendor No

7 B07VR Solution Architect COTS Vendor / Integrator No

8 B08VR COTS Developer COTS Vendor / Integrator No

9 B09BU Business Lead Host Organization/Business Yes

10 B10BU Data Architect Host Organization/Business No

11 B11IT Enterprise Architect Host Organization/ ITS No

12 B12IT Infrastructure Support/Server Host Organization/ ITS No


3. Case C Informant list:

Table 51: Case C Informant List


Check

1 C01PT Senior Project Manager Host Organization/ Core Yes

2 C02PT Project Technical Lead Host Organization/ Core No

3 C03PT COTS Developer Host Organization/ Core No

4 C04PT Business Analyst Host Organization/ Core No

5 C05VR Data Migration Specialist COTS Vendor Yes

6 C06VR COTS Specialist/Consultant COTS Vendor No

7 C07BU Business Lead Host Organization/Business No

8 C08BU Business Analyst Host Organization/Business Yes

9 C09IT Solution Architect Host Organization/ ITS No


Page 297 of 307

4. Case D Informant list:

Table 52: Case D Informant List


Check

1 D01PT Senior Project Manager Host Organization/ Core No

2 D02PT Project Technical Lead Host Organization/ Core Yes

3 D03PT Senior COTS Developer Host Organization/ Core No

4 D04PT Business Analyst Host Organization/ Core No

5 D05VR SharePoint Specialist COTS Vendor Yes

6 D06VR COTS Specialist/Consultant COTS Vendor No

7 D07BU Business Authority Host Organization/Business Yes

8 D08BU Business User Host Organization/Business Yes

9 D09IT Enterprise Solution Architect Host Organization/ ITS No


Page 298 of 307

Appendix D – Strategic Investment Gating Process Gate 1 Approval Process

Page 299 of 307

Gate 2 Approval Process

Page 300 of 307

Gate 3 Approval

Page 301 of 307

Gate 4 Approval Process

Page 302 of 307

Appendix E

Table 53: Informants Summary for All Cases


Face to Face Semi-


Duration for each



with Project team

# of interviews

with vendor and

solution integrator

# of interviews

for Client/

Business Units

Project Manager 4 2

Business Lead 4

Business Analyst 4 1 2


Data Architect 1 2

Solution Architect 2 2

Project team

member

9 5 1

21 11 9

Total # of

interviews

41

Appendix F: Acronym Table

Table 54: Acronyms

Acronyms Meaning

AOC Agency Operations Centers

BRD Business Requirement Document

COTS Commercial Off The Shelf

CSF Critical success factors

CRM Customer Relationship Management

ECM Enterprise Content Management

ERP Enterprise Resource Planning

ES Enterprise systems

EIS Enterprise Information Systems

FDD feature-driven development

FMO Financial Market Operations

FRS Financial Regulatory System

FI Financial Institutions

FBSR Financial Billing Systems Replacement

GUI Graphic user interfaces

Page 303 of 307

IS Information Systems

SLA Service level agreement

TCD Technical Conceptual Design

PMO Project Management Office

RUP Rational Unified Process

RTM Requirement Traceability Matrix

SRS System Requirements Specifications

SUC System Use Cases

UAT User Acceptance Testing

XP eXtreme programming

Appendix G: Data Collection Instrument

Project Name : Participant Code :

Participant association : Host / Partner/ Vendor / Solution Integrator

Date of interview :

Location :

1. Did you have this phase in this Project? Were you involved?

_____ Yes ______No

2. Which other phase does this phase relate to in terms of information flow (other than

immediate next phase)? i.e. Information generated on this phase results in immediate

actions on what other phases

3. What are the Key Activities performed during this Phase?

4. What are the key processes used during this phase?

Page 304 of 307

Not Significant =1 ….2…..3…..4…….5 = Very Significant

Initia

tion

Re

qu

irem

en

t &

Pla

nn

ing

An

aly

sis &

Co

nce

ptu

al

De

sign

D

esig

n &

A

rchite

cture

De

ve

lop

me

nt

De

live

ry

Clo

se-O

ut

Was this phase present in this project?

How signification this phase was in terms to activities?

Contribution level By:

VIZOR

OSFI/CDIC

Bank of Canada

DELOITTE

General Implementation Related Questions

5. What defined the success for this project? [Success factors]

Time, Budget, Scope, Quality, And Other Factors: ------------------------------------------

6. Which part was most critical for Success? And Why ?

Pre-development / Development / Delivery

7. Which part was most High Risk? And Why ?

Pre-development / Development / Delivery

8. Overall challenges for this project?

Page 305 of 307

9. What would you do differently if you do the same project again?

10.A. If this project were to fail, what are the reasons you can think of?

10.B. And how/what phase this should have been ideally detected to minimize the damage?

11. How did vendor's agile approach affect the PM process in the Bank? What is positive and

what is negative about it?

12. Something(s) that contributed to your personal experience and will be beneficial for future projects involvements, something that surprised you etc.

13. Some elements that are valuable for Tri-agency but not captured through standard project management templates (could be a process, could practices, standards or anything that you can think of)

14. Any generalized model that appears to be a good standard for future projects of this scale and

context for the Tri-Agency or Organizations of similar size/environment.

15. What do you recommend for an organization like us (the BANK) can do enhance our chances of

being successful with this challenge (mentioned on #18) thus enhancing the chance of overall

project success with other similar COTS products?

16. Do you see any other opportunity for the Tri-Agency to make things better in the future? like

creating more certain kind of roles / different project management style or changing any

internal process or different technical unit to handle COTS based projects like this ?

17. Project Cost was higher than average ERP implementation or $10 to $15 million, what are your

thoughts on that and is there any way to reduce it for similar projects?

18. What were the major in-efficiencies for this project?

19. General discussion around RFP, Requirement, Design and Development phases

20. General Discussion around Migration and Metadata Conversion activities

21. Project Governance discussion – How it was structure and why?

Page 306 of 307

Control Related

Low (L=1 to 3) Moderate (M=4 to 8) High (H=over 8)

1.

Initia

tion

2.

Re

qu

irem

en

t &

Pla

nn

ing

3.

An

aly

sis &

Co

nce

ptu

al

De

sign

4

. D

esig

n &

A

rchite

cture

5.

De

ve

lop

me

nt

6.

De

live

ry

7.

Clo

se-O

ut

What was the control configuration on this phase?

Was the control effective (Y/N)?

Control Trigger events

Level of meetings (per week)

Status requests and deadlines

Joint activities among partners

Level of missed deadlines or expected deliverables?

Level of escalations due to issues

Procedural Control = P Trust Based Control= T Coordinated control=C

22. Why control configuration was changed from phase 1 to phase 2?

23. Was this change produced desired results?










Page 307 of 307


Stakeholder Related

Low (Less than 30% stakeholders were involved)

Moderate (50% stakeholders were involved)

High (Over 70% stakeholders were involved)

Initia

tion

Re

qu

irem

en

t &

Pla

nn

ing

An

aly

sis &

Co

nce

ptu

al

De

sign

D

esig

n &

A

rchite

cture

De

ve

lop

me

nt

De

live

ry

Clo

se-O

ut

What was the level of stakeholder involvement at this phase? (L/M/H)

Stakeholder Influence

Host

Partners

Vendor

Solution Integrators

External/public

Clients

34. General discussion about stakeholder orientation and reasons for such orientation