10
International Journal of Information Management 23 (2003) 259–268 Case study Data warehousing in decision support for pharmaceutical R&D supply chain Sarmad Alshawi, Isabel Saez-Pujol, Zahir Irani* Department of Information Systems and Computing, Information Systems Evaluation and Integration Network Group (ISEing), Brunel University Uxbridge, Uxbridge, Middlesex UB8 3PH, UK Abstract The expanding technology of data warehousing is providing organisations with a powerful decision support utility that can be effectively used to support supply chain activities throughout a business or industry. Pharmaceutical Research and Development (R&D) activity represents a unique type of information-based supply chain that utilises a huge amount of data and involves a large number of decision-making points along its stages. By analysing the processes of drugs R&D in a pharmaceutical case study (company unnamed), the authors identify the main types of internal and external information sources utilised by the principle decision-making levels within the drug R&D supply chain. A classification of the information sources and the decision-making levels is then presented. The paper also discusses how by integrating these information sources, data warehouse technology can facilitate effective decision support leading to a shortening of the drug development life cycle. r 2003 Elsevier Science Ltd. All rights reserved. Keywords: Decision support; Data warehouse; Pharmaceutical R&D; Supply chain 1. Introduction With the fast development of Information Systems during the last three decades, the amount of information available to the public has dramatically increased. In order to be able to capture and use this information to its full potential, organisations need to use the appropriate technology at their disposal to keep ahead of their competitors. Some of these tools will be facilitators for obtaining the right information at the right time, which could be critical to the evolution of the company and its competitiveness (Orr, 2000). *Corresponding author. Tel.: +44-1895-816-211; fax: +44-1895-816-242. E-mail address: [email protected] (Z. Irani). 0268-4012/03/$ - see front matter r 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0268-4012(03)00028-8

Datawarehousing In Pharma

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Datawarehousing In Pharma

International Journal of Information Management 23 (2003) 259–268

Case study

Data warehousing in decision support for pharmaceuticalR&D supply chain

Sarmad Alshawi, Isabel Saez-Pujol, Zahir Irani*

Department of Information Systems and Computing, Information Systems Evaluation and Integration Network Group

(ISEing), Brunel University Uxbridge, Uxbridge, Middlesex UB8 3PH, UK

Abstract

The expanding technology of data warehousing is providing organisations with a powerful decisionsupport utility that can be effectively used to support supply chain activities throughout a business orindustry. Pharmaceutical Research and Development (R&D) activity represents a unique type ofinformation-based supply chain that utilises a huge amount of data and involves a large number ofdecision-making points along its stages. By analysing the processes of drugs R&D in a pharmaceutical casestudy (company unnamed), the authors identify the main types of internal and external information sourcesutilised by the principle decision-making levels within the drug R&D supply chain. A classification of theinformation sources and the decision-making levels is then presented. The paper also discusses how byintegrating these information sources, data warehouse technology can facilitate effective decision supportleading to a shortening of the drug development life cycle.r 2003 Elsevier Science Ltd. All rights reserved.

Keywords: Decision support; Data warehouse; Pharmaceutical R&D; Supply chain

1. Introduction

With the fast development of Information Systems during the last three decades, the amount ofinformation available to the public has dramatically increased. In order to be able to capture anduse this information to its full potential, organisations need to use the appropriate technology attheir disposal to keep ahead of their competitors. Some of these tools will be facilitators forobtaining the right information at the right time, which could be critical to the evolution of thecompany and its competitiveness (Orr, 2000).

*Corresponding author. Tel.: +44-1895-816-211; fax: +44-1895-816-242.

E-mail address: [email protected] (Z. Irani).

0268-4012/03/$ - see front matter r 2003 Elsevier Science Ltd. All rights reserved.

doi:10.1016/S0268-4012(03)00028-8

Page 2: Datawarehousing In Pharma

Many organisations are increasingly starting to recognise the potential of the informationstored within their systems as well as the ability to capture, access and analyse information. Itsdistribution inside the enterprise as well as internal communication needs to be fast, accurate andefficient (Hackathorm, 1998). However, in practice, it is common that information is created,captured and stored in databases by different functions, in different ways and with little co-ordination throughout the different phases of supply chains (Boar, 1996). In more detail, datasharing throughout the different business areas can become a complicated process inmultinational companies, where data is stored and accessed differently according to the culture,development of the economy and use of Information Systems across international locations. Someof the consequences of this lack of co-ordination are difficulties in accessing information whichslow down the development process of the supply chain and create unnecessary rework, with thecorresponding related cost (Jonathan, 1999).The use of a tool such as an integrated Data Warehouse could provide the solution to these

problems. Modern data-warehousing tools provide the opportunity to store, access and distributecorporate data in an integrated way (Labio, Quass, & Adelberg, 1997; Cameron, 1998).Moreover, the analysis of historical data can support management decision making, thereforeoptimising company strategy (Tanrikorur, 1998; Thomsen, 1998).Supply chain management is understood as the integration of supplier, distributor, and

customer logistics requirements into one cohesive process. Traditionally, this definition is usedin manufacturing chains; however, this paper will focus on the process of supply chainmanagement in Research and Development (R&D). In the pharmaceutical industry, beforethe drug is manufactured and launched into the market, approximately 7.5 years for thediscovery and another 9.5 years for the development of the drug precede it. These 17pre-manufacturing years are known as the pharmaceutical R&D supply chain. In some industries,using data in the right way is not only a means of obtaining competitive advantage, butalso crucial to comply with regulatory constraints (Harrington, 1999), as the use of the wrongdata could harm third parties. This is the case in the pharmaceutical industry, where using thewrong information could lead to human damage and other drastic consequences. This paperstudies the role data warehousing can play in supporting decision making in the pharmaceuticalR&D processes. It also highlights the importance of data warehousing in the speed ofdevelopment of the drug’s life cycle.

2. Case study profile

The research is based on the case study of the R&D divisions at a medium size multinationalpharmaceutical organisation, whose operations are located in three different countries (UK,France and Spain). The research method followed consisted, besides general observations, ofstructured interviews and questionnaires directed at key employees throughout the R&D supplychain. The information obtained from the interviews and questionnaires was mainly related totheir specific departments. The selection of the interviewees followed the guidance of the Head ofR&D Informatics, who provided the contacts. An interview guideline was prepared and sent tothe interviewees before the discussions. These guidelines were prepared following suggestions ofKimball (1996, 1998), and were approved by the Director of Data Management of the

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268260

Page 3: Datawarehousing In Pharma

organisation. The approach aimed to support consistency throughout all interviews and obtainthe following information:

* The role of the departments in the R&D of a drug.* The business processes involved performing this role.* The types of data required and generated during the business processes as well as the toolsutilised to capture, access and store this data.

The information obtained during the interviews was analysed using qualitative data analysistechniques recommended by Miles and Huberman (1984). The use of these analysis techniquesensured that the data obtained from the interviews was accurate. After the one-to-one, face-to-face interviews and questionnaire activities were completed, a report was drafted and given to theinterviewees for their feedback, comments and the filling in of any gaps that may have resulted.Once all the departmental information had been analysed and documented, a cross-departmentanalysis was performed in order to produce a high-level review of the internal and externalinformation flow. This cross-department analysis was then reviewed by key employees in R&Dthat provided comments and feedback.

3. Pharmaceutical R&D

Although most literature discusses supply chain management as the production ofphysical products, not all industries have the same supply chain characteristics. As Tanrikorur(1998) discusses, ‘‘every company has a uniquely different culture, environment andrequirements’’. In certain industries, supply chains may also be non-physical, i.e. intellectual orinformational (Crowley, 1997), where data or information is required and also produced whiledeveloping an intellectual concept or product. This is the case of R&D supply chains in thePharmaceutical industry, where the time spent exchanging information in the discovery and thedevelopment of a drug concept can be extremely costly. Before a drug is manufactured and ismade available to the public market, it follows a non-physical supply chain, where intellectual(rather than physical) information is required and produced. The R&D of a drug is initiated byscientists generating hypotheses for new compounds with both proprietary and public domaininformation. It continues through the discovery, development and testing of new drugs andfinalises when the product receives marketing authorisation from the relevant regulatoryauthorities.In the pharmaceutical R&D supply chain, suppliers are the different sources of data

and information required to develop the drug. These sources can be classified as externalsources (e.g. Hospitals, Contract Research Organisations, consultants or the public sector)and internal sources (e.g. other departments or offices in the organisation). The R&Dintellectual production of a drug is initiated with some basic research that will generatehypotheses for new compounds. It continues through the development and testing of newdrugs and finalises when the product receives marketing authorisation from the relevantregulatory authorities. The customer is the Regulatory Agency to which registration of the drugwill be submitted.

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268 261

Page 4: Datawarehousing In Pharma

4. Phases of the drug research process

Although it is not the purpose or scope of this paper to describe the particular details of thedifferent stages in the discovery and development of a drug, we will provide a brief description ofthese activities in order to give the reader an insight of the processes.

5. Research process

The research process of the drug R&D supply chain starts with the generation of hypothesescompounds and ends with the study of the compounds’ reactions against the disease. The differentstages of the research are concept, screening, target identification, chemical lead andpharmacology.

Concept: The first step of the discovery is defining the idea or concept of the drug that is goingto be developed.

Screening: During this phase thousands of potential candidate compounds are screened. Theaim of this phase is matching the molecular structure of potential medicines with the patternsfound in human genes. Screening can take up to 2.5 years to be completed.

Target identification: This phase studies the effectiveness of the potential drugcandidates against the specific disease. The tests are generally carried out for a length of1.5 years.

Chemical lead: The chemical lead identification consists on the mapping of the compound’sstructure. A chemical name is chosen according to the activities of the compound. This isgenerally completed in 1.5 years.

Pharmacology: Pharmacology is the detailed study of the way the compounds react towards thedisease and other body functions. The study generally takes approximately 1.5 years.

6. Development process

During the development of the drug, samples are tested on both animals andhumans in different doses. The process includes the pre-clinical testing stage, followed byPhases I–IV.

Pre-clinical testing: In this phase, laboratories evaluate the different compounds and thetoxicological testing on animals is studied. The length of this process is approximately 3.5 years.

Phase I: This phase consist on the testing of the drug on a high number of healthy volunteers(between 20 and 80 people). Phase I tests are generally carried out for a length of 1 year.

Phase II: This phase studies the dose that is the most efficient and also determines side effects.The drug is administered to volunteer patients (between 100 and 300 people). The study generallytakes approximately 2 years to complete.

Phase III: This phase aims to verify the effectiveness of the drug by comparison to another drugor placebo. It also monitors adverse reactions from long-term use. The drug is administered to alarge number of volunteer patients (between 1000 and 3000 people). The duration of the process isabout 3 years.

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268262

Page 5: Datawarehousing In Pharma

Phase IV: Following Phase III, a process of review and approval by the Food and DrugAssociation (FDA) begins. It can take up to 2.5 years for the FDA to authorise the launch of thedrug into the market.

7. Data warehouse and pharmaceutical supply chain integration

Most big pharmaceutical companies such as Eli Lilly (Cronin, 1997) and Pfizer (Richards, 1999)have initiated data-warehousing integration projects during the last 5 years. The aim of theseprojects is to harmonise data-warehousing tools throughout the international locations to supportthe easy retrieval and storage of data as well as being able to share information internally andexternally. As Bernard (1996) puts it ‘‘improving data flows internally within the organisation as

well as externally to R&D partners and regulatory agencies will dramatically shorten product-to-market times’’.For these projects to be successful, an in depth examination of the variety of databases

currently utilised, detailed data requirements and business processes analysis needs to beperformed for each phase of the supply chain (Birkhead & Schirmer, 1999). Also, the impact ofthe different regulatory constraints on the supply chain need to be understood. An example ofthese constraints is the achievement of Regulatory Compliance for Electronic Records andSignatures (21 CFR Par 11) imposed by the FDA. The difficulty of these projects increases withthe fact that most phases of the drug development are divided into different business departmentsacross countries with different cultures, languages and regulations that need to be also considered.In some industries, using data in the right way is not only a means of obtaining competitive

advantage, but also crucial to comply with regulatory constraints (Harrington, 1999), as the use ofthe wrong data could harm third parties (Leiheiser, 2001). This is particularly the case in thepharmaceutical industry, where using the wrong information could lead to human damage andother drastic consequences.Shortening time-to-market is becoming a critical IT function in the pharmaceutical industry

(Hibbard, 1998). Data-warehousing integration seems to be the key to success of improving thequality and speed of data sharing and communication across the different phases of the drugR&D supply chain. The data warehouse receives information from both internal and externalsources and compiles the information required by the different R&D phases. Departmentsworking in R&D may play both roles: internal information source and DW informationcustomer. Once the intellectual production is finalised with the regulatory preparation of thedossier, the Regulatory Agency receives the final product, which is in this case the marketingauthorisation application. Moreover, the Regulatory Agency also plays the role of externalinformation source when providing guidance and regulatory advice to the organisation. Theamount of data required and generated by the different phases of the drug R&D supply chain andthe cost involved are extensive. It is a two-way flow: data is made available by suppliers to thecustomer and also from the drug developers to the suppliers as a means to communicate anddistribute information. As currently no common and integrated repository of data is availableacross the R&D supply chain, unnecessary time is spent in communicating, exchanging and tryingto obtain the required information from the related source. Moreover, as different sources keeptheir own type of storage, data quality is also compromised. Again, an integrated data warehouse

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268 263

Page 6: Datawarehousing In Pharma

would not only speed the access to data but also improve the quality and consistency across theR&D supply chain.

8. Identified information sources

During the research for this study, a total of 32 different types of information sources have beenidentified. The variety of source types supports the initial suggestion that individual databasescould not handle the analysis of external and across-business units’ data, as information isprovided from a wide variety of sources and data formats. A data warehouse could then be thetool facilitating the integration of the different type of sources. The sources have been classifiedaccording to their relationship with the organisation. They are considered to be external if theyare not part of the company. Internal sources are departments, individuals or other offices that arepart of the company (R&D and non-R&D), independent of the country of their location. A totalof 13 main types of external sources providing a wide range of information to the R&D divisionswere identified. Also, 19 high-level types of internal sources have been identified, of which threecan be considered to be non-R&D information sources, five R&D Support information sourcesand 11 R&D intellectual production information sources. Non-R&D sources are departmentsinternal to the organisation that are outside the R&D domain but that interact with R&Ddepartments to exchange information. R&D support sources provide support or services to thewhole R&D supply chain to ensure that the organisation runs smoothly and to optimise futureand strategic decisions. This includes departments taking care of the R&D intellectual productionphases, and are classified by the two main divisions in the intellectual production: the Discoveryand the Development divisions.

9. Classification of decisions

Some of the potential decisions that a data warehouse could support during the drug R&Dprocesses have been identified with the analysis of the research results. These decisions could beclassified according to the four decision-making levels presented by Laudon and Laudon (1998)namely strategic, management control, knowledge-level and operational control. This classifica-tion also combines the way the decision affects the different levels of the supply chain (internaland external sources, the R&D divisions and the ultimate submission to the Regulatory Agency).The following organisational decisions are typical examples:

At a Strategic level, a data warehouse could provide information supporting decisions on whichconcepts or compounds should be researched in the future. It could also support decisions onwhich market to target for regulatory submissions. Historical information obtained from externalsources would be essential for this type of strategic decision, making a data warehouse the idealinformation/data storage to be used as a support system.

At a Management control level, a central repository of data would provide management withhigh-level project information and would support awareness of current project status.Consequently, the data warehouse would support management decisions on how to proceedwith departmental projects, being able to plan the time and required resources more accurately.

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268264

Page 7: Datawarehousing In Pharma

At a Knowledge-level, a data warehouse would support knowledge sharing and distributionacross departments and locations. It could also support general awareness, which could have anexcellent indirect impact on the individual and team performance. Time spent on rework would bereduced increasing the consistency across the organisation and optimising standardisation, whichare essential for quality improvement.Finally, at an Operational level, a data warehouse could provide details on key information

about previous projects. This data could optimise decisions on how to proceed with futureprojects of similar characteristics, avoiding rework, saving time and also reducing the cost of theresearch. Decisions on the type of population to be used for clinical trials could also be supportedby the data stored in the data warehouse providing information on previous projects’ approachand success.Table 1 presents a summary of the above and other decisions that a data warehouse could

support at the drug R&D supply chain.

10. Concluding comments

The drug R&D supply chain is a long and expensive process. A drug may take up to 17 yearsbefore regulatory authorities approve it for manufacture and marketing. During these years, thedifferent stages include the discovery phases of concept, screening, target identification, chemicallead and pharmacology as well as the development pre-clinical phases. Data warehousing canbecome a key success factor in the pharmaceutical industry. It is expected that an integrated data-warehousing approach will facilitate communication, improve the accuracy of the data andtherefore will shorten the expensive and lengthy life cycle of drug development. However, thecurrent lack of experience in this area makes such integration projects the biggest challenge for thepharmaceutical IT departments. During the research undertaken by the authors, 32 different typeof information sources were identified, from which 13 were external and 19 internal sources to theorganisation. This variety of source types suggests that no single database could handle theamount and diversity of data that these information sources provide. Only a data warehousewould have the capacity and capability to store, relate and analyse the information, so that it cansupport decision making at the different organisational levels.Some of the cost-related benefits associated with employing data warehouse technology would

include the reduction of IT storage tools and the required IT knowledge and support to maintainthem. Also, the reduction of time to access information, improvement of data quality andimproved productivity would support shortening the time to market. Shortening the drug time tomarket provides a pharmaceutical organisation with high competitive advantage against theircompetitors, as it increases the time that the drug is on the market with patent protection. Such anadvantage would also enhance relationships with the Regulatory Agency, which would be acustomer-related benefit. Some of the strategic benefits include improvements in knowledgesharing, accuracy and expansion of regulatory knowledge and general pharmaceutical marketawareness.Finally, data warehousing provides the organisation with the required information to support

decision making at the different organisational levels: strategic, management control, knowledge-level and operational control. In this paper, some examples of the type of decision making that a

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268 265

Page 8: Datawarehousing In Pharma

Table 1

Decision making at R&D

R&D supply chain Strategic Management

control

Knowledge-level

decision making

Operational

control

Information

sources

Internal Supports

decisions on

departmental

strategic

management,

such as location

in the building,

mergers between

departments, etc.

Supports analysis

of the

departmental

performance

Supports

knowledge

sharing across

departments

Supports tracking

of departmental

projects, which

supports

decisions on

which tasks to

perform next

External Supports

decisions on how

to proceed in the

future in order to

reduce the cost of

obtaining

information (e.g.

negotiating with

CROs, etc.)

Supports decision

making on how to

improve external

communications

with partners

Supports and

improves

knowledge

sharing across the

company as

previous

information

searches can be

shared from

project to project

and from office to

office

Supports the

selection of

CROs/hospitals

Supports choice

of pharmaceutical

consultants

providing reports

on their previous

quality of work

Supports strategic

decisions on how

to deal with

projects (increase

or decrease the

use of consultants

and CROs to

work on projects

or develop in-

house resources—

open new

company

laboratories, etc.)

Supports

management

control on

company

expenses in

obtaining

knowledge and

information

Supports the

decision on

external source

choice according

to quality of

information and

service

Production Research Supports

decisions on

which areas of the

market provide

an opportunity of

Supports

decisions at an

organisational

level on which

offices should

Supports

reducing time and

cost of projects by

knowledge

sharing across

Supports analysis

on the

performance of

the research tools

used and

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268266

Page 9: Datawarehousing In Pharma

Table 1 (continued)

R&D supply chain Strategic Management

control

Knowledge-level

decision making

Operational

control

competitive

advantage

deal with each of

the phases of the

research process

geographical

locations and also

across

departments

decisions on how

to improve them

Supports

decisions on

whether specific

drug research

projects are worth

developing any

further (providing

previous

company research

of abandoned

projects or other

pharmaceutical

companies

information)

Supports project

awareness and

planning (time

and resources)

Supports

knowledge on

regulatory

constraints

reducing

possibilities of

project failure

Supports

optimisation of

decisions on doses

administrated

in vitro and

in vivo and the

type of chemical

and biological

test that need to

be performed

Supports

decisions on

which other

concepts and

compounds

should be

researched

Supports the

clinical candidate

selection

providing detailed

information on

the compounds

Development Supports strategic

decisions on the

type of drug and

market the

company wants

to target in the

long/medium/

short term

Supports

decisions at an

organisational

level on which

offices should

deal with each of

the phases of the

development

process

Supports

reducing time and

cost of projects by

knowledge

sharing across

geographical

locations and also

across

departments. It

also increases the

knowledge on the

research phases

and previous

phases of the drug

development

Supports

optimisation of

decisions on doses

administrated

and type of

population for

clinical trials

Supports the

decision on when

to submit the

filing of a patent

Supports project

awareness and

planning (time

and resources)

Supports

knowledge on

regulatory

constraints

reducing

possibilities of

project failure

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268 267

Page 10: Datawarehousing In Pharma

DW could support have been identified during the analysis. A data warehouse could, e.g. providethe market information required to optimise the time the organisation decides to file thecompound of a patent. This is a very important decision, as it will have a direct impact on theamount of time the drug will be in the market with patent protection. The longer the drug will bein the market patent protected, the higher the organisational revenue will be.

References

Bernard, S. (1996). Make way for the next world. Pharmaceutical Executive, 50 (Advanstar Communications, Inc.)

Birkhead, N., & Schirmer, R. (1999). Add value to your supply chain. Transportation & Distribution, 15(9), 51.

Boar, B. (1996). Understanding data warehousing strategically. In R. Barquin, & H. Edelstein (Ed.). Building, using and

managing the data warehouse, 1997 (pp. 277–299), New Jersey: Prentice-Hall PTR.

Cameron, D. (1998). Do you really need a data warehouse? Direct Marketing, 61(2), 43–45.

Cronin, M. J. (1997). Getting drugs to market fast. Fortune, 10, 263.

Crowley, A. (1997). Delivering a healthy dose of sales data. PC Week, 14(49), 53–54.

Hackathorm, R. (1998). Rouging the Web for your data warehouse. DBMS, 11(9), 36–37.

Harrington, L. H. (1999). Focus on pharmaceuticals: Put good ideas to work. Transportation and distribution, 41(9), 41.

Hibbard, J. (1998). Research gains from IT boom—Top drug makers depend on technology for more than just speeding

production. Information Week, 700, 217.

Jonathan, R. (1999). Pfizer’s prescription for a healthy supply chain. Enterprise Systems Journal, 14(10), 26.

Kimball, R. (1996). The data warehouse toolkit. Practical techniques for building dimensional data warehouses. New

York: Wiley.

Kimball, R. (1998). Professional boundaries (the data warehouse manager’s job). DBMS, 11(8), 14–15.

Labio, W., Quass, D., & Adelberg, B. (1997). Physical database design for data warehouses. Proceedings of the

International Conference on data engineering, Dallas, USA.

Laudon, K. C., & Laudon, J. P. (1998). Management information systems. New Jersey: Prentice Hall.

Leiheiser, R. (2001). Data quality in health care data warehouse environment. Proceedings of the 34th Hawaii

International Conference on System Sciences, Hawaii, USA, IEEE computers.

Miles, M. B., & Huberman, A. M. (1984). Qualitative data analysis: A sourcebook of new methods. Newbury Park, CA:

Sage Publications.

Orr, K. (2000). Data warehousing technology. The Ken Orr Institute, URL: http://www.kenorrinst.com/dwpaper.html.

Richards, J. (1999). Pfizer’s prescription for a healthy supply chain. Enterprise Systems Journal, 14(10), 26.

Tanrikorur, T. (1998). Enterprise DSS architecture: a hybrid approach. DM review online, URL: http://

www.DMReview.com.

Thomsen, E. (1998). Smart decision support systems. Database programming design (pp. 59–61).

Professor Zahir Irani is the Director of Postgraduate Studies in the Department of Information Systems and

Computing, Brunel University (UK). Having worked for several years as a project manager, Professor Irani retains

close links with industry, and is a non-executive director to a leading engineering company. He consults for

international organisations such as Royal Dutch Shell Petroleum, DERA, BMW and Adidas, and has also taken part in

UK-Government funded trade missions to the Middle-East and Gulf region. Professor Irani leads a multi-disciplinary

group of International PhD students that research information systems evaluation and application integration.

S. Alshawi et al. / International Journal of Information Management 23 (2003) 259–268268