Copyright © HSG/IWI/R.Winter, 2000. Supplied by The CRM-Forum at http://www.crm-forum.com

Prof. Dr. Robert Winter University of St.Gallen

Copyright © HSG/IWI/R.Winter, 2000

The Current and Future Role of

Data Warehousing in

Corporate Application Architecture

CRM Forum Resources http://www.crm-forum.com

1

ABSTRACT

Data Warehouse systems are widely accepted as a new middleware layer between operational applications and deci-

sion support applications, thereby decoupling systems focussed on efficient handling of business transactions from

systems focussed on efficient support of business decisions. However, the increasing importance of channel-

oriented, horizontal applications leads to the question whether the Data Warehousing approach can be extended to

cover this new type of applications or whether alternative integration approaches are more appropriate. As a founda-

tion for that analysis, a general application architecture model along the dimensions ‘process’, ‘product’, and ‘func-

tion’ is proposed. Traditional, vertical applications, decision support applications, and the Data Warehouse system

are located within that model. Cross-product applications and channel-oriented, horizontal applications are inte-

grated into the model. Being another important phenomenon in current information logistics, operational data stores

are integrated into the model. The extended role of the Data Warehouse system that reflects the existence of cross-

product and horizontal applications as well as the necessity of on-line data integration by operational data stores is

discussed, and respective Data Warehouse systems are integrated into the application architecture. The paper closes

with research questions that result from the proposed, extended role of Data Warehouse systems in corporate appli-

cation architecture.

1. INTRODUCTION

Data Warehouse systems1 are established as a new middleware layer of applications. Such a middleware layer is

necessary because the direct, individual access of Decision Support applications to data of operational, transaction

oriented applications has proved to be technically or economically infeasible: Data quality problems and complex

integration requirements usually make it impossible to supply consistent, integrated data real-time to various Deci-

1 In analogy to the pair of terms ‘Database’ – ‘Database system’, we use the term ‘Data Warehouse system’ forthe entirety of applications and databases that are needed to utilize a Data Warehouse (i.e. the databases that

2

sion Support applications. Even if technically feasible, the development and maintenance of mn interfaces between

m Decision Support applications and n transactional applications cannot be economically useful. As an intermediate

systems layer, the Data Warehouse system is decoupling Decision Support applications and operational applications,

thereby reusing integration mechanisms and derived data for various Decision Support applications and allowing

maintenance to be focussed on few, well-defined interfaces. The role of the Data Warehouse system as middleware

layer is illustrated by Figure 1.

Extrac-tion /

Integr.

Inte-gration

Inte-gration

Operational applicationse.g.

Order processing (retail)Order processing (wholesale)

Materials managementHuman resources management

Financials

Operational applicationse.g.

Order processing (retail)Order processing (wholesale)

Materials managementHuman resources management

Financials

Extrac-tion Extrac-

tion

Extrac-tion

Extrac-tion

Extrac-tion /

Integr.

Extrac-tion /

Integr.

Extrac-tion /

Integr.

Extrac-tion /

Integr.

Extrac-tion /

Integr.

Data Warehouse

Aggre-gation /Filtering

Aggre-gation /Filtering

Aggre-gation /Filtering

Decision Support applicationse.g.

MarketingControlling

Extrac-tion

Extrac-tion

Aufbe-reitung

Extrac-tion

Aufbe-reitung

Extrac-tion

Extrac-tion

Inte-gration

Inte-gration

Inte-graztion

Decision Support applicationse.g.

MarketingControlling

ExternalData

Figure 1: Data Warehouse system as a middleware layer

The foundation of a Data Warehouse are operational applications or, more accurately, the databases of a company’s

operational applications. These databases cover a wide range from flat files to hierarchical, relational or even object-

oriented database systems. In addition, often external data (e.g. panel / market research data) are to be integrated into

the Data Warehouse [9]. On this foundation, interface systems and extraction procedures are based which extract,

transform, and load data from operational applications into the Data Warehouse. After usually being integrated and

corrected in the Data Warehouse system, consistent data are then filtered and / or further aggregated to be trans-

comprise subject-oriented, integrated, historical, and aggregate data) [1, p.17]. ‘Data Warehousing’ denotes theentirety of activities that are associated with the development and operations of the Data Warehouse system.

3

ferred into data management modules of Decision Support applications or directly accessed by Decision Support ap-

plications. The application architecture for the Data Warehouse system in Figure 1 has been simplified; a detailed

description can be found in Section 4.

In this paper, the current and future role of Data Warehousing as an important component of corporate application

architecture is analyzed. In Section 2, a model of the corporate application architecture is proposed, and the Data

Warehousing system is positioned in that model. Recently, more and more channel-oriented and product-

independent, ‘horizontal’ applications (e.g. Call Center, WWW portal, WAP portal) have been introduced by many

large companies. These additional components of the corporate application architecture are introduced into the

model in Section 3. Another actual development in Data Warehousing is the introduction of operational data stores.

Operational data stores are introduced into the application architecture model in Section 4. Both horizontal applica-

tions and operational data stores lead to an extended role of the Data Warehouse system which is discussed in Sec-

tion 5. The paper closes with research questions that result from the proposed, extended role of Data Warehouse

systems in corporate application architecture.

2. APPLICATION ARCHITECTURE MODEL

Although both scientific community and practitioners agree on the importance of Information Systems architecture

[7, p.59], a general, precise architectural model is still lacking [13, p.13]. E.g., Scheer defines IS architecture as a

model of Information Systems components and their relationships [12, p.3], while Österle defines IS architecture as

a high-level model of organization, business functions, business data, software systems, and databases [10, p.26]

which can be used as a foundation for organizational and application development [10, p.69].

In analogy to the usage of the term ‘architecture’ in construction industries [7, p.58][15, p.2], Information Systems

architecture should comprise specifications and documentation of applications and their relationships with regard to

all relevant aspects (e.g. data flow, control flow, roles and responsibilities) as well as appropriate construction rules.

Allowing to document the most important corporate applications and their relationships, the application architecture

model is an important component of the Information Systems architecture. In many companies, the application ar-

chitecture has grown over a long period of time. In the worst case, applications have been introduced unsystematic-

ally. If at least a rudimentary architectural planning has been maintained, application development is usually ori-

4

ented towards functional specialization (e.g. order processing, materials management, financials) or towards busi-

ness areas (i.e. usually products or product groups). As an example, a retail bank usually has developed separate ap-

plications for loans, cash deposits, custody, etc., and a telecommunications company usually has developed separate

applications for fixed line telephony, cellular telephony, data communications, Internet access, etc. In most cases,

every functional area or business area maintains separate customer data and product data. As a consequence, cus-

tomers receive different bills from different applications, have to report inquiries or address changes to different

contact points, and are bothered by marketing campaigns for products that they already bought. From the viewpoint

of the company, redundant data create inefficiencies and inconsistencies, cross-selling potentials cannot be ex-

ploited, and customer knowledge is distributed over numerous applications.

A simple application architecture model represents applications as cubes along the dimensions ‘function’, ‘product

(group)’, and ‘process’. The dimension ‘function’ lines up the various functional areas of the corporation (e.g. order

processing, materials management, financials). The dimension ‘product (group)’ lines up the various divisions or

product groups of the corporation (e.g. loans, cash deposits, custody). The dimension ‘process’ represents the course

of business processes (e.g. information request, negotiation, contract, fulfillment/clearing, archiving).

Process

Partner-ApplikationPartner-Applikation

Partner-Applikation

Product / product group

Function(e.g. order

processing,production

planning,materials

management)Application for product group / division A

Application for product group / division BApplication for product group / division C

Application for product group / division D

Figure 2: Simple application architecture model comprising vertical applications

5

Figure 2 illustrates the positioning of most traditional operational applications in such a three dimensional space.

Most applications comprise modules that cover all functional aspects of a (more or less) complete business process

for a specific product, product group, or division [3, p.2-3]. Hence, the application architecture comprises a rela-

tively small number of components that can be designated ‘vertical’ applications due to their optical appearance in

our model.

EinzellebensversicherungPartn

er ap

plicat

ion

Produ

ct app

licati

on

Process

Product / product group

Function(e.g. order

processing,production

planning,materials

management)Application A

Application BApplication C

Application D

Figure 3: Application architecture model comprising vertical applications and cross-product applications

A first important extension of a ‘vertical’ application architecture is the transfer of selected, cross-product functions

into dedicated applications. E.g., customer management should be transferred from vertical applications and inte-

grated into one single, cross-product ‘partner management application’ to avoid the problems of redundant customer

data management and create opportunities for cross-selling and customer bonus programs.2 Similar effects can be

created by transferring all configuration and pricing functions into a single, cross-product ‘product management ap-

plication’ or by transferring all reporting functions into a single, cross-product reporting application [6][8][14]. Al-

though the data managed by such cross-product applications are processed by all other applications and thereby be-

2 These advantages will only be effective until the application architecture is changing significantly – e.g. by amerger or an acquisition – and the integration process has to be repeated.

6

come ‘reference data’, they should be treated as operational data [2, pp.141-142]. The extended application archi-

tecture model resulting from the separation of cross-product applications is illustrated by Figure 3.

In an application architecture comprising vertical and cross-product operational applications that create transaction

data, the Data Warehouse system represents an intermediate layer that derives subject-oriented information for Deci-

sion Support applications. To avoid the development and maintenance efforts for numerous individual interfaces

between operational applications and Decision Support applications, source data are integrated into a (logically)

centralized, consistent database. This database is then used by all Decision Support applications as a single source of

consistent data. An application architecture model that is extended by the Data Warehouse system and Decision

Support applications is illustrated by Figure 4.

Externaldata

Product / product group

Function

Einzellebensversicherung

Process

Extraction, transformation,integration, correction

Data Warehouse

Selection, aggregation,supplementation

Operational applications

Business Intelligence(= Decision Support)

applications

Figure 4: Data Warehouse system as intermediate layer of the application architecture

For simplification, the Data Warehouse system is represented as an unstructured layer in Figure 4. However, in real-

ity usually several architectural components are created for data extraction, data transformation, data integration,

data correction, data quality control, data load, data security, data selection and filtering, data aggregation, etc.

Moreover, the Data Warehouse system does not have to be implemented as a single, centralized system, but can also

be partially or entirely implemented in a decentralized or even virtual way (see e.g. [1, pp.24-25]).

7

3. HORIZONTAL APPLICATIONS

While the transfer of functions like customer data management or product configuration and pricing from vertical

applications into dedicated cross-product applications has started in the 1990ies, channel-specific functions have not

been transferred into dedicated channel-specific applications until recently. This is due to the fact that a strong de-

mand for multi-channel (i.e. face-to-face, letter-based, phone-based, and electronic) access to corporate applications

is associated with increased mobility and the recent advent of electronic business and widespread access to the

Internet. Channel-specific applications integrate access and/or distribution functions which are specific for a certain

channel, but may be implemented identically for different products or product groups. If access and/or distribution

channels have to be flexibly assigned to products or services, channel-specific functions, product-specific functions,

and cross-product functions should be implemented in separate applications.

The clustering of channel-specific functions into applications is determined by the respective access media: Custom-

ers demand access to information and products/services by phone, by Internet, by cellular phone and WAP, by tra-

ditional written communication, by using self-service terminals (e.g. ATMs), or via sales representatives. As a con-

sequence, vertical applications have to be complemented by alternative, channel-specific application add-ons like

Call Center support, WWW portal, WAP portal, Letter Center/Document Management support, ATM support, and

traditional, inhouse transaction applications. These applications may differ not only by supporting a different access

and/or distribution channel, but also by different security mechanisms.

In the application architecture model, channel-specific applications are represented by cubes that comprise various

products for selected functions and for a certain portion of the underlying business processes. Due to their optical

appearance, channel-specific applications can be designated as ‘horizontal’ applications. The positioning of hori-

zontal applications in the application architecture model is illustrated by Figure 5.

8

Product / product group

Function

Einzellebensversicherung

Process

Application APartn

er ap

plicat

ion

Produ

ct app

licati

on

...

WAPportal

WWW portal

ATMapplication

Call Center application

Application B

Application CApplication D

Figure 5: Application architecture model comprising vertical applications, cross-product applications, and

horizontal applications

In principle, the introduction of horizontal applications has no influence on the positioning of Data Warehouse sys-

tems in the corporate application architecture: Like vertical applications and cross-product applications, horizontal

applications are operational, transaction-oriented applications and, therefore, create operational data. The same in-

termediate layer that transforms operational data created by vertical applications and cross-product applications, can

be used to transform data created by horizontal applications into input for Decision Support applications.

4. OPERATIONAL DATA STORES

The transformation of operational data into input for Decision Support applications by a Data Warehouse system

consumes a certain amount of time and creates non-volatile, aggregate information. However, often there is a strong

demand to access integrated and consistent data from operational applications in real-time. E.g., Call Center appli-

cations need access to both actual customer data from a ‘partner management application’ as well as actual product-

related functions from vertical applications. Since access in real-time is not possible for integrated, consistent Data

Warehouse data (and since these data have usually been aggregated), the concept of operational data stores has been

introduced [2, pp.143-144][5, p.20]. In an operational data store, transaction data from operational applications are

9

integrated in (nearly) real-time. By operational data stores, an additional architectural component is introduced

whose data are subject-oriented, actual, volatile, detailed, integrated, and, most important, accessible in real-time [3,

p.4][4, pp.1-2]. Hence, data managed by operational data stores have characteristics that differ from data managed

by operational applications as well as from data managed by the Data Warehouse. The different characteristics of

data managed by operational applications, by operational data stores, and by the Data Warehouse are summarized by

Table 1. It becomes evident that operational data stores can be positioned „between“ operational applications and

the Data Warehouse system.

Operationalapplication

Operationaldata store

DataWarehouse

Basicorientation

transaction informationobject

informationobject

Tim ereferences

actual actual actual andhistorical

Access read-w rite read-w rite read-onlyAggregationlevel

detailed detailed mostlyaggregated

Integrationlevel

isolated integrated integrated

Accessibility real-tim e real-tim e delayed (due toderivation)

Table 1: Comparison of operational applications, operational data stores, and the Data Warehouse

Whether it is really necessary to introduce operational data stores in addition to a Data Warehouse system should be

decided in each case carefully: If the focus is on providing actual data for reporting, often cross-product applications

or more frequent Data Warehouse updates are sufficient. If, on the other hand, operational applications exchange

operational data, i.e. if read-write access to integrated, actual data is necessary in real-time, the introduction of an

additional architectural layer is inevitable [5, p.20].

5. THE CHANGING ROLE OF THE DATA WAREHOUSE SYSTEM

Similar to the Data Warehouse system that is decoupling operational applications and Decision Support applications,

operational data stores are decoupling vertical and horizontal operational applications [3, p.5]. It is usually much

10

more efficient to integrate a large number of operational applications by a small number of operational data stores

than to develop and maintain a large number of individual interfaces between applications.

Since both the Data Warehouse and the operational data store are decoupling application layers, it was proposed that

operational data stores should be implemented as a part of the Data Warehouse system or that the Data Warehouse

system should be directly utilized for operational services like Customer Relationship Management or e-Commerce

(‘Closed loop model’, e.g. [11]). However, it was shown in Section 4 that operational data stores are fundamentally

different from the Data Warehouse system due to real-time processing needs, read-write access to data, etc. Hence,

the Data Warehouse system and operational data stores have to be represented by two different architectural compo-

nents. An application architecture model that is extended by operational data stores is illustrated by Figure 6.

Product / product group

Function

Verticaloperationalapplications

(integration withregard to

transactions)

Data S

taging Dat

a S

tagi

ngOpe-ratio-nal

DataStores

Process

Horizontaloperationalapplications

(integration withregard to

access channels)

Extraction, transformation,integration, correction

Data Warehouse

Selection, aggregation,supplementation

Business Intelligence(= Decision Support)

applications

Figure 6: Data Warehouse system and operational data stores as different intermediate layers of the applica-

tion architecture

By using operational data stores, an efficient ‘local’ closed-loop approach can be implemented on the operational

level, i.e. between vertical and horizontal operational applications. By using the Data Warehouse system, efficient

information supply may be implemented between operational applications and Decision Support applications. Since

information ‘backflows’ take their way indirectly (i.e. by actions of decision makers) into the operational systems

11

and not directly through the Data Warehouse system, however, only an open-loop can be implemented on the deci-

sion support level.

Although operational data stores and the Data Warehouse systems are different in their architectural positioning and

their functionality, many synergies can be utilized in systems development and operations:

• Data managed by operational data stores are already integrated to some extent and are accessible in real-time.

Therefore, they are an ideal data source for the Data Warehouse system. Although it is neither possible nor use-

ful to feed the Data Warehouse exclusively from operational data stores [4, p.4], every opportunity should be

utilized to source data from operational data stores without having to replicate data extraction and integration

procedures redundantly.

• The meta data infrastructure which has to be developed and maintained for Data Warehouse operations should

be reusable to a large extent for the development and operations of operational data stores. On a meta level, ex-

perience shows that data integration for operational applications and information logistics for decision support

do not differ significantly.

6. CONCLUSIONS

In this paper, we introduced various types of operational applications, Decision Support applications, the Data

Warehouse system, and operational data stores as complementary components of a general application architecture

for large companies. While being different in nature, the Data Warehouse system and operational data stores share

certain roles so that synergies in development and operations should be exploited.

Since experience from many large Data Warehousing projects has just began to be integrated and transformed into

general findings and methodologies and since operational data stores have introduced into information logistics

systematically only recently, many research questions still have to be addressed conceptually and empirically:

• What extent of meta data (and meta data management) of Data Warehouse systems can be reused for opera-

tional data stores? Should proven concepts from Data Warehousing be adapted for application integration pur-

poses, or can they be extended resulting in a general approach to meta data management for information logis-

tics?

12

• Can models and methodologies for Data Warehouse development be adapted for the development of opera-

tional data stores? Can these models and methodologies be extended resulting in a general methodological ap-

proach to application integration?

• In contrast to most of the vertical applications in large service companies, horizontal applications are often im-

plemented by using standardized packages (e.g. Customer Relationship Management applications, E-commerce

applications, Call Center applications). As a consequence, there is a chance to identify reference models at

least for the operational levels of information logistics.

• Does the permanent organization of Data Warehousing differ from the permanent organization of application

integration?

REFERENCES

[1] Böhnlein, M., Ulbrich-vom Ende, A.: Grundlagen des Data Warehousing – Modellierung und Architektur.

[2] Devlin, B.: Data Warehouse – from Architecture to Implementation. Addison-Wesley: Reading u.a. 1997.

[3] Imhoff, C.: The Corporate Information Factory, DM Review, December 1999,http://www.dmreview.com/editorial/dmreview, 29-03-2000.

[4] Imhoff, C.: The Operational Data Store: Hammering Away, DM Review, July 1998,http://www.dmreview.com/editorial/dmreview, 29-03-2000.

[5] Kimball, R., Reeves, L., Ross, M., Thornthwaite, W.: The Data Warehouse Lifecycle Toolkit: Expert Methodsfor Designing, Developing and Deploying Data Warehouses. John Wiley & Sons: New York u.a. 1998.

[6] Koch, G.: Ein Datenmodell als Schlüssel einer flexiblen Gestaltung von Versicherungsprodukten, in: Versi-cherungswirtschaft, 16/1993, 1052-1055, 1993.

[7] Lehner, F., Maier, R., Hildebrand, K.: Wirtschaftsinformatik: theoretische Grundlagen; Hanser: Wien 1995.

[8] Leist, S., Winter R.: Nutzung generischer Produktmodelle im Finanzdienstleistungsbereich am Beispiel des Er-gebniscontrolling, Wirtschaftsinformatik, 40, 4, 281-289, 1998.

[9] Möller, F.: Data Warehouse als Warnsignal an die Datenschutzbeauftragten. In: Datenschutz und Datensicher-heit (DuD), 22, 10, 555-560, 1998.

[10] Österle, H., Brenner, W., Hilbers, K.: Unternehmensführung und Informationssystem: der Ansatz des St.GallerInformationssystem-Managements, 2. Aufl.; Teubner: Stuttgart 1992.

[11] OVUM Evaluates: CRM Strategies: Technology Choices for the Customer-focussed Business; OVUM Ltd.,London 1999.

[12] Scheer, A.-W.: Architecture of Integrated Information Systems, Springer: New York, Berlin 1992.

[13] Schmalzl, J.: Architekturmodelle zur Planung der Informationsverarbeitung von Kreditinstituten; Physica: Hei-delberg 1995.

[14] Schönsleben, P., Leuzinger, R.: Innovative Gestaltung von Versicherungsprodukten: flexible Industriekonzeptein der Assekuranz, Gabler: Wiesbaden 1996.

[15] Sinz, E.J.: Architektur betrieblicher Informationssysteme, in: Rechenberg, P., Pomberger, G. (Eds.): Handbuchder Informatik; Hanser: München 1997.