Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
1
1998 Executive Information Systems, Inc.
Executive Information Systems, Inc.
Architectural Evolution in DataWarehousing: The Coming of Distributed
Knowledge Management Architecture (DKMA)
By
Joseph M. Firestone, [email protected]
September 9, 1998
EIS Web Site: http://www.dkms.com
2
8The Dynamic Integration Problem is the problemof proactively and automatically monitoring andmanaging evolutionary change in datawarehousing systems without imposing atraditional and constraining “Top-Down”architecture.
8 It is the problem of providing managers of bothdata warehouses and data marts the power toinnovate, while still maintaining the integrationand consistency of the system.
Dynamic Integration and Data Warehousing
1998 Executive Information Systems, Inc.
3
Dynamic Integration andArchitectural Evolution in Data Warehousing
8 Data Warehousing is now a complex systemsintegration problem. A full-blown DataWarehousing System may encompass8 the following database servers:
8 The data warehouse
8 various data marts (department, function, orapplication-specific DSSs, using Relational, Multi-dimensional (MOLAP), or Column-based Servers)
8 One or more Operational Data Stores (ODSs)
8 One or more Data Staging Areas
1998 Executive Information Systems, Inc.
4
Dynamic Integration and ArchitecturalEvolution in Data Warehousing (Two)
8 the following application servers8 Web Servers
8 ETML Servers
8 Data Mining servers
8 Stateless Transaction Servers (e.g., MTS, Jaguar CTS,etc.
8 Business Process engines (e.g. Persistence Power-Tier,DAMAN InfoManager)
8 Document Servers
8 ROLAP Support Servers (e.g., MicroStrategy,Information Advantage)
8 Report Servers
8and various front-end OLAP and reporting tools
1998 Executive Information Systems, Inc.
5
Dynamic Integration and ArchitecturalEvolution in Data Warehousing (Three)
8 The Dynamic Integration problem in the contextof this complexity of components and interactionsis three-fold:8First, we need an integrated view of all server-
based assets;
8Second, we need to manage flows of data,information, and knowledge throughout this systemto maintain the common view in the face of changein form and content, and to distribute the system’sdata, information, and knowledge bases asrequired, and
1998 Executive Information Systems, Inc.
6
Dynamic Integration and ArchitecturalEvolution in Data Warehousing (Four)
8Third, we need such management to occurautomatically and without centralizing the systemso that the authority and responsibility for addingnew data and information to the system isdistributed.
8Automated dynamic integration is a capability notnow provided by data warehousing vendors. It isincreasing recognition of the need for thiscapability that drives architectural evolution in theDW system.
1998 Executive Information Systems, Inc.
7
Architectures forManaging DSS Integration
8 Here are six architectures for managing andviewing the problem of integration.
8 Top-Down Architecture(Inmon, Prism Solutions, Carleton Corporation, ETI)
8 Bottom-Up Architecture(Broadbase Information Systems, Sagent, ArdentDataStage)
8 Enterprise Data Mart Architecture (EDMA)(Informatica, Carleton)
8 Data Stage/Data Mart Architecture (DS/DMA)(Informatica, Carleton, Sagent)
1998 Executive Information Systems, Inc.
8
Architectures forManaging DSS Integration (Two)
8 Distributed Data Warehouse/Data MartArchitecture (DDW/DMA) (Sybase, Platinum (HP)Intelligent Warehouse)
8 Distributed Knowledge Management Architecture(DKMA) (The sole vendor targeting this architectureis DAMAN Consulting)
8 There are variations of each architectureincorporating an ODS
1998 Executive Information Systems, Inc.
9
Top-Down Architecture
8 The first Data Warehousing Systems architecture
8 Begins with Extraction, Transformation,Migration, and Loading (ETML) process
8 Establishes the data warehouse first, along withcentralized metadata repository
8 Data Marts are constituted from extracted andsummarized data warehouse data and metadata
8 The Data Warehouse has atomic layer anddetailed historical data
1998 Executive Information Systems, Inc.
10
Top-Down Architecture (Two)
8 The Data Warehouse uses Normalized E-RModels
8 Data Marts have highly and lightly summarizeddata
8 Integration is automatic as long as the disciplineof constituting data marts as subsets of the datawarehouse is maintained
8 Tools exist to generate data marts from the datawarehouse “by pushing a button”
1998 Executive Information Systems, Inc.
11
DataMarts A . . .N
LegacyApplications and
Data Sources
DataWarehouse
Data StagingArea
Extraction,Transformation, &Migration
Figure One -- Top-Down Architecture
Metadata Stores in DW and DMs
1998 Executive Information Systems, Inc.
12
Bottom-Up Architecture
8 The second Data Warehousing Systemsarchitecture
8 Became popular because the Top-downarchitecture took too long to implement, was oftenpolitically unacceptable, and was too expensive
8 Begins with ETML for one or more Data Marts
8 Requires no common data staging area
8 Uses Dimensional Data models for Data Marts
8 Uses Atomic, lightly summarized, and highlysummarized data
1998 Executive Information Systems, Inc.
13
Bottom-Up Architecture (Two)
8 Provides no common metadata
8 Constructs thedata warehouse incrementally overtime from data marts
8 Adopted initially by second generation toolvendors Informatica, Sagent, and Ardent
8 Met expectations in building data marts, but soonwas perceived as unacceptable for the long termbecause lacking common metadata, it is difficultto construct the data warehouse, and it also leadsto new “stovepipes” or”legamarts” over time.
1998 Executive Information Systems, Inc.
14
1998 Executive Information Systems, Inc.
Figure Two -- Bottom-Up Architecture
DataMarts A . . .N
LegacyApplications and
Data Sources
DataWarehouse
Data StagingAreas
Extraction,Transformation, &Migration
Metadata Stores in DW
and DMs
15
Enterprise DataMart Architecture (EDMA)
8 A response of the Bottom-up supporters to“legamart” argument
8 Begins with ETML for one or more data marts
8 Establishes a common staging area called aDynamic Data Store (DDS) for ET results,including a common or global metadata repository
8 Constructs EDMA before beginning data marts
8 EDMA includes enterprise subject areas, andcommon dimensions, metrics, business rules,sources, all represented in a logically common (butnot necessarily physically centralized) metadatarepository
1998 Executive Information Systems, Inc.
16
Enterprise DataMart Architecture (EDMA) (Two)
8 Uses Dimensional Data Models for Data Marts
8 Develops the data warehouse and data marts fromthe metadata repository, data marts and the DDSusing an incremental approach
8 Informatica’s PowerCenter Tool was the first toimplement this architecture. Informatica supportsmetadata management through monitoring andreporting mechanisms, not through an automatedprocess. Informatica makes some use of ObjectTechnology. Carleton is also close to thiscapability.
1998 Executive Information Systems, Inc.
17
1998 Executive Information Systems, Inc.
Figure Three -- EDM Architecture
DataMarts A . . .N
LegacyApplications and
Data Sources
DataWarehouse
Dynamic DataStore Staging
Area
Extraction,Transformation, &Migration
Shared Metadata Repository Layer
Application Servers
18
Data Stage/Data MartArchitecture (DS/DMA)
8 The same as EDMA with the important exceptionthat no physical enterprise-wide data warehouse isimplemented
8 Instead, the data warehouse is viewed as theconjunction of the data marts in the context of anEDMA-like metadata repository
8 The repository provides a common view ofenterprise DSS resources, but not necessarily anenterprise-wide view, because there is no
1998 Executive Information Systems, Inc.
19
Data Stage/Data MartArchitecture (DS/DMA) (Two)
8 guarantee that the conjunction of data marts willprovide access to enterprise-wide global attributes,as would a data warehouse
8 Leading tool providers are again Informatica andCarleton, and also Sagent which is advocatingDS/DMA in association with Ralph Kimball
1998 Executive Information Systems, Inc.
20
1998 Executive Information Systems, Inc.
Figure Four -- DS/DM Architecture
DataMarts A . . .N
LegacyApplications and
Data Sources
The Data Warehouseis the Conjunction of the Data Marts
Dynamic Data Store Staging
Area
Extraction,Transformation, &Migration
Shared Metadata Layer
Application Servers
21
Distributed Data Warehouse/DataMart Architecture (DDW/DMA)
8 Again similar to EDMA. Also:
8 Provides common view of metadata across theenterprise
8 Provides a logical database layer mapping aunified logical data model to physical tables in thevarious data marts and the data warehouse
8 Provides transparent querying of a unified logicaldatabase across data marts along with caching andjoining services.
1998 Executive Information Systems, Inc.
22
Distributed Data Warehouse/DataMart Architecture (DDW/DMA) (Two)
8 Leading tool providers are Informatica, Carleton,Sybase Adaptive Server, and HP (now Platinum)Intelligent Warehouse
8 These tools (except IW) are all offered as part ofSybase’s Warehouse Studio
8 This is the most adaptable of the architecturesdiscussed to this point, but it still reflects thelimitations of the relational viewpoint when itcomes to handling objects and processes, and itstill doesn’t support distributed and automatedchange capture and management
1998 Executive Information Systems, Inc.
23
1998 Executive Information Systems, Inc.Figure Five -- DDW/DM Architecture
DataMarts A . . .N
LegacyApplications and
Data Sources
The Logical/Physical Data
Warehouse Layer Maps a Unified
Data Model to the Physical Tables
in the Data Warehouse and
Data MartsDynamic Data Store Staging
Area
Extraction,Transformation, &Migration
Shared Metadata Layer
Application Servers
24
Distributed KnowledgeManagement Architecture (DKMA)
8 An evolving O-O/Component-based architecture
8 Top - Down and Bottom-Up architectures may beviewed as two-tier architectures utilizing clientsand local or remote databases
8 EDMA, DS/DMA, and DDW/DMA may beviewed as adding Middleware and Tuple layers toearlier architectures to provide the capability tomanage warehouse systems integration throughunified logical views, monitoring, reporting, andintentional DBA maintenance activity. But this
1998 Executive Information Systems, Inc.
25
Distributed KnowledgeManagement Architecture (DKMA) (Two)
8 form of management still doesn’t provideautomatic feedback of changes in one componentto others
8 DKMA may be viewed as adding an object layerto EDMA or to DDW/DMA to provide integrationthrough automated change capture andmanagement
1998 Executive Information Systems, Inc.
26
Distributed KnowledgeManagement Architecture (DKMA) (Three)
8 The object layer contains process distributionservices, an in-memory active, object model, andconnectivity to a variety of data store andapplication types. The layer requires anarchitectural component called an ActiveKnowledge Manager (AKM).
1998 Executive Information Systems, Inc.
27
1998 Executive Information Systems, Inc.Figure Six -- DKM Architecture
DataMarts A . . .N
LegacyApplications and
Data Sources
The Logical/Physical Data
Warehouse Layer Maps a Unified
Data Model to the Physical Tables
in the Data Warehouse and
Data Marts Dynamic Data Store Staging
AreaExtraction,
Transformation, &Migration
Shared Object Layer with AKM
Process Control ServicesActive In-Memory Object Model
Connectivity Services
PersistentObject Store
Application Servers
28
DKM Architectureand the AKM
8 An AKM provides Process Control Services, anObject Model of the Distributed KnowledgeManagement System (DKMS) (the systemcorresponding to the DKM architecture), andconnectivity to all enterprise information, datastores, and applications8Process Control Services:
8In memory proactive object state management andsynchronization across distributed objects
8Component management
8Workflow management
8Transactional multithreading 1998 Executive Information Systems, Inc.
29
DKM Architectureand the AKM (Two)
8In-memory Active Object Model/Persistent ObjectStore is characterized by:8Event-driven behavior
8DKMS-wide model with shared representation
8Declarative business rules
8Caching along with partial instantiation of objects
8A Persistent Object Store for the AKM
8Reflexive Objects
8Connectivity Services should have:8Language APIs: C, C++, Java, CORBA, COM
8Databases: Relational, ODBC, OODBMS,hierarchical, network, flat file, etc.
1998 Executive Information Systems, Inc.
30
DKM Architectureand the AKM (Three)
8Wrapper connectivity for application software:custom, CORBA, or COM-based.
8Applications include all the categories mentioned inthe earlier discussion of the Dynamic Integrationproblem, whether these are mainframe, server, ordesktop - based.
8 The DKM Architecture and the AKM provide thesolution to the Dynamic Integration Problem,because only the DKMA among the precedingarchitectures supports distributed proactivemonitoring and management of change in the webof data warehouse, data mart, web information
1998 Executive Information Systems, Inc.
31
DKM Architectureand the AKM (Four)
8 servers, component transaction servers, datamining servers, ETML servers, other applicationservers, and front-end applications comprisingtoday’s Enterprise DSS/Data WarehousingSystem.
1998 Executive Information Systems, Inc.
32
ODS Variations
8 Each of the architectures covered may vary withthe addition of an Operational Data Store (ODS)
8 According to Inmon: “An ODS is a collection ofdata containing detailed data for the purpose ofsatisfying the collective, integrated operationalneeds of the corporation . . . The ODS is:8subject-oriented,
8 integrated,
8volatile,
8current-valued,
8detailed.”
1998 Executive Information Systems, Inc.
33
ODS Variations (Two)
8 The ODS is like a data warehouse in its first twocharacteristics, but it is like an OLTP system in itslast three characteristics. Its purpose is to supportoperational, tactical decisions
8 The workload of an ODS involves four kinds ofprocessing: loading data, updating, accessprocessing, and DSS-style analysis across manyrecords.
8 The four types of ODS processing are the sourceof difficulties in optimizing ODS processing. It isdifficult to optimize performance over all fourtypes.
1998 Executive Information Systems, Inc.
34
ODS Variations (Three)
8 Look at the above architectures in relation to theODS. It is clear that an architecture that willsupport both DSS and OLTP-style processing isneeded in order to optimally integrate the ODSinto the broader data warehousing architecture. Inparticular,8process control services will be very important for
the OLTP-style of processing we find in the ODS.
8Also, distribution of ODS objects across multipleservers will help ODS performance.
1998 Executive Information Systems, Inc.
35
ODS Variations (Four)
8Finally, in-memory processing in distributedAKMs can do much to upgrade performance in adistributed ODS.
8 Of course, only one of the above architecturescan provide these capabilities for the ODS: theDKM Architecture.
1998 Executive Information Systems, Inc.
36
DKM Architectureand Data Mining
8 A key emerging capability in DKMS and datawarehousing systems is Knowledge Discovery inDatabases (KDD) or Data Mining.
8 The key mechanism for KDD is the data miningserver.
8 Here are some difficulties with current data miningserver products:8It’s difficult to incorporate new data mining
algorithms, and therefore keep pace with newdevelopments coming out of the research world;
1998 Executive Information Systems, Inc.
37
DKM Architectureand Data Mining (Two)
8Many products require that data must be transportedto proprietary data stores before data mining canoccur;
8Models produced by the data mining algorithms arenot freely available to power users unless they usethe data mining tool itself,
8It is difficult to incorporate validation criteria notinitially incorporated in the data mining tool intothe KDD process,
8There are few “open architecture” commercial datamining tools.
1998 Executive Information Systems, Inc.
38
DKM Architectureand Data Mining (Three)
8 To solve these problems a product class called AnAnalytical Data Mining Workbench (ADMW)should be developed.
8 The ADMW needs:8Easy and convenient encapsulation of new
algorithms into object model classes;
8Capability to mine data from any data source in theenterprise;
8Incorporation of analytical models into an objectmodel repository;
8A modifiable validation model, 1998 Executive Information Systems, Inc.
39
DKM Architectureand Data Mining (Four)
8Integration of legacy data mining applications withthe ADMW.
8 An ADMW with these capabilities would meet allof the difficulties specified above.
8 The DKM Architecture can help in developing theabove because:8New algorithms can be encapsulated in objects
through the “wrapping” capabilities of the AKM;
8Data can be brought into the AKM’s in-memoryobject model for data mining without relocating itfrom its data store (“chunks,” partial instantiation),
1998 Executive Information Systems, Inc.
40
DKM Architectureand Data Mining (Five)
8Data mining can be performed by executing theanalytical models in memory on data chunks andpartially instantiated objects;
8Analytical Models produced by an AKM -basedapplication would be placed in an object modelrepository where they can be accessed by PowerUsers;
8Customized validity criteria could be added bymodifying the validation model in the repository,because the validation model is just another objectwhose attributes and methods can be modified ;
1998 Executive Information Systems, Inc.
41
DKM Architectureand Data Mining (Six)
8Legacy data mining applications could beintegrated using AKM connectivity services whichwould “wrap” them.
8 When viewing the above, keep in mind that thereis no ADMW with the above capabilities atpresent. Data Mining is a rapidly growing field,but the market niche represented by the ADMW isempty.
8 On the other hand, there are software tools that canbe used as a foundation to rapidly develop theADMW as a facility within the AKM.
1998 Executive Information Systems, Inc.
42
DKM Architectureand Software Tools
8 To implement DKM Architecture in a DKMS youneed the full range of tools now used to create datawarehousing systems. In addition though, you needadditional tools for the AKM component(including the ADMW facility and the ability tointegrate the ODS into the DKMS). These include:8An object modeling RAD environment providing
extensive process control services and connectivity( e.g. DAMAN's InfoManager, TemplateSoftware’s Enterprise Integration Template (EIT),Forte, a combination of Ibex's DAWN workflowproduct along with its Itasca Active Object
1998 Executive Information Systems, Inc.
43
DKM Architectureand Software Tools (Two)
8Database, a combination of Rational Rose,Persistence Power-Tier; and Iona’s Orbix)
8Technology for constructing software agents toproactively monitor components of the DKMS (e.g.CA Unicenter TNG, ObjectSpace’s Voyager,Persistence Power-Tier, DAMAN’s InfoManager)
8An OODBMS to serve as a persistent objectrepository for the AKM component (ObjectStore,Objectivity/DB. Jasmine, Versant, Itasca)
1998 Executive Information Systems, Inc.
44
45
Back-Up Slides
4Distributed Knowledge Management Systems (DKMS
4Why DKMS?
4What is the Knowledge Management System (KMS)?
4The Knowledge Base and Knowledge
4The Knowledge Management Process and KnowledgeManagement
4Data, Information, Knowledge, and Wisdom
4Organizational Knowledge
46
Distributed KnowledgeManagement Systems (DKMS)
8 A DKMS is a system that manages the integration of distributed objectsinto a functioning whole producing, maintaining, and enhancing abusiness knowledge base.
8 A business knowledge base is the set of data, validated models,metamodels, and software used for manipulating these, pertaining tothe enterprise, produced either by using a DKMS, or imported fromother sources upon creation of a DKMS. A DKMS, in this view,requires a knowledge base to begin operation. But it enhances its ownknowledge base with the passage of time because it is a self-correctingsystem, subject to testing against experience.
8 The DKMS must not only manage data, but all of the objects, objectmodels, process models, use case models, object interaction models,and dynamic models, used to process data and to interpret it to producea business knowledge base. It is because of its role in managing andprocessing data, objects, and models to produce, enhance, and maintaina knowledge base that the term Distributed Knowledge ManagementSystem is so appropriate.
1998 Executive Information Systems, Inc.
47
Why DKMS?
8Other reasons for adopting the term DKMS include:
8business knowledge production and management iswhat business intelligence is all about;
8DKMS plays off DBMS, and therefore capitalizes on afamilar term while favorably contrasting with it, i.e.knowledge management is clearly better than mere datamanagement;
8DKMS also highlights the point that data is notknowledge, but only a part of it;
8“DKMS” is a product/results-oriented name likely toappeal to business decision makers (that is, they getvaluable and valid knowledge that they can use to gaincontrol and produce results);
1998 Executive Information Systems, Inc.
48
What is the KnowledgeManagement System (KMS)?
8 The Knowledge Management System (KMS) is the on-going, persistentinteraction among agents within a system that produces, maintains, andenhances the system's knowledge base. This definition is meant toapply to any intelligent, adaptive system composed of interactingagents.
8 An agent is a purposive, self-directed object.
8 Knowledge Base will be defined in the next section.
8 In saying that a system produces knowledge we are saying that it (a)gathers information and (b) compares conceptual formulationsdescribing and evaluating its experience, with its goals, objectives,expectations or past formulations of descriptions, or evaluations.
8 Further, this comparison is conducted with reference to validationcriteria. Through use of such criteria, intelligent systems distinguishcompeting descriptions and evaluations in terms of closeness to thetruth, closeness to the legitimate, and closeness to the beautiful.
1998 Executive Information Systems, Inc.
49
What is the KnowledgeManagement System (KMS)? (Two)
8 In saying that a system maintains knowledge we are saying that itcontinues to evaluate its knowledge base against new information bysubjecting the knowledge base to continuous testing against itsvalidation criteria. We are also saying that to maintain its knowledge, amore complex system must ensure both the continued dissemination ofits currently validated knowledge base, and continued socialization ofintelligent agents in the use and content of its knowledge base.
8 Finally, in saying that a system enhances its knowledge base, we aresaying that it adds new propositions and new models to its knowledgebase, and also simplifies and increases the explanatory and predictivepower of its older propositions and models. That is, one of thefunctions of the KMS is to provide for the growth of knowledge.
1998 Executive Information Systems, Inc.
50
The Knowledge Base and Knowledge
8 A system's knowledge base is: the set of remembered data; validatedpropositions and models (along with metadata related to their testing);refuted propositions and models (along with metadata related to theirrefutation); metamodels; and (perhaps, if the system produces such anartifact) software used for manipulating these, pertaining to the systemand produced by it.
8 A knowledge management system, in this view, requires a knowledgebase to begin operation. But it enhances its own knowledge base withthe passage of time because it is a self-correcting system, and subjectsits knowledge base to testing against experience.
8 Finally, the emphasis on a system's knowledge base, rather than itsknowledge, recognizes that an identification of knowledge as individualconceptions, propositions, or models is inconsistent with the reality thatacceptance of a piece of information into a system's body of knowledgeis dependent on the background knowledge already within theknowledge base. This background knowledge is used to filter andinterpret the information being evaluated.
1998 Executive Information Systems, Inc.
51
The Knowledge Base and Knowledge (Two)
8 This definition of knowledge base contrasts with a popular definition ofknowledge as "justified, true belief." The definition does agree with thenecessity of justification as a necessary condition for knowledge; but itinsists that justification be specific to the validation criteria used by asystem to evaluate its descriptions and evaluations. The definition alsoagrees that knowledge is a particular kind of belief, provided that beliefextends beyond cognition alone, to evaluation.
8 The biggest discrepancy with the popular definition is in not requiringthat justified beliefs be "true." Truth can be used as a regulating idealby a system producing descriptive knowledge. "Right" can be used as aregulating ideal by a system producing evaluative or normativeknowledge. But the system in question can never say for sure that aproposition or a model within its knowledge base is "true," or "right;"but only that it has survived refutation by experience better than itscompetitors, and therefore that the system "believes" it is true or right.
1998 Executive Information Systems, Inc.
52
The Knowledge Base and Knowledge (Three)
8 So instead of knowledge as "true, justified belief," the position taken hereis that knowledge equals justified belief that some conceptual formulation,fact, or evaluation, is true or right as the case may be.
8 In a very real sense, a system's knowledge is the analytical network ofpropositions and models constituting the knowledge base. It istherefore, just for convenience, that one may refer to a particularproposition or model as something a system "knows," because it knowsthat "something," only if one assumes that numerous unspecifiedbackground propositions and models are also known by it.
1998 Executive Information Systems, Inc.
53
The Knowledge Management Processand Knowledge Management
8The Knowledge Management Process (KMP) is an on-going persistent interaction among human-based agentswho aim at integrating all of the various agents,components, and activities of the knowledge managementsystem into a planned, directed process producing,maintaining and enhancing the knowledge base of theKMS.
8Knowledge Management is the human activity within theKMP aimed at creating and maintaining this integration,and its associated planned, directed process.
1998 Executive Information Systems, Inc.
54
The Knowledge Management Processand Knowledge Management (Two)
8 A good way to look at the human activity called knowledgemanagement is through the concept of the Use Case. In a use case ahuman-based agent, within the KMS, called an actor, participates in theKMP to get an outcome from the KMS that has value for the actor.The KMP can be represented as a set of Business Process Use Caseseach classified within one of four business sub-process categories:planning, acting, monitoring, and evaluating. A way of decomposingknowledge management activity then, is in terms of the use cases thatconstitute it.
8 The set of all use cases aimed at creating and maintaining theintegrated, planned, directed process producing, enhancing andmaintaining the KMS knowledge base, is an alternative characterizationof knowledge management. The set of these use cases represents all ofthe organizational knowledge management activity of the actorsmaking use of the KMS through the KMP. In other words, the set ofuse cases is what we mean by knowledge management in a humansystem.
1998 Executive Information Systems, Inc.
55
Data, Information,Knowledge, and Wisdom
8 What is the difference between data, information, knowledge, andwisdom?
8 To begin with, organizational data, information, knowledge, andwisdom, all emerge from the social process of an organization, and arenot private. In defining them, we are not trying to formulate definitionsthat will elucidate the nature of personal data, information, knowledge,or wisdom. Instead, to use a word that used to be more popular indiscourse than it is at present, we are trying to specify intersubjectiveconstructs and to provide metrics for them.
8 A datum is the value of an observable, measurable or calculableattribute. Data is more than one such attribute value. Is a datum (or isdata) information? Yes, information is provided by a datum, or by data,but only because data is always specified in some conceptual context.At a minimum, the context must include the class to which the attributebelongs, the object which is a member of that class, some ideas aboutobject operations or behavior, and relationships to other objects andclasses.
1998 Executive Information Systems, Inc.
56
Data, Information,Knowledge, and Wisdom (Two)
8 Data alone and in the abstract therefore, does not provide information.Rather, information, in general terms, is data plus conceptualcommitments and interpretations. Information is data extracted, filteredor formatted in some way (but keep in mind that data is alwaysextracted filtered, or formatted in some way).
8 Knowledge is a subset of information. But it is a subset that has beenextracted, filtered, or formatted in a very special way. Morespecifically, the information we call knowledge is information that hasbeen subjected to, and passed tests of validation. Common senseknowledge is information that has been validated by common senseexperience. Scientific knowledge is information (hypotheses andtheories) validated by the rules and tests applied to it by some scientificcommunity.
8 Wisdom, lastly, has a more active component than data, information, orknowledge. It is the application of knowledge expressed in principles toarrive at prudent, sagacious decisions about conflict situations.
1998 Executive Information Systems, Inc.
57
Organizational Knowledge
8 Organizational knowledge in terms of this framework is informationvalidated by the rules and tests of the organization seeking knowledge.The quality of its knowledge then, will be largely dependent on thetendency of its validation rules and tests to produce knowledge thatimproves organizational performance (the organization’s version ofobjective knowledge).
8 From the viewpoint of the definition given of organizationalknowledge, what is an organization doing when it validates informationto produce knowledge? The validation process is an essential aspect ofthe broader organizational learning process, and that validation is aform of learning. So, though knowledge is a product and not a processderived from learning, knowledge validation (validation of informationto admit it into the knowledge base) is certainly closely tied to learning,and depending on the definition of organizational learning, may beviewed as derived from it.
1998 Executive Information Systems, Inc.