Upload
jose-monteiro
View
220
Download
0
Embed Size (px)
Citation preview
8/3/2019 AdrienneTannenbaum2 - Congres BI
1/119
2002Database Design Solutions, Inc.40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metadata Solutions areHere!
And we are finallyready.
8/3/2019 AdrienneTannenbaum2 - Congres BI
2/119
22002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Remember
o Data affects thequality andextensiveness of
informationo We need information
to widen ourknowledge
o Knowledge is whatmakes us intelligent
8/3/2019 AdrienneTannenbaum2 - Congres BI
3/119
32002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Whats In Store
for this Afternoon?
o The evolution ofintelligence
o Metadata a full
discussion
o The Metadata Solution
o Metadata SolutionArchitectures
o Bringing the MetadataSolution to YourOrganization
8/3/2019 AdrienneTannenbaum2 - Congres BI
4/119
42002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Case Study 1
Data and Metadata
o Pharmaceutical Company
o Quite distributedo Functionally
o Financiallyo Information Creation
8/3/2019 AdrienneTannenbaum2 - Congres BI
5/119
52002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
FDA Requirement
o How much does it cost to release a drug?
o How do you justify your prices
o Costs involved:o Researcho Manufacturing
o Clinical Trials
o Marketingo Sales
o Employee Costs
8/3/2019 AdrienneTannenbaum2 - Congres BI
6/119
62002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
How Do We Do That?
o Each division keeps their own data
o Each division stores data in a different way
o Each database is designed differentlyo Not all data exists everywhere
o Some data by researcher
o Some data by employee
o Some data by drugo Some data by manufacturing site
o Etc.
8/3/2019 AdrienneTannenbaum2 - Congres BI
7/119
72002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Putting the
Data Together
o What data do wehave?
o What does it mean?
o Where is it?
o How did it get there?
o How do I get it?
8/3/2019 AdrienneTannenbaum2 - Congres BI
8/119
82002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Their Solution
o A data warehouse
o Major extract andtranslation efforts
o Solid metadata supporto Reporting Tools did nothave appropriatemetadata coverage
o Separate customizedmetadata store
o Metadata access beyondthe data warehouse
Ive got it!
8/3/2019 AdrienneTannenbaum2 - Congres BI
9/119
92002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Case Study 2 IntegratingData Without Metadata
o Internet ServiceProvider
o Another month,
another acquisitiono Another acquisition,
another billing system
o Another billing system,more customer data
o More customer data,more integrationproblems
8/3/2019 AdrienneTannenbaum2 - Congres BI
10/119
102002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Their System
Architecture
Billing System 2
Billing System 1
Billing System n
Std.Translation
Std.Extract
Customer DW
8/3/2019 AdrienneTannenbaum2 - Congres BI
11/119
112002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Their
Environmental Snapshot
o Big Web Believers
o No Formal SystemsDevelopment
Methodology
o Anti-Big Picture
o DevelopmentSupport Focused onTechnical
Infrastructure
8/3/2019 AdrienneTannenbaum2 - Congres BI
12/119
122002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Their Information Picture
o Which numbersshould we use?
o Why are they alldifferent?
8/3/2019 AdrienneTannenbaum2 - Congres BI
13/119
132002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Their
Information Dilemmas
o Definition of Customer
o Definition of Service
o Product Variationso Revenue Calculations
8/3/2019 AdrienneTannenbaum2 - Congres BI
14/119
142002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Their Metadata Solution
Data Warehouse
Development
Tool
Development Library
DW Reporting Tool
Metadata Store
Standard Reports
8/3/2019 AdrienneTannenbaum2 - Congres BI
15/119
152002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Lessons Learned
o Metadata can be moreimportant than data
o Metadata does not have
to all be stored in oneplace
o Metadata can bestandardized and shared
o Information Managementis a practice not amethodology
8/3/2019 AdrienneTannenbaum2 - Congres BI
16/119
162002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Case Study 3Beginning a Painful Process
o A Major Health CareProvider
o HIPAA reporting
requirements
o Data everywheremetadata in someplaces
8/3/2019 AdrienneTannenbaum2 - Congres BI
17/119
172002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Getting Started
o HIPAA ComplianceTaskforce
o Begin with a data
search
8/3/2019 AdrienneTannenbaum2 - Congres BI
18/119
182002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
How Painful Is It?
o Still assessing theamount of timeneeded to LOCATE
the datao Cannot assess yet the
amount of timerequired to
INTERPRET andCONNECT the data
8/3/2019 AdrienneTannenbaum2 - Congres BI
19/119
192002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Learning Lessons?
o Maybe.
o Determine whatmetadata you need
o Create the metadataBEFORE you need it
o What comes first, thepain or the prevention?
8/3/2019 AdrienneTannenbaum2 - Congres BI
20/119
202002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Beyond Data
More 5 Questions
1. What metadata do I have?
2. What does it mean?
3. Where is it?4. How did it get there?
5. How do I get it? (Go get it for me)
8/3/2019 AdrienneTannenbaum2 - Congres BI
21/119
2002Database Design Solutions, Inc.40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Information
TheUltimate
Objective
8/3/2019 AdrienneTannenbaum2 - Congres BI
22/119
222002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Wheres the info?
o Locating information
o Interpreting
informationo Validating
interpretations
o Finalizing information
8/3/2019 AdrienneTannenbaum2 - Congres BI
23/119
232002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
A Formal Definition?
o The set of available data,whether internal orexternal, numeric,
textual, graphic, visual,audio, and so on, thatwhen combined with anyset of manual ormechanized processesand integrated with pre-existing informationassists the decisionmaker.
Information
8/3/2019 AdrienneTannenbaum2 - Congres BI
24/119
242002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The 7 Rules
wrt Information
1. Information can takeon virtually anyformat (as long as it
is identifiable,storable, andretrievable)
2. Information shouldnot be restricted to
fulfill the roleidentified by itscreating process
3. Not all data isintended to becomeinformation
4. Information shouldnot be derived fromother informationall information
should be derivedfrom data
8/3/2019 AdrienneTannenbaum2 - Congres BI
25/119
252002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The 7 Rules wrt
Information - contd
5. Informationsusability is directlyrelated to its
relationship to basicdata
6. The averageinformationconsumer will go to
extreme lengths tofind it, get it, andapply it
7. Unless theinformationconsumer is aware of
the informationsexistence, it is highlyunlikely that even thebest information will
be used optimally
8/3/2019 AdrienneTannenbaum2 - Congres BI
26/119
262002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Typical
Information Requests?
o Who are our threelargest customers?
o How long did it take
to roll-out our latestrelease of ProductXYZ?
o How many of our
executives were hiredvia employmentagencies?
8/3/2019 AdrienneTannenbaum2 - Congres BI
27/119
272002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Information vs. Knowledge
o Today, nothing becomes information unlesssome processing or context, usually of aparticular specialty, is applied to it.
o Information evolves.
Art
AttendanceList
ReportCard
Student Name
Course Name
8/3/2019 AdrienneTannenbaum2 - Congres BI
28/119
282002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Information EvolutionJ 5 0 4 3
Part Inventory Guide
Characters without Context
Context
InformationJ 5 0 4 3
Part Number for Windshield Wiper
8/3/2019 AdrienneTannenbaum2 - Congres BI
29/119
292002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Corporate Information
o Information Technology (IT) adds some newtypes of context
o Applications/Systems
o Tools
o Packages
o Databases
8/3/2019 AdrienneTannenbaum2 - Congres BI
30/119
302002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Informations Role Today
o Providing the Answer
o Supplying the Trend
o Pointing Us to a Solutiono Getting Us There
o Assisting the Completion of a Task
o Solving a Problem
8/3/2019 AdrienneTannenbaum2 - Congres BI
31/119
312002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Role of Information
o Helps us do our jobs!
(or at least it should)
P tti
8/3/2019 AdrienneTannenbaum2 - Congres BI
32/119
322002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Putting
Information Together
o First we locate ito What types of
information are welooking for?
o Then we identifywhich pieces we wanto There are often
supplemental piecesof information
involved
o We connect andrelate as we interpret
8/3/2019 AdrienneTannenbaum2 - Congres BI
33/119
332002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Types of Information
o Structured
o Database contents
o Documents
o Models
o
o Unstructured**
o Raw data, raw text
o Program Code
o Graphicso Sound
o Web sites**
o .
**What makesthese thingsunstructured?
C ti Diff t
8/3/2019 AdrienneTannenbaum2 - Congres BI
34/119
342002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Connecting Different
Types of Information
o Traditionally it was amatter of matchingcommon values
o Gets complicatedwhen the data doesnot equate or when itresides in physically
different types ofstructures
I f ti
8/3/2019 AdrienneTannenbaum2 - Congres BI
35/119
352002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Information
Integration Styles
o DatabaseConsolidation
o Data Management
o Data Modeling
o Data Warehousing
o Corporate Intranets
8/3/2019 AdrienneTannenbaum2 - Congres BI
36/119
362002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Database Consolidation
CorporateAccountingMaster
FieldSales DB Wholesale
Sales DBCustomerHistory DB
WholesaleSales DB
Corporate Sales
8/3/2019 AdrienneTannenbaum2 - Congres BI
37/119
372002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
CRMA Special Type
FieldSales DB Customer
History DB
WholesaleSales DB
CorporateAccounting
CRM Package
Local Sales Data Mart
Document ManagementSystem
8/3/2019 AdrienneTannenbaum2 - Congres BI
38/119
382002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Data Management
o Data Modeling
o Enterprise Data Modelso Standards
8/3/2019 AdrienneTannenbaum2 - Congres BI
39/119
402002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Data Warehousing
Data Warehouse Result
Application DB2
Application DB 3
CleanMergeTransitionAggregate
Application DB 1
Th T i l
8/3/2019 AdrienneTannenbaum2 - Congres BI
40/119
412002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Typical
Data Warehouse -contd
Data Warehouse Result
End-User Query ToolsMulti-DimensionalAnalysis
ToolsDirectoryData
WarehouseMiddleware
8/3/2019 AdrienneTannenbaum2 - Congres BI
41/119
422002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Data Warehousing
o What happened?
o Data Accuracy?
o Data Credibility?
o Integrating Unplanned Data?
8/3/2019 AdrienneTannenbaum2 - Congres BI
42/119
432002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Corporate Intranets
o Best Way to answer Question Number 1:o What data do I have?
o Many also answered Question Number 2:o What does it mean?
o From whose perspective?
o Fewer answered Question Number 3:o Where is it?
o Even fewer answered Question Number 5:o How do I get it (Go get it for me!)?
8/3/2019 AdrienneTannenbaum2 - Congres BI
43/119
2002Database Design Solutions, Inc.40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metadata
The reason weare here
8/3/2019 AdrienneTannenbaum2 - Congres BI
44/119
452002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
What is Metadata?
o Its one of those
things we need andwant but just cant
get our fingers on..
R b
8/3/2019 AdrienneTannenbaum2 - Congres BI
45/119
462002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Remember
Metadatas Role
J 5 0 4 3
Part Inventory Guide
Characters withoutContext
Context
InformationJ 5 0 4 3Part Number forWindshield Wiper
Instance Data
Metadata
Information
8/3/2019 AdrienneTannenbaum2 - Congres BI
46/119
472002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
This Example of Instance Data
is What Most People Relate to:
Employee SS#: 387-33-0595
Employee Name: Jacqueline ReedEmployee Start Date: 03-09-1994Department Code: 476Payroll Code: 256
8/3/2019 AdrienneTannenbaum2 - Congres BI
47/119
482002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Employee SS#: 387-33-0595
Employee Name: Jacqueline Reed
Employee Start Date: 03-09-94
Department Code: 476
Payroll Code: 256
Table Name: EMPLOYEE
EMPLOYEE_SSNO, X(9) Primary Key
EMPLOYEE_LAST_NAME, X(20) Index
EMPLOYEE_FIRST_NAME, X(15)
EMPLOYEE_DEPT_CODE, X(3) Foreign Key
EMPLOYEE_PAY_CODE, X(3) Foreign Key
Table Name: DEPARTMENT
DEPT_CODE, X(3) Primary Key
DEPT_NAME, X(15)PAYROLL_CODE, X(3) Foreign Key
InstanceData example
Is this Metadata?
Or this?
8/3/2019 AdrienneTannenbaum2 - Congres BI
48/119
492002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Intermediate Processes
CustomerMaster
Customer Number: 8938
Customer Name:Smith, RussellTable Name:CustomerColumn Name:Customer_No
Column Name:Customer_Name
Edit
SalesTransactionsETL
ETL
Sales
DataWarehouse
8/3/2019 AdrienneTannenbaum2 - Congres BI
49/119
502002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Generic Metadata Role
o Making sure that the covered metadata is:
o Identifiable (What metadata do we have?)
o Understandable (What does it mean?)
o Locatable (Where is it?)
o Traceable (How did it get there?)
o Accessible (How do I get it?)
R l ti
8/3/2019 AdrienneTannenbaum2 - Congres BI
50/119
512002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Relating
Metadata to Data
o Gives us Information
o Provides answers tothe 5 Questions
from eachperspective
o Data will be:o Identified
o Defined
o Locatedo Sourced
o Accessed
Th Lib A l
8/3/2019 AdrienneTannenbaum2 - Congres BI
51/119
522002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
o Instance Data: TheBook
o Metadata: The CardCatalog Entry
The Library Analogy
Comparing Data to Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
52/119
2002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Information/MetadataConnection
Why this allhas a
purpose
8/3/2019 AdrienneTannenbaum2 - Congres BI
53/119
542002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Data and Metadata
Data + Metadata =Information
8/3/2019 AdrienneTannenbaum2 - Congres BI
54/119
552002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Data/Metadata/Information
o What data do I have?
o What does it mean?
o Where is it?o How did it get there?
o How do I get it?
o Product_price
o The gross charge for
the firms offerings,before any discounts
o SalesDatabase
Product Table
o Update_ProductProgram
o Product_price Query
Data:$46.30
8/3/2019 AdrienneTannenbaum2 - Congres BI
55/119
562002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Data/Metadata/
Informationcontd
o What data do I have?
o What does it mean?
o Where is it?
o How did it get there?
o How do I get it?
o Entity Name
o Any person or organizationwith whom we do business
o Marketing Data Model
mktg.erx File
ERWin Modeling Tool
o David Myers ModelerdmyersUserid
o ERWin Executable
Data:Customer
8/3/2019 AdrienneTannenbaum2 - Congres BI
56/119
2002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Metadata Solution
What We StriveFor
What is a
8/3/2019 AdrienneTannenbaum2 - Congres BI
57/119
582002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
What is a
Metadata Solution?
o An organized andintegrated set ofrelated metadata
o Logically connected,often physicallyseparate
o Common access
points and methods
The Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
58/119
592002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Metadata
Solution Role
o Providing a unifiedviewof the metadata
that we need toidentify, access, andinterpret theinformation that weneed to do our jobs
Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
59/119
602002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Metadata
Solution Role - contd
o Puts some purposeand order arounddisparate metadata!
8/3/2019 AdrienneTannenbaum2 - Congres BI
60/119
612002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Job/Role Variability
o Depending on who I
am, I can use thesame metadatasolution for adifferent purpose
Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
61/119
622002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Metadata
Solution Components
o Metadata
o Metamodel(s)
o Meta-metamodelo Metadata stores
o Software
8/3/2019 AdrienneTannenbaum2 - Congres BI
62/119
632002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Component Roleso Metadata You should know that by now!
o Metamodel The organized collection of metadata,representing a particular perspective (source, function,or other metadata grouping)
o Meta-Metamodel The organized collection ofmetamodels, used as gateway info in a MetadataSolution
o Metadata Store where the metadata is physically kept
o Software Everything required to create, access,update, and display metadata, populate itsoccurrences, and share/exchange them with othermetadata solution architectural components
8/3/2019 AdrienneTannenbaum2 - Congres BI
63/119
642002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Component Variability
o Metamodel Standard, Customized, or Built-from-scratch?
o Metadata Store none, one, or many?
Standard or custom built?o Software Herein lies the rub!
o Metadata Display (with or without the web)
o Metadata Access and Processing
o Metadata Exchange and Update
o Metadata Security
Types of
8/3/2019 AdrienneTannenbaum2 - Congres BI
64/119
652002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Types of
Metadata Solutions
o Metadata Repository(ies)
o Enterprise Portals
o Standalone or Distributed Metadata Storeso XML-Based Metadata Exchange
o Information Directories
o Others
o Integrated Metadata Solution Architectures
o a combination of the above
8/3/2019 AdrienneTannenbaum2 - Congres BI
65/119
2002Database Design Solutions, Inc. 40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The MetadataRepository
Option 1
The Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
66/119
672002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Metadata
Repository
o The vendor-neutral metadata store
o Can be virtual
o Can store metadata gateways to external
metadata
o Integrated
o Standard Architecture
The Generic
8/3/2019 AdrienneTannenbaum2 - Congres BI
67/119
682002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Generic
Metadata Repository
UnderlyingDatabase
Repository Metamodel
Repository Software
The Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
68/119
692002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
The Metadata
Repository Database
o Stores metadata contents as well asaddressing and execution data necessary toget to external metadata
o Open (commercial) or closed (proprietary)
8/3/2019 AdrienneTannenbaum2 - Congres BI
69/119
702002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Repository Metamodel
o The Meta-metamodel
o Correlates with theunderlying database
o Tracks relationshipso Handles security
o Tool focused
o RepositoryBlueprint
8/3/2019 AdrienneTannenbaum2 - Congres BI
70/119
712002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Repository Software
o Go Get It Routines
- APIs, RPCs
o Repository Policieso Repository
Templates
o Repository Utilities
8/3/2019 AdrienneTannenbaum2 - Congres BI
71/119
722002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Distributed Repositories
o Not that simple with todays tools
o Special distribution considerations dependon tool architecture
o Application Logic
o Presentation Logic
o Database Logic
Client/Server
8/3/2019 AdrienneTannenbaum2 - Congres BI
72/119
732002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Client/Server
Processing Layers
Server
DatabaseServer
Execute database
Database extract access program
Application Logic
Display results
in GUI format
Point and click
request for
information
Presentation Logic
Database Processing
Database
Calls
DBMS Access Language
8/3/2019 AdrienneTannenbaum2 - Congres BI
73/119
2002Database Design Solutions, Inc.40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Enterprise Portals
Option 2
8/3/2019 AdrienneTannenbaum2 - Congres BI
74/119
752002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Enterprise Portals
o InformationDistribution Hubs
o Web Browser or
Customized URLfront-end
o Customized Searchand Retrieve
o Tag or URL based
8/3/2019 AdrienneTannenbaum2 - Congres BI
75/119
762002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Enterprise Portals - contd
o Fancy name for acompanys intranet?
o Accurate name for an
organizedmetamodel-basedintranet
o Able to get to the
same stuff with thesame request everytime
8/3/2019 AdrienneTannenbaum2 - Congres BI
76/119
772002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Portal Componentso Meta-metamodel (of
sorts)
o Taken care of whenyou buy portalsoftware
o Metadata/datatransfer mechanism
o Portal display/front-end
Metadata/Data
8/3/2019 AdrienneTannenbaum2 - Congres BI
77/119
782002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
XML Based Portals
o Most Popular
o XML is used toexchange queries
and query results toand from the portalsoftware
o Metadata wrappers
(tags) are generatedthroughout the portalarchitecture
8/3/2019 AdrienneTannenbaum2 - Congres BI
78/119
792002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
An XML Based Portal
DesktopWord
ProcessingTool
DocumentManagement
System
CorporateLibrarySystem
XML-BasedPortal
Access
PUBLICATIONS
AUTHOR = Adrienne
Tannenbaum
Standard
Publication
DTD
Another XML
8/3/2019 AdrienneTannenbaum2 - Congres BI
79/119
802002Database Design Solutions, Inc.
40 Morristown Road, Suite 2BBernardsville, NJ 07924www.dbdsolutions.com
Based Portal
Delimited
File
Common
Metadata
Store
DBMS
Catalog
Program
Library
Shared
Metadata
Store
Extract/Translate
Process
Extract/
Translate
Process
XML
Parser
Extract
Process
Extract/
Translate
Process
Application
Development
Application
Design
Spreadsheets
Word Processing
Simple databases
Graphics
Desktop Office
Productivity Tools
Intranet Communication
(HTML)
8/3/2019 AdrienneTannenbaum2 - Congres BI
80/119
2002Database Design Solutions, Inc.40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metadata Stores
Option 3
8/3/2019 AdrienneTannenbaum2 - Congres BI
81/119
82
2002Database Design Solutions, Inc.40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metadata Stores
o Standalone or Distributed
o Most Popular
o Easiest to implement (depending on design)o Home-grown databases which follow a
home-grown metamodel
Distributed
8/3/2019 AdrienneTannenbaum2 - Congres BI
82/119
83
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Distributed
Metadata Stores
o Plan first, Design next, Implement last
o Each metadata store must be relatedto theothers
o Common metamodel
o Different ways of populating each metadatastore
Distributing
8/3/2019 AdrienneTannenbaum2 - Congres BI
83/119
84
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Distributing
Metadata Stores - contd
o Common Metamodel Variations
o Same constructs, different metadatainstances
o Different constructs, different metadatainstances
Common
8/3/2019 AdrienneTannenbaum2 - Congres BI
84/119
85
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metamodel Population
SubjectArea
DataElement
DataStore
Application
ApplicationComponents
Categorizes
Resides In
Accessed By
Consists of
Common
Metamodel
Relational
MetadataStore 1
SubjectArea
DataElement
Data
Store
Application
ApplicationComponents
Categorizes
Resides In
Accessed By
Consists of
CommonMetamodel
Relational
MetadataStore 2
SubjectArea
DataElement
Data
Store
Application
ApplicationComponents
Categorizes
Resides In
Accessed By
Consists of
CommonMetamodel
Relational
MetadataStore 3
Common
8/3/2019 AdrienneTannenbaum2 - Congres BI
85/119
86
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metamodel Population
Object
Object
Type
Metamodel
Data
Store
Physical
Location
Integrating Meta-Metamodel Meta-metaData Store
8/3/2019 AdrienneTannenbaum2 - Congres BI
86/119
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
XML Based Exchange
Option 4
XML Based
8/3/2019 AdrienneTannenbaum2 - Congres BI
87/119
88
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metadata Exchange
o Has been a silverbullet for quite some
time
o Leave everything asis, just pass it backand forth?
o Standard XML
documents totranslate from andto?
eXtensible
8/3/2019 AdrienneTannenbaum2 - Congres BI
88/119
89
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Markup Language
o The W3C calls XML a common syntax forexpressing structurein data
o XML is a method of putting structure in a text
file
o XML is a markup specification language andXML files contain both data and metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
89/119
90
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
More important.
o XML is ametalanguage whichdescribes a markup
language
o XML is anarchitecture not anapplication
8/3/2019 AdrienneTannenbaum2 - Congres BI
90/119
91
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Why The Popularity
o It is a standardized way to exchange data
o It is Flexible You can do a lot with it
o It is Highly adaptable
o Custom usage is not dependent on acommittee
o It permits the automated exchange of databetween two organizations via the web
8/3/2019 AdrienneTannenbaum2 - Congres BI
91/119
92
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
How XML works
o XML is composed of markup and content
o Markup is used to describe content at thedesired level of detail
o XML has nameable tags that are used to definewhat the content is
o An XML document must be well-formed
8/3/2019 AdrienneTannenbaum2 - Congres BI
92/119
93
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
How XML works - contd
o If a Document Type Declaration (DTD) exists andthe XML document conforms to the DTD the XMLdocument is said to be valid.
o Very Simplified DTD exampleo
o
8/3/2019 AdrienneTannenbaum2 - Congres BI
93/119
94
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Simplified XML Sample
File
File
type
has
File has
File Type
XML Based
8/3/2019 AdrienneTannenbaum2 - Congres BI
94/119
95
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Information Exchange
o Choices in scopeaffect the overallmetadata solution
architecture
8/3/2019 AdrienneTannenbaum2 - Congres BI
95/119
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Information Directories
Option 5
8/3/2019 AdrienneTannenbaum2 - Congres BI
96/119
Information
8/3/2019 AdrienneTannenbaum2 - Congres BI
97/119
98
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Directories - contd
o Directories are a webbrowser without the
access
o A portal that does notfetch
o Optional use of theweb
o Retail repositoriesusually offer thiscapability
Metadata Categorization
8/3/2019 AdrienneTannenbaum2 - Congres BI
98/119
99
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Within a Directory
o Categorization of a set of scoped information(could be metadata or data)
o Association of the categories with the
information
o Identification of the categorized information
o Optional access of the metadata-identified
information
8/3/2019 AdrienneTannenbaum2 - Congres BI
99/119
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Other Metadata
Solutions
Just as an aside..
Other
8/3/2019 AdrienneTannenbaum2 - Congres BI
100/119
101
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Metadata Solutions
o File ManagementSystems
o Database
ManagementSystems
o Object OrientedComponent Libraries
o Etc.
File
8/3/2019 AdrienneTannenbaum2 - Congres BI
101/119
102
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Management System
o A file typeis afiletypeis a file type
o Why are file types soimportant to FileManagement
Systems?
File typesguarantee
reusability and
standardization
The File
8/3/2019 AdrienneTannenbaum2 - Congres BI
102/119
103
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
System Metamodel
File
File Type
AccessProcedure
Has Associated
Has Associated
Database
8/3/2019 AdrienneTannenbaum2 - Congres BI
103/119
104
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Management Systems
o Cant forget that catalog!
o Physical definitions of all defined databases
o Database is related to all of its components
Object Oriented
8/3/2019 AdrienneTannenbaum2 - Congres BI
104/119
105
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Component Libraries
o Potential standardization of the way data isdefined and accessed in an organization
o Standard software components
o Standard object classes
o Who has access to the library?
8/3/2019 AdrienneTannenbaum2 - Congres BI
105/119
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Integrated Metadata
Solution Architectures
Tying them all
together
Integrated Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
106/119
107
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Solution Architecture
o A Unified Set of Software
o Shared Metadata Store
o Reflective of Common Metamodel
o Active Component Connections
8/3/2019 AdrienneTannenbaum2 - Congres BI
107/119
108
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
What Goes on Inside?
DBMS Metamodel
Metadata Store
DBMS Catalog
ETLApplication
Metadata Store
Planning Analysis
Construction Application
Development
MetamodelIntegrated Development
Environment
Decision Support
Metamodel
Data Warehouse
Reporting Tool Shared Metadata
Repository
DataWarehouse
What Goes
8/3/2019 AdrienneTannenbaum2 - Congres BI
108/119
109
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
on Inside? - contd
Metamodel
MetamodelConstruct
MetadataStore
RepositoryProcedure
Contains
Resides In
Implemented As
Accessed by
Illustrated Metadata
S
8/3/2019 AdrienneTannenbaum2 - Congres BI
109/119
110
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Solution
Microsoft SQL ServerCustom Schema
ModelMart
System CatalogsDB2OracleMSSQLTandem
Windows NTUser client
iexplore.exe
Batchscheduler
Cool:GenPublic Interface
Tables
Windows NTAdministrative client
Maintenance
0
ModelMart API
Load program
MTS
COMcomponents
MS InternetInformation
Server 4
PresentationActive Server
Pages
Metadata Solution
A E l
8/3/2019 AdrienneTannenbaum2 - Congres BI
110/119
111
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Access Example
Get library collectionSet libraryCollection = ISession.Objects.Collect(,
LibraryTypeCode)\
Loop through libraries
For Each libraryReference in libraryCollectionDebug.Print libraryReference.Name
Get diagram collection for librarySet diagramCollection = Isession.Objects.Collect(
libraryReference.Handle, DiagramTypeCode)
Loop through diagramsFor Each diagramReference in diagramCollection
Debug.Print vbTab & diagramReference.Name
Introducing Metadata
8/3/2019 AdrienneTannenbaum2 - Congres BI
111/119
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Introducing Metadata
Solutions to Your
Organization
8/3/2019 AdrienneTannenbaum2 - Congres BI
112/119
113
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Most Common Mistake
o Making theinvestment
o Purchasing/developin
g the solutiono Just letting it loose!
o Without thinking abouthow it will changethings
Thi T C id
8/3/2019 AdrienneTannenbaum2 - Congres BI
113/119
114
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Things To Consider
o Changes in daily activitieso How will the metadata be actively maintained?
o Will application or database development
processes need to change?o Changes in organization structure
o Who will be using the metadata solution?
o Who will administer the metadata solution?
o Who will enhance or maintain the metadatasolution?
o Etc.
A i R di
8/3/2019 AdrienneTannenbaum2 - Congres BI
114/119
115
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Assessing Readiness
o They wanted metadata
o You gave them metadata
o What will they do with it?
o When?
o How?
o How will you ensure their objectives?
B i R l
8/3/2019 AdrienneTannenbaum2 - Congres BI
115/119
116
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Basic Rules
o Make standardized metadata part of allinformation creation
o Information access should be the result of
metadata-based searches
o Industry standards should have a positiveimpact
o Needing common metadata stores shouldbecome of less and less importance
Finally
8/3/2019 AdrienneTannenbaum2 - Congres BI
116/119
117
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Finally
o Be sure the originalobjectives are met
o Now
o As new architecturalcomponents are added
o Always rememberwhy the metadatasolution was created
to begin with
Q ti ?
8/3/2019 AdrienneTannenbaum2 - Congres BI
117/119
118
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
Questions?
Wh t W C d
8/3/2019 AdrienneTannenbaum2 - Congres BI
118/119
119
2002Database Design Solutions, Inc.
40 Morristown Road, Suite 2B
Bernardsville, NJ 07924www.dbdsolutions.com
What Weve Covered
o The evolution ofintelligence
o Metadata a full
discussiono The Metadata Solution
o Metadata SolutionArchitectures
o Bringing the MetadataSolution to YourOrganization
F M I f
8/3/2019 AdrienneTannenbaum2 - Congres BI
119/119
For More Info..
o Metadata Solutions (2001, Addison Wesley)
o www.dbdsolutions.com
o Publications
o Discussion Group
o Upcoming Conferences (USA)
o Metadata Conference, San Antonio, Texas
o Email:o [email protected]