Data warehousing theory and modelling techniques Building Dimensional Models

Data warehousing theory Data warehousing theory and modelling techniquesand modelling techniques

Building Dimensional Building Dimensional ModelsModels

1. Matrix Method for getting1. Matrix Method for getting started. started. 2. 2.

1.11.1 Build the Matrix Build the Matrix

1.21.2 Use the four-step method Use the four-step method

22. Managing the dimensional . Managing the dimensional Modelling Project.Modelling Project.

2.1 Data Warehouse Bus Architecture 2.1 Data Warehouse Bus Architecture MatrixMatrix

2.22.2 Fact Table Diagram Fact Table Diagram

2.32.3 Fact Table Detail Fact Table Detail

2.4 Dimension Table Detail2.4 Dimension Table Detail

2.5 Steps for the Modelling Team2.5 Steps for the Modelling Team

2.6 Identifying the sources2.6 Identifying the sources

2.7 using a Data Modelling Tool2.7 using a Data Modelling Tool

SummarySummary

1.11.1 Build the Matrix Build the Matrix

Table RowsTable Rows : Data Marts : Data Marts

Table ColumnsTable Columns : Dimensions : Dimensions

ExampleExample Data Warehouse Bus Architecture Matrix Data Warehouse Bus Architecture Matrix

1.1 Build the Matrix 1.1 Build the Matrix List the Data Marts (rows)List the Data Marts (rows)

Example: data marts for a telephone co.Example: data marts for a telephone co.– customer billing statementscustomer billing statements– scheduled service and installation ordersscheduled service and installation orders– trouble reportstrouble reports– marketing promotionsmarketing promotions– call detail for a billing perspectivecall detail for a billing perspective– customer inventorycustomer inventory– labor and payrolllabor and payroll– ......

1.1 Build the Matrix 1.1 Build the Matrix List the Dimensions List the Dimensions

(columns)(columns)

Example: Dimensions for the telephone co.Example: Dimensions for the telephone co.in data mart: in data mart: Customer billing statementCustomer billing statement– Time (date of billing)Time (date of billing)– CustomerCustomer– ServiceService– Rate categoryRate category– Local service providerLocal service provider– ......

Data Warehouse Bus Architecture MatrixData Warehouse Bus Architecture Matrix

1.21.2 Use the four-step Use the four-step Method to design Each Method to design Each

Fact TableFact Table

Step 1. Choose the Data MartStep 1. Choose the Data Mart

Step 2. Declare the GrainStep 2. Declare the Grain

Step 3. Choose the DimensionsStep 3. Choose the Dimensions

Step 4. Choose the FactsStep 4. Choose the Facts

Example: Telephone co.Example: Telephone co.

Four-steps e.g.Telephone Four-steps e.g.Telephone co.co.

1. 1. Data Mart:Data Mart: Customer billing Customer billing 2. 2. Grain:Grain: the individual line item on the individual line item on

each monthly customer billeach monthly customer bill 3. 3. Dimensions:Dimensions: Time, Customer, Time, Customer,

Service, PromotionService, Promotion 4. 4. Facts:Facts: Line item amount, Line item amount,

Line item quantityLine item quantity

2.2 Fact Table Diagram2.2 Fact Table Diagram

The fact table Diagram:The fact table Diagram:– Names the fact tableNames the fact table– Clearly states its grainClearly states its grain– Shows dimensions to which it is connectedShows dimensions to which it is connected– Shows all the other dimensions without Shows all the other dimensions without

connectionsconnections Example : fact table diagram for the Example : fact table diagram for the

telephone billing line itemtelephone billing line item Example :Example : Supporting information Supporting information

The telephone billing The telephone billing Fact Table DiagramFact Table Diagram

Dimension information Dimension information TableTable

Supporting information for the Fact Supporting information for the Fact Table Diagram includes the Name Table Diagram includes the Name and Description of each dimension and Description of each dimension

Dimension Name Dimension Description

2.3 Fact Table Detail2.3 Fact Table Detail

Complete list of all the facts Complete list of all the facts available through the fact tableavailable through the fact table

List includes:List includes:– actual facts in the physical tableactual facts in the physical table– derived facts presented through viewsderived facts presented through views– facts calculated from first two groupsfacts calculated from first two groups

Example: Customer billing Line item Example: Customer billing Line item fact table detail for the telephone cofact table detail for the telephone co

Customer billing Line item Customer billing Line item

Fact table detailFact table detail

2.4 Dimension Table 2.4 Dimension Table DetailDetail

Shows attributes in a single dimensionShows attributes in a single dimension Shows explicit grain of the dimensionShows explicit grain of the dimension Shows the approximate cardinality of each Shows the approximate cardinality of each

dimension attributedimension attribute Shows hierarchies and relationships Shows hierarchies and relationships

between the attributesbetween the attributes Example: Time dimension Table detailExample: Time dimension Table detail Example:Example: Dimension attribute detail Dimension attribute detail

descriptionsdescriptions

Time Dimension Table detail Time Dimension Table detail

diagramdiagram

Dimension attribute detail Dimension attribute detail descriptionsdescriptions

Documentation: Full descriptive Documentation: Full descriptive information to support the diagraminformation to support the diagram– Attribute Name, description, cardinality Attribute Name, description, cardinality – Slowly Changing Policy, Sample ValuesSlowly Changing Policy, Sample Values

Example: Time Dimension attributeExample: Time Dimension attribute detail descriptions detail descriptions

Example:Example: Many-to-many relationships and Many-to-many relationships and slowly changing dimension attributesslowly changing dimension attributes

Example:Example: Correlated attributesCorrelated attributes

Time Dimension attributeTime Dimension attribute detail descriptions detail descriptions

Many-to-many relationships and slowly Many-to-many relationships and slowly changing dimension attributeschanging dimension attributes

Correlated attributesCorrelated attributes

2.5 Steps for the Dimensional 2.5 Steps for the Dimensional Modelling TeamModelling Team

Create the Initial draft: data marts, Create the Initial draft: data marts, dimensions, data matrix and diagramsdimensions, data matrix and diagrams

Track Base FactsTrack Base Facts Track derived Facts (Example)Track derived Facts (Example) Present initial design to rest of IS teamPresent initial design to rest of IS team Select some key users to work on Select some key users to work on

projectproject Present to Business users Present to Business users

Derived Fact worksheetDerived Fact worksheet

2.6 Identifying the Sources for 2.6 Identifying the Sources for Each Fact Table and Dimension Each Fact Table and Dimension

TableTable

SourceSource. Name of the source system.. Name of the source system. Business ownerBusiness owner. Name of the primary contact within . Name of the primary contact within

the business who is responsible for this data.the business who is responsible for this data. IS ownerIS owner. Name of the person who is responsible.. Name of the person who is responsible. PlatformPlatform. Operating environment where system runs. . Operating environment where system runs. Location.Location. The actual location of the system. The city The actual location of the system. The city

and the specific machine where this system runs. and the specific machine where this system runs. DescriptionDescription. A brief description of what system does. A brief description of what system does Example: Data Source DefinitionsExample: Data Source Definitions Example:Example: Mapping data from source to Target Mapping data from source to Target

Data Source definitionsData Source definitions

Mapping data from source to Mapping data from source to TargetTarget

Source-to-target data map:Source-to-target data map:

Foundation for the development of Foundation for the development of the data staging processthe data staging process

To document specifically where the To document specifically where the data can be locateddata can be located

Example: Sample Source-to-target Example: Sample Source-to-target data mapdata map

Sample Source-to-target data map Sample Source-to-target data map (1/2)(1/2)

Sample Source-to-target data map Sample Source-to-target data map (2/2)(2/2)

Table nameTable name. The name of the logical table in the data warehouse.. The name of the logical table in the data warehouse. Column nameColumn name. The name of the logical column in the warehouse.. The name of the logical column in the warehouse. Data type.Data type. The data type of the logical column (char, number, date). The data type of the logical column (char, number, date). Length.Length. The length of the field of the logical column. The length of the field of the logical column. Target column descriptionTarget column description. A description of the logical column.. A description of the logical column. Source systemSource system. The name of the source system where data feeds . The name of the source system where data feeds

the target logical column.the target logical column. Source table/fileSource table/file. The name of the specific table or file where data . The name of the specific table or file where data

feeds the target logical column. feeds the target logical column. Source column/fieldSource column/field. The name of the specific column or field . The name of the specific column or field

where data feeds the target logical column.where data feeds the target logical column. Data transformData transform. Notes about any transformations that are required . Notes about any transformations that are required

to translate the source information into the format required by the to translate the source information into the format required by the target column. target column.

2.7 Using a Data Modelling Tool2.7 Using a Data Modelling Tool

Used to develop the physical data Used to develop the physical data modelmodel

Preferably one that stores your Preferably one that stores your model’s structure in a relational model’s structure in a relational databasedatabase

Summary (1/2)Summary (1/2)

Process used to apply dimensional Process used to apply dimensional modelling techniques modelling techniques

Bus Architecture Matrix to lay out Bus Architecture Matrix to lay out the data marts and dimensionsthe data marts and dimensions

Four-step method to design a single Four-step method to design a single data martdata mart

Diagramming techniques Diagramming techniques Data sourcing and mappingData sourcing and mapping

Summary (2/2)Summary (2/2)

In this section we focused on In this section we focused on the the

DATADATA

Next: different path of lifecycleNext: different path of lifecycleTechnical architecture

Supporting Templates Supporting Templates collected on CD-Romcollected on CD-Rom

Template 7.1 Data Mart MatrixTemplate 7.1 Data Mart Matrix Template 7.2 Dimensional Model Template 7.2 Dimensional Model

DocumentDocument Template 7.3 Derived Fact WorksheetTemplate 7.3 Derived Fact Worksheet Template 7.4 Logical table designTemplate 7.4 Logical table design Template 7.5 Data Source Definition Template 7.5 Data Source Definition

DocumentDocument Template 7.6 Source to Target Data MapTemplate 7.6 Source to Target Data Map