38
© 2013 KMS Technology

Agile data warehouse

  • Upload
    dao-vo

  • View
    507

  • Download
    2

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Agile data warehouse

© 2013 KMS Technology

Page 2: Agile data warehouse

Confidential 2

AGILE DATA WAREHOUSE DESIGNDao Vo

Page 3: Agile data warehouse

Confidential 3

AGENDA

• Overview of data warehousing • Designing and implementing a data

warehouse• Waterfall BI/WH development• Agile BI/WH development framework• Q&A

Page 4: Agile data warehouse

Confidential 4

OVERVIEW OF DATA WAREHOUSINGWhat is a data warehouse?

Page 5: Agile data warehouse

Confidential 5

OVERVIEW OF DATA WAREHOUSING

• The business problem• What is a data warehouse?• BI/WH Architectures

Page 6: Agile data warehouse

THE BUSINESS PROBLEM

• Key business data is distributed across multiple systems

Page 7: Agile data warehouse

THE BUSINESS PROBLEM

• Finding the information required for business decision making is time-consuming and error-prone

Page 8: Agile data warehouse

THE BUSINESS PROBLEM

• Fundamental business questions are hard to answer

Page 9: Agile data warehouse

WHAT IS A DATA WAREHOUSE?

Page 10: Agile data warehouse

WHAT IS A DATA WAREHOUSE?

• A centralized store of business data for reporting and analysis

• Typically, a data warehouse:– Contains large volumes of historical data– Is optimized for querying data (as

opposed to inserting or updating)– Is incrementally loaded with new

business data at regular intervals– Provides the basis for enterprise

business intelligence solutions

Page 11: Agile data warehouse

Confidential 11

DESIGNING AND IMPLEMENTING A DATA WAREHOUSE

How to design a data warehouse and BI solution?

Page 12: Agile data warehouse

Confidential 12

DESIGN AND IMPLEMENT WH

• Introduction to Dimensional Modeling• Star Schemas• Considerations for Dimension Tables• Considerations for Fact Tables• Snowflake Schemas

Page 13: Agile data warehouse

Confidential 13

WAREHOUSE MODELING

Page 14: Agile data warehouse

INTRODUCTION TO DIMENSIONAL MODELING

• Business questions focus on measures that are aggregated by business dimensions

• Measures are facts about the business

• Dimensions are ways in which the measures can be aggregated

Product Line

Salesperson Product

Time

CustomerRegionQuantityRevenue

CostProfit

Page 15: Agile data warehouse

STAR SCHEMAS• Group related

dimensions into dimension tables

• Group related measures into

fact tables• Relate fact tables

to dimension tables by using foreign keys

DimSalesPersonSalesPersonKeySalesPersonNameStoreNameStoreCityStoreRegion

DimProductProductKeyProductNameProductLineSupplierName

DimCustomerCustomerKeyCustomerNameCityRegion

FactOrdersCustomerKeySalesPersonKeyProductKeyShippingAgentKeyTimeKeyOrderNoLineItemNoQuantityRevenueCostProfit

DimDateDateKeyYearQuarterMonthDay

DimShippingAgentShippingAgentKeyShippingAgentName

Page 16: Agile data warehouse

SNOWFLAKE SCHEMASDimSalesPersonSalesPersonKeySalesPersonNameStoreKey

DimProductProductKeyProductNameProductLineKeySupplierKey

DimCustomerCustomerKeyCustomerNameGeographyKey

FactOrdersCustomerKeySalesPersonKeyProductKeyShippingAgentKeyTimeKeyOrderNoLineItemNoQuantityRevenueCostProfit

DimDateDateKeyYearQuarterMonthDay

DimShippingAgentShippingAgentKeyShippingAgentName

DimProductLineProductLineKeyProductLineName

DimGeographyGeographyKeyCityRegion

DimSupplierSupplierKeySupplierName

DimStoreStoreKeyStoreNameGeographyKey

Page 17: Agile data warehouse

Confidential 17

WAREHOUSE MODELING

Page 18: Agile data warehouse

Confidential 18

WATERFALL BI/WH DEVELOPMENTTraditional SDLC to develop a BI/WH product

Page 19: Agile data warehouse

Confidential 19

WATERFALL BI/WH DEVELOPMENT

• SDLC Overview

Page 20: Agile data warehouse

Confidential 20

WATERFALL BI/WH DEVELOPMENT

Page 21: Agile data warehouse

Confidential 21

SDLC OVERVIEW

Page 22: Agile data warehouse

Confidential 22

AGILE BI/WH DEVELOPMENT FRAMEWORK

Incremental development framework for BI/WH product

Page 23: Agile data warehouse

Confidential 23

AGILE BI/WH DEVELOPMENT

FRAMEWORK• Agile BI/WH life cycle• Agile DW design overview• Agile ETL Solution

Page 24: Agile data warehouse

Confidential 24

AGILE BI/WH LIFE CYCLE

Page 25: Agile data warehouse

Confidential 25

AGILE BI/WH LIFE CYCLE

Page 26: Agile data warehouse

Confidential 26

AGILE DW DESIGN OVERVIEWHow to design to answer business question?

Page 27: Agile data warehouse

Confidential 27

AGILE DW DESIGN OVERVIEW

• How do we ask question?• The 7Ws framework• Design using natural language• Straightforward methodology• Model storming• BEAM methodology

Page 28: Agile data warehouse

Confidential 28

HOW DO WE ASK QUESTION?

• Events/Transactions– A immutable "fact" that occurs in a time

and place• Interrogatives:–Who, What, When, Where, Why– Descriptive context that fully describes

the event– A set of “dimensions" that describe

events

Page 29: Agile data warehouse

Confidential 29

THE 7WS FRAMEWORK

WhyWhere

How

WhoWhenWhat

HowMany

Page 30: Agile data warehouse

THE 7WS FRAMEWORK

HOW – FACTsMuchManyOften£$€

WhoCustomerEmployee

SellerOrganization

WhatProductService

TransactionsBooking

Event

WhyCausal

PromotionReason

WeatherCompetition

WhereLocation

GeographicStore

Ship toHospital

WhenTimeDay

MonthYear

Page 31: Agile data warehouse

Confidential 31

DESIGN USING NATURAL LANGUAGE

• Verbs – Events – Relationships – Fact Tables

• Nouns – Details – Entities – Dimensions

• Main Clause – Subject-Verb-Object• Prepositions – connect additional

details to the main clause• Interrogatives – The 7Ws – Dimension

Types

Page 32: Agile data warehouse

STRAIGHTFORWARD METHODOLOGY

Confidential

Who

What

When

Where

How (many)

Why

How

1

3

11111

4

5

2

6

7

8

Declare Event Type

Subject-Verb-Object

Quantities - Facts

Sufficient Detail Fact Granularity

Initial Data Examples

9

Page 33: Agile data warehouse

Confidential 33

DESIGN USING NATURAL LANGUAGE

• Verbs – Events – Relationships – Fact Tables

• Nouns – Details – Entities – Dimensions

• Main Clause – Subject-Verb-Object• Prepositions – connect additional

details to the main clause• Interrogatives – The 7Ws – Dimension

Types

Page 34: Agile data warehouse

Confidential 34

BUSINESS EVENT ANALYSIS AND MODELING (BEAM✲)

An agile approach to dimensional modeling

Page 35: Agile data warehouse

MODEL STORMING

Confidential 35

Quick

Data Modeler BI Stakeholders

Inclusive

Interactive

Fun

Page 36: Agile data warehouse

Confidential 36

BEAM ✲ METHODOLOGYStructured, non-technical, collaborative working conversation directly with BI

Users

• BI User’s Business Process, Organizational, Hierarchical, and Data Knowledge• Focused Data

Profiling

• Logical and Physical Dimensional Data Models

• Example data• Detailed and

Testable ETL Specification• DW

Prototype

BEAM✲

Data

Modeler

BI Stakeholders

Page 37: Agile data warehouse

37

Q&A

Page 38: Agile data warehouse

© 2013 KMS Technology

THANK YOU.