SQL SERVER SQL SERVER SQL SERVER SQL SERVER

ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES

INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES

ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES

DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING

DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING

INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES

SQL SERVER SQL SERVER SQL SERVER SQL SERVER

MS SQL Server Analysis Services 2008 and Enterprise

Data Warehousing

About Me

Slava KokaevE m a i l : v k o k a e v @ b o s t o n b i . o r gP e r s o n a l w e b s i t e : w w w. b i p r o . o r gB l o g : w w w. b o s t o n b i . o r g / b l o g . a s p x

Agenda

Understanding Business Architecture

Understanding BI Architecture

Introduction to Dimensional Modeling

“You can’t manage what you can’t measure. You can’t measure what you can’t

describe”

Robert Kaplan and David Norton Authors of “The Balanced Scorecard”

Drive Corporate PerformanceGiving a purpose to business intelligence

Bike FactoryTires FactoryStill Factory

Color Factory

Warehouse Resellers

Accessory Factory

Plastic Factory AdventureWorksHeadquarter

Operational System

Management System

Understanding The Business System

Business Intelligence System

Microsoft BI Platform

EnterpriseData WarehouseSystem

BusinessAnalysisSystem

Understanding a BI System

Source SystemETL System

DW SystemDA System

Microsoft’s BI platform

END USER TOOLS & PERFORMANCE MANAGEMENT APPS

Excel Power Pivot

BI PLATFORM

SQL Server Reporting Services

SQL Server Analysis Services

SQL Server DBMS

SQL Server Integration Services

SharePoint Server

Reports Dashboards Excel Workbooks

AnalyticViews Scorecards Plans

Microsoft’s BI Strategy and VisionTo improve organizations by providing business insights to all employees, leading to better, faster, more relevant decisions• Microsoft has a long-term commitment

to delivering a complete and integrated BI offering

• SQL Server has led innovation in the BI space for more than a decade

• There is widespread delivery of intelligence through Microsoft Office

• The platforms are enterprise-grade and affordable

Enterprise Business Analysis System

Customers Business partners

Vendors, Suppliers, Channel partners

IT providers Financial service providers

Monitoring Systems Analysis Systems

Business Processes and Operations

Controlling Systems Strategy and Planning Systems

Abstract Functional Business Model

Plan

Act

Check

Do

Data Mining Reporting Services ANALYSIS SERVICESSQL Server

Input Data

Resources

Plans, Business Rule and KPI

Result Data

Process Output (Facts /Measures)

Feedback (Improvement)

IDEF0 Modeling Notation

SQL Server 2008 BI Platform Components

Data acquisition from source systems and integration

Data transformation and synthesis

Data enrichment, with business logic, hierarchical views

Data discovery via data mining

Data presentation and distribution

Data access for the masses

Integrate Analyze Report

MS BI Platform

Data Mining

Analysis Services

Integration Services

Reporting

Services

Data Mining

Analysis Services

Reporting Services

Understanding BI Architecture

Source System ETL System Clients ToolsData Warehouse

Enterprise Data Source Structure

Data Warehouse

CRM

ERPHR

Call Center

Web Apps

FinanceInventory

ETL SystemExtract data from the source systems

Transform the data to convert it to a desired state

Load the data into the data warehouse

E T L

ETL Workflow

OLTP SSIS DW OLAP

Analysis SystemMultidimensional databases are also called online analytical processing (OLAP) databases and…Contain structures optimized for rapid ad hoc information retrievalPre-calculate and store aggregated valuesInclude calculation engines for fast, flexible transformation of base dataDesigned to reveal business trends and statistics not directly visible in the data retrieved from a data warehouseData mining models discover patterns in data, typically for prediction analysis

ProductAssociation

Sales FinanceProduction

Data Visualization SystemClient access and distribution mechanisms can include:

Static report viewers and browsers Ad hoc query tools Report writers Modeling applications Scorecard applications Portals and dashboards

Delivering data is a process of continuous business improvement:

Monitor Analyze Plan

What is a dimensional model ?A dimensional model is made up a central fact table (or tables) and its associated dimensions. The dimensional model is also called a star schema because it looks like a star with the fact table in the middle and the dimensions serving as the points on the star. From a relational data modeling perspective, the dimensional model consists of a normalized fact table with denormalized dimension tables.

Fact and Dimensions together or “Star Schema” Database

DimensionsDimensions are the foundation of the dimensional model, describing the objects of the business, such as employee, product, customer, service.They describe the surrounding measurement events. The business processes (facts) or actions of the business in which the dimensions participate. Each dimension table links to all the business processes in which it participates. A single dimension that is shared across all these processes is called a conformed dimension.

Fact TablesEach fact table contains the measurements associated with a specific business process. A record in a fact table is a measurement, and a measurement event can always produce a fact table record. These events usually have numeric measurements that quantify the magnitude of the event, such as quantity ordered, sale amount, or call duration. These numbers are called facts (or measures in Analysis Services).The key to the fact table is a multi-part key made up of a subset of the foreign keys from each dimension table involved in the business event.

Sales Business Process

Plan Sales

Analyze Sales

Monitor Sales

Resellers Sales

SQL Server DB

SALES MANAGERSales Representative

Stock Data

Reseller (Dimension)

Sales Quota

Sales Result

Sale Orders (Facts /Measures)

Sales corrections and Improvement

Sales Summary

Balance Scorecards

Sales Transaction

Product Hierarchies

Designing DimensionsDatabase Table

DataSourceView

DatabaseDimension

DimensionHierarchy

CubeDimension

HierarchiesA hierarchy is a collection of logically structured levels based on attributes. In some hierarchies, each member attribute uniquely implies the member attribute above it.

Surrogate KeysPrimary key purpose

Identifies uniqueness Relates to foreign keys in a fact table

Two candidates Business key

Represents source primary key

Surrogate keyConsolidates multiple data sources

Consolidates multi-value business keys

Allows tracking of dimension history

Limits fact table width for optimization

Using a surrogate key is considered best practice

Surrogate Keys

MS-1981163MS-1981

Source OLTP Table Target DW Table

Surrogate Key

Business Key

SnowflakingSnowflaking is the practice of connecting lookup tables to fields in the dimension tables. Sometimes it's easier to maintain a dimension in the ETL process when it's been partially normalized or snowflaked.

Reviewing Star Schema Benefits • Transforms normalized data into a simpler

model• Delivers high-performance queries• Delivers higher performing queries using

Star Join Query Optimization• Uses mature modeling techniques that are

widely supported by many BI tools• Requires low maintenance as the data

warehouse design evolves

VT

COM

A

TXSt

ate

Understanding Cube Structure

12451576

29543007

1383

16542145

2012

645

8451082

905

275

345875

745 745

905

2012

3007

761

745

1365

1575

234

2455

988

700

645

2322

18741479

15531576 2954 3007

1455 1874445 1575

995 1945 945 1479

1164 1893 13761245

Quarter 1Quarter 2

Quarter 3 Quarter 4

Semester 1

Semester 2

Calendar Year - 2009

Australia

United States

Canada

France

Coun

try

Accessories

Bikes

Clothing

Components

Prod

uct

Line

Reviewing Star Schema Benefits

Normalized(OLTP)

Denormalized (Star Schema)

MDX vs. T-SQLcalculate YTD monthly average and compare it over several years for the same selected month

WITH MEMBER Measures.MyYTD AS SUM(YTD([Date].[Calendar]),[Measures].[Internet Sales Amount])

MEMBER Measures.MyMonthCount AS SUM(YTD([Date].[Calendar]),(COUNT([Date].[Month of Year])))

MEMBER Measures.MyYTDAVG AS Measures.MyYTD / Measures.MyMonthCount

SELECT {Measures.MyYTD, Measures.MyMonthCount,[Measures].[Internet Sales Amount],Measures.MyYTDAVG} On 0, [Date].[Calendar].[Month] On 1FROM [Adventure Works]WHERE ([Date].[Month of Year].&[7])

Slowly Changing DimensionsSupport primary role of data warehouse to describe the past accuratelyMaintain historical context as new or changed data is loaded into dimension tables

Slowly Changing Dimension (SCD) types Type 1: Overwrite the existing dimension record Type 2: Insert a new ‘versioned’ dimension record Type 3: Track limited history with attributes

The concept of Slowly Changing Dimensions was introduced by Ralph Kimball

Slowly Changing Dimensions Type 1Existing record is updated

History is not preserved

LastName update to Valdez-Smythe

Slowly Changing Dimensions Type 2Existing record is ‘expired’ and new record insertedHistory is preservedMost common form of Slowly Changing Dimension

SalesTerritoryKey update to 10

Slowly Changing Dimensions Type 2Existing record is updated

Limited history is preserved

Implementation is rare

SalesTerritoryKey update to 10

ResourcesSQL Server 2008 Books Online,msdn2.microsoft.com/en-us/library/bb543165(sql.100).aspx

The Microsoft Data Warehouse Toolkit by Joy Mundy, Warren Thornthwaite, and Ralph Kimball

The Data Warehouse Lifecycle Toolkit by Ralph Kimball, et al.