DSS & DWH Concepts

8/17/2019 DSS & DWH Concepts

1/41

1

IntroductionIntroduction

to DSSto DSS

&&Data WarehousingData Warehousing

ConceptsConceptsTATA

INFOTECH Ltd

Atul Gandre


2/41

2

ContentsContents

• What is DSS ?What is DSS ?• DSS architecture and its componentsDSS architecture and its components

•Extraction, Transformation & LoadingExtraction, Transformation & Loading•Data Access & AnalsisData Access & Analsis

• Data martsData marts• Data !iningData !ining

TATA

INFOTECH Ltd


3/41

"

What is DSS?What is DSS?


4/41

#

ManagementManagement

ObjectivesObjectives• $ncreased profits• $mpro%ed margins

• educed o%erheads• Larger mar'et share

TATA

INFOTECH Ltd


5/41

(

Typical Questions a DecisionTypical Questions a Decision

Maer may as !!!Maer may as !!!“ Give peformance of all TV’s, over the past 3years ”

• Sho) Sales * %olume, %alue and margin contri*ution

• + different time periods

• + product model• + region

• ompare " ears sales month)ise

• ompare sales %-s tpe of promotion, * region

• +est distri*utors - Worst distri*utors• !argin Analsis, ost +rea'up

• apacit utilisation o%er time

TATA

INFOTECH Ltd


6/41


7/41

• Sales %alue

• 1 of target met

• 1 o%er last ear

• + region

• + product

“ Top !" / Bottom !" #ales men this year “ y


Maer may as !!!Maer may as !!!

TATA

INFOTECH Ltd


8/41

3

• + month - 4uarter - ear

• + product - product group - all products

• 5or a region - all regions

• an'ing o%er the last 62 months

• eco%er of dues

“ #ho$ the Top %" &ustomers “


Maer may as !!!Maer may as !!!

TATA

INFOTECH Ltd


9/41

7

Can TransactionCan Transaction

systems ans"er suchsystems ans"er such#ueries $#ueries $ The pro*lems 8

• 9ighl normalised structures ma'e 4ueries more complex

• $ncrease in complexit of 4ueries due to 8

Aggregation, Summarisation, an'ing, umulations, unning totals,omparison

/arious dimensions

• Little historical data stored on:line for comparison

• 9igh resource utilisation )ill result in slo) response to

complex 4ueries

• Difficult in ma'ing ad:hoc 4ueries

TATA

INFOTECH Ltd


10/41

6;

DSS - a definition DSS - a definition

Decision Support Systems use computers tofacilitate the decision main! process of semistructured tass" These systems are desi!ned not

to replace mana!erial #ud!ement $ut to supportit and mae the decisions more effecti%e" DSShelps mana!ers react &uicly to chan!in! needs"

' ( H Inmon

TATA

INFOTECH Ltd


11/41

66

DSS % a paradigm orDSS % a paradigm or

analysisanalysis• ather strategic information

• onstant prototpe mode

• Detailed and summaried data• edundanc allo)ed

• Data is normall loaded

• Amount of data used in a process is

large

• Ser%es the managerial communit

' ( H Inmon

Transactional Applications DSS Applications

TATA

INFOTECH Ltd


12/41

62

DSS architecture and its DSS architecture and its

componentscomponents

TATA

INFOTECH Ltd

• Data )arehouse architecture

• Data extraction, transformation and loading

• Data access and analsis@


13/41


14/41

6#

!!!DSS 'rchitecture!!!DSS 'rchitecture

The DSS Architecture consists of@@@

Extraction of data from %arious operational sstems

on different platforms, then transforming and

loading to the Data Warehouse

• The Data Warehouse contains historical data as )ell

as current data@

• The data in the Data Warehouse is accessed * the

front:end tools

TATA

INFOTECH Ltd


15/41

6(

Data Warehouse % theData Warehouse % the

heart o DSSheart o DSS The Data Warehouse is that portion of an

o%erall architected data en%ironment that

ser%es as the single integrated source ofdata for decision support sstems B

' ( H Inmon

TATA

INFOTECH Ltd


16/41

6.

Data Warehouse (Data Warehouse (

'nother de)nition 'nother de)nitionA Data Warehouse is a• su*Cect:oriented,

• integrated,

• time %ariant and

• non:%olatile

collection of data in support of

managements decision:ma'ing process@B

-- ( H Inmon

TATA

INFOTECH Ltd


17/41

6

** Characteristics o aCharacteristics o a

Data WarehouseData Warehouse• The DW pro%ides access to

corporate - organiational data

• The data in the DW is consistent

• The data in the DW can *e separated and com*ined *

means of e%er possi*le measure in the *usiness

• The DW is )here data is pu*lishedB

TATA

INFOTECH Ltd


18/41

DW Data Model CharacteristicsDW Data Model Characteristics

• Data centric not process *ased

• Simple to understand

• 5lexi*le to add-modif

• Design reflects *usiness information

•uer dri%en design

• Denormalised

$ntuiti%e and eas to use


19/41

The Dimensional ModelThe Dimensional Model

• The E< of a ompan sas :

• We sell products in %arious markets and )e

measure our performance o%er timeB Time

Product

Market


20/41

The Dimensional ModelThe Dimensional Model

• Each ell in the cu*e contains *usiness

measures for a particular com*ination of

• Product , Market and Time

•


21/41

East (est North

1234

5234

6237

8237

1237

0u$y

Emerald

Saffire

Sales0e%enue

• North

• Emerald

• 5234

DW Data modelling - DW Data modelling -

Multidimensionality : An example Multidimensionality : An example


22/41

DW Data modelling :DW Data modelling :

Star SchemaStar SchemaSales

Dimension ta*les

Time

egion

0roduct

ustomer

5act ta*les


23/41

Star Schema FeaturesStar Schema Features

•


24/41

Snowflake DesignSnowflake Design

• Sno)fla'e refers to normalising dimension

ta*les

• reating Outrigger ta*les containing• containing descriptions of codes in dimension ta*le

• containing additional attri*utes


25/41

Snowflake Schema ExampleSnowflake Schema Example

Sales Fact

Product

Seller

Supplier

Location

Time

Store

District

Region

Month

Day

Season

Sales Assoc.

Sales Dept.


26/41

Data Modelling StepsData Modelling Steps

Starting point : End users and source data

• $dentif a su*Cect area

• 5ind out the Gfacts• Associate the facts )ith the *usiness dimensions

• Define the attri*utes in the dimensions

• Decide on the le%el of detail : the granularit• Decide the Gsummarise and purge period


27/41

2

Data extraction transformation Data extraction transformation

and upload and upload

Operational

Systems Data Warehouse !"S

Data !xtraction Data

Transformation

Data #pload

TATA

INFOTECH Ltd


28/41

23

Data +,tractionData +,traction

The Extract program :

• ummages through a file or data*ase

• Hses some criteria for selection• $dentifies 4ualified data and• Transports the data o%er onto another

file or data*ase@

TATA

INFOTECH Ltd


29/41

27

Data +,traction %Data +,traction %

CleanupCleanup• estructuring of records or fields• emo%al of


30/41

";

Data transormationData transormation

• $ntegrating dissimilar data tpes• hanging codes• Adding a time attri*ute• Summarising data

• alculating deri%ed %alues• Denormalising data

TATA

INFOTECH Ltd


31/41

"6

Data loadingData loading

• $nitial and incremental loading

• Hpdation of metadata

• Hpdation of log

• oll*ac' in case of loading errors


32/41

"2

+T- Tools+T- Tools

• $nformatica

• Ardent DataStage

•


33/41

""

Data 'ccess &Data 'ccess &

'nalysis 'nalysis

• Ease of na%igation across screen• /alue addition * *etter information presentation

I graphs, charts and mapsJ

• 9ighlighting exception information *

Alarms and Alerts

• Drill:do)n - roll:up through successi%e le%els of

data

• What :if analsis

TATA

INFOTECH Ltd


34/41

"#

.eporting Tools.eporting Tools

• !icrostrateg

• +usiness


35/41

"(

What is O-'/$What is O-'/$


36/41

".

O-'/ CharacteristicsO-'/ Characteristics

• Al)as in%ol%es interacti%e 4uer and analsis of the

data@ The interaction is usuall multiple passes

• $n%ol%es drilling do)n into successi%el lo)er le%els

of detail data

• $n%ol%es roll:ups to higher le%els of summariation

and aggregation@

•


37/41

"

Data martsData marts

• Architecture

• haracteristics

• Example


38/41

"3

E)ternal

Sources

Data

E)traction*

Scru$$in!* +

Transformation

Data Sources

,Operational

Systems-

EIS

OLA.

Data /inin!

Data Access

Data mart in the DSS 'rchit Data mart in the DSS 'rchit

TATA

INFOTECH Ltd

Data

(arehouse

O0

AI

Data /art


39/41

"7

• Data marts are scaled:do)n and lessexpensi%e %ersions of data )arehouses

• Data marts utilie large:scale data

)arehousing concepts on a smaller, more focussed le%el

• Data marts are focussed at departmental

users• Decentralised approach

What are Data marts $What are Data marts $

TATA

INFOTECH Ltd


40/41

#;

What is data mining$What is data mining$

Data !ining, the extraction of hidden

information from large data*ases@ $t is a po)erful ne) technolog )ith great potential

to help companies focus on the most

important information in the data )arehouse@

TATA

INFOTECH Ltd


41/41

#6

Data miningData mining

capabilitiescapabilities• Disco%er of un'no)n patterns

• 0rediction of trends and *eha%iors• Disco%er of anomalies in data

TATA

INFOTECH Ltd

Documents

DSS & DWH Concepts