Embed Size (px)
Citation preview
Data Warehouse Automation
Retail - Case Study
Michael Tantrum & Douglas Barrett(WhereScape)
How data warehouse automation was able to help save people and planet!
Customer
Wikipedia
• IT is too slow
• Overcomplicated
• Hard to change
• Slow
REQUIREMENTS
DW framework
Profile
Logical Model
Physical Model
DB Architecture
Storage Mgmt
Index Mgmt
OLAP Design
ETL Mapping
ETL Dev
Versioning
Testing
Scheduling
Deployment
Word/Excel
3NF and Star
BI Tools
SQL
PowerDesigner
Modelling
JIRA
Toad
Cognos
Control M
Change tool
Change tool
Change tool
Change tool
Change tool
Change tool
Informatica
DB Management
Documentation
Word/Excel
Git
ETL
DW Traditional Approach
First Attempt
FinancialsSAP
Data WarehouseOracle Cognos Reporting & Analytics
After 2 years the project stakeholders called time out
Framework Manager
ReferenceOracle
SalesiSeries
Informatica
Second Attempt
FinancialsSAP
Data WarehouseOracle Cognos Reporting & Analytics
Informatica
After 6 months the project stakeholders called time out
Framework Manager
ReferenceOracle
SalesiSeries
Third time a charm
Framework Manager
FinancialsSAP
Data WarehouseOracle Cognos Reporting & Analytics
Informatica
Scaled out after successful POC – 6 weeks into production
ReferenceOracle
SalesiSeries
WhereScape
DW Framework
Profile
Physical Model
DB Architecture
Storage Mgmt
Index Mgmt
OLAP Design
ETL Mapping
ETL Dev
Version Control
Workflow
Deployment
Maintenance
Logical Model
Documentation
REQUIREMENTS
Blends development tasks1 development task instead of 6Efficient changeAgile projects
DW Better Approach - Automate
WhereScape 3DWhereScape RED
Format:
Under the radar Started with IT Champion. 1 part time developer kept 3 full time BAs busy
Result:4 Facts2 Aggregates12 Dimensions
Engaged Stakeholders
POC
• Why automate• What to automate• How to automate• Who it automates
Automation
Automate the doing, not the thinking.
Today
1960s
1 person
1920s
3 people one car
Robots
Why Automate
• Productivity• Consistency
2000s
1990s
More Programmers
1980s
Programmers
Even More Programmers!
DW Automation
• Productivity• Consistency
Anything with a pattern – design & build:
1. Data Vault – Hubs / Links / Sats
2. Star Schema – Type 1 / Type 2 / Snapshot
3. Time Variant Processing – Data Store / 3NF
4. Loading Data
5. Data type mapping
6. Indexing
7. Workflow dependencies
8. Deployments
9. Documentation
What to Automate
Who to Automate
Repurpose the team from developing to collaborating
Demo
