Upload
xander-less
View
223
Download
4
Tags:
Embed Size (px)
Citation preview
SQL Server Fast Track &
Project Madison – SQL MPP
Roger Moore – Data Warehouse [email protected]
972-955-0426
Microsoft Confidential
Agenda
Microsoft Data Warehouse StrategySQL DW & BISQL Server Fast Track Madison Overview – SQL MPP (DATAllegro)
Hub and SpokeMulti-TemperatureMTP – Technology Preview (PoC)
Summary
END USER TOOLS & PERFORMANCE MANAGEMENT APPS
ExcelPerformancePoint
Server
BI PLATFORM
SQL Server Reporting Services
SQL Server Analysis Services
SQL Server DBMS
SQL Server Integration Services
SharePoint Server
DELIVERY
Reports Dashboards Excel Workbooks
AnalyticViews Scorecards Plans
Our Integrated BI-DW Offering
Microsoft Is Serious About Data Warehousing
Heterogeneous Connectivity & Workloads
Data Integrity & Quality
Compliance & Security
Data Warehouse Scale
Data Warehouse Management
2005 2008 Futures
PB Warehouses>64 Core ProcessingScale out through MPP
Perf. Management ToolsBI Resource GovernanceImproved Predictability
Mixed workload supportContinuous Loading
Integrated DQ Services (Zoomix)Master Data Management(Stratature Integration)
Rights Management
10s of TB WarehousesParallel partitioningData compressionNew Reference
Architectures
Policy Based Admin.DB Resource Governance
High Perf. Connectors(Oracle, Teradata, SAP BW)
Data Profiling
Policy based auditing
Multi TB WarehousesEnterprise scalabilityDW Reference Architectures
Unified manageability
Enterprise class ETL tool
Data Cleansing(Fuzzy lookup/matching)
Data Protection & Tracing
The Appliance Model for Data Warehousing• Building a traditional
DW• Time consuming• Expensive• Performance varies• Scalability issues
Potential bottlenecks in standard DW architecture
• The DW appliance model• Pre-built & tuned h/w + s/w• Views entire stack holistically• Known performance &
scalability• Encapsulates best practices• Leverages Sequential I/O
Lower TCOFaster
deployment
Better performanc
e
Minimised DBA time
Benefits
What is SQL Server Fast Track Data Warehouse?
• An appliance approach to SMP data
warehouse reference architectures• Pre-built & tuned h/w +
s/w• Views entire stack
holistically• Known performance &
scalability• Encapsulates best
practices• Leverages Sequential
I/O• Seven distinct reference architectures• Delivered with SI Partners – • QuickStart assessments• Solution templates
Helping Customers & Partners Accelerate Their Data Warehouse Deployments
Fast Track Data Warehouse ComponentsKey Principle 1: Tight Specifications
7<Session Name> Microsoft NDA-only
Software:• SQL Server 2008
Enterprise• Windows Server 2008
Hardware:• Tight specifications for
servers, storage and networking
• ‘Per core’ building block
Configuration guidelines:• Physical table
structures• Indexes• Compression• SQL Server settings• Windows Server
settings• Loading
Key Principle 2: Balanced Across All Components
FCHBA
AB
AB
FCHBA
AB
AB FC
SW
ITCH
STORAGECONTROLLER
AB
ABCA
CHE
SERV
ER
CACH
ESQ
L SE
RVER
WIN
DO
WS
CPU
CO
RES
CPU Feed Rate HBA Port Rate Switch Port Rate SP Port Rate
A
BDISK DISK
LUN
DISK DISK
LUN
SQL Server Read Ahead Rate
LUN Read Rate Disk Feed Rate
SQL Server 2008 Potential Performance Bottlenecks
Key Principle 3: Sequential I/O
Sequential I/OIdeal for data warehousingScalable, predictable performanceLarge reads & writesRequires 1/3 or fewer drives for same performance
Random I/OIdeal for OLTPNot as predictable & scalable for data warehousingSmall reads and writesRequires large number of drives
Best practices focus on preserving the sequential order of data
SQL Server Fast Track Data Warehouse for HP
2 Processor ConfigurationServer: HP ProLiant DL385 G5p with 2 Quad-core AMD Opteron processorsStorage server: EMC or MSA StorageScalability: up to 8 TB
4 Processor ConfigurationServer: HP ProLiant DL 585 G5 with 4 Quad-core AMD Opteron processorsStorage server: EMC or MSA StorageScalability: 4 – 16 TB
8 Processor ConfigurationServer: HP ProLiant DL 785 G5 with 8 Quad-core AMD
Opteron processorsStorage server: EMC or MSA StorageScalability: 16 – 32 TB
• Note - Compression assumes 2.5:1
SQL Server Fast Track Data Warehouse for DELL
2 Processor ConfigurationServer: Dell Power Edge 2950 MLK with 2 Quad-core Intel Xeon processorsStorage server: EMC CX4-240 & AX4Scalability: up to 8 TB
4 Processor ConfigurationServer: Dell Power Edge R900 with 4 6-core Intel Xeon processorsStorage server: EMC CX4-240 & AX4Scalability: 12 – 24 TB
• Note - Compression assumes 2.5:1 - Fully loaded only adds drives to minimum HW required - Data space can be increased by using 450GB drives
Fast Track Case Study - Environment
Current EnvironmentTeradata 4-node (5450 model) with 6TB of user dataBI: Business ObjectsETL: Informatica and BTEQ scripts
Proposed Microsoft PlatformSQL Server Fast Track Data WarehouseHP DL580 Server - 4 Quadcore Processors (16 core total)256 GB MemorySAN Storage: MSA 2000 (Qty 4) – 8TB User Data CapacityBI: Business ObjectsETL: SQL Server and SSIS
Fast Track Case Study - Results
Teradata SQL Server Fast Track DW
Comparison
Loading – Subject Area 1
5:10:21 total time 51:31 total time R SQL Server 6x faster
Loading – Subject Area 2
4:36:08 total time 1:50.01 total time R SQL Server 2.5x faster
Query times – Subject Area 1
3:03 avg query time(using 9 benchmark queries)
0:15 avg query time(using 9 benchmark queries)
R SQL Server 12x faster
Query times – Subject Area 2
56:44 avg query time(using 4 benchmark queries)
8:09 avg query time(using 4 benchmark queries)
R SQL Server 7x faster
Fast Track Benefits Summary
14<Session Name> Microsoft NDA-only
Appliance-like time to valueReduces DBA effort; fewer indexes, much higher level of sequential I/O
Choice of HW PlatformsDell, HP, Bull – more in future
Low TCO ThroughCommodity Hardware and value
pricing; Lower storage costs.
High ScaleNew reference architectures scale
up to 32 TB (assuming 2.5x compression)
Reduced RiskTested by Microsoft; better choice of hardware; application of Best
Practice
The Bridge to Project "Madison"
Fast Track offers appliance-like ease of deployment, scalability and performance for SMPMadison to offer massively parallel (MPP) scale and performanceMadison hub-and-spoke architecture to include support for SMP spokes
Scale Out
Scale Up
Scaling SQL Server 2008
INDUSTRY STANDARDNETWORKING
INDUSTRY STANDARDSTORAGE
INDUSTRY STANDARDSERVERSFast Track
Data Warehouse
s
Project“Madiso
n”
MPP (Madison) Overview
INDUSTRY STANDARDNETWORKING
INDUSTRY STANDARDSERVERS
Reference Hardware Platforms
ProjectMadison
INDUSTRY STANDARDSTORAGE
Madison – SQL MPP Architecture Sample
Compute Nodes
Compute Nodes
Du
al
Infi
nib
an
d
Spare Compute Node
Storage Node
Control Nodes
Active / Passive
Landing Zone
Backup Node
Storage Servers
Date Dim
D_DATE_SK
D_DATE_ID
D_DATE
D_MONTH
…
Store Sales
Ss_sold_date_sk
Ss_item_sk
Ss_customer_sk
Ss_cdemo_sk
Ss_store_sk
Ss_promo_sk
Ss_quantity
…
Promotion
P_PROMO_SK
P_PROMO_ID
P_START_DATE_SK
P_END_DATE_SK
…
Customer
C-CUSTOMER_SK
C_CUSTOMER_ID
C_CURRENT_ADDR
…
Item
I_ITEM_SK
I_ITEM_ID
I_REC_START_DATE
I_ITEM_DESC
…
Store
S_STORE_SK
S_STORE_ID
S_REC_START_DATE
S_REC_END_DATE
S_STORE_NAME
…
Customer
Demographics
CD_DEMO_SK
CD_GENDER
CD_MARITAL_STATUS
CD_EDUCATION
…
1
Trillion
Rows
100 Million73, 049
1.92 Million1, 902
2, 500
502, 000
Project “Madison” Demonstration Architecture TPCDS – 150+ Terabytes
Date
Dim
D_DATE_SK
D_DATE_ID
D_DATE
D_MONTH
…
Item
I_ITEM_SK
I_ITEM_ID
I_REC_START_DATE
I_ITEM_DESC
…Store Sales
Ss_sold_date_sk
Ss_item_sk
Ss_customer_sk
Ss_cdemo_sk
Ss_store_sk
Ss_promo_sk
Ss_quantity
…
Promotion
P_PROMO_SK
P_PROMO_ID
P_START_DATE_SK
P_END_DATE_SK
…
Store
S_STORE_SK
S_STORE_ID
S_REC_START_DATE
S_REC_END_DATE
S_STORE_NAME
…
Customer
C-CUSTOMER_SK
C_CUSTOMER_ID
C_CURRENT_ADDR
…
Customer
Demographics
CD_DEMO_SK
CD_GENDER
CD_MARITAL_STATUS
CD_EDUCATION
…
Database Distributed & Replicated Tables
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
C I
D
CD
S
P
SS
Data Distribution with Replication
Processor Utilization
Madison and Fast Track Hub and Spoke
22<Session Name> Microsoft NDA-only
Central EDW Hub
Regional Reporting
Departmental Reporting
ETL Tools
High Performance HQ
Reporting
Madison Multi-Temperature
Auto Publish
FR
ES
H D
ATA
L
OA
DIN
G
Most Recent - 3 Months
2 Years 7 Years
User Queries
BI Server
Queries
• User Data• Hot -> Warm -> Cold• Stage -> ODS ->
Prod
•Back-up / Archive• Data structure in
synch• Fast response to
users
• Easy Data Movement
• High Availability
Case study: Tier 1 Carrier - CDR Architectureincluding Multi Temperature Archive
UP TO 500M ROWS/DAY
HIGH-SPEEDPARALLELUPDATES
COSTMGT
REVENUEASSURANCE
MARGINANALYSIS
120 TB HIGH CAPACITY‘WARM’ CDRs
FRAUD DETECTION
BILLING60 TB HIGH PERFORMANCEFOR MEDIATION & AUGMENTATION USING ETL TOOLS
220TB ARCHIVE DW
ROLL OFF TO ARCHIVE
"DW Appliance" Experience
All hardware from a single vendorMultiple vendors to chose fromOrderable at the rack or cluster Vendor will
Assemble appliancesImage appliances with OS, SQL Server and Madison software
Appliance installed in less than a daySupport –
Vendor provides hardware supportMicrosoft provides software support
Madison Beta Programs
Two ProgramsMTP – Madison Technology Preview
15-20 participantsDuration of 4 to 6 weeks
TAP – Beta production implementation4-6 customersFirst iteration 9 to 12 weeks
RequirementsFocus on EDW and large data martsMigration projects, not green fieldOpen to customers & prospects
DW QuickStart – Data Warehouse Roadmap Service
RequirementsExisting DWVolume of end-user data 1TB+Considering change to BI or DW infrastructure
On site surveyInterview of key stake holders in Data Warehouse environmentPerformed by Microsoft Architect Service also available from selected Microsoft partners with deep Data Warehouse expertise2-5 days duration
DeliverablesPresentation of key findingsReport detailing findingsResults delivered approximately 10 days after survey
Summary
Microsoft has a compelling EDW visionBI, ETL, scale up and outHub & Spoke architecture
Fast Track available todayUp to 30TB
Scale up today with SMP, scale out tomorrow with MPPMTP and TAP for Madison in June 2009
Scales up SQL Server to >1PBSets a new bar in appliance pricing and performance
Hub-and-Spoke will integrate Fast Track with Madison
END USER TOOLS & PERFORMANCE MANAGEMENT APPS
ExcelPerformancePoint
Server
BI PLATFORM
SQL Server Reporting Services
SQL Server Analysis Services
SQL Server DBMS
SQL Server Integration Services
SharePoint Server
DELIVERY
Reports Dashboards Excel Workbooks
AnalyticViews Scorecards Plans
Our Integrated BI-DW Offering
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.