SQL Server 2005 Real World Data WarehousesDouglas Barrett (WhereScape)Data Warehouse Architect
VOC202
MeCurrent Life
Technical Presales ConsultantData Warehouse ArchitectProjects:
Previous livesMicrosoft ConsultingASB Bank DWH Team
ADHBASBMRPTelecomVero InsuranceAirNZ
MetrowaterSky CityFonterraetc
AgendaData Warehouse 101
Talk the same languageSQL Server 2005 and Data Warehousing
The Good and the BadDWH Projects
Secrets for successCustomers
ADHB – Mr Paul McGrath (DWH Manager)ASB - Mr Steve Letford (BI Manager)
Questions and Answers
Data Warehouse 101Organisations use functional systems to run their businessData is locked away
Financials
Sales
HR
BI SolutionData Warehouse
Data Warehouse 101Financials
Sales
Financials
Other
Inventory
CRM
Reformat Data
SQL Server 2005
AS, RS and Excel or PP
BI SolutionData Warehouse
Data Warehouse ProjectsFinancials
Sales
Financials
Other
Inventory
CRM
Reformat Data
Data Warehouse Risk / Time
Reformat DataExtract Transform Load
SQL 2005 BI Platform - ProsRelational Database
PartitioningDMVs
Analysis ServicesUnified Data ModelKPIs
Integration ServicesPipeline Processing
Reporting ServicesCube supportReport Model / Report Builder
SQL 2005 BI Platform - ProsRelational Database
PartitioningDMVs
Analysis ServicesUnified Data ModelKPIs
Integration ServicesPipeline Processing
Reporting ServicesCube supportReport Model / Report Builder
SQL 2005 BI Platform - ProsRelational Database
PartitioningDMVs
Analysis ServicesDevelopment EnvironmentKPIsUDM
Integration ServicesPipeline processing
Reporting ServicesCube supportModel / Report Builder
Excel ServicesBI Portal
Performance PointForecasting / PlanningDashboarding
SQL 2005 BI Platform - ConsMetadata missing Logging controlIntegration Services
Consistency of ImplementationAuditing / loggingMetadata
SQL 2005 BI Platform - Considerations
Disk layoutDataLogsTempdbExclusivity
Processor – 64 bitServer MemoryStats and Indices
DWH Projects – Secrets to success1. Find a sponsor2. Prototype3. User driven design4. Maintain momentum5. Remove Mysteries6. The DWH is a process not a project7. Small team of smart people
DWH Projects – Secrets to success
Secrets to success 1Find the sponsor
As high up as possible (Executive)In the businessFind out their painThey will determine success / failure Clears the way – makes life easy
Secrets to success 2Prototype – users don’t know what they want
Build what is possible – then show users1 – 2 weeks using real dataRequirements are generally a waste of timeFind out KPIs and core reportsProvides a picture / estimate of the full project
Secrets to success 3User driven design
Show them what is possibleLet them decide how to handle issues / rules“Socialise the design”No surprises at the end of a project
Secrets to success 4Maintain momentum
Shorter focused projects – single business area3 months maxShort intense projects retain business involvement
Secrets to success 5Remove Mysteries
Data Warehouse processing will always have problemsProcessing should make it easy to find out “what happened”Documentation is pretty handy
Secrets to success 6The DWH is a process not a project
Needs BAU resourceThe data warehouse is never staticAlways new data problems / enhancements / tuning
Secrets to success 7Small Team of Smart People
Teams don’t scale wellAble to talk to the businessAble to develop complex transformsAble to model the data based on implied requirements
ADHBMr Paul McGrathData Warehouse Manager
ADHB – Previous SituationTeam of 5 SQL Server 2000 and DTSBogged down by BAUUnable to manage change quicklyUnable to deliver new dataNo consistency or standardsChange control was a mess
Business getting frustrated with slow delivery.
ADHB – DWH ProjectWhereScape RED
Single ProductSpeed of DevelopmentStandardsBuilt in Change ControlSelf DocumentingMethodology - Prototype and iterateManages database
Generates SQL Stored procedures and Analysis Services Cubes
ADHB – DWH ProjectWhereScape RED3 month project to redevelop Inpatient and some Outpatient systems for AnalysisPrototype and iterateFull ownership by ADHB Development TeamInvolve analystsOngoing support and enhancements
Upgraded to SQL Server 2005 and AS 2005Upgraded to 64bit ServersWhereScape RED already SQL Server 2005 compliant including Analysis Services 2005Benefits
Make use of the latest features, fully supported by RED PartitioningRankingNew Analysis Services features including KPI’s
Pain – minimal
ADHB – SQL 2005
EDW with multiple data sources Approx 150Gb and growingExpanding to other areas
RadiologyHaematology/OncologyMental HealthFinance, HR etc
Catch up on a back log of workStart being proactive to usersManagement Information Portal
MOSS and Excel ServicesInteractive Reporting Excel 2007Customizable
ADHB – Current Situation
ADHB – Information Portal
ADHB – Information Portal
ADHB – DWH ProjectExtend to other areas:
OutpatientsHaematologyMental
Catch up on a back log of workStart being proactive to usersDelivery via Reporting Services
ADHBMr Paul McGrathData Warehouse Manager
ASBMr Steve LetfordBI Solution Manager
ASB – Previous SituationNew Data mart project support regulatory requirementsRelatively large team of developers and designersSQL Server 2005 and SSISProject got stuck with design, implementation issues, unable to engage with the business.Unable to deliver the project
Business getting frustrated with IT for slow delivery.
ASB – DWH ProjectWhereScape REDProject to redevelop key data martPrototype and iterateInvolve Business representativesOngoing support and enhancementsSuccessfully delivered
Utilising SQL Server partitioning, Analysis Services 2005 and Reporting Services
ASB – DWH OngoingCreate a group wide governance committeeExtend data mart into a set of federated data martsRollout reports Group wideCreate a services
modelMove to 64bit
ASBMr Steve LetfordBI Solution Manager
Questions?
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after
the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.