Upload
mara-crosland
View
216
Download
2
Tags:
Embed Size (px)
Citation preview
Data Warehouse Data Warehouse AcceleratorAccelerator
Michael Wallace
Principal Systems Consultant
2
Why a Data Warehouse Accelerator?Why a Data Warehouse Accelerator?
“Forces related to the new business climate may be fueling an acceleration in the growth of the data
warehouse.... Doing more with less is both the mandate and the mantra across the board.
“More than ever, businesses need dramatic increases in the price/performance of their
systems.
-Richard Winter, VLDB Expert, Winter Corporation
3
Sound All Too Familiar?Sound All Too Familiar?
How do I protect my revenue that is at risk?How can I better leverage detailed (non-aggregate)data?How can I keep more data online without incurring prohibitive storage expense?How do I perform real-time analysis?How do I run ad hoc queries anytime?How do I get queries to return within seconds or minutes, not hours or days?How do I free up my DBAs from constant warehouse tuning?How do I add users and data without causing performance or architectural disruptions?How can I help my business users be more independent and productive?
• How do I get this project up and running this quarter?• How do I demonstrate a quick and solid return on my investment?• How do I minimize my risk?
4
Need to get your data back under control?Need to get your data back under control?
5
Hundreds of companies have benefited from the from the
power, flexibility, scalability and low, low TCO of the IQ
Warehouse Accelerator
Use all your warehouse data….
a subset of your warehouse data….
Or a new set of data…..
Here’s how.
6
Typical Data Warehouse Architecture
7
The IQ Accelerator enhances the performance and ROI of any existing data warehouse.
Gauranteed!
8
Sybase IQ – What is it?Sybase IQ – What is it?
The Database for Analytic Applications
The Only RDBMS 100% designed for decision support
It’s Faster It’s More Scalable
It’s More Economic It has huge market momentum
9
Sybase IQ Architecture Sybase IQ Architecture
An IQ Database
Is a database with tables
The tables have columns
Uses “indexes” to speed retrieval
Schema Independent
Star
Relational
Flat
Applications Connect to IQ via:
ODBC
JDBC
Open Client
An IQ Database Provides:
Stored Procedures, Functions, and Batches
Views
On-Line Backup
Concurrent Readers/Writers
Transactions
User Defined Functions and User Defined data types
Crash Recovery and logging
Common Language ProcessorANSI 92 SQL
Transact SQL
JAVA
10
Sybase IQ – What’s Different?Sybase IQ – What’s Different?
Data is Stored Vertically
Each column is stored separately
Bit-Mapped Index
Index on every column
Optimized Storage
Input data is typically compressed
Usually = 30-40%
Database smaller than input data
Even with all the indexes
NOT an MPP Solution
Much simpler implementation
Much simpler management
Query Engine Retrieves Only Columns Used in the Query
Reduces system I/O dramatically
Average 90% Less than competition
Permits better data manipulation
Schema Design Not Restricted
Design based on application use
Flat, Star, Relational, Snowflake
Any Schema
11
IQ Multiplex ArchitectureIQ Multiplex Architecture Scalability Scalability
A single copy of data shared across multiple computer nodes
All data and indexes stored in the shared database
No partitioning of data required
No distributed lock management
System does not lock on queries or refresh
Individual nodes are Independent of Other Nodes.
Each IQ Node has its own local Temp Space and catalog.
Individual nodes can be different configurations (CPUs, memory, disk).
Data Store (SAN)
Fiber Channel Backbone
ASIQ ReaderASIQ
Writer/Reader
12
IQ Multiplex ArchitectureIQ Multiplex ArchitectureVertical ScalabilityVertical Scalability
Data Store (SAN)
Fiber Channel Backbone
ASIQ ReaderASIQ
Writer/Reader
ASIQ Reader
Individual nodes can be different configurations (CPUs, memory, disk).
Each IQ engine runs independently, using all available CPUs on its own node.
Additional CPUs scale linearly when added to existing nodes
IQ is CPU, not I/O, bound
13
IQ Multiplex ArchitectureIQ Multiplex ArchitectureHorizontal ScalabilityHorizontal Scalability
Data Store (SAN)
Fiber Channel Backbone
ASIQ Reader
ASIQ Writer/Reader
ASIQ Reader
No data redistribution
No change in schema
Start small and grow HUGE.
Load balancing can be used to spread out users.
Up to 120 nodes
14
Data Store (SAN)Data Store (SAN)
IQ Multiplex ArchitectureIQ Multiplex ArchitectureFlexible ScalabilityFlexible Scalability
Data Store (SAN)
Fiber Channel Backbone
ASIQ Reader
ASIQ Writer/Reader
ASIQ Writer/Reader
ASIQ Reader ASIQ Reader
Re-use old hardware
Grow writer node as needed
Increase disk storage without adding nodes
Up to 30+ PetaBytes of disk storage
SMP-like management & tuning
High Availability provided through multiplexing
15
IQ Multiplex ArchitectureIQ Multiplex ArchitectureUser ScalabilityUser Scalability
EnterpriseEnterprise
WorkgroupWorkgroup
VLDBVLDB
WebWeb
Flexible Scalability = User Scalability
16
Why Sybase IQ?Why Sybase IQ?
Speed: IQ speeds up queries of your existing warehouse by 10-1000X
Scalability: Scales to thousands to users with virtually no degradation of performance
Flexibility: Allows ad hoc queries anytime with no additional tuning required
Low risk: Already tested, tuned, configured to insure success (world’s largest data warehouse built and certified on IQ/Sun boasts 48.2TB input data, 22TB final size)
Simplicity and elegance: Leading-edge, patented architecture guarantees quick installation, low management and maintenance costs.
Economy: Saves approx. $1 million per terabyte of input data
17
Need Proof?Need Proof?
Best price-performance in both 300GB and 1000 GB scales
Lowest disk to data ratio: 3 to 10 X better than any other systemAt 1000GB, IQ used 54 disks compared to 1263 and 1408 for competing systems
Best storage efficiency by factor of 10 -- 1TB raw data = 2.4TB storage in IQ, 22TB in competitor
At 300 and 1000 GB scales, IQ is 9 to 25 times less expensive than competitive systems
IQ scores big in TPCH benchmarks
18
Need proof?Need proof?GIGA study illustrates hefty ROI*GIGA study illustrates hefty ROI*
“The organizations interviewed by Giga information group showed actual or expected returns on their investment ….that ranged from 72% to 175%.
“Simply put, for every dollar invested in the Sun/Sybase RA (IQ running on Sun hardware), $1.63 would be returned to the organization in direct cost savings or increased bottom-line profit as a result of increased business.
“Giga Information Group projects that a composite organization facing some of the same business and IT pressures will likewise achieve a return on investment greater than most standard IT hurdles, and such an investment will pay back its investment in a period of between 13 and 15 months of use.”
*The Total Economic Impact of Deploying the Sun-Sybase Enterprise Data Warehouse Reference Architecture, c. 2003
19
Need Proof?Need Proof?
“If you are used to queries that take 24 or more hours to run, and then you are told that you can run them in a matter of minutes with Sybase IQ, this may be hard to swallow. The truth is that using a column-based approach really can produce such performance improvements.… IT managers should be ready to fall off their chairs.”
--Bloor Research, 2002
We are able to deliver one data warehouse for all of our applications at one third the storage of conventional technologies, while seeing performance gains as advertised with IQ.
-Kim Ross, CIO Nielsen Media Research
“Conservatively speaking, the CDW’s (Compliance Data Warehouse) Return on Investment is expected to be 200 to 1.”
--Jeffrey Kmonk IRS
Sybase IQ reduced loading and indexing from 30 minutes to 2.5 to 3 minutes. Query speeds were 20 – 50 times faster than Oracle. Time to add a column was reduced from 4 hours with Oracle to 15 minutes with IQ.
Jeff ButlerDepartment of TransportationBureau of Transportation Statistics
20
Need Proof?Need Proof?Fortis Bank Fortis Bank
Before After
ONE Data Mart (Marketing) Many Data Marts
4 days to load and create all mainframe data into the data warehouse
1.5 TB data loaded every month
8 days to run (in batch) all Business objects reports
115,000 ad-hoc queries/month
Average response time is 4 hours 75% of queries executed within 3 seconds.
Raw Data is only 6 Gigabyte 450 Gigabyte IQ storage
Data warehouse can only be refreshed once a quarter
Data warehouse refreshed every day
20 users 1,000 users
7 external (PWC) consultants write each report
0 external consultants, users write reports themselves.
Full time DBA work DBA work = 10 min. a day
“This is the strongest product I have come across in my career, something I wouldn’t admit that often. There is no doubt whatsoever that another technology would not have offered our users the same service as Sybase IQ.”
--Jean-Louis Catin, IT
21
Hard to believe? Challenge Us!
Nothing to loose –Everything to Gain!
Proof of Concept.
Sybase IQSybase IQ::
The Ultimate Data Warehouse Accelerator