Technical presentation

Courtney Claussen, SAP Sybase IQ Product ManagementOctober 19, 2012

TEC1227Complete Business Analytics with SAP Sybase IQ and SAP BusinessObjects Software

© 2012 SAP AG. All rights reserved. 2

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.


Agenda

SAP BusinessObjects and SAP Sybase IQ Better together

What is so cool about SAP Sybase IQ? Some of the key features

Proof points Customer stories and Monsta benchmark

SAP BusinessObjects integration with SAP Sybase IQ Now and going forward

SAP BusinessObjects Analytics Editions A great combination of database, BI and EIM

Summary Wrapping it up

SAP BusinessObjects and SAP Sybase IQ


Operational reporting

Executives & Manager

Business Analysts

“normal” user =information consumers

Tactical users

Trends, graphics & dashboards

Flexible, analytical tools

Static, easy-to-use reports

Explorer / Web Intelligence

Explorer / Crystal Reports

ECC Reports

Make BI pervasive with SAP BusinessObjects

Xcelsius


BI Platform

Dashboards

CrystalReports

WebIntelligence Analysis

Explorer

Trusted Data

SAP BusinessObjects BI 4.1


BI Also Needs Solid Data Foundation


SAP Sybase IQ delivers a solid foundation

. . .

BI Platform

Dashboards

CrystalReports

WebIntelligence Analysis

Explorer

Trusted Data


Stands out as the leading enterprise data warehouse among the largest banks, insurance agencies, and telecom operators worldwide

Manage and analyze statistical measures for the entire nation of Canada

Analyze complex models in more than 200 financial institutions worldwide

Analyze ALL Federal tax returns in the US

Store and analyze massive amounts of industry segment data in 30 of the largest information providers in the world, including Transunion, Nielsen and Axiom

SAP Sybase IQ Big Data AnalyticsPervasive across data intensive industries worldwide


SAP Sybase IQMature, industrial strength analytic DBMS

LEADERSHIP• Industry leading

performance & scale benchmarks

• Recognized EDW market leader by Gartner, Forrester

• Pioneering technology with 10+ patents

ADOPTION

• 4500+ installations in 2150+ accounts

• ~200 new customer wins per year (last 4 years)

• Consistently 96%+ customer satisfaction rates

MOMENTUM

• 2 x DW market growth rate (last 4 years)

• Fast paced product releases

• v15, v15.1 (2009), v15.2 (2010), v15.3, v15.4 (2011)

Ericsson • Sungard • Nielsen • BNP Paribas • Telefonica • hmv.com • comScore • Agricultural Bank of China


SAP BusinessObjects and SAP Sybase IQ#1 BI platform and column-based analytics server

Profil

ing

Mat

chin

g

Clean

sing

ETL/CDC

Calcu

late

Model

s

Prese

ntatio

n

SBOP Data Services SBOP BI Tools SBOP Applications SAP Sybase IQ

Applicat

ions

• Column Store Analytics DBMS

• Governance, Risk, Compliance Analytics• Business Analytics Applications - Banking - Telco - Retail - Insurance - Hi-Tech - Public Sector

• SAP BusinessObjects BI Platform - SAP Crystal Reports - Web Intelligence - Dashboards -Explorer -Predictive Workbench

• Data Integrator (ETL)• Data Quality Mgmt• Data Replication• Text Analytics

What is so cool about SAP Sybase IQ?


Key characteristics Data is stored vertically — each column is stored separately The data is the index Large page sizes (128 KB–512 KB) Persistent row identifiers Bitmap-driven

Column store processor

Key Benefits Unsurpassed concurrent, mixed workload performance, and storage efficiencies Ability of queries and updates to access only referenced columns and pages Single data type and domain per page for significant enhancement of the effectiveness of dictionary-based

compression Large page size for better use of today’s disk and I/O subsystems Ability of queries to evaluate multiple predicates on the same table using index-based access methods Bitmaps for significant efficiencies — compact representation, and perfect pre-fetch of rows to avoid cache misses

and Least Recently Used (LRU)

Sybase IQ

1 2 3 4 5 6 7 8 9 …

r1

r2

r3

r4

r5


SAP Sybase IQ 15.4A complete platform for data analytics

Most mature column store

Comprehensive lifecycle tiering

MPP queries + Virtual Marts + User scaling

High Speed loads

Structured + Unstructured Store

Comprehensive ANSI SQL w/OLAP

Built-in Full Text Search

InDB Analytics w/ MapReduce + simulator

Web 2.0 APIsBig Data

OpnSrc APIs

Optimized BI,EIM, Model, Replicate

Dev and admin tools Predictive Analytics Packaged ILM apps

Bradmark, Symantec,

Whitesands, Quest, ZEND

SAS, SPSS, KXEN, Fuzzy Logix,

Zementis, Visual Numerics

BMMSoft, SOLIX, PBS

Sybase PowerDesigner,Sybase Replication Server,

SAP BusinessObjectsISYS, Panopticon

Eco-System

App. Services

DBMSHadoop,

R


Mathematical• Matrix algebra• Area under curve• Interpolation methods• Distance measures

Statistical• Descriptive statistics• Distance measures• Hypothesis testing• Cross tabulation• Anova

Univariate Distributions• Monte Carlo simulation• Univariate distributions

Data Mining• Linear regression• Logistic regression• Principal component

analysis (PCA)• Cluster analysis• Naive Bayes

OLAP & Time Series• Correlation• LAG, LEAD• Weighted average

In-database analytics


Total time, in seconds, to load the data from the database and perform the computations

In-database analytics performance

http://www.kth.se/en


Full text queries

Full text queries SELECT * FROM myTable WHERE CONTAINS (TextCol, ‘d’); – returns rows SELECT * FROM myTable CONTAINS (TextCol, ‘d’); – returns rows and scoring SELECT * FROM myTable WHERE CONTAINS (TextCol, ‘a AND NOT b’); – Boolean SELECT *FROM myTable WHERE CONTAINS (TextCol, ‘a NEAR b’); – proximity

Text IndexID Term Pos Info

0 a 1,3,4

1 b 1,5

2 c 1

3 d 2,3,4

4 e 2,4,5

5 f 2,5

TextCol

a b c

f e e d

d a d

d e a d

b e e f

…

…

Full Text Queries

Visualization

Text Load File ingestion into blob, clob

Text Filtering Filtering to plain text, formatting

Schema Transform

Hierarchical to relational

EntityExtraction

CategorizationTokenization


Massively parallel processing• Leader node: Receives and initiates queries, including UDFs

– Any node can be a leader, one leader per query, many concurrent leaders possible– Leader node may satisfy query within itself

• Worker node: Nodes picking up work units from leader– Many worker nodes per query, same worker node can serve multiple queries– Worker nodes are enlisted only if leader cannot satisfy query on its own

SAP Sybase IQ shared everything MPP architectureDistributed query processing

Query 24 node DQP

Query 15 node DQP

SAN Fabric


Support different user communities with virtual data marts

SAN Fabric

Full Mesh High Speed Interconnect

VDM1

VDM2

Shared

� Characteristics • VDM can isolate applications, workload, users,

data• DQP within VDM boundaries only• Dynamic/scheduled resource reassignment b/w

VDM

� Advantages• Effective user group assignment to

resources• Effective workload isolation and partitioning• Economical scalability


Accelerate performance with distributed query processing (DQP)

61 sec

6 TB

296 sec 17 sec

SAP Sybase IQ 15.3 with PlexQ guarantees SLAs now and into the future

SAP Sybase IQ 15.3 with PlexQSAP Sybase IQ 15 before PlexQ

394 sec 1341 sec

12 sec

124 sec 51 sec

6.5x

4.5x

7.3x 4.3x

Watch CoreLogix LoanPerformancevideo on YouTube:

http://www.youtube.com/watch?v=L8ilR4vrIh0


SAP Sybase IQ Hadoop Integration

Technology

ETL Hadoop data into IQ Fetch HDFS subsets into IQ store; Process entirely in IQ

Results and Applications

Client side federation Join pre-computed IQ and Hadoop results

SAP Sybase

IQ

SAP Sybase IQ

Federate Hadoop data into IQ

Fetch HDFS subsets by materializing inside IQ as in-memory virtual tables Triggered on the fly as part of query

SAP Sybase IQ

Federate Hadoop jobs into IQ

Trigger MapReduce job in both IQ and Hadoop and join the results on the fly

C++/JAVA UDF

Files

SAP Sybase IQ

C++/JAVA UDF

Quest


Real world benefits of SAP Sybase IQ and SAP BusinessObjects#1 BI platform and column-based analytics server

• Up to 1000 times faster than traditional relational databases• Critical for ad hoc analyses of large datasets• Scales to a large number of concurrent users• Can handle the most complex of queries and thousands of users

High Performance

• Compresses data to save on maintenance and storage costs• Minimizes the need for aggregates, indexes and cubes• No need to tune and optimize each query• Rapid time to value

Lowest TCO

• Industry leader for column-based databases• 4500+ installations in 2150+ accounts• Technology leader with 10+ patents• Leader in Gartner Magic Quadrant and Forrester Wave

Industry Leader

• Mature product line at 15 years in the market place• 96% customer satisfaction rate• 2x DW market growth rate (last 4 years)• Fast paced product releases with innovative features

Proven Technology

Proof points


Dramatically improving BI together

BNP was experiencing substantial growth in the use of their Web-based BOBJ reporting application and chose SAP Sybase IQ to accelerate online transactions.RESULT: online customers generated twice as many reports 3x faster, freeing up OLTP resources.

Spice Telecom was able to populate a data warehouse with raw mobile and landline call data, analyzing data sets not previously possible, opening access to all departments.RESULT: achieved a 10x performance improvement with SAP Sybase IQ.

REPORTING (3x)

Oracle Sybase IQ Oracle Sybase IQ

DISK SAVINGS (50%)TransUnion was able to process more than half a billion records each month with different data formats from different sources. RESULT: achieved enhanced data management, higher query speeds, lower costs, and a future-proofed system with SAP Sybase IQ.

DB2 Sybase IQ

ANALYTICS (6x)

Wellmark chose SAP Sybase IQ as its analytics server to decrease reporting times and claims processing, as well as centralizing data silos at the enterprise level.RESULT: reduced software costs by 200-300% and dramatically reduced people and hardware costs.

Hardware DBAs Software Storage

DB2 Sybase IQ

SOFTWARE SAVINGS(300%)

http://www.wellmark.com/scripts/redirect.asp?List=/index.aspx





Monsta benchmark

SAP Co-Innovation Lab project:

Validate performance and scalability of a large BI deployment deployment capable of sustaining 10,000 concurrent users! Software

– SAP BusinessObjects BI 4 as the BI platform and SAP Sybase IQ 15.4 as the reporting database– Application delivery control system from F5– Continuous energy monitoring and management from OsiSoft

Commodity hardware (super micro Twinblades and SuperServers) running on RedHat RHEL 6 Simulated user traffic with Soasta CloudTest. Data:

– TPCH Scale Factor 30 (34 GB data)– 4.5 million customers– 300K vendors– 45 million orders and 180 million line items

Reports:– 2500, 5000, 10,000 and 25,000 line reports


Hardware available to the project team

Hardware (turned out to be much more than needed!) 80 hosts - 40 Supermicro Twinblades each with 24 core Intel

Westmere processor and 96GB RAM - total of 960 core and 3840GB RAM

– 12 hosts allocated to SOASTACloud Test Maestro to manage all load generation, load monitoring and reporting

– 21 hosts are allocated for SAP Sybase IQ as the reporting DB– The balance of blades allocated to BI4

One Supermicro 2U Super Server with Westmere 40 core and16GB RAM

– Dedicated to Sybase ASE as CMS DB


Monsta benchmark results

Report refresh time was consistently 6-7 seconds on average under load (3 seconds when the environment was completely idle) and within 19 seconds for 90% of all requests

At a load of 10,000 concurrent users, the physical footprint was contained to:– just 10 host machines for the entire BI 4

environment (excluding the File Repository Servers, which in a normal scenario would run on a CMS host)

– 10 SAP Sybase IQ servers for the reporting database

– 1 SAP Sybase ASE instance.

SAP BusinessObjects integration with SAP Sybase IQ


BusinessObjects UniverseA common semantic layer


SELECT RESORT_COUNTRY.COUNTRY, RESORT.RESORT, SUM( INVOICE_LINE.DAYS * INVOICE_LINE.NB_GUESTS * SERVICE.PRICE )

FROM INVOICE_LINE INNER JOIN SERVICE ON ( INVOICE_LINE.SERVICE_ID=SERVICE.SERVICE_I D) INNER JOIN SERVICE_LINE ON ( SERVICE.SL_ID=SERVICE_LINE.SL_ID ) INNER JOIN RESORT ON ( SERVICE_LINE.RESORT_ID=RESORT.RESORT_I D) INNER JOIN COUNTRY RESORT_COUNTRY ON ( RESORT.COUNTRY_ID=RESORT_COUNTRY.COUNTRY_ID ) GROUP BY RESORT_COUNTRY.COUNTRY, RESORT.RESORT

BO universe

Pre-requirements for ad-hoc reporting with relational dataSimple query creation without SQL or database expertise


SELECT RESORT_COUNTRY.COUNTRY, RESORT.RESORT, SUM(INVOICE_LINE.DAYS * INVOICE_LINE.NB_GUESTS *

SERVICE.PRICE)

FROM INVOICE_LINE INNER JOIN SERVICE ON

(INVOICE_LINE.SERVICE_ID=SERVICE.SERVICE_ID) INNER JOIN SERVICE_LINE ON

(SERVICE.SL_ID=SERVICE_LINE.SL_ID) INNER JOIN RESORT ON

(SERVICE_LINE.RESORT_ID=RESORT.RESORT_ID) INNER JOIN COUNTRY RESORT_COUNTRY ON

(RESORT.COUNTRY_ID=RESORT_COUNTRY.COUNTRY_ID)

GROUP BY RESORT_COUNTRY.COUNTRY, RESORT.RESORT

BO universe

Pre-Requirements for ad-hoc reporting with relational dataSimple query creation without SQL or database expertise


Best-in-class Optimized & Differentiated Business Analytics Software Platform

From Certification and Support:

Towards a unified out of the box Business

Analytics StackSAP BusinessObjects

Data Services

SAP Sybase IQ

Analytics DBMS

SAP BusinessObjects

Business Intelligence

SAP BusinessObjects

Analytical Applications

Best-in-class Certified & SupportedBusiness Analytics Software Stack

Certified

and

sup

po

rted acro

ss stack

SAP BusinessObjects

Data Services

SAP Sybase IQ Analytics Server

SAP BusinessObjects

Business Intelligence

SAP BusinessObjects

Analytical ApplicationsOp

timizatio

n an

d D

ifferentiatio

n

across stack

Single Sign-On Access Control

Unified global support

Today Tomorrow

• Integrated technologies for lower administration and system costs• Cutting Edge Text Analytics

BENEFITS• Rapid deployment of new analytics applications • Leading Performance for Ad Hoc Analysis• Handles huge numbers of users

Disclaimer: The forward looking statements above represent our current judgment on what the future holds, they are subject to risks and uncertainties that could cause actual results to differ materially. This is not an announcement of product availability.

SAP Sybase IQ & SAP BusinessObjectsBusiness Analytics Vision

SAP BusinessObjects Analytics Editions


Today's world: end-to-end best-of-breed stack from SAP

Analys

is

Mod

eling

Calcula

te

Mod

els

Prese

ntat

ion

Discov

ery

Answers

IQ

AnalyticDBMS

SAP Data Integrator SAP BusinessObjects BISAP Sybase IQData Sources e.g. Oracle 11g, IBM DB2,

Microsoft SQL Server ….

Heter

ogen

eous

Data

Operational DataTr

ansfo

rmLo

adCDC

Extrac

t


Now in one complete package from SAP

Analys

is

Mod

eling

Tran

sform

Load

CDC

Calcula

te

Mod

els

Prese

ntat

ion

Discov

ery

Answers

IQ

Heter

ogen

eous

Data

Operational Data

SAP BusinessObjects BI, analytics edition

Extrac

t

AnalyticDBMS

Data Sources e.g. Oracle 11g, IBM DB2,

Microsoft SQL Server ….

Summary


SAP Sybase IQ & SAP BusinessObjects Complete Business Analytics Portfolio

Answers

SAP BusinessObjects EIM SAP Business Objects BI

IQ

SAP Sybase IQ

Data Warehousing

Data preparation Data Usage

Data Modeling / Database Design Tool

Event Processing

Fast ROI, Solution Deployment

An Integrated Best of Breed StackEach functional area – EIM, DBMS, BI recognized as market leader - analyst ratings and customer adoption

Acquire - one vendor solution and supportDevelop - build apps on total system optimized stackDeploy – pre-certified, market proven BI stack / appsSustain - coordinated support, long term commitment

Value Proposition


Further Information

SAP Public Web

Download SAP Sybase IQ Express Edition (Free):http://www.sybase.com/iqexpressedition

Watch SAP TechEd Onlinewww.sapteched.com/online

http://www.sapteched.com/online

FeedbackPlease complete your session evaluation for TEC 1227.

Thanks for attending this SAP TechEd session.

Click icon to add picture

[email protected]

mailto:[email protected]

Documents

Technical presentation