WHAT IS NLS

SAP NetWeaver BW Near-Line Storage

Implementation for Sybase IQ

SAP NetWeaver BW PM

July 2013

2013 SAP AG. All rights reserved. 2

Disclaimer

This presentation outlines our general product direction and should not be relied on

in making a purchase decision. This presentation is not subject to your license

agreement or any other agreement with SAP. SAP has no obligation to pursue any

course of business outlined in this presentation or to develop or release any

functionality mentioned in this presentation. This presentation and SAP's strategy

and possible future developments are subject to change and may be changed by

SAP at any time for any reason without notice. This document is provided without a

warranty of any kind, either express or implied, including but not limited to, the

implied warranties of merchantability, fitness for a particular purpose, or non-

infringement. SAP assumes no responsibility for errors or omissions in this

document, except if such damages were caused by SAP intentionally or grossly

negligent.

Agenda

What is Near-Line-Storage (NLS)?

What is the NLS Interface in SAP NetWeaver Business Warehouse (BW)?

What is an implementation for the BW NLS Interface details?

Whats new with the SAP owned BW NLS implementation for Sybase IQ?

Agenda Topic: What is Near-Line-Storage (NLS)?

Strategy & Definition

Differentations

Design and Runtime Aspects

Key Facts


Strategy & Definition Near-Line Storage (NLS)

SQL-based direct access for reporting and ETL

Extracted NLS partitions are deleted in RDBMS

NLS partitions of an InfoProvider are read-only

NLS is primarily partitioned by time/age


Strategy & Definition BW powered by SAP HANA and Sybase IQ NLS

HANA DB

InfoProvider

Near-line

Storage

Acquisition

NLS Interface

BW

Access - very frequently frequently not frequently rarely

Optimized NLS load performance using IQ

Loader functionality

SAP HANA and IQ share the same columnar paradigm

NLS data compression around 90%

Can handle large data volumes

Suitable for ad-hoc queries with long history

Minimum administrative effort


Data is read and/or written frequently In memory No restrictions, all features available

Infrequent access On disk, no need to keep in memory all the time No restrictions, all features available

Sporadic access Not stored in HANA DB; stored in Near-line Storage Restricted to NLS capabilities

Differentations: Multi Temperature Data Approach

Non-Active Data Concept

Providing lower TCO by optimized data volume management

hot

warm

cold

NLS Management for read-only data


DBMS

InfoProvider

BW

DB

Design and Runtime Aspects Part 1

Q3/2010

Q4/2010

Q1/2011

Newer

Q1/2010

Q2/2010

NLS Implementation


NLS Implementation

Design and Runtime Aspects - Part 2

DBMS

InfoProvider

BW

DB

Q3/2010

Q4/2010

Q1/2010

Q2/2010

Q1/2011

Newer

for

Locked

update

Q3/2010

NLS - Interface


Design and Runtime Aspects - Part 3

DAP Design Time Define partitioning criteria Generate DAP

DAP Scheduling / Monitoring / Runtime

Describe partition to be extracted Generate request InfoProvider Manage Menu Process chain Archive / Extract (DAP menu, process chain

Restore (DAP menu)

Additional Features

Query access to NLS DTP access to NLS Look up during transforms

Data Archiving Process (DAP)


BW - Key Facts about NLS Interface

NLS Interface Features

Supports archiving of InfoCubes and DataStore Objects

Mainly time-based archiving, but can also be based on other characteristics

Lock of the archived time slices in the original InfoProvider

NLS process steps in process chain handling

Copes with changes in the metadata to the BW objects of the archived data

Deletion of NLS partitions supported

Direct load from ADK archive files into NLS supported

NLS should be part of a data-aging strategy

Based on well-established SAP/BW archiving concepts

Data consistency guaranteed before deleting the data from source

Data archived in NLS can be incorporated into reporting

Saves storage costs and other system resources

High compression rate up to 90% (not on top of HANA compression)

Increases retention period for analysis data

Included in the query statistic data collection (RSRT)

Agenda Topic: What is the NLS Interface in BW?

Functionality

Data Management with Near-Line Storage

Modeling Objects

Online Impressions


Functionality Overview: Internal NLS Interface (ABAP Objects)

A kind of NLS Development Kit

Set of APIs

Select time slice of an InfoProvider for NLS

handover

Copy data to NLS Solution Provider and

wait for success message

Verification of copied data

Deletion of data in BW DB

Reload and delete handling for NLS data

partitions


Near-Line Storage Implementation Challenges

Analysis and Reporting operate on a combination of online data

and near-line data. Data consistency is of paramount importance.

Transition processes that affect several storage types have to

guarantee consistency:

Archiving and deletion of online data has to be one logical

unit of work (LUW).

Rollback mechanisms for single archiving steps have to be

available.

The Archive is getting the attidude of a database

Processing of a data archiving process (DAP) is therefore

structured in different phases:

The request is generated

The Data area of the request is locked against changes

The write phase is completed

The verification phase ends successfully

The deletion phase is confirmed, and the request completed


Data Management with Near-Line Storage Implementation Aspects

1) Create a data archiving process

2) Create and schedule archiving requests

3) Restore archiving requests

4) Load data to subsequent data targets

5) Look-up during transformation

6) Query settings

7) MultiProvider settings

Data Propagation

Layer

Corporate

Memory

Data

Acquisition

Layer

DataSource

InfoSource

InfoPackage

Near-Line Storage

Reporting Layer

(Architected Data Marts)

MultiProvider SAP Sales InfoCube

DAP

DTP DTP

PSA

DTP

1

2 3

4

Near-Line Storage

6

5

7

Near-Line Storage

DTP

LS

A


Online Impressions: BW NLS Connection to Sybase IQ 16


Online Impressions: BW NLS Archiving Request Log


Online Impressions: BW NLS Runtime Latencies

Agenda Topic: What is an implementation for the

BW NLS Interface?

Architecture

Partner Solutions

Differentiators


Architecture: The Near-Line Storage Solution for BW

The near-line storage interface allows partners to implement a connection between BW and their archiving/ NLS solution

A typical third Party NLS Solution

is implemented in the SAP BW ABAP stack in partner-specific namespaces

has to pass a certification process

can offer a specific Application Area in the SAP Support Portal

has to be licensed in addition to SAP licenses

can have a different release cycle to SAP NetWeaver BW

SAP BI Frontend

Bex-Query

SAP NetWeaver BW

DB Interface

Layer

NLS Partner

Solution

BW OLAP Engine

DBMS

Interface

General Nearline

Interface

Nearline Partner

Implementation

BW Online DB


SAP-Defined Certification Category: Near-Line Storage Existing Partner Solutions

The full list of NLS certified

partners is provided under:

http://www.sap.com/ecosystem

/customers/directories/SearchS

olution.epx

Source: Certified Partners


Differentiators: Overview of BW NLS Partner Solutions

Different storage types supported

Databases IBM DB6 (from IBM)

Sybase IQ (from PBS)

MaxDB (for SAPs T&E implementation) Informatica Nearline (former SAND DNA/

from Informatica)

BW database inbound Underlying BW DB (from Datavard)

File system ADK storage systems (from PBS)

Different license models

Volume based User based Mixed approach

In general, customers would like to have

the entire NLS solution from one provider


BW powered by HANA NLS Example with PBS CBW and Sybase IQ


Save Yourself Time

The Healthy System Dont start thinking about data archiving when your system is about

to crash!

Timely Planning Proactive action to prepare

sustainable system performance

Interdisciplinary Process

Data archiving requires a large amount of

coordination between IT and people responsible

for applications.


Data Analysis and Assistance for ROI Analysis

Sizing of near-line storage solutions:

Hardware sizing of the near-line storage solution has to be performed by the vendor

Various near-line storage technologies on

the market

From database solutions, to file-based

solutions, to column-based storage

solutions

Data volume services by SAP Active Global

Support (AGS):

http://service.sap.com/dvm Delivers a thorough analysis of BW objects distribution

Can help to estimate the data volume that can be archived/transferred to NLS for the

largest InfoProviders in the system

Only covers technical facts (and not the customers business requirements)

Agenda Topic: Whats new with the SAP-owned BW NLS Implementation for Sybase IQ?

Motivation

Implementation Details

Product Roadmap Focusing BW NLS Implementation for Sybase IQ

Performance: Prelimenary Results

Demo: BW 7.30 SP09: NLS Interface Implementation for Sybase IQ


Motivation

An SAP- owned BW NLS implementation is

urgently required by the field organization to

be able to offer a seamless solution (without

third party involvement)

Main aspects of NLS:

Separation of frequently accessed (hot) data from less frequently accessed (cold)

data, keeping HANA in memory costs to a

reasonable level

Complementary to active/non-active

classification in HANA

Data in NLS is highly compressed in cost-efficient storage, with fewer backups and

reduced operational costs


Strategy & Definition BW powered by SAP HANA and Sybase IQ NLS

HANA DB

InfoProvider

Near-line

Storage

Acquisition

NLS Interface

BW

Access - very frequently frequently not frequently rarely

Optimized NLS load performance using IQ

Loader functionality


NLS data compression around 90%





Manages and analyzes statistical measures of the entire nation of Canada / Analyzes all federal tax

returns in the US

Stores and analyzes massive amounts of industry segment data in 30 of the largest information

providers in the world

Analyzes complex models in more than 200 financial institutions worldwide

Stands out as the leading enterprise data warehouse amongst the largest banks, insurance agencies,

and telecom operators in Korea

Repeatedly demonstrated leading performance in the industry standard TPC-H benchmark

Did you know? SAP Sybase IQ Facts


SAP Sybase IQs Secret Ingredient: Column-Based Architecture

Data is stored vertically each column is stored separately

The data is the index

Retrieve only columns used in the query reduces system I/O dramatically

Data is stored horizontally

Querying without indexes and views is extremely I/O intensive

Building indexes and views is a huge drain on time and resources

Database footprint must be dramatically expanded to make the environment

efficient for querying


BW powered by HANA and Sybase IQ Near-Line Storage (NLS) Architecture - Overview

BI Clients

SAP

HANA Sybase

IQ

MultiProvider Transient

Provider InfoCube/DSO

Near-Line SDK

SAP Netweaver BW 7.3x

Partner

OEM

BW NLS4IQ

SAP Native

An SAP - owned BW NLS

implementation for Sybase IQ

offers a fully integrated solution

from one provider

Main aspects:

Deliver an ABAP-based implementation of the BW NLS

interfaces

Deliver a Sybase IQ DBSL light that covers all the needs of the above-mentioned NLS

implementation

Sybase IQ to deliver reliable, high-performance execution of

the DBSL driven loads and

queries

Availability in Q2/2013

SAP owned alternative to

existing NLS-Partner

Solutions


ODBC Driver from IQ

HANA DB

BW NLS Implementation for Sybase IQ (DBSL-Based

Solution)

InfoProvider

NLS Interface

Near-Line Adapter (ABAP based )

BW

DBSL for

Sybase ASE (/w IQ ext.)

(Kernel based; Open SQL + Native IQ

SQL via ADBC)

BW Porting

General Porting

DB Client


Characteristics of SAPs BW NLS Implementation for Sybase IQ

Tight integration with Sybase IQ via DBSL (direct communication)

Mass insertion into Sybase IQ with IQ loader (LOAD INTO USING CLIENT FILE) ~ 200 Million rows per hour

Integrated backup & recovery strategy (TRIGGER EVENT)

Partitioning

Available since BW 7.30 SP09


Planned Innovations Future Direction Today

SAP NetWeaver BW Product Roadmap Focusing BW NLS Implementation for Sybase IQ

Upcoming planned

release

Platform Aspects

Sybase IQ ODBC NLS client for BW Application Server on AIX

Supporting Sybase IQ 16.0

Error DTP Handling for locked NLS

Partitions

During DSO Load

BW NLS downport to BW 7.0

BW NLS Enhancements

NLS support for BW Inventory Cubes/DSOs

Downport option open

Future innovations

HANA Integration

Performance optimizations for NLS queries using SAP HANA data

federation

BW NLS Enhancements

DAP proposal/generation based on DVMS (delegation to external resources)

Update options for NLS partitions based on error DTP

Data Management Aspects

SAP DBA Cockpit supporting Sybase IQ

Disaster recovery handling consistency between BW/IQ

High availability in BW not endangered

Additional Support

For other platforms covered by Sybase IQ ODBC,

like IQ Client Support for HP-UX

(until end of 2013) (* 7.31/SP07 begin Q2/2013)

SAP NW BW 7.30

SP09*

Implementation specifications

DBSL-based direct connection to Sybase IQ 15.4

Performance optimization for data loads based on IQ Loader

technology

Using Sybase IQ techniques for very large data base

(VLDB) and large binary

objects (LOB)

Supported Platforms

Sybase IQ ODBC NLS client for BW Application Server on

Windows and Linux (x86_64)

Supporting Sybase IQ 15.4 ESD#3


Preliminary Results from Ongoing Performance Tests

Environment:

BW 7.30 on HANA Rev. 42 (Online DB) IBM M-Box Intel Xeon X7560 64 Cores + HT 2,27 GHz 512 GByte RAM

Sybase IQ 15.4 (Near-Line Storage) Nehalem BWA-Blade Intel Xeon X5670 12 Cores + HT 2,93 GHz 7

2 GByte RAM

Setup:

SPO partitioned by 10 InfoCubes with 10 million rows each

Single target table on IQ side partitioned by archiving request (time slice/data slice)

Results:

64 million rows/hour (26 GByte/hour) for single relocation run (1 cube)

238 million rows/h (97 GByte/hour) for 4-fold parallel relocation run (4 cubes)

Further potential for improvement in single process mode and scalability (for example IQ 16.x)

Demo: BW 7.30 SP09: NLS Interface Implementation for

Sybase IQ


Features of the BW NLS Implementation for Sybase IQ

Optimized NLS data transfer throughput using IQ Loader

functionality


Expected data compression around 90%





Further Information SAP Public Web

SAP NetWeaver Business Warehouse SCN Space http://scn.sap.com/community/data-warehousing/netweaver-bw

BW Powered by HANA Overview on experiencesaphana.com https://www.experiencesaphana.com/docs/DOC-1522/

SAP NetWeaver BW 7.3 powered by SAP HANA Release Notes on SAP Help

http://help.sap.com/nw73bwhana/#section1

SAP NetWeaver BW 7.3 Documentation Section Using the SAP HANA Database http://help.sap.com/saphelp_nw73/helpdata/en/AF/81F38183C

F46B9B919F82D7BB7EC47/frameset.htm


More Information ...

NLS Landing Page in SCN:

http://scn.sap.com/docs/DOC-39944

Thank you

201 SAP AG. All rights reserved.

201 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the United States and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries.

Oracle and Java are registered trademarks of Oracle and its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc.

HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of Technology.

Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc.

IOS is a registered trademark of Cisco Systems Inc.

RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited.

Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are trademarks or registered trademarks of Google Inc.

INTERMEC is a registered trademark of Intermec Technologies Corporation.

Wi-Fi is a registered trademark of Wi-Fi Alliance.

Bluetooth is a registered trademark of Bluetooth SIG Inc.

Motorola is a registered trademark of Motorola Trademark Holdings LLC.

Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360, and B2B 360 Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.

Documents

WHAT IS NLS