What´s New? SAP HANA SPS 07 - blog.scherbinek.de´s-New... · What´s New? SAP HANA SPS 07 SAP HANA Application Function Library (AFL) (Delta from SPS 06 to SPS 07) SAP HANA Product

What´s New? SAP HANA SPS 07 SAP HANA Application Function Library (AFL) (Delta from SPS 06 to SPS 07)

SAP HANA Product Management November, 2013

© 2013 SAP AG. All rights reserved. 2 Public

Contents

Overview

What’s New for SPS 07: Predictive Analysis Library

What’s New for SPS 07: Application Function Modeler

Overview


Application Function Library: General Overview – 1-

Application Function Library (AFL) – what is it?

Pre-delivered commonly utilized business, predictive and other types of algorithms for use in projects

or solutions that run on SAP HANA. The technology framework enabling the use of these algorithms

is called the Application Function Library .

What are it’s benefits?

These algorithms can be leveraged directly in development projects, speeding up projects by

avoiding writing custom complex algorithms from scratch. AFL operations also offer very fast

performance, as AFL functions run in the core of SAP HANA in-memory DB.

What are its primary libraries?

The Predictive Analysis Library (PAL) is a set of functions in the AFL. It contains pre-built, parameter-

driven, commonly used algorithms primarily related to predictive analysis and data mining.

The Business Function Library (BFL) is a set of functions in the AFL. It contains pre-built, parameter-

driven, commonly used algorithms primarily related to finance.


The Application Function Library Framework in SAP HANA

Application Functions (C++)

SQLScript

HANA Clients (App Server, Analytics Technology, etc)

SAP HANA

Business Function Library

Predictive Analysis Library

AFL Framework

…

AFL Technology includes

Application Functions

Written in C++ and delivered as AFL content by SAP

Predictive Analysis and Business Function Library have been

released in SPS 05 as AFL content

AFL Framework

On demand library loading framework for registered and supported

libraries

AFL are consumed for use from SqlScript via so-called wrapper-

procedures. Consumption can be controlled via permissions.

Beyond the initial script-based approach, the Application

Function Modeler is released with SPS 06 as a graphical editor

to facilitate the design-time process of creating the

wrapper-procedures and can easily be re-used as part of

development workflow.

Parameter Table

AFM


Predictive Analysis Library – General Overview

Predictive Analysis Library (PAL)

• Compiled analytic function library for predictive analysis in HANA SPS 05

• Support multiple algorithms: K-Means, Association Analysis, C4.5 Decision Tree, Multiple

Linear Regression, Exponential Smoothing…

Know Your Business

Decide with Confidence

Compute Quickly

Uncover deep insights & patterns about the business: association rules,

customer clustering, or sales prediction

Drive more advanced analyses. Decision is made with support from

analysis numbers

Query and analyze data in real-time with high-performance

computation in-memory

Help Customers To

Bring decision support capabilities to the business users through

simplified experience and pre-built scenarios Empower the business


The Predictive Analysis Library in SAP HANA Functional Overview as of SPS 06

The Predictive Analysis Library (PAL) is a built-in C++ library to perform in-database data mining

and statistical calculations, designed to provide excellent performance on large data sets.

Embedded in SAP HANA via the Application Function Library

Data Preparation

Sampling

Binning

Scaling

Convert Categorical to Binary

Outlier Detection

Inter-Quartile Range Test (Tukey’s Test)

Variance Test

Anomaly Detection

Cluster Analysis

ABC Classification

DBSCAN

K-Means

Kohonen Self Organized Maps

Time Series Analysis

Single Exponential Smoothing

Double Exponential Smoothing

Triple Exponential Smoothing

Link Prediction

Different Measurements: Common

Neighbors; Jaccard’s Coefficient;

Adamic/Adar; Katzβ

Other

Weighted Scores Table

Classification Analysis

C4.5 Decision Tree Analysis

CHAID Decision Tree Analysis

K Nearest Neighbour

Multiple Linear Regression

Polynomial Regression

Exponential Regression

Bi-Variate Geometric Regression

Bi-Variate Logarithmic Regression

Logistic Regression

Naïve Bayes

Association Analysis

Apriori

Apriori Lite

Status: SAP HANA SPS 06.

Y

X Z


Application Function Modeler

Application Function Modeler A graphical editor to facilitate a faster and easier design-time process of creating the wrapper-procedures

AFL Models are stored as repository objects and can easily be re-used as part of development workflow.

Library Selection

Function List and

Search

Parameters and

specifications

for table types

Model Editor

Drag‘n drop of functions

Template for table types

Data source selection and

automatic mappings to table

types

Sample SQL for

procedure

consumption

What’s New: Predictive

Analysis Library


New Algorithms: Statistical Functions

Univariate

Mean, median, variance, standard deviation

Kurtosis

Skewness

Multivariate

Covariance matrix

Pearson correlations coefficient matrix

Chi-squared tests:

– test of quality of fit

– test of independence

Variance equal test (F-test)


New Algorithms: Substitute Missing Values; Partitioning

Substitute missing values

Continuous variable: replace the ‘null’ value with ‘mean’ or ‘median’

Categorical variable: replace the ‘null’ value with ‘mode’

Partitioning Splits an input table into 3 parts (training table, testing table and validation table).

Union could be a subset of the input table and the validation part could be empty

Two partition methods are supported:

– Random Partition

– Stratified Partition (stratified sampling)


New Algorithms: Support Vector Machine; Forecast Smoothing

Support Vector Machine

A family of supervised machine learning algorithms that analyze data and

recognize patterns, typically used for classification and regression analysis

Holds advantages over other supervised learning models: SVM models can be

either linear or non-linear

Forecast Smoothing

Calculates a set of optimal parameters for PAL functions: Single Smoothing,

Double Smoothing and Triple Smoothing

Outputs the forecasting result by using the best parameter

Parameter optimization is based on global and local search algorithms


New Clustering Algorithms: Agglomerate Hierarchical; Affinity Propagation

Agglomerate Hierarchical Clustering

Widely used clustering method which can find natural groups within a set of data

Groups the data into a hierarchy or a binary tree of the subgroups

The PAL version follows the agglomerate approach: merges the clusters with a

bottom-up strategy.

Affinity Propagation Clustering

A relatively new clustering algorithm based on the concept of "message passing"

between data points

Holds the advantage that the number of clusters does not necessarily have to be

predetermined


Enhancements to Regression Algorithms

Linear regression

Support p-value for each coefficient

Stepwise linear regression (forward and backward only)

Logistic regression

Logistic regression with L1 regularization

Categorical support for input variable

p-value for coefficient


Enhancements to Other Algorithms

Apriori

Output rules in relational format

Add lift as a filter for rules

Provide an option which controls the number of items in right hand side (1 or many)

Decision Tree

Missing value handling in decision tree (mean/mode)

What’s New: Application

Function Modeler


Application Function Modeler SPS 07 Features Overview

Drive AFM achieve a more stable and usable solution

Better integrate with AFL Framework.

UI Improvement: more friendly and usable

Better support the E2E Development Scenarios: create, activation, calling, delivery etc.

Others: Performance improvement, Error Handling

Objectives

Performance improvement: creation, model consistence

check

Error Message: in studio side and server side

Support open file offline but read-only

…

Other

Better support the table type editing

Better naming rule and icon

Better UI Text

UI Improvement

Update table type constrains and default parameter

Support new function in PAL (11 new functions)

Use new AFL API to generate wrapper procedure

Refactor the data model

AFL Framework

View support

Better undo/redo

Calling parameter remove/add/restore/persistence

Schema mapping

…

E2E Development Scenarios


AFM: New Functions

New Algorithm in AFM in SPS 07 Description Category AFM PAL

DBSCAN DBSCAN Clustering SPS 07 SPS 06

LINKPREDICTION Link Prediction Social network analysis SPS 07 SPS 06

CONV2BINARYVECTOR Convert category Type to

Binary Vector Preprocessing SPS 07 SPS 06

SLIGHTSILHOUETTE Slight Silhouette Clustering SPS 07 SPS 06

NBCTRAIN NBCTRAIN Classification SPS 07 SPS 06

NBCPREDICT NBCPREDICT Classification SPS 07 SPS 06

HCAGGLOMERATE Hierarchical Clustering Clustering SPS 07 SPS 07

SVMTRAIN Train with Support Vector

Machines Classification SPS 07 SPS 07

SVMPREDICT Predict with Support Vector

Machines Classification SPS 07 SPS 07

PARTITION Data Partition Preprocessing SPS 07 SPS 07

SUBSTITUTE_MISS Substitute Missing Values Preprocessing SPS 07 SPS 07

….

11 new functions will be supported


AFM: Updated Generated Procedure Definition

New AFL Generator API in Runtime Plugin SPS 06: CREATE PROCEDURE ... LANGUAGE LLANG AS BEGIN ...

SPS 07: CREATE PROCEDURE ... LANGUAGE AFLLANG

SQL SECURITY INVOKER READS SQL DATA AS BEGIN ..


AFM: Table Type Editor & Parameter Editing

Table Type Editor

Parameter Editing

Support user add/remove parameter

Provide restore defaults action


AFM: New Error Messages

Provide better error messages and provide the possible next step

Case Error Message (Client Side)

No required permission User does not have the AFL__SYS_AFL_AFLBFL_EXECUTE role. Assign the role first.

Table Type Configuration Error Table column setting does not match function specification

Calling with activation AFL Model (.aflpmml) file has not been activated. Activate the file before calling it.

… …

(Server side Activation)

No schema sleeted Repository: Error in repository runtime extension. No schema selected. Select a schema.

No Function Selected Repository: Error in repository runtime extension. No function in the model. Select a function.

Table Type has not been initialized Repository: Error in repository runtime extension. Add columns to the Dataset table.

File corrupt Repository: Error in repository runtime extension. Invalid file. Check the file or create the file again.

… …


Disclaimer

This presentation outlines our general product direction and should not be relied on in making

a purchase decision. This presentation is not subject to your license agreement or any other

agreement with SAP.

SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and

SAP’s strategy and possible future developments are subject to change and may be changed

by SAP at any time for any reason without notice.

This document is provided without a warranty of any kind, either express or implied, including

but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or

non-infringement. SAP assumes no responsibility for errors or omissions in this document,

except if such damages were caused by SAP intentionally or grossly negligent.

Thank you

Contact information

Ron Silberstein

SAP HANA Product Management

[email protected]

To get the best overview of what’s new in SAP HANA SPS 07, read this blog.

mailto:[email protected]

http://www.saphana.com/community/blogs/blog/2013/11/20/whats-new-in-sap-hana-sps07


© 2013 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG.

The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and

SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in

the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other

countries.

Please see http://www.sap.com/corporate-en/legal/copyright/index.epx#trademark for additional trademark information and notices.

http://www.sap.com/corporate-en/legal/copyright/index.epx




© 2013 SAP AG. Alle Rechte vorbehalten.

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche

Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.

Einige der von der SAP AG und ihren Distributoren vermarkteten Softwareprodukte enthalten proprietäre Softwarekomponenten anderer Softwareanbieter.

Produkte können länderspezifische Unterschiede aufweisen.

Die vorliegenden Unterlagen werden von der SAP AG und ihren Konzernunternehmen („SAP-Konzern“) bereitgestellt und dienen ausschließlich zu Informationszwecken.

Der SAP-Konzern übernimmt keinerlei Haftung oder Gewährleistung für Fehler oder Unvollständigkeiten in dieser Publikation. Der SAP-Konzern steht lediglich für Produkte

und Dienstleistungen nach der Maßgabe ein, die in der Vereinbarung über die jeweiligen Produkte und Dienstleistungen ausdrücklich geregelt ist. Keine der hierin

enthaltenen Informationen ist als zusätzliche Garantie zu interpretieren.

SAP und andere in diesem Dokument erwähnte Produkte und Dienstleistungen von SAP sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP

AG in Deutschland und verschiedenen anderen Ländern weltweit. Weitere Hinweise und Informationen zum Markenrecht finden Sie unter http://www.sap.com/corporate-

en/legal/copyright/index.epx#trademark.




Documents

What´s New? SAP HANA SPS 07 - blog.scherbinek.de´s-New... · What´s New? SAP HANA SPS 07 SAP HANA Application Function Library (AFL) (Delta from SPS 06 to SPS 07) SAP HANA Product