What's new on SAP HANA Workload Management

1© 2015 SAP SE or an SAP affiliate company. All rights reserved. 1© 2014 SAP AG or an SAP affiliate company. All rights reserved.

SAP HANA SPS 11 - What’s New? Workload Management

SAP HANA Product Management December, 2015(Delta from SPS 10 to SPS 11)

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 2Public

Agenda

Introduction What is Workload Management ?

Strategies to handle mixed workload Thread Pool Sizes Memory Consumption Limit Degree of Parallelism and Execution Priority

Dynamic Workload Mapping Workload Classes and Workload Mapping Monitoring

Customer Feedback

Summary

Introduction


Mixed Workload in the Context of SAP HANA

Workload is a set of requests with common characteristics, e.g. Source of request (user, application) Query types (ad-hoc, batch, analytical, transactional) Business importance (high, low, best effort) System Level Objectives (SLOs), e.g. response time, throughput

Workload has certain resource demands to the system, e.g. CPU Memory Disk I/O Network bandwidth

Mixed workloads compete for shared system resources If a request consumes a significant amount of system resources, other requests are subject

queuing effect Waiting workloads fail to meet their expected performance Overall system performance degradation may occur

DRAM DRAM DRAM

SQL SQL SQL

ETL Analytics Transactions


What is Workload Management ?

Objectives Runs the database in an optimal resource state Maximize overall system performance Balance resource demand between workloads Ensures that request meet their required SLOs Continuous maintenance and improvement of overall

workload efficacy, in an iterative manner

Understand your WorkloadBusiness Importance

Application TypeQuery Types

Resource DemandUsers

Conflicts between Workloads

Map your Workload to System Resources

CPU MemoryPriority

Degree of ParallelismWorkload Classes

Analyze System PerformanceCPU utilizationMemory consumptionExpensive Statements (Memory)Over-Parallelization

Workload- Management

Process

Strategies to handle mixed workloads


Thread Pools Sizes

Defintion:Thread Pools are managed inside of the HANA database per index server and provide the ability to expand or throttle how much of available CPU SAP HANA can consume

HANA Thread-Pools Job-worker thread pool Job Executor Request thread pool SQL Executor

Job Executor Handles complex, long-running statements (typically OLAP-like) Manages job queues (1-2 per CPU) Configure the thread pool size

– global.ini -> execution -> max_concurrency

SQL Executor Handles simple, short-running statements (typically OLTP-like) Configure the thread pool size

– indexserver.ini -> sql -> sql_executors– indexserver.ini -> sql -> max_sql_executors

SQL SQL

OLAP OLTP

SAP HANA(indexserver)

max_concurrency max_sql_executor

Key Point – max concurrency parameter: control overall number of threads that can be busy at any one time for complex operations (1 thread equates to 1 logical core)


Memory Consumption Limit

Global memory allocation limit per database (GAL) Max amount of memory that can be used by the database

– global.ini -> memorymanager -> global_allocation limit = <MB> Default: 90% of physical memory

Memory allocation limit per statement Max amount of memory that can be used by a single statement

– global.ini -> memorymanager -> statement_memory_limit = <GB> Default: 0 <unlimited>

Memory limit threshold per statement Controls if/when parameter statement_memory_limit is applied; Statement memory limit s applied only if total memory (for a particular statement) exceeds the global allocation limit by the value derived by the

statement_memory_limit_threshold (as a percentage of global_allocation_limit) Means of ensuring an inordinate amount of memory is not consumed by a single statement

– global.ini -> memorymanager -> statement_memory_limit_threshold = <%> Default: 0%, i.e. statement_memory_limit is always applied


Degree of Parallelism and Execution Priority

Parallel Execution SAP HANA , as much as it possibly can, executes with maximum parallelization in an automated manner -> highest system performance SAP HANA auto-determines degree of parallel statement execution (execution plan, job graph)

Degree of Parallelism Default: unlimited use of threads for parallel statement execution (best effort) Can be controlled by statement thread limit per statement in workload classes Default parallel degree can be controlled with parameter default_statement_concurrency_limit

– e.g. settting value to 8 means max 8 threads can be engaged at any one time for any one statement

Execution Priority is applied to jobs of the Job Executor all generated jobs for a statement get an “external priority” execution priority for all statements per user can be set between 0 … 9; default is 5 Syntax:

– alter user SYSTEM set parameter PRIORITY = '9';– select priority from m_connections where connection_id = CURRENT_CONNECTION;

New for SPS11

Dynamic Workload Mapping


What is Dynamic Workload Mapping ?

Motivation System-wide resource settings are static and does not fit to all workloads Manual admin tasks to set parameter for every workload and user are error prone and complex

Objectives Dynamic mapping of system resources to DB-clients with different workload profiles Flexible resource mapping on session and statement granularity Focus on avoiding severe resource abuse by single statements vs. resource pre-allocation No changes in the application code of the DB-clients


Workload Mapping

Desktop Datamart

Administrator Tasks: • create workload classes• define mapping rules

Mobile Dashboard

db-connect

SAP HANA Session Layer

Workload Class

“DATAMART”

db-connect

sessioncontext

Workload Class

“BW”

Workload Class

“CRM”

Workload Class

“MOBILE”

Mapping Rules

sessioncontext


Defining Workload Classes

Workload Classes represent individual and named sets of resource properties

statement thread limit = max #of parallel threads to execute a statement statement memory limit = max #of memory allocated per statement statement priority = execution priority for a statement

SQL Syntax: <CREATE|ALTER|DROP> WORKLOAD CLASS <NAME> [SET|UNSET <PROPERTY LIST>]Example: create workload class “DATAMART" set 'PRIORITY' = '6', 'STATEMENT MEMORY LIMIT' = '2','STATEMENT THREAD LIMIT' = '8‘;

Note You can define all resource properties or only a subset in a workload class Workload Class definitions can be checked in the HANA system view WORKLOAD_CLASSES

– select * from workload_classes;


Defining Mapping Rules

The HANA session layer reads the client context information

application name = name of the application application user name = name of the application user (usually the user logged into the application) user name = name of the db-user (where the application is connected with HANA technically) client = ABAP client number (“mandant”) or can be used for setting

SQL Syntax: <CREATE|ALTER|DROP> WORKLOAD MAPPING <NAME> WORKLOAD CLASS <WC NAME> [SET <PROPERTY LIST>]Example: create workload mapping “BICLIENT" workload class “DATAMART" set 'USER NAME' = ‘DATA001‘;

Note You can define all client context information as mapping attributes or only a subset Workload Mapping definitions can be checked in the HANA system view WORKLOAD_MAPPINGS

– select * from workload_mappings;


Monitoring Views for Workload Mapping

SYS.M_CONNECTIONS_, SYS.M_PREPARED_STATEMENTS_; SYS.M_ACTIVE_STATEMENTS_ global.ini-> resource_tracking -> enable_tracking = on / memory_tracking = on


Additional Remarks about Workload Classes - 1 -

Privileges Managing workload classes requires the ‘WORKLOAD ADMIN’ privilege

Changing Workload Classes Changes of workload classes or mappings will only be applied, if a (connected) db-client re-connects You can change the running session-context of a connected db-client via SQL-command ‘ALTER SYSTEM ALTER SESSION SET‘ (not recommended)

Multi-Database Containers (MDC) Workload classes work also in a multi-database environment, but you need to define workload classes for each

single db-container

Scale-out Workload classes are applied to the complete HANA-db in a scale-out environment, but not to each single node


Additional Remarks about Workload Classes – 2-

Dropping users / workload classes If you drop a user in the HANA-db, all related workload classes are dropped If you drop a workload class, the related mappings are dropped

Setting the client context in the application The client info object contains the client context information as key/value pairs The ABAP Server sets the client context information automatically for all ABAP applications The client interface of the corresponding driver (jdbc/odbc/python) provides methods to set client info object

properties (see SAP note 2215929)

New for SPS11: Setting Session-Specific Client Information is now documented in the SAP HANA Developer Guide (see section 12.1)

New for SPS11

Customer Feedback from POC


Customer POC Scenario: How SAP HANA WLM helps web requests in peak load situations

Description

This scenario is done to capture how WLM helps web requests in peak load situations Captures the impact of other workloads on the web request with workload class and with out workload class Started 50 web request/second and kept it constant and slowly increase analytical and ETL load by adding 5 every 5 minutes. Stressed the system CPU utilization to Max and CPU was showing 100% most of the time

Workload Class Specification

Web User ( no limitation on resource) ETL User ( Threads limited to 1 and memory limit to 5 GB per sql) ANALYTIC User ( Threads limited to 1 per SQL and memory limit 5GB per sql)


Results

Outcome There is huge difference in hits per seconds

of the web request with workload class and with out workload class

Response times are stable even after increasing load from other low priority workloads.

Hits /Second Without WLM With WLMMIN 0 6.922AVG 71.641 152.010MAX 94.406 157.297

Summary


Summary

SAP HANA is a database platform to run various applications with different workload profiles on the same data

SAP HANA provides various strategies to manage and balance resources between applications

Thread Pool Sizes Memory Allocation Limits Degree of Parallel Execution Execution Priorities

SAP HANA provides a dynamic workload mapping to manage resources on statement and session level

SAP HANA Workload Management is an ongoing development effort

For more details on SAP HANA Workload Management see: SAP TechEd 2015 Video – Workload Management

http://events.sap.com/teched/en/session/22704


Disclaimer

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP.

SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP’s strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice.

This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

© 2015 SAP SE or an SAP affiliate company. All rights reserved.© 2014 SAP AG or an SAP affiliate company. All rights reserved.

Thank you

Contact information

Ruediger Karl / Ron SilbersteinSAP HANA Product [email protected]

mailto:[email protected]

Data & Analytics

What's new on SAP HANA Workload Management