Empower Db2 for z/OS with Machine Learning to Boost Performance · 2019. 5. 15. · IBM z Analytics Db2 for z/OS 12 Db2ZAI V1.2 System Requirements 12 — Db2 12 for z/OS •Required

IBM Db2 AI for z/OS

Empower Db2 for z/OS with Machine Learning to Boost Performance

Tom Beavin DB2 for z/OS [email protected]

Michigan Db2 Users GroupMay 15, 2019 Detroit, MI

IBM z Analytics

Db2ZAI Session Objectives

—Introduction

—Architecture

—Setup and Configuration

—SQL Optimization Features

—System Assessment Feature

—Future Direction and Feedback

Db2 for z/OS

© Copyright IBM Corporation 2019

IBM z Analytics

Introduction

Db2 for z/OS


IBM z Analytics

Factors driving ML adoption across industries

1. Velocity of change is increasing and buying power is shifting from IT to the LOB

• IT is held accountable for SLAs at a business application level

• Risk of outages increase, as rate of change accelerates and operational complexity increases

• New workloads volumes, such as mobile, can be unpredictable

2. Growth of hybrid cloud applications is increasing complexity – new layers of IT Systems built

out on top of previous layers

• The mapping between business applications and mainframe resources are unclear

• Silos among people and environment inhibits SLAs attainment

3. IT Operations environment is increasingly strained

• Organizations face skills issues as they go through a generational shift

• Operations teams are asked to reduce cost while maintaining operational integrity


IBM z Analytics Db2 for z/OS

Many Organizations are turning to ML technology

Machine learning…• Constantly learns and adapts

• Avoids making the same mistakes

• Provides faster, deeper, improved insights

Resulting in…✓ More-favorable business outcomes

✓ Reduced risks and costs

✓ New opportunities

Identify suspicious behavior, predict

and prevent threats / fraud – continually

reduce business risks and costs

Product recommendation, next purchase

prediction, targeted offers – individually

tailored shopping experience

Learn, predict weather patterns and

energy production from renewable sources

and integrate into grid more effectively

Detect and understand life-threatening

medical conditions and design ever

more effective treatment programs

Churn analysis helps identify cause of

churn and facilitates implementation

of effective retention strategies



Similar benefits driving Db2 to ML technology

Machine learning…• Constantly learns and adapts

• Avoids making the same mistakes

• Provides faster, deeper, improved insights

Resulting in…✓ More-favorable business outcomes

✓ Reduced risks and costs

✓ New opportunities

Data obtained from prior query

execution can help the Db2 Optimizer

choose better performing access paths

Analysis for system performance data,

coupled with expert domain knowledge,

can be used to detect anomalies and

recommend corrective actions

Analysis of query performance data can

be used to detect and self-heal from

performance regressions

Many more possibilities! Self-tuning

access paths, self-tuning buffer pools,

etc.


IBM z Analytics

New offering: IBM Db2 AI for z/OS (Db2ZAI)

• Learns patterns from workload data collected in an organization’s unique operating environment

and uses derived insight to help determine optimal access paths for SQL statements

• Detects anomalies from Db2 Statistics and provides performance tuning recommendations

• Built on top of the IBM Watson Machine Learning for z/OS (WML on Z) Technology stack

• Leverages WML on Z services without requiring data scientist support

• Db2 generates model training data

• Deploys, monitors and retrains models via WML on Z services

• Internal IBM benchmarks have shown up to 25% reduction in CPU time

Db2ZAI product ID: 5698-CGN


IBM z Analytics

Architecture

Db2 for z/OS


IBM z Analytics

Db2ZAI V1.2 Architecture

—All z/OS architecture

• No Linux or x86 requirement (new in V1.2)

—Learning is driven in the background (within Db2)

• Asynchronous SRB tasks (zIIP eligible)

—WML on Z provides ML services

• Model training/retraining

• Scoring

—Db2ZAI provides a user interface (UI)

• Frontend to communicate with end user

• Backend to co-ordinate processing on Db2 side and WML on Z side


IBM z Analytics

Db2ZAI V1.2 Architecture Diagram

10

PerformanceHistory

Db2 for z/OS V12

Db2

Optimizer

“Learning”

Task (zIIP

Offloaded*)

Db2 IFC Statistics

trace &

zparm

Db2ZAI UI

WML on Z model management UI

Db2ZAI

Performance

Assessment

Performance Tuning Recommendations

z/OS z/OSX86 or

zLinux

WML on Z Training &

Scoring Services

(includes Open Data

Analytics - runtime)

WML


IBM z Analytics

Db2ZAI V1.2 UI Architecture



12

Db2ZAI V1.2 System Requirements

12

— Db2 12 for z/OS

•Required Function Level FL 500

•Plus required APARs - PH07782, PH09051, and

PH09052

•UI is delivered under 5698-CGN (FMID HCOY120)

and APAR PH08098 and PH09689

— IBM Watson Machine Learning for z/OS V2.1

•No longer requires zLinux or X86

•WML V2.1 and APAR PH08693 (this includes

Db2StatsProfiler model for Db2ZAI V1.2)


IBM z Analytics

Setup and Configuration

Db2 for z/OS


IBM z Analytics

Db2ZAI Configuration

Db2 for z/OS


IBM z Analytics

Db2ZAI Configuration

Db2 for z/OS


IBM z Analytics

Db2ZAI Add Connection

Db2 for z/OS


IBM z Analytics


Db2 for z/OS


IBM z Analytics


Db2 for z/OS


IBM z Analytics


Db2 for z/OS


IBM z Analytics


Db2 for z/OS


IBM z Analytics

Db2ZAI Select Packages

Db2 for z/OS


IBM z Analytics

Db2ZAI Db2 Dashboard

Db2 for z/OS


IBM z Analytics

Db2ZAI Db2 Dashboard

Db2 for z/OS


IBM z Analytics

IBM Knowledge Center Documentation: https://www.ibm.com/support/knowledgecenter/SSGKMA_1.2.0/src/ai/ai_home.html

IBM z Analytics

SQL Optimization Features

Db2 for z/OS


IBM z Analytics

What is the Db2 Optimizer? An analogy….

• The optimizer is responsible for

• Choosing the most efficient method of accessing the data for a given SQL statement

• Think of your transportation choices• Start/end location, time of day, construction, traffic, available options/routes

• All can impact the “quickest” route

Db2 for z/OS

?

?

?

?© Copyright IBM Corporation 2019

IBM z Analytics

An Optimization Challenge

—Determining the degree of filtering for each predicate (i.e. the filter factor) is a key

input to the optimizer’s access path cost formulas

—Question: how many of you were born before parameter marker?

• It depends…

o What if the substitution value is ‘1900-01-01’?

o What if the substitution value is ‘2018-12-31’?

—When we have a range predicate with a parameter marker or host variable…

• …we use a default filter factor (these are documented)

Db2 for z/OS

BIRTHDATE < ?


IBM z Analytics

Tuning just ONE query: the user challenge

— Access path analysis can be a very complex, time-consuming task

• Depends on several factors (user skill level, available tooling, etc.)

— Solutions that we like to see:

• Determine RUNSTATS options for targeted database objectso Source: SYSIBM.SYSSTATFEEDBACK (system-level) or DSN_STAT_FEEDBACK (query-level)

• Indexing – add or modify

• Add OPTIMIZE FOR n ROWS to SQL (“paging-style”– subset of result set fetched)

— Solutions that are less preferred:

• BIND/REBIND with APREUSE or OPTHINTS

• Query rewrite

• “Fudging” catalog statistics

• Query tricks (0=1 for example)

Db2 for z/OS

Skill often missing

Organizations often revert to these

IBM z Analytics

Large-scale REBIND can present another challenge for users

— Suppose you REBIND 10,000 static packages using APCOMPARE(WARN)

• After migration to new Db2 version, or after applying maintenance

— If 90% of packages show no access path changes, and 10% (1000 packages) do - how do you

validate that changed access paths are OK?

• Review changes in REMARKS column of PLAN_TABLE if REBIND executed with EXPLAIN(YES)

o Do access paths look better (e.g., increase in MATCHCOLS)? Any red flags?

• If access path change looks OK, go with new instance of package

• What if access path change does NOT look OK?

o REBIND with SWITCH to revert to previous instance of package?

o Use OPTHINT to force use of an alternate access path?

o Try to tune affected queries?

— What if access path change validation work has to be done for several hundred packages?

Db2 for z/OS

Approaches most commonly taken


IBM z Analytics

Predict program variable values

— SQL uses host variables or parameter markers in a predicate• Db2ZAI predicts likely literal values based on historical data

• Can lead to much better filtering estimates

— Example SQL:SELECT ….

FROM CUSTOMER C, ORDER O, LINEITEM L, NATION N

WHERE C_CUSTKEY = O_CUSTKEY

AND L_ORDERKEY = O_ORDERKEY

AND C_NATIONKEY = N_NATIONKEY

AND O_ORDERDATE >= ?

AND O_ORDERDATE < ? -- Db2ZAI uses historical data to predict ?’s

AND L_RETURNFLAG = ? for access path selection purposes

GROUP BY C_CUSTKEY,

C_NAME, C_ACCTBAL, C_PHONE, N_NAME, C_ADDRESS, C_COMMENT

ORDER BY RENENUE DESC;

— Depending on the values predicted, the access path can change• New table join sequence, join methods, etc. may be chosen by optimizer


IBM z Analytics

Program variable prediction - Details

— Program variable values collected during query execution

• First 1000 executions collected (max per query)

• Qualified query / qualified program variables only

— Create models from collected program variable values

• 1 model per program variable

• Algorithm = random forest

• Prediction = program variable value

• Minimum amount of training data required = 100 executions of the query

— Model usage

• For each program variable of interest, optimizer invokes scoring routine associated with program

variable

• Predicted program variable value used to determine filter factor for predicates involving program

variable

— Current Limitations

• Host var expressions (example: BIRTHDATE < ? – 1 YEAR)


IBM z Analytics

Partial result set fetch

— SQL might qualify hundreds or thousands of rows

— Application only fetches a subset of the qualifying rows (one screen)

— Db2ZAI learns application behavior and adjusts access path selection appropriately

— Example SQL:SELECT L_SUPPKEY FROM LINEITEM -- index on L_SUPPKEY (low CR)

WHERE L_DISCOUNT IN (0.01, 0.02)

AND L_TAX IN (2.99, 3.99, 5.99, 4.99)

AND L_QUANTITY <= 5 -- non indexed columns

ORDER BY L_SUPPKEY; --Application only fetches first 25 rows

— With Db2ZAI, access path may change from table space scan + sort -> index access w/o sort

— In tests, 95% reduction in CPU and elapsed time. 90% reduction in get pages (due to sort

avoidance)


IBM z Analytics

Partial result set fetch - Details— Actual number of rows fetched collected during query execution

• First 1000 executions only (max per query)• Query must qualify for machine learning (be in “scope”)

• Query must not already have OPTIMIZE FOR n ROWS clause or FETCH FIRST n ROWS clause

— Create models from data collected• 1 model per query• Algorithm = linear regression• Prediction = number of rows fetched by application

• Minimum amount of training data required = 100 executions of the query

— Model Usage• For each query, optimizer invokes the scoring routine for that query to obtain the predicted number of rows

that will be fetched• Optimizer generates an OPTIMIZE FOR n ROWS clause using the predicted number for ‘n’

— Limitations• Only for the top “query block”. Does not apply to subqueries, etc. • Same limitation as OPTIMIZE FOR n ROWS clause


IBM z Analytics

Query Parallelism— SQL using DEGREE=ANY and currently not eligible for //’ism due to optimizer underestimating actual

elapsed

— Db2ZAI learns actual run time behavior and allows “modest” parallelism where appropriate

— Example SQL:SELECT L_SHIPMODE. -- Query bound with DEG=ANY or using SET CURRENT DEGREE=ANY

,SUM(CASE WHEN O_ORDERPRIORITY = '1-URGENT’ OR O_ORDERPRIORITY = '2-HIGH’

THEN 1 ELSE 0 END) AS HIGH_LINE_COUNT,

SUM(CASE WHEN O_ORDERPRIORITY <> '1-URGENT'

AND O_ORDERPRIORITY <> '2-HIGH'

THEN 1 ELSE 0 END) AS LOW_LINE_COUNT

FROM ORDERS, LINEITEM

WHERE O_ORDERKEY = L_ORDERKEY

AND L_SHIPMODE IN (? ,? )

AND L_COMMITDATE < L_RECEIPTDATE

AND L_SHIPDATE < L_COMMITDATE

AND L_RECEIPTDATE >= ?

AND L_RECEIPTDATE < ?

GROUP BY L_SHIPMODE

ORDER BY L_SHIPMODE; -- Blocked from //’ism because optimizer’s cost estimate is too low

— With Db2ZAI, the access path may change from no parallelism to query parallelism with DEG=2 or 4

— In tests, elapsed time reduced 85% and chargeable CPU (GP) reduced 95%. Total CPU (GP + SE) increased 5%


IBM z Analytics

Query Parallelism - Details

— Collect actual cpu time during query execution• First 1000 executions collected (max per query)• Query must qualify for machine learning (be in “scope”)• Query must qualify for parallelism (i.e. bound with DEGREE=ANY)

— Create models from collected cpu time data• 1 model per query• Algorithm = linear regression• Prediction = cpu time• Minimum training data required = 100 executions of the query

— Model Usage• For each query, optimizer invokes the scoring routine for that query to get the predicted cpu time• Low degree parallelism allowed if predicted cpu time > parallelism threshold

— Limitations• Not allowed for dynamic• Only allowed for static with DEGREE=ANY


IBM z Analytics

SQL Sort

— SQL using sort

— Db2ZAI learns SQL execution behavior and makes adjustments to sort algorithm and allocations in order to optimize sort CPU and memory use

— Does not require models• 1 execution is all that’s needed to feedback to the next execution

— SQL that will benefit most

• SQL with DISTINCT

• ORDER BY with larger FETCH FIRST n ROWS values

• Longer sort rows (> 4K row length)

— Limitations• If cache is cleared then feedback is also cleared

• Benefit is lost on the initial execution of the sort after cache is cleared

— In tests, up to 50% reduction in CPU for sweet spot queries


IBM z Analytics

Db2ZAI SQL Optimization

Db2 for z/OS


IBM z Analytics

System Assessment Feature

Db2 for z/OS



Motivation and problem statementAssessment steps Db2zAI V1.2 Externals

Db2 AI for z/OS V1.2 System Assessment

Embed machine learning into Db2 12 for enhanced performance



40Think 2019 / 6469 / February 13, 2019 / © 2019 IBM Corporation

>1500 Performance Counters

Db2 for z/OS produces

Motivation


IBM z Analytics

Too

Many

Numbers

Db2 for z/OS

41

Motivation

41



42

Approach

42

Expert knowledge Deep domain knowledge requires years of experience embedded into assessment

Faster turnaround Generate next actions to take when the key metrics exceed thresholds

Smarter insightsLearns from data to understand your specific workload characteristics



43

Inputs and Outputs

Assessment Input Statistics data captured directly in Db2 tables

Customized metrics based on Db2 statistics counters

Pre-defined exception rules

• Exception thresholds are based on domain expert plus custom profiler output

Assessment Result List of Exceptions

Recommendations

(Correlation) Graph

Workload profile using the key metrics



Exception Categories

SQL related EDM and

Statement cache

System levelLog read and

write Authorization

Locking RID

processing Work file

CPU Parallelism

Address Space CPU

time

DRDA activities

Memory usage

Db2 internal latches

Local Buffer Pools

Group Buffer Pools



45

Examples of Exceptions

SQL and application related

Too many rows deleted per delete operations

Excessive Getpages per commit

Excessive parallel execution

EDM and statement cache

Approaching to the defined EDM pool values

Global or local cache hit ratio is lower than normal

Log writes

High number of Async CF lock sync up time during log write process

High numbers of log buffer shortage



46

Analysis and Recommended Actions - Example 1

Exception

Excessive Parallel Group Execution

If PARAMDEG = 0 (no override), then recommend to use PARAMDEG with number of zIIPs. Otherwise, recommend to reduce the concurrency.

Recommendation

What is current system parameter value on controlling parallel degree (PARAMDEG) ?

ZPARM

value



47

Analysis and Recommended Actions – Example 2

Exception

Db2 Log Buffer Shortage

If workload increase, and if log buffers are not max, increase log buffers. Otherwise, investigate ...

Recommendation

This may be explained by the log record increase (= update workload increase ) ?

Create

Correlation

graph

What is current system parameter value on controlling log buffers (OUTBUFF) ?

ZPARM

value



48

System Assessment Steps

Set up work period, training data range, and assessment

schedule

Start ML from UI - Statistics

collection starts

Triggers Training

Training completes by WML custom

profiler

Continue to collect data

Triggers Performance Assessment

(based on recurring Schedule)

Review Assessment

Results

Corrective Action Taken

or new applications

deployed.



49

Db2ZAI System Assessment Externals

— Externals are all through UI (same as Db2ZAI SQL optimization)

— Once enabled via UI, Db2ZAI will collect Db2 statistics information for every minute, and Db2 system parameters

— Once enough data is accumulated, training can be triggered through UI — Recommend to train with at least 1 week of data — Optionally, user can define Work Periods

— Once enough data is accumulated, periodical System Assessments can be scheduled

— Assessment results are published through UI

Checkout Knowledge Center Documentation: https://www.ibm.com/support/knowledgecenter/SSGKMA_1.2.0/src/ai/ai_home.html



50


Db2ZAI Dashboard


51

Db2ZAI System Assessment Dashboard

51



52

Db2ZAI System Assessment Exception & Recommendations



53

Db2ZAI System Assessment Exception & Recommendations



54

Db2ZAI System Assessment Graphs


IBM z Analytics

Future Direction

Db2 for z/OS


IBM z Analytics

Future Direction – Where we are now

—Db2ZAI V1.2 • SQL Optimization for improved access path selection and better query performance

• System Assessment for exception detection and recommendations

—Just a first step toward automation and deeper analysis• SQL Optimization requires user

o Select packages for ML enablement

o Perform recommended REBINDs (static SQL)

o Clear dynamic statement cache when needed (dynamic SQL)

• System Assessment requires user

o Review the list of exceptions

o Take the recommended actions (if any)

o Continue analysis where Db2ZAI left off (if no recommendation, or recommendation does not

resolve issue)

Db2 for z/OS


IBM z Analytics

Future Direction – Where we are going—SQL Optimization with self-managing access paths

• Automatically recover back to better performing access path if a new access path

causes regression (self-healing capability)

• Automatically introduce a new access path when Db2ZAI detects performance can

be improved by phasing in a new plan (self-tuning capability)

• Automatically classify queries running on the system for optimized processing. Goal

is to match the optimization effort spent on a query to the amount that is needed by

the query. In other words, avoid unnecessary overhead but don’t miss out on

opportunities to improve.o Simple OLTP (avoid overhead for RLF, Db2ZAI, etc.)

o Complex OLTP

o OLAP / HTAP

Db2 for z/OS


IBM z Analytics

Future Direction – Where we are going (continued)—Add analysis of application data (i.e. Db2 accounting data)

• Application Assessment to complement and build upon System Assessment

—Potential Use Cases – What would be most important use case for you?

• “Health Check”: I want “health check” similar to System Assessment for my selected

applications (what would be the criteria to select applications?)

• “Outlier Detection”: I have a base line and would like to compare the latest

performance and detect the outliers against the base

• “Transaction Monitoring”: I have a set of critical transactions to monitor to keep them

healthy and report any possible performance issues/tuning opportunity

• “Drill down”: When System Assessment finds Db2 subsystem level exceptions, I want

to track down to applications or statements causing the problems

Db2 for z/OS


Thank you!

Tom Beavin

Db2 for z/OS Development

[email protected]


Notices and disclaimers

— © 2019 International Business Machines Corporation. No part of this document may be reproduced or transmitted in any form without written permission from IBM.

— U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

— Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event, shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted per the terms and conditions of the agreements under which they are provided.

— IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”

— Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

— Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those

— customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

— References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

— Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

— It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer follows any law.


Notices and disclaimers continued

— Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products about this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a purpose.

— The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

— IBM, the IBM logo, ibm.com and [names of other referenced IBM products and services used in the presentation] are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml


http://www.ibm.com/legal/copytrade.shtml

Documents

Empower Db2 for z/OS with Machine Learning to Boost Performance · 2019. 5. 15. · IBM z Analytics Db2 for z/OS 12 Db2ZAI V1.2 System Requirements 12 — Db2 12 for z/OS •Required