IBM DB2 Analytics Accelerator in Production at NLB

IBM DB2 Analytics Accelerator in Production at NLB (Nova ljubljanska banka)

Mitja Makovec NLB d.d. Namik Hrle IBM

Session Code: E5 Tuesday, 15 October 2013| Platform: DB2 for z/OS

Click to edit Master title style

Nova Ljubljanska Banka d.d.

2

Over 1.5 million customers

43 % market share in Slovenia

6,735 employees

46 companies in 14 countries

NLB Group: 47 companies in 13 countries

Total Assets: ~EUR 13,000 million

3,713 employees

26.6 % market share (TA) in Slovenia


NLB d.d. Group

Banks: 11

Leasing: 7

Trade finance: 9

Insurance: 4

Asset management: 1

Other activities: 15

3


POC (proof of Concept) in 2010

Platforms tested:

• Netezza Twinfin (CRMT) as a stand alone database (at that time Netezza was still an independent company)

• Oracle Exadata

• IBM Smart Analytic Optimizer

All platforms showed extraordinary increase in performance

What was the key IBM Smart Analytics Optimizer V1 benefit?

• Unlike other options it required no or minimal changes to the application

What were its disadvantages?

• V1 supported only full table refresh

• It can be connected to all the subsystems of the database, but the capacity is shared between environments

4


POC in figures

In all three cases extraordinary increase in speed

IBM DB2 Analytics Accelerator for z/OS added for comparison

5

Query No DB2 on z9 DB2 on z196

Stand-alone Netezza TwinFin

DB2 with ISAO V1

Oracle Exadata

DB2 with IBM DB2 Analytics

Accelerator V2

Query 1 00:06:29 00:01:50 00:00:04 0:00:02 00:00:09 00:00:03

Query 2 04:01:30 00:75:31 00:00:09 0:00:07 00:00:39 00:00:04

Query 3 00:21:14 00:00:46 00:00:05 N/A 00:00:13 00:00:02


NLB environment

• System z196, 5CP, 2zIIP, 128 GB RAM

• Storage Hitachi USP and VSP

• z/OS V1.12.

• DB2 10 for z/OS

• IBM DB2 Analytics Accelerator for z/OS Version 2

• IBM Netezza 1000-3:

• (3 S-blades, 24cpu, 8Tb uncompressed)

• 4 LPARs: • Production, UAT, Test, Development

• 8 DB2 subsystems – 2 per LPAR

6


NLB BDW architecture view

7


Tools

• IBM banking data warehouse model

• NDM-MQ Series – transport of data

• DB2 Utilities suite – load into data warehouse

• Microsoft BI tools: • Integration: SSIS (Integration Services)

• Analytics: SSAS (OLAP)

• Reporting: SSRS (Reporting Services)

• Viewing: SharePoint (integration OLAP and RS)

• Supervisor of batches

• Web applications, reconciliation

8


BDW

9

9


What is IBM DB2 Analytics Accelerator?

10

Blending System z and Netezza

technologies to deliver unparalleled,

mixed workload performance for

complex analytic business needs.

Accelerating decisions to the speed of business

Leverages the strengths of System z

and Netezza

Extreme Performance for Complex

Analytics

Breakthrough technologies - Hardware

acceleration

Tightly integrated into DB2 for z/OS

Creates a highly secure environment for

sensitive data analysis

Performance, availability and scalability

10


Design Principles

• DB2 continues to own data (both OLTP and DW) • Access to data (authorization, privileges, …)

• Data consistency and integrity (backup, recovery, …)

• Enables extending System z QoS characteristics to BI/DW data as well

• Applications access data (both OLTP and DW) only through DB2 • DB2 controls whether to execute query in DB2 mainline or route to DB2 Analytics

Accelerator for z/OS

• DB2 returns results directly to the calling application

• Existing applications do not have to change.

• Enables mixed workloads and selection of optimal access path (within DB2 or Analytics Accelerator) depending on access pattern

• DB2 Analytics Accelerator for z/OS as a virtual DB2 component • DB2 provides key Analytics Accelerator status and performance indicators as well as

typical administration tasks by standard DB2 interfaces and means

• No direct access (log-on) to IBM DB2 Analytics Accelerator for z/OS

• Enables operational cost reduction through skills, tools and processes consolidation

11


Deep Integration with DB2 for z/OS

12


13

DB2 for z/OS

Optimizer

IDA

A D

RD

A R

eq

ue

sto

r

DB2 Analytics Accelerator

Application

Application

Interface

Queries executed with DB2 Analytics Accelerator

Queries executed without DB2 Analytics Accelerator

Heartbeat (DB2 Analytics Accelerator availability and performance indicators)

Query execution run-time for

queries that cannot be or should

not be off-loaded to DB2 Analytics

Accelerator for z/OS

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SM

P H

ost


IBM DB2 Analytics Accelerator for z/OS TCP/IP configuration

• DB2 Analytics Accelerator for z/OS connected through zBX TOR to System z196

• Topology chosen for redundancy

14

z196 zBX IDAA

BPH

OSX

OSX

INMN

INMN

IEDN

IEDN

HOST 1

HOST 2


Enabling DB2 subsystem for IBM DB2 Analytics Accelerator for z/OS

• SMP/E to install required PTFs

• IBM DB2 Analytics Accelerator Studio or IBM Data Studio with Analytics Accelerator plugin

• WLM and IBM DB2 Analytics Accelerator for z/OS Stored procedures

• DB2 zPARM • ACCEL=AUTO

• QUERY_ACCELERATION=NONE

• ACCEL_LEVEL=V2 – ONLY FOR V9

• SET CURRENT QUERY ACCELERATION special register • ENABLE

• ENABLE WITH FAILBACK

15

• ELIGIBLE • ALL

APAR: PM72265 - v10 PM72264 – v9


• IBM DB2 Analytics Accelerator for z/OS is used for: • Rapid development and data analysis

• ETL process for populating DWH

• On-line applications

• Reporting

• OLAP cubes (MOLAP -> ROLAP)

• Prerequisites • Table must be enabled and populated in IDAA

• Only dynamic SQL

• Redirect enabled (SET command or zPARM)

Using IBM DB2 Analytics Accelerator for z/OS

16


Populating IBM DB2 Analytics Accelerator for z/OS

Tables need to be defined in DB2 Analytics Accelerator for z/OS

Options for populating IBM DB2 Analytics Accelerator for z/OS V2 • IBM DB2 Analytics Accelerator Studio / Data Studio

• Stored procedure sysproc.ACCEL_LOAD_TABLES

• JCL and C program (AQTSJI03)

• JCL and USS (CLP call, no error handling)

17


Populating Analytics Accelerator – Our way!

• Own procedure that calls sysproc.ACCEL_LOAD_TABLES for loading tables: • Generates XML statement for table/partition

• Log of execution in db2 table – admin web report – Partition level!

18

Call DW.BI_IDAA_LOAD(‘tblcreator’, ’tblname’, ‘partition’)


Accelerating ETL

• Cross loader

• Stored procedures

• Insert – select

19


Accelerating ETL

• Cross loader (CLP limit 10k! – ODBC/JDBC 32k)

CALL SYSPROC.DSNUTILU (

'IDAALOAD' ,

'NO' ,

'EXEC SQL

SET CURRENT QUERY ACCELERATION = ENABLE;

ENDEXEC

EXEC SQL

DECLARE C1 CURSOR FOR

select CLAN as MSR_PRD_ID, count(*) as obr_visina

from BII.CX_STANJE_ARH

where name = ‘‘ABCD’’

group by clan;

ENDEXEC

LOAD DATA INCURSOR C1 REPLACE INTO TABLE TEST_IDAA’, ?);

20

SYSPROC.DSNUTILU EBCDIC/UNICODE conversion problem where name = x’’C1C2C3C4’’


Accelerating ETL

• SQL Stored procedures • Only dynamic sql can be accelerated

DECLARE cursor1 CURSOR WITH HOLD FOR V_DYNAMIC; -- declare cursor for dynamic sql

SET STMT=‘SELECT * … WITH UR’ -- define sql statement

SET CURRENT QUERY ACCELERATION = ENABLE; -- enable accelerator

PREPARE V_DYNAMIC FROM STMT; -- prepare cursor

FETCH INSERT LOOP -- fetch cursor, insert into table

21


Accelerating ETL

• Insert from select • Functionality introduced by APAR PM60921

• DSN6SPRM.QUERY_ACCEL_OPTIONS – zPARM

• OPTION 2:

• For INSERT statements where a fullselect specifies the data to be inserted, the fullselect part is not blocked from executing on IDAA although the data operated on by the fullselect may not be current in IDAA.

22


Accelerating queries

• No need to optimize queries • Execution time: - db2 ~1min, Analytics Accelerator 1,5 seconds

23


Accelerating queries - tuning

• Visual explain – Data Studio/IDAA Studio

• Explain: Set current query acceleration = enable;

Explain all for select count(*) from ecb1.stanje_ecb_arh;

• DSN_QUERYINFO_TABLE

• DB2 analytics accelerator tuning options • Distribution keys

• Organizing keys

24



• Execution in db2 blocked by resource limit facility: DSNT408I SQLCODE = -495, ERROR: ESTIMATED PROCESSOR COST OF 000000004345 PROCESSOR SECONDS

(000120703888 SERVICE UNITS) IN COST CATEGORY A EXCEEDS A RESOURCE LIMIT ERROR THRESHOLD OF 00090000000 SERVICE UNITS

• Tables added in accelerator - no tuning on accelerator <execution state ="DONE" submitTimestamp="2013-07-09T19:42:40.817269“

elapsedTimeSec="563" executionTimeSec="563" resultRows="4155309"

resultBytes="3250724471"/> <sql><![CDATA[INSERT INTO

ECB1.STANJE_ECB_TC_ARH ( MSR_PRD_ID, STANJE_ID, POSEL_NID, PD_ID,

MIS_KONTO, KRATEK_DOLG, IZLOCEN, POR_POST_NIVO]]></sql>

• Tables added in accelerator -distribution keys defined

<execution state ="DONE" submitTimestamp="2013-07-09T20:57:19.094606“

elapsedTimeSec="274" executionTimeSec="274" resultRows=“4155309“

resultBytes="3250724471"/> <sql><![CDATA[INSERT INTO ECB1.STANJE_ECB_TC_ARH

( MSR_PRD_ID, STANJE_ID, POSEL_NID, PD_ID, MIS_KONTO, KRATEK_DOLG, IZLOCEN,

POR_POST_NIVO]]></sql>

25

0 500

1000 1500 2000 2500 3000 3500 4000 4500 5000

DB2 Accelerator - random

distribution

Accelerator - distributon keys

seco

nd

s



26


IBM db2 analytics accelerator - monitoring

• Db2 commands

• dis accel(*) detail DSNX810I )DSNW DSNX8CMD DISPLAY ACCEL FOLLOWS -,

DSNX830I )DSNW DSNX8CDA,

ACCELERATOR MEMB STATUS REQUESTS ACTV QUED MAXQ,

-------------------------------- ---- -------- -------- ---- ---- ----,

ACCEL2W DSNW STARTED 11678 1 0 0,

LOCATION=ACCEL2W HEALTHY,

DETAIL STATISTICS,

LEVEL = AQT02012,

STATUS = ONLINE,

FAILED QUERY REQUESTS = 243,

AVERAGE QUEUE WAIT = 37 MS,

MAXIMUM QUEUE WAIT = 962 MS,

TOTAL NUMBER OF PROCESSORS = 24,

AVERAGE CPU UTILIZATION ON COORDINATOR NODES = 1.00%,

AVERAGE CPU UTILIZATION ON WORKER NODES = 12.00%,

NUMBER OF ACTIVE WORKER NODES = 3,

TOTAL DISK STORAGE AVAILABLE = 8024544 MB,

TOTAL DISK STORAGE IN USE = 5.25%,

DISK STORAGE IN USE FOR DATABASE = 205457 MB,

DSN9022I )DSNW DSNX8CMD '-DISPLAY ACCEL' NORMAL COMPLETION,

27

• dis thread(*) accel(*) DSNV401I -DSND DISPLAY THREAD REPORT FOLLOWS -

DSNV402I -DSND ACTIVE THREADS -

NAME ST A REQ ID AUTHID PLAN ASID TOKEN

SERVER AC * 937 db2bp.exe APLMAM DISTSERV 0070 58622

V437-WORKSTATION=NSMA140149, USERID=aplmam,

APPLICATION NAME=db2bp.exe

V442-CRTKN=10.23.100.32.64565.130716111952

V445-GA176420.FC35.130716111952=58622 ACCESSING DATA FOR

::FFFF:10.0.209.127

V444-GA176420.FC35.130716111952=58622 ACCESSING DATA AT

ACCEL2D-::FFFF:10.0.63.198..1400

DISPLAY ACTIVE REPORT COMPLETE

DSN9022I -DSND DSNVDT '-DIS THREAD' NORMAL COMPLETION



• Omegamon XE for DB2 (PM80739 – classic interface)

28



• Data Studio with IBM DB2 Analytics Accelerator Studio Feature

29



• TDS/z (PM66403 - TDSZ IDAA TOLERATION )

• DB2_IDAA_ACC_x

• DB2_IDAA_STAT_x

• DB2_APPLICATION_x

30

• DB2_TRANSACTION_x

• DB2_USER_APPL_x

• DB2_USER_TRAN_x

0

50

100

150

200

250

300

350

400

zIIP

CP

0

10

20

30

40

50

60

70

80

90

100

0:0

0:0

0

1:0

0:0

0

2:0

0:0

0

3:0

0:0

0

4:0

0:0

0

5:0

0:0

0

6:0

0:0

0

7:0

0:0

0

8:0

0:0

0

9:0

0:0

0

10

:00

:00

11

:00

:00

12

:00

:00

13

:00

:00

14

:00

:00

15

:00

:00

16

:00

:00

17

:00

:00

18

:00

:00

19

:00

:00

20

:00

:00

21

:00

:00

22

:00

:00

23

:00

:00

CPU_MAX

CPU_AVG

CPU_MIN


IBM db2 analytics accelerator - usage

31

0

2

4

6

8

10

12

14

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

1.6

.20

13

2.6

.20

13

3.6

.20

13

4.6

.20

13

5.6

.20

13

6.6

.20

13

7.6

.20

13

8.6

.20

13

9.6

.20

13

10

.6.2

01

3

11

.6.2

01

3

12

.6.2

01

3

13

.6.2

01

3

14

.6.2

01

3

15

.6.2

01

3

16

.6.2

01

3

17

.6.2

01

3

18

.6.2

01

3

19

.6.2

01

3

20

.6.2

01

3

21

.6.2

01

3

22

.6.2

01

3

23

.6.2

01

3

24

.6.2

01

3

25

.6.2

01

3

26

.6.2

01

3

27

.6.2

01

3

28

.6.2

01

3

29

.6.2

01

3

30

.6.2

01

3

QUERIES EXECUTED

AVG_ELAP_TIME(sec)



32

0

200

400

600

800

1000

1200

MAX_ELAP_TIME(sec)



33

0

2

4

6

8

10

12

14

16

18

0

20000

40000

60000

80000

100000

120000

2013-01 2013-02 2013-03 2013-04 2013-05 2013-06

Query count

AVG_ELAP_TIME(sec)


IBM db2 analytics accelerator – disk storage usage

34

0

50

100

150

200

250

300

30.1.2013 28.2.2013 31.3.2013 30.4.2013 31.5.2013 30.6.2013

Disk Usage (gb)


IBM db2 analytics accelerator – query limit

35

0

2

4

6

8

10

12

14

16

18

30.1.2013 28.2.2013 31.3.2013 30.4.2013 31.5.2013 30.6.2013

MAX REQUEST

MAX_REQ

• Currently there is a 100 concurrent queries limit on DB2 analytics accelerator


Db2 analytics accelerator – is it really that fast?

• Reporting – MS Reporting services

36

0,00

5,00

10,00

15,00

20,00

25,00

30,00

35,00

40,00

Aktivni klasični računi novi

Aktivni osebni računi

Aktivni varčevalni produkti

Aktivni zlati računi

Poslovni račun Sodobne tržne poti

Število aktivnih storitev FO

Število novih storitev FO

Študentski računi

T - IDAA

T-1 year - DB2


Maintenance

• Order of update • DB2 PTFs

• IBM DB2 Analytics Accelerator studio

• IBM DB2 Analytics Accelerator server

• DB2 stored procedures

• Our deployment cycle: • Development Test IUAT Production

• DB2 PTFs + new IBM DB2 Analytics Accelerator server first upgraded on all test LPARs

• After verification PTFs are applied to production DB2

• In the meantime IBM DB2 Analytics Accelerator SP may throw warnings about potential incompatibilities

37

Upgrade of NPS software on Netezza Problem: No test environment if only one Netezza


IBM DB2 Analytics Accelerator for z/OS Tips&Tricks • Know your data – data in IBM DB2 Analytics Accelerator for z/OS is snapshot of db2 data

(applies to db2 analytics accelerator v2 only)

• WLM policy – be carefull to set correct priority for IBM DB2 Analytics Accelerator for z/OS procedures

• Regular maintenance is important! - PM57960 - UNLOAD Utility with INTERNAL format for decreasing CPU consumption in ACCEL_LOAD_TABLES

• Table ~18 mio rows • -- PTF2a--

• TOTAL TCB CPU TIME= 2.92 TOTAL ELAPSED TIME= 6.1 • TOTAL TCB CPU TIME= 3.12 TOTAL ELAPSED TIME= 4.8 • TOTAL TCB CPU TIME= 3.20 TOTAL ELAPSED TIME= 7.4

• -- PTF3 -- • TOTAL TCB CPU TIME= 1.32 TOTAL ELAPSED TIME= 2.5 • TOTAL TCB CPU TIME= 1.36 TOTAL ELAPSED TIME= 2.9

• DSN_QUERYINFO_TABLE – info about whether query is eligible for offloading to IBM DB2

Analytics Accelerator for z/OS

• Redbook – Optimizing DB2 Queries with IBM DB2 Analytics Accelerator for z/OS

38


• Less time for tuning of sql statements

• No db maintenance – define tables/refresh data

• Faster agile development

• Coexistence of OLTP and DWH databases on same LPAR

• CPU saving because of redirecting execution to IBM DB2 Analytics Accelerator for z/OS

IBM DB2 Analytics Accelerator for z/OS Benefits

39


• IBM Smart Analytics Optimizer V1 • In-memory, columnar database

• Massive parallelism, multi-core and SIMD friendly algorithms

• Generally available November 2010

• Discontinued and replaced by IBM DB2 Analytics Accelerator

• IBM DB2 Analytics Accelerator V2 • Successor of IBM Smart Analytics Accelerator V1

• Preserves key V1 value propositions and adds many more

• Generally available November 2011

• Numerous new features added via PTFs in 2012

• IBM DB2 Analytics Accelerator V3 • Generally available November 2012

• More performance, more capacity, new use cases

Fast Evolution of IBM DB2 Analytics Accelerator

40


• Propagating DB2 changes to the accelerator as they happen: Incremental Update

• Reducing disk storage cost by archiving data in the accelerator and maintaining the excellent performance for analytical queries: High Performance Storage Saver

• Workload Manager integration

• Automatic detection of needs to refresh data in the accelerator

• More query routing control for applications (all, eligible)

• More query offload (e.g. DB2 OLAP functions)

• Speeding-up data refresh and reducing associated CPU cost on System z (1)

• Accelerating in-database transformation (1)

• Enhancing high availability and scaling out (1)

• Improving performance of queries that generate very large result sets (1)

• Supporting multi-byte EBCDIC data encoding (phase 1) (1)

• Increasing capacity to more than 1 petabyte (1)

• Support for SAP workloads (1)

IDAA V3 Highlights

41

(1) – features retrofitted to V2

Generally available since November 2012


• Additional query engine: PureData System for Analytics N2001

• Support for Netezza operating system 7

• Further reduction of CPU time associated with IDAA load process • Enhancements in BSAM routines managing data on the USS pipes

• DFSMS APAR OA41706

• Multiple time zones in the same accelerator

• Limited support for LOCAL DATE setting

• Support for BITAND and TIMESTAMPDIFF functions

• Support for DECFLOAT when used as implicit cast

• Smaller enhancements to incremental update

IDAA V3 Highlights

42

Additions since GA (work-in progress, short term delivery expected)


• Support for DB2 11, including first-day ESP support

• Extending query routing eligibility to static SQL

• Support for queries from local applications that use multi-row fetch

• High Performance Storage Saver enhancements

• Built-in restore

• Better access control for archived partitions

• Explicit protection for image copies created by archiving process

• Loading from image copy

• New RTS column: 'last-changed-at' timestamp

• Explicitly used by stored procedure for detecting which tables/partitions need to be refreshed

• Can be used when deciding which partitions are eligible for archiving by HPSS

• Automatic workload balancing with multiple accelerators

What We Are Working On

43

• Incremental Update enhancements: • Supporting more DB2 systems connected to the same

accelerator

• Better performance due to fewer log records that need to be processed by the Capture component (thanks to new DB2 IFI Filtering enhancement)

• Improving performance for large result sets (thanks to new DRDA enhancements)

• WLM support for local applications

• Automated NZKit installation • Significantly reduced need for remote service access

to System z and IDAA and on-site support

• Richer monitoring at system, thread and statement level

• Supporting EBCDIC and Unicode in the same DB2 system and accelerator

• Prospective CPU cost savings and elapsed time improvement(SMF)

• Profile controlled special registers

• Separation of duty for ACCEL_CONTROL_ACCELERATOR

Mitja Makovec NLB d.d. [email protected]

Session IBM DB2 Analytics Accelerator in Production at NLB (Nova ljubljanska banka)

Namik Hrle IBM [email protected]

mailto:[email protected]



Documents

IBM DB2 Analytics Accelerator in Production at NLB