THE WHAT, HOW AND WHY
OF DB2 ACCELERATORS
CRISTIAN MOLAROMConsulting
IBM GOLD Consultant
IBM Information Champion
DISCLAIMER
PLEASE BE AWARE THAT THE ACTUAL PROGRAMMING TECHNIQUES,
ALGORITHMS AND ALL NUMERICAL PARAMETERS USED IN EXAMPLES
GIVEN IN THIS PRESENTATION ARE SUBJECT TO CHANGE AT SOME FUTURE
DATE EITHER BY A NEW VERSION OF DB2, A NEW RELEASE, A SMALL
PROGRAMMING ENHANCEMENT (SPE) OR A PROGRAMMING TEMPORARY
FIX (PTF).
THE INFORMATION CONTAINED IN THIS PRESENTATION HAS NOT BEEN
SUBMITTED TO ANY FORMAL REVIEW AND IS DISTRIBUTED ON AN “AS IS”
BASIS WITHOUT ANY WARRANTY EITHER EXPRESS OR IMPLIED. THE USE
OF THIS INFORMATION OR THE IMPLEMENTATION OF ANY OF THESE
TECHNIQUES IS A CUSTOMER RESPONSIBILITY AND DEPENDS ON THE
CUSTOMER’S ABILITY TO EVALUATE AND INTEGRATE THEM INTO THE
CUSTOMER’S OPERATIONAL ENVIRONMENT. WHILE EACH ITEM MAY HAVE
BEEN REVIEWED FOR ACCURACY IN A SPECIFIC SITUATION, THERE IS NO
GUARANTEE THAT THE SAME OR SIMILAR RESULTS WILL BE OBTAINED
ELSEWHERE. CUSTOMERS ATTEMPTING TO ADAPT THESE TECHNIQUES TO
THEIR OWN ENVIRONMENTS DO SO AT THEIR OWN RISK.
DB2 IS A TRADEMARK OF INTERNATIONAL BUSINESS MACHINE
CORPORATION. THIS PRESENTATION USES MANY TERMS THAT ARE
TRADEMARKS. WHEREVER WE ARE AWARE OF TRADEMARKS THE NAME
HAS BEEN SPELLED IN CAPITALS.
[email protected] ©® 2012
Agenda
The WHAT in DB2 Accelerators
The HOW in DB2 Accelerators
The WHY in DB2 Accelerators
Conclusions
[email protected] ©® 2012
What is a DB2 Accelerator?
Definition of “Accelerators for DB2 for z/OS queries”
– Accelerator is a solution that can run queries on behalf of DB2
– Generally a high-performance appliance
– An accelerator server is a specific instance of an accelerator
For certain types of queries processing on an accelerator can be
faster (or much faster) than processing on DB2:
– Business Intelligence queries
– Analytics
IMPORTANT: No changes are required to existing applications
IMPORTANT Transparent to users and applications: unaware of Accelerator
IMPORTANT: Users and applications continue to connect to DB2 for z/OS
[email protected] ©® 2012
What is IDAA?
IDAA = IBM DB2 Analytics Accelerator V2
– Successor of ISAO = IBM Smart Analytics Optimizer
– Improved V2 of ISAO offering
• Complete different platform � Netezza
– GA since November 2011
– Effective Capacity from 32 to 1280 TB
• (Assumes 4x compression ratio)
Requirements:
– z/OS 1.11, or later
– DB2 V9 or DB2 V10.1 for z/OS
– IBM System zEnterprise 196 or 114
– Verify at www.ibm.com/support/
docview.wss?id=swg27022331
IBM land page: www.ibm.com/software/data/db2/zos/analytics-accelerator/
[email protected] ©® 2012
What is IDAA?
IDAA allows integration of OLTP and DWH workload in the SAME DB2
[email protected] ©® 2012
The Appliance concept
A computer appliance is:
– A separate hardware device with integrated software (firmware)
– Specifically designed to provide a specific computing resource
• DB2 Accelerator � PERFORMANCE!
Known as “appliances” because of their similarities to home
appliances:
– Generally “closed and sealed”
– Not serviceable by the owner
NOT designed to allow the customer to:
– Change the software
– Reconfigure the hardware
Plug & Play Performance solution
IMPORTANT: IDAA provides Transparent integration with DB2 for z/OS
[email protected] ©® 2012
Appliance: isolation and integration
Cannot directly access the appliance
– Service is done through z/OS
– Data is owned and controlled by DB2
Provides many advantages:
– Easy maintenance and administration
– Very fast to deploy and get operative
DB2 integration
– All information about the Accelerator is routed via DB2
– The heartbeat provides active monitoring of the accelerator
– DB2 instrumentation (traces) for IDAA does not require
additional classes or IFCIDs to be started
• New fields were added to existing records
NOTE: IDAA is NOT another database: Total integration with DB2 for z/OS
[email protected] ©® 2012
DB2 Accounting Report Long: Accelerator section
Accounting contains a Distributed activity section
– Even if the application does not do ANY distributed access
– DB2 communicates with IDAA using DRDA
ACCELERATOR IDENTIFIER ACCELERATOR AVERAGE TOTAL ACCELERATOR AVERAGE TOTAL
----------- ---------- ----------- -------- ------ ------------ ----------- ------------
PRODUCT AQT02012 OCCURRENCES 1.00 24 ELAPSED TIME
SERVER IDAATF3 CONNECTS 1.00 24 SVCS TCP/IP 6:31.492821 2:36:35.8277
REQUESTS 2.00 48 ACCUM ACCEL 5:56.198064 2:22:28.7535
TIMED OUT 0.00 0 CPU TIME
FAILED 0.00 0 SVCS TCP/IP 0.000341 0.008187
SENT ACCUM ACCEL 3.357125 1:20.570989
BYTES 8026.33 192632 WAIT TIME
MESSAGES 11.00 264 ACCUM ACCEL 0.037185 0.892430
BLOCKS 0.00 0
ROWS 0.00 0 DB2 THREAD
RECEIVED CLASS 1
BYTES 7467.00 179208 ELAPSED N/A 10:47:16.523
MESSAGES 11.00 264 CP CPU N/A 16:36.545365
BLOCKS 0.00 0 SE CPU N/A 0.000000
ROWS 0.00 0 CLASS 2
ELAPSED N/A 2:47:43.0033
CP CPU N/A 16:35.173834
SE CPU N/A 0.000000
[email protected] ©® 2012
DB2 Statistics: Accelerator section
The DB2 Statistics Class 1 trace provides IDAA information
– The states of the accelerators that are in use
– The amount of processing time that is spent in accelerators
– Counts of the amounts of sent and received information
IDAATF3 ACCELERATION QUANTITY IDAATF3 CONTINUED QUANTITY
-------------------------------- --------------- ----------------------------------- --------------
QUERIES SUCCESSFULLY EXECUTED 24.00 AVG QUEUE LENGTH (LAST 3 HRS) 1.68
QUERIES FAILED TO EXECUTE 0.00 AVG QUEUE LENGTH (LAST 24 HRS) 1.68
ACCELERATOR IN INVALID STATE 0.00 MAXIMUM QUEUE LENGTH 16.00
CURRENTLY EXECUTING QUERIES 5.74 AVG QUEUE WAIT ELAPSED TIME 0.000001
MAXIMUM EXECUTING QUERIES 24.00 MAX QUEUE WAIT ELAPSED TIME 0.000083
CONNECTS TO ACCELERATOR 44.00 WORKER NODES 2.22
REQUESTS SENT TO ACCELERATOR 88.00 WORKER NODES AVG CPU UTILIZATION (%) 7.65
TIMED OUT 0.00 COORDINATOR AVG CPU UTILIZATION (%) 0.31
FAILED 0.00
BYTES SENT TO ACCELERATOR 354814.00 DISK STORAGE AVAILABLE (MB) 5948861.42
BYTES RECEIVED FROM ACCELERATOR 254580.00 IN USE (%) 5.81
MESSAGES SENT TO ACCELERATOR 484.00 IN USE FOR DATABASE (MB) 354179.00
MESSAGES RECEIVED FROM ACCEL 484.00 DATA SLICES 16.31
BLOCKS SENT TO ACCELERATOR 0.00 DATA SKEW (%) 91.69
BLOCKS RECEIVED FROM ACCELERATOR 0.00
ROWS SENT TO ACCELERATOR 0.00 PROCESSORS 17.79
ROWS RECEIVED FROM ACCELERATOR 0.00
TCP/IP SERVICES ELAPSED TIME 4:46:07.281505 ELAPSED TIME IN ACCELERATOR 4:31:41.389834
WAIT TIME IN ACCELERATOR 1.665544 CPU TIME SPENT IN ACCELERATOR 2:14.299979
[email protected] ©® 2012
DB2 commands
-DIS ACCEL DETAIL
-DIS THD (*) ACCEL(*)
DSNV401I -DA12 DISPLAY THREAD REPORT FOLLOWS -
DSNV402I -DA12 ACTIVE THREADS -
NAME ST A REQ ID AUTHID PLAN ASID TOKEN
BATCH AC * 17 IDAA102 IDAA1 DSNTEP10 0032 15480
V437-WORKSTATION=BATCH, USERID=IDAA1,
APPLICATION NAME=IDAA102
V444-DEIBMIPS.IPWASA12.C9272E5E3939=15480 ACCESSING DATA AT
IDAATF3 -::FFFF:10.101.8.100..1400
DISPLAY ACTIVE REPORT COMPLETE
DSN9022I -DA12 DSNVDT '-DIS THD' NORMAL COMPLETION
***
DSNX830I -DA12 DSNX8CDA
ACCELERATOR MEMB STATUS REQUESTS ACTV QUED MAXQ
-------------------------------- ---- -------- -------- ---- ---- ----
IDAATF3 DA12 STARTED 1757 0 0 62
LOCATION=IDAATF3 HEALTHY
DETAIL STATISTICS
LEVEL = AQT02012
STATUS = ONLINE
[email protected] ©® 2012
A new access path
TIP: IDAA can be viewed as an additional access path for dynamic SQL
[email protected] ©® 2012
Query Execution: Process Flow
16
DB2 for z/OS
Optimizer
IDA
A D
RD
A R
eq
ue
sto
r
DB2 Analytics Accelerator
App
Application
Interface
Queries executed with DB2 Analytics Accelerator
Queries executed without DB2 Analytics Accelerator
Query execution run-time for
queries that cannot be or should
not be off-loaded to IDAA
SPUSPU
CPU FPGA
Memory
SPUSPU
CPU FPGA
Memory
SPUSPU
CPU FPGA
Memory
SPUSPU
CPU FPGA
Memory
SM
P H
ost
Controlling acceleration
Setting the CURRENT QUERY ACCELERATION special register
Default value is NONE
TIP: There is no way to FORCE IDAA Query execution only. Use RLF for
controlling execution in DB2Q yet
IMPORTANT: be careful with SET CURRENT QUERY ACCELERATION
ENABLE WITH FAIL BACK in case of forgotten or big queries because them
coming back to DB2 may cause performance issues
.-=-.
>>-SET CURRENT QUERY ACCELERATION--+---+------------------------>
>--+-NONE-----------------+------------------------------------><
+-ENABLE---------------+
+-ENABLE WITH FAILBACK-+
'-host-variable--------'
[email protected] ©® 2012
Why a query may not be offloaded to IDAA?
Mandatory criteria
– Cursor is NOT defined as scrollable or row set cursor
– The query is defined as read-only
– The query is dynamic
– The query is a SELECT statement
Main reasons for NOT offloading a Query
– It uses CURRENT QUERY ACCELERATION = NONE (default)
– The accelerator or tables are disabled
– It uses static SQL or a plan with DBRMs
– It is not read-only or the cursor is scrollable or is a rowset cursor
– It references a table or column that is not loaded/enabled
– The optimizer decides DB2 for z/OS can do better
[email protected] ©® 2012
Or7It contains syntax that is not supported
SQL restrictions
SOME SQL restrictions
– The Query uses a special register OTHER than:
• CURRENT DATE
• CURRENT TIME
• CURRENT TIMESTAMP
– It contains a sequence expression (NEXTVAL or PREVVAL).
– It contains a user-defined function (UDF)
– It uses a MIN or MAX function with string values or more than
four arguments
Can identify non eligible queries in DSN_QUERYINFO_TABLE
IMPORTANT: More considerations apply, refer to SG24-8005 for more details
TIP: In some cases, a query can be rewritten to make it eligible
[email protected] ©® 2012
Workload Assessment
Done by IBM, free or charge
Collect information from DSC
– IFCID 316-318
Assessment of:
– Queries Acceleration
potential
– Estimated Elapsed time
– CPU cost
Documentation
and REXX procedure
Upload data to
IBM IBM lab
Database
Pre-process and
load
Quick
Workload Test
Tool Assessment
Report
AssessmentCustomer
Database
[email protected] ©® 2012
Virtual Accelerators
You can install virtual accelerators with or without IDAA installed
– Applies to DB2 9 for z/OS and DB2 10 for z/OS
EXPLAIN only
DSN_QUERYINFO_TABLE
– Contains information about acceleration of query blocks
EXPLAIN with SET CURRENT QUERY ACCELERATION ENABLE
REASON_CODE=0 � the query is eligible for IDAA offload
EXPLAIN ALL SET QUERYNO=228101 FOR
SELECT SUM(L_TAX)
FROM ONETB.LINEITEM
WHERE L_LINENUMBER=1
AND L_DISCOUNT > 10000
GROUP BY L_SUPPKEY
FETCH FIRST 10 ROWS ONLY;
[email protected] ©® 2012
Adding tables to an Accelerator
Queries eligible for acceleration will be executed in accelerator
ONLY IF all the tables the query is referencing has been defined
and loaded in the accelerator
Defining tables to IDAA: Add Table wizard in Accelerator Studio
Loading tables into an Accelerator
– Stored procedures and batch pipes
– Either from the Accelerator Studio or using batch jobs
[email protected] ©® 2012
Partitions belonging to the same table can be loaded in parallel
User defined degree of parallelism
Updates are done on a per-table or per-partition level
Up to 3TB / hour
– 200 GB / hour for single stream
Loading data into IDAA
NOTE There is no automatic indication that a table needs to be refreshed
[email protected] ©® 2012
Performance, performance, performance
Example of observed performance changes (figures from IBM)
– International securities company
[email protected] ©® 2012
IDAA Effects
Class 1
Elapsed
Class 1
CPU
Class 2
CPU
545% 17,921% 27,669%
618% 21,075% 32,545%
442% 22,187% 34,299%
473% 20,123% 31,135%
487% 12,256% 18,924%
719% 1,155,561% 2,233,215%
894% 1,274,866% 2,475,147%
Reduction of CPU consumption in z/OS
Potentially:
– A good portion of the CPU cycles consumed by queries running
in DB2 can be eliminated
• Queries are diverted to execute in IDAA
Amount of reduction depends on:
– The volume of queries that are eligible to run in IDAA
– It is likely complex queries will be selected to run in IDAA
• Normally these queries contribute with much CPU utilization
– Even with only a small subset of queries the potential z/OS CPU
savings could be significant
IMPORTANT: Offloaded queries returning a large result set could consume a
noticeable amount of CPU time in DB2
TIP: IDAA gives an opportunity to reduce costs by CPU reduction
[email protected] ©® 2012
Benchmark workload example
80 concurrent users executing Cognos reports
Reports are a mix of short, intermediate, and long running reports
Total of 26 queries against the DB2 for z/OS database
[email protected] ©® 2012
CPU distribution per report
Exploit WLM Client Info to identify CPU utilization per report
Workload driven from Cognos or from zLinux scripts
IMPORTANT: even a few offloaded queries can save lots of resources
[email protected] ©® 2012
Scalability
IMPORTANT: The maximum level of concurrency is 100 queries in IDAA
[email protected] ©® 2012
News from Development
What to expect in the near future
– Reduce the cost of synchronyzing data between DB2 and IDAA
– Low latency replication of changes
– Integration with z/OS WLM
– Removing limitations for routing queries to IDAA
– Performance enhancements
– Adding advanced analytics capabilities
– Providing multi-temperature data capabilities
– Extension of query acceleration to new usage cases
[email protected] ©® 2012
Enable DB2 transition into a truly universal DBMS that provides best
characteristics for both OLTP and analytical workloads
Conclusions
WHAT
HOW
WHY
�A lot better performance
�Reduction of Mainframe TCO
�DB2 integration
�Transparency to applications
�Accelerator = appliance
�OLTP + Analytics in DB2
* Inspired from Simon Sinek’s “Start with Why”
[email protected] ©® 2012
Summary
What we discussed about:
– The WHAT in DB2 Accelerators
– The HOW in DB2 Accelerators
– The WHY in DB2 Accelerators
QUESTIONS?
[email protected] ©® 2012