Real-World Performance Training · RWP Training Extreme DW Hands-On Exercise: Star Transformation q.high.sql Runs in 45 seconds Star transformation Spending most of our time in random

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Real-World Performance TrainingHands-On Exercise

Real-World Performance Team


Solution Progress, Step 1Initial Observations


RWP Training Extreme DW Hands-On Exercise: Baselineq.low.sql

Runs in 5 seconds

Nested Loops joins and B*Tree index access on fact

table

Almost all the time on random I/O against fact table

index and table

Cardinality estimate problem on PART


RWP Training Extreme DW Hands-On Exercise: Baselineq.med.sql

Runs in 183 seconds


table


index and table



RWP Training Extreme DW Hands-On Exercise: Baselineq.high.sql

Runs in 474 seconds


table


index and table



Progress

q.low.sql q.med.sql q.high.sql Total

Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds


Solution Progress, Step 2Gather Stats on PART


RWP Training Extreme DW Hands-On Exercise: Stats on PARTq.low.sql

Runs in 3 seconds


table

Now performing hash join to DATE_DIM

Better cardinality estimates on PART, but still off


RWP Training Extreme DW Hands-On Exercise: Stats on PARTq.med.sql

Runs in 3 seconds

Now using Bloom Filter for PART and HASH JOIN

Hash Joins to other dimensions

Better cardinality estimates on PART, but still off


RWP Training Extreme DW Hands-On Exercise: Stats on PARTq.high.sql

Runs in 6 seconds

Filtering on SUPPLIER Bloom Filter with Hash Join

Hash Joins to other dimensions


Progress



Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds


Solution Progress, Step 3Extended Stats on PART


RWP Training Extreme DW Hands-On Exercise: Extended Stats on PARTq.low.sql

Runs in 3 seconds

Now doing NL join between DATE_DIM and LINEORDER

Hash joining to PART and SUPPLIER

Accurate cardinality estimates on PART changed

join orderReference to CUSTOMER

indicates lack of Join Elimination (and constraints)


RWP Training Extreme DW Hands-On Exercise: Extended Stats on PARTq.med.sql

Runs in 3 seconds

Accurate cardinality estimate on PART yield a slightly

different right-deep plan

Accurate cardinality estimates on PART changed

join orderReference to CUSTOMER

indicates lack of Join Elimination (and constraints)


RWP Training Extreme DW Hands-On Exercise: Extended Stats on PARTq.high.sql

Runs in 6 seconds

No plan change from previous step

Reference to CUSTOMER indicates lack of Join

Elimination (and constraints)


Progress




Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds


Solution Progress, Step 4Constraints (PK, FK, NOT NULL)


RWP Training Extreme DW Hands-On Exercise: Constraintsq.low.sql

Runs in 3 seconds

No reference to CUSTOMER; Join Elimination takes place

Still doing NL join & index access for

DATE_DIM/LINEORER join


RWP Training Extreme DW Hands-On Exercise: Constraintsq.med.sql

Runs in 4 seconds


Now doing Hash Joins with Bloom Filters for both DATE_DIM and PART

dimensions

Scanning all partitions; not performing Bloom pruning


RWP Training Extreme DW Hands-On Exercise: Constraintsq.high.sql

Runs in 4 seconds


Now doing Hash Joins with Bloom Filters for both

DATE_DIM and SUPPLIER dimensions

Scanning all partitions; not performing Bloom pruning


Progress





Constraints 3 seconds 4 seconds 4 seconds 11 seconds


Solution Progress, Step 5Partition LINEORDER on LO_ORDERDATE


RWP Training Extreme DW Hands-On Exercise: Partitioningq.low.sql

Runs in 2 seconds

Using BF and Hash Join with PART dimension

Now iterating over fact table partitions for DATE_DIM NL

join


RWP Training Extreme DW Hands-On Exercise: Partitioningq.med.sql

Runs in 31 seconds

No Bloom Filters used

Execution time partition iteration due to data type

mismatch between DATE_DIM and LINEORDER

No Bloom Pruning


RWP Training Extreme DW Hands-On Exercise: Partitioningq.high.sql

Runs in 2 seconds

Bloom Filtering, Bloom Pruning, and Hash Joins


Progress






Partitioning 2 seconds 31 seconds 2 seconds 35 seconds


Solution Progress, Step 6Fix DATE_DIM Data Type


RWP Training Extreme DW Hands-On Exercise: Data Typesq.low.sql

Runs in 3 seconds

With proper data types, reverted to index access path

and NL joins

Performing partition range scan after NL join between

PART/LINEORDER


RWP Training Extreme DW Hands-On Exercise: Data Typesq.med.sql

Runs in 0.2 seconds

Right-deep, BF plan with hash joins

Bloom pruning possible with proper data types


RWP Training Extreme DW Hands-On Exercise: Data Typesq.high.sql

Runs in 1 seconds

Bloom Filtering, Bloom Pruning, and Hash Joins –

same as previous test


Progress







Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds


Solution Progress, Step 7Bitmap Indexes and Star Transformation


RWP Training Extreme DW Hands-On Exercise: Star Transformationq.low.sql Runs in 0.2 seconds

Star transformation yielded decent response time


RWP Training Extreme DW Hands-On Exercise: Star Transformationq.med.sql Runs in 5 seconds

Here, the transformation isn’t engaged but we’re still accessing the fact table via a bitmap index

Spending most of our time in random I/O


RWP Training Extreme DW Hands-On Exercise: Star Transformationq.high.sql Runs in 45 seconds

Star transformation

Spending most of our time in random I/O. Clearly, star transformation & bitmap indexes isn’t the right choice for this workload. This is

the risk with small bitmap indexes; Oracle often will cost these types of plans in when full scans

are faster


Progress








Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds


Solution Progress, Step 8Exadata, No Star Transformation, No Bitmap Indexes, DoP=4 on LINEORDER


RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4q.low.sql Runs in 1 second

In parallel, query runs quite fast


RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4q.med.sql Runs in 0.55 seconds



RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4q.high.sql Runs in 1 second



Progress









Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds


Solution Progress, Step 9Database In-Memory, DoP=4 on LINEORDER


RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4q.low.sql Runs in 0.03 seconds

TABLE ACCESS STORAGE FULL becomes TABLE ACCESS INMEMORY

FULL


RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4q.med.sql Runs in 1 second

TABLE ACCESS STORAGE FULL becomes TABLE ACCESS INMEMORY

FULL


RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4q.high.sql Runs in 0.52 seconds


Progress










DBIM with DoP=4 0.03 seconds 1 second 0.52 seconds 1.55 seconds


Solution Progress, Step 10In-Memory Aggregation (DoP = 1)


RWP Training Extreme DW Hands-On Exercise: IMAq.low.sql Runs in 0.03 seconds

The IMA transformation


RWP Training Extreme DW Hands-On Exercise: IMAq.med.sql Runs in 0.09 seconds

The IMA transformation


RWP Training Extreme DW Hands-On Exercise: IMAq.high.sql Runs in 1 second


Progressq.low.sql q.med.sql q.high.sql Total










IMA with DoP=1 0.03 seconds 0.09 seconds 1 seconds 1.12 seconds


Solution Progress, Step 11Materialized Views In-Memory, Clustered


RWP Training Extreme DW Hands-On Exercise: MJV DBIMq.low.sql Runs in 0.017 seconds


RWP Training Extreme DW Hands-On Exercise: MJV DBIMq.med.sql Runs in 0.026 seconds


RWP Training Extreme DW Hands-On Exercise: MJV DBIMq.high.sql Runs in 0.127 seconds

You can make things really fast if you take it this far. A total of 0.17

seconds for each of the 3 “typical” star queries!


Progressq.low.sql q.med.sql q.high.sql Total










IMA with DoP=1 0.03 seconds 0.09 seconds 1 seconds 1.12 seconds

MJV with DBIM 0.017 seconds 0.026 seconds 0.127 seconds 0.17 seconds


Appendix AInstructor Notes


RWP Training Extreme DW Hands-On Exercise

Objective: Demonstrate multiple steps to achieve query optimization

1. Download SQL*Monitor reports from https://stbeehive.oracle.com/content/dav/st/Real%20World%20Training/Documents/3.5%20Day%20Course/New%20PPTX%20October%202014%20Week%202/handson_sqlmon.zip

2. Show reports in “baseline” and “constraints” section – just open a browser to show them. The demo application will take too long to run them

3. Go to the demo system and launch Star Query Demo. Use Windows for display/VGA purposes and Chrome

4. Click through each of the numbered queries on the list. There 8 steps with 3 queries each; the steps are “Constraints and Stats”, “Extended Stats”, “Partitioning on the right column”, “Star transform w/ bitmap indexes”, “Exadata DoP=4”, “DBIM DoP=4”, “IMA with DoP=1”, and “MV In-Memory DoP=4”

5. You can maximize any window and click on the Description to generate the SQL*Monitor report

If things go wrong:

1. Login to scao08adm01 as oracle

2. Go to /home/oracle/users/jclarke/training/hands_on/solution

3. SQL*plus as DW_JC/DW_JC

4. Run solution.sql

Instructor Notes

https://stbeehive.oracle.com/content/dav/st/Real World Training/Documents/3.5 Day Course/New PPTX October 2014 Week 2/handson_sqlmon.zip


Appendix BCode Examples


ConstraintsNOT NULL, Primary Key, and Foreign Key Constraints


StatisticsUnlocking, Gathering Stats & Extended Stats


Star TransformationBitmap Indexes and Star Transformation


Star TransformationPartitioning LINEORDER on correct column


Star TransformationMark Tables for In-Memory Column Store and Populate


Star TransformationCheck In-Memory Population Status