115
Optimizer Yin and Yang Thomas Kyte http://asktom.oracle.com/

Optimizer Yin and Yang Thomas Kyte

Embed Size (px)

Citation preview

Page 1: Optimizer Yin and Yang Thomas Kyte

Optimizer Yin and Yang

Thomas Kytehttp://asktom.oracle.com/

Page 2: Optimizer Yin and Yang Thomas Kyte

Programming to fail…

Page 3: Optimizer Yin and Yang Thomas Kyte

Programming to fail…

• Too smart for their own good• Parse count• Parse count (failures)

• Quick stories about parsing…

Page 4: Optimizer Yin and Yang Thomas Kyte

Programming to fail…

Begin

execute immediate

‘begin internal_pkg.some_code; end;’;

Exception

when others then null;

End;

Page 5: Optimizer Yin and Yang Thomas Kyte

Programming to fail…

Sandeep - my math was wrong, I said 40% of your hard parses were failed parses.  In looking at the numbers again:

Statistic                                     Total     per Second     per Trans-------------------------------- ------------------ -------------- -------------parse count (failures)                      389,176          109.0           3.0parse count (hard)                          607,096          170.1           4.7parse count (total)                       6,775,397        1,898.0          52.3

It would be correct to say that 64% (yes, 64%!!!!!!!!!!!!!) of your parses are *FAILED* parsed.  The parse count hard included failed and successful parses - therefore, it is 389k/607k*100 to get the right percentage.

2 out of 3 SQL statements FAIL PARSING.  That is sick

Page 6: Optimizer Yin and Yang Thomas Kyte

Abusing Functions

Page 7: Optimizer Yin and Yang Thomas Kyte

Function Abuse

• Cardinality estimation issues• May reduce access paths• Can increase CPU needs (repeated function calls)• Could lead to partition elimination elimination

Page 8: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issues

ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from all_objects 5 /

Table created.

Page 9: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issues

ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where created >= to_date( '5-sep-2010', 'dd-mon-yyyy' ) 4 and created < to_date( '6-sep-2010', 'dd-mon-yyyy' ) 5 /

COUNT(*)---------- 65925

ops$tkyte%ORA11GR2> select count(*), 0.01 * count(*), 0.01 * 0.01 * count(*) 2 from t 3 /

COUNT(*) 0.01*COUNT(*) 0.01*0.01*COUNT(*)---------- ------------- ------------------ 72926 729.26 7.2926

Page 10: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issues

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T' );

PL/SQL procedure successfully completed.

• Why did I wait till here to gather statistics?

Page 11: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issues

ops$tkyte%ORA11GR2> select count(*) 2 from t t2 3 where created >= to_date( '5-sep-2010', 'dd-mon-yyyy' ) 4 and created < to_date( '6-sep-2010', 'dd-mon-yyyy' ) 5 /

COUNT(*)---------- 65925

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);

---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 291 (100)| || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 65462 | 511K| 291 (1)| 00:00:04 |---------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------

2 - filter(("CREATED"<TO_DATE(' 2010-09-06 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "CREATED">=TO_DATE(' 2010-09-05 00:00:00', 'syyyy-mm-dd hh24:mi:ss')))

Page 12: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issuesops$tkyte%ORA11GR2> select count(*) 2 from t t1 3 where trunc(created) = to_date( '5-sep-2010', 'dd-mon-yyyy' ) 4 /

COUNT(*)---------- 65925

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 294 (100)| || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 729 | 5832 | 294 (2)| 00:00:04 |---------------------------------------------------------------------------

Predicate Information (identified by operation id):--------------------------------------------------- 2 - filter(TRUNC(INTERNAL_FUNCTION("CREATED"))=TO_DATE(' 2010-09-05 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

Page 13: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issues

ops$tkyte%ORA11GR2> select count(*) 2 from t t1 3 where trunc(created) = to_date( '5-sep-2010', 'dd-mon-yyyy' ) 4 and substr( owner, 1, 3 ) = 'SYS' 5 /

COUNT(*)---------- 33535

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 292 (100)| || 1 | SORT AGGREGATE | | 1 | 14 | | ||* 2 | TABLE ACCESS FULL| T | 7 | 98 | 292 (1)| 00:00:04 |---------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------

2 - filter((SUBSTR("OWNER",1,3)='SYS' AND TRUNC(INTERNAL_FUNCTION("CREATED"))=TO_DATE(' 2010-09-05 00:00:00' 'syyyy-mm-dd hh24:mi:ss')))

Page 14: Optimizer Yin and Yang Thomas Kyte

Cardinality Estimation Issues

ops$tkyte%ORA11GR2> select count(*) 2 from t t1 3 where trunc(created) = to_date( '5-sep-2010', 'dd-mon-yyyy' ) 4 and substr( owner, 1, 3 ) = 'SYS' 5 and mod(object_id,100000) > 1 6 / COUNT(*)---------- 33535

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 292 (100)| || 1 | SORT AGGREGATE | | 1 | 19 | | ||* 2 | TABLE ACCESS FULL| T | 1 | 19 | 292 (1)| 00:00:04 |---------------------------------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 2 - filter((SUBSTR("OWNER",1,3)='SYS' AND MOD("OBJECT_ID",100000)>1 AND TRUNC(INTERNAL_FUNCTION("CREATED"))=TO_DATE(' 2010-09-05 00:00 'syyyy-mm-dd hh24:mi:ss')))

23 rows selected.

Page 15: Optimizer Yin and Yang Thomas Kyte

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1315

Compile with warnings…

SQL> alter session set plsql_warnings='enable:all’;

SQL> create or replace procedure p 2 as 3 begin 4 dbms_output.put_line( 'hello world' ); 5 exception 6 when others 7 then null; 8 end; 9 /Warning: Procedure created with compilation errors.c##tkyte%CDB1> show errorsErrors for PROCEDURE P:

LINE/COL ERROR---- -----------------------------------------------------------------6/6 PLS-06009: procedure "P" OTHERS handler does not end in RAISE or RAISE_APPLICATION_ERROR

Page 16: Optimizer Yin and Yang Thomas Kyte

16

Increased CPU

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_date varchar2(30) := '01-jan-2011'; 4 l_start number := dbms_utility.get_cpu_time; 5 begin 6 for i in 1 .. 10 7 loop 8 for x in ( select owner, object_name 9 from big_table.big_table 10 where created = l_date ) 11 loop 12 null; 13 end loop; 14 end loop; 15 dbms_output.put_line( 'CPU: ' || 16 to_char( dbms_utility.get_cpu_time-l_start ) ); 17 end; 18 /SP2-0804: Procedure created with compilation warningsops$tkyte%ORA11GR2> exec pCPU: 132

Page 17: Optimizer Yin and Yang Thomas Kyte

17

Increased CPU

ops$tkyte%ORA11GR2> show errors procedure pErrors for PROCEDURE P:

LINE/COL ERROR-------- -----------------------------------------------------------------10/36 PLW-07204: conversion away from column type may result in sub-optimal query plan

… 7 loop 8 for x in ( select owner, object_name 9 from big_table.big_table 10 where created = l_date ) 11 loop 12 null; 13 end loop;…

Page 18: Optimizer Yin and Yang Thomas Kyte

18

Increased CPU

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_date date := to_date('01-jan-2011','dd-mon-yyyy'); 4 l_start number := dbms_utility.get_cpu_time; 5 begin 6 for i in 1 .. 10 7 loop 8 for x in ( select owner, object_name 9 from big_table.big_table 10 where created = l_date ) 11 loop 12 null; 13 end loop; 14 end loop; 15 dbms_output.put_line( 'CPU: ' || 16 to_char( dbms_utility.get_cpu_time-l_start ) ); 17 end; 18 /Procedure created.ops$tkyte%ORA11GR2> exec pCPU: 94 30% less CPU in this case

Page 19: Optimizer Yin and Yang Thomas Kyte

19

Reduced Access Paths

ops$tkyte%ORA11GR2> create table t 2 ( x varchar2(20) constraint t_pk primary key, 3 y varchar2(30) 4 );Table created.

ops$tkyte%ORA11GR2> insert into t 2 select user_id, username 3 from all_users;47 rows created.

ops$tkyte%ORA11GR2> commit;Commit complete.

Page 20: Optimizer Yin and Yang Thomas Kyte

20

Reduced Access Paths

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_rec t%rowtype; 4 l_key number := 5; 5 begin 6 select * into l_rec from t where x = l_key; 7 for x in (select plan_table_output 8 from TABLE( dbms_xplan.display_cursor())) 9 loop 10 dbms_output.put_line( x.plan_table_output ); 11 end loop; 12 end; 13 /

SP2-0804: Procedure created with compilation warnings

Page 21: Optimizer Yin and Yang Thomas Kyte

21

Reduced Access Paths

… 5 begin 6 select * into l_rec from t where x = l_key; 7 for x in (select plan_table_output…

ops$tkyte%ORA11GR2> show errorsErrors for PROCEDURE P:

LINE/COL ERROR-------- -----------------------------------------------------------6/42 PLW-07204: conversion away from column type may result in sub-optimal query plan

Page 22: Optimizer Yin and Yang Thomas Kyte

22

Reduced Access Paths

ops$tkyte%ORA11GR2> exec p

SQL_ID 18796jgha0hwz, child number 0-------------------------------------SELECT * FROM T WHERE X = :B1

Plan hash value: 1601196873

--------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |--------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 3 (100)| ||* 1 | TABLE ACCESS FULL| T | 1 | 29 | 3 (0)| 00:00:01 |--------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------1 - filter(TO_NUMBER("X")=:B1)

Page 23: Optimizer Yin and Yang Thomas Kyte

23

Reduced Access Paths

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_rec t%rowtype; 4 l_key varchar2(5) := '5'; 5 begin 6 select * into l_rec from t where x = l_key; 7 for x in (select plan_table_output 8 from TABLE( dbms_xplan.display_cursor())) 9 loop 10 dbms_output.put_line( x.plan_table_output ); 11 end loop; 12 end; 13 /Procedure created.

ops$tkyte%ORA11GR2> show errorsNo errors.

Page 24: Optimizer Yin and Yang Thomas Kyte

24

Reduced Access Paths

ops$tkyte%ORA11GR2> exec p

SQL_ID 18796jgha0hwz, child number 1-------------------------------------SELECT * FROM T WHERE X = :B1

Plan hash value: 1303508680------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 1 (100)| || 1 | TABLE ACCESS BY INDEX ROWID| T | 1 | 29 | 1 (0)| 00:00:01 ||* 2 | INDEX UNIQUE SCAN | T_PK | 1 | | 1 (0)| 00:00:01 |------------------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------2 - access("X"=:B1)

Page 25: Optimizer Yin and Yang Thomas Kyte

25

Partition Elimination Eliminated

ops$tkyte%ORA11GR2> CREATE TABLE t 2 ( 3 dt date, 4 x int, 5 y varchar2(30) 6 ) 7 PARTITION BY RANGE (dt) 8 ( 9 PARTITION part1 VALUES LESS THAN(to_date('31-jan-2011', 'dd-mon-yyyy')), 10 PARTITION part2 VALUES LESS THAN(to_date('28-feb-2011', 'dd-mon-yyyy')) 11 ) 12 /

Table created.

Page 26: Optimizer Yin and Yang Thomas Kyte

26

Partition Elimination Eliminated

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_date timestamp := timestamp'2011-01-15 00:00:00.000'; 4 l_count number; 5 begin 6 select count(*) into l_count from t where dt = l_date; 7 8 for x in (select plan_table_output 9 from TABLE( dbms_xplan.display_cursor() ) ) 10 loop 11 dbms_output.put_line( '.'||x.plan_table_output ); 12 end loop; 13 end; 14 /

SP2-0804: Procedure created with compilation warnings

Page 27: Optimizer Yin and Yang Thomas Kyte

27

Partition Elimination Eliminated

… 5 begin 6 select count(*) into l_count from t where dt = l_date; 7 …

SP2-0804: Procedure created with compilation warnings

ops$tkyte%ORA11GR2> show errorsErrors for PROCEDURE P:

LINE/COL ERROR-------- --------------------------------------------------------------6/47 PLW-07204: conversion away from column type may result in sub-optimal query plan

Page 28: Optimizer Yin and Yang Thomas Kyte

28

Partition Elimination Eliminated

SQL_ID 0t5m83d3m67q7, child number 0-------------------------------------SELECT COUNT(*) FROM T WHERE DT = :B1

Plan hash value: 3225603066---------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |---------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 2 (100)| | | || 1 | SORT AGGREGATE | | 1 | 9 | | | | || 2 | PARTITION RANGE ALL| | 1 | 9 | 2 (0)| 00:00:01 | 1 | 2 ||* 3 | TABLE ACCESS FULL | T | 1 | 9 | 2 (0)| 00:00:01 | 1 | 2 |---------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------

3 - filter(INTERNAL_FUNCTION("DT")=:B1)

Page 29: Optimizer Yin and Yang Thomas Kyte

29

Partition Elimination Eliminated

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_date date := to_date( '2011-01-15', 'yyyy-mm-dd' ); 4 l_count number; 5 begin 6 select count(*) into l_count from t where dt = l_date; 7 8 for x in (select plan_table_output 9 from TABLE( dbms_xplan.display_cursor() ) ) 10 loop 11 dbms_output.put_line( '.'||x.plan_table_output ); 12 end loop; 13 end; 14 /Procedure created.ops$tkyte%ORA11GR2> show errorsNo errors.

Page 30: Optimizer Yin and Yang Thomas Kyte

30

Partition Elimination Eliminated

.SQL_ID 0t5m83d3m67q7, child number 1

.-------------------------------------

.SELECT COUNT(*) FROM T WHERE DT = :B1

.

.Plan hash value: 3660200434

.

.------------------------------------------------------------------------------------------------

.| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |

.------------------------------------------------------------------------------------------------

.| 0 | SELECT STATEMENT | | | | 2 (100)| | | |

.| 1 | SORT AGGREGATE | | 1 | 9 | | | | |

.| 2 | PARTITION RANGE SINGLE| | 1 | 9 | 2 (0)| 00:00:01 | KEY | KEY |

.|* 3 | TABLE ACCESS FULL | T | 1 | 9 | 2 (0)| 00:00:01 | KEY | KEY |

.------------------------------------------------------------------------------------------------

.

.Predicate Information (identified by operation id):

.---------------------------------------------------

.

. 3 - filter("DT"=:B1)

Page 31: Optimizer Yin and Yang Thomas Kyte

31

Partition Elimination Eliminated

ops$tkyte%ORA11GR2> alter session set Plsql_Warnings = 'error:all‘;

ops$tkyte%ORA11GR2> create or replace procedure p authid definer 2 as 3 l_date timestamp := timestamp'2011-01-15 00:00:00.000'; 4 l_count number; 5 begin 6 select count(*) into l_count from t where dt = l_date; 7 8 for x in (select plan_table_output 9 from TABLE( dbms_xplan.display_cursor() ) ) 10 loop 11 dbms_output.put_line( '.'||x.plan_table_output ); 12 end loop; 13 end; 14 /

Page 32: Optimizer Yin and Yang Thomas Kyte

<Insert Picture Here>

Gathering stats when you shouldn’t

Page 33: Optimizer Yin and Yang Thomas Kyte

33

Right command, wrong time

• Temporary tables– Empty at 3am, full at 3pm

• Queue Tables– Same as above

• Partitioned Transactional Tables– Partition starts empty– Grows rapidly– Gather stats every hour? (no…)

Page 34: Optimizer Yin and Yang Thomas Kyte

34

Right command, wrong time – Temporary tables

• Temporary tables– Dynamic Sampling might be appropriate (might

be shared over sessions)

– Fill with representative data, Gather and Lock

– Cardinality/Opt_Estimate Hints on a session by session basis

– 12c Session Private Statistics• Hard parse…

Page 35: Optimizer Yin and Yang Thomas Kyte

35

Copies stats from source partition to destination partition

Adjusts min & max values for partition column at both partition & global level

Copies statistics of the dependent objects

Columns, local indexes etc.

Does not update global indexes

Maybe use DBMS_STATS.COPY_TABLE_STATS();

Sales Table

SALES_1995

:

SALES_Q4_2003

SALES_Q1_2004

DBMS_STATS.COPY_TABLE_STATS(‘SH’,

'SALES’,

'SALES_Q4_2003’,

'SALES_Q1_2004’);

Right command, wrong time – Partitioned tables

Page 36: Optimizer Yin and Yang Thomas Kyte

<Insert Picture Here>

Not Having Representative Stats When You Should

Page 37: Optimizer Yin and Yang Thomas Kyte

Is 10% enough?

• 1,000,000 rows• Grows at a rate of 10,000 rows per month• 10 months – no stats…• What might happen?

Page 38: Optimizer Yin and Yang Thomas Kyte

38

Is 10% right?

ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from ( 5 select add_months(sysdate,-100) + mod( rownum, 3000 ) dt 6 from dual 7 connect by level <= 1000000 8 ) 9 where dt < trunc(sysdate,'y') 10 /

Table created.

Page 39: Optimizer Yin and Yang Thomas Kyte

39

Is 10% right?

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T' );

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR2> select num_rows from user_tables where table_name = 'T';

NUM_ROWS---------- 994672

Page 40: Optimizer Yin and Yang Thomas Kyte

40

Is 10% right?

ops$tkyte%ORA11GR2> insert into t 2 select trunc(sysdate,'y') + mod( rownum, 150 ) dt 3 from dual 4 connect by level <= 50000 5 /

50000 rows created.

ops$tkyte%ORA11GR2> commit;

Commit complete.

Page 41: Optimizer Yin and Yang Thomas Kyte

41

Is 10% right?

ops$tkyte%ORA11GR2> select trunc(dt,'mm'), count(*) 2 from t 3 where dt >= add_months( trunc(sysdate,'y'),-3) 4 group by trunc(dt,'mm') order by 1;

TRUNC(DT, COUNT(*)--------- ----------01-OCT-13 1032301-NOV-13 999001-DEC-13 1032301-JAN-14 1035301-FEB-14 934401-MAR-14 1032301-APR-14 999001-MAY-14 9990

8 rows selected.

Page 42: Optimizer Yin and Yang Thomas Kyte

42

Is 10% right?

ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where dt between to_date( '01-dec-2013' ) 4 and to_date( '31-dec-2013' );

COUNT(*)---------- 9990

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 506 (100)| || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 10517 | 84136 | 506 (2)| 00:00:07 |---------------------------------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 2 - filter(("DT">=TO_DATE(' 2013-12-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "DT"<=TO_DATE(' 2013-12-31 00:00:00', 'syyyy-mm-d hh24:mi:ss')))

Page 43: Optimizer Yin and Yang Thomas Kyte

43

Is 10% right?

ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where dt between to_date( '01-jan-2014' ) 4 and to_date( '31-jan-2014' );

COUNT(*)---------- 10353

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 506 (100)| || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 333 | 2664 | 506 (2)| 00:00:07 |---------------------------------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 2 - filter(("DT">=TO_DATE(' 2014-01-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "DT"<=TO_DATE(' 2014-01-31 00:00:00', 'syyyy-mm-d hh24:mi:ss')))

Page 44: Optimizer Yin and Yang Thomas Kyte

44

Is 10% right?

ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where dt between to_date( '01-sep-2014' ) 4 and to_date( '30-sep-2014' );

COUNT(*)---------- 0

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 506 (100)| || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 333 | 2664 | 506 (2)| 00:00:07 |---------------------------------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 2 - filter(("DT">=TO_DATE(' 2014-09-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "DT"<=TO_DATE(' 2014-09-30 00:00:00', 'syyyy-mm-d hh24:mi:ss')))

Page 45: Optimizer Yin and Yang Thomas Kyte

45

Is 10% right?

When might be the time to gather stats?

At what %?

“it depends”

Page 46: Optimizer Yin and Yang Thomas Kyte

46

Is 10% right?

ops$tkyte%ORA11GR2> create or replace function stats_to_date(p_raw raw) 2 return date 3 as 4 l_dt date; 5 begin 6 dbms_stats.convert_raw_value(p_raw,l_dt); 7 return l_dt; 8 end; 9 /Function created.

ops$tkyte%ORA11GR2> select stats_to_date( low_value ), stats_to_date( high_value ) 2 from user_tab_columns 3 where table_name = 'T';

STATS_TO_ STATS_TO_--------- ---------31-OCT-05 31-DEC-13

Page 47: Optimizer Yin and Yang Thomas Kyte

47

Is 10% right?

ops$tkyte%ORA11GR2> declare 2 l_distcnt number; 3 l_density number; 4 l_nullcnt number; 5 rec_srec dbms_stats.statrec; 6 datevals dbms_stats.datearray; 7 l_avgclen number; 8 l_low date; 9 l_high date; 10 BEGIN 11 dbms_stats.get_column_stats 12 (user, 'T', 'DT', 13 distcnt => l_distcnt, 14 density => l_density, 15 nullcnt => l_nullcnt, 16 srec => rec_srec, 17 avgclen => l_avgclen ); 18 19 select stats_to_date(low_value), stats_to_date(high_value) 20 into l_low, l_high 21 from user_tab_col_statistics 22 where table_name = 'T' 23 and column_name = 'DT';

25 l_high := add_months(l_high, 5); 26 datevals := dbms_stats.datearray (l_low, l_high); 27 rec_srec.minval:=NULL; 28 rec_srec.maxval:=NULL; 29 rec_srec.bkvals:=NULL; 30 rec_srec.novals:=NULL; 31 32 dbms_stats.prepare_column_values (rec_srec, datevals); 33 34 dbms_stats.set_column_stats 35 (user, 'T', 'DT', 36 distcnt => l_distcnt, 37 density => l_density, 38 nullcnt => l_nullcnt, 39 srec => rec_srec, 40 avgclen => l_avgclen ); 41 END; 42 /

Page 48: Optimizer Yin and Yang Thomas Kyte

48

Is 10% right?

ops$tkyte%ORA11GR2> select stats_to_date( low_value ), stats_to_date( high_value ) 2 from user_tab_columns 3 where table_name = 'T';

STATS_TO_ STATS_TO_--------- ---------31-OCT-05 31-MAY-14

Page 49: Optimizer Yin and Yang Thomas Kyte

49

Is 10% right?

ops$tkyte%ORA11GR2> select count(*) 2 from t t2 3 where dt between to_date( '01-jan-2014' ) 4 and to_date( '31-jan-2014' );

COUNT(*)---------- 10353

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor);---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 506 (100)| || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 10188 | 81504 | 506 (2)| 00:00:07 |---------------------------------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 2 - filter(("DT">=TO_DATE(' 2014-01-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND "DT"<=TO_DATE(' 2014-01-31 00:00:00', 'syyyy-mm-d hh24:mi:ss')))

Page 50: Optimizer Yin and Yang Thomas Kyte

50

Is 10% right?

Knowledge of your data is a mandatory prerequisite to

developing a sensible statistics implementation

Page 51: Optimizer Yin and Yang Thomas Kyte

Hints are a bad idea (in general)

Page 52: Optimizer Yin and Yang Thomas Kyte

Hints are a bad idea

• Who here thinks they know how to hint?• Two types of hints– Good hints– Bad hints

• Hints are fragile– Slightly different environments..– Upgrades/Patches

• When done correctly, they are still bad (forces a plan forever if you do it right)

Page 53: Optimizer Yin and Yang Thomas Kyte

Hinting is hard

ops$tkyte%ORA11GR2> create table sales as select * from sh.sales;

Table created.

ops$tkyte%ORA11GR2> create table customers as select * from sh.customers;

Table created.

ops$tkyte%ORA11GR2> create index sales_cust_bix on sales(cust_id);

Index created.

ops$tkyte%ORA11GR2> alter table customers add constraint customers_pk primary key(cust_id);

Table altered.

Page 54: Optimizer Yin and Yang Thomas Kyte

Hinting is hard

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats ( user, 'SALES' );

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats ( user, 'CUSTOMERS' );

PL/SQL procedure successfully completed.

Page 55: Optimizer Yin and Yang Thomas Kyte

Hinting is hard

ops$tkyte%ORA11GR2> select /*+ use_nl_with_index(s sales_cust_bix) */ 2 c.cust_id, c.cust_first_name, 3 c.cust_last_name, s.amount_sold 4 from customers c, sales s 5 where c.cust_id = s.cust_id 6 /

Execution Plan----------------------------------------------------------Plan hash value: 2056508761

--------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Ti--------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 918K| 26M| | 2690 (1)| 00|* 1 | HASH JOIN | | 918K| 26M| 1736K| 2690 (1)| 00| 2 | TABLE ACCESS FULL| CUSTOMERS | 55500 | 1083K| | 405 (1)| 00| 3 | TABLE ACCESS FULL| SALES | 918K| 8973K| | 1236 (1)| 00--------------------------------------------------------------------------------

Page 56: Optimizer Yin and Yang Thomas Kyte

Hinting is hard

atom_hint=(@=0x5b1f10 err=0 resol=1 used=0 token=898 org=1 lvl=3 txt=USE_NL_WITH_INDEX ("S" "SALES_CUST_BIX") )

Page 57: Optimizer Yin and Yang Thomas Kyte

Hinting is hard

ops$tkyte%ORA11GR2> select /*+ leading(c s) use_nl_with_index(s sales_cust_bix) */ 2 c.cust_id, c.cust_first_name, 3 c.cust_last_name, s.amount_sold 4 from customers c, sales s 5 where c.cust_id = s.cust_id 6 /

Execution Plan----------------------------------------------------------Plan hash value: 4100337089

--------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%C--------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 918K| 26M| 852K| 1 | NESTED LOOPS | | | || 2 | NESTED LOOPS | | 918K| 26M| 852K| 3 | TABLE ACCESS FULL | CUSTOMERS | 55500 | 1083K| 405|* 4 | INDEX RANGE SCAN | SALES_CUST_BIX | 130 | | 2| 5 | TABLE ACCESS BY INDEX ROWID| SALES | 17 | 170 | 107--------------------------------------------------------------------------------

Page 58: Optimizer Yin and Yang Thomas Kyte

Hinting is hard

ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor(format=>'+outline'));..

--------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%C--------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | | | 852K(1| 1 | NESTED LOOPS | | | || 2 | NESTED LOOPS | | 918K| 26M| 852K| 3 | TABLE ACCESS FULL | CUSTOMERS | 55500 | 1083K| 405|* 4 | INDEX RANGE SCAN | SALES_CUST_BIX | 130 | | 2| 5 | TABLE ACCESS BY INDEX ROWID| SALES | 17 | 170 | 107--------------------------------------------------------------------------------Outline Data------------- /*+ BEGIN_OUTLINE_DATA IGNORE_OPTIM_EMBEDDED_HINTS OPTIMIZER_FEATURES_ENABLE('11.2.0.2') DB_VERSION('11.2.0.3') ALL_ROWS OUTLINE_LEAF(@"SEL$1") FULL(@"SEL$1" "C"@"SEL$1") INDEX(@"SEL$1" "S"@"SEL$1" ("SALES"."CUST_ID")) LEADING(@"SEL$1" "C"@"SEL$1" "S"@"SEL$1") USE_NL(@"SEL$1" "S"@"SEL$1") NLJ_BATCHING(@"SEL$1" "S"@"SEL$1") END_OUTLINE_DATA */

Page 59: Optimizer Yin and Yang Thomas Kyte

Hinting is bad

If you can hint it, you can baseline it. If you baseline it – it will use that plan, and that plan can evolve over

time (if we let it)

Page 60: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> CREATE TABLE t AS SELECT * FROM all_objects;

ops$tkyte%ORA11GR2> ALTER TABLE t ADD CONSTRAINT t_id_pk PRIMARY KEY (object_id);

ops$tkyte%ORA11GR2> CREATE INDEX t_idx_type ON t(object_type);

ops$tkyte%ORA11GR2> exec DBMS_STATS.GATHER_TABLE_STATS(user,'T');

Page 61: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> SELECT * 2 FROM t t1 3 WHERE t1.object_type = 'TABLE' 4 AND t1.object_id > (SELECT MAX(t2.object_id) - 500000 FROM t t2);

------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 94 | 9118 | 32| 1 | TABLE ACCESS BY INDEX ROWID | T | 94 | 9118 | 30| 2 | BITMAP CONVERSION TO ROWIDS | | | || 3 | BITMAP AND | | | || 4 | BITMAP CONVERSION FROM ROWIDS| | | || 5 | SORT ORDER BY | | | ||* 6 | INDEX RANGE SCAN | T_ID_PK | 1871 | | 3| 7 | SORT AGGREGATE | | 1 | 5 || 8 | INDEX FULL SCAN (MIN/MAX)| T_ID_PK | 1 | 5 | 2| 9 | BITMAP CONVERSION FROM ROWIDS| | | ||* 10 | INDEX RANGE SCAN | T_IDX_TYPE | 1871 | | 6------------------------------------------------------------------------------

Page 62: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> begin 2 dbms_output.put_line( 3 dbms_spm.load_plans_from_cursor_cache 4 ( sql_id => 'crvk9z6mx9n4d' ) 5 ); 6 end; 7 /1 ops$tkyte%ORA11GR2> select sql_handle, 2 substr(sql_text,1,10)||'...'|| 3 substr(sql_text,length(sql_text)-10) stext, 4 plan_name, enabled 5 from dba_sql_plan_baselines 6 where sql_text like 7 'SELECT%FROM t t1%(SELECT MAX(t2.object_id) - 500000 FROM t t2)';

SQL_HANDLE STEXT PLAN_NAME ENA-------------------- ------------------------ ------------------------------ ---SQL_e738c19a5191e8fd SELECT * SQL_PLAN_fff61m98t3u7xda64b1bb YES ... FROM t t2)

Page 63: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> begin 2 dbms_output.put_line( 3 dbms_spm.alter_sql_plan_baseline 4 ( sql_handle => 'SQL_e738c19a5191e8fd', 5 attribute_name => 'enabled', 6 attribute_value => 'NO' ) 7 ); 8 end; 9 /1

Page 64: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> select sql_handle, 2 substr(sql_text,1,10)||'...'|| 3 substr(sql_text,length(sql_text)-10) stext, 4 plan_name, enabled 5 from dba_sql_plan_baselines 6 where sql_text like 7 'SELECT%FROM t t1%(SELECT MAX(t2.object_id) - 500000 FROM t t2)';

SQL_HANDLE STEXT PLAN_NAME ENA-------------------- ------------------------ ------------------------------ ---SQL_e738c19a5191e8fd SELECT * SQL_PLAN_fff61m98t3u7xda64b1bb NO ... FROM t t2)

Page 65: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> SELECT /*+ first_rows(1) */ * 2 FROM t t1 3 WHERE t1.object_type = 'TABLE' 4 AND t1.object_id > (SELECT MAX(t2.object_id) - 500000 FROM t t2);

Execution Plan----------------------------------------------------------Plan hash value: 1289158178

------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 97 | 4 (0)|* 1 | TABLE ACCESS BY INDEX ROWID| T | 1 | 97 | 2 (0)|* 2 | INDEX RANGE SCAN | T_IDX_TYPE | 1871 | | 1 (0)| 3 | SORT AGGREGATE | | 1 | 5 || 4 | INDEX FULL SCAN (MIN/MAX)| T_ID_PK | 1 | 5 | 2 (0)------------------------------------------------------------------------------

Page 66: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> begin 2 dbms_output.put_line( 3 dbms_spm.load_plans_from_cursor_cache 4 ( sql_id => '5mn39tz7fpjnu', 5 plan_hash_value => 1289158178, 6 sql_handle => 'SQL_e738c19a5191e8fd' ) 7 ); 8 end; 9 /1

Page 67: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> select sql_handle, 2 substr(sql_text,1,10)||'...'|| 3 substr(sql_text,length(sql_text)-10) stext, 4 plan_name, enabled 5 from dba_sql_plan_baselines 6 where sql_text like 7 'SELECT%FROM t t1%(SELECT MAX(t2.object_id) - 500000 FROM t t2)';

SQL_HANDLE STEXT PLAN_NAME ENA-------------------- ------------------------ ------------------------------ ---SQL_e738c19a5191e8fd SELECT * SQL_PLAN_fff61m98t3u7x971f1a3f YES ... FROM t t2)

SQL_e738c19a5191e8fd SELECT * SQL_PLAN_fff61m98t3u7xda64b1bb NO ... FROM t t2)

Page 68: Optimizer Yin and Yang Thomas Kyte

Don’t hint - baseline

ops$tkyte%ORA11GR2> SELECT * 2 FROM t t1 3 WHERE t1.object_type = 'TABLE' 4 AND t1.object_id > (SELECT MAX(t2.object_id) - 500000 FROM t t2);------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 97 | 4 (0)|* 1 | TABLE ACCESS BY INDEX ROWID| T | 1 | 97 | 2 (0)|* 2 | INDEX RANGE SCAN | T_IDX_TYPE | 1871 | | 1 (0)| 3 | SORT AGGREGATE | | 1 | 5 || 4 | INDEX FULL SCAN (MIN/MAX)| T_ID_PK | 1 | 5 | 2 (0)------------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------

1 - filter("T1"."OBJECT_ID"> (SELECT /*+ PUSH_SUBQ INDEX ("T2" "T_ID_PK") * MAX("T2"."OBJECT_ID")-500000 FROM "T" "T2")) 2 - access("T1"."OBJECT_TYPE"='TABLE')

Note----- - SQL plan baseline "SQL_PLAN_fff61m98t3u7x971f1a3f" used for this statement

Page 69: Optimizer Yin and Yang Thomas Kyte

Baseline

And consider this a patch – a quick fix. The real solution will be in

cardinalities…

Page 70: Optimizer Yin and Yang Thomas Kyte

What Estimate Percent to Use?

Page 71: Optimizer Yin and Yang Thomas Kyte

What Estimate Percent?

The two most important statistics are…

Page 72: Optimizer Yin and Yang Thomas Kyte

What Estimate Percent?

The two most important statistics are…

NUM_ROWS

NDV

Page 73: Optimizer Yin and Yang Thomas Kyte

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.73

How to gather statistics

# 1 most commonly asked question– “What sample size should I use?”

Controlled by ESTIMATE_PRECENT parameter From 11g onwards use default value AUTO_SAMPLE_SIZE

– New hash based algorithm

– Speed of a 10% sample

– Accuracy of 100% sample

Sample Size

More info in the following paper http://dl.acm.org/citation.cfm?id=1376721

Page 74: Optimizer Yin and Yang Thomas Kyte

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.74

How to gather statistics

Speed of a 10% sample

Accuracy of 100% sample

Sample Size

Run Num AUTO_SAMPLE_SIZE 10% SAMPLE 100% SAMPLE

1 00:02:21.86 00:02:31.56 00:08:24.10

2 00:02:38.11 00:02:49.49 00:07:38.25

3 00:02:39.31 00:02:38.55 00:07:37.83

Column Name

NDV with AUTO_SAMPLE_SIZE

NDV with 10% SAMPLE

NDV with 100%

SAMPLE

C1 59852 31464 60351

C2 1270912 608544 1289760

C3 768384 359424 777942

Page 75: Optimizer Yin and Yang Thomas Kyte

“X” is bad

Page 76: Optimizer Yin and Yang Thomas Kyte

“X” is bad

• Nothing is 100% true in software• Conversely, nothing is 100% false

• Every thing, every feature, has a time and a place

• Never say never– Never say always• I always say……

• Do not disable things globally

Page 77: Optimizer Yin and Yang Thomas Kyte

select * from t1, t2 where t1. id = t2. id and t1.small_distinct = :x

Is therea best way

to do something – every time?

• T1 is large, where small_distinct = :x returns much of the table

• T2 is large

Page 78: Optimizer Yin and Yang Thomas Kyte

<Insert Picture Here>

select * from t1, t2 where t1. id = t2. Id and t1.small_distinct = :x

Is therea best way

to do something – every time?

HASH JOIN TABLE ACCESS FULL T1 TABLE ACCESS FULL T2

SELECT STATEMENT NESTED LOOPS TABLE ACCESS BY INDEX ROWID(T1) INDEX RANGE SCAN T1_IDX TABLE ACCESS BY INDEX ROWID(T2) INDEX UNIQUE SCAN T2_PK

Page 79: Optimizer Yin and Yang Thomas Kyte

Is therea best way

to do something – every time?

HASH JOIN TABLE ACCESS FULL T1 TABLE ACCESS FULL T2

SELECT STATEMENT NESTED LOOPS TABLE ACCESS BY INDEX ROWID(T1) INDEX RANGE SCAN T1_IDX TABLE ACCESS BY INDEX ROWID(T2) INDEX UNIQUE SCAN T2_PK

call count cpu elapsed disk query

Fetch 35227 5.63 9.32 23380 59350

Fetch 35227 912.07 3440.70 1154555 121367981

Page 80: Optimizer Yin and Yang Thomas Kyte

Is therea best way

to do something – every time?

HASH JOIN TABLE ACCESS FULL T1 TABLE ACCESS FULL T2

SELECT STATEMENT NESTED LOOPS TABLE ACCESS BY INDEX ROWID(T1) INDEX RANGE SCAN T1_IDX TABLE ACCESS BY INDEX ROWID(T2) INDEX UNIQUE SCAN T2_PK

call count cpu elapsed disk query

Fetch 1 4.55 5.16 12152 12456

Fetch 1 0.05 0.09 12 15

Page 81: Optimizer Yin and Yang Thomas Kyte
Page 82: Optimizer Yin and Yang Thomas Kyte
Page 83: Optimizer Yin and Yang Thomas Kyte

“Tune” this query

Page 84: Optimizer Yin and Yang Thomas Kyte

“Tune” this query

I need a volunteer…

Page 85: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> set autotrace traceonly explain

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

---------------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

---------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | 65 | 1 (0)| 00:00:01 |

| 1 | SORT AGGREGATE | | 1 | 65 | | |

| 2 | NESTED LOOPS | | 1 | 65 | 1 (0)| 00:00:01 |

| 3 | NESTED LOOPS | | 1 | 52 | 1 (0)| 00:00:01 |

| 4 | TABLE ACCESS BY INDEX ROWID| T3 | 1 | 26 | 1 (0)| 00:00:01 |

|* 5 | INDEX RANGE SCAN | T3_IDXON_SOME_OTHER_ID | 1 | | 1 (0)| 00:00:01 |

| 6 | TABLE ACCESS BY INDEX ROWID| T2 | 1 | 26 | 0 (0)| 00:00:01 |

|* 7 | INDEX RANGE SCAN | T2_IDXON_T2_ID | 1 | | 0 (0)| 00:00:01 |

|* 8 | INDEX RANGE SCAN | T1_IDXON_T1_ID | 1 | 13 | 0 (0)| 00:00:01 |

---------------------------------------------------------------------------------------------------------

Page 86: Optimizer Yin and Yang Thomas Kyte

Tune this query

• Impossible task given the information you have• You can remove (+) and that is about it (but we already do

that)

• So, let’s see what the developer gave us to work with…

ops$tkyte%ORA11GR2> set autotrace traceonly explain

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

Page 87: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> CREATE TABLE T1

2 (

3 T1_ID NUMBER(18) ,

4 data varchar2(1000)

5 );

Table created.

ops$tkyte%ORA11GR2> create index t1_idxon_t1_id on t1(t1_id);

Index created.

ops$tkyte%ORA11GR2> CREATE TABLE T2

2 (

3 T2_ID NUMBER(18) ,

4 T1_ID NUMBER(18) ,

5 data varchar2(1000)

6 );

Table created.

ops$tkyte%ORA11GR2> create index t2_idxon_t2_id on t2(t2_id);

Index created.

Page 88: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> CREATE TABLE T3

2 (

3 T3_ID NUMBER(18) ,

4 SOME_OTHER_ID NUMBER(18),

5 data varchar2(1000)

6 );

Table created.

ops$tkyte%ORA11GR2> create index t3_idxon_t3_id on t3(t3_id);

Index created.

ops$tkyte%ORA11GR2> create index t3_idxon_some_other_id on t3(some_other_id);

Index created.

Page 89: Optimizer Yin and Yang Thomas Kyte

Tune this query

• Still impossible task given the information you have

• You don’t know what or how the tables relate to each other – 1:1, 1:M?

• You don’t know if the relationships are mandatory or optional

• “Application enforces them” you are told– So you ask for them – primary keys, foreign keys, all constraints

Page 90: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> ALTER TABLE T1 ADD CONSTRAINT T1_PK1 PRIMARY KEY (T1_ID);

Table altered.

ops$tkyte%ORA11GR2> ALTER TABLE T2

2 ADD CONSTRAINT T2_PK1

3 PRIMARY KEY (T2_ID);

Table altered.

ops$tkyte%ORA11GR2> ALTER TABLE T3

2 ADD CONSTRAINT T3_ORDER_PK1

3 PRIMARY KEY (T3_ID);

Table altered.

• Now we know primary keys and quite a few “NOT NULL” constraints

Page 91: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> ALTER TABLE T2

2 ADD CONSTRAINT T2_OSO_FK1

3 FOREIGN KEY (T1_ID)

4 REFERENCES T1 (T1_ID);

Table altered.

ops$tkyte%ORA11GR2> ALTER TABLE T3

2 ADD CONSTRAINT T3_OLS_S_FK1

3 FOREIGN KEY (T3_ID)

4 REFERENCES T2 (T2_ID);

Table altered.

ops$tkyte%ORA11GR2> alter table t2

2 modify t1_id not null;

Table altered.

• Along with foreign keys – and a NOT NULL constraint on T2

Page 92: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> set autotrace traceonly explain

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

T1( T1_ID primary key )

T2( T2_ID primary key, T1_ID foreign key to T1 and NOT NULL )

T3( T3_ID primary key, T3_ID foreign key to T2(T2_ID) )

T1

T3 T2

t1.t1_id = t2.t1_id

t2.t2_id = t3.t3_id

Page 93: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

ops$tkyte%ORA11GR2> select count(*)

2 from t3

3 where t3.some_other_id = to_number(:v0);

--------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

--------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1 | 13 | 1 (0)| 00:00:01 |

| 1 | SORT AGGREGATE | | 1 | 13 | | |

|* 2 | INDEX RANGE SCAN| T3_IDXON_SOME_OTHER_ID | 1 | 13 | 1 (0)| 00:00:01 |

--------------------------------------------------------------------------------------------

• These are now semantically equivalent queries.• How did I get there?

Page 94: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

• First, we know the outer join is not necessary– Where t2.col = t3.col(+) and t3.anything = ‘something’– Implies the (+) is not necessary

• If the outer join ‘happened’, then t3.anything would be NULL! And t3.anything = to_number(:v0) would never be satisfied

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id

5 and t3.some_other_id = to_number(:x);

Page 95: Optimizer Yin and Yang Thomas Kyte

Tune this query

• Second, we know that T1 is not relevant to the query– Nothing is projected from T1 in the output– T1(t1_id) is the primary key, joined to T2(t1_id) – so T2 is

“key preserved” – T2(t1_id) is NOT NULL and is a foreign key to T1– Therefore, when you join T1 to T2 – every row in T2 appears

at least once and at most once in the output

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

ops$tkyte%ORA11GR2> select count(*)

2 from t2, t3

3 where t2.t2_id = t3.t3_id

5 and t3.some_other_id = to_number(:x);

Page 96: Optimizer Yin and Yang Thomas Kyte

Tune this query

• Lastly, we know that T2 is not relevant to the query– Nothing is projected from T2 in the output– T2(T2_ID) is the primary key, joined to T3(T3_ID) – so T3 is

“key preserved” – T3(T3_ID) is NOT NULL and is a foreign key to T2– Therefore, when you join T2 to T3 – every row in T3 appears

at least once and at most once in the output

ops$tkyte%ORA11GR2> select count(*)

2 from t1, t2, t3

3 where t1.t1_id = t2.t1_id

4 and t2.t2_id = t3.t3_id(+)

5 and t3.some_other_id = to_number(:x);

ops$tkyte%ORA11GR2> select count(*)

2 from t3

3 where t3.some_other_id = to_number(:x);

Page 97: Optimizer Yin and Yang Thomas Kyte

Tune this query

ops$tkyte%ORA11GR2> SELECT COUNT(*)

2 FROM T1, T2, T3

3 WHERE T2.order_id = T1.order_id

4 AND T2.service_order_id = T3.service_order_id (+)

5 AND T3.related_service_order_id = TO_NUMBER(:v0);

ops$tkyte%ORA11GR2> SELECT COUNT(*)

2 FROM T3

3 WHERE T3.related_service_order_id = TO_NUMBER(:v0);

Is the same as…. But only because of the constraints in

place…

Actually.. We could probably ‘tune’ this more…

Page 98: Optimizer Yin and Yang Thomas Kyte

Tune this query

• So, do your developers have to be this smart?• Nope.. 10053 trace (after constraints added) shows:

SQL:******* UNPARSED QUERY IS *******

SELECT COUNT(*) "COUNT(*)" FROM "OPS$TKYTE"."T1" "T1","OPS$TKYTE"."T2" "T2","OPS$TKYTE"."T3" "T3" WHERE "T2"."T1_ID"="T1"."T1_ID" AND "T2"."T2_ID"="T3"."T3_ID"(+) AND "T3"."SOME_OTHER_ID"=TO_NUMBER(:B1)

JE: eliminate table: T1 (T1)

...

SQL:******* UNPARSED QUERY IS *******

SELECT COUNT(*) "COUNT(*)" FROM "OPS$TKYTE"."T2" "T2","OPS$TKYTE"."T3" "T3" WHERE "T2"."T2_ID"="T3"."T3_ID"(+) AND "T3"."SOME_OTHER_ID"=TO_NUMBER(:B1)

Query block SEL$FFB75F5A (#0) simplified

...

OJE: Converting outer join of T3 and T2 to inner-join.

...

SQL:******* UNPARSED QUERY IS *******

SELECT COUNT(*) "COUNT(*)" FROM "OPS$TKYTE"."T2" "T2","OPS$TKYTE"."T3" "T3" WHERE "T3"."T3_ID"="T2"."T2_ID" AND "T3"."SOME_OTHER_ID"=TO_NUMBER(:B1)

JE: eliminate table: T2 (T2)

SQL:******* UNPARSED QUERY IS *******

SELECT COUNT(*) "COUNT(*)" FROM "OPS$TKYTE"."T3" "T3" WHERE "T3"."SOME_OTHER_ID"=TO_NUMBER(:B1)

Page 99: Optimizer Yin and Yang Thomas Kyte

Tune this query

• Datatypes are constraints, affect cardinality estimates• Check constraints used for query rewrite• NOT NULL as well• And foreign key/primary key/unique constraints• Dimensions are used to rewrite

• What about a data warehouse?

• What about deferrable constraints?

Page 100: Optimizer Yin and Yang Thomas Kyte

Fake Default Values

Page 101: Optimizer Yin and Yang Thomas Kyte

Fear of Nulls

• Use some out of range value– Which obviously changes the high/low values– Which impacts cardinality estimates

• Afraid of not being able to use indexes– Nulls are not indexed – NOT TRUE

• Could the use of fake values lead to data integrity issues?

Page 102: Optimizer Yin and Yang Thomas Kyte

Fake Values

ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from ( 5 select add_months(sysdate,-100) + mod( rownum, 3000 ) dt 6 from dual 7 connect by level <= 1000000 8 ) 9 where dt < trunc(sysdate,'y') 10 /

Table created.

ops$tkyte%ORA11GR2> insert into t 2 select * 3 from ( 4 select null dt 5 from dual 6 connect by level <= 1000000 7 ) 8 /

1000000 rows created.

Page 103: Optimizer Yin and Yang Thomas Kyte

Fake Values

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T' );

PL/SQL procedure successfully completed.

Page 104: Optimizer Yin and Yang Thomas Kyte

Fake Values

ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where dt between to_date( '01-jun-2013' ) and to_date( '30-jun-2013' );

COUNT(*)---------- 9657

Execution Plan----------------------------------------------------------Plan hash value: 2966233522

---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 4 | 949 (2)| 00:00:12 || 1 | SORT AGGREGATE | | 1 | 4 | | ||* 2 | TABLE ACCESS FULL| T | 10337 | 41348 | 949 (2)| 00:00:12 |---------------------------------------------------------------------------

Page 105: Optimizer Yin and Yang Thomas Kyte

Fake Values

ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from ( 5 select add_months(sysdate,-100) + mod( rownum, 3000 ) dt 6 from dual 7 connect by level <= 1000000 8 ) 9 where dt < trunc(sysdate,'y') 10 /

Table created.

ops$tkyte%ORA11GR2> insert into t 2 select * 3 from ( 4 select to_date( '01-jan-9999') dt 5 from dual 6 connect by level <= 1000000 7 ) 8 /

1000000 rows created.

Page 106: Optimizer Yin and Yang Thomas Kyte

Fake Values

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T' );

PL/SQL procedure successfully completed.

Page 107: Optimizer Yin and Yang Thomas Kyte

Fake Values

ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where dt between to_date( '01-jun-2013' ) and to_date( '30-jun-2013' );

COUNT(*)---------- 9657

Execution Plan----------------------------------------------------------Plan hash value: 2966233522

---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 8 | 1018 (2)| 00:00:13 || 1 | SORT AGGREGATE | | 1 | 8 | | ||* 2 | TABLE ACCESS FULL| T | 1356 | 10848 | 1018 (2)| 00:00:13 |---------------------------------------------------------------------------

Page 108: Optimizer Yin and Yang Thomas Kyte

Null Values

ops$tkyte%ORA11GR2> create table t 2 as 3 select case when mod(rownum,1000)=0 then null else object_type end otype, 4 stage.* 5 from stage 6 /Table created.

ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T' );PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR2> create index t_idx on t(otype);Index created.

ops$tkyte%ORA11GR2> analyze index t_idx validate structure;Index analyzed.

ops$tkyte%ORA11GR2> select lf_rows, (select count(*) from t) , 2 lf_rows- (select count(*) from t) diff 3 from index_stats;

LF_ROWS (SELECTCOUNT(*)FROMT) DIFF---------- --------------------- ---------- 79920 80000 -80

Page 109: Optimizer Yin and Yang Thomas Kyte

Null Values

ops$tkyte%ORA11GR2> select * from t where otype is null;--------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |--------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 80 | 3760 | 163 (1)| 00:00:02 ||* 1 | TABLE ACCESS FULL| T | 80 | 3760 | 163 (1)| 00:00:02 |--------------------------------------------------------------------------

ops$tkyte%ORA11GR2> select /*+ index( t t_idx ) */ * from t where otype is null;--------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |--------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 80 | 3760 | 163 (1)| 00:00:02 ||* 1 | TABLE ACCESS FULL| T | 80 | 3760 | 163 (1)| 00:00:02 |--------------------------------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------

1 - filter("OTYPE" IS NULL)

Page 110: Optimizer Yin and Yang Thomas Kyte

Null Values

ops$tkyte%ORA11GR2> drop index t_idx;Index dropped.

ops$tkyte%ORA11GR2> create index t_idx on t(otype,0);Index created.

Page 111: Optimizer Yin and Yang Thomas Kyte

Null Values

ops$tkyte%ORA11GR2> select * from t where otype is null;

Execution Plan----------------------------------------------------------Plan hash value: 470836197

-------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time-------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 80 | 3760 | 5 (0)| 00:0| 1 | TABLE ACCESS BY INDEX ROWID| T | 80 | 3760 | 5 (0)| 00:0|* 2 | INDEX RANGE SCAN | T_IDX | 80 | | 2 (0)| 00:0-------------------------------------------------------------------------------

Predicate Information (identified by operation id):--------------------------------------------------- 2 - access("OTYPE" IS NULL)

Page 112: Optimizer Yin and Yang Thomas Kyte
Page 113: Optimizer Yin and Yang Thomas Kyte

Birmingham Traffic Engineer Gregory Dawkins says the city may change the system to keep Roberson from receiving more tickets. He says "maybe we just need to leave that part blank altogether."

Page 114: Optimizer Yin and Yang Thomas Kyte
Page 115: Optimizer Yin and Yang Thomas Kyte