If you can't read please download the document
Upload
hester-hines
View
215
Download
1
Embed Size (px)
Citation preview
SQL TuningOhio Oracle Users GroupOctober 2002 Copyright, Kris T. Mason, 2002
Kris T. MasonInformation Control Corp (ICC) Consultant10+ years with OracleOracle 7x, 8x, 9i70% Application 30% ProductionCSCC Adjunct FacultyOracle ProgrammingComputer Information Systems Concepts
IntroductionOverview of the SQL Tuning processDatabase InternalsSchema Objects involved with queriesSQL Optimizer and Execution PlansApplication construction
Why Tune?
PhilosophyBe a Good NeighborConsume only those resources you needRelease resources you are not usingThe more you know ...
Reality CheckApplications can not treat the database as a black-box and expect optimal performance. They must contribute to efficiency.Overall efficiency can be gained by tuning small, frequently executing queries as well as large infrequent onesTuning cant overcome bad designTune SQL before tuning the DB InstancePolitics, Balance, Judgement
Preparing to tuneProgrammingLanguagesApplication segmentationBind VariablesServer ConfigurationHardware (CPUs, Memory, Disks, Network)Non-Database applicationsDatabase ConfigurationInit.oraGather statistics select value, name from v$sysstat order by name;Analyze Tables & Indexes (DBMS_STATS package)DB Links and ReplicationKnow where you are, want to be and how to get thereCurrent performanceWhat is acceptable performanceHow long it will take
MinimizingLogical ReadsDisk ReadsTemp SpaceMemorySortsParsingNetwork TrafficApplication Round TripsResponse Time (Perceived & Real)
Database InternalsSystem Global Area (SGA)Database Block Buffers Block Size usually 8k - 16kHolds portions of Tables and IndexesMinimizes Expensive Disk ReadsV$DB_CACHE_ADVICEBuffer Cache Hit RatioLibrary Cache Dictionary and recent SQLTwo SQL statements are identical only if they match byte for byteProgrammers and Dynamic Query Rewrite (Cursor Sharing) can force queries to be identical via bind variablesIdentical Queries accumulate statistics in V$SQLAREA providing better visibility to statement use
Oracle FeaturesDynamic Query Rewrite (Cursor Sharing)Parallel QueryOptimizer ModesHintsAlter Session statementTable & Index PartitioningStatspack
Dynamic Query Rewrite(Cursor Sharing)Init.oraQUERY_REWRITE_ENABLED=TRUEQUERY_REWRITE_INTEGRITY=TRUSTEDTurns literal strings and values into bind variablesBypasses the programmerDifferent queries become identicalWHERE STATUS = 1 AND NAME = SMITHWHERE STATUS = 3 AND NAME = JONESWHERE STATUS = :b1 AND NAME = :b2Statistics are accumulated under the single query
Parallel QueryAllows multiple CPUs to work parts of the same query (divide and conquer)My experience:Best when 6+ CPUsCan be over done, causing different concurrently executing queries to compete for resourcesConfigure PQ Servers (#CPUs 1)*2
OptimizerRule / Choose / CostGenerates alternative execution plansPlans can be seen and understoodSET AUTOTRACE ONSET AUTOTRACE TRACEONLY
Primary Schema ObjectsTablesFull Table ScanRandom Data Block ReadROWID Direct AccessIndexIndex Organized TableIndexesB*Tree / Bitmap / Index-Only Table / Function BasedIdeal when highly selectiveIndex Only Processing avoids Data Block ReadEntirely NULL values are not stored in the indexALTER INDEX REBUILDTables & Indexes require Data block reads if not in memory already
IndexesSelectivity = (# of Distinct Values / # of Rows) * 100Selectivity = 0% when all the values have the same valueSelectivity = 100% when all the values have different values (PKs)Indexes loose effectiveness when Selectivity falls below 80%Indexes gain effectiveness when Selectivity approaches 100%
Index use can be invalidated (Index Invalidation) when the column reference in the WHERE is involved with a functionSELECT ... WHERE ID = 5 /* Uses an index */SELECT ... WHERE ID + 0 = 5 /* Does not use an index */SELECT ... WHERE NAME BETWEEN SMITH AND THOMAS /* Uses an index */SELECT ... WHERE UPPER(NAME) BETWEEN SMITH AND THOMAS /* Does not use an index */
Secondary Schema ObjectsViewsStored SELECT statementSome Hints do not propagate inside a complex viewTriggersPL/SQL codeRegistered against a specific tableFired upon specific DML events
Tuning ProcessFind SQLSQL Analyze & Top SQLTop SessionsSQL*Plus & SQL Worksheet V$SQLAREALook ForHigh Disk Reads vs. Low Rows ReturnedHigh ExecutionsDISTINCTIdentify ProblemSelectivityAccess PathJoin Method and OrderConsider Possible AlternativesHintsIndexesDocument, Measure & Test AlternativesShow your workDeploy SolutionGet credit for your work
Finding SQL to tuneV$SQLAREAset linesize 1000set pagesize 1000column disk_reads format 999,999,999,999column executions format 999,999,999,999column rows_processed format 999,999,999,999column sql_text format a800 word_wrappedselect executions, disk_reads, rows_processed, sorts, sql_text from v$sqlarea where disk_reads > 3000 order by disk_reads DESC;
Look for high resource consumers
DISK_READSEXECUTIONSSORTS
Limited to 1000 bytes of SQL Statement including white space. Be sure you have the whole statement!The entire statement exists in v$sqltext but it is broken into pieces
Helper PL/SQL FunctionsCREATE OR REPLACE FUNCTION SQUISH (P in varchar2) return varchar2 asI number;V varchar2(32000);Begin/* Change special chars to blanks */V := trim(rtrim(translate(P,chr(9)||chr(10)||chr(13),' '))); /* One blank between quotes */Loop I := instr (V,' '); /* Two blanks between quotes */ Exit when I = 0;/* V := replace two blanks with one */ V := replace (V, , );End loop;Return V;End Squish;
CREATE OR REPLACE FUNCTION CAT_SQL_TEXT (P in varchar2) return varchar2 as V varchar2(32000) := ''; Beginfor C in (select sql_text from v$sqltext where address = P order by piece) Loop V := V || C.sql_text;End loop;Return V;End cat_sql_text;
Grant select on v$sqltext to [function owner];
Finding SQL To Tune with Helper FunctionsV$SQLTEXTset linesize 1000set pagesize 1000column disk_reads format 999,999,999,999column executions format 999,999,999,999column rows_processed format 999,999,999,999column sql_text format a800 word_wrapped
select address, hash_value, executions, disk_reads, rows_processed, sorts, squish(cat_sql_text(address)) sql_text from v$sqlarea where disk_reads > 3000 order by disk_reads DESC;
Finding SQL to tuneDISTINCT
EXECUTIONS DISK_READS ROWS_PROCESSED SORTS SQL_TEXT------------- ------------- ---------------- ------- ---------------------------------------------------- 81 3,499 5,103 81 SELECT DISTINCT TABLE_B.NAME FROM TABLE_A, TABLE_B WHERE ( TABLE_A.ID = TABLE_B.ID )
The Intent of this query is to show the names in Table_B that have any Matching rows in Table_A
OriginalDocument the query as originally writtenSELECT DISTINCT TABLE_B.NAMEFROM TABLE_A, TABLE_BWHERE ( TABLE_A.ID = TABLE_B.ID );
Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=921 Card=306 Bytes=5814) 1 0 SORT (UNIQUE) (Cost=921 Card=306 Bytes=5814) 2 1 HASH JOIN (Cost=33 Card=73605 Bytes=1398495) 3 2 TABLE ACCESS (FULL) OF TABLE_B' (Cost=1 Card=307 Bytes=4298) 4 2 INDEX (FAST FULL SCAN) OF 'FK_TABLE_A_ID' (NON-UNIQUE) (Cost=14 Card=73605 Bytes=368025)
Original StatisticsStatistics---------------------------------------------------------- 43 recursive calls 8 db block gets 247 consistent gets 0 physical reads 0 redo size 1368 bytes sent via SQL*Net to client 591 bytes received via SQL*Net from client 6 SQL*Net roundtrips to/from client 5 sorts (memory) 0 sorts (disk) 63 rows processed
AlternativeDocument each version of the query as rewrittenSELECT TABLE_B.NAME FROM TABLE_BWHERE EXISTS ( SELECT 1 FROM TABLE_A WHERE TABLE_A.ID = TABLE_B.ID );
Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=16 Bytes=224) 1 0 FILTER 2 1 TABLE ACCESS (FULL) OF TABLE_B' (Cost=1 Card=16 Bytes=224) 3 1 INDEX (RANGE SCAN) OF 'FK_TABLE_A_ID' (NON-UNIQUE) (Cost=4 Card=1169 Bytes=5845)
Alternatives StatisticsStatistics---------------------------------------------------------- 0 recursive calls 4 db block gets 638 consistent gets 0 physical reads 0 redo size 1409 bytes sent via SQL*Net to client 591 bytes received via SQL*Net from client 6 SQL*Net roundtrips to/from client 2 sorts (memory) 0 sorts (disk) 63 rows processed
DocumentationOriginal versionReformed versionRewrite by hand in a form comfortable to youMake sure it is equivalent to the originalMatch querys joins to data modelEach alternativeA record of your workRecommended solution
TestingFull volume production testingTest or Personal copy of DatabaseDBMS_STATSImport Statistics from Production
What Can Be Done?Analyze Tables & IndexesHistogramsSee Oracles Execution PlanSET AUTOTRACE [ON, TRACEONLY]PLAN tableQuery Costs (Oradollars)Hints You know more than the Optimizer!Optimizer GoalFIRST_ROWSALL_ROWSAccess PathFULLINDEXQuery TransformationORDEREDSTARJoin OperationUSE_NLUSE_MERGEUSE_HASHDRIVING_SITELEADING
Execution PlansA roadmap for executing the queryTop to bottomInside to outsideExecution TechniquesFull Table Scan Not necessarily a bad thingUnique Index ScanIndex Range ScanConcatenationSortAggregationOrder ByGroup ByNested LoopsMerge JoinHash JoinIN-List Iterator
HintsSELECT /*+ hint goes here */ ...Features:Syntax errors are not reported and not usedSpecify the table alias not the table namePut preceding and trailing blankYou can use multiple hintsHints dont have to be used by the optimizer
Other Techniques to Optimize PerformanceALTER SESSION SET SORT_AREA_SIZE=1000000SET ROLLBACK_SEGMENT=RBS_BIGMinimized Chained RowsHow full are the data blocks? Random Deletes reduce efficiency of table scansUse Plan Stability to insure optimizer does not change the plan over time
Distributed QueriesQueries across multiple instancesVia a DB_LinkCreate a view of the remote tablesMinimize round-trips to the remote DBPull data to the local DB via a snapshotWire the local & remote servers together
Effects of tuning
Chart1
2440031
2755646
3002076
3128057
3255563
2732933
4977331
4607288
4439704
3050420
3216624
3359287
1461685
3972817
4634971
2480144
2078049
2247093
2387418
1217823
1100976
1460628
473575
1888641
2022442
2158865
1594354
3351965
1093388
2400641
729951
858740
1300059
939603
418737
654228
491860
628695
767432
400953
726420
988249
667529
470863
780128
940692
1005888
842897
2454981
1120904
1155951
1318518
1473361
896222
416322
931174
535284
361329
511517
664176
509920
853205
1550388
1318985
372583
547183
733474
656377
350939
473671
857583
551996
730219
906466
737113
1279820
498207
1031463
349448
618190
803809
229606
839138
755849
741377
666703
855536
1040570
2373227
58412
5798005
922272
552873
797905
985632
849963
623027
896700
698837
466540
650848
836111
542713
1008870
786579
1680805
1457768
1644372
1830549
889365
797671
1506566
1780922
Time
Reads
Physical Reads
Disk Reads
6/12/02 20:002,440,031
6/13/02 20:002,755,646
6/14/02 20:003,002,076
6/15/02 20:003,128,057
6/16/02 20:003,255,563
6/17/02 20:002,732,933
6/18/02 20:004,977,331
6/19/02 20:004,607,288
6/20/02 20:004,439,704
6/21/02 20:003,050,420
6/22/02 20:003,216,624
6/23/02 20:003,359,287
6/24/02 20:001,461,685
6/25/02 20:003,972,817
6/26/02 20:004,634,971
6/27/02 20:002,480,144
6/28/02 20:002,078,049
6/29/02 20:002,247,093
6/30/02 20:002,387,418
7/1/02 20:001,217,823
7/2/02 20:001,100,976
7/3/02 20:001,460,628
7/4/02 20:00473,575
7/5/02 20:001,888,641
7/6/02 20:002,022,442
7/7/02 20:002,158,865
7/8/02 20:001,594,354
7/9/02 20:003,351,965
7/10/02 20:001,093,388
7/11/02 20:002,400,641
7/13/02 20:00729,951
7/14/02 20:00858,740
7/15/02 20:001,300,059
7/16/02 20:00939,603Murali ran several queries that used 7M disk reads These have been subtracted out
7/17/02 20:00418,737
7/18/02 20:00654,228
7/19/02 20:00491,860
7/20/02 20:00628,695
7/21/02 20:00767,432
7/22/02 20:00400,953
7/23/02 20:00726,420
7/24/02 20:00988,249
7/25/02 20:00667,529
7/26/02 20:00470,863
7/27/02 20:00780,128
7/28/02 20:00940,692
7/29/02 20:001,005,888
7/30/02 20:00842,897
7/31/02 20:002,454,981Roger was running statistical analysis without debugging in Desi first. No adjustment made to #
8/1/02 20:001,120,904
8/2/02 20:001,155,951
8/3/02 20:001,318,518
8/4/02 20:001,473,361
8/5/02 20:00896,222
8/6/02 20:00416,322
8/7/02 20:00931,174
8/8/02 20:00535,284
8/9/02 20:00361,329
8/10/02 20:00511,517
8/11/02 20:00664,176
8/12/02 20:00509,920
8/13/02 20:00853,205
8/14/02 20:001,550,388
8/15/02 20:001,318,985
8/16/02 20:00372,583
8/17/02 20:00547,183
8/18/02 20:00733,474
8/19/02 20:00656,377
8/20/02 20:00350,939
8/21/02 20:00473,671
8/22/02 20:00857,583
8/23/02 20:00551,996
8/24/02 20:00730,219
8/25/02 20:00906,466
8/26/02 20:00737,113
8/27/02 20:001,279,820
8/28/02 20:00498,207
8/29/02 20:001,031,463
8/30/02 20:00349,448
8/31/02 20:00618,190
9/1/02 20:00803,809
9/2/02 20:00229,606
9/3/02 20:00839,138
9/4/02 20:00755,849
9/5/02 20:00741,377
9/6/02 20:00666,703
9/7/02 20:00855,536
9/8/02 20:001,040,570
9/9/02 20:002,373,227
9/10/02 20:0058,412
9/11/02 20:005,798,005
9/12/02 20:00922,272
9/13/02 20:00552,873
9/14/02 20:00797,905
9/15/02 20:00985,632
9/16/02 20:00849,963
9/17/02 20:00623,027
9/18/02 20:00896,700
9/19/02 20:00698,837
9/20/02 20:00466,540
9/21/02 20:00650,848
9/22/02 20:00836,111
9/23/02 20:00542,713
9/24/02 20:001,008,870
9/25/02 20:00786,579
9/26/02 20:001,680,805
9/27/02 20:001,457,768
9/28/02 20:001,644,372
9/29/02 20:001,830,549
9/30/02 20:00889,365
10/1/02 20:00797,671
10/2/02 20:001,506,566
10/3/02 20:001,780,922
10/4/02 20:00
10/5/02 20:00
Disk Reads
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Time
Reads
Physical Reads
Network
17/31/02 20:002,091,902256,325,1502,197,206,494
18/1/02 20:002,239,260282,333,5712,063,379,523
18/2/02 20:002,207,351270,428,4042,090,123,662
18/3/02 20:003,815,472447,995,3213,817,521,664
18/4/02 20:005,408,174623,036,4735,536,644,656
18/5/02 20:002,212,273268,739,6112,132,318,108
18/6/02 20:002,127,845260,046,4912,046,233,540
18/7/02 20:002,209,658274,765,8572,148,478,896
18/8/02 20:002,170,982266,005,5772,144,696,112
18/9/02 20:002,168,244271,212,4922,130,432,980
18/10/02 20:003,771,721447,292,8033,861,970,642
18/11/02 20:005,381,314624,133,4775,597,558,784
18/12/02 20:002,456,341314,562,8182,162,479,117
18/13/02 20:002,214,900276,328,0762,134,099,931
18/14/02 20:002,381,743292,092,2212,370,914,263
18/15/02 20:002,204,978266,998,9672,494,680,428
18/16/02 20:002,211,136269,789,2262,264,446,668
18/17/02 20:003,963,588469,882,9324,216,100,542
18/18/02 20:005,650,599659,571,0506,143,883,169
18/19/02 20:002,240,032274,784,5692,324,785,124
18/20/02 20:002,159,883261,385,8712,233,512,731
18/21/02 20:002,162,317264,297,4262,214,789,552
18/22/02 20:002,226,966276,852,4092,294,296,996
18/23/02 20:002,247,475274,817,4072,314,773,070
18/24/02 20:003,985,713467,509,3654,264,394,853
18/25/02 20:005,725,333658,426,0906,199,726,285
18/26/02 20:002,205,118266,720,0922,237,353,398
18/27/02 20:002,243,834274,945,3942,343,590,402
18/28/02 20:002,207,405271,316,1422,254,007,953
18/29/02 20:002,444,890318,179,2972,332,966,736
18/30/02 20:002,460,768301,712,3752,411,194,958
18/31/02 20:004,327,800513,699,0334,394,270,919
19/1/02 20:006,088,029706,755,0536,400,283,288
19/2/02 20:001,745,520191,989,3552,006,168,562
19/3/02 20:002,440,004301,463,0272,407,361,517
19/4/02 20:002,347,529288,013,1572,358,188,928
19/5/02 20:002,359,229282,060,7952,367,129,485
Network
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Network Bytes Sent Server to Client
Sheet3
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Network Bytes Sent Client to Server
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Network Round Trips (SQL*Net)
ApplicationsStart with optimized queries so when code is cloned it has a better chance of working wellReview 3rd party produced code such as PowerBuilderWrite code for the proper platform (application segmentation)PL/SQL is not cool but it is fast and effectiveUse Bind Variables whenever possibleAvoid Dynamic SQL if possible
ReferencesOracles TechnetDatabase Performance Tuning Guide & Referencehttp://otn.oracle.com/docs/products/oracle9i/doc_library/release2/server.920/a96533/toc.htm
www.iccohio.com
Summary
SQL Tuning is not necessarily hard, it just takes some practiceand the willingness to try