ITEC 450

1

MODULE 4 DATABASE TUNING AND DATA LOADSection 1 Database Tuning and Monitoring

Fall 2012

2

ITEC 450BULK DATA IMPORTS AND

EXPORTS USING DATA PUMP Previously, export (exp) and import (imp)

utilities were used to import/export data from a DB

Data Pump export (expdp) and import (impdp) utilities are more versatile and much faster Can be executed in parallel, failed/stopped jobs

can be restarted, metadata can be filtered out, etc.

Exporting/importing is allowed at all logical layers: tables, schemas, groups of objects, or an entire DB You export data and metadata Database exports can used to migrate and

upgrade between different versions of Oracle

Fall 2012

3

ITEC 450BULK DATA IMPORTS AND

EXPORTS USING DATA PUMP Fall 2012

4

ITEC 450

EXPORTING USING DATA PUMP

Steps:1. Start a shell and2. Type expdp help=y3. Do a simple export

expdp classmate/classpass@oraclass DIRECTORY=dmpdir DUMPFILE=classmate.dmp

4. Create a directory object for Data Pump C/CREATE OR REPLACE DIRECTORY DP AS 'C:\';

5. Execute the expdp utility and exportexpdp system/password@oraclass DIRECTORY=dp

DUMPFILE=system.dmp NOLOGFILE=Y

6. Export the CLASSMATE schemaexpdp system/password@oraclass DIRECTORY=dp

DUMPFILE=classmate2.dmp SCHEMAS=('CLASSMATE') NOLOGFILE=Y

Fall 2012

5

ITEC 450

IMPORTING USING DATA PUMP

Data Pump is only available to DB- and OS level administrative personnel Older utils. allow exports/imports across a

network Are secure because users have to

have the schema password names to access schemas

Steps:1. Start a shell2. Type impdp help=y

The most likely use of these utilities are as fast backups and as an easy method of copying DBs

IMP and EXP are easier to use than Data Pump, with the same options, just fewer options

Fall 2012

6

ITEC 450BULK DATA LOADS WITH

SQL*LOADER

SQL*Loader can perform magnificently in direct path, parallel mode, and using external tables Direct path loads allow appending to tables Some situations will cause single tables and even

entire SQL*Loader executions to execute using a conventional path load

SQL*Loader is not limited to individual table loads It can load into more than one table at once,

considering all constraints SQL*Loader can also perform fast direct

loads with external tables

Fall 2012

7

ITEC 450BULK DATA LOADS WITH

SQL*LOADERFall 2012

8

ITEC 450DIRECT VERSUS CONVENTIONAL

PATH LOADS In a conventional path load rows are parsed into

an array structure according to field specification, as defined by SQL Loader and table to be added to

A direct path is similar to a conventional one; but, load writes data to end of table into new block and extent structures, creating new blocks as it goes Potentially much faster than conventional path loads

A parallel direct path load is a variation on a direct path load and is potentially faster than a direct one

The external table load creates and loads an external table

Fall 2012

9

ITEC 450

SQL LOADER INPUT DATA FILES The input data files provide the data loaded

in SQL Loader Input data file contents can be fixed-length rows,

variable-length rows, or a single stream (string) Input data can consist of a single data file in

control file, or a separate data set in the input data file File does not have to exist and that input data can all

be included in the control file To avoid confusion, it is best to divide things into the

different appropriate files

Fall 2012

10

ITEC 450

THE SQL LOADER CONTROL FILE

Control file contains a mapping between input data and table structures into which data is to be loadedLOAD DATA

INFILE 'data\Chapter11\ch11.dat'INTO TABLE client APPENDFIELDS TERMINATED BY "," TRAILING NULLCOLS(

CLIENT_ID INTEGER,FIRST_NAME CHAR(10),LAST_NAME CHAR(20),CONTACT_PHONE CHAR(15),CONTACT_EMAIL CHAR(30)

)

The bad rows and discard rows can be defined tooBADFILE 'solutions\Chapter11\bad.dat'DISCARDFILE 'solutions\Chapter11\discard.dat'

Fall 2012

11

ITEC 450

ROW LOADING OPTIONSLOAD DATA INFILE ... BADFILE ... DISCARDFILE ...INTO TABLE table1 [ INSERT | REPLACE | TRUNCATE | APPEND ]FIELDS TERMINATED BY "," TRAILING NULLCOLS(

FIELD1 INTEGER EXTERNAL,FIELD2 INTEGER EXTERNAL,...FIELDn ...

) INSERT is the default

Fall 2012

12

ITEC 450

COLUMN DELIMITERS Two examples:

LOAD DATAINTO TABLE table1 TRUNCATE(

FIELD1POSITION(001:010) CHAR(10) TERMINATED BY WHITESPACE,FIELD2 POSITION(011:030) CHAR(20) TERMINATED BY WHITESPACE,... ,FIELDn ...

)

LOAD DATAINTO TABLE table1 TRUNCATE FIELDS TERMINATED BY "," TRAILING

NULLCOLS(

FIELD1 INTEGER EXTERNAL,FIELD2 INTEGER EXTERNAL,...FIELDn ...

)

Fall 2012

13

ITEC 450

THE PARAMETER FILE SQL*Loader can include a parameter file

containing repeated settings, across multiple executions of SQL*LoaderUSERID = CLASSMATE/CLASSPASS@ORACLASSDISCARDMAX = 2ERRORS = 1000000

To load your new client rows:sqlldr control=<path>\data\Chapter11\ch11.ctl log=<path>\data\

Chapter11\ch11.log parfile=<path>\data\Chapter11\ch11.par

Some of the most likely uses of SQL*Loader are to bulk load large amounts of data into a data warehouse, or when importing data from outside, into an existing database

Fall 2012

14

ITEC 450

EXAMPLES OF PERFORMANCE ISSUES A query has a response-time problem A query used to run for 5 minutes, now for an

hour and still running A tablespace runs out of disk storage space The batch window extends into day Someone submitted a “query from hell” that

just won’t stop running, and take away CPU and I/O resources

Fall 2012

15

ITEC 450PERFORMANCE INFLUENTIAL FACTORS Workload – a combination of online

transactions, batch jobs, ad hoc queries, and other system activities outside of database

Throughput – the overall capability of the computer to process data, such as I/O speed

Resources – hardware and software tools, such as chips, memory, and database kernel

Optimization – query tuning, configuration change

Contention – two or more components of the workload are competing a single resource

Fall 2012

16

ITEC 450

DATABASE PERFORMANCE TUNINGDatabase performance tuning can be defined as

the optimization of resource use to increase throughput and minimize contention, enabling the largest possible workload to be processed.

Identify the most troublesome areas Application tuning – 80% of database

performance problems are from poorly coded SQL or application logic

Database system tuning areas: Memory allocation (buffer/cache for data) Logging option (log cache, log size, log switch) I/O efficiency (data contention, fragmentation) Enabled features (parallelism, query rewrite)

Fall 2012

17

ITEC 450PERFORMANCE MANAGEMENT COMPONENTS Monitoring – scanning the environment to

collect performance information Analysis – analyzing collected information to

identify problem Correction - modifying problematic behaviors

Fall 2012

18

ITEC 450

SERVICE-LEVEL MANAGEMENTSLM is the disciplined methodology and

procedures used to ensure adequate levels of services are delivered

A service level is a measure of operational behavior.

SLM focus on mainly availability, sometimes performance. 99.9% uptime from 8:00 AM to 6:00 PM weekdays Average response time for the transaction will be two

seconds or less for workloads of 500 or fewer users All involved parties, DBA and users, have to

agree the service levels on stated objectives for availability and performance.

Fall 2012

19

ITEC 450

DBMS PERFORMANCE BASICS Do not over-tune – stop tuning when

performance reaches a pre-defined service level

Remain focused – don’t tune multiple things at once

Do not panic – “I don’t know, but I’ll find out.”

Communicate clearly – keep users informed with progress, and coordinate discussions among all involved parties

Accept reality – Proactive is the goal, but reality is often that dealing with problems as they occur.

Fall 2012

ITEC 450

20

MODULE 4 DATABASE TUNINGSection 2 Managing and Monitoring Oracle Database

Fall 2012

21

ITEC 450PROACTIVE DATABASE MONITORINGAlerts are used to monitor the database when

particular metric thresholds are crossed, or can issue simply because an event has occurred.

For each alert, you can set critical and warning threshold

Default server-generated alerts: CPU Time Per Call (warning at 8000 ms, critical

at 10,000 ms) Table space usage (warning at 85% full, critical

at 97% full) SQL Service Response Time Recovery Area Space Usage

Fall 2012

22

ITEC 450Fall 2012

ORACLE ENTERPRISE MANAGER

23

ITEC 450

MANAGING ALERTSUsing Oracle Enterprise Manager

Warning and critical thresholdsResponse action, setting notification rulesRead Chapter 10 in Oracle 2 Day DBA; and Chapter 5 in Oracle 2

Day DBA Tuning Guide

Using DBMS_SERVER_ALERT PackageUse DBMS_SERVER_ALERT.GET_THRESHOLD, SET_THRESHOLDSQL> connect sys/password as sysdba;SQL> desc dbms_server_alertSQL> select reason from dba_outstanding_alerts;SQL> select reason, resolution from dba_alert_history;Metrics and alerts information – v$sysmetric, v$servicemetric,

v$sysmetric_historyInformation about alert types – v$alert_typesSQL> select distinct object_type from v$alert_types

Fall 2012

24

ITEC 450

ALERTS SECTION OF OEM Fall 2012

25

ITEC 450AUTOMATIC WORKLOAD REPOSITORY (AWR)

AWR automatically collects and stores database performance statistics relating to problem detection and tuning.

A temporary in-memory collection of statistics in the SGA, accessible through dynamic views – v$sysstat, v$sessstat, v$session, v$session_wait, etc.

A persistent type of performance data, access by data dictionary views – dba_users, dba_source, dba_errors, etc.

By default, AWR generates snapshots once every hour, and retains the statistics for 8 days.

Read Chapter 3 in Oracle 2 Day DBA Tuning Guide

Fall 2012

26

ITEC 450Fall 2012

METRICS PAGE FROM AWR

27

ITEC 450

AWR REPORTSOracle provides a script generate summary

reports: Top five timed events Wait events and latch activity Time-model statistics Buffer pool and PGA statistics and advisories

SQL> @C:\app\Administrator\product\11.2.0\dbhome_1\RDBMS\ADMIN\awrrpt.sql;

Type Specified: htmlEnter value for num_days: 1Enter value for begin_snap: 84Enter value for end_snap: 89A report is generated under: C:\app\Administrator\product\11.2.0\

dbhome_1\BIN. The file name can be specified or default.

Fall 2012

28

ITEC 450

ACTIVE SESSION HISTORY (ASH)Oracle also provides statistics for all active

sessions every second, instead of AWR with 60 minutes by default.

Current active session data – v$session, v$active_session_historySQL> select username, status, sid, serial#, program from

v$session 2 where username is not null;SQL> select sample_time, session_id, event, module from

v$active_session_history 2 where user_id = (select user_id from dba_users where

username = 'SYSTEM'); Read Chapter 4 in Oracle 2 Day DBA Tuning

Guide

Fall 2012

29

ITEC 450AUTOMATIC DATABASE DIAGNOSTIC MONITORADDM identifies performance problems, and

recommends actions, after the AWR takes a new snapshot.

ADDM diagnoses problems such as: Expensive SQL statements I/O performance issues Resource bottlenecks, including memory and

CPU bottlenecks ADDM findings and recommendations

Finding types – problem, symptom, and informational

Recommendations – actions for each problem finding

Fall 2012

30

ITEC 450

MANAGING ADDMUsing Oracle Enterprise Manager

Performance findings of the stored ADDM reportsUsing DBMS_ADVISOR Package

CREATE_TASK – creates a new advisor taskSET_DEFAULT_TASK – modifies default values of parameters within

a taskDELETE_TASK – deletes a specific task from the repositoryGET_TASK_REPORT – Displays the most recent ADDM report

Using Oracle Provided ScriptSQL> @ C:\app\Administrator\product\11.2.0\dbhome_1\

RDBMS\ADMIN \addmrpt.sql;A report is generated under: C:\app\Administrator\product\11.2.0\

dbhome_1\BIN by default. The file name should start with addmrpt.

Fall 2012

31

ITEC 450MANAGEMENT ADVISORY FRAMEWORKThe advisors help in the performance tuning,

identifying bottlenecks and suggesting optimal sizes for key database resources.

Memory advisor – recommendations regarding the optimal sizing of the SGA and PGA.

Tuning-related advisors SQL tuning advisor – analyzing complex SQL

statements and recommending way to improve performance

SQL access advisor – advising on creating new indexes, materialized views, or materialized view logs.

Space-Related advisors – segment advisor, undo advisor

Fall 2012