Oracle Db 11g Readiness Technical Uk Sep07 2

8/8/2019 Oracle Db 11g Readiness Technical Uk Sep07 2

1/154

PL/SQL Enhancements in 11g


2/154

Enhanced SQL

PIVOT and UNPIVOT

100Q3Jeans

1000Q4Jeans

500Q3Jeans

1000Q1Jeans

1000Q2Shoes

2000Q1Shoes

AMOUNTQUARTERPROD

1000600Null1000Jeans

NullNull10002000Shoes

Q4Q3Q2Q1PROD

Enhanced performance of ETL in your warehouse

SELECT * FROM sales

PIVOT

(sum(amount)

FOR quarter in

Q1,Q2,Q3,Q4);


3/154


Fine Grained Dependency Tracking


4/154

The challenge

View vends up invalid in 10.2 because we know only

that its dependency parent has changed at thegranularity of the whole object

create table t(a number)

/

create view v as select a from t

/alter table t add(Unheard_Of number)

/

select status from User_Objects

where Object_Name = 'V'

/


5/154

Fine Grained Dependency Tracking

In 11.1 we track dependencies at the level of element

within unit so we know that these changes have no consequence

create table t(a number)

/

create view v as select a from t

/alter table t add(Unheard_Of number)

/

select status from User_Objects

where Object_Name = 'V'

/


6/154


Real Native Compilation


7/154

The challenge

Through 10.2, PL/SQL compiled to a native DLL issignificantly faster than PL/SQL compiled for

interpretation by the PVM Oracle translates PL/SQL source to C code and

leaves the last step to a 3rd partyC compiler

BUT some customers religion forbids a C compileron a production box!

AND other customers religion forbids paying tolicense a C compiler when theyve already paid tolicense Oracle Database!


8/154

Real native compilation

In 11.1, Oracle translates PL/SQL source directlytothe DLL for the current hardware

Moreover, Oracle does the linking and loading so thatthe filesystem directories are no longer needed

So PL/SQL native compilation will work out of the box and without compromising religion

Only one parameter remains: the on/off switch,PLSQL_Code_Type


9/154

Real native compilation

As a bonus, its faster!

Compilationwith real native is twice as fast as with C native

The Whetstone benchmark is 2.5x faster with real nativethan with C native at run-time

Contrived tests have shown 20x run-time speedup


10/154


Sequence in a PL/SQL expression


11/154

create or replace trigger Trg

before insert on My_Table for each row

declare

s number;begin

-- Annoying locution

select My_Seq.Nextval into s from Dual;

:New.PK := s;

end;

/

The challenge


12/154

Sequence in a PL/SQL expression

create or replace trigger Trg

before insert on My_Table for each row

begin

:New.ID := My_Seq.Nextval;

end;

/


13/154


Tools support


14/154

Challenge Inherit code, find a problem in one area that links elsewhere, got

lost following reference after reference Impact Analysis: how do you determine who references a

particular variable?

Solution PL/Scope

Usage

PL/Scope

alter session set PLScope_Settings = 'identifiers:all'

create or replace...

Select . from User_Identifiers where Object_Name = ...

UI expected in SQL Developer 2.0


15/154

Challenge Where is time really being spent in PL/SQL code?

Solution PL/SQL Hierarchical Profiler

reports dynamic execution profile organised by subprogram accounts for SQL and PL/SQL separately

generates hypertexted HTML reports

Usage

PL/SQL Hierarchical Profiler

> EXECUTE DBMS_HPROF.START_PROFILING('PLSHPROF_DIR', 'test.trc');

% plshprof

UI expected in SQL Developer 2.0


16/154

XML DB Enhancements in 11g

Fast, better and easier

See:- New Features in Oracle XML DB, for Oracle Database 11g Release 1,

An Oracle White Paper, June 2007


17/154

In Place Schema Evolution

10.2 copyEvolve() unloads and reloads the data when the XMLSchema changes

very flexible but expensive: time to evolve schema is proportionate

to the amount of data being managed 11.1 In Place Schema Evolution allows simple changes to

registered XML schemas with zero down-time

no data copy required

schema change takes a few seconds regardless of amount of data. changes to the XML must not invalidate existing documents

add optional elements and attributes

adding new values to enumerations

increase in length


18/154

XMLType Partitioning

Challenge

repeating elements typify XML documents and aremanaged using nested tables

Solution

leverage 11.1 REF partitioning

nested Tables are partitioned using the same keyas the top level XMLType table

supports partition maintenance operations

supports partition pruning


19/154

XML Misc Improvements

Significant performance improvements

Large node handling eliminates current 64K limit on

size of a text-node. Stream based replication support for LOB-based

XML storage

Events: trigger support for the XML DB repository. e.g. inserting a purchase order into approved folder triggersordering process


20/154

Binary XML

Challenge

10.2 schema-less XML not well optimizedSolution

11.1 binary XML optimized for schema-less andflexible schema-based XML

Post Parse compact representation of XML

Flexible XML Schema Support

High performance fragment access and extractionwith new XML Index


21/154

Binary XML

AppServer

WebCache

DatabaseClient

Binary XML Binary XML Binary XML

Oracle Binary XML

Single format for on-disk, in-memory and on-the-wire Reduced CPU and Memory overhead

single format avoids parse and serialize issue between tiers

Reduced Network Overhead

post-parse format used when XML moves between tiers


22/154

XML Index

New universal index for Binary and LOB basedXMLType

All possible paths are indexed by default

XML index optimization

asynchronous operation possible

path sub-setting controls what is indexed

partial re-indexing (binary XML and securefiles only)

Repository integration with XML Index on XML DBrepository


23/154

Java Enhancements in 11g

Doing it inside like you do outside

See:- Java Developer Center on OTNhttp://www.oracle.com/technology/tech/java/index.html


24/154

JDBC PerformanceSaving 1 Million Roundtrips per Day

Prefetch in First Roundtrip (JDBC-Thin)

saves 1 database roundtrip per SELECT Statement

benefit for ad-hoc queries

New Native AQ protocol (JDBC-Thin)

AQ/OJMS operations 40%-300% Faster

reduced server-side CPU Utilization


25/154

Java Standards

JDBC 4.0 SQLXML

RowId

NChar & Nclob

Wrapper Interface

SQLException Hierarchy

conn.createBlob, createClob, etc

Disconnect Java VM upgrade from database Java 5 (i.e. JDK 1.5) in base 11.1 release

Java 6 (i.e. JDK 1.6) and newer JDKs, in patch sets

From Black Box to White Box Monitoring JMX Instrumentation

Support standard JMX-enabled tools(i.e., JConsole)


26/154

Java in the Database

Mass AdoptionChallenge Performance

Solution

New JIT compiler compiles Java in the database natively and transparently (onthe fly) without the need of a C compiler

Enabled by default, Out-of-the-Box

Up to an Order of magnitude (~x 10) speed-up for pure Java, 10g versus 11g,out-of-the-box

Challenge Ease of Use

Solution JDK-Like interface, with code on filesystem, using familiar notation,

e.g. {classpath}, -D system properties Eases the migration for developers from JDK VM world


27/154

Enhancements for PHP developers

Accessing the database will never be the same

See:- PHP Developer Center on OTNhttp://www.oracle.com/technology/tech/php/index.html


28/154

PHP users have asked for

Database Connection Pooling Web apps issue hundreds connects per second

Connection establishment is expensive

PHP does not allow real multi-threading

persistent vs non persistent connections

Existing client-side connection pooling not possible

Options: apache process can repeatedly connect/disconnect

apache process can hold onto private connection

Results in inefficient use of the database resources


29/154

Database Resident Connection Pool

No Connection Pooling 11g Database Resident Connection Pooling


30/154

DB

WS

Native Database Web Service

PL/SQL

SQL

XQuery

SOA

DBA

?

WSDLAuto-

Gene

rate

SOAP

Easy: a few steps, no other software installation and no coding

Secure: HTTP authentication and easily integrate with WS Security Products

High performance : C-based engine, natively build in to the Oracle DB


31/154

Oracle Scheduler

enhancements in Oracle Database 11g

Local OS jobs run as any OS user

Remote OS jobs

no Oracle database required

on node run as any OS user

Agent

11g

ScheduleJobs

Execute remoteOS job as OS user X

Execute remoteOS Job as OS user Z

Agent

Execute local OSjob A as OS user X &job B as OS user Y


32/154

OLAP Data Mining

Business Intelligence

See:-

Oracle Database 11g for Data Warehousing

and Business Intelligence

An Oracle White Paper

July 2007


33/154

What is Oracle OLAP Option?

A multidimensional calculation and aggregation engineembedded in the database

multidimensional data types (Cubes and Dimensions)

OLAP API for cube definition and multidimensional query

and coming in Oracle Database 11g . transparent SQL interface to OLAP Cubes and dimensions

cube-organized materialized views

OLAP metadata in the Oracle Data Dictionary


34/154

Oracle OLAP 11g

New Features in Oracle Database 11g

Data accessibility: Enhanced SQL access to OLAPcubes

SQL Optimizer awareness of OLAP cubes

Cube Organized Materialized Views

System-maintained generation of cube and dimension

views Feature accessibility: Shift the focus from the skilled

OLAP specialist to the database generalist

Cube and dimensions in the Oracle Data Dictionary

Cost-based aggregation

Dimensionally aware SQL-like calculation syntax


35/154

Cube-Organized Materialized Views

Materialized view with data stored in an OLAP cube

Automatic query write of summary queries into the cube

for improving performance of summary queries

Materialized view managed refresh of cubes anddimensions

Similar to materialized view on pre-built table; data is

stored in cube and MV is a meta data object


36/154

Cube-Organized Materialized Views

Cube MVCube OrganizedCube OrganizedMV representingMV representing

summary datasummary data

Tables

Detail data managed inDetail data managed in

relational tablesrelational tables

SQL QuerySQL Query

Summary dataSummary data

managed in cubesmanaged in cubes

DatabaseDatabase--managedmanaged

refreshrefresh

QueryQuery

RewriteRewrite

Detail data is stored inrelational tables

Application queriesrelational tables with SQL

Summary data ismanaged by OLAP cube

Access to summary dataoccurs via automaticquery rewrite to the cube

Application is unchanged,but updates and queriesare faster


37/154

Query Rewrite to Cube

select a13.FAMILY_ID FAMILY_ID,

a13.FAMILY_DSC FAMILY_DSC,

a12.CALENDAR_YEAR_ID CALENDAR_YEAR_ID,

a12.CALENDAR_YEAR_DSC CALENDAR_YEAR_DSC,

sum(a11.SALES) WJXBFS1from UNITS_FACT a11

join TIME_DIM a12

on (a11.MONTH_ID = a12.MONTH_ID)

join PRODUCT_DIM a13

on (a11.ITEM_ID = a13.ITEM_ID)

group by a13.FAMILY_ID,

a13.FAMILY_DSC,

a12.CALENDAR_YEAR_ID,

a12.CALENDAR_YEAR_DSC;

Select from relational tables

GROUP BY

SUM(SALES)


38/154

Query Rewrite to Cube

OPERATION

---------------------------------------

SELECT STATEMENT

HASH

HASH JOINHASH JOIN

CUBE SCAN CB$UNITS_CUBE

VIEW

HASH

TABLE ACCESS PRODUCT_DIMVIEW

HASH

TABLE ACCESS TIME_DIM

Cube Access


39/154

What is Oracle Data Mining?

In-Database Mining Engine wide range of algorithms

anomaly detection attribute importance

association rules clustering classification & regression nonnegative matrix factorization structured & unstructured data (text mining) BLAST (life sciences similarity search algorithm)

Oracle Data Miner (GUI) simplified, guided data mining

Spreadsheet Add-In for Predictive Analytics 1-click data mining from a spreadsheet

PL/SQL API & Java (JDM) API develop advanced analytical applications

O l D Mi i 11


40/154

Simplified development and deployment of models

supermodels: data preparation combined with mining model

tighter database integration DMSYS schema eliminated; improvedsharing of objects, security, manageability

New algorithms: General Linear Models

multivariate linear regression; logistic regression

basic tool for microeconomics analysis and business decision-making. Indicatethe impact of different attributes on a key performance indicator.

What is the impact on SUVs sales if the price of gas goes up by a $1.00?

Oracle Data Mining 11g

New Features in Oracle Database 11g

Why In Database Analytics ?


41/154

Why In-Database Analytics ?

Fewer moving parts : data remains in the databaseat all times with appropriate access security

control mechanisms Straightforward inclusion within interesting

and arbitrarily complex queries

Real-world scalability and performance Fast scoring: 2.5 million records scored in 6 seconds on a single CPU

system

Real-time scoring: 100 models on a single CPU: 0.085 seconds

Integration with Oracle BI EE


42/154

Integration with Oracle BI EE ODM provides likelihoodof fraud and other

important questions.

S it


43/154

Security

User Management Access Control

Data Protection

Monitoring

See:- Oracle Database 11g Release 1 TransparentSolutions for Security and Compliance


June 2007

K D i f D t S it


44/154

Key Drivers for Data Security

Sarbanes-Oxley (SOX),

Foreign Exchange Instruments andExchange Law (J-SOX) EU Privacy Directives, CA SB 1386. Payment Card Industry (PCI) Adequate IT controls, COSO, COBIT

Separation of duty, Proof of compliance,Risk Assessment and Monitoring

Large percentage of threats go undetected Outsourcing and off-shoring trend

Customers want to monitor insider/DBA

Regulatory Compliance

Insider Threats

O l D t b S it P d t


45/154

Oracle Database Security Products

Core

PlatformSecurity

User Management Oracle Identity Management Enterprise User Security

Data Protection Oracle Advanced Security Oracle Secure Backup

Access Control Oracle Database Vault Oracle Label Security

Monitoring Oracle Audit Vault EM Configuration Pack

Oracle Database 11g


46/154

User Management for Compliance Centralized User Management

Consolidate database accounts withshared database schemas

Centrally managed DBAs

Validated with Oracle Virtual Directory

Enterprise Strong Authentication

Kerberos (MSFT, MIT) PKI (x.509v3)

Password

SYSDBA Strong Authentication

Database Enterprise Edition Feature Requires Oracle Identity Management

Available since Oracle 8.1.6

Oracle Database 11gEnterprise User Security

FinancialDatabase

CustomerDatabase

HRDatabase

Oracle IdentityManagement

EUS EUS



47/154


Core

PlatformSecurity





Oracle Label Security


48/154

Oracle Label Security

Manageability

Policy based model

Multiple policies

supported ACME, HR, Legal

Policies are umbrellasapplying to one or

more tables, schemas,users

Web based management

Integrated with OracleIdentity Management



49/154


Core

PlatformSecurity





Oracle Advanced Security


50/154


Tablespace Encryption

Define a new tablespace as encrypted

cannot convert existing, un-encrypted tablespaces

however, content can be moved into encrypted tablespaces

Always salted for higher security

Overcomes limitation of column-based TDE:

supports indexes other than b-tree supports foreign keys

No additional management overhead

integrated into TDE key management, same wallet used asfor column based Transparent Data Encryption

No storage overhead (!)



51/154


Master key stored in HSM device

Store the Master key in an external hardware device

Master key never leaves the device

Standard PKCS #11 API allows customers to choose froma wide range of HSM vendors

Encryption and decryption done on the database server

Simplifies key management in distributed environments(data guard, RAC)



52/154


SECUREFILE LOB Encryption

SECUREFILE LOB encryption

All SECUREFILE LOBs in an encrypted column areencrypted In-line (in table) and out-of-line (in tablespace) are both

encrypted

BFILEs are not encrypted Always salted for higher security



53/154


Core

PlatformSecurity





Oracle Database 11g


54/154

Oracle Database 11g

Security Manageability

Integrated with EM

Label Security

Virtual Private Database

Application Context

Enterprise SecurityManager

Transparent DataEncryption

11g Database Manageability


55/154


Memory Management AWR Baselines

Automatic SQL Tuning

Advisors

Upgrade Considerations

See:- Oracle Database 11g:

Manageability Overview


August 2007


56/154


Automatic Memory Management

Memory Management in 10g/11g


57/154

e o y a age e t 0g/ g

SGA_TARGET

Key shared memory components

tuned automatically Single parameter controls all

shared memory

10g Automatic Shared Memory

Management (ASMM)

10g Automatic PGA Memory

Management

PGA_AGGREGATE_TARGET

Various PGA sort areas tuned

automatically Single parameter controls all PGA

memory

MEMORY_TARGET

MEMORY_MAX_TARGET

Improves memory utilization

Eases memory management

11g Automatic Memory Management

Automatic Memory Management in 11g


58/154

y g g

Unifies system (SGA) and process(PGA) memory management

Dynamic parameters for all databasememory:

MEMORY_TARGET

MEMORY_MAX_TARGET

Automatically adapts to workloadchanges

Maximizes memory utilization Available on:

Linux

Windows

Solaris

HPUX

AIX

O/S Memory

O/S Memory

PGA

SGA

PGA

SGA

Memory Target Benefits


59/154

y g

Balances dynamic memory needs of SGA and PGA

Previously either or both could end up undersized

PGA can grow but not to the detriment of SGA performance Best-effort memory transfers

Tunable PGA can help relieve 4031 from SGA

Built-in safety nets:

Decides victim components

Paced memory donation avoids thrashing

Run-away workload cannot starve resources Component-specific settings still honored


60/154


AWR Baseline and Adaptive Thresholds

Performance Monitoring and Baselines


61/154

g

Performance Page shows certain metrics against 99th

percentile over selected baseline

Adaptive Thresholds over select system metrics

Enabled by System Moving Window baseline

AWR Compare Period Report

Use Baseline as one of the periods

Other Baselines Features


62/154

Baseline Templates enable future Baseline creation

Pre-specify known interesting times, e.g. Thanksgiving

System Moving Window Baseline

Always available

Defined by AWR retention or user specified Adaptive thresholds computed over this baseline

Creating Single AWR Baseline


63/154

Quick Configure Adaptive Thresholds


64/154

Adaptive Thresholds


65/154

Adaptive threshold valuesfollow baseline values

AWR Baseline


66/154


Automatic SQL Tuning

Manual SQL Tuning Challenges


67/154

Complex - requires expertise in several domains SQL optimization: adjust the execution plan

Access design: provide fast data access SQL design: use appropriate SQL constructs

Time consuming

Each SQL statement is unique Potentially large number of statements to tune

Never ending task SQL workload always evolving

Plan regressions can happen

SQL Tuning in Oracle Database 10g


68/154

Workload

SQL Tuning CandidatesSQL Tuning

Advisor

ADDM

AWR

one hour

Generate

Recommendations

DBA

InvokeAdvisor

Implement

DBA

Some meaningful automation,but the DBA is still required

Evaluate

Recommendations

DBA

SQL Tuning Automation in 11g


69/154

Fully automatic SQL Tuning task

Runs automatically under Autotask framework:

Maintenance Window, CPU resource controlled, on/off switch Identifies, ranks and tunes candidate SQL

Leverages SQL Tuning Advisor

Candidate SQL automatically chosen Excluded: parallel queries, DML/DDL, recursive, ad-hoc (infrequent)

Tests and (optionally) implements SQL profiles

Only implements significantly improved plans (3x) Time budgeted to avoid run-away plans

Automatic SQL Tuning in Oracle


70/154

Its Automatic!

ChooseCandidate

SQLone

week

Database 11g

Workload

SQL TuningCandidates

Test SQL ProfilesImplement

SQL Profiles

Generate

Recommendations

AWRDBA

View Reports /ControlProcess

Result Summary


71/154

Result Details


72/154

Result Recommendations


73/154


74/154


SQL Access Advisor

SQL Access Advisor: Partition Advice


75/154

Indexes Materializedviews

Materializedviews log

SQL AccessAdvisor

Hypothetical

SQL cache

FilterOptions

STS

CompleteWorkload

Partitionedobjects

Hashpartitions?

Intervalpartitions?

SQL Access Advisor Results


76/154

Access Advisor Recommendations


77/154


78/154

Editable implementation script


79/154

Oracle Database 11g R1


80/154

Fault Diagnostic Automation

Realistic Testing

and

Automatic Health

Checks

AutomaticDiagnosticWorkflow

IntelligentResolution

ProactivePatching

Goal: Reduce Time to Problem Resolution

Prevention Resolution

Diagnostic Solution Delivery

Automatic Diagnostic Workflow

Automatic


81/154

DBA

CriticalError

AutomaticDiagnostic

Repository

Knownbug?

DBA

EM Support Workbench:

Apply patch / Data Repair

Yes

DBA

No

Alert DBATargeted health checks

Assisted SR filling

Auto incident creation

First failure capture

EM Support Workbench:

Package incident info

Data Repair

1 2

3

4

Problems and Incidents


82/154

Problems are fundamental code or configurationissues that can cause execution failures

They exist until they are corrected, e.g by patch They are assigned a signature-based key value

An incident is a single occurrence of a problem

They happen at point(s) in time and thus have timestamps

They induce diagnostic actions like dumps and traces

They are managed by ADR and packaged by IPS

Automatic Diagnostic Repository


83/154

diag

rdbms

DB

Name

SID

ADR

Base

$ORACLE_HOME/log

DIAGNOSTIC_DEST

ADRHome

$ORACLE_BASE

ADRCIlog.xml

alert_SID.log

V$DIAG_INFO

BACKGROUND_DUMP_DEST

USER_DUMP_DEST

CORE_DUMP_DEST

alert cdump (others)hmincpkg incident

metadata

incdir_1 incdir_n

trace

Support Workbench

Important points about ADR


84/154

Location of traces and dumps has changed See V$DIAG_INFO for new locations

Symbolic links can provide work around for scriptcompatibility

Format of alert.log is changed

New XML format more parse-able Old style still available for backward compatibility

Space management

Incident flood control minimizes number of traces ADR auto-purging reduces disk footprint


85/154

EM Support Workbench Overview


86/154

Primary customer-facing interface into IPS and ADR

Task-oriented interface over problems and incidents View problem and incident details

Create and manage IPS packages

Run health checks or generate additional diagnostic data

Run advisors to help resolve problems

Linkages to Oracle Support Generate and send incident package files using OCM

Create and track Metalink service requests from problem

Close problem SR once resolved

EM Support Workbench


87/154


88/154

Package details and IPS in EM


89/154


90/154

Prevention: Health Monitoring


91/154

Proactive Health Checks

E.g. Dictionary Validation

Reactive Checks

Trigger on error via Intelligent dumping Rules

E.g. Checks blocks around corrupted block

All Checks can be Activated on Demand

Incident Meter for EM

Intelligent Repair Advisors


92/154

Data Recovery Advisor

Guided expert data recovery system using diagnostic data

and Health Check output

SQL Test Case Builder

Automatically retrieves exact environment information fromADR to build SQL test cases and replicate SQL issues

SQL Repair Advisor

Analyzes failing SQL statements to isolate bug May recommend SQL Patch as work around


93/154



94/154


Enterprise Manager Support


95/154

Enterprise Manager Support


96/154



B i R i


97/154

Business Requirement

Bug resolution

Test case required for fast bug resolution

Not always easy to provide a test case What information should be provided?

How much data is need?

Getting the test case to Oracle can be tricky

Solution

Oracle automatically creates a test case

Collects necessary information relating to a SQL incident

Collected data is packaged to be sent to Oracle

Collected data allows a developer to reproduce the problem


98/154

SQL Repair Advisor

SQL Repair Advisor

B i R i t


99/154

Business Requirement The most common types of SQL problems - exception, performance

regression etc., are hard to diagnose

A lot of time is spent trying to reproduce the problem If a workaround is found it has to be applied to entire system

Solution

Advisor

Investigates the incident locally

Automatically determines the root cause

Provides a workaround (SQL Patch) for just the effected SQL If not, sends necessary diagnostic information to Oracle

SQL Repair Advisor Flow

SQLstatement Statement

Generate


100/154

DBA

statement Execute Statementcrashes

incident in ADRautomatically

DBA run

SQL Repair Advisor

DBA gets

alerted

SQLRepairAdvisor

investigates

SQL patchgenerated

DBA accept

SQL patch

Statementexecutes

successfullyagain

SQL statementpatched

Execute

Trace files

DB Management Pack Enhancements


101/154

Diagnostic Pack

ADDM for RAC

AWR Baselines and Adaptive Thresholds Transportable AWR

Tuning Pack

Automatic SQL Tuning Advisor

Partition Advisor

Real-time SQL Monitoring

SQL Plan Management and Plan Evolution


102/154

Upgrade Considerations

New Background Processes in 11g

I RAC i hi i (A i C lfil M S i ) i

Description FoundProcess

Name


103/154

ASMMaintains disk membership in ASM disk groupsGMON

ALL.The process archives historical rows for tracked tables into flashback data archives and manages the

flashback archives.FBDA

ALL with anyregistered event

activity

(optional non-fatal)

The Event Monitor Coordinator coordinates the event management and notification activity in the

database which includes Streams Event Notifications, Continuous Query Notifications and FastApplication Notifications.

EMNC / e0xx

ALLPerforms diagnostic dumps and executes global oradebug commands.

DIAG

ALLResponsible for hang detection and deadlock resolution.

DIA0

ALLThe Resource Manager process is responsible for setting Resource Plans and other Resource Manager

related tasks.DBRM

RAC

In an RAC environment, this per-instance process (Atomic Controlfile to Memory Service) is an agentwhich contributes to ensuring a distributed SGA memory update is either globally committed on success

or globally aborted in the event of a failure.ACMS

RAC onlyProvides transparent support for XA global transactions in a RAC environment. The database

autotunes the number of these processes based on the workload of XA global transactions.GTX0-j


104/154

Oracle Database 11g Release 1Upgrade Paths


105/154

Direct upgrade to 11g is supported from 9.2.0.4 or higher,10.1.0.2 or higher, and 10.2.0.1 or higher.

If you are not at one of these versions you need to perform adouble-hop upgrade

For example:

7.3.4 -> 9.2.0.8 -> 11.1

8.1.7.4->9.2.0.8->11.1

Choose an Upgrade Method


106/154

Database Upgrade Assistant (DBUA)

Automated GUI tool that interactively steps the user throughthe upgrade process and configures the database to run withOracle Database 11g Release 1

Manual Upgrade

Use SQL*Plus to perform any necessary actions to prepare

for the upgrade, run the upgrade scripts and analyze theupgrade results

Export-Import

Use Data Pump or original Export/Import

CREATE TABLE AS SQL statement

Oracle Database 11g InstallationChanges


107/154

Addition of new products to the install

SQL Developer

Movement of APEX from companion CD to main CD Warehouse Builder (server-side pieces)

Oracle Configuration Management (OCM)

New Transparent Gateways

Removal of certain products and features from the installation: OEM Java Console

Oracle Data Mining Scoring Engine

Oracle Workflow

iSQL*Plus

Fine Grained Access Controlfor Utl_TCPand its cousins

ChallengeO SQ


108/154

Oracle Database provides packaged APIs for PL/SQLsubprograms to access machines (specified by host and port)using bare TCP/IP and other protocols built on it (SMTP and

HTTP) Utl_TCP, Utl_SMTP, Utl_HTTP

If you have Executeon the package, you can access ANYhost-port

Solution

an Access Control List (ACL) specifies a set of users and roles

you assign an ACL to a host and port range you may need to explicitly grant this access in 11.1


109/154

Log files changes

A t ti Di ti R it


110/154

Automatic Diagnostic Repository

$ORACLE_BASE/diag

alert.log

xml format

$ORACLE_BASE/diag/rdbms/orcl/orcl/alert/log.xml

adrci> show alert tail

$ORACLE_BASE/diag/rdbms//trace/alert_SID.log

Moving On

RBO


111/154

RBO

Still there, same behavior as 10.2, but even move

tools to ease the move SQL performance analyzer

SQL plan management

Moving On

These deprecated features / privileges are stillavailable for backward compatibility but the


112/154

Stored Outline SQL Plan Management

Ultrasearch Secure Enterprise Search

CTXPath Index XML Index

SYSDBA for ASM administrationSYSASM

No longer supported

Workflow BPEL

available for backward compatibility but therecommendation is to move on

Real Application Testing


113/154


Enterprise ManagerManagement PacksEnhancements

See:- Oracle Database 11g: Real Application

Testing & Manageability Overview


June 2007

What is Real Application Testing?

New database option available with EE only

I l d f


114/154

Includes two new features

Database Replay (DB Replay)

SQL Performance Analyzer (SPA)

SQL Tuning Set (STS) functionality is shared betweenReal Application Testing option and EM Tuning Pack

Advanced functionality requires additional packs Diagnostic Pack for DB Replay

Tuning Pack for SPA

License Requires licenses for both capture and replay system CPUs


Value


115/154

RemediateRemediate

TestTestTest

DeployDeploy

Value

Rapid technology adoption

Higher testing quality

Business Benefit

Lower cost

Lower risk

Solution for the Agile Business

Change


116/154

Real Application TestingDatabase Replay

The Need for Database Replay

Businesses want to adopt new technology that adds value

E i i d lid i i i i i d


117/154

Extensive testing and validation is expensive in time and cost

Despite expensive testing success rate low

Many issues go undetected System availability and performance negatively impacted

Cause of low success rate

Existing tools provide inadequate testing

Simulate synthetic workload instead of replaying actualproduction workload

Provide partial workflow coverage

Database Replay makes real-world testing possible

Database Replay

Replay actual production database workload in testenvironment

Identify analyze and fix potential instabilities before making


118/154

Identify, analyze and fix potential instabilities before makingchanges to production

Capture Workload in Production Capture full production workload with real load, timing &concurrency characteristics

Move the captured workload to test system

Replay Workload in Test

Make the desired changes in test system Replay workload with full production characteristics

Honor commit ordering

Analyze & Report

Errors Data divergence

Performance divergence

Analysis & Reporting

Database Replay: Supported Changes

Changes

ClientClient

Client


119/154

ChangesUnsupported

Changes Supported

Database Upgrades, Patches

Schema, Parameters

RAC nodes, Interconnect

OS Platforms, OS Upgrades

CPU, Memory

Storage

Etc.

Middle Tier

Storage

Recording ofExternal Client

Requests

80

Comparison of LoadRunner & DB ReplayTesting e-Business Suite


120/154

0

20

40

60

80

TimeTaken(Days)

Install & Setup Understand

Application

Usage

Identify Key

Transactions

Generate

Workload

Run Test

DB Replay

LoadRunner

2

205

4

2420

DB Replay: 2 weeks

LoadRunner: 30 weeks

Total Testing Time

500

Why DB Replay?


121/154

To:From:

Complete workflowsPartial workflows

Low riskHigh risk

AutomatedManual intensive

Production workloadsArtificial workloads

Days of developmentMonths of development

150

Days

10

Days

Database Replay Workflow

Production (10.2.0.4) Test (11.1)

Replay Driver

Clients


122/154

Capture Replay Analysis &Reporting

Process

StorageStorage

Mid-Tier

p y

Step 1: Workload Capture

Production System

All external client requestscaptured in binary files

ClientClientClient


123/154

File 1

File 2

File n

System background, internalactivity excluded

Minimal performanceoverhead for capture

For RAC, shared and local

file system supported

Specify interesting timeperiod for capture, e.g., peakworkload, month-end

processing, etc.

Can capture on 10.2.0.4 andreplay on 11g

File System

ClientClient

Client

Middle Tier

Storage

Capture Options

Workload can be filtered to customize what is captured Filter Types Inclusion Filters: Specifies which sessions should be captured


124/154

p p

Exclusion Filters: Specifies which sessions should NOT becaptured

Filter Attributes: Workload capture can be filtered using any of thefollowing session attributes

User

Program

Module

Action

Service

Session ID

Workload capture can be run on-demand or scheduled to run at

later time

Step 2: Process Workload Files

Test System Setup test system Test DB is at same point in timeas before production capture


125/154

File 1

File 2

File n

File 1

File 2

File n

Metadata

Capture Files Replay Files

Use RMAN to physically restoreproduction db from backup

Use Snapshot standby

Use imp/exp, Data Pump, etc.

Processing transforms captureddata into replayable format

Once processed, workload can bereplayed many times

For RAC copy all capture files tosingle location for processing

Step 3: Replay Workload

Replays workload preservingtiming, concurrency anddependencies of the capture

Test System


126/154

p psystem

Replay Driver is a specialclient program thatconsumes processedworkload and sends requeststo the replay system

Replay Driver consists of oneor more clients. Forworkloads with high

concurrency, it may benecessary to start multipleclients to drive workload

File 1

File 2

File n

Metadata

Replay Files

Replay Driver

Replay Options

Synchronized Replay (Default) Workload is replayed in full synchronized mode

Exact same concurrency and timing as production workload


127/154

Exact same concurrency and timing as production workload

Transaction commit order is honored

Ensures minimal data divergence Unsynchronized Replay

Workload can be replayed in unsynchronized mode

Useful for load/stress testing

High data divergence Three (3) parameters provided to control degree of synchronization

Think time synchronization

Connect (logon) time synchronization

Commit order synchronization


128/154

Analysis & Reporting

Comprehensive reports are provided for analysis purposes

There (3) types of divergences are reported


129/154

Data Divergence: Number of rows returned by each call arecompared and divergences reported

Error Divergence: For each call error divergence is reported

New: Error encountered during replay not seen during capture

Not Found: Error encountered during capture not seen duringreplay

Mutated: Different error produced in replay than during capture Performance Divergence

Capture and Replay Report: Provides high-level performanceinformation

ADDM Report: Provides in-depth performance analysis AWR, ASH Report: Facilitates comparative or skew analysis

Database Replay Summary Report


130/154

Performance Page Database Replay


131/154

Top Activity Page: Database Replay


132/154

Performance Page Database Replay


133/154


134/154

Current Restrictions

Database Replay does not support the followingfeatures in the current release

Di h l d i /


135/154

Direct path load, import/export

OCI based object navigation (ADTs) and REF binds Streams, non-PL/SQL based AQ

Distributed txns, remote describe/commit operations

Flashback

Shared Server

Best Practices

Capture

Provide adequate disk space for captured workload (binary files) Database restart (Optional): Recommended to minimize divergence

For RAC, use shared file system

T t S t S t


136/154

Test System Setup Ensure data in test is identical to production as of capture start time to

minimize data divergence during replay Use RMAN backup/restore or Snapshot Standby to setup test system

For performance analysis test system capacity should be similar toproduction

Reset system clock to same time as production if application logic involves

SYSDATE usage Process Workload

Processing workload has performance overhead and can possibly take along time

Process workload on test system instead of production

Replay Use Client Calibration Advisor to identify number of replay clients needed to

replay workload properly


137/154

Real Application TestingSQL Performance Analyzer (SPA)

The Need for SQL Performance Analyzer (SPA)

Businesses want systems that are performant andmeet SLAs

SQL performance regressions are #1 cause of poor


138/154

SQL performance regressions are #1 cause of poor

system performance Solution for proactively detecting allSQL regressionsresulting from changes not available

DBAs use ineffective and time-consuming manual

process to identify problems

SPA automates identification of all SQL performanceregressions resulting from changes

To:

Why SQL Performance Analyzer?

From:


139/154

Complete workload

Low risk

Automated analysis in minutes

Automated workload capture

Production workload

Partial workload

High risk

Months of manual analysis

Manual workload capture

Synthetic workload

Test impact of change on SQL query performance

Capture SQL workload in production including statistics & bindvariables

Re-execute SQL queries in test environment

SQL Performance Analyzer


140/154

ClientClient

Client

Capture SQL

q

Analyze performance changes improvements and regressions

Middle Tier

Storage

Oracle DB

Re-execute SQL Queries

Production Test

Use SQL TuningAdvisor to tuneregression

Enables identification of SQL performance regressions beforeend-users can be impacted

SPA h l ith h th t i t SQL ti l

SPA Benefits


141/154

SPA can help with any change that impacts SQL execution plan

DB upgrades Optimizer statistics refresh

New indexes, Materialized Views, Partitions, etc.

Automates SQL performance tracking of hundreds of thousands ofSQL statements impossible to do manually

Captures SQL workload with low overhead

Integrated with SQL Tuning Advisor and SQL Plan Baselines forregression remediation


142/154


143/154


144/154


145/154

Step 4: Execute SQL After Making Change

SQL Tuning Set

Fetch Next SQL Manually implement the planned change

Database upgrade, patches

Optimizer statistics refresh


146/154

Re-execute SQL after change

Gathers new SQL execution plansand statistics

SQL Performance Analyzer

BeforeChange

Execution Plan &

Statistics

Test Execute

Save Result

AfterChange

Completed

Optimizer statistics refresh

Schema changes Database parameter changes

Tuning actions, e.g., SQL Profile creation


147/154

SPA Report


148/154


149/154

Best Practices

Workload Capture Use incremental STS capture Peak or representative workloads can be more accurately captured

with this mechanism

T


150/154

Test system

Run SPA on test system instead of production as analysis can beresource intensive

Ensure test system has similar configuration and comparableoptimizer statistics as production

Performance comparison Use several different metrics, e.g., elapsed time, CPU time, etc., tocompare pre- and post-change performance for reliable results

Regression remediation Use SQL Tuning Advisor and SQL Plan Baselines to fix regressions


151/154


152/154

Q U E S T I O N SQ U E S T I O N S

A N S W E R SA N S W E R S

Thank you for your attention !


153/154


154/154

Documents

Oracle Db 11g Readiness Technical Uk Sep07 2