11g overview

<Insert Picture Here>

Oracle 11g IntroductionJames HardingSenior Sales [email protected]


Agenda

Oracle 11gR2 DBMS Advanced Compression Real Application Testing Transparent Data Encryption Data Masking Partitioning Advanced Security Edition Based Redefinition New Indexing Features Analytical & OLAP Features

“This presentation is for informational purposes only and may not be incorporated into a contract or agreement. ”

Customer Data Management ProblemsReal World Solutions

• Modern Enterprise Grids – Real Application Clusters– Management packs– TimesTen In-Memory Database

• Information Lifecycle Management– Partitioning– Advanced Compression

• Data Warehousing– Oracle Information Appliances – OLAP, Mining, Warehouse Builder

• Governance, Risk & Compliance– Security Options– Total Recall

• Reduce time, cost and risk of change– Real Application Testing


• Flashback Archive• Data Guard• Streams• Online Maintenance• Data Recovery Advisor

High Availability


Flashback RecoverySustained Innovation…

FlashbackQuery

FlashbackDrop

FlashbackDatabase

FlashbackData Archive


Error Investigation Using Flashback

• Flashback Query• Query all data at point in time

Tx 1

Tx 2

Tx 3

select * from Emp AS OF ‘2:00 P.M.’ where …

select * from Emp VERSIONS BETWEEN ‘2:00 PM’ and ‘3:00 PM’ where …

select * from FLASHBACK_TRANSACTION_QUERY where xid = ‘000200030000002D’;

Flashback Transaction Query– See all changes made by a transaction

Flashback Version Query– See all versions of a row between times– See transactions that changed the row


Flashback Drop

• Allows users to quickly recover objects after a SQL drop operation

• Provides self-service recovery for dropped objects• Eliminate the need for TSPITR• Dropped objects are placed in a Recycle Bin

• Objects remain in the recycle bin until you permanently drop them with the PURGE command or recover them with the Flashback Drop command.

• Objects will remain in the recycle bin until there is no room in the tablespace for new rows or updates to existing rows or until the tablespace needs to be extended

• Purged in the order they were dropped

Drop table emp;

Emp

Mistake made

Emp

Recycle bin

Flashback Table emp

before drop;

Mistake undone



Flashback Data Archive



• Long term history- years• Automatically stores all

changes to selected tables in Flashback Data Archive• Archive cannot be modified• Old data purged per retention

policy• View table contents as of any

time using Flashback SQL• Uses:

• Change Tracking• ILM• Long term history• Auditing• Compliance

ORDERS

User Tablespaces


ArchiveTables

Oracle Database

Changes

Total Recall

Select * from orders AS OF ‘Midnight 31-Dec-2004’



Data Guard


Unlocking the Value of Standby DBs

Standbyfor OnlineUpgrade,

Auto Failover

Standbyfor Testing,ReadablePhysical

Standbyfor DR

and Backup

Logical Standby

for RealtimeQuery


Physical Standby with Real-Time Query

Physical Standby Database

Primary Database

Real-time Queries

Continuous Redo Shipment and Apply

Concurrent Real-Time

Query

• Read-only queries on physical standby concurrent with redo apply• Supports RAC on primary / standby• Queries see transactionally consistent results

• Handles all data types, very fast, operationally simple• But not as flexible as logical standby

• Immediate appeal to the many users of physical standby• DR with real time query is unique in the industry – no idle resources

Now supports Incremental

backups!


Data Recovery Advisor


Time to Repair

Data Recovery AdvisorThe Motivation

• Oracle provides robust tools for data repair:

RMAN – physical media loss or corruptions

Flashback – logical errorsData Guard – physical or logical problems

• However, problem diagnosis and choosing the right solution can be error prone and time consuming

• Errors more likely during emergencies

Recovery

Investigation & Planning


Data Recovery AdvisorEnterprise Manager Support


• Real Application Clusters• Automatic Storage Management• Direct NFS• Optimizer Enhancements

Grid and OLTP


Real Application Clusters


Standard Oracle Architecture

Instance Database


Shared Disk Architecture

DatabaseInstance 1

DatabaseInstance 2

DatabaseInstance 3

Table ATable BTable C


RAC – Cache Fusion Protocol

• Locality Optimized Fusion Protocol (10.2)• Oracle detects when most segment accesses are coming from a single

instance• Optimizes access by that instance

• Reader Optimized Fusion Protocol• Highly read-intensive segments are automatically converted to a reader

optimized messaging protocol• Improved performance for read-intensive workloads

• improves any read from disk (not cache) whether short random reads or large table scans

• Throughput improved up to 70% for internal read-only benchmark• Long Query Optimized Fusion Protocol

• After all modified cache buffers at start of query are written to disk, no more need for RAC communication

• Direct reads for non-parallel table scans• Update Optimized Fusion Protocol

• Update block in parallel to readers releasing the block


Database Control 11gTiled Instance Charts


Inst 1

• Automatic Database Diagnostics Managers (ADDM) for Real Applications Cluster (RAC)

• RAC expert in a box• Identifies performance

problems for the entire RAC cluster database

• Database-wide analysis of:• Global cache interconnect

issues• Global resource contention,

e.g. IO bandwidth, hot blocks

• Globally high-load SQL• Skew in instance response

times• Runs proactively every hour

when taking AWR snapshots (default)AWR 1 AWR 2 AWR 3

Inst 2 Inst 3

Self-Diagnostic Engine

Database-Level ADDM11g

Instance-Level ADDM

ADDM for RAC



Automatic Storage Management



• The preferred and best storage manager for Oracle Databases

• Easier to manage than file systems• Performance of raw volumes• Built-in to Oracle database • Shared storage pool for all databases

• Free, and widely adopted• >65% of 10g RAC deployments on ASM• >25% of 10g customers already using ASM• Many VLDB over 10TB

ASM DiskASM Disk

ASM DiskASM Disk

ASM Disk



• Spreads database files evenly across storage arrays

• Storage arrays can be easily added or remove• transparent data redistribution

• Data mirrored across arrays• Tolerates failure of disks or arrays

New ASM features in Oracle 11g:• ASM Fast Disk Resync• ASM Preferred Mirror Read• ASM Rolling Upgrade• Larger extent, allocation unit sizes

ASM DiskASM Disk

ASM DiskASM Disk

ASM Disk


New Optimizer Features In 11g


Inside the Oracle Database 11g Optimizer Removing the black magic

• Plans change unexpectedly especially during upgrades

• Cardinality estimate is wrong so plan goes wrong

• Gathering Optimizer Statistics takes too long

• Bind peeking doesn’t work when there is a data skew


Inside the Oracle Database 11g Optimizer Removing the black magic

• Plans change unexpectedly especially during upgrades• Guaranteed plan stability and controlled plan evolution• Controlled statistics publication

• Cardinality estimate is wrong so plan goes wrong• Collect appropriate statistics• Eliminate wrong cardinality estimates

• Gathering Optimizer Statistics takes too long• Faster statistics gathering • Improved statistics quality

• Bind peeking doesn’t work when there is a data skew• Enhanced plan sharing with binds



SQL Plan ManagementGuaranteed plan stability and

controlled plan evolution


Without SQL Plan Management

• Something changes in the environment• Statistics are re-gathered, DB upgrade or parameter change

• Changes result in new plan• New plan implemented regardless of resulting performance

NL

NL

GB

Parse

HJ

HJ

GB

Parse

• SQL statement is parsed for the first time and a plan is generated• Does plan gives good performance? Plan is “verified by execution”

Execute Plan Acceptable

Execute Plan NOT Acceptable


With SQL Plan Management• Repeatable SQL populates statement log, plan history and creates a

baseline

Parse

HJ

HJ

GB

Statement log

Plan history

HJ

HJ

GB

Plan baseline

Execute Plan Acceptable


Statement log

Plan history

HJ

HJ

GB

Plan baseline

With SQL Plan Management• Something changes in the environment• SQL statement is parsed again and a new plan is generated

NL

NL

GB

Parse

GB

NL

NL

• New plan is not the same as the baseline – new plan is not executed but marked for verification


With SQL Plan Management• Something changes in the environment• SQL statement is parsed again and a new plan is generated• New plan is not the same as the baseline – new plan is not executed

but marked for verification • Execute the known plan baseline “performance guaranteed by history”

Execute Plan AcceptableParse

HJ

HJ

GB

Statement log

Plan history

HJ

HJ

GB

Plan baselineGB

NL

NL

• Partitioning• Information

Lifecycle Management

• Compression

VLDB


Partitioning in Oracle Database 11g

37

Oracle Partitioning10 years of innovation

Core functionalityOracle8 Range partitions, global range index

Oracle8i Hash and composite range-hash partitioning

Oracle9i List partitioning

Oracle9i R2 Composite range-list partitioningOracle 10g Global hash indexes

Oracle 10g R2 1M partitions per tablePartitioning by referenceVirtual column partitioningAutomatic interval partitioningNew composite partitioning: range-range, list-range, list-list, list-hash

38

Range List HashRange 9i 8i

List

• Partition on virtual (computed) columns • New composite partitioning

Range List HashRange 11g 9i 8i

List 11g 11g 11g

JAN FEB

>5000

1000-

5000

ORDERS

RANGE-RANGEOrder Date by

Order Value

USA EUROPE

>5000

1000-

5000

ORDERS

LIST-RANGERegion by

Order Value

USA EUROPE

Gold

SilverORDERS

LIST-LISTRegion by

Customer Type

Enhanced Partitioning

39

The Concept of PartitioningMaintain Consistent Performance as Database Grows

SALES SALES

Jan Feb

SALES

Jan Feb

Europe

USA

Large Table•Difficult to Manage

Partition•Divide and Conquer

•Easier to Manage

•Improve Performance

Composite Partition•Higher Performance

•Match to business needs

40

Partition for PerformancePartition Pruning

What was the total sales amount for May 20 and May 21 2010?

Select sum(sales_amount)

From SALES

Where sales_date between

to_date(‘05/20/2010’,’MM/DD/YYYY’)

And

to_date(‘05/22/2010’,’MM/DD/YYYY’);

5/20

5/21

5/22

5/19

Sales Table

• Performs operations only on relevant partitions

• Dramatically reduces amount of data retrieved from disk

• Improves query performance and optimizes resource utilization

41

Partition to Manage Data Growth Compress Data and Lower Storage Costs

• Distribute partitions across multiple compression tiers

• Free up storage space and execute queries faster

• No changes to existing applications

Active Data

3x OLTP Compression

Read Only Data

10-15x DW Compression

Archive Data

15-50x Archive

Compression

42Content is property of Oracle Corp. and is provided for Data Warehousing student education

Partitioning • Partition level management

• On-line addition and removal of partitions• Data management operations (loading, index builds)• Range, hash, composite range-hash, list, composite range-

list

• Improved availability• Localized disk failures, backup and recoveryorder table

Drop

Nov 2006

Add

Jan 2006 Feb 2006 Mar 2006 Apr 2006

Local Index

Other data is not affected


Virtual Column based Partitioning

Virtual Columns

• ANSI syntax• Look just like regular columns from SQL perspective• Support for partitioning, indexes, constraints,

statistics, histograms• Used by expression evaluation when applicable

Oracle Confidential

Create table t1 ( first_name varchar2, last_name varchar2, full_name as (first_name || ‘ ‘ || last_name) virtual)

Virtual Columns - Example

• Base table with all attributes ...

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ...

12500 Adams12507 Blake1266612875 Smith

King


12500 Adams12507 12Blake12666 1212875 12Smith

King

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))

12

• Base table with all attributes ...• ... is extended with the virtual (derived) column


12500 Adams12507 12Blake12666 1212875 12Smith

King

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))partition by list (acc_branch) ...

12

• Base table with all attributes ...• ... is extended with the virtual (derived) column• ... and the virtual column is used as partitioning key

32320 Jones32407 32Clark32758 3232980 32Phillips

32

... Hurd


REF Partitioning

Before REF Partitioning

Table ORDERS

Jan 2006

... ...

Feb 2006

Table LINEITEMS

Jan 2006

... ...

Feb 2006

• Redundant storage of order_date• Redundant maintenance

• RANGE(order_date)• Primary key order_id

• RANGE(order_date)• Foreign key order_id

REF Partitioning

Table ORDERS

Jan 2006

... ...

Feb 2006

Table LINEITEMS

Jan 2006

... ...

Feb 2006

• RANGE(order_date)• Primary key order_id

• RANGE(order_date)• Foreign key order_id

PARTITION BY REFERENCE• Partitioning key inherited

through PK-FK relationship


Advanced Compression Option

• Oracle 9i compresses data only during bulk load; useful for DW and ILM

• Oracle 11g compresses w/ inserts, updates• Typical compression ratio of 2x to 3x• Database directly reads compressed data

eliminating decompression overhead• Strategy: compress db’s 10 largest tables

• Shrink table data by 50%, increase CPU by 5%• Savings cascade to all db copies: test, dev,

standby, mirrors, archiving, backup, etc.

Data Compressionfor All Applications

Content is property of Oracle Corp. and is provided for Data Warehousing student education

Advanced Compression OptionNew in Oracle Database 11g

• Compress Large Application Tables • Transaction processing, data warehousing

• Compress All Data Types• Structured and unstructured data types

• Typical Compression of 2-3 X• Cascade storage savings throughout data center

Content is property of Oracle Corp. and is provided for Data Warehousing student education

Advanced CompressionReduces storage requirements across all tiers

5% Active 35% Less Active 60% Historical

$16,600 $22,600 $19,400

Lets use compression factor of 3

$49,800 $67,700 $58,000

OLTP Table Compression

Overhead

Free Space

Uncompressed

Compressed

Inserts are uncompresse

d

Block usage reaches PCTFREE –

triggers Compression

Inserts are again

uncompressed

Block usage reaches PCTFREE –

triggers Compression

• Adaptable, continuous compression• Compression automatically triggered when block usage

reaches PCTFREE• Compression eliminates holes created due to deletions

and maximizes contiguous free space in block

Security

• Access Control• Data Protection• Monitoring• User Management

Oracle Database Vault Compliance and Insider Threats

• Controls on privileged users• Restrict DBA access to application

data• Provide Separation of Duty• Security for database and

information consolidation• Enforce data access security

policies• Control who, when, where and how

is data accessed• Make decision based on IP

address, time, auth…• Back Ported to Oracle9i R2• Validated with PeopleSoft• E-Biz & other Apps validation

underway, including 3rd party

Reports

Realms

Multi-FactorAuthorization

Separationof Duty

CommandRules

Oracle Audit Vault OverviewTrust-but-Verify

• Collect and Consolidate Audit Data• Oracle 9i Release 2 and higher

• Simplify Compliance Reporting• Built-in reports• Custom reports

• Detect and Prevent Insider Threats• Alert suspicious activity

• Scale and Security• Robust Oracle Database technology• Database Vault, Advanced Security• Partitioning

• Lower IT Costs with Audit Policies• Centrally manage/provision audit settings

10gR210gR1

Oracle 9iR2(Future)

Other Sources,Databases

Monitor Policies

Reports Security

Database 11gCore Database Security Enhancements

• Secure Configuration• Continuation of Secure By Default initiative started in Oracle9i• Password management settings• Audit sensitive administrative operations

• Stronger password verifier• Case sensitive passwords• Backward compatibility mode

• Expanded Kerberos support• Support principal names up to 2000 characters in length• Cross realm support

Fine Grained Access Controlfor Utl_TCP and its cousins

Challenge• Oracle Database provides packaged APIs for PL/SQL

subprograms to access machines (specified by host and port) using bare TCP/IP and other protocols built on it (SMTP and HTTP)• Utl_TCP, Utl_SMTP, Utl_HTTP…• If you have Execute on the package, you can access ANY

host-port

Solution• an Access Control List (ACL) specifies a set of users and roles• you assign an ACL to a host and port range• you may need to explicitly grant this access in 11.1


Data Masking

What is data masking?

What• The act of anonymizing customer,

financial, or company confidential data to create new, legible data which retains the data's properties, such as its width, type, and format.

Why• To protect confidential data in test

environments when the data is used by developers or offshore vendors

• When customer data is shared with 3rd parties without revealing personally identifiable information

LAST_NAME SSN SALARYAGUILAR 203-33-3234 40,000

BENSON 323-22-2943 60,000D’SOUZA 989-22-2403 80,000FIORANO 093-44-3823 45,000

LAST_NAME SSN SALARYANSKEKSL 111—23-1111 40,000

BKJHHEIEDK 111-34-1345 60,000KDDEHLHESA 111-97-2749 80,000FPENZXIEK 111-49-3849 45,000

Major features• Automatic database referential

integrity when masking primary keys• Implicit – database enforced• Explicit – application enforced

• Data mask format library• View sample data before masking• Application masking templates• Define once; execute multiple times

Enterprise ManagerData Masking Pack

Production Staging

Mask Test

TestCloneClone

64

• Real Application Testing• Database Management

Management and Change

Oracle Database 11gReal Application Testing

66

Database Replay

67

The Need for Database Replay

• Businesses want to adopt new technology that adds value• Extensive testing and validation is expensive in time and cost• Despite expensive testing success rate low

• Many issues go undetected • System availability and performance negatively impacted

• Cause of low success rate• Existing tools provide inadequate testing

• Simulate synthetic workload instead of replaying actual production workload

• Provide partial workflow coverage

Database Replay makes real-world testing possible

68

Database Replay

• Replay actual production database workload in test environment

• Identify, analyze and fix potential instabilities before making changes to production

• Capture Workload in Production• Capture full production workload with real load, timing &

concurrency characteristics• Move the captured workload to test system

• Replay Workload in Test• Make the desired changes in test system• Replay workload with full production characteristics• Honor commit ordering

• Analyze & Report• Errors• Data divergence • Performance divergence

Analysis & Reporting

69

Database Replay: Supported Changes

Changes Unsupporte

d

Changes Supported•Database Upgrades, Patches

•Schema, Parameters•RAC nodes, Interconnect

•OS Platforms, OS Upgrades•CPU, Memory

•Storage•Etc.

ClientClient

…Client

Middle Tier

Storage

Recording of External Client

Requests

70

…

…

Database Replay WorkflowProduction (9.2.0.8, 10.2.0.3+) Test (11.1)

Capture Replay Analysis & Reporting Process

Storage Storage

Mid-Tier

Replay DriverClients

71

Database Replay Summary Report

72

Performance Page – Database Replay

73

Top Activity Page: Database Replay

74

SQL Performance Analyzer (SPA)

75

The Need for SQL Performance Analyzer (SPA)

• Businesses want systems that are performant and meet SLA’s

• SQL performance regressions are #1 cause of poor system performance

• Solution for proactively detecting all SQL regressions resulting from changes not available

• DBA’s use ineffective and time-consuming manual process to identify problems

SPA automates identification of all SQL performance regressions resulting from changes

76

……

ClientClient

…Client

Capture SQL

• Test impact of change on SQL query performance• Capture SQL workload in production including statistics & bind

variables• Re-execute SQL queries in test environment• Analyze performance changes – improvements and regressions

Middle Tier

Storage

Oracle DB

Re-execute SQL Queries

Production Test

Use SQL Tuning Advisor to tune regression

SQL Performance Analyzer

77

…

SQL Performance Analyzer Workflow

Storage

Production (9iR2+)

Test (10.2+)

Capture SQL

Transport SQL

Execute SQL Pre-change

Execute SQL Post-change

Compare Perf

Storage

Mid-Tier

Clients

78

Real Application Testing for Prior Releases

Helps Smooth Transition to Newer Releases

• Database Replay: Capture on older release; Replay on 11.1 and above• SQL Performance Analyzer: Execute tests on 10.2 and above

† For more details: Note: 560977.1: Real Application Testing for Earlier Releases

Feature Upgrade From Upgrade To

Database Replay

10g R2 11g

9i R2 11g

SQL Performance Analyzer10g R2 10g R2 or 11g

10g R1 10g R2 or 11g9i R2 10g R2 or 11g

79

SPA Report

Oracle Database 11gDatabase Manageability

81

Manageability Evolution

Stor

age

Bac

kup

Mem

ory

App

s/SQ

L

Sche

ma

RA

C

Rec

over

y

Rep

licat

ion

Auto-TuningTuning

Advisory

InstrumentationLow Impact

Integrated

Adaptive

82

• Identifies cluster-wide critical performance problems

• Runs automatically when taking AWR snapshots• Snapshots are synchronized across cluster

• Performs database-wide analysis of:• Global resources, e.g. IO, global locks• High-load SQL, hot blocks• Global cache interconnect traffic• Network latency issues• Skew in instance response times

• Used by DBAs to analyze cluster performance

ADDM for RAC

83

Enhancements to ADDM Findings• Directives can instruct ADDM to suppress findings and reduce noise

from known un-actionable findings

84

Automatic SQL Tuning

• Captures high-load SQL

• Tunes SQL by creating SQL profiles

• Optionally implements greatly improved SQL plans

• Reports analysis

• Runs runs in maintenance window

Nightly

Well-tunedSQL

SQL Workload

PackagedApps

Custom Apps

Automatic SQL Tuning

SQLProfiles

SQLAnalysis

Report

Manually implement

85

SQL Access Advisor: Partition Advice

Indexes Materializedviews

Materializedviews log

SQL Access Advisor

Hypothetical

SQL cache

Filter Options

STS

Complete Workload

Partitionedobjects

Hash partitions?

Interval partitions?

86

Automatic Memory Management in 11g

• Unifies system (SGA) and process (PGA) memory management

• Single dynamic parameter for all database memory – MEMORY_TARGET

• Automatically adapts to workload changes

• Maximizes memory utilization• Available on:

• Linux• Windows• Solaris • HPUX• AIX

O/S MemoryO/S Memory

PGA

SGA

PGA

SGA

87

Fault Diagnostic Automation

88

Automatic Diagnostic Repository

diag

rdbms

DBName

SID

ADRBase

$ORACLE_HOME/log

DIAGNOSTIC_DEST

ADRHome

$ORACLE_BASE

ADRCIlog.xml alert_SID.log

V$DIAG_INFO

BACKGROUND_DUMP_DEST

USER_DUMP_DESTCORE_DUMP_DEST

alert cdump (others)hmincpkg incident

metadata

incdir_1 incdir_n…

trace

Support Workbench

89

Online Patching of One-off Patches

• Patch a running Oracle instance with no downtime• Many one-off patches can be online patched

• Subset of RAC online upgradeable patches• Great for diagnostic patches

• Enable, disable and de-install one-off patches with no downtime• Integrated with OPatch and inventory

• Initially available on Linux and Solaris• Long term goal is online patching of Critical

Patch Updates (CPUs).


SecureFiles

Oracle SecureFilesConsolidated Secure Management of Data

• SecureFiles is a new 11g feature designed to break the performance barrier keeping file data out of databases

• Next-generation LOBs - faster, and with more capabilities• transparent deduplication, compression and encryption• leverage the security, reliability, and scalability of database• superset of LOB interfaces allows easy migration from LOBs

• Enables consolidation of file data with associated relational data• single security model• single view of data• single management of data• scalable to any level using SMP scale-up, or grid scale-

out

Designed from Scratch

• SecureFiles is a major rearchitecture of how the database handles unstructured (file) data• not an incremental improvement to LOBs

• Entirely new:• disk format• network protocol• versioning and sharing mechanisms• caching and locking• redo and undo algorithms• space and memory management• cluster consistency algorithms


11g Statistics & SQL Analytics

11g Statistics & SQL Analytics• Ranking functions

• rank, dense_rank, cume_dist, percent_rank, ntile

• Window Aggregate functions (moving and cumulative)

• Avg, sum, min, max, count, variance, stddev, first_value, last_value

• LAG/LEAD functions• Direct inter-row reference using offsets

• Reporting Aggregate functions• Sum, avg, min, max, variance, stddev, count,

ratio_to_report• Statistical Aggregates

• Correlation, linear regression family, covariance

• Linear regression• Fitting of an ordinary-least-squares

regression line to a set of number pairs. • Frequently combined with the COVAR_POP,

COVAR_SAMP, and CORR functions

Descriptive Statistics• DBMS_STAT_FUNCS: summarizes numerical

columns of a table and returns count, min, max, range, mean, median, stats_mode, variance, standard deviation, quantile values, +/- n sigma values, top/bottom 5 values

• Correlations• Pearson’s correlation coefficients, Spearman's

and Kendall's (both nonparametric). • Cross Tabs

• Enhanced with % statistics: chi squared, phi coefficient, Cramer's V, contingency coefficient, Cohen's kappa

• Hypothesis Testing• Student t-test , F-test, Binomial test, Wilcoxon

Signed Ranks test, Chi-square, Mann Whitney test, Kolmogorov-Smirnov test, One-way ANOVA

• Distribution Fitting• Kolmogorov-Smirnov Test, Anderson-Darling

Test, Chi-Squared Test, Normal, Uniform, Weibull, Exponential

Note: Statistics and SQL Analytics are included in Oracle Database Standard Edition

Statistics

Split Lot A/B Offer testing• Offer “A” to one population and “B” to another

• Over time period “t” calculate median purchase amounts of customers receiving offer A & B

• Perform t-test to compare• If statistically significantly better results achieved from one offer over another, offer everyone higher performing offer

Independent Samples T-Test (Pooled Variances)

• Query compares the mean of AMOUNT_SOLD between MEN and WOMEN within CUST_INCOME_LEVEL ranges

SELECT substr(cust_income_level,1,22) income_level,avg(decode(cust_gender,'M',amount_sold,null)) sold_to_men,avg(decode(cust_gender,'F',amount_sold,null)) sold_to_women,stats_t_test_indep(cust_gender, amount_sold, 'STATISTIC','F') t_observed,stats_t_test_indep(cust_gender, amount_sold) two_sided_p_value

FROM sh.customers c, sh.sales sWHERE c.cust_id=s.cust_idGROUP BY rollup(cust_income_level)ORDER BY 1;

SQL Worksheet


Oracle OLAP 11gOptimizing BI Solutions with Oracle OLAP


Presentation Agenda

• Oracle OLAP Overview• Enhancing BI Solutions Transparently• Delivering Rich Analytics Easily

Oracle Database Strategy for DW Embedded Analytics

Data Mining

OLAP Statistics

SQL Analytics

• Bring the analytics to the data• Leverage core database infrastructure

Oracle OLAPLeveraging Core Database Infrastructure

• Single RDBMS-MDDS process• Single data storage• Single security model• Single administration facility• Grid-enabled• Accessible by any SQL-based tool• Connects to all related Oracle data

Oracle Database 11g Data Warehousing

Warehouse Builder

OLAP

Data Mining

Oracle OLAP Goals

• Improve the delivery of information rich queries by SQL-based business intelligence tools and applications• Fast query performance• Simplified access to analytic calculations • Fast incremental update• Leverage existing Oracle Database expertise

Materialized ViewsAutomatic Query Rewrite

• Most DW/BI customers use Materialized Views (MV) today to improve summary query performance

• Define appropriate summaries based on query patterns

• Each summary is typically defined at a particular grain

• Month, State• Qtr, State, Item• Month, Continent, Class• etc.

• The SQL Optimizer automatically rewrites queries to access MV’s whenever possible

SALES_YCyear_id

continent_idquantityrevenue

Year, Continent

SALES_MSmonthstate

quantiyrevenue

Month, Stateselect month, district, sum(revenue)from sales, time, cust

group by month, district

rewrite

SALESday_idprod_idcust_idchan_idquantity

pricerevenue

Materialized ViewsChallenges in Ad Hoc Query Environments

• Creating MVs to support ad hoc query patterns is challenging

• Users expect excellent query response time across any summary

• Potentially many MVs to manage• Practical limitations on size and

manageability constrain the number of materialized views

SALES_MCCmonth_id

category_idcity_idquantiyrevenue

Month, City, Category

SALES_YCCyear_id

category_idcity_idquantiyrevenue

Year, City, Category

SALES_YCCyear_id

category_idcontinent_id

quantiyrevenue

Year, Continent, Category

SALES_QSIqtr_id

item_idstate_idquantiyrevenue

Qtr, State, Item

SALES_XXXXXX_idXXX_idXXX_id

expense_amountpotential_fraud_cost

Cust, Time, Prod, Chan Lvls

SALES_XXX

XXX_idXXX_idXXX_id


SALES_XXXXXX_idXXX_idXXX_id


SALES_XXXXXX_idXXX_idXXX_idquantiyrevenue

SALES_YCTyear_idtype_id

continent_idquantiyrevenue

Year, District

SALESday_idprod_idcust_idchan_idquantityrevenue

SALES_MSmonthstate

quantiyrevenue

Month, State

SALES_YCyear_id

continent_idquantityrevenue

Year, Continent

Cube-based Materialized ViewsBreakthrough Manageability & Performance

SALESday_idprod_idcust_idchan_idquantity

pricerevenue

TIMEday_idmonthquarter

year

CUSTOMERcust_id

citystate

country

PRODUCTitem_id

subcategorycategory

type

rewrite

• A single cube provides the equivalent of thousands of

summary combinations• The 11g SQL Query

Optimizer treats OLAP cubes as MV’s and rewrites queries

to access cubes transparently

• Cube refreshed using standard MV procedures

CHANNELchan_id

class

SALESCUBErefresh

Cost Based AggregationPinpoint Summary Management

• Improves aggregation speed and storage consumption by pre-computing cells that are most expense to calculate

• Easy to administer• Simplifies SQL queries by

presenting data as fully calculated

NY25,000

customers

Los Angeles35

customers

Precomputed

Computed when queried


DemonstrationTransparently Improving Performance of BI Solutions

Easy AnalyticsFast Access to Information Rich Results

• Time-series calculations• Calculated Members• Financial Models• Forecasting

• Basic• Expert system

• Allocations• Regressions• Custom functions• …and many more

Snapshot of some functions

Easy AnalyticsOptimized Data Access Method

• Data stored in dense arrays• Offset addressing – no joins

• More powerful analysis• Better performance

Time

Category

Hotel

ExpensesLunch

Food

Q1 Q2 Q3SF

WestNortheast

Market

How do Expenses compare this Quarter versus Last Quarter

What is an Item’s Expense contribution to its Category?

BNP ParibasAdvanced Time-Series Analyses in Real-Time

• Large European financial institution

• Used by traders to help decrease susceptibility to market volatility

• Replacing FAME Time Series Database

• Forecasting, Analysis and Modeling Environment

• Three billion stored facts on RAC• Data updated every 2 seconds –

processing approximately 1m records daily

• SQL-based custom application used by 1500 concurrent users

Cube Represented as Star ModelSimplifies Access to Analytic Calculations

• Cube represented as a star schema

• Single cube view presents data as completely

calculated• Analytic calculations

presented as columns• Includes all summaries

• Automatically managed by OLAP

SALES_CUBEVIEWday_idprod_idcust_idchan_id

salesprofit

profit_yragoprofit_share_parent TIME_VIEW

day_idquartermonthyear

CUSTOMER_VIEWcust_id

citystate

region

PRODUCT_VIEWprod_id

subcategorycategory

group

CHANNEL_VIEWchan_id

classtotal

SALESCUBE

The Gallup OrganizationHealthcare Group

• Gallup asks over 1 billion questions annually• Gallup Healthcare Group

• Conduct surveys measuring quality of care and patient loyalty• Originally developed a reporting infrastructure that delivered static

reports to hospitals across the US• Enhanced the interactivity and analytic content of solution

• Support over 1000 concurrent users• Response time less than 2 seconds per query

• Reduced cost and complexity• Leveraged Oracle Database investment• Integrated OLAP into existing infrastructure (security, navigation,

XML/XSL application underpinnings)• Lowered application development costs• Reduced complexity for users

Empowering Any SQL-Based Tool Leveraging the OLAP Calculation Engine

SELECT cu.long_description customer, f.profit_rank_cust_sh_parent, f.profit_share_cust_sh_parent, f.profit_rank_cust_sh_level,

f.profit,f.gross_margin

FROM time_calendar_view t, product_primary_view p,

customer_shipments_view cu, channel_primary_view ch,

units_cube_view f

WHERE t.level_name = 'CALENDAR_YEAR‘ AND t.calendar_year = 'CY2006‘

AND p.dim_key = 'TOTAL‘ AND cu.parent = 'TOTAL‘ AND ch.dim_key = 'TOTAL‘ AND t.dim_key = f.TIME

AND p.dim_key = f.product AND cu.dim_key = f.customer AND ch.dim_key = f.channel;

Application Express on Oracle OLAP

Oracle OLAP 11g Summary

• Improve the delivery of information rich queries by SQL-based business intelligence tools and applications• Fast query performance• Simplified access to analytic calculations • Fast incremental update• Centrally managed by the Oracle Database


Other observations

Oracle Database 11g Release 1 Upgrade Paths

• Direct upgrade to 11g is supported from 9.2.0.4 or higher, 10.1.0.2 or higher, and 10.2.0.1 or higher.

• If you are not at one of these versions you need to perform a “double-hop” upgrade

• For example:• 7.3.4 -> 9.2.0.8 -> 11.1• 8.1.7.4->9.2.0.8->11.1

Oracle Database 11g Installation Changes• Addition of new products to the install

• SQL Developer• Movement of APEX from companion CD to main CD• Warehouse Builder (server-side pieces)• Oracle Configuration Management (OCM)• New Transparent Gateways

• Removal of certain products and features from the installation:• OEM Java Console• Oracle Data Mining Scoring Engine• Oracle Workflow• iSQL*Plus


Possible Upgrade Gotchas

Case Sensitive Password

• By default:• Default password profile is enabled• Account is locked after 10 failed login attempts

• In upgrade:• Passwords are case insensitive until changed• Passwords become case sensitive by ALTER USER

• On creation:• Passwords are case sensitive

• Review:• Scripts• Database links with stored passwords• Consider backward compatibility parameter, SEC_CASE_SENSITIVE_LOGON

Log files changes

Automatic Diagnostic Repository• $ORACLE_BASE/diag

alert.log• xml format• $ORACLE_BASE/diag/rdbms/orcl/orcl/alert/log.xml• adrci> show alert -tail

AQ&


The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Education

11g overview