Upload
others
View
45
Download
0
Embed Size (px)
Citation preview
Oracle Database 10gExtensibility Framework:Building, Deploying, and Managing Data Cartridges
Geoff LeePrincipal Product Manager,Oracle [email protected]
Agenda
OverviewOracle Database 10g Extensibility FrameworkDemoSummaryQ & A
Specialty Data in Life SciencesGenomic Proteomic Chemical
Virtual ScreeningMicroarrays
Managing All Your Data
DynamicWeb PagesDynamic
Web Pages E-MailE-Mail
StructuredData
StructuredData
Consistent, universal accessEasy to search and find any contentEasy to develop applications Easy to deploy with enterpriseQuality of ServiceEasy to manage
LegacyData
LegacyDataDocumentsDocuments
VideoVideoVideo LocationsLocationsTextText
StaticWeb Pages
StaticWeb Pages
AudioAudio ImagesImagesImages
Oracle10gOracle10g
Oracle Extensible Platform
Optimizer
Query Engine
Index EngineType Manager
Extensibility
Text
TextSpatialUtilities Tools
Image
Image
Text
Spatial
Image
Data CartridgesSpatIal
Specialty Data Cartridges
Oracle8iServer
Service Interfaces
DataCartridge
Extensibility Interfaces
TypeSystem
QueryProcessing Data
IndexingServer
Execution . . .
Database Extensibility Services
Oracle10gServer
Oracle8/8i
Evolution of the Extensibility Framework
Oracle9i/9i R2
Range PartitionedLocal Domain Indexes
ODCIEnv TypeAlter IndextypeDBMS_ODCI PackagePipelined Parallel
Table FunctionsUser-Defined AggregatesGeneric SQL TypesMulti-Threaded Extproc
Parallel Domain Indexe Scan DBMS_STATS integration
Array InsertsAlter OperatorDebug Info (29891)Trans. Tablespace
for Domain IndexesParallel Index Create/
Index Rebuild
Extensible Typesystem
Extensible ServerExecution Env.
Extensible IndexingExtensible Optimizer
Agenda
OverviewOracle Database 10g Extensibility Framework
– Building a data cartridge– Deploying a data cartridge– Managing data cartridges
DemoSummaryQ & A
Building Data Cartridges
Designing the schema (data types, LOBs, etc.)User defined types, operators, indexes, and optimizer
– Using the right languages (pl/sql, Java, C/C++)Boosting performance
– Array insert– Parallel pipelined table functions– Parallel (inter-partition) local domain indexes
Create, alter rebuild, queryUser defined aggregatesDebugging trace events
Key Steps to Building a Data Cartridge
1. User-Defined Types2. User-Defined Operators3. User-Defined Indexes4. Extensible Optimizer
User-Defined Types (Step 1 of 4)
Type System– Object Types (inheritance, attributes, methods, etc.)– Methods (dynamic method dispatch, PL/SQL, Java, or C/C++) – Object References (Object ID, dot notation)– Collection Types – LOBs
Physical Model– Column objects and row objects– Object views and object tables
Type evolution, replication, import/export, SQL*Loader, etc.Versatile PL/SQL, Java, C/C++, OLE DB and XML accessSQL 2003 standard
LOBs• LOBs help capture unstructured data• Three types of LOBS supported
• Binary LOB (BLOB)• Character LOB (CLOB, NCLOB)• Binary File (BFILE)
• LOBs replace LONG,LONG RAW• Multiple LOBs per table or UDT• Piece-wise access of LOB data• Streaming Interface• LONG to LOB APILOBs can be stored in a
separate tablespace or in operating system files
LOBs can be stored in a separate tablespace or in operating system files
Table
BFILE
BLOB/CLOB
LOB Performance Improvements
5x performance gain for accessing inline (< 4KB) LOBsTemporary LOBs use reference counting to provide orders of magnitude performance gain
– Reference on Read– Copy on Write
Temporary LOBs use aggressive garbage collection at row level
Native Floating-PointData Types
Two new numeric data types BINARY_FLOAT, BINARY_DOUBLE
– IEEE 754 Standard for binary floating point arithmetic
– Part of numerous other standards (e.g, Java, XML Schema) and hardware platforms
– Prevalent in Business Intelligence, Life Sciences, Engineering/Scientific Computation, etc.
User-defined Operators(Step 2 of 4)• Domain specific operation• Evaluated using functions or indexes
CREATE FUNCTION SssMatchBlob (a BLOB, b VARCHAR2)RETURN NUMBER AS …/CREATE OR REPLACE OPERATOR sssBINDING
(BLOB, VARCHAR2) RETURN NUMBERWITH INDEX CONTEXT, SCAN CONTEXT cdc_ixmdlUSING cdcsss.SssMatchBlob,
(BLOB, CLOB) RETURN NUMBERWITH INDEX CONTEXT, SCAN CONTEXT cdc_ixmdlUSING cdcsss.SssMatchBlob ;
User-Defined Indexes (Step 3 of 4)Registers a new indexing scheme to support a set of operatorsImplements domain indexing scheme as ODCIIndexinterface functions
CREATE OR REPLACE TYPE cdc_ixmdl AS OBJECT {STATIC FUNCTION ODCIIndexCreate(..) RETURN NUMBER,STATIC FUNCTION ODCIIndexInsert(..) RETURN NUMBER,STATIC FUNCTION ODCIIndexUpdate(..) RETURN NUMBER,… };
CREATE OR REPLACE INDEXTYPE ixmdl FORsss(BLOB, VARCHAR2), sss(BLOB, CLOB) USING cdc_ixmdl;
CREATE INDEX acd2d_mol_mdlix ON acd2d_mol(ctab)INDEXTYPE IS c$mdlimol2.ixmdlPARAMETERS ('RCG_DB = ACD2D');
Extensible Indexing Architecture
Logical index
Physical index table
UserApp.
User-definedindexes/stats
CREATE INDEX
SELECT
ODCIIndexCreateODCIIndexInsert, etc.
ODCIIndexStartODCIIndexFetchODCIIndexClose
DBMS_STATS ODCIStatsCollectDMLs
Simple QuerySELECT cdbregno as cdbregno,
chime_string(ctab) structure FROM acd2d_mol WHERE sss(ctab,’querystring’)=1;
Operation Options Object--------------------------- ---------------- --------------SELECT STATEMENT Cost=1229TABLE ACCESS BY INDEX ROWID ACD2D_MOL
DOMAIN INDEX ACD2D_MOL_MDLIX
SELECT cdbregno as cdbregno, chime_string(ctab) structure
FROM acd2d_mol WHERE sss(ctab,’querystring’)=1;
Operation Options Object--------------------------- ---------------- --------------SELECT STATEMENT Cost=1229TABLE ACCESS BY INDEX ROWID ACD2D_MOL
DOMAIN INDEX ACD2D_MOL_MDLIX
Extensible Optimizer (Step 4 of 4)SELECT /*+ ORDERED */ to_char(acd.cdbregno) as ACD_regno,
ch.ch_mdlnum as MDL_number, ch.ch_casrn as CAS_Number,ch.ch_prdcnt as Product_Count, chime_string(ctab) as structure
FROM acd991.ACD2D_moltableb acd, acd991.ACD2D_mol mol,acd991.chemical ch
WHERE sss(ctab,<Chime String>)=1 AND ch.ch_prdcnt > 1AND acd.cdbregno = mol.cdbregnoAND mol.mdlnumber = ch.ch_mdlnum
Aware of user-defined indexes User can specify cost & selectivityUser can collect domain specific statisticsSimple defaults or comprehensive functions
Cost-Based Optimizer
Query
Hints
Statistics
Execution Plan:join order,Join methods,access methods,…
C(P1)
C(P2)
C(P3)
C(Pn)
CBO (min)
System State
Estimating Costs
I/O cost: number of data blocks fetched– DBMS_STATS package
CPU cost: number of machine instructions
SQL> DECLARE2 a INTEGER;3 BEGIN4 a := DBMS_ODCI.ESTIMATE_CPU_UNITS(10);5 DBMS_OUTPUT.PUT_LINE
('CPU units = '|| a*1000);6 END;7 /
CPU units = 1198114000
Complex QuerySELECT /*+ ORDERED */ to_char(acd.cdbregno) as ACD_regno,
ch.ch_mdlnum as MDL_number, ch.ch_casrn as CAS_Number,ch.ch_prdcnt as Product_Count, chime_string(ctab) as structure
FROM acd991.ACD2D_moltableb acd, acd991.ACD2D_mol mol,acd991.chemical ch
WHERE sss(ctab,<Chime String>)=1 AND ch.ch_prdcnt > 1 AND acd.cdbregno = mol.cdbregno AND mol.mdlnumber = ch.ch_mdlnum
Query Plan-------------------------------------------------SELECT STATEMENT Cost = 11689
NESTED LOOPSNESTED LOOPS
TABLE ACCESS BY INDEX ROWID ACD2D_MOLTABLEBDOMAIN INDEX ACD2D_MOLTABLEB_MDLIX
TABLE ACCESS BY INDEX ROWID ACD2D_MOLINDEX RANGE SCAN IX_ACD2D_MOL
TABLE ACCESS BY INDEX ROWID CHEMICALINDEX UNIQUE SCAN CH_MDLNUM
Agenda
OverviewOracle Database 10g Extensibility Framework
– Building a data cartridge– Deploying a data cartridge– Managing data cartridges
DemoSummaryQ & A
Deploying
Cross platform transportable tablespacesCollecting statisticsRuntime architecture (J2EE, RAC, Grid, etc.)User privileges (invokers rights)Multi-threaded agents
Transportable TablespacesCopy database subsets (Tablespaces) between databases
– Operating system file copy for data– Managed transfer of metadata (indexes,
constraints, stats, etc.) between databases
– “Plug and go”
Introduced in Oracle8iEnhanced in Oracle9i to support different block sizesResult: extremely fast Result: extremely fast bulk data transport bulk data transport between databasesbetween databases
Oracle10g Cross Platform Transportable Tablespace
Solaris (SPARC)
Open VMS (Intel IA64)
Windows (Intel IA32/64)
Linux (Intel IA32/64)
AIX (PowerPC)
HP-UX (PA-RISC)
Tru64 UNIX (Alpha)
Little Endian Platforms
Simple File Copy Within These
HP-UX (Intel IA64)
Big Endian Platforms
Simple File Copy Within These
RMANHigh SpeedFile Convert
Automatic Statistics Gathering
DBMS_STATS.GATHER_SCHEMA_STATS(‘SCOTT’,options => ‘GATHER AUTO’)
This simple statement gathers statistics for the entire schema, and implicitly determines:
– what tables need to be sampled– how much sampling is needed– which columns need histograms– what degree of parallelism to use when sampling
Dynamic and Scalable Parallel Execution
SQL
querycoordinator
queryserver
queryserver
queryserver
queryserver
queryserver
queryserver
querycoordinator
results
• Query coordinator allocates query servers, and divides query into separate subtask
• Each “set” of query servers perform different task (e.g. scanning, sorting, joining)
• Results are “pipelined” from one set of query servers to the next
RAC Configuration
Order Entry Supply Chain
High SpeedInter-connects
High Speed Switches
Agenda
OverviewOracle Database 10g Extensibility Framework
– Building a data cartridge– Deploying a data cartridge– Managing data cartridges
DemoSummaryQ & A
Managing Data CartridgesData cartridge upgrades
– Alter type/operator/indextype/indexPerformance Tuning
– Current statistics, Explain plan, Trace event– Inlined LOBs, Persistent Data interface, chunk sizes, loading,
compressionData Management
– Partitioning, Import/export, SQL*Loader, data pump– ALL_TYPES, ALL_TYPE_ATTRS, ALL_TYPE_METHODS– ALL_OPERATORS, ALL_INDEXTYPES,
ALL_INDEXTYPE_OPERATORS, ALL_INDEXES views– ALL_IND_STATISTICS, ALL_TAB_STATISTICS,
ALL_TAB_COL_STATISTICSSpace Mgmt
– ASSM/Locally Managed Tablespaces– Inlined LOBs storage
Type Evolution
• Non-structural changes– adding, dropping or modifying
methodsStructural changes
– adding, dropping or modifying attributes
ALTER TYPE person_t
ADD/MODIFY/DROP attribute (ssn
VARCHAR2(11)) CASCADE;
Alter Operator
Add bindings to or drop bindings from an existing operator
ALTER OPERATOR Ordsys.Contains ADD BINDING (music.artist, music.artist) RETURN NUMBER USING music.contains_func;
Alter Indextype
Associate the added operator binding to the indextype
ALTER INDEXTYPE TextIndexType ADD lob_contains(CLOB, CLOB);
Alter IndexODCIIndexAlter( ia ODCIIndexInfo,
parms IN OUT VARCHAR2, alter_option NUMBER, env ODCIEnv) RETURN NUMBER
alter_option is one of the following options:• AlterIndexNone: ALTER INDEX [PARTITION] PARAMETERS• AlterIndexRename: ALTER INDEX RENAME [PARTITION]• AlterIndexRebuild: ALTER INDEX REBUILD [PARTITION]
[PARALLEL (DEGREE deg)] [PARAMETERS]• AlterIndexUpdBlockRefs: ALTER INDEX [schema.]index
UPDATE BLOCK REFERENCES
Agenda
OverviewOracle Database 10g Extensibility FrameworkDemoSummaryQ & A
D E M O N S T R A T I O N
A Data Cartridge in Action
Agenda
OverviewOracle Database 10g Extensibility FrameworkDemoSummaryQ & A
Summary
Manage All of Your Data with Oracle10g Database
– Consolidate all your data– Simplify application development, deployment,
and management– Explore versatile capabilities– Ensure RAS, manageability, and security
Managing All of Your Data with Oracle10g Database!
Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S