Upload
erick-mclaughlin
View
226
Download
0
Tags:
Embed Size (px)
Citation preview
40183 :Oracle XML DB Performance and Tuning
Mark D. Drake
Senior Product Manager
Session id:
IntroductionIntroduction
Managing XML Content
XML Stored as XMLType SQL functions allow XPath expressions to
operate on XML content– extract(), extractValue(), updateXML(),
existsNode()
Two modes of operation– Functional Evaluation– Query-Rewrite
Functional Evaluation
Works with all XMLType columns and Tables. Parse the XML document and construct a
DOM Evaluate XPath expressions using DOM API Based on proven XDK ‘C’ XML Parser and
DOM Most appropriate to operations on small
numbers of small documents
XPath Re-write
Query and Update operations Requires Schema based XMLType Operates directly on the underlying Object
Store Relational database engine used to evaluate
XPath expression– XPath Expression translated into equivalent SQL– Eliminates need to parse XML and construct DOM
Allows efficient operations on large collections of XML Documents
CollectionsCollections
What is a collection
A Collection is a set of elements with the same name and similar structure
<LineItem ItemNumber="1"> <Description>Duel at Ichijoji Temple</Description> <Part Id="37429125526" UnitPrice="29.95"
Quantity="3"/</LineItem><LineItem ItemNumber="2"> <Description>The Red Shoes</Description> <Part Id="37429128220" UnitPrice="39.95"
Quantity="4"/></LineItem>
Defining a Collection
In XML Schema, Collections are defined using maxOccurs attribute
<xs:complexType name="LineItemsType“
<xs:sequence>
<xs:element name="LineItem“
type="LineItemType“
maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
Working with Collections
A Typical XML document contains one or more collections
Collections may be Nested– A collection may contain a collections
Query and Updating collections present the biggest challenge
Querying Collections
Sample Query on a CollectionGet the Reference and Item Number of each LineItem that contains a reference to Part “717951002372“
XPATH :='/PurchaseOrder/LineItems/LineItem[Part/@Id="717951002372"]';select extractValue(p.object_value,'/PurchaseOrder/Reference') extractValue(value(l),'/LineItem/@ItemNumber') from PURCHASEORDER p, table (xmlsequence(extract(p.object_value,XPATH))) l
Querying Collections
Use the extract() function to get the set of nodes that match an XPath expression
XPATH :='/PurchaseOrder/LineItems/LineItem[Part/@Id="717951002372"]';select extractValue(object_value,'/PurchaseOrder/Reference') extractValue(value(l),'/LineItem/@ItemNumber') from PURCHASEORDER p, table (xmlsequence(extract(p.object_value,XPATH))) l
Querying Collections
The extract function will be evaluated for each document in the table.
An XMLType will be generated from each document that contains a node matching the XPath expression.
The XML Type will be contain an XML Fragment – The Fragment will consist of the set of nodes that match
the specified XPATH expression.
Querying Collections
Use the xmlSequence function to generate a set of XMLType Objects from each fragment
XPATH :='/PurchaseOrder/LineItems/LineItem[Part/@Id="717951002372"]';select extractValue(object_value,'/PurchaseOrder/Reference') extractValue(value(l),'/LineItem/@ItemNumber') from PURCHASEORDER p, table (xmlsequence(extract(p.object_value,XPATH))) l
Querying Collections
xmlSequence will create a set of XMLType objects from the result of the extract.
Each set will contain one XMLType object for each top level node in the XML fragment.
Each XMLType will consist of a well-formed XML document.
Querying Collections
Use the TABLE function to convert the set of XMLType objects into a virtual table
XPATH :='/PurchaseOrder/LineItems/LineItem[Part/@Id="717951002372"]';select extractValue(p.object_value,'/PurchaseOrder/Reference') extractValue(value(l),'/LineItem/@ItemNumber') from PURCHASEORDER p, table (xmlsequence(extract(p.object_value,XPATH))) l
Join the virtual table with the base table to get required result set
ManagingCollectionsManaging
Collections
Collection Management
Four options for storing collections– BLOB Storage
SQL objects stored in a single BLOB Column
– Nested Table Storage SQL objects stored as rows in a Nested Table
– CLOB Storage XML text
– XMLType Table Storage XMLType objects stored in an XMLType Table
Why different storage models ?
Applications have different requirements– Querying within in a collection– Indexing within a collection– Updating within a collection
Choosing the correct storage model is vital
Collections and VARRAYS
VARRAY: SQL concept for managing collections
VARRAY can be collection of Scalars or collection of SQL Type
– AUTHOR_V VARRAY(n) of VARCHAR2(40)– LINE_ITEM_V VARRAY(n) of LINE_ITEM_T
In XML DB the members of a collection are mapped to a SQL VARRAY.
LOBLOB
LOB model
Default : No Annotation Required Collection stored as VARRAY of SQL Objects SQL Objects serialized into a single BLOB Indexing using CtxXPath index No partial updates
– update requires re-writing entire BLOB
Query re-write for XPath expressions
LOB : Objects<xs:complexType name="LineItemsType" xdb:SQLType="LINEITEMS_T"> <xs:sequence> <xs:element name="LineItem"
type="LineItemType" maxOccurs="unbounded" xdb:SQLName="LINEITEM" xdb:SQLCollType="LINEITEM_V"/>
</xs:sequence></xs:comlexType>
SQL> desc LINEITEMS_T LINEITEMS_T is NOT FINAL Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T LINEITEM LINEITEM_V
LOB : StorageSQL> describe LINEITEM_V LINEITEM_V VARRAY(2147483647) OF LINEITEM_T LINEITEM_T is NOT FINAL Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T ITEMNUMBER NUMBER(38) DESCRIPTION VARCHAR2(256 CHAR) PART PART_T
LOB Storage
ReferenceReference User IdUser Id …… LineItemLineItem
ABANDAABANDA ……ABANDA-20..
LINEITEM_V( LINEITEM_T(...), LINEITEM_T(...), LINEITEM_T(...), LINEITEM_T(...))
LOB : Query Re-Write
SELECT P.XMLDATA."Reference", L."ItemNumber" FROM PURCHASEORDER P, TABLE (P.XMLDATA."LineItems"."LineItem") L WHERE L."Part"."PartNumber" = '717951002372'
Evaluated using SQL to process VARRAY of LINEITEM objects
Entire VARRAY has to be loaded into Memory
NestedTablesNestedTables
Nested Table model
Annotation: xdb:storeVarrayAsTable=“true” Collection converted into a set of SQL objects Each SQL Object stored as a separate row in
a Nested Table. Nesting of Nested Tables for collections within
collections SQL Based Fragment access and update Index with B-Tree, Functional and CtxXPath
indexes
Nested Table : Objects<xs:complexType name="LineItemsType" xdb:SQLType="LINEITEMS_T"> <xs:sequence> <xs:element name="LineItem"
type="LineItemType" maxOccurs="unbounded" xdb:SQLName="LINEITEM" xdb:SQLCollType="LINEITEM_V"/>
</xs:sequence></xs:comlexType>
SQL> desc LINEITEMS_T LINEITEMS_T is NOT FINAL Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T LINEITEM LINEITEM_V
Nested Table : ObjectsSQL> describe LINEITEM_V LINEITEM_V VARRAY(2147483647) OF LINEITEM_T LINEITEM_T is NOT FINAL Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T ITEMNUMBER NUMBER(38) DESCRIPTION VARCHAR2(256 CHAR) PART PART_TSQL> describe LINEITEM_TABLE Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T ITEMNUMBER NUMBER(38) DESCRIPTION VARCHAR2(256 CHAR) PART PART_T
Nested Table : Storage
ReferenceReference User IdUser Id …… LineItemLineItem
ABANDAABANDA ……ABANDA-20..
IDID ItemNumberItemNumber DescriptionDescription PartPart
Good Morn…Good Morn… ……1
Uriah Hee…Uriah Hee… ……21
SistersSisters ……31
The Prince…The Prince… ……41
1
1
Nested Table : Query Re-Write
SELECT P.XMLDATA."Reference", L.ITEMNUMBER FROM PURCHASEORDER P, LINEITEM_TABLE l WHERE L."Part."PartNumber"='717951002372'
AND L.NESTED_TABLE_ID = SETID(P.XMLDATA."LineItems"."LineItem“)
Evaluated using SQL to query contents of the Nested Table
Working with Nested Tables
xdb:storeVarrayAsTable=“true” is a “shotgun” approach
– Every collection stored as a Nested Table– Nested Tables have ‘unfriendly’ names
Simple cases– Rename tables created by Schema Registration
Complex cases – Create tables manually using CREATE TABLE
Renaming Nested Tables
Query USER_NESTED_TABLES to find the direct descendant of the Parent Table
SQL> select PARENT_TABLE_COLUMN, TABLE_NAME 2 from USER_NESTED_TABLES 3 where PARENT_TABLE_NAME = 'PURCHASEORDER' 4 /
PARENT_TABLE_COLUMN TABLE_NAME-------------------------------- -----------------------"XMLDATA"."LINEITEMS"."LINEITEM" SYS_NTDfLwYKWcRxmyssvmCvRMqw=="XMLDATA"."ACTIONS"."ACTION" SYS_NTqxF5epLrSniXaAWq5A4Uig==
Renaming Nested Tables
SQL> rename "SYS_NTDfLwYKWcRxmyssvmCvRMqw==" to LINEITEM_TABLE 2 /Table renamed.
SQL> rename "SYS_NTqxF5epLrSniXaAWq5A4Uig==" to ACTION_TABLE 2 /Table renamed.
SQL> select PARENT_TABLE_COLUMN, TABLE_NAME 2 from USER_NESTED_TABLES 3 where PARENT_TABLE_NAME = 'PURCHASEORDER' 4 /PARENT_TABLE_COLUMN TABLE_NAME--------------------------------- ----------------------"XMLDATA"."ACTIONS"."ACTION" ACTION_TABLE"XMLDATA"."LINEITEMS"."LINEITEM" LINEITEM_TABLE
Multi-Level Nested Tables
Collections within Collections result in Storage Models consisting of multiple levels of Nested Tables
Repeat the process using the Nested Table name as the PARENT_TABLE_NAME
Manual Table Creation
Can create Nested Tables as part of create Table
Need to know the name of the VARRAYs that will stored as Nested Tables
Allows flexibility, only use Nested Table Storage only where it adds value
Must use Nested Table storage with collections that contain an element stored as a CLOB.
Manual Table Creation Example
create table PURCHASEORDER of XMLTYPEXMLSCHEMA
"http://localhost:8080/home/SCOTT/xsd/purchaseOrder.xsd"ELEMENT "PurchaseOrder"varray "XMLDATA"."ACTIONS"."ACTION" store as table ACTION_TABLE ( (primary key (NESTED_TABLE_ID, ARRAY_INDEX)) organization index overflow )varray "XMLDATA"."LINEITEMS"."LINEITEM" store as table LINEITEM_TABLE ( (primary key (NESTED_TABLE_ID, ARRAY_INDEX)) organization index overflow )
CLOBCLOB
CLOB model
Annotation: xdb:SQLType=“CLOB” Collection stored as XML text
– Reduced complexity– Fast Ingestion and Retrieval of Collection – Lower memory and processing requirements
Index using CTXPATH index No Optimization of Fragment Level operations
– XPath based Functions evaluated via DOM API– Fetching Fragments requires Parsing– Updating Requires Parsing and Rewriting
CLOB : Objects
<xs:complexType name="LineItemsType" xdb:SQLType="LINEITEMS_T"> <xs:sequence> <xs:element name="LineItem"
type="LineItemType" maxOccurs="unbounded" xdb:SQLName="LINEITEM" xdb:SQLType="CLOB"/>
</xs:sequence></xs:comlexType>
SQL> describe LINEITEMS_T LINEITEMS_T is NOT FINAL Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T LINEITEM CLOB
CLOB : Storage
ReferenceReference User IdUser Id …… LineItemLineItem
ABANDAABANDA ……ABANDA-20..
<LineItem ItemNumber="1"> <Description>Good … <Part Id="3742914 …</LineItem><LineItem ItemNumber="2“ <Description>Uriah.. <Part Id="6950030…</LineItem>
CLOB: Query Re-Write
SELECT P.XMLDATA."Reference", EXTRACTVALUE(value(l),'/LineItem/@ItemNumber') FROM PURCHASEORDER P, TABLE (
XMLSEQUENCE (
EXTRACT (
VALUE(P), '/PurchaseOrder/LineItems/LineItem/' || 'Part/@Id="717951002372"]' ) ) ) l
Evaluated using by Parsing CLOB and using DOM API
XMLTypeTable
XMLTypeTable
XMLType Table model
Annotation xdb:SQLInline=“false” Also known as ‘out-of-line’ storage Collection converted into a set of XMLType objects
stored in a separate XMLType table. Parent table contains a VARRAY of REF XMLTYPE
that points to rows in the child table Index using B-Tree, Functional and CtxXPath indexes SQL Based Fragment access and update. Must allow Schema Registration to create the Tables.
XMLType Table : Objects<xs:complexType name="LineItemsType" xdb:SQLType="LINEITEMS_T"> <xs:sequence> <xs:element name="LineItem"
type="LineItemType" maxOccurs="unbounded" xdb:SQLName="LINEITEM" xdb:SQLInline="false“ xdb:defaultTable="LINEITEM_TABLE"/>
</xs:sequence></xs:comlexType>
SQL> describe LINEITEMS_T LINEITEMS_T is NOT FINAL Name Null? Type ------------------- -------- ------- SYS_XDBPD$ XDB.XDB$RAW_LIST_T LINEITEM XDB.XDB$XMLTYPE_REF_LIST_T
XMLType Table : ObjectsSQL> describe XDB.XDB$XMLTYPE_REF_LIST_TXDB.XDB$XMLTYPE_REF_LIST_T VARRAY(2147483647)
OF REF XMLTYPE
SQL> describe LINEITEM_TABLEName Null? Type ------------------------- -------- ----------------------------TABLE of SYS.XMLTYPE
( XMLSchema "http://localhost:8080/home/SCOTT/xsd/purchaseOrder.xsd"
Element "LineItem“) STORAGE Object-relational TYPE "LINEITEM_T"
XMLType with Ref Table
Combine with xdb:storeVarrayAsTable=“true” to force VARRAY of REFS to be stored in a nested table
Each row in the nested table consists of SETID and a REF to a ROW in the child table
Scope the REFS in the child table to allow full optimization of queries
rename "SYS_NTDfLwYKWcRxmyssvmCvRMqw==" to LINEITEM_REF_TABLE
alter table LINEITEM_REF_TABLE add (scope for (COLUMN_VALUE) is LINEITEM_TABLE)
Scope REF of XMLType
select PARENT_TABLE_COLUMN, TABLE_NAME from USER_NESTED_TABLES where PARENT_TABLE_NAME = 'PURCHASEORDER‘/PARENT_TABLE_COLUMN TABLE_NAME-------------------------------- -----------------------"XMLDATA"."LINEITEMS"."LINEITEM" SYS_NTDfLwYKWcRxmyssvmCvRMqw==
rename "SYS_NTDfLwYKWcRxmyssvmCvRMqw==" to LINEITEM_REF_TABLE/alter table LINEITEM_REF_TABLE add (scope for (COLUMN_VALUE) is LINEITEM_TABLE)/create index LINEITEM_REF_INDEX on LINEITEM_REF_TABLE (COLUMN_VALUE)/
XMLType Table : Query Rewrite
SELECT P.XMLDATA."Reference", L.XMLDATA."Item" FROM PURCHASEORDER p, TABLE("P.XMLDATA."LineItems"."LineItem") R, LINEITEM_TABLE L WHERE L."Part"."PartNumber" = '717951002372' AND VALUE(R)= Ref(l)
Evaluated using SQL to query contents of the XMLType table
Relative FTP Insert performance
00.5
11.5
22.5
33.5
44.5
5
CLOB LOB NestedTable
XMLTypeTable
ScopedXMLType
Table
Un-Indexed Query Performance
0
2
4
6
8
10
12
14
CLOB LOB NestedTable
XMLTypeTable
ScopedXMLType
Table
IndexingCollections
IndexingCollections
Indexing Collections
Collections stored as LOB or CLOB can only be indexed using CtxXPath Index
Collections stored as Nested Tables or Out-of-Line tables can be index using B-Tree Indexes
– Indexes on collections are created on the nested table or Out-of-Line Table, not the Parent Table
Indexing with Nested Tables
Indexes on Nested Table are defined in terms of the SQL Type structure
Nested Tables contain a foreign key called NESTED_TABLE_ID
Consider adding NESTED_TABLE_ID to any secondary indexes
Nested Table: Secondary Index
create index iPartNumberIndex on LINEITEM_TABLE l(l.PART.PART_NUMBER, l.NESTED_TABLE_ID)
Query Plan showing use of Index
---------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 662 | 17M| 89 (2)| 00:00:02 || 1 | NESTED LOOPS | | 662 | 17M| 89 (2)| 00:00:02 ||* 2 | INDEX UNIQUE SCAN | LINEITEM_TABLE_IOT | 82 | 370K| 6 (0)| 00:00:01 ||* 3 | INDEX RANGE SCAN | IPARTNUMBERINDEX | 33 | | 3 (34)| 00:00:01 || 4 | TABLE ACCESS BY INDEX ROWID| PURCHASEORDER | 8 | 176K| 2 (50)| 00:00:01 ||* 5 | INDEX UNIQUE SCAN | SYS_C004261 | 1 | | 1 (100)| 00:00:01 |---------------------------------------------------------------------------------------------------
Indexing with XMLType Tables
Indexes on Out-Of-Line XMLType tables are created using XPath syntax
Index is defined in terms of the extractValue() functions
Index is not a function index. XPath re-write allows the index to be converted into Object-Relational SQL.
XPath is from the root element of the Out-of-Line table, not the root of the parent document
When storing REFs as nested table remember to index the nested table
XMLType Table: Secondary Index
create index iPartNumberIndex on LINEITEM_TABLE l(extractValue(value(l),'/LineItem/Part/@Id'))create index LINEITEM_REF_INDEX on LINEITEM_REF_TABLE(COLUMN_VALUE)
Query Plan showing use of Index----------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |----------------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 4626 | 100M| 591 (2)| 00:00:08 || 1 | NESTED LOOPS | | 4626 | 100M| 591 (2)| 00:00:08 ||* 2 | HASH JOIN | | 571 | 136K| 14 (15)| 00:00:01 || 3 | TABLE ACCESS BY INDEX ROWID| LINEITEM_TABLE | 571 | 115K| 6 (17)| 00:00:01 ||* 4 | INDEX RANGE SCAN | IPARTNUMBERINDEX | 571 | | 2 (50)| 00:00:01 || 5 | INDEX FAST FULL SCAN | LINEITEM_REF_INDEX | 8168 | 295K| 8 (13)| 00:00:01 || 6 | TABLE ACCESS BY INDEX ROWID | PURCHASEORDER | 8 | 176K| 2 (50)| 00:00:01 ||* 7 | INDEX RANGE SCAN | SYS_C004315 | 8 | | 1 (100)| 00:00:01 |----------------------------------------------------------------------------------------------------
Indexed Query Performance
0
0.5
1
1.5
2
2.5
3
3.5
Nested Table XMLType Table Scoped XMLTypeTable
DOMFidelity
DOMFidelity
Dom Fidelity
Requires additional processing and storage overhead Specific meta data maintained for each instance
– Meta data managed at the ‘Type’ Level in the SYS_XDBPD$ attribute
– SYS_XDBPD$ is stored as a (in-line) LOB.
Do you need it ?
Can disable on a Type by Type basis using annotation xdb:maintainDOM=“false”
Not using Namespaces No Comments or Processing Instructions Do not care about empty Vs missing elements Elements in an All and Choice will be ordered
as per the Schema, not the instance Not worried about defaults Type does not allow Mixed text
Type defn with DOM Fidelity
desc LINEITEMS_T LINEITEMS_T is NOT FINAL Name Null? Type ------------------------------- -------- ------------------ SYS_XDBPD$ XDB.XDB$RAW_LIST_T LINEITEM LINEITEM_V
desc LINEITEM_V LINEITEM_V VARRAY(2147483647) OF LINEITEM_T LINEITEM_T is NOT FINAL Name Null? Type ------------------------------ -------- ------------------ SYS_XDBPD$ XDB.XDB$RAW_LIST_T ITEMNUMBER NUMBER(38) DESCRIPTION VARCHAR2(256 CHAR) PART PART_T
xdb:maintainDOM
<xs:complexType name="LineItemsType" xdb:SQLType="LINEITEMS_T" xdb:maintainDOM="false"> <xs:sequence> <xs:element name="LineItem" type="LineItemType" maxOccurs="unbounded" xdb:SQLName="LINEITEM" xdb:SQLCollType="LINEITEM_V"/> </xs:sequence></xs:complexType>
<xs:complexType name="LineItemType" xdb:SQLType="LINEITEM_T" xdb:maintainDOM="false"> <xs:sequence> <xs:element name="Description" type="DescriptionType“ xdb:SQLName="DESCRIPTION"/> <xs:element name="Part" type="PartType" xdb:SQLName="PART"/> </xs:sequence> <xs:attribute name="ItemNumber" type="xs:integer" xdb:SQLName="ITEMNUMBER" xdb:SQLType="NUMBER"/></xs:complexType>
Type defn without DOM Fidelity
desc LINEITEMS_T LINEITEMS_T is NOT FINAL Name Null? Type ------------------------------- -------- ------------------ LINEITEM LINEITEM_V
desc LINEITEM_V LINEITEM_V VARRAY(2147483647) OF LINEITEM_T LINEITEM_T is NOT FINAL Name Null? Type ------------------------------ -------- ------------------ ITEMNUMBER NUMBER(38) DESCRIPTION VARCHAR2(256 CHAR) PART PART_T
Dom Fidelity
00.10.20.30.4
0.50.60.70.80.9
1
Files DOM Fidelity No Fidelity
Insert TimeSize on Disk
HierarchicalTables
HierarchicalTables
Hierarchical Tables
XMLType Tables can be Hierarchically Enabled
– Table is linked to the Oracle XML DB repository– Each Row in the table corresponds to a
Resource (document) in the XML DB repository– Row Level Security defined by associating an
ACL with a resource– DML operations on the row reflected in the
resource (Last Modified Date, Delete)
Hierarchical Considerations
Tables created by Schema Registration are Hierarchically Enabled
– DML operations constrained by ACL– Table Level Triggers sync DML operations on
base-table with XDB repository
Cannot bypass ACL security by accessing content via SQL
Delete of a row in a table will delete the corresponding Resource
ACL Management
XML DB architected to support systems with thousands or millions of ACLs
ACL based security automatically enforced by calling the sys_checkAcl() function
ACL check is based on ACLOID and OID Dedicated ACL cache ensures that Oracle XML DB
can locate ACLs very efficiently ACLOID and OID are stored as hidden columns on an
XMLType table
ACL Checking
ACL checking can affect query performance– Need ACLOID and OID in addition to other
columns
ACL Based security can be disabled using dbms_xdbz.disable_hierarchy()
– Turns off ACL Evaluation and repository synchronization
In some cases disabling the hierarchy can lead to significant performance benefits
AQ&Q U E S T I O N SQ U E S T I O N S
A N S W E R SA N S W E R S