Upload
avis-short
View
216
Download
2
Embed Size (px)
Citation preview
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
Streams new features in 11g
Zbigniew Baranowski
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 2
Outline
• Combined capture and apply• Automatic split and merge procedures• Compare and converge• Streams performance advisor• Other features
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 3
Combined capture and apply• Optimization of data flow management• Uses direct connections between capture
and apply process• It is applied automatically if required
conditions are satisfied
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 4
Combined capture and apply• CC&A requirements
– The capture process's queue must have a single publisher, and it must be the capture process
– A propagation must be configured between the capture process's queue and the apply process's queue. There can be no intermediate queues
– Each apply process that applies changes from the same source database must use a different queue
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 5
Combined capture and apply• Improved flow control (new capture process
behavior)– No subscribers => capturing paused– Apply or propagation disabled => target removed from
subscribers list => no LCRs accumulation in source queue– Target re-enabled => target added to the subscribers list
=> automatic re-capturing of missing changes– Controlling the oldest applied SCN of subscribers– Constant max queues capacity
• 10000 messages for source queue• 1000 messages for destination queue
– No spilling from queues
• Max throughput increased about x10
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 6
Combined capture and apply• Point in time recovery automatic handling
– No additional steps are required to restart replication
– Capture needs an access to the old archive logs before recovery
– Recovery steps• Stop capture, apply and propagation• Perform point in time recovery• Start apply, propagation and capture• Start SCN is determinated automatically
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 7
Combined capture and apply
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 8
Automatic split & merge
• Are split & merge procedures needed with combined capture and apply?
• Split of broken stream procedure is required:– to keep replicas in archive log window– to reduce number of LCRs to be re-captured by
capture process
• Start SCN = min {SCN app1, SCN app2, …}• All recaptured LCRs will be propagated to
all destinations, but applied only by one.
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 9
Automatic split & merge
• Replacement of our custom scripts• Three ways of performing split and merge
operation– SQL script generation for manual execution– Splitting and merging procedures
• DBMS_STREAMS_ADM.SPLIT_STREAMS• DBMS_STREAMS_ADM.MERGE_STREAMS
– Automatic replication management (new in 11.2)• New capture parameters
– split_threshold– merge_theshold
• Monitor job running periodically
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 10
Automatic split & merge
• DBA_STREAMS_SPLIT_MERGE
• DBA_STREAMS_SPLIT_MERGE_HIST
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 11
Compare and converge
• DBMS_COMPARSION– Shared object synchronization package
• Compare rows in an object at 2 database• Converge objects in case of differences
– Supported database object types• Tables• Single-tables views• Materialized views• Synonyms for tables, single-tables views and
materialized views
– Version supporting:• Local >= 11.1• Remote >=10.1.0
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 12
Compare and converge
– Comparison against different object types• e.g. table vs. materialized view• Column level synchronization
– Scan options• Identifying row differences (in separate table)• Random portion of data
– Percentage of data to be scanned– Cyclical way
• Custom portion of data (rowid range)
– No support for:• LONG, LONG RAW, ROWID, CLOB, NCLOB, BLOB,
BFILE• User defined types
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 13
Compare and converge
DBA_COMPARISON General information about comparsions in the local databases
DBA_COMPARISON_COLUMNS Information about columns compared by each comparsion in local database
DBA_COMPARISON_SCAN Information about each scan run result
DBA_COMPARISON_SCAN_VALUES Information about range of rows compared during scan
DBA_COMPARISON_ROW_DIF Detailed information about row differences found in the each scan run. Populated when compare run with perform_row_dif parameter.
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 14
Compare and converge
• Compare & converge steps– Create an comparison on object
• DBMS_COMPARISON.CREATE_COMPARISON
– Perform comparison• DBMS_COMPARISON.COMPARE
– If necessary do the conversion• DBMS_COMPARISON.CONVERGE
– Local wins– Remote wins
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 15
Compare and converge
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 16
Compare and converge
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 17
Compare and converge
• Test result for artificial data with data set of 20 x varchar2 columns 17B each– Comparisons of
• consistent data– 18 MB/s• inconsistent data – 1,7 MB/s• purged table – 6 MB/s
– Convergence of• inconsistent data – 4 MB/s• purged table – 3,5 MB/s
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 18
Streams performance advisor
• DBMS_STREAMS_ADVISOR_ADM– Analyzes local and remote (via dblink)
database configuration– Provides
• Oracle streams topology representation– By assignment of to each stream path
• Streams performance statistics– Component level– Session level
– Gathers streams related metrics with snapshots and compares them
• DBMS_STREAMS_ADVISOR_ADM.ANALYZE_CURRENT_PERFORMANCE;
– Stores statistics in set of dictionary views
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 19
Streams performance advisor dictionary views
DBA_STREAMS_TP_COMPONENT contains information about each Oracle Streams component at each database
DBA_STREAMS_TP_COMPONENT_LINK contains information about how messages flow between Oracle Streams components
DBA_STREAMS_TP_COMPONENT_STAT contains temporary performance statistics and session statistics about each Oracle Streams component.
DBA_STREAMS_TP_DATABASE contains basic information about each database that contains Oracle Streams components (name, version, last advisor run etc.)
DBA_STREAMS_TP_PATH_BOTTLENECK contains temporary information about Oracle Streams components that might be slowing down the flow of messages in a stream path
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 20
Streams performance advisor
• UTL_SPADV Package– Replacement of strmmon command line tool– Monitoring of whole stream path– Live streams data collector– Implemented with pl/sql procedures
• Stores measurements in db tables
– Monitoring management• UTL_SPADV.START_MONITORING• UTL_SPADV.STOP_MONITORING• UTL_SPADV.SHOW_STATS
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 21
Streams performance advisor
• UTL_SPADV Package– Pre run actions
• @utlspadv.sql• grant execute on SYS.DBMS_LOCK to …
– Example outputPATH 1 RUN_ID 3 RUN_TIME 2008-MAR-20 10:52:22 CCA Y |<C> CAPTURE_HNS=>HUB.EXAMPLE.COM 10267 10040 3 LMR 95% 0% 3.3% "" LMP (1) 86.7% 0% 11.7% "" LMB 86.7% 0% 11.7% "" CAP+PS 10040 1.46E+06 3 16.7% 71.7% 11.7% "" |<PR> SPOKE1.EXAMPLE.COM=> 98.5% 0% 0.6% "" |<Q> "STRMADMIN"."DESTINATION_SPOKE1" 25 0 0 |<A> APPLY_SPOKE1 10042 100 4 APR 93.3% 0% 6.7% "" APC 98.1% 0% 1.8% "" APS (1) 953 272 76.7% "CPU + Wait for CPU" |<B> "NO BOTTLENECK IDENTIFIED"
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 22
Other features
• LCRs tracking– By adding a tag to each captured LCR– V$STREAMS_MESSAGE_TRACKING
• Synchronous Capture– Uses internal mechanism for capturing
changes (DML only) instead of logminer
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 23
Other features
• Oracle Streams Jobs Use Oracle Scheduler instead of DBMS_JOB
• Automated Alerts for Oracle Streams Clients and Thresholds in EM
• Support for – Transparent Data Encryption– XMLType Columns– Virtual Columns– Compressed table (in 11.2)
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 24
Bugs discovered
• Propagation view population– After recreation of processes all propagation
metrics in v$propagation_sender– Assigned to the Oracle development
• Unable to split a stream as strmadmin user– ORA-02289: sequence does not exist– Fixed by: grant select on STREAMS$_SM_ID to
strmadmin
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
Zbigniew Baranowski 25
Conclusions
• Real time replication– About 10 times faster than 10g– Reduced resource consumption
• Easier streams recovery– Split & merge– Point in time recovery– Compare & converge synchronization
• More diagnostic and monitoring tools– Enterprise Manager– Streams Performance Advisor– LCRs tracking