Logical Standby Database for Reporting
Mark Bole
NoCOUG Nov 10, 2005
Introduction
Mark Bole
Independent Consultant
Oracle, Unix, Perl since 1991
http://www.bincomputing.com
Today’s Session
• DataGuard Logical Standby — going outside the DG box
• What is Logical Standby?
• How to set it up
• Reporting/Batch: refresh cycle, materialized views, maintenance
DataGuard vs. Streams From the Oracle docco:
“While Streams and Data Guard do share some common underlying […] technology, they are both independent features that are built to solve different business needs. [see references]
“Oracle Data Guard is designed for protecting from data failure and disasters.
“Streams is designed for information sharing and distribution but can also provide a very efficient high availability solution.”
Logical Standby Product Placement?
Integration
8i
9i
10g
High Availability
Oracle Streams Oracle Data Guard –
SQL Apply (Logical Standby)
Basic Replication
Advanced Replication
Basic readable standby database
Oracle Data Guard – Redo Apply (Physical Standby)
Logical Standby Product Placement?
Integration
8i
9i
10g
High Availability
Oracle Streams Oracle Data Guard –
SQL Apply (Logical Standby)
Basic Replication
Advanced Replication
Basic readable standby database
Oracle Data Guard – Redo Apply (Physical Standby)
Why Not Logstdby for HA?
“Parity, one-to-one-ness, having a failover environment that is a mirror image of the production [site]. That is the ‘ideal’ situation.”
http://asktom.oracle.com
Too Many Ways for the Logical Standby to deviate from the primary — intentional or otherwise!
“Bugs” – A Real-Life StoryThe following is an excerpt from a recent posting on the oracle-l list.Need some help or opinions regarding Oracle logical standby database.
Logical standby, version 10, release x, patch x. Primary and standby are tightly coupled (maximum availability mode). [...]
2) From time to time, standby breaks, but this seems due mostly to schema changes [...]
3) DB Guard leaks memory on the primary. This was confirmed via the OS, not V$-views. . Had to disable it. Not really happy about that. […]
5) Primary is not happy about unexpected standby restarts.[...]
6) To sum up: something's a bit shaky in my environmentt. Should I give up with maximum availability mode and reduce the database to maximum performance mode? I am interested to hear from others.
Logical Standby has a documented list of
restrictions*Unsupported Datatypes (partial list)
LONGLONG RAWBFILEROWIDuser-defined typesvarraysnested tables
Unsupported Tables, Sequences, and Views (partial list)
• Tables using data segment compression
• Index-organized tables
*Note: many removed in 10g
Robustness: less more
ASM
Evolution of Robustness
8i
9i
10g
RMAN
SQL ApplyRAC
Assumptions: Logstdby for Reporting
• Not running DG Manager/Broker• No intention of ever switching DG roles • Log transport MAXIMUM PERFORMANCE mode• Implies existence of separate Physical Standby for
“real” recovery• NOARCHIVELOG mode (all report/batch data can be
re-created if necessary)• Service Level less than 24 x 7 (planned maintenance
outages)• Not running SQL Apply continuously
Today’s Session
• DataGuard Logical Standby — going outside the DG box
• What is Logical Standby?
• How to set it up
• Reporting/Batch: refresh cycle, materialized views, maintenance
The Goal:logical, not physical,
replication• Physical standby is an image copy of the
primary — every datafile block is the updated to match as of point in time.
• Other existing options for logical: Export / Import Traditional Advanced Replication Transportable tablespace (maybe)
DB Link
Previous Solution
Primary
Redo apply(once every 24 hours)
Physical Standby
Reporting Database
Daily Cycle:1. Close Physical Standby
2. Mount Physical Standby
3. Recover day's worth of redo
4. Open Physical Standbyread-only for rest of day
5. Run batch and reporting processes using dblink
6. Loop to step 1
DB Link
Previous Solution - Problems
Physical Standby (open read-only)
Reporting Database
• Need to close and re-open physical standby
• Two databases on same server creates inefficiencies
• Harder to tune distributed queries
• Cannot use MV logs in standby, hence cannot use MV's in reporting database
SQL Apply
“SQL Apply technology […] first transforms the redo data into SQL statements and then executes the generated SQL statements on the logical standby database.”
SQL Block-level changes
redo logminer SQL
Physical
Logical
GUI Log MinerI:\>Oracle\product\10.1.0\Client_1\BIN\oemapp.bat lmviewer
Select archived redo logs to mine
Example: schema in primarycreate user schema_blueidentified by xyzdefault tablespace TESTDB_Atemporary tablespace tempquota 100M on TESTDB_Aquota unlimited on temp;
grant create session, create table to schema_blue;
connect schema_blue/[email protected]
CREATE TABLE test_table (name varchar2(50), value varchar2(50), timestamp date,constraint pk_test_table primary key (name) using index);
insert into test_table values ('test1', 'value1', sysdate);
insert into test_table values ('test2', 'value2', sysdate);
commit;
Logical Change Records
Logical Standby = Streams Made Simple
• Streams is configured from the bottom up — individual tables, schemas, capture processes, apply processes, queues
• Logical Standby is configured from the top down — start with entire database, then specify only what you don’t want
• Less muss, less fuss
Today’s Session
• DataGuard Logical Standby — going outside the DG box
• What is Logical Standby?
• How to set it up
• Reporting/Batch: refresh cycle, materialized views, maintenance
Logstdby Pre-requisites
• Enterprise Edition license• Same operating system and platform
architecture as primary• Same RDBMS version (10g supports
rolling upgrades)• FORCE LOGGING at database level
Supplemental Logging “If you’re going to use supplemental logging, you need to be aware that you are actually asking for quite a lot of new information to be included in the redo stream… and all that extra information might just be enough to tip LGWR and ARCH over into serious performance problems.”
Howard J. Rogers, New Features in Oracle 9i
Much less of a problem if you have a unique or primary key on every table!
Create the Logstdby• Start with any kind of hot backup• Mount or quiesce the primary (requires
outage, restriction removed in 10g)• EXECUTE DBMS_LOGSTDBY.BUILD to put
the LogMiner dictionary into the redo stream• Rename datafiles, online redo logs in standby• Reset database name — DBNEWID utility• Register archive logfile containing LogMiner
dictionary, start SQL Apply
My favorite LogStdby settings•INIT.ORAremote_archive_enable = receivestandby_archive_dest = '/opt/oracle/admin/binc01dg/stbyarch'
•DBA_LOGSTDBY_PARAMETERSDBMS_LOGSTDBY.APPLY_SET ('TRANSACTION_CONSISTENCY', 'NONE');DBMS_LOGSTDBY.APPLY_SET ('MAX_EVENTS_RECORDED', '2000');DBMS_LOGSTDBY.APPLY_SET ('RECORD_SKIP_DDL', 'FALSE');DBMS_LOGSTDBY.APPLY_SET ('RECORD_APPLIED_DDL', 'FALSE');DBMS_LOGSTDBY.APPLY_SET ('_EAGER_SIZE', 1000);
•alter database guard standby;
Today’s Session
• DataGuard Logical Standby — going outside the DG box
• What is Logical Standby?
• How to set it up
• Reporting/Batch: refresh cycle, materialized views, maintenance
DB Link
Previous Solution
Primary
Redo apply(once every 24 hours)
Physical Standby
Reporting Database
Daily Cycle:1. Close Physical Standby
2. Mount Physical Standby
3. Recover day's worth of redo
4. Open Physical Standbyread-only for rest of day
5. Run batch and reporting processes using dblink
6. Loop to step 1
The Logstdby Advantage
Primary
OLTP application
LogStdby
Reporting/Batch application
SCHEMA_REDTABLESINDEXESPL/SQL
SCHEMA_BLUETABLESINDEXESPL/SQL
SCHEMA_GREENTABLESINDEXESPL/SQL
SCHEMA_BLUETABLESINDEXESPL/SQL
SCHEMA_ORANGETABLESINDEXESPL/SQL
DG SQL Apply
The Logstdby Advantage
LogStdby
Reporting/Batch application
SCHEMA_BLUETABLESMV LOGSINDEXESPL/SQL
SCHEMA_ORANGETABLESINDEXESMATL_VIEWSPL/SQL
Now we can create Materialized View logs directly in the “source” schema without impacting the primary!
Guard Standby (read-only)
Controlling what is replicated-- workaround for bug in 9.2.0.5exec DBMS_LOGSTDBY.SKIP('PROCEDURE', 'XYZ', '%', null);
exec DBMS_LOGSTDBY.SKIP('SCHEMA_DDL', 'VCS_MONITOR', '%', null);
exec DBMS_LOGSTDBY.SKIP('DML', 'VCS_MONITOR', '%', null);
-- NON_SCHEMA_DDL (for grants)
Example: MV logsschema_blue> select table_name from user_tables;TEST_TABLE
schema_blue> delete from test_table where name = 'test1';
ORA-01031: insufficient privileges <= table is guarded
schema_blue> CREATE MATERIALIZED VIEW LOG on test_table 2 with sequence, primary key, rowid 3 (value, timestamp) 4 INCLUDING NEW VALUES;
Materialized view log created.
Example: Materialized Viewschema_orange> create materialized view mv_test_table 2 build immediate 3 refresh fast on demand 4 ENABLE QUERY REWRITE 5 AS 6 SELECT 7 count(*) cnt, 8 count(name) cnt_name, 9 tt.value, 10 trunc(tt.timestamp) trunc_timestamp 11 FROM 12 schema_blue.test_table tt 13 group by 14 tt.value, 15 trunc(tt.timestamp);
Materialized view created.
Fast Refreshable? Check!
schema_orange> SELECT OWNER, 2 MVIEW_NAME, 3 FAST_REFRESHABLE 4 FROM DBA_MVIEWS 5 /
OWNER MVIEW_NAME FAST_REFRESHABLE-------------------- --------------- ------------------SCHEMA_ORANGE MV_TEST_TABLE DIRLOAD_DML
Query Rewrite? Check!
schema_orange> exec DBMS_MVIEW.EXPLAIN_REWRITE ('select count(*) from schema_blue.test_table', 'mv_test_table');
PL/SQL procedure successfully completed.schema_orange> commit;
schema_orange> SELECT MV_OWNER, MV_NAME, QUERY, MESSAGE, PASS FROM REWRITE_TABLE ;[headings omitted]SCHEMA_ORANGE MV_TEST_TABLEselect count(*) from schema_blue.test_tableQSM-01033: query rewritten with materialized view,MV_TEST_TABLEYES
The Logstdby Advantage
LogStdby
Reporting/Batch application
SCHEMA_BLUETABLESMV LOGSINDEXESPL/SQL
SCHEMA_ORANGETABLESINDEXESMATL_VIEWSPL/SQL
Now we can create Materialized View logs directly in the “source” schema without impacting the primary!
Guard Standby (read-only)
SQL Apply — can be slow
• Redo contains uncommitted transactions• Necessary overhead of logical vs. physical
in general• Be sure to read and follow Best Practices
document• Be prepared, in worst case, to recreate your
Logical Standby for reporting!
Recover Perl scriptrecover_logstdby.pl - recover logical standby
Connect to databasealter database start logical standby applyDie on error unless ORA-16105: Logical Standby is already running in
background
While (BehindTime = select trunc( (newest_time - applied_time)*1440 ) from dba_logstdby_progress)
if (BehindTime < specified recovery window)done, successful
if (exceeded timeout)done, unsuccessful, wake me up
Endalter database stop logical standby applyDisconnect
ReferencesCreating a Logical Standby with Minimal Production Downtime
Note:278371.1
Oracle Data Guard Readme for SQL Apply Release 9.2.0.6
Note:286787.1
Oracle Data Guard Readme for SQL Apply Release 10.1.0.3
Note:304059.1
SQL Apply Best Practices (9i) http://otn.oracle.com/deploy/availability/pdf/DataGuardSQLApplyBestPractices.pdf
SQL Apply Best Practices (10g) http://www.oracle.com/technology/deploy/availability/pdf/MAA_WP_10gSQLApplyBestPractices.pdf
Oracle10g Data Guard SQL Apply Troubleshooting
Note:312434.1
ORA-1 Occurring on Logical Standby Note:257356.1
Synchronizing tables in a Logical Standby Database
Note:271455.1
Troubleshooting 9i Data Guard Network Issues
Note:241925.1
Streams and Data Guard Role Transitions http://www.oracle.com/technology/deploy/availability/pdf/MAA_WP_10gDataGuardRoleTransitionsStreams.pdf
Thank you!
Copy of presentation will be available at NoCOUG web site
Questions?