Symmetry Nms Database

SYMMETRY NMS Database Maintenance Guide (v2.40)

033-100692-240001, Issue 1A

The information in this manual is subject to change without notice. All statements, information and recommendations in this manual are believed to be accurate, but are presented without warranty of any kind, expressed or implied. Users must take full responsibility for their use of any products.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, without prior written consent from SR Telecom Inc.

SR TELECOM, AIRSTAR, ANGEL, INSIGHT NMS, METROFLEX, METROPOL, SR500, SR500IP, SWING, SYMMETRY and WL500 are trademarks of SR Telecom Inc. All rights reserved 2005. All other trademarks are property of their owners. Information subject to change without notice.

© 2005, SR Telecom Inc.All rights reserved. 5/30/05Printed in Canada

Head OfficeSR Telecom Inc.8150 Trans-Canada Hwy.Montreal, QuebecCanada H4S 1M5

Tel.: +1 514 335 1210Fax: +1 514 334 77831 888 SRTELECOM (778 3532)(U.S. and Canada)

www.srtelecom.com

http://www.srtelecom.com

3

Chapter 1 Introduction .............................................................................................................................................. 5

1.1 Introduction ......................................................................................................................... 6

1.2 Database Data Files............................................................................................................ 6

1.3 Database Redo Logs .......................................................................................................... 7

1.4 Database Archive Logs ....................................................................................................... 7

1.5 Database Control Files........................................................................................................ 7

1.6 Database Cold Backup ....................................................................................................... 8

1.7 Database Hot Backup ......................................................................................................... 8

Chapter 2 NMS Database Monitor ........................................................................................................................... 9

2.1 Installing the NMS Database Monitor................................................................................ 10

2.2 Scheduling NMS Database Monitor .................................................................................. 11

2.3 Reviewing NMS Database Monitor Reports...................................................................... 122.3.1 Error Alerts ........................................................................................................................ 122.3.2 Tablespaces with < 15% and < 1Gb Free Space.............................................................. 122.3.3 Segments Can’t Allocate NEXT Extent ............................................................................. 132.3.4 Tablespaces with Over 20 Free Fragments ...................................................................... 132.3.5 Segments Nearing Max Extents........................................................................................ 14

2.4 Preventive Actions ............................................................................................................ 152.4.1 Resizing a Data File .......................................................................................................... 152.4.2 Coalescing a Tablespace.................................................................................................. 172.4.3 Increasing a Segment Maximum Number of Extents ........................................................ 182.4.4 Rebuilding a Large Raphael Tablespace .......................................................................... 18

Chapter 3 NMS Database Cold Backup ................................................................................................................. 21

3.1 Installing the NMS Database Cold Backup Tool ............................................................... 22

3.2 Performing an NMS Database Cold Backup..................................................................... 23

Chapter 4 NMS Database Hot Backup................................................................................................................... 25

4.1 Installing the NMS Database Hot Backup Tool ................................................................. 26

4.2 Performing an NMS Database Hot Backup ...................................................................... 28

Chapter 5 NMS Database Restore......................................................................................................................... 29

5.1 Restoring the NMS Database from a Cold Backup........................................................... 30

Table of Contents

4

SYMMETRY NMS Database Maintenance Guide

5.2 Restoring the NMS Database from a Hot Backup............................................................. 32

5.3 Reducing the NMS Database Recovery Time................................................................... 36

5.4 Restoring the NMS Database Completely......................................................................... 37

Chapter 6 NMS Standby Database ........................................................................................................................ 39

6.1 Standby NMS Configuration.............................................................................................. 40

6.2 Oracle Standby Database ................................................................................................. 41

6.3 Setting Up an NMS Standby Database............................................................................. 43

6.4 Monitoring and Managing the NMS Standby Database .................................................... 466.4.1 Using the NMS Database Monitor Tool............................................................................. 466.4.2 Determining the Most Recent Archived Log...................................................................... 476.4.3 Reviewing the Archive Trace Files.................................................................................... 476.4.4 Resolving a Gap Sequence .............................................................................................. 48

6.5 Opening the Standby Database as Read-only .................................................................. 51

6.6 Activating the NMS Standby Database ............................................................................. 536.6.1 Activating the Standby NMS Database ............................................................................. 536.6.2 Recreating the Primary After Standby Activation .............................................................. 55

5

Chapter 1 IntroductionThis chapter introduces some basic database concepts that you need to understand in order to effectively use the tools. Chapter 2 describes a monitor tool that checks the health and state of the NMS database and sends out reports using e-mail. Chapter 3 describes how to install and perform NMS database cold backup, and Chapter 4 describes how to install and perform NMS database hot backup. Chapter 5 describes how to restore an NMS database from backups and suggests the preparation steps that you should follow to reduce the database recovery time as well as restore database 100%. Chapter 6 describes how to prepare, manage, and maintain an Oracle standby database using SYMMETRY NMS.

6


1.1 IntroductionOne of the most important tasks for SYMMETRY NMS system administrators is to protect the NMS database from failures. Particularly, the database must be backed up frequently so that the database can be restored in the event of media failure. SYMMETRY NMS provides tools to help system administrators maintain database operations. This document describes these tools and how to use them.

The SYMMETRY NMS Database Maintenance Guide also contains some information about common database maintenance tasks, including how to purge old Raphael data records, how to purge old NMS alarm history records and event records. It also explains how to schedule these tasks to run automatically. It is important for a system administrator to perform these tasks to ensure proper NMS operations.

d All database system files described in this chapter must not be modified by anyone. Doing so will cause the NMS database to be corrupted.

1.2 Database Data FilesThe database writes its modified buffer cache to files on disk. The files containing user data are called data files. For the NMS database, these files are located in /data/oradata and are updated by NMS applications. If you have installed the entire NMS including Gabriel, Raphael, and Michael Order Manager (MOM), you will see the following subdirectories, corresponding to each of the systems: GMS, RAPHAEL, and MICHAEL.

Introduction

7

1.3 Database Redo LogsAll changes made to data are also recorded as redo entries in the redo log buffer. A change is not committed until its redo entries in the buffer are written to a redo log file on the disk. Redo log files (or simply redo logs) are sometimes called online redo logs (or simply online logs) since the redo entries in these files are not archived.

For the NMS database, the redo log files for Gabriel, Raphael, and MOM are located in /data/oradata/GMS.

1.4 Database Archive LogsAn optional background process called Archiver only starts when the database is run in ARCHIVELOG mode with automatic archiving enabled. A redo log file is archived when the log file is full and the database needs to switch to another redo log file. When this occurs, the Archiver copies the filled redo log file to a specified location. These copied log files are called archive logs and they can be used to recover data when the media fails.

By default, the database runs in NON_ARCHIVELOG mode. In this mode, a redo log file is not archived before it is overwritten. In addition, the database can only be backed up after you have shutdown the database. The archive logs generated by the database running in ARCHIVELOG mode make database online backup possible.

For the NMS database, the archive log files are located in /data/oradata/GMS/archive directory.

1.5 Database Control FilesThe control files help the database synchronize the write operations between the redo log files and the data files. At the synchronization point (check point), all modified data buffers are written to disk. After the check point process is completed, the file headers are updated and marked as current, and a check point sequence number is recorded in the control file. During the database startup, the control file confirms that all the files are at the same check point number.

Control files can not be directly backed up when the database is online. The database is constantly being updated. A simple UNIX copy of the file may not give you a valid copy.

For the NMS database, the control files are located in /data/oradata/GMS.

8


1.6 Database Cold BackupDatabase cold backup is also called closed database backup. You need to shutdown the database before performing the cold backup. Although cold backup makes the database unavailable during the backup, cold backup should still be performed from time to time during maintenance windows. A cold backup can backup the entire database, including data files, redo logs, control files, archive logs, and parameter files. Since the database is not available for use during the backup, all the database files are consistent to a specific point in time. This makes a cold backup extremely reliable and, as a result, you can always restore your database from a good cold backup.

1.7 Database Hot BackupAlthough the cold database backup is reliable and can backup the entire database, the NMS database can not be shut down often to perform backups. If the database runs in the ARCHIVELOG mode with automatic archiving enabled, then you can perform a database hot backup while it is online. Therefore, hot backups should be performed often and cold backups can be performed less frequently.

Hot backup also cleans out the archive log files so that the archive log directory does not become full. The hot backup will copy the archive log files to the backup location and delete them from the original location.

9

Chapter 2 NMS Database MonitorSYMMETRY NMS provides a script that checks the health and state of the NMS database and generates a report. The report is then e-mailed to system administrators. The script can be scheduled as a system task which will enable system administrators to monitor the NMS database activities so that potential issues can be identified and addressed before they become problems. This chapter describes how to install, run, and schedule the SYMMETRY NMS database monitor tool.

10


2.1 Installing the NMS Database MonitorTo install the NMS database monitor

1. Login NMS server as user oracle and copy the supplied monitor.tar file to /tmp.

2. Extract the tar file with the UNIX tar command.

%cd /tmp%tar xvf monitor.tar

This will produce two files, installmonitor.ksh and monitor_app.tar.

3. Run the install script to install and configure the NMS database monitor.

%/tmp/installmonitor.ksh

When prompted, please be sure to enter correct e-mail address(es). NMS database monitor will send reports to the specified e-mail address(es). Multiple e-mail addresses must be separated by a space.

4. To verify the installation, run the DB checkup script manually.

%/export/apps/oracle/admin/db_monitor/scripts/db_CHECKUP.sh

If the installation is successful, you should receive an e-mailed database report.

NMS Database Monitor

11

2.2 Scheduling NMS Database MonitorSchedule the database monitor so that it runs automatically and periodically, for example, once every day. This way, you can be proactive and address the issues before they become problems.

To schedule the NMS database monitor as a periodic UNIX task

1. Login NMS server as user oracle.

2. Open the user oracle cron table with these commands:

%export EDITOR=vi%crontab -e

This opens a UNIX vi editor session.

3. Using vi, add this line on its own line (case-sensitive).

0 1 * * * /export/apps/oracle/admin/db_monitor/scripts/db_CHECKUP.sh 2>&1

4. Exit the vi session to complete the crontab -e command.

This entry will run the NMS database monitor at 0 minute, 1 hour (1 AM), every day. Please consult the Software Installation Setup Guide provided with your SYMMETRY NMS system for details on how to use default UNIX editor vi to add and save the line above.

To verify the entry

Enter the following command as user oracle:

%crontab -l

If you have performed the above steps correctly, you should automatically receive NMS database reports via e-mail, as scheduled.

12


2.3 Reviewing NMS Database Monitor ReportsThe following sections describe some important items that you will find in NMS database monitor reports.

2.3.1 Error AlertsThe error alert section lists all Oracle errors that have been generated since yesterday. For example, the following error alerts indicate that the Oracle database was not able to insert new alarm history records because it was unable to extend the tablespace. Therefore, old alarm history records need to be purged to make room for new ones, or the tablespace needs to made larger.

<< Alert Errors Since Yesterday >>

ORA-1653: unable to extend table GMS_S1.ALARMHISTORY by 1024 in tablespace ALMA_DATAORA-1653: unable to extend table GMS_S1.ALARMHISTORY by 1024 in tablespace ALMA_DATA

All errors in the error alerts section should be investigated and addressed immediately.

2.3.2 Tablespaces with < 15% and < 1Gb Free SpaceAll tablespaces with less than 15% and less than 1 GB space will be listed in this section of the report. For example:

<< Tablespaces with < 15% and < 1Gb Free Space >>

GMS_DATA (3645440 bytes or 2.31% free)

This is a warning that these tablespaces may need to be given more space soon so that they will not run over the currently allocated space limits.

For information about growing tablespaces, refer to Section 2.4.1 “Resizing a Data File” on page 15.


13

2.3.3 Segments Can’t Allocate NEXT ExtentThe first field is the tablespaces name, and the second field is the table name. For example:

<< Segments Can't Allocate NEXT Extent >>

GMS_DATA: GMS_S1.HSDRESOURCE (TABLE, next_extent 8388608)GMS_DATA : GMS_S1.LOG_RECORD (TABLE, next_extent 8388608)

This means that if the Oracle database needs to extend the spaces used by these tables, it will not be able to because there is not going to be enough contiguous free space for the table to grow.

For information about growing tablespaces, refer to Section 2.4.1 “Resizing a Data File” on page 15.

f You can ignore the warnings related to tablespace raphael_dimension_data, for example, RAPHAEL_DIMENSION_DATA : RAPHAEL.DAY (TABLE, next_extent 8388608). This tablespace is no longer used.

2.3.4 Tablespaces with Over 20 Free FragmentsThe first field is the tablespace name, and the second field is the number of free fragments. For example:

<< Tablespaces with Over 20 Free Fragments >>

RAPHAEL_CDR_DATA_P1 has 89 Free Fragments

The available free space for a tablespace can become fragmented. When a segment is dropped, its extents are deallocated and marked as free, but they are not combined to be a larger, contiguous extent. Unless these free extents are coalesced, they may not be used within the tablespace during the next space request.

For information about combining neighbouring free extents, refer to Section 2.4.2 “Coalescing a Tablespace” on page 17.

14


2.3.5 Segments Nearing Max Extents<< Segments Nearing Max Extents >>

RAPHAEL.BASE_CALL_UTIL_MEASUREMENT (TABLE, 90% extents used)

Each segment has a maximum allowable number of extents. Any transaction that causes the table to attempt to exceed its maximum extent number will fail. This section of the monitor report warns you that you need to increase the maximum extent number for a table.

The example report above warns that the table BASE_CALL_UTIL_MEASUREMENT in schema RAPHAEL has reached 90% of its maximum number of extents.

For information about changing a maximum extent number for a segment, refer to Section 2.4.3 “Increasing a Segment Maximum Number of Extents” on page 18.


15

2.4 Preventive ActionsThe following sections describe the common maintenance tasks that you may need to perform when addressing the issues reported in the NMS database monitor report.

2.4.1 Resizing a Data FileAssume that the NMS database monitor report has entries as described in Section 2.3.2 on page 12 or in Section 2.3.3 on page 13.

To increase the size of the correct data file

1. Login as user oracle.

2. Invoke Oracle server manager tool by typing svrmgrl to get SVRMGR prompt.

3. At SVRMGR prompt, type in the following commands. Press the return key or ENTER after each command.

SVRMGR>connect internalSVRMGR>select file_name from dba_data_files where tablespace_name='GMS_DATA';

Note the semicolon at the end of the select command above. The select command gives you a file name. This is the name of the data file for the tablespace. In this example, for tablespace GMS_DATA, the data file would be /data/oradata/GMS/gms_data01.dbf.

4. On a different terminal window, issue the following UNIX command:

ls -l /data/oradata/GMS

� Take note of the file size for /data/oradata/GMS/gms_data01.dbf file. For example, let us assume that it is 150 MB.

16


5. Return to the terminal window with the SVRMGR prompt and type in the following command at the SVRMGR prompt:

SVRMGR>alter database datafile '/data/oradata/GMS/gms_data01.dbf' resize 210M;

Note the single quotes between the file name. The above command will take a short while and it will increase the file size from 150 MB to 210 MB.

6. Type exit to close the svrmgrl tool.

SVRMGR>exit

7. Verify the file size change for /data/oradata/GMS/gms_data01.dbf by entering the following command:

ls -l /data/oradata/GMS

f For Raphael tablespaces such as RAPHAEL_CDR_DATA_P1, you may have to keep increasing the data file sizes as your network deployment grows when CPEs and base stations are added. This is true even you are purging the database to only keep the CDR records for the last 30 days, as described in the Software Installation Setup Guide provided with your SYMMETRY NMS system. If a data file size is close to 1 GB, you should rebuild the tablespace; refer to Section 2.4.4 “Rebuilding a Large Raphael Tablespace” on page 18.


17

2.4.2 Coalescing a TablespaceAssume that the NMS database monitor report has entries as described in Section 2.3.4 on page 13. Coalesce a tablespace to try to reduce space fragments.

To coalesce the tablespace RAPHAEL_CDR_DATA_P1




SVRMGR>connect internalSVRMGR>set heading offSVRMGR>set pagesize 0SVRMGR> select 'alter table ' || owner || '.' || segment_name || ' deallocate unused;' from dba_segments where tablespace_name = 'RAPHAEL_CDR_DATA_P1’;

The above select command provides one or more alter table statements for each table in the tablespace RAPHAEL_CDR_DATA_P1:

alter table RAPHAEL.VOICE_CDR_SEGMENT deallocate unused;alter table RAPHAEL.RU_CDR_SEGMENT deallocate unused;alter table RAPHAEL.VOICE_V52_MEASUREMENT deallocate unused;

4. Cut and paste the above alter table commands at the prompt, one by one, to execute them.

5. Coalesce the tablespace by entering the following commands:

SVRMGR>alter tablespace RAPHAEL_CDR_DATA_P1 coalesce;SVRMGR>exit

18


2.4.3 Increasing a Segment Maximum Number of ExtentsAssume that you have entries in the NMS database monitor report as described in Section 2.3.5 on page 14.

To set the maximum extent number to be unlimited for table BASE_CALL_UTIL_MEASUREMENT




SVRMGR>connect internalSVRMGR>alter table raphael.base_call_util_measurement storage (maxextents unlimited);SVRMGR>exit

2.4.4 Rebuilding a Large Raphael TablespaceRaphael uses SQL*Loader direct path load to insert new records. Direct path load bypasses normal database processing by adding new records after the high-water-mark. This dramatically increases the insertion speed, and it is necessary for Raphael. However, the free space below the high-water-mark is not searched and reused. This is why you may have to keep increasing the size of the datafile, even though the records older than 30 days are being deleted everyday by the purge script.

If a tablespace is becoming too large (close to 1 GB), you should rebuild the tablespace by truncating the tables. Truncating a table removes all records of the table and resets the high-water-mark to the beginning. Of course, the existing records need to be exported first before truncating and imported back after truncating. This section describes the steps for performing this task.

f The steps below use Raphael tablespace RAPHAEL_CDR_DATA_P1 as an example. For a different tablespace, you will need different export and import parameter files, and the truncate and alter table commands below will also be different. Contact your SR Telecom support personnel first.

Before you rebuild a tablespace, you must stop the Raphael loading processes.


19

To stop the Raphael loading processes

1. Login as user gabriel and invoke stopraphael to stop the Raphael loading processes.

2. Confirm that the Raphael parser (dta) and loader processes (rdm) are stopped using the following UNIX command:

ps -ef | grep dta and ps -ef | grep rdm

To rebuild a large Raphael tablespace

1. Login as oracle and create a directory exp_imp.

2. Place the export parameter file (exp_rap_cdr_data_p1) and import parameter file (imp_rap_cdr_data_p1) in the new directory exp_imp.

3. Export the three tables within the RAPHAEL_CDR_DATA_P1 tablespace.

%cd /export/home/oracle/exp_imp%exp sys/live parfile=exp_rap_cdr_data_p1

After the export is completed, there should be a export dump file called cdr_data_p1.dmp and a log file called exp_cdr_data_p1.log.

4. Review the log file to make sure that the export has been finished before you continue. Do not attempt to view the dump file.

5. Truncate the tables and coalesce the tablespace. Truncating a table deletes all the records and also reset its storage parameter to the initial values when the table was created. This will cause the new records to be inserted from the beginning of the data file.

%svrmgrlSVRMGR>connect internalSVRMGR>truncate table RAPHAEL.VOICE_CDR_SEGMENT;SVRMGR>truncate table RAPHAEL.RU_CDR_SEGMENT;SVRMGR>truncate table RAPHAEL.VOICE_V52_MEASUREMENT;SVRMGR>alter tablespace RAPHAEL_CDR_DATA_P1 coalesce;SVRMGR>exit

6. Import back.

%imp sys/live parfile=imp_rap_cdr_data_p1

20


Exporting tables does not take long. Importing takes some time. For example, importing approximately 600,000 records for each of the three tables in the raphael_cdr_data_p1 tablespace took roughly 45 minutes in our testing.

The following are the contents of the export and import parameter files used in the steps above:

File exp_rap_cdr_data_p1:DIRECT=YROWS=YCONSISTENT=YCOMPRESS=NSTATISTICS=NONERECORDLENGTH=57344#THE FOLLOWING PARAMETERS ARE TABLE OR TABLESPACE SPECIFICFILE=cdr_data_p1.dmpLOG=exp_cdr_data_p1.logTABLES=(RAPHAEL.VOICE_CDR_SEGMENT, RAPHAEL.RU_CDR_SEGMENT,RAPHAEL.VOICE_V52_MEASUREMENT)

File imp_rap_cdr_data_p1:FROMUSER=RAPHAELTOUSER=RAPHAELIGNORE=Y#THE FOLLOWING PARAMETERS ARE TABLE OR TABLCESPACE SPECIFICFILE=cdr_data_p1.dmpLOG=imp_cdr_data_p1.log

21

Chapter 3 NMS Database Cold BackupCold backup should be a part of an overall NMS database backup strategy. This chapter describes how to install and perform an SYMMETRY NMS database cold backup.

22


3.1 Installing the NMS Database Cold Backup ToolYou must have already installed the NMS database monitor tool. If not already installed, follow the steps in Chapter 2 to do so before installing cold backup tool.

To install the SYMMETRY NMS database cold backup tool

1. Login NMS server as user oracle and copy the supplied backup_cold.tar file to /tmp.

2. Extract the tar file using the following UNIX tar command:

%cd /tmp%tar xvf backup_cold.tar

This will produce two files, installcoldbackup.ksh and backup_cold_app.tar.

3. Run the install script to install and configure the NMS database cold backup tool.

%/tmp/installcoldbackup.ksh

This installs the cold backup tool.

NMS Database Cold Backup

23

3.2 Performing an NMS Database Cold BackupTo perform a cold backup of the NMS database

1. Login NMS server as user gabriel and shutdown all NMS applications.

%stopgabriel%stopraphael%stopmichael

2. Ensure that all NMS GUI applications are closed, particularly any NMS Admin Viewer and Raphael PM Viewer since they access database directly. Do not shutdown the Oracle database.

f You must leave the database up and running as the cold backup scripts will need to access the database before it starts the backup. The scripts will shutdown the database during the cold backup and will restart the database after the cold backup is completed.

3. Login as user oracle and perform the cold backup.

%su - oracle%cd /export/apps/oracle/admin/backups% ./coldbackup.sh

The time it takes to complete the cold backup depends on the size of the NMS database. You can monitor the backup progress by tailing the backup log file.

24


To monitor backup progress

1. Open another terminal window.

2. Type in the following UNIX commands:

%cd /export/apps/oracle/admin/GMS/bkup/cold% tail -f coldbp_GMS_11Feb04.log

The actual log file name for the tail command will be different. The above example assumes that you are performing the cold backup on Feb 11, 2004. When the cold backup completes, the log file will print lines to inform you. You should receive an e-mail at the e-mail address(es) that you have specified when you installed the NMS database monitor tool.

The backup files are located in /export/apps/oracle/admin/GMS/bkup/cold. This directory also contains the restore scripts, that were generated during the backup, that will be required to restore the database from these backup files. This is the directory that you should tar up and save a copy on another disk or on tape every time that you complete a cold backup.

25

Chapter 4 NMS Database Hot BackupA hot backup can be performed while the database is online. Therefore, hot backups should be performed in addition to cold backups to allow the database to be backed up more frequently. This chapter describes how to install and perform an ANGLE NMS database hot backup.

26


4.1 Installing the NMS Database Hot Backup ToolYou must have already installed the NMS database monitor tool. If not, follow the steps in Chapter 2 to do so before you continue with the steps below.

f You should perform a database cold backup before attempting the steps listed here. This ensures that you can always restore the database to the current state using the cold backup, if necessary. For information about performing a database cold backup, refer to the steps in Chapter 3.

To install the SYMMETRY NMS database hot backup tool

1. Login NMS server as user gabriel and shutdown all NMS applications.

%stopgabriel%stopraphael%stopmichael

2. Ensure that all NMS GUI applications are closed, particularly any NMS Admin Viewer and Raphael PM Viewer, since they access the NMS database directly. Do not shutdown the Oracle database.

f You must leave the database up and running. The hot backup installation scripts need to access to the database. The database will be restarted automatically when the installation completes.

3. Login NMS server as user oracle and copy the supplied backup.tar file to /tmp.

4. Extract the tar file with the UNIX tar command.

%cd /tmp%tar xvf backup.tar

This will produce two files, installbackup.ksh and backup_app.tar.

NMS Database Hot Backup

27

5. Run the install script to install and configure the NMS database hot backup tool.

%/tmp/installbackup.ksh

This script will shutdown the NMS database and restart it in archive log mode to allow for the backing up of the database while it is online. Please be patient while waiting for the script to complete, as a proper database shutdown may take a little while.

6. Verify that the database is in archive log mode.

%svrmgrlSVRMGR>connect internalSVRMGR>archive log list

The output of the last SVRMGR command should show that: the database is in archive mode, the automatic archival is enabled, the archive destination is set to /export/apps/oracle/admin/GMS/multiplex/archives/.

7. Type exit to quit svrmgrl.

SVRMGR>exit%

You have just installed the NMS database hot backup tool. You can now restart NMS applications. Login as user gabriel and follow the steps described in the Software Installation Setup Guide to start Gabriel, Michael Order Manager, and Raphael.

To confirm the successful installation, you should perform a NMS database hot backup right away.

28


4.2 Performing an NMS Database Hot Backup To perform a hot backup of the NMS database

1. Login the NMS server as user oracle.

2. Execute the hot backup script.

%cd /export/apps/oracle/admin/backups% ./hotbackup.sh

The time it takes to complete the hot backup will depend on the size of the NMS database. You can monitor the backup progress by tailing the backup log file.

To monitor backup progress

1. Open another terminal window.

2. Type in the following UNIX commands:

%cd /export/apps/oracle/admin/GMS/bkup/hot% tail -f hotbp_GMS_11Feb04.log

The actual log file name for the above tail command will be different. The above example assumes that you are performing the hot backup on Feb 11, 2004. When the hot backup completes, the log file will print lines to inform you. You should also get an e-mail at the e-mail address(es) that you have specified when you installed the NMS database monitor tool.

The backup files are located in /export/apps/oracle/admin/GMS/bkup/hot. This directory also contains the restore scripts, that were generated during the backup, that will be required to restore the database from these backup files. This is the directory that you should tar up and save a copy on another disk or on tape every time you that complete a hot backup. This step can be automated by using UNIX cron to schedule daily hot backups, followed by simple UNIX tar and move commands.

29

Chapter 5 NMS Database RestoreThis chapter describes how to restore the entire NMS database from backups. It also makes suggestions on what you should do to reduce database restore time so that the NMS database downtime can be minimized.

30


5.1 Restoring the NMS Database from a Cold BackupTo restore the entire NMS database from a cold backup

1. If necessary, reinstall the entire SYMMETRY NMS. This is required if the entire NMS is lost due to a disk crash. You may also need to install and configure Solaris if the damaged disk also hosted the OS. Refer to the Software Installation Setup Guide provided with your SYMMETRY NMS system to install the OS and SYMMETRY NMS. Installing the OS alone may take several hours. To reduce NMS downtime, you should have a second Sun server available, with the OS and the NMS installed and ready to use.

2. If necessary, reinstall the NMS database monitor and backup tools. Follow the installation instructions in Chapter 2 and Chapter 3, respectively. Note that you may need to start up the database first before installing the NMS database monitor and backup tools. This step can be avoided by having another server available with the NMS database monitor and backup tools already installed.

3. Ensure that all NMS applications have been stopped. Login as user gabriel and follow the steps in the Software Installation Setup Guide to shutdown the Gabriel, Raphael, and MOM applications.

4. Login in as user oracle and make sure that the database and its listener have all been stopped. Follow the steps in the Software Installation Setup Guide to check this and issue the proper stop commands. If you are not able to shutdown the database using the SVRMGR shutdown immediate command described in the Software Installation Setup Guide, you will have to use the shutdown abort command.

5. Get the last cold backup files and place them in the following directory: /export/apps/oracle/admin/GMS/bkup/cold. In addition to other files, there are two restore scripts in this directory and their names depend upon the date when the cold backup was taken. Make sure that you are still logged in as user oracle and perform the following two commands to change them to be executable:

%cd /export/apps/oracle/admin/GMS/bkup/cold%chmod u+x restore_cold_backup_11Feb04.sh%chmod u+x restore_cold_backup_GMS_11Feb04.sh

The above commands assume that the cold backup was taken on Feb 11, 04.

NMS Database Restore

31

6. Run the restore scripts in the following order:

%./restore_cold_backup_11Feb04.sh%./restore_cold_backup_GMS_11Feb04.sh

The first script restores the database configuration parameter files and takes very little time. The second script restores the database data files; the time it takes depends on the size of the database. As the second script executes, it reports the progress status and you should start to see the data files appearing in the directories under /data/oradata.

7. After the restore scripts complete the execution, verify that the Oracle files tnsnames.ora and listener.ora have the correct host name. This is particularly important if you are using a cold backup file taken on a different host.

% cd $TNS_ADMIN

You will see the two files in this directory. Make proper changes if the values for variable HOST in these two files do not match the correct name of the host.

8. You can proceed to start up the NMS database and applications. Please follow the instructions in the Software Installation Setup Guide provided with your SYMMETRY NMS system.

32


5.2 Restoring the NMS Database from a Hot Backup

f If you have copies of the latest online redo logs, archive logs, and control files, the hot backup restore is also easier and you can recover the changes made after the hot backup was taken. If this is the case, ensure that you make and store a copy of these file somewhere. Do not put them in /data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex, since these locations will be overwritten during the steps below. Refer to Section 5.4 “Restoring the NMS Database Completely” on page 37.

To restore the entire NMS database from a hot backup

1. If necessary, reinstall the entire SYMMETRY NMS This is required if the entire NMS is lost due to a disk crash. You may also need to install and configure Solaris if the damaged disk also hosted the OS. Refer to the Software Installation Setup Guide provided with your SYMMETRY NMS system to install the OS and SYMMETRY NMS. Installing the OS alone may take several hours. To reduce NMS downtime, you should have a second Sun server available, with OS and NMS installed and ready to use.

2. If necessary, reinstall the NMS database monitor and backup tools. Follow the installation instructions Chapter 2 and Chapter 4, respectively. Note that you may need to start up the database first before installing the NMS database monitor and backup tools. This step can be avoided by having another server available with the NMS database monitor and backup tools already installed.

3. Ensure that all NMS applications have been stopped. Login as user gabriel and follow the steps in the Software Installation Setup Guide to shutdown the Gabriel, Raphael, and MOM applications.

4. Login in as user oracle and make sure that the database and its listener have all been stopped. Follow the steps in the Software Installation Setup Guide to check this and issue the proper stop commands. If you are not able to shutdown the database using the SVRMGR shutdown immediate command described in the Software Installation Setup Guide, you will have to use the shutdown abort command.


33

5. Get the last hot backup files and place them in the following directory: /export/apps/oracle/admin/GMS/bkup/hot. In addition to other files, there is a restore script in this directory and its name depends upon the date when the hot backup was taken. Make sure that you are still logged in as user oracle and perform the following command to change it to be executable:

%chmod u+x restore_GMS_11Feb04.sh

The above command assumes that the hot backup was taken on Feb 11, 04.

f If you have copies of the latest control file, redo log files, and archived redo log files, stop here and do not perform the steps below. Refer to the steps in Section 5.4 “Restoring the NMS Database Completely” on page 37 to continue so that even the changes after the hot backup was taken can be restored.

6. Run the restore script to restore database data files, archive logs, and database parameter files. You must specify the database name GMS on the command line.

%./restore_GMS_11Feb04.sh GMS

7. The hot backup cannot directly copy the database control files because hot backup is performed while database is open and running. The steps given here will create the control files back from a control file trace created during the hot backup. You can avoid losing control files so that these steps here will not be necessary. Having a most recent control file also ensures a safe database recovery. There are ways for maintaining a copy of a recent control file and they will be discussed in a later section. Follow the steps below to recreate the database control files from the trace file.

%cp control_GMS_11Feb04.sql /data/oradata/GMS

This copies the control trace file created during the hot backup to the original location of the control file.

34


8. Edit the file control_GMS_11Feb04.sql in /data/oradata/GMS to comment out the last line by adding -- (two dashes) to the beginning of the line. After this is done, the last line should read as:

-- ALTER DATABASE OPEN RESETLOGS;

It is important to comment out this line before you continue with the next step.

9. Perform the following commands to start the recovery of control files:

%cd /data/oradata/GMS%svrmgrlSVRMGR>connect internalSVRMGR>@control_GMS_11Feb04.sqlSVRMGR>recover database until cancel using backup controlfile;

You will be prompted with a message like:

ORA-00279: change 733543036 generated at 02/11/2004 00:00:10 needed for thread 1ORA-00289: suggestion : /export/apps/oracle/admin/GMS/multiplex/archives/1_99.gmsORA-00280: change 733543036 for thread 1 is in sequence #99Specify log: {<RET>=suggested | filename | AUTO | CANCEL}

10. Type and enter AUTO and all of the archive logs which are present in the backup will be applied. After all of the available logs have been applied, you may see some error message like:

ORA-00308: cannot open archived log '/export/apps/oracle/admin/GMS/multiplex/archives/1_101.gms'ORA-27037: unable to obtain file statusSVR4 Error: 2: No such file or directoryAdditional information: 3

This means that archive log file it is looking for is not there. If this number is higher than the numbers in the archive log directory, it is acceptable because it has not been written to disk yet.

At this point, the database has been recovered. Because the control file was recreated, you must synchronize the data files with the control files and redo log files.


35

11. Enter the following database alter command to start up the database and synchronize files:

SVRMGR>alter database open resetlogs;SVRMGR>exit%

12. At this point you should perform a cold backup of the database before launching the NMS. This ensures that you can easily restore the database up to this point using the cold backup. Follow the instructions provided in Chapter 3 to perform a cold backup.

At the end of the cold backup, the database will be restarted automatically.

13. Restart the listener as user oracle.

%lsnrctl start

14. Verify that you can login database using sqlplus.

%sqlplus gms_s1/gms_s1@GMS

You may need to wait for a minute or so for the above command to work, since Oracle listener takes a little while to start up.

15. Follow the Software Installation Setup Guide provided with your SYMMETRY NMS system to start up the NMS applications.

36


5.3 Reducing the NMS Database Recovery TimeBacking up the NMS database is critical to prevent loss of your network and customer data. Nothing can replace a good backup in the event of an NMS server hardware failure. However, you will not be able to start restoring the NMS database unless the NMS and its database software are properly installed and configured. If you do not have a spare server, in the event of an NMS server hardware failure, you may need to first repair the server and you may also need to reinstall the OS and the NMS. This will incur substantial NMS downtime.

You should have another server to act as an emergency standby. On this server, you should install the same version of the NMS, including the NMS database monitor and backup tools. If your active NMS server failed and could not be repaired quickly, you can immediately start restoring the NMS database on the standby server by invoking the restore script. This will dramatically reduce the NMS downtime.


37

5.4 Restoring the NMS Database CompletelyAlthough it is reliable and complete, the cold backup cannot be performed often because it requires shutting down the database. Hot backup allows the database to stay online and available to users and therefore can be performed more frequently, even daily. However, unless you have redundant copies of the latest online redo logs and archive logs, you may still lose the database updates that have taken place since your last hot backup.

Another factor to consider is the database control file. As mentioned in the hot backup restore section, database control files are key, but they can not be backed up by hot backup since the database is open and running. Therefore the hot backup restore process described above has one step for recreating the control files. This step would not be needed and the restore would be much safer if you have redundant copies of the control files.

To be able to fully restore the NMS database, you need to multiplex the database redo logs, archive logs, and control files.

As soon as you install the NMS database hot backup tool and the database is put in ARCHIVELOG mode, the NMS database will start maintaining two copies of these files. The first set of copies of these files are located in /data/oradata as described in Chapter 1, and the default location of the second set of copies of these files are saved in /export/apps/oracle/admin/GMS/multiplex. These two locations should reside on two different disks so that the failure of one disk does not cause both sets to be lost.

If you have copies of the latest redo logs, archive logs, and control files, the hot backup restore is also easier.

38


To start the database with automatic recovery

1. Make sure that you make and store a copy of the latest redo logs, archived redo logs, and control file somewhere. Do not put them in /data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex, since these locations will be overwritten during steps below.

2. Refer to Section 5.2 on page 32 and complete Step 1 on page 32 to Step 5 on page 33.

3. Run the restore script to restore database the data files, archive logs, and database parameter files from the hot backup. You must specify the database name GMS on the command line.

%./restore_GMS_11Feb04.sh GMS

4. Place a copy of the control files, latest redo log files, and archive log files in both locations, /data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex.

5. Do the following to start up the database with automatic recovery:

%svrmgrlSVRMGR>connect internalSVRMGR>startup mountSVRMGR>recover automatic databaseSVRMGR>alter database open;SVRMGR>exit

This restores the database including the changes that have taken place since your last hot backup. After this step, continue with the remaining steps described Section 5.2 “Restoring the NMS Database from a Hot Backup” on page 32 to perform a cold backup, to start the database listener, and to start NMS applications (Step 12 on page 35 to Step 15 on page 35).

39

Chapter 6 NMS Standby DatabaseSYMMETRY NMS redundancy architecture recommends the use of a standby database. The standby database is synchronized with the primary database and therefore can be activated at any time, as required. This chapter describes how to prepare, manage, and maintain an Oracle standby database with SYMMETRY NMS.

40


6.1 Standby NMS ConfigurationSYMMETRY NMS redundancy architecture recommends using one host as the standby NMS regardless of how many hosts are used for the primary NMS system (see Figure 6.1). On the standby host, you must install the same version of SYMMETRY NMS software as the primary, start the standby database and synchronize it with the active database, but keep the NMS background processes down.

To failover to the standby NMS, you first activate the standby database, and then launch all NMS backgrounds. After all the processes come up, the standby system becomes the new primary and it connects all existing base stations.

Figure 6.1 SYMMETRY NMS Redundancy Configuration

NMS Standby Database

41

6.2 Oracle Standby DatabaseAn Oracle standby database is created from a backup of the primary database and is synchronized by applying the redo logs archived by the primary database. After it is configured for the managed standby environment and managed recovery mode, the primary database will automatically archive the redo logs to the standby host and they are automatically applied to the standby database. See Figure 6.2 and Figure 6.3 on page 42.

Figure 6.2 SYMMETRY NMS Primary Database and Standby Database

42


Figure 6.3 Failover to Standby Database


43

6.3 Setting Up an NMS Standby Database

f To use the standby NMS feature, you must install the NMS database monitor, cold backup, and hot backup tools on the primary database. If you have not done so, you should first install them on the primary host and perform some monitor and backup operations before continuing to set up the standby database. Refer to Chapter 2, Chapter 3, and Chapter 4.

To prepare the NMS standby database

1. Install a complete SYMMETRY NMS system on the standby host, including the NMS database maintenance packages described in previous chapters.

2. Take a complete backup of the primary database

� Shutdown the primary NMS background processes.

� Perform a cold backup.

� Keep the processes down.

3. From the primary database, create a control file for the standby database and then shutdown the primary database.

% svrmgrlSVRMGR>connect internalSVRMGR>alter database create standby controlfile as ‘/tmp/control01.ctl’;SVRMGR>shutdownSVRMGR>exit%

44


4. Restore the standby database using the primary database backup.

� Place the backup file in /export/apps/oracle/admin/GMS/bkup/cold on the standby host.

� Run the restore scripts. Refer to the procedure in Section 5.1 on page 30.

� Since you are using a backup from one host and restoring it on another, remember to change the Oracle TNS files tnsnames.ora and listener.ora to use the standby host name in these two files, as described in Section 5.1 on page 30.

� Do not start the standby database at this time.

5. Place the standby control file that you created above into the following two directories on the standby host: /data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex/control.

6. Configure the standby database to automatically apply the archived redo logs to be received from the primary database. As user oracle on the standby host, place the standby package standby.tar that you are given in the Oracle home directory /export/home/oracle, extract the tar file, and then run the setup script for the standby database.

% cd /export/home/oracle% tar xvf standby.tar% cd stdby_config/scripts/setup% ./setupstandby.ksh


45

7. Start the standby database listener, mount the database as standby, and then place it in managed recovery mode. As user oracle on the standby host:

% lsnrctl start listener_standby_gms% svrmgrlSVRMGR>connect internalSVRMGR>startup nomountSVRMGR>alter database mount standby database;SVRMGR>recover managed standby database

f After you execute the above command, the SVRMGR prompt does not return and stay on the next line. This is normal. Do not press CTRL+C. Pressing CTRL+C will cancel the managed recovery, and the logs from the primary database will not be applied. Closing the terminal window does not affect the managed recovery; the recovery continues in the background.

8. On the primary host, configure the primary database to automatically archive the redo logs to the standby database. As user oracle, place the standby package standby.tar that you are given in the Oracle home directory /export/home/oracle, extract the tar file, and then run the setup script for the primary database.

% cd /export/home/oracle% tar xvf standby.tar% cd stdby_config/scripts/setup% ./setupprimary.ksh

9. Start the primary database, its listener, and all NMS processes. Refer to the Software Installation Setup Guide provided with your SYMMETRY NMS system for the correct steps.

This completes the setup steps for both the primary and the standby database. When it is time to archive a redo log file on the primary database, it will also be archived and applied to the standby database.

46


6.4 Monitoring and Managing the NMS Standby DatabaseYou should monitor the archiving status of your primary and standby database to check that the standby database is indeed synchronized with the primary database. This will help make sure that no data is lost if the primary host fails and you need to perform a failover. This section describes how to check the synchronization status and what to do if the standby database is no longer synchronized with the primary database.

6.4.1 Using the NMS Database Monitor ToolRecall that the NMS database monitor generates a database report that you can configure to automatically send you the report daily. If the primary database is not able to archive redo logs to the standby, there will be errors in the report’s Alert section. For example:

<< Alert Errors Since Yesterday >>

ARC0:Error 12543 connecting to standby host ‘STANDBY_GMS’ARC0: Error 12543 creating archive log file ‘STANDBY_GMS’

These errors inform you that the primary database is not able to connect to the standby database, possibly due to some network issues.


47

6.4.2 Determining the Most Recent Archived LogAs user oracle, run the script maxseq.ksh to query the database for the most recent archived redo log.

% cd /export/home/oracle/stdby_config/scripts/utils% ./maxseq.ksh

If the archiving of redo logs from the primary to the standby databases is working properly, the value you get from both the primary and standby databases should match.

6.4.3 Reviewing the Archive Trace FilesThere is a trace file on both the primary and standby hosts that records the redo log archiving activities. On the primary host, they are in directory /export/oracle/admin/GMS/bdump and have names like gms_arc0_20210.trc. On the standby host, they are in directory /export/oracle/admin/GMS/udump and have names like gms_rfs_4361.trc. You may want to review recent files when you are troubleshooting as they contain detailed information generated from the database’s archiving background process (ARC).

48


6.4.4 Resolving a Gap SequenceThe Oracle database assigns a sequence number for each redo log file. When a primary database archives a redo log file but it is not able to transfer it to the standby host, the standby database will miss a set of archived redo log files. The sequence numbers for the these missed archived logs are called gap sequences. A gap sequence is usually caused by some network problem between the primary host and the standby host.

Another common reason that can cause a gap sequence is when you shutdown the standby database while the primary database is still up and open. Therefore, you should avoid shutting down the standby database without first shutting down the primary database.

Even after you fix the network problems and the transfer of the archived logs resumes, the newly archived logs are not going to be automatically applied to the standby database unless you resolve the gap sequence manually. Therefore, when a gap sequence occurs, the standby database is no longer synchronized with the primary database. You should resolve the gap sequence as soon as possible so that the log files received from the primary database can again be applied automatically.

To identify and resolve the gap sequence


2. Identify the gaps by running the script gap.ksh on the standby host.

% cd /export/home/oracle/stdby_config/scripts/utils% ./gap.ksh

Let us assume the script produces the following result:

THREAD# LowSeq# HighSeq#----------------------------1 460 463

If there is no gap, then the script should return either no result, or the LowSeq# will be equal to the HighSeq#.


49

3. Using FTP or making a CD, transfer the archived log files from seq# 460 to seq# 463 in /data/oradata/GMS/archive on the primary host to the standby host and place them in both of the following directories: /data/oradata/GMS/archive and /export/apps/oracle/admin/GMS/multiplex/archives.

f You must place these log files in both directories, otherwise the step below will fail and you will receive error messages about not finding a log file in the directory.

4. On the standby host, cancel the managed recovery that is still in progress and then apply the log files within the gap sequences.

% svrmgrlSVRMGR> recover managed standby database cancelSVRMGR> recover automatic standby database

5. After recovering the available logs, Oracle prompts for the name of a log that does not exist. The reason is that the recovery process does not know about the logs archived to the standby site by the primary database. For example, you might see:

ORA-00308: cannot open archived log ’/oracle/standby/standby_logs/arcr_1_464.arc’ORA-27037: unable to obtain file statusSVR4 Error: 2: No such file or directoryAdditional information: 3Specify log: {<RET>=suggested | filename | AUTO | CANCEL}

50


6. Cancel the recovery.

SVGMGR> cancel

You may see these messages (they are acceptable):

Media recovery cancelled.ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error belowORA-01194: file 1 needs more recovery to be consistentORA-01110: data file 1: ’some_filename’ORA-01112: media recovery not started

7. You are now ready to put the managed recovery mode back on.

SVRMGR> recover managed standby database

The standby database will now again be able to automatically apply the archived logs to be received from the primary database.


51

6.5 Opening the Standby Database as Read-onlyWhen you place the standby database into recovery mode by executing the recover managed standby database command, the database does not allow users to connect to the database to perform queries. Sometimes you may want to query the standby database to make sure that the records match the primary database. You can do this by placing the standby database in read-only mode. But you must first cancel the managed recovery mode before you can place the database in read-only mode.

The log files received from the primary database while the standby database is in read-only mode will not be applied, and this will create gap sequence situation described in Section 6.4.4 “Resolving a Gap Sequence” on page 48. Therefore, you should avoid placing the standby database in read-only mode while the NMS background processes on the primary host are up, and may cause the updating of the primary database.

To place the standby database in read-only mode


2. Cancel the managed recovery.

% svrmgrlSVRMGR> connect internalSVRMGR> recover managed standby database cancel

3. Open the database in read-only mode.

SVRMGR> alter database open read only;SVRMGR> exit%

Now you can start SQL*Plus sessions on the standby host the same way you do on the primary host, and perform SQL select commands against any Gabriel, Raphael, and MOM tables.

You should put the standby database back to managed recovery mode as soon as you have completed your queries.

52


To place the standby database back to managed recovery mode

1. Make sure that you exit all SQL*Plus sessions that you opened during the read-only mode.

2. Issue the following statements:

% svrmgrlSVRMGR> connect internalSVRMGR> recover managed standby databaseSVRMGR> exit%

If you have not caused a gap sequence while the database was in read-only mode, the managed recovery will continue when a new log file is received from the primary database. If there is any gap sequence, you must resolve the gap; refer to Section 6.4.4 “Resolving a Gap Sequence” on page 48.


53

6.6 Activating the NMS Standby DatabaseActivating a standby database is a permanent action and it can not be undone. As soon as it is activated, the standby database will be completely “destroyed”, and you would have to go through all the steps described in Section 6.3 “Setting Up an NMS Standby Database” on page 43 if you want to create the standby database again. Therefore, you should not activate your standby database just to verify if your standby database is being synchronized with the primary database.

You should also make a complete cold backup as soon as possible after you activate your standby database. All of your previous hot backups of your old primary database are no longer valid after you have activated your standby database.

6.6.1 Activating the Standby NMS DatabaseFollow these steps to activate the standby NMS database.

To activate the standby NMS database

1. Cancel the ongoing managed recovery, activate the standby, and then shut down all standby database background processes. As user oracle on the standby host:

% lsnrctl stop listener_standby_gms% svrmgrlSVRMGR> connect internalSVRMGR> recover managed standby database cancelSVRMGR> alter database activate standby database;

d If you receive an error message from the above command, please stop here and follow the steps described in Step 2 on page 54. If not, continue with Step 3 on page 54.

SVRMGR> shutdown immediateSVRMGR> exit%

54


2. You may get error messages when you execute the above alter database activate standby database command. If you do, please follow these steps to restart the standby database, try the alter database command again, and then shutdown the standby database, before you proceed to Step 3 on page 54 for starting up the new primary database:

SVRMGR> shutdown immediateSVRMGR> exit% svrmgrlSVRMGR>connect internalSVRMGR>startup nomountSVRMGR>alter database mount standby database;SVRMGR> alter database activate standby database;SVRMGR> shutdown immediateSVRMGR> exit%

3. Start the new primary database. As user oracle on the activated standby (the new primary) host

% lsnrctl start listener_gms% svrmgrlSVRMGR> connect internalSVRMGR> startup mountSVRMGR> alter database open;SVRMGR> exit%

You can now proceed to start all NMS background processes on the new primary host. As soon as you have done this, the new primary NMS systems are ready for use by users.

f If your Raphael system is standalone (i.e., the Raphael system is running on its own host), you may also need to run the script gabrielhost_script to configure the new primary database so that Gabriel will continue to send CDR data to the Raphael system. Please consult the Software Installation Setup Guide provided with your SYMMETRY NMS system.


55

6.6.2 Recreating the Primary After Standby ActivationYou may want to recreate the primary NMS system on a new host, or on the original primary host after the problems which caused the failover have been resolved. Please refer to proper NMS manuals for each of the steps listed below.

To recreate the primary NMS system on a new host

1. On the new primary host, you may need to reinstall all SYMMETRY NMS software, such as Gabriel, Raphael, MOM, as well as database maintenance packages.

2. Make a cold backup of the activated standby database (the current primary system).

3. Restore the cold backup to the original primary database by following the steps in Section 5.1 “Restoring the NMS Database from a Cold Backup” on page 30.

4. Shut down the activated standby database and NMS processes on the current primary host.

5. Start up the restored database on the original primary host and NMS processes.

Users can now access the original primary host as their SYMMETRY NMS systems.

6. Make a cold backup of the primary database and use the backup to recreate the standby database.

Documents

Symmetry Nms Database