Upload
dinhdan
View
223
Download
0
Embed Size (px)
Citation preview
Backup and Recovery-Exchange 2010 DAG
1
Backup and Recovery – Exchange 2010 DAG
By Francis Sabarinath [frasa03]
Backup and Recovery-Exchange 2010 DAG
2
Table of Contents BACKUP AND RECOVERY – EXCHANGE 2010 DAG ........................................................................................ 3
1. Exchange Backup................................................................................................................................... 3
2. Backup Sequence Diagram – using Store Writer .................................................................................. 3
3. Backup Sequence Diagram – using Replica Writer ............................................................................... 4
4. Difference between VSS backup of Exchange 2010 using Windows Server Backup and that done by
CA ARCserve Agent for Microsoft Exchange ............................................................................................. 4
5. Important Tool for Troubleshooting Exchange Backup issues – MS Event Viewer .............................. 5
6. User Problems related to Backup ......................................................................................................... 8
6.1 User Problem 1 -Exchange Database Level Backup fails ................................................................. 8
6.2 User Problem 2 -Exchange Database Level Backup fails - Client halt error .................................... 8
6.3 User Problem 3 - Transaction Logs not truncated .......................................................................... 9
6.4 User Problem 4 – Unable to find Exchange Database .................................................................. 10
7. Exchange Restore ................................................................................................................................ 10
8. Restore Sequence Diagram – only Store Writer is used and not Replica Writer ................................ 11
9. User Problems related to Restore ....................................................................................................... 11
9.1 User Problem 1-Restore using Recovery Database – How to extract the Mailbox ...................... 11
9.2 User Problem 2 –Unable to find the Recovery Database ............................................................. 12
Backup and Recovery-Exchange 2010 DAG
3
BACKUP AND RECOVERY – EXCHANGE 2010 DAG
1. Exchange Backup Microsoft provides 2 ways of backup – legacy streaming and VSS. Legacy streaming backup is not
supported with Exchange 2010. The VSS communicates with the Windows Server File system and with
the mass storage device driver through a third-party (or customer) provider. The backup/restore
application is also called the VSS Requestor. The requestor communicates with the VSS to obtain
information about the Exchange 2010, to command the creation of shadow copies and to gain access to
the data for backup. To support the VSS, Exchange 2010 includes an Exchange writer that it built into the
Exchange Store. The Exchange Writer coordinates with the Exchange Store, to freeze and dismount the
database before backing it up, and then to unfreeze and mount the database after the backup is
complete. Backup from the active copy uses the Store Writer, and backup from the replica (copy
database) uses the Replica Writer. It is good to understand the Backup sequence using the Store and the
Replica writer which is diagrammatically provided in the following section.
2. Backup Sequence Diagram – using Store Writer
Backup and Recovery-Exchange 2010 DAG
4
3. Backup Sequence Diagram – using Replica Writer
Ref: http://msdn.microsoft.com/en-us/library/exchange/aa579076(v=exchg.140).aspx
4. Difference between VSS backup of Exchange 2010 using Windows Server
Backup and that done by CA ARCserve Agent for Microsoft Exchange
The sequence of backup will be the same except that a physical consistencies check that will be initiated
by ARCserve Exchange Agent. Our agent uses the exported functions from chksgfiles.dll to check the
consistency.
The CHKSGFILES API ships as part of Exchange 2010. The API can be used on a computer that has
Mailbox server role installed, or a computer that has the Administrator-only install. CHKSGFILES is
available only as a 64-bit DLL in Exchange 2010. By default, the CHKSGFILES DLL is installed in the
C:\Program Files\Exchange 2010\bin directory.
The backup/restore application may choose to validate the integrity of a shadow copy backup set.
During a backup operation managed by the Volume Shadow Copy Service, there is no opportunity for
Microsoft Exchange Server 2010 to read each database file in its entirety and verify its checksum
integrity. Therefore, database and transaction log file integrity should be verified by the backup
application. Microsoft recommends that the backup application verify the physical consistency of the
Backup and Recovery-Exchange 2010 DAG
5
shadow copy set prior to informing the Exchange Writer that the backup is complete. Upon successful
backup, Exchange store updates the headers of the backed-up databases to reflect the last successful
backup times and removes transaction logs from the server that are no longer required to roll forward
from the last successful backup.
In order to provide better granularity and control during the consistency check process, Exchange 2010
includes a set of unmanaged public APIs packaged into a DLL (chksgfiles.dll). The CHKSGFILES DLL is
installed with all Exchange server roles as well as part of the Admin-only option to enable running
consistency checks off of an Exchange server on a proxy backup machine.
More Reference:
http://msdn.microsoft.com/en-us/library/bb204049.aspx
http://msdn.microsoft.com/en-us/library/bb204219.aspx
5. Important Tool for Troubleshooting Exchange Backup issues – MS Event
Viewer Microsoft Event Viewer (application events) is an important tool to be used for troubleshooting an
Exchange Backup. All the events will be recorded in sequence – from the Backup initiation till the end.
Sequence of the Exchange Backup can be understood by looking at the application event during an
Exchange VSS backup. Provided below is a sample set of application events, from a backup taken from
the replica, and it is important to understand this sequence for troubleshooting purpose:
Replica Events:
SNo TIME SOURCE EVENT ID
TASK Category
DESCRIPTION
1 3:27:34 AM MSExchangeRepl 2021 Exchange VSS Writer
The Microsoft Exchange VSS Writer has successfully collected the metadata document in preparation for backup.
2 3:27:38 AM MSExchangeRepl 2110 Exchange VSS Writer
The Microsoft Exchange VSS Writer instance cfda266f-d071-47fd-9295-8ec31213f378 has successfully prepared for a full or a copy backup of database 'FRANCISTEST'. The following database will be backed up: FRANCISTEST.
3 3:27:38 AM MSExchangeRepl 2023 Exchange VSS Writer
The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-d071-47fd-9295-8ec31213f378) successfully prepared for backup.
4 3:27:49 AM MSExchangeIS Public Store
9539 General General "The Microsoft Exchange Information Store database ""<NULL>"" was stopped. “
5 3:27:49 AM ESE BACKUP 960 General msexchangerepl (2444) This computer is performing a surrogate backup. The master server is USSDWIN2K8R1.
6 3:27:49 AM MSExchangeRepl 2027 Exchange VSS Writer
The Microsoft Exchange VSS Writer instance cfda266f-d071-47fd-9295-8ec31213f378 has successfully frozen the databases.
7 3:27:49 AM MSExchangeRepl 2025 Exchange VSS Writer
The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-d071-47fd-9295-8ec31213f378) successfully prepared for a snapshot.
8 3:27:49 AM MSExchangeRepl 2027 Exchange VSS Writer
The Microsoft Exchange VSS Writer instance cfda266f-d071-47fd-9295-8ec31213f378 has successfully frozen the databases.
Backup and Recovery-Exchange 2010 DAG
6
9 3:27:49 AM MSExchangeRepl 2029
Exchange VSS Writer
The Microsoft Exchange VSS Writer instance cfda266f-d071-47fd-9295-8ec31213f378 has successfully thawed the databases.
10 3:27:50 AM MSExchangeRepl 2035 Exchange VSS Writer
The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-d071-47fd-9295-8ec31213f378) has successfully processed the post-snapshot event.
11 3:27:54 AM Storage Group Consistency Check
100 Initialization Instance 1: A physical consistency check has been successfully initiated. Database '\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\FRANCISTEST.edb' and the transaction log files in '\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\' with a base name of 'E05' will be validated.
12 3:27:54 AM Storage Group Consistency Check
200 Database Header Validation
Instance 1: Database headers have been successfully validated. All databases are in a dirty shutdown state. To bring these databases to a clean shutdown state, log generations 3 (0x00000003) to 3 (0x00000003) will be required.
13 3:27:57 AM Storage Group Consistency Check
300 Log File Validation
Instance 1: Transaction log files in '\\? \GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\' with a base name of 'E05' have been successfully validated. No errors were detected.
14 3:27:57 AM ESE 103 General "caagstart (788) Instance 1: Log File Integrity Check (\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\E05): The database engine stopped the instance (0). Internal Timing Sequence: [1] 0.000, [2] 0.000, [3] 0.000, [4] 0.000, [5] 0.000, [6] 0.000, [7] 0.000, [8] 0.000, [9] 0.000, [10] 0.000."
15 3:27:57 AM ESE 105 General "caagstart (788) Instance 1: Log File Integrity Check (\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\E05): The database engine started a new instance (0). (Time=0 seconds) Internal Timing Sequence: [1] 0.000, [2] 0.000, [3] 0.000, [4] 0.000, [5] 0.000, [6] 0.000, [7] 0.000, [8] 0.000, [9] 0.000, [10] 0.000, [11] 0.000.“
16 3:27:57 AM ESE 102 General "caagstart (788) Instance 1: Log File Integrity Check (\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\E05): The database engine (14.00.0692.0000) is starting a new instance (0).
17 3:27:57 AM Storage Group Consistency Check
400 Termination Instance 1: The physical consistency check has completed successfully. All files were validated and no errors were detected.
18 3:28:47 AM MSExchangeRepl 2021 Exchange VSS Writer
The Microsoft Exchange VSS Writer has successfully collected the metadata document in preparation for backup.
19 3:29:03 AM ESE BACKUP 963 General msexchangerepl (2444) The surrogate backup procedure to USSDWIN2K8R1 has been successfully completed.
20 3:29:03 AM MSExchangeRepl 2046 Exchange VSS Writer
"The Microsoft Exchange Replication service VSS Writer instance cfda266f-d071-47fd-9295-8ec31213f378 has successfully completed the backup of database 'FRANCISTEST'. Database log truncation has been requested for this database. Log truncation will occur on the active copy after the next log generation is created. Log truncation will occur automatically on the passive copies after that log file is copied."
21 3:29:03 AM MSExchangeRepl 2033 Exchange The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-
Backup and Recovery-Exchange 2010 DAG
7
VSS Writer d071-47fd-9295-8ec31213f378) has successfully processed the backup completion event.
22 3:29:03 AM MSExchangeRepl 2037 Exchange VSS Writer
The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-d071-47fd-9295-8ec31213f378) backup has been successfully shut down.
Active Events:
SNo TIME SOURCE EVENT ID TASK Category DESCRIPTION
1 3:27:51 AM ESE 210 Logging/Recovery "Information Store (3400) FRANCISTEST: A full backup is starting.
2 3:27:51 AM ESE BACKUP 910 General Information Store (3400) A full surrogate backup is starting. This server is the master. The requestor is USSDWIN2K8R1.
3 3:29:04 AM MSExchangeRepl 3156 Service Active Manager successfully mounted database FRANCISTEST on server ussdwin2k8r1.gotir.ca.com.
4 3:29:04 AM ESE BACKUP 913 General Information Store (3400) The surrogate backup procedure has been successfully completed by USSDWIN2K8R1.
5 3:29:04 AM ESE 213 Logging/Recovery "Information Store (3400) FRANCISTEST: The backup procedure has been successfully completed.
Will the transaction logs be purged from the Active or the Replica – when an Exchange backup is
performed via the Replica?
The log truncation will happen on the Active Database only and not from the Copy databases.
Once the backup is completed from the Replica, it will signal the Active database. The logs will be
truncated from the Active database and sync will take place with the copies, which will affect the
transaction logs on the Copy databases.
Backup initiated transaction log file truncation will be triggered based on the type of backup chosen. In
non-DAG configurations, the Store Writer will truncate the transaction log files at the completion of
successful Full or Incremental backups. In DAG replicated configurations, log truncation will be delayed
by the Replication service until all necessary log files are replayed into all other copies. The Replication
service will delete the backed up log files both from the active and the copy log file paths after it verifies
that the to-be-deleted log files have successfully been applied to the copy database and both active
database and the database copies checkpoint has passed the log files to be deleted.
Log truncation doesn't occur on the active mailbox database copy when one or more passive copies are
suspended. If your planned maintenance activities are going to take an extended period of time (for
example, several days), you may have considerable log file buildup. To prevent the log drive from filling
Backup and Recovery-Exchange 2010 DAG
8
up with transaction logs, you can remove the affected passive database copy instead of suspending it.
When the planned maintenance is completed, you can re-add the passive database copy.
6. User Problems related to Backup
6.1 User Problem 1 -Exchange Database Level Backup fails
From the Application event, check at which step the backup failed.
Run a Windows Server Backup on that mailbox database to narrow down the case.
Refer the link below on how to run a VSS backup of Exchange using Windows Server Backup:
http://technet.microsoft.com/en-us/library/dd876854(v=exchg.141).aspx
6.2 User Problem 2 -Exchange Database Level Backup fails - Client halt error
The following errors will be observed from the Application event and it will occur during the backup
(after sequence 18 from the Application Events in section 5)
Active Events: Event ID: 215
Source: ESE Category: Logging/Recovery Level: Error Message: "Information Store (3084) EXCHDB1_08: The backup has been stopped because it was halted by the client or the connection with the client failed.
Replica Events: Source: ESE BACKUP Event ID: 914 Category: General Level: WarningMessage: General Information Store (3084) The surrogate backup by VMEXCHANGEMBS1A has stopped with error 0xFFFFFFFF.
In this case as well, run a Windows Server Backup of Exchange, as explained above to narrow down the case. If the Windows Server backup is successful, then do the steps below to disable TCP Offloading
TCP Offloading has to be disabled on the NIC from the Backup Server and the Exchange Nodes.
On the Backup servers:
IPv4 Checksum Offload Off
IPv4 Large Send Offload Disabled
IPv6 Checksum Offload Off
IPv6 Large Send Offload Disabled
Backup and Recovery-Exchange 2010 DAG
9
Receive Side Scaling Disabled
On the Exchange Nodes:
IPv4 Checksum Offload Disabled
IPv4 Giant TSO Offload Disabled
IPv4 TSO Offload Disabled
IPv4 TCP Segmentation Offload Disabled
Offload IP Options Disabled
TCP Checksum Offload (IPv4) Disabled
TCP Checksum Offload (IPv6) Disabled
6.3 User Problem 3 - Transaction Logs not truncated
Truncation of Transaction logs takes place after a full or an incremental backup of the Exchange
Database. If there is an issue where the transaction logs are not getting truncated, then the step is to
run a Windows Server backup of the Exchange Database to check if the logs are getting truncated.
If the transaction log truncation does not take place from the Windows Server Backup of the Exchange,
then check the status of the copy databases, in case of an Exchange 2010 DAG.
Log truncation doesn't occur on the active mailbox database copy when one or more passive copies are
suspended. If your planned maintenance activities are going to take an extended period of time (for
example, several days), you may have considerable log file buildup. To prevent the log drive from filling
up with transaction logs, you can remove the affected passive database copy instead of suspending it.
When the planned maintenance is completed, you can re-add the passive database copy.
Backup and Recovery-Exchange 2010 DAG
10
6.4 User Problem 4 – Unable to find Exchange Database
When the backup is initiated, you will get an error as “E3577 Cannot find DB on Exchange server”
In such cases, run the adsiedit.msc and use the same domain controller and credentials as provided in
the ARCserve Backup UI. Mostly, there will be ldap connectivity issues, or the Exchange containers
cannot be seen. The solution is to use a different domain controller or credentials, so that the Exchange
objects can be viewed correctly.
A sample screenshot of the adsiedit.msc is provided below:
7. Exchange Restore During a restore, the backup/restore application instructs the Exchange Writer to coordinate with the
Exchange Store to dismount the database, replace the database files, mount the database and then
replay the transaction logs as needed. When restoring, the requestor also communicates with the VSS to
prepare the system for the restore, and then to put the data back on to the mass storage device. The
backup/restore application is also responsible for working with the Windows Server to read data from
and write data to the backup storage media, whether a tape archive, a storage area network, or the
other backup medium. The restored database can be mounted either as a regular, active database, or as
the Exchange 2010 Recovery Database. Only one mounted database can be designated as a recovery
database on each Exchange Server.
Backup and Recovery-Exchange 2010 DAG
11
8. Restore Sequence Diagram – only Store Writer is used and not Replica
Writer
Ref: http://msdn.microsoft.com/en-us/library/exchange/bb204079(v=exchg.140).aspx
9. User Problems related to Restore
9.1 User Problem 1-Restore using Recovery Database – How to extract the Mailbox
After the mailbox database is restored to the Recovery Database, ensure that it is mounted.
Mount the Recovery Database either from Exchange Management Console or from Exchange Management Shell by running the command Mount-Database "Recovery Database"
Make sure the destination mailbox is enabled and existing in the Exchange Server. Make sure the database corresponding to the destination mailbox is mounted.
Before proceeding to recover the mailbox contents from Recovery Database, run the following command to get the details of mailboxes available in the Recovery Database.
Get-MailboxStatistics –Database "Recovery Database"
Backup and Recovery-Exchange 2010 DAG
12
The above command can be run to make sure that the mailbox that we are trying to restore is existing in the
Recovery Database.
Restore a single mailbox
1. Run the following command to restore the content of mailbox 'production mailbox' in the Recovery
Database to the production mailbox:
Restore-Mailbox –Identity "production mailbox" –RecoveryDatabase "Recovery Database"
2. Say 'Yes' for the confirmation asked. Restore process will start immediately.
3. When completed, you will get statistics about the restore mailbox job. You can see the information about
the source and target mailbox database etc.
Restore a single mailbox to a different mailbox
1. Run the following command to restore the content of 'production mailbox' in the Recovery Database to
'test mailbox' under the folder 'Recovered Contents Folder' within 'spare mailbox':
Restore-Mailbox -Identity "test mailbox" -RecoveryDatabase "Recovery Database" -
RecoveryMailbox "production mailbox" -TargetFolder "Recovered Contents Folder"
NOTE : The above command will create a new folder "Recovered Contents Folder" inside "test mailbox"
and will restore the content of "production mailbox" into the new folder "Recovered Contents Folder"
2. Say 'Yes' for the confirmation asked. Restore process will start immediately.
3. When completed, you will get statistics about the restore mailbox job. You can see the information about
the source and target mailbox database etc.
http://msexchangeguru.com/2010/05/23/exchange2010-recovery-database/
http://technet.microsoft.com/en-us/library/ee332351(v=exchg.141).aspx
9.2 User Problem 2 –Unable to find the Recovery Database
When a restore to a Recovery Database is attempted, it will fail with “unable to find the Recovery
Database”. In order to narrow down the problem do the following:
Have the adsiedit.msc installed. Then connect to the same Domain Controller and check if the
containers can be displayed, and check if the “Recovery Database” created can be viewed from the
Active Directory (Refer 6.4 User Problem 4 for a screenshot of the adsiedit.msc)