Hyper-V Backup: A look under the hoodSoumya Das Bhaumik (soumyadb@microsoft.com)Program ManagerMicrosoft Corporation

SESSION CODE: VIR322

OVERVIEW

Basics

Hyper-V VSS writer

Clustered Shared Volumes and Hyper-V

Hyper-V backup and Storage Array technologies

GOALS OF HYPER-V BACKUP

Enable centralized backup application to orchestrate backup of VMs in the physical machine

Ensure application consistency of applications within backed up VMs

Participate in the VSS framework for backup

Challenges to backup scenarios in the absence of VSS:

For applications:Lack of a standard way to describe data and backup restore semantics

For backup applications (Requesters):Excessive number of APIs to deal withOpen filesPoint in time consistency of a group of filesHW subsystems have varying capabilities; brittle interfaces

VSS - BASICS

VSS – THE BIG PICTURE

Exchange

Volume Shadow Copy Service (VSS)

Backup ApplicationISV Apps

Writer Metadata

Windows (volsnap.sys)SAN Vendors

IIS

Writer

Hyper-V

FilesComponentsSettings

FilesComponentsSettings

FilesComponentsSettings

FilesComponentsSettingsState Restore

VSSProviders

HYPER-V APPLICATION LEVEL BACKUP

Application-levelRun backup inside the VM as if it were a physical machine

Application-specific granularityCapture data on iSCSI/pass-through media

Parent

VSS-aware App

VSS

BackupApp

VM

HYPER-V HOST LEVEL BACKUPHyper-V VSS Writer

Host levelBackup run from OS in physical machine

Leverage “container” provided by a VM Single Agent for all VMs in a physical machine

HYPER-V WRITER METADATA

Global Authorization Manager Config file

List of Virtual Machines.

Subcomponents of each VM:Configuration XMLChain of virtual hard disksSaved State fileMemory .bin fileList of Snapshots, each of which contain:

Configuration XMLSaved State file.avhd files

8

9

PREPARING FOR A VSS SNAPSHOTEnsuring Application consistency

Requestor (Backup App)

VSS Hyper-V VSS Writer Application (SQL, Exchange)

VSSHyper-V VSS IC (guest requestor)

Parent VM

PrepareForSnapshot

time

Take snapshot

Take snapshot

Quiesce the app

PrepareForSnapshotDoSnapshotSet

DoSnapshotSet

NEED FOR A POST SNAPSHOT STEP

10

Time lag between snapshot in guest and snapshot in hostPotential data corruption if left as isNeed the post-snapshot step to fix the data

Requestor (Backup App)

VSS Hyper-V VSS Writer Application (SQL, Exchange)

VSSHyper-V VSS IC (guest requestor)

Parent VMtime

Take snapshot

Take snapshot

Δt

THE POST-SNAPSHOT STEP

11

Snapshots need to be auto-recoverable to allow this to work

Parent Volume

snapshot

Vhd

Vhd Mounted as a volume on the host

Parent

Vhd

Vhd

The VM is currently running against this VHD

THE RESTORE PROCESSPurely a host-side process

12

Hyper-V VSS WriterPreRestore:

Check if VM exists and turnoff and delete VM

Backup ApplicationRestores the files and fixes the ACLs

Hyper-V VSS WriterPostRestore:

Check if all files were restoredRegister the virtual machineCheck for restores to alternate locations and update configFix Network configuration

ONLINE BACKUP

Requires the VSS IC to be installed in the guest

Communicates with the VSS writers running in the VM to prepare for a snapshot

Creates the Volume snapshot while the VM is running

Ensures file system and application consistency of the snapshot in the guest

13

BACKUP USING SAVED STATE

Used when the Integration components are not installed or the Guest OS does not support VSS

Linux Windows 2000

Saves the state of the VM before taking a snapshot in the host

Does not communicate with the writer in the guest

Ensures crash consistency of the writers running in the VM

14

ONLINE BACKUP VS. BACKUP USING SAVED STATE

Decision to take online snapshot is based on:

If the VSS IC is running in the guest

If the guest is in the running state at the time when the snapshot is requested.

Storage configuration in guest:The shadowstorage of each volume is explicitly set to be in the same volumeThe Guest has only basic disks configuredAll of the fixed disks in the guest have a volumes that support snapshots.

15

Hyper-V in WS08 (v1): If one VM fails to back up, the backup operation for all the VMs fails.

Hyper-V in WS08 R2:Partial Failure.Allows for backup to continue backing up even if some of the VMs fail to back up.

16

HYPER-V VSS PARTIAL FAILURE

CLUSTER SHARED VOLUMES (CSV) AND VSS

Coordination NodeThe cluster node where the CSV volume is mounted.

Direct I/OSend NTFS metadata operations to the coordination node over the network while sending data directly to the disk class driver for the storage device

17

HOW DIRECT I/O WORKS

VHD VHD

Storage

NTFS

CSV Filter

Storage

NTFS

CSV Filter

Move VM

Direct I/O

VM 1 VM 2 VM 2

Storage

NTFS

CLUSTER SHARED VOLUMES (CSV) AND VSS

Coordination NodeThe cluster node where the CSV volume is mounted.

Direct I/OSend NTFS metadata operations to the coordination node over the network while sending data directly to the disk class driver for the storage device

Redirected I/OSend NTFS metadata as well as data over the network. Data is not sent directly to the disk.

19

CSV BACKUP

VHD VHD

CSV Filter CSV Filter

ClusterPrepareSharedVolumeForBackup

VHD

VM 3VM 1 VM 2

VHD VHD VHD

Backup App

All I/O to this CSV volume is re-directed over SMB until backup is complete

(System) or snapshot complete (H/W)

Re-directed I/O

NTFS

Storage

NTFS

Storage

NTFS

Storage

Direct I/O

BACKUP PROCESS TIMELINE - SYSTEM PROVIDER

21

Backup Timeline...

ClusterPrepareSharedVolumeForBackup() BackupComplete Event

Backup/Snapshot Timeline using System Provider (VolSnap)

Volume mounted locallyDirect I/O suspended

CSV Volume Must Be Local and Direct I/O Disabled During Entire Backup Procedure

Volume can be moved, Direct I/O can be enabled

BACKUP PROCESS TIMELINE - HARDWARE PROVIDER

22

Backup Timeline...

ClusterPrepareSharedVolumeForBackup()

VSS Freeze VSS Thaw

PostSnapshot Event

Backup/Snapshot Timeline using Hardware Provider

Volume mounted locallyDirect I/O suspended

CSV Volume Must Be Local and Direct I/O Disabled During

Snapshot

Volume can be movedDirect I/O enabled

HYPER-V OFF-HOST BACKUPProduction Server Backup Server

Storage Arrays

DBs1TB

Logs200 GB

DBs1TB

Logs200 GB

FAST RECOVERY – LUN RESYNC

LUN 1’

V’:

LUN 1

V:Snapshot creation

LUN 1’

V’:

LUN 2

R: Resync

LUN 1’

V’:

LUN 2

V:

FAST RECOVERY – LUN SWAP

Production Server

Storage Array

DBs1TB

Logs200 GB

DBs1TB

Logs200 GB

If you do not care about maintaininga pristine snapshot

Backup and Recovery using diskshadow

DEMO

Diskshadow script for backupset context persistentset metadata c:\diskshadowdata\demo.cabset verbose onbegin backupadd volume c: alias SystemVolumeShadow

create

expose %SystemVolumeShadow% x:

#Custom script to copy the data from shadow volumeexec c:\diskshadowdata\backupscript.cmd

end backup

Diskshadow script for Restoreset context persistentload metadata c:\diskshadowdata\demo.cabset verbose on

begin restore

#Custom script to copy data from backupexec c:\diskshadowdata\restoreScript.cmd

end restore

BACKUP PRODUCTSVendor/Partner Product version Value add features

(in addition to support of Hyper-V and CSV)DPM DPM 2010 Checksum based incrementals

Item level recoveryAuto protection of new VMs

Symantec Symantec BackupExec 2010

Application granular recovery Auto protection of new VMsAuto exclude powered off VMs from backup

NetApp NetApp Snap Manager for Hyper-V (SMHV)

Integrates with NetApp SnapMirror to get cross site DR

Computer Associates

CA ARCServe Auto protection of new VMsItem level recoveryIncremental backupsDedupe of VM backup sessions

A POINT ABOUT PASS-THROUGH DISKS

Pass-through and iSCSI disks are not backed upCurrently left out from the list of volumes in a VM that need to be backed up.Technical limitations in storage stack prevent us from supporting this.

30

CALL TO ACTIONFeedback on what you would like to see in Hyper-V backup going forward

APPENDIX - IMPORTANT HYPER-V QFES

Windows Server 2008 R2KB975354: Hyper-V VSS writer roll-up

Ability to exclude application writers in an online backupAbility to run post-snapshot when CSV volume is not localAlternate location recovery of Hyper-V snapshots

Windows Server 2008 SP2If you have Windows Server 2008 RTM, upgrade to SP2.KB975925:

Ability to include only the system writers in an online backup

32

APPENDIX - IMPORTANT WINDOWS SERVER 2008 R2 VSS QFES

33

KB Description

KB975921 VSS needs to clean up on-disk shadow copy metadata flags after failed commit during shadow copy creation

KB975803 shadow copies deleted after consecutive revert operations on multiple system software shadow copies

KB976099 shadow copy creation fails after ResyncLuns

KB975354 Parallel VM backups fail on CSV volumes if CSV ownership is taken over by another node right after shadow copy operation is complete

KB975688 VSS Hardware shadow copy corruption – VSS doesn’t throw an error if provider takes longer than 10 seconds for shadow copy creation

APPENDIX - CLUSTER API’S FOR CSV BACKUP

ClusterIsPathOnSharedVolumeIs this path on a CSV?

ClusterGetVolumePathNameSame as standard GetVolumePathName

ClusterGetVolumeNameForVolumeMountPointSame as standard GetVolumeNameForVolumeMountPoint

ClusterPrepareSharedVolumeForBackupWas PrepareVolumeForSnapshotSet

Makes volume local and “pins” volume to nodeEnables Re-directed I/O

ClusterClearBackupStateForSharedVolumeClear CSV backup state

34

Related Content

Breakout SessionsVIR01-INT - Disaster Recovery and Virtualization Protection with Microsoft System Center Data Protection Manager (DPM) 2010VIR312 - Realizing a Dynamic Datacenter Environment with Windows Server 2008 R2 Hyper-V and Partner Solutions

Virtualization Track ResourcesStay tuned into virtualization at TechEd NA 2010 by visiting our event website, Facebook and Twitter pages. Don’t forget to visit the Virtualization TLC area (orange section) to see product demos, speak with experts and sign up for promotional giveawaysMicrosoft.com/Virtualization/Events Facebook.com/Microsoft.VirtualizationTwitter.com/MS_Virt Like this session? Write a blog on 2 key learning's from this session and send it to #TE_VIR and you could win a Lenovo IdeaPad™ S10-3 with Windows 7 Netbook! Review the rules on our event websiteMicrosoft.com/Virtualization/Events

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

Complete an evaluation on CommNet and enter to win!

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st

http://northamerica.msteched.com/registration

You can also register at the

North America 2011 kiosk located at registrationJoin us in Atlanta next year

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

JUNE 7-10, 2010 | NEW ORLEANS, LA