Cluster Shared Volume – Effective Virtual Machine Migration 

Mohammed  Yasser A K 

Dell Windows OS Engineering 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table of Contents 

 

Overview 

  What is CSV? 

  Benefits of CSV 

  Requirements – Hardware 

  Requirements – Software 

Creating a CSV 

  Creating and Configuring Virtual Disks on storage array 

  Create and Manage a Cluster using Failover Cluster Manager 

  Enabling Cluster Shared Volume 

Creating Virtual Machines on CSV 

  Hyper‐V Manager Settings when creating a VM on CSV 

Configure Virtual Machines Service on the Cluster 

  Virtual Machine Service Configuration 

  Live Migration 

  Quick Migration 

Known Issues of Live and Quick Migration 

Tools and References 

 

 

 

DISCLAIMER THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT OF THIS WHITE PAPER ARE PROVIDED AS IS, WITH NO WARRANTIES, EXPRESS OR IMPLIED. DELL EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY. UNDER NO CIRCUMSTANCES, INCLUDING, BUT NOT LIMITED TO, NEGLIGENCE, SHALL DELL BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES, INCLUDING, BUT NOT LIMITED TO, LOSS OF DATA OR PROFIT, ARISING OUT OF THE USE, OR THE INABILITY TO USE, THE INFORMATION, EVEN IF DELL OR A DELL AUTHORIZED REPRESENTATIVE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.  AT THE TIME OF WRITING THIS PAPER, WINDOWS SERVER 2008 R2 AND WINDOWS 7 ARE BETA PRODUCTS.  THE  FEATURES  AND  FUNCTIONALITY  OF  THESE  PRODUCTS  AS  DESCRIBED  IN  THIS  PAPER MAY  CHANGE  IN  THE FUTURE.  © 2009 Dell Inc. All rights reserved. Trademarks used in this text: Dell, PowerEdge, EqualLogic and PowerVault are trademarks of Dell, Inc.; Microsoft, Windows Server, Hyper‐V, and Windows are registered trademarks of Microsoft Corporation. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and trade names other than its own. March 2009 

 

 

 

 

 

 

 

 

 

 

 

Overview 

What is CSV? 

CSV – Definition:

a) Windows 2008 R2’s CSVs are shared disks in available storage (LUN) that is visible and accessible to all nodes.

b) These shared disks appear as folders in %systemroot%\ClusterStorage folder. c) Each volume is labeled Volume1, Volume2 and so on.

Limitations in a non-CSV setup:

a) Export and Import of Virtual Machines is slow and is not automated b) A VM cannot be moved from one host to another without interrupting services running

inside the VM. c) No failover modes to ensure availability of Virtual Machines in an event of failed Network

Link, IO or Virtualization Hosts. d) In a SAN setup, each disk or LUN is owned by a single node at any point in time, and

only that node can perform I/O to it leading to a complex setup NTFS – Reason for the above limitations:

a) NTFS is not a clustered file system and concurrent access from systems to a NTFS volume is not allowed.

b) In a failover cluster, only one node is active for a service/application, and the LUN containing pertinent data is owned by that active node.

c) If a service/application has to be moved from one node to another, the volume has to unmounted and remounted to the new active node which would cause downtime.

d) This was the only blocking factor for Live Migration feature to work on Microsoft Hyper-V

Fig:1 – NTFS Volume Access

  

Benefits of CSV 

How will Cluster Shared Volume help?

a) Virtual Machines are stored onto a subfolder of one of the Volumes. b) The great feature of CSV is that all nodes in the cluster can access the content of a CSV

concurrently. c) Each VM is placed in its own sub-folder of a volume which in turn enables Live Migration

to work efficiently.  How is CSV managed and used?

a) For each CSV, one node in the cluster acts as the MAIN/COORDINATOR node. b) All nodes trying to write metadata to a CSV send the metadata updates to this main node

for the volume. c) All nodes in the cluster can write normal block level information to the volume directly

which is the majority of write activity. d) CSV is actually implemented via the CSVFilter.sys file system mini-filter driver which is

responsible for the interception of the NTFS metadata requests and all I/O requests in the event a node loses communication with the target volume.

e) A node can ask the coordinator to perform all its I/O if it can no longer communicate with the target volume.

  

Requirements – Hardware 

CPU and Memory requirement

a) Processors should support Virtualization Technology (Intel VT or AMD-V) b) Do not mix processors from different vendors, based on different family from the same

vendor, difference in number of cores and features.

c) Live and Quick Migration might fail when trying to migrate from newer family of a specific processor vendor to older version

d) Have matching hardware components for best cluster performance e) Make sure all nodes have enough spare memory to take control of another VM when its

parent node fails

Storage Requirement:

a) Make sure the SAN storage array is compatible to Windows 2008 R2 Clustering Service (Refer to Microsoft website for more details on storage compatibility – This is to ensure the miniport driver used for the storage will work with Microsoft Storport storage driver)

b) Storage Arrays are configured using its respective management software like MDSM or Navisphere Utility. Test for their access from disk management before proceeding with cluster creation.

c) Storage disks that are going to be part of CSV should not be a boot or system volume d) Supported storage interfaces include iSCSI, FC and SAS

For more details on configuring SAN storage arrays from Dell refer to links below,

a) iSCSI - MD3000i – Click here, EMC Storage – Click here, Dell EqualLogic Storage – Click

here b) FC – Click here c) SAS – Click here

Network Requirement

a) Have separate network card for storage (iSCSI) and network communication b) SAS setup would have limitations to the number of nodes in a cluster

Requirements – Software 

CSV Requirement:

a) Nodes that will be part of the cluster should be running the latest and same build of Windows Server 2008 R2

Cluster Requirement:

a) Do not mix Full and Server Core Edition (Standard Edition does not support Failover Clustering)

b) Make sure all nodes are configured to connect to a domain controller and a shared storage (Only FC, SAS and iSCSI is supported from Microsoft)

c) Make sure the entire Cluster is validated with no errors d) All nodes should have Failover Clustering Feature and Hyper-V Manager Role installed

 

Fig:2 – Complete Hardware Setup  

 

 

 

Creating a CSV 

Creating and Configuring Virtual Disks on storage array 

Storage Management Software/Utility:

a) Each Storage array vendor provides a management software or utility that enables administrators to create and configure virtual disks from a management station

b) Make sure the firmware on the storage array is upto date to ensure compatibility with Windows Server 2008 R2

c) Storage Arrays would also require administrators to install host drivers on each node before proceeding

d) In iSCSI and FC interfaces, host to virtual disk mapping has to be done using this utility. For more details, refer to the storage array manual

e) iSCSI storage would need iSCSI initiator usage on all hosts to get access f) Use Disk Management within the Operating System to ensure the disks are listed,

initialized and in online state g) Create partitions and format it with NTFS 

Create and Manage a Cluster using Failover Cluster Manager 

Failover Cluster Manager:

a) This feature in Windows Server 2008 onwards will help administrators build redundancy of services and applications on their network and eliminate single points of failure.

b) This helps reduce downtime, guard against data loss, and reduce total cost of ownership. c) The feature is not installed by default. It has to be added using Server Manager on all

nodes that will be part of the Cluster

Creating a Cluster:

a) Open Failover Cluster Manager through Administrative Tools from on one of the Nodes to Create and Manage the Cluster

b) Click on Create a cluster and it will take the administrator through below steps • Select servers (Nodes) that will be part of the cluster

• The wizard would give warnings if the nodes do not meet the requirements of being part of the cluster. Some warnings include;

1. The node is already part of another Cluster

2. OS types (Full and Core) do not match 3. Failover Clustering service is not added on the node

4. Node is not configured to the domain 5. Node is not available or cannot be reached

• The wizard will prompt the user to run the Cluster Validation Wizard that will test the complete setup including Network, SAN and Node Configurations

c) Review the Cluster Validation Report • Any errors on the wizard will block the creation of the Cluster • Warnings would still allow me to create a cluster but might impact different

failover levels on the cluster

d) Manage your cluster

 

Enabling Cluster Shared Volume 

Enabling Cluster Shared Volume and Adding disk to it

a) Refer to figure above where it shows on how to Enable CSV on a Cluster in Failover Cluster Manager.

b) Add available storage to the CSV using the option of Add Storage. There should be enough available storage to be added to the CSV. You cannot add storage assigned for Disk Quorum Settings

c) Check Disk Management on all nodes and you would see the state changed to Reserved

d) On Failover Cluster Manager, check for the current owner of the CSV. This is the

Active/Coordinator Node I was referring to in the beginning of the article

 

 

 

 

Creating Virtual Machines on CSV 

Hyper‐V Manager Settings when creating a VM on CSV 

Node Configuration:

a) Make sure Hyper-V Role is added on all the Nodes using Server Manager b) Name for Virtual Network Switch on all Nodes should be the same

Virtual Machine Configuration: a) If pass-through disks are configured for a specific VM on a node, make sure the disk is

on the same network as other nodes b) Make sure Integration Components are installed on all the guest Operating Systems

c) Mapped network drives and Image files should be on the same network as other nodes d) Virtual Machine Configuration Settings and respective VHDs to be stored on the

C:\ClusterStorage\Volume path

e) Make sure Remote Desktop is enabled on all the Virtual Machines for purpose of testing

Live Migration f) Test for Virtual Machine functionality through Remote Desktop from another client

Configure Virtual Machine Service on the Cluster 

Virtual Machine Service Configuration 

Prerequisites:

a) Before configuring Virtual Machine Service to the cluster, make sure the above steps were considered while creating the Virtual Machine

b) Turn off all the virtual machines that you intend to add to the cluster

Adding Virtual Machine Service:

a) Click on Configure a Service or Application under the cluster you had created

b) Select Virtual Machine and Click on Next c) A small validation check is performed in the background to check if all the nodes on the

cluster have Hyper-V Role added and they share the same Hypervisor version

d) If VMs are in the running state, the wizard will display an error as below;

e) It will list all virtual machines on the cluster and prompt the user to select f) Once selected, it will validate every virtual machine and generate a report

g) Report would have warnings or errors if any of the steps mentioned above for VM creation wasn’t followed

Virtual Machine Summary and Ownership Details:

a) Each Virtual Machine Properties will provide settings for configuring Failover and setting preferred owners

b) Virtual Machines can be managed directly from the Failover Cluster Manager c) Administrators will also have the option to manually perform Live or Quick Migration of

Virtual Machines if a planned downtime is set for any node

 

 

 

 

 

 

 

 

 

 

Live Migration 

Definition:

a) This is a process that can be initiated by user or be automatic in event of failure wherein the VM state and settings are migrated from one node to other

b) Users or Clients accessing the VM will still be having access to the VM during this process c) Slower than Quick Migration but is seamless

Steps Involved:

a) Scenario: The complete hardware and VM setup is shown as below with three VMs being hosted by Node 1 on the Cluster.

b) Step 1: For migrating VM2 hosted by Node1 to Node2, the process first starts of with creating and transferring a full memory copy of the current VM state

c) Step 2: After this, Dirty pages of memory that include the current changes/process on VM starts getting copied

d) Step 3: Cluster Manager should show the Migration to be complete and the respective VM host ownership would be changed to Node 2

During all of the above, the Remote Client/User has still access to the VM and applications on it without any interruption.

Quick Migration 

Definition:

a) This is a process that can be initiate by user wherein the VM goes into a Saved state b) Users or Clients accessing the VM will temporarily lose access to the VM till the migration

is complete c) Much faster than Live Migration and best suited when the node that is hosting the VM

has a planned downtime or when migration was planned

 

 

 

Known Issues of Live and Quick Migration 

Error 1: VM Migration between Intel and AMD– Error Message:

Reason: Working as designed

Workaround: Do an Export and Import of Virtual Machine (Move VMs through Failover Cluster Manager) instead of Quick and Live Migration

Error 2:

VM Migration between different families of same processor vendor of different families may fail

Reason: Working as designed

Workaround: Enable Migration through processors where features do not match on the Virtual Machine Processor Setting. Please note this process is not reversible.

Tools and References http://www.microsoft.com/virtualization/default.mspx