Windows Server 2012 Hyper-V Architecture Deep Divedownload.microsoft.com/download/5/C/E/5CEB23E0-C814-4E4B-ADE… · One or more computers running Windows Server 2012 with the Hyper-V

Windows Server 2012 Hyper-V Architecture

Deep Dive

Natthaphol Suntudkarn, MCITP Server Administrator 2008, Messaging 2010

MCTS Windows Server 2008, Virtualization, System Center Virtual Machine Manager 2008

RICOH (Thailand) Limited

Title of Presentation

• Speaker Name, Title

• Microsoft Corporation2

More secure multitenancy

Features

3



• Microsoft Corporation

• IT organizations and hosting providers have begun offering infrastructure as a service (IaaS) to customer

• IT organizations and hosting providers must offer customers enhanced security and isolation from one another

4




• Isolating different departments’ or customers’ virtual machines can be a challenge on a shared network

Scale beyond VLAN with Hyper-V Network Virtualization

5

Contoso Bank Woodgrove Bank

Multiple customers

on shared infrastructure

Finance Sales

Multiple business units

on shared infrastructure

Multi-Tenant

Datacenter




• Traditionally, VLANs have been used to isolate networks, but VLANs are very complex to manage on a large scale Cumbersome reconfiguration of production switches

VLANs have limited scalability

VLANs cannot span multiple subnets


6

VLAN tags

Aggregation

Switches

VMs




Virtual machine isolation with PVLANs

• When a virtual machine doesn’t need to communicate with other virtual machines

• Use PVLANs to isolate it from other virtual machines in your datacenter

7




Hyper-V Extensible Switch • Is a layer-2 virtual network switch that provides

security and isolation capabilities Multitenant virtual machine isolation through private virtual LANs

(PVLANs).

Protection from Address Resolution Protocol/Neighbor Discovery (ARP/ND) poisoning (also called spoofing).

Protection against Dynamic Host Configuration Protocol (DHCP) snooping and DHCP guard.

Isolation and metering using virtual port access control lists (ACLs).

The ability to trunk traditional VLANs to virtual machines.

Monitoring.

Windows PowerShell/Windows Management Instrumentation (WMI).

8




• Add more functionality to their virtual machines and networks

• Such as adding firewalls, intrusion detection systems, and network traffic monitoring tools

• Extensions are implemented by using Network Device Interface Specification (NDIS) filter drivers and Windows Filtering Platform (WFP) callout drivers

• Open platform plug-in for firewalls, antivirus software, diagnostic software, and other types of applications and services

9




Type of Extension

10




• Virtual machine IP address assignment presents other key issues when organizations move to the cloud: Required renumbering of service workloads

Policies that are tied to IP addresses

Physical locations that determine virtual machine IP addresses

Topological dependency of virtual machine deployment and traffic isolation

• When moving to the cloud, the addresses must be changed to accommodate the physical and topological restrictions of the datacenter


11




• Hyper-V Network Virtualization enables you to isolate network traffic from different business units or customers on a shared infrastructure

• Without having to use VLANs

• Allow to move virtual machines as needed within your virtual infrastructure while preserving their virtual network assignments

• use Network Virtualization to transparently integrate these private networks into a preexisting infrastructure on another site.


12





13

• Network Virtualization to isolate network traffic that belongs to two different customers

• a Blue virtual machine and a Yellow virtual machine are hosted on a single physical network, or even on the same physical server

• Because they belong to separate Blue and Yellow virtual networks, the virtual machines cannot communicate with each other even if the customers assign these virtual machines IP addresses from the same address space





14

To virtualize the network, Hyper-V Network Virtualization uses the following elements:

• Two IP addresses for each virtual machine Customer Address, CA IP address that the customer assigns based on the customer’s own

intranet infrastructure

Provider Address (PA) is the IP address that the host assigns based on the host’s physical network infrastructure

• Generic Routing Encapsulation (GRE)

• IP address rewrite

• Policy management server





15

• Generic Routing Encapsulation (GRE)

• Windows Server 2012 uses GRE IP packets to map the virtual network to the physical network

• The GRE IP packet contains the following information: One customer address per virtual machine.

One provider address per host that all virtual machines on the host share.

A Tenant Network ID embedded in the GRE header Key field.

Full MAC header.





16

• IP Address Rewrite

• Hyper-V Network Virtualization uses IP Address Rewrite to map the CA to the PA

• Each virtual machine CA is mapped to a unique host PA

• This information is sent in regular TCP/IP packets on the wire

• Little impact on performance.





17





18

Network Virtualization benefits include:

• Tenant network migration to the cloud with minimum reconfiguration or effect on isolation

• Tenant virtual machine deployment anywhere in the datacenter

• Works with today’s hardware (servers, switches, appliances)

• Full management through Windows PowerShell and WMI

Flexible Infrastructure

Features

19







Hyper-V over SMB

21

• Enabling Hyper-V to use SMB file shares for virtual storage

• Inexpensive to deploy

• Offer performance and features

• Hyper-V over SMB require: One or more computers running Windows Server 2012 with the Hyper-V

and File and Storage Services roles installed.

A common Active Directory infrastructure. (The servers running Active Directory Domain Services do not have to run Windows Server 2012.)

Failover clustering on the Hyper-V side, or the File and Storage Services side, or both. Failover clustering is not required

Hyper-VHyper-VHyper-VHyper-VHyper-VHyper-V

File Server

File Server

Shared

Storage

Hyper-V

SQL

Server

IIS

VDI

Desktop

Hyper-V

SQL

Server

IIS

VDI

Desktop

Hyper-V

SQL

Server

IIS

VDI

Desktop

Hyper-V Cluster

File Server Cluster




Hyper-V over SMB, SMB Transparent Failover

22

• Failover transparent to server application Zero downtime – small IO delay during failover

• Supports planned and unplanned failovers HW/SW Maintenance

HW/SW Failures

Load Rebalancing

• Resilient for both file and directory operations

Requires:

• Windows Failover Clusters

• Both server running application and file server cluster must be Windows Server 2012

• Shares enabled for ‘continuous availability’

File Server Cluster

Hyper-V

Failover share - connections and handles lost,

temporary stall of IO

2

2

Normal operation1

Connections and handles auto-recovered

Application IO continues with no errors3

1 3

File Server

Node AFile Server

Node B

\\fs1\share \\fs1\share




Hyper-V over SMB, SMB Multichannel

23

• Full Throughput scalable connections to SMB shares Bandwidth aggregation with multiple

NICs

Multiple CPUs cores engaged when NIC offers Receive Side Scaling (RSS)

• Automatic Failover SMB Multichannel implements end-to-end

failure detection

Leverages NIC teaming (LBFO) if present, but does not require it

• Automatic Configuration SMB detects and uses multiple paths

Multiple RDMA NICsMultiple 1GbE NICsSingle 10GbE RSS-capable NIC

SMB Server

SMB Client

SMB Server

SMB Client

SMB Server

SMB Client

Sample Configurations

Multiple 10GbE in LBFO team

SMB Server

SMB Client

LBFO

LBFO

Switch10GbE

NIC10GbE

NIC10GbE

Switch10GbE

NIC10GbE

NIC10GbE

NIC10GbE

NIC10GbE

Switch1GbE

NIC1GbE

NIC1GbE

Switch1GbE

NIC1GbE

NIC1GbE

Vertical lines are logical channels, not cables

Switch10GbE/IB

NIC10GbE/IB

NIC10GbE/IB

Switch10GbE/IB

NIC10GbE/IB

NIC10GbE/IB

Switch10GbE




Hyper-V over SMB, SMB Direct (SMB over RDMA, Remote Direct Memory Access)

24

• Advantages Scalable, fast and efficient storage access

High throughput with low latency

Minimal CPU utilization for I/O processing

Load balancing, automatic failover and bandwidth aggregation via SMB Multichannel

• Scenarios High performance remote file access for application servers like

Hyper-V, SQL Server, IIS and HPC

Used by File Server and Clustered Shared Volumes (CSV) for storage communications within a cluster

• Required hardware RDMA-capable network interface (R-NIC)

Three types: iWARP, RoCE and Infiniband

SMB Client SMB Server

SMB ServerSMB Client

User

Kernel

Application

Disk

R-NIC

Network w/

RDMA support

NTFS

SCSINetwork w/

RDMA support

R-NIC




Hyper-V over SMB, SMB Scale-Out

25

• Targeted for server app storage

Example: Hyper-V and SQL Server

Increase available bandwidth by adding cluster nodes

• Key capabilities:

Active/Active file shares

Fault tolerance with zero downtime

Fast failure recovery

CHKDSK with zero downtime

Support for app consistent snapshots

Support for RDMA enabled networks

Optimization for server apps

Simple management

Single File System Namespace

Cluster Shared Volumes

Single Logical File Server (\\FS\Share)

Hyper-V Cluster

(Up to 64 nodes)

File Server Cluster

(Up to 8 nodes)

Data Center Network(Ethernet, InfiniBand or combination)




Hyper-V over SMB, configuration option

26




Storage Space

27

• The cost to acquire and manage highly available and reliable storage can represent a significant part of information technology budgets

• Storage Spaces as part of the Windows Server 2012 storage platform

• Provides an alternative option for companies that require advanced storage capabilities at a lower price




Storage Space

28

• Spaces introduces a new class of sophisticated storage virtualization enhancements

• Storage pools : Virtualized units of administration that are aggregates of physical disk units

• Storage spaces : Virtual disks with associated attributes that include Level of resiliency

Thin or fixed provisioning




Storage Pools

Physical Disk UnitsSAS or SATA in JBOD

Enclosure aggregated to create

a storage pool




Storage Spaceor Virtual Diskspartitioned and formatted with NTFS

Storage Spacesupportsmirror and parity mode resiliency

Thin or Fixed provisioning




Storage Space

31

• Reduction in costs with storage

• Mixed deployment SSD and SAS

• Resilient storage, mirror or parity and Designated Hot-Spares in a Pool

• Thin or Fixed provisioning

• Integrated with failover clustering and Cluster Shared Volumes (CSV)




Migrate virtual machines without downtime

32

• Move virtual machines whenever necessary without disrupting business

• Live migration outside a clustered environment

• Simultaneous migration, perform multiple simultaneous live migrations to quickly move many virtual machines

• Faster migration, live migrations are now able to utilize higher network bandwidths (up to 10 Gigabit) to complete migrations faster.




Live migration outside clustered environment, SMB-based live migration

33

In this instance, each virtual machine’s hard disk is stored on a central SMB file share. You then

perform a live migration of the virtual machines from one server to another while their storage

remains on the central SMB share.

1. Live migration setup

A. source host creates a TCP connection

with the destination host

B. This connection transfers the virtual

machine configuration data to the

destination host.

C. A skeleton virtual machine is set up on

the destination host





34

2. Memory page transfer, the memory

that is assigned to the migrating

virtual machine is copied over the

network from the source host to the

destination host





35

3. Moving the storage handle from source to

destination, control of the storage that is

associated with VM such as any virtual hard disk

files or physical storage attached through a

virtual Fibre Channel adapter, is transferred to

the destination host

4. Bringing the virtual machine online on the

destination server: the destination server has the

resume VM operation

5. Network cleanup:, the migrated virtual machine

runs on the destination server sent a message to

the network switch, which causes it to obtain the

new the MAC addresses of the migrated virtual

machine




Live migration outside clustered environment, Shared-nothing live migration

36

1. During the move operation, disk reads and

writes go to the source virtual hard disk

2. While reads and writes occur on the source

virtual hard disk, the disk contents are

copied over the network to the new

destination virtual hard disk.

3. After the initial disk copy is complete, disk

writes are mirrored to both the source and

destination virtual hard disks

In this case, the live migration of a virtual machine from one non-clustered Hyper-V host to another begins

when the virtual machine’s hard drive storage is mirrored to the destination server over the network.




Live migration outside clustered environment, Shared-nothing live migration

37

4. After the source and destination virtual

hard disks are synchronized, the virtual

machine live migration is initiated, following

the same process that was used for live

migration with shared storage.

5. After the live migration is complete and the

virtual machine is successfully running on

the destination server, the files on the

source server are deleted.




Move virtual storage with no downtime

38





39

1. During the move operation, disk reads and

writes go to the source virtual hard disk

2. While reads and writes occur on the source

virtual hard disk, the disk contents are

copied over the network to the new


3. After the initial disk copy is complete, disk

writes are mirrored to both the source and

destination virtual hard disks





40

4. After the source and destination virtual

hard disks are synchronized, the virtual

machine switches over to using the


5. The source virtual hard disk is deleted.




Merge snapshots while the virtual machine is running

41

Scale, performance, and density

Features

42













Host NUMA

Memory

Processors

NUMA node 1 NUMA node 2

Memory

Processors





Host NUMA

Memory

Processors


Memory

Processors





vNUMAnode A vNUMAnode B vNUMAnode A vNUMAnode B

NUMA node 1 NUMA node 2 NUMA node 3 NUMA node 4

Disable virtual NUMA support on virtual machine configured to used Dynamic Memory




Host

Network I/O path without SRIOV

Root Partition

Hyper-V

Switch

Physical

NIC

Virtual

Machine

Virtual

NIC

Routing

VLAN Filtering

Data Copy

Network I/O path with SRIOV

HostRoot Partition

Hyper-V

Switch

SR-IOV Physical NIC

Virtual

Machine

Virtual

Function

Routing

VLAN Filtering

Data Copy




Network I/O path with SRIOV

HostRoot Partition

Hyper-V

Switch

SR-IOV Physical NIC

Virtual

Machine

Virtual

Function

Routing

VLAN Filtering

Data Copy
















• New SAN hardware feature copy offload innovations to copy large amount of data to one location to another such as merging disk and storage migration

• Offloaded Data Transfer (ODX) support is a feature of the storage stack of Hyper-V in Windows Server 2012

• ODX lets a storage device perform a file copy operation without the main processor of the Hyper-V host actually reading the content from one storage place and writing it to another

• Virtual machine are fully ODX aware and enabled

VHD Stack

Token

Token




• Traditional Data Copy Model

Server Issues Read Request To SAN

Data Is Read Into Memory

Data Is Written From Memory To SAN

• Problems

Increased CPU & Memory Utilization

Increased Storage Traffic

Inefficient For SAN

VHD Stack




VHD Stack

Token

Token

Offload Enabled Data Copy Model

• Server Issues Offload Read Request (ODX) To SAN

• SAN Returns Token Representing Request

• Server Issues Write Request To SAN Using Token

• SAN Completes Data Copy Internally

• SAN Confirms Data Was Copied

• Reduce Maintenance Time

• Merge, Mirror, VHD/VHDX Creation

• Increased Workload Performance

• VM’s Are Fully ODX Aware and Enabled




VHD Stack

Token

Token




• Data Center Bridging, or DCB, refers to enhancements to Ethernet LANs used in datacenter environments

• Consolidate the various forms of network into a single technology known as a Converged Network Adapter (CNA)

• Hyper-V in Windows Server 2012 can take advantage of DCB-capable hardware to converge multiple types of network traffic on a single network adapter with a maximum level of service

• It converges different types of traffic, including network, storage, management, and live migration traffic

• Support for DCB-enabled 10 GigE network adapters is one of the new Quality of Service (QoS) bandwidth management features in Windows Server 2012

59




• Provides Fibre Channel ports within the guest operating system enabling direct connections to Fibre Channel directly from within virtual machines

• Running Windows Failover Clustering feature inside the virtual machine connect to shared Fibre Channel storage

• Support Live Migration while maintaining Fibre Channel connectivity

• Using N_Port ID Virtualization (NPIV) to create multiple NPIV ports on top of physical Fibre Channel ports

• A new NPIV port is created on the host each time a virtual HBA is created inside a virtual machine

• Up to four virtual Fibre Channel adapters on a virtual machine

• Microsoft MultiPath I/O (MPIO) functionality

60

WWN WWN




• Hosting providers that host customers on a Hyper-V server must deliver performance that is based on SLAs

• Enterprises want to run multiple application servers on a Hyper-V server with the confidence that each application server will perform predictably.

• Assign a minimum bandwidth to virtual machine or services (live migration, management, storage, virtual machine)

• Enhance scheduler software in Windows and DCB network adapter mechanisms

61




• A technology that helps you track historical data on the use of virtual machines and gain insight into the resource use of specific servers

• Use this data to perform capacity planning, to monitor consumption by different business units or customers, or to capture data needed to help redistribute the costs of running a workload

• Hyper-V cmdlets in Windows PowerShell, and the new APIs in the Virtualization WMI Provider.

62




http://www.microsoft.com/en-us/server-cloud/windows-server/2012-capabilities.aspx

http://download.microsoft.com/download/5/D/B/5DB1C7BF-6286-4431-A244-438D4605DB1D/WS%202012%20White%20Paper_Hyper-V.pdf

http://download.microsoft.com/download/A/B/E/ABE02B78-BEC7-42B0-8504-C880A1144EE1/WS%202012%20White%20Paper_Storage.pdf

http://download.microsoft.com/download/7/E/6/7E63DE77-EBA9-4F2E-81D3-9FC328CD93C4/WS%202012%20White%20Paper_Networking.pdf

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012

http://www.microsoft.com/en-us/server-cloud/windows-server/2012-capabilities.aspx

http://download.microsoft.com/download/5/D/B/5DB1C7BF-6286-4431-A244-438D4605DB1D/WS 2012 White Paper_Hyper-V.pdf

http://download.microsoft.com/download/A/B/E/ABE02B78-BEC7-42B0-8504-C880A1144EE1/WS 2012 White Paper_Storage.pdf

http://download.microsoft.com/download/7/E/6/7E63DE77-EBA9-4F2E-81D3-9FC328CD93C4/WS 2012 White Paper_Networking.pdf

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012

64

Documents

Windows Server 2012 Hyper-V Architecture Deep Divedownload.microsoft.com/download/5/C/E/5CEB23E0-C814-4E4B-ADE… · One or more computers running Windows Server 2012 with the Hyper-V