Embed Size (px)
Citation preview
vSphere Distributed Switch: Technical Deep Dive
NET2745
Jason Nash, Varrow Chris Wahl, AHEAD
NET2745
Senior Solutions Architect @ AHEAD VCDX #104, vExpert Blogger – WahlNetwork.com Author – Networking for VMware Administrators Author – Pluralsight IT Pro training
CCNA Data Center, vCloud Director Host – VUPaaS and IT Engine Builders Podcasts
Chief Technology Officer @ Varrow VCDX #49, vExpert Blogger – JasonNash.com Author – Pluralsight IT Pro training
XtremIO, Cisco UCS, Nexus 1000v, vC Ops, + more
• Overview • Requirements • Construction • Alternatives • Tips • Real World Use Cases
ü Enterprise Plus licensing ü VMware skillset ü ESXi host versions ≥ to VDS version
o Lives in vCenter o All 5.1+ features are Web Client only L o VDS is a Data Center level object o VDS database syncs with each ESXi host
o Lives on the ESXi host o We suggest • Use Elastic ports • Connect uplinks (vmnics) to a single network segment o If you need multiple network segments • Possible, but requires workarounds
Servers Segment
DMZ Segment
vmnic0
vmnic1
vmnic2
vmnic3
Port Group
VM1
VM2
VM3
VM4
VM5
Servers Segment
DMZ Segment
vmnic0
vmnic1
vmnic2
vmnic3
Port Group
VM1
VM3
VM5
VM2
VM4
Port Group
vmnic2
vmnic3
vmnic0
vmnic1
o vSphere Standard Switch (VSS) o Cisco Nexus 1000v o IBM 5000V aka “Chupacabra”
ü Use 802.1Q tags for port groups ü At least 2 vmnics (uplinks) per VDS ü A 2x 10 GbE configuration can work fine ü Put QoS tagging in VDS or physical, not both ü Use descriptive naming everywhere o No one knows what “dvPortGroup-1” does
Migration VSS to VDS
Mixing 1Gb and 10Gb
Hosts
Handling vMotion
Saturation
vSphere Replication Bandwidth
Quality of Service Tagging
Load Based Teaming vs
Link Aggregation
Triggers: Ø Licensing (purchased Enterprise Plus) Ø Consume features found only in VDS Ø Reduce operational overhead Ø Separate control planes and related responsibilities
Tips and Advice: Ø Have a detailed plan in place Ø Test the process on a single host with non-prod
VMs Ø Test network convergence time and ping drops Ø Become comfortable with the steps Ø Put in a change control Ø Execute change during maintenance window
Standard Switch
Distributed Switch
vmnic0
vmnic1
VM Port Group VM
VMKs VMK Port Group
VM Port Group
VMK Port Group
Standard Switch
Distributed Switch
vmnic0 VM Port Group VM
VMKs VMK Port Group
VM Port Group
VMK Port Group vmnic1
Standard Switch
Distributed Switch
vmnic0 VM Port Group VM
VMK Port Group
VM Port Group
VMK Port Group vmnic1 VMKs
Standard Switch
Distributed Switch
vmnic0 VM Port Group
VMK Port Group
VM Port Group
VMK Port Group vmnic1 VMKs
VM
Standard Switch
Distributed Switch
VM Port Group
VMK Port Group
VM Port Group
VMK Port Group vmnic1 VMKs
VM vmnic0
Distributed Switch
VM Port Group
VMK Port Group vmnic1 VMKs
VM vmnic0
Triggers: Ø Purchase of new server / switch hardware Ø Staged migration to 10 GbE Ø Data Center transformation process
Tips and Advice: Ø Use a single network segment Ø Use a single VDS Ø Hosts should be entirely 1 GbE or 10 GbE Ø VM Traffic can traverse any uplink Ø Control teaming policies on VMK networks
dvUplinks
VM Port Groups VMK 1Gb (Mgmt, vMotion, etc)
ESXi 1 Gb
vmnic1 vmnic2 vmnic3 vmnic4
dvUplink1
1 2 3 4 1 2 3 4
dvUplink2 dvUplink3 dvUplink4
vmnic0 was left off to make the numbers
match
dvUplinks
VM Port Groups
ESXi 1 Gb
vmnic1 vmnic2
dvUplink1
VMK 10Gb (Mgmt, vMotion, etc)
1 2 1 2
dvUplink2 dvUplink3 dvUplink4
dvUplinks
VM Port Groups VMK 1Gb (Mgmt, vMotion, etc)
ESXi 1 Gb
vmnic1 vmnic2 vmnic3 vmnic4
ESXi 1 Gb
vmnic1 vmnic2
dvUplink1
VMK 10Gb (Mgmt, vMotion, etc)
1 2 3 4 1 2 3 4 1 2
dvUplink2 dvUplink3 dvUplink4
dvUplinks
VM Port Groups
ESXi 1 Gb
vmnic1 vmnic2
dvUplink1
1 2 3 4
dvUplink2 dvUplink3 dvUplink4
No Link
Triggers: Ø Multiple hosts migrate VMs to a single host Ø 2+ host maintenance mode Ø DRS migrations Ø DRS affinity and anti-affinity rules
Tips and Advice Ø Know how Ingress vs Egress works in VDS Ø Use NIOC for source-based control Ø Use Traffic Shaping for destination-based control
Triggers: Ø Sharing a WAN pipe with other traffic Ø Paying for bandwidth at certain % of peak Ø Multiple VR tenants between data centers Ø Contention with other backup or replication jobs
Tips and Advice: Ø Use NIOC with Limits (per vmnic) Ø Use Network Resource pool for VR Ø Alternative is to limit based on VR ports
Ø Run script for different limits during day/night Ø Requires PowerCLI and vCenter service account Short URL = http://goo.gl/dAgqBz
Triggers: Ø Tag traffic for various SLAs Ø Use L2 Priority Code Point (PCP) Ø Use L3 Differentiated Services Code Point (DSCP) Ø Data Center Bridging extensions in 802.1 Ø Priority-based Flow Control (PFC) – 802.1Qbb Ø Enhanced Transmission Selection (ETS) – 802.1Qaz
Tips and Advice: Ø KISS: QoS solves contention problems Ø Pick a place to tag traffic – virtual or physical Ø Try not to enforce QoS in too many ways Ø Use clearly defined tagging when needed Ø Avoid hard limits on traffic flows
0 1 2 3 4 5 6 7
Triggers: Ø Network and Server teams not cooperating Ø Pop out of those silos! Ø Poor convergence times during link failover Ø Poor use of uplink throughput Ø Excessive Topology Change Notifications (TCN) Ø Excessive vMotion activity
Load Distribution Ø Assigning workloads to uplinks based on identifiers Ø Example: L2, L3, L4, and VLAN values
Load Balancing Ø Assigning workloads to uplinks based on traffic Ø Example: “Route based on physical NIC load”
Load Distribution Ø No iSCSI Binding or Multi-NIC vMotion Ø Potential Layer 2 Path Optimization
Load Balancing Ø Imbalanced NIC saturation Ø Network can tolerate TCN and MAC table updates
Load Distribution Ø Link Aggregation Group (LAG) Ø Static (EtherChannel) or Dynamic (LACP)
Load Balancing Ø Set PG to Route based on physical NIC load Ø Also known as Load Based Teaming (LBT)
Chris Wahl @ChrisWahl
WahlNetwork.com
Jason Nash @TheJasonNash JasonNash.com
Thank You
Fill out a survey Every completed survey is entered into a
drawing for a $25 VMware company store gift certificate
vSphere Distributed Switch: Technical Deep Dive
NET2745
Jason Nash, Varrow Chris Wahl, AHEAD
