Embed Size (px)
DESCRIPTION
Citation preview
Intel® Ethernet
Evolving New SR-IOV Configuration Tools
Mitch Williams
Intel LAN Access Division
Intel® Ethernet
SR-IOV in One Minute
2 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
NIC HW
VMM
Virtual Switch
Network
VM2
PV DD
VMn
PV DD
VM1
PV DD
Emulated NICs
Intel® Ethernet
SR-IOV in One Minute
3 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
NIC HW
VMM
Virtual Switch
Network
VM2
PV DD
VMn
PV DD
VM1
PV DD
VT-D
NICNIC HW
VMM
Virtual Switch
Network
VM2
PV DD
VMnVM1
PV DD NIC DD
Emulated NICs Direct Assigned NIC
Intel® Ethernet
SR-IOV in One Minute
4 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
NIC HW
VMM
Virtual Switch
Network
VM2
PV DD
VMn
PV DD
VM1
PV DD
VT-D
NICNIC HW
VMM
Virtual Switch
Network
VM2
PV DD
VMnVM1
PV DD
NIC (PF)
VMM
VM2
VF DD
VMn
VF DD
NIC Switch
VM1
VF DD
VF1 VF2VFn
SR
IOV
Network
NIC DD
Emulated NICs Direct Assigned NIC SR-IOV
Intel® Ethernet
Intel SR-IOV NICs• Intel® 82576 Gigabit Ethernet Controller
– Up to 8 VFs per port
– Dual Port / Quad Port
– Copper / Fiber / Backplane
– igb / igbvf driver
– Upstream kernel / RHEL 5.4 / XCP 0.1.1
• Intel® 82599 10 Gigabit Ethernet Controller– Up to 64 VFs per port
– Single Port / Dual Port
– Copper / Fiber / DA / Backplane
– ixgbe / ixgbevf
– Upstream kernel
5 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel® Ethernet
Mitch’s SR-IOV Pain• MAC addresses assigned randomly
– Reassigned each time host reboots
– Causes net device proliferation in VMs
– Annoying for DHCP users
– FATAL for fixed IP addresses
6 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel® Ethernet
Mitch’s SR-IOV Pain• MAC addresses assigned randomly
– Reassigned each time host reboots
– Causes net device proliferation in VMs
– Annoying for DHCP users
– FATAL for fixed IP addresses
7 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel Test Subject #796:
Seen here using fixed IP addresses
with SR-IOV.
Results Inconclusive.
Intel® Ethernet
Mitch’s SR-IOV Pain Reliever• Step 1: Have a meeting
• Step 2: Send a flurry of email
• Step 3: Fix it!
– Kernel PCI subsystem
– Kernel netdevice layer
– Kernel RT-netlink interface
– igb driver
– iproute2 package (AKA the “ip” command)
8 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel® Ethernet
How It Worksroot@avec:~# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:30:48:cc:6d:ee brd ff:ff:ff:ff:ff:ff
vf 0: MAC ee:d4:38:30:91:14
vf 1: MAC 2a:da:93:d7:7d:ec
vf 2: MAC aa:c6:38:a5:8b:a5
vf 3: MAC 4a:d2:64:7c:a8:17
root@avec:~# ip link set dev eth0 vf 0 mac 00:11:22:33:44:55
root@avec:~# ip link set dev eth0 vf 0 vlan 10
root@avec:~# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:30:48:cc:6d:ee brd ff:ff:ff:ff:ff:ff
vf 0: MAC 00:11:22:33:44:55, vlan 10
vf 1: MAC 2a:da:93:d7:7d:ec
vf 2: MAC aa:c6:38:a5:8b:a5
vf 3: MAC 4a:d2:64:7c:a8:17
root@avec:~#
9 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel® Ethernet
Other Pain-Reducing Features• VF VLAN setting
– Put a VF on VLAN, without the VM noticing
• TX Rate Limiting
– Set max transmit rate in MBps
– Coming soon…
10 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel® Ethernet
Near Term Needs Part 1• 10 Gig Support
– Done, in validation
– Plan for submittal to 2.6.35
• Update Distros
– Backport patches to XCP and RHEL 6
– Add support to VM management tools
• Add Migration Support
– Use channel bonding + hotplug in guests
– Add (more) support to VM management tools
11 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Intel® Ethernet
NIC (PF)
VMM
Near-Term Needs Part 2
12 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
Virtual Switch
VMB
VNIC
VMA
VNIC
VM2
VF DD
VMn
VF DD
NIC Switch
VM1
VF DD
VF1 VF2VFn
SR
IOV
Network
• VMs with emulated NICs
can’t talk to VMs with VF
NICs.
• Need to add MAC
addresses of emulated
NICs to PF NIC switch.
• No mechanism in place for
this.
Intel® Ethernet
Call For Comments• Please help us!
• What do you need to make SR-IOV work for you?
• What do you want to make SR-IOV easy for you?
13 *Other names and brands may be claimed as the property of others
Copyright © 2010 Intel Corporation. All products, dates and figures are preliminary,
for planning purposes only and are subject to change without notice.
