Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
OpenStack Neutron Networking
Paul Sim Technical Account Manager [email protected]
● OpenStack overview
● OpenStack Components
● Nova-network
● Network as a Service : Neutron
● Network Virtualization - Overlay
● Neutron - Modular Layer 2
● Neutron High Availability
● Distributed Virtual Router
Index
OpenStack overview
OpenStack Components
● Identity Keystone
● Compute Nova
● Block Storage Cinder
● Object Storage Swift
● Image Storage Glance
● Network Quantum/Neutron
● Dashboard Horizon
● Metering Ceilometer
● Orchestration Heat
● Database as a Service Trove
● Hadoop as a Service Sahara
● File-share Service Manila
OpenStack network model
1. Nova-network
a. Flat Network Manager
b. Flat DHCP Network Manager
c. VLAN Network Manager
2. Neutron with plugins
a. ML2 : OpenvSwitch
b. VMware NSX
c. Software Defined Networking
OpenDaylight, Ryu
d. MidoNet
e. OpenContrail
f. ...
OpenStack networking with Nova-network
Controller node
Keystone
Compute node - 2 Compute node - 3
Nova
Glance Horizon
Nova compute
eth1
eth0
eth1
eth0
eth1
eth0
Management
External network
Nova network
Nova compute
Nova network
Compute node - 1
Nova compute
eth1
eth0
Nova network
Nova-network
eth0
Flat DHCP Network Manager VLAN Network Manager
VM VM VM
Bridge dnsmasq
G/W
VM VM VM
Bridge 1 Bridge 2
eth0
vlan 100 vlan 101
dnsmasq dnsmasq
G/W G/W
Nova-network
VM VM VM
Bridge 1 Bridge 2
eth0
vlan 100 vlan 101
dnsmasq dnsmasq
G/W G/W
Compute node-2
VM VM VM
Bridge 1 Bridge 2
eth0
vlan 110 vlan 100
dnsmasq dnsmasq
G/W G/W
Compute node-1
Switch
vlan 100,110
vlan 100,101
Nova-network
Deprecation of Nova Network
With the introduction of the full software-defined networking stack provided by OpenStack Networking (neutron) in the
Folsom release, development effort on the initial networking code that remains part of the Compute component has
gradually lessened. While many still use nova-network in production, there has been a long-term plan to remove the
code in favour of the more flexible and full-featured OpenStack Networking.
An attempt was made to deprecate nova-network during the Havana release, which was aborted due to the lack of equivalent functionality (such as the FlatDHCP multi-host high availability mode mentioned in this guide), lack of a migration path between versions, insufficient testing, and simplicity when used for the more straightforward use cases nova-network traditionally supported. Though significant effort has been made to address these concerns, nova-network will not be deprecated in the Icehouse release. In addition, the Program Technical Lead of the Compute project has indicated that, to a limited degree, patches to nova-network will now again begin to be accepted. This leaves you with an important point of decision when designing your cloud. OpenStack Networking is robust enough to use with a small number of limitations (IPv6 support, performance issues in some scenarios) and provides many more features than nova-network. However, if you do not have the more complex use cases that can benefit from fuller software-defined networking capabilities, or are uncomfortable with the new concepts introduced, nova-network may continue to be a viable option for the next 12 to 18 months. Similarly, if you have an existing cloud and are looking to upgrade from nova-network to OpenStack Networking, you should have the option to delay the upgrade for this period of time. However, each release of OpenStack brings significant new innovation, and regardless of your use of networking methodology, it is likely best to begin planning for an upgrade within a reasonable time frame of each release. As mentioned, there's currently no way to cleanly migrate from nova-network to neutron. We recommend that you keep a migration in mind and what that process might involve for when a proper migration path is released. If you must upgrade, please be aware that both service and instance downtime is likely unavoidable.
http://docs.openstack.org/trunk/openstack-ops/content/nova-network-deprecation.html
Compute Node
Neutron API
Agent
Controller Neutron plugins
Nova Horizon UI
Compute Node
Agent
pSwitch
API, Agent
L4, F/W, VPN
Network as a Service - Neutron
API, Agent
Neutron Plugins
● Modular Layer 2
● OpenvSwitch
● VMware NSX
● Software Defined Networking
o OpenDaylight, Ryu
● MidoNet
● OpenContrail
Controller node
Keystone
Network node Compute node - 1 Compute node - 2
Nova
Glance Horizon
Neutron server
Neutron openvswitch-plugin
Nova compute
eth1 eth2
eth0
eth1 eth2
eth0
eth1 eth2
eth0
eth1 eth2
eth0
Management 192.168.20.0/24
Data 192.168.10.0/24
External network 192.168.122.0/24
Neutron openvswitch-plugin
Neutron metadata-agent
Neutron L3/dhcp-agent
Neutron openvswitch-plugin
Nova compute
OpenStack networking with Neutron
Compute node - 3
Compute node - 2
Network node
vRouter A
Network Virtualization
Compute node - 1
Subnet 1
Subnet 2
Subnet 4
vRouter B
vRouter C
vRouter D
Subnet 3
Tenant X
Tenant Y
Tenant Z Subnet 3
Subnet 4
Subnet 2
GRE/VxLAN Tunneling
Network Topology
● ext_net : external network - 192.168.122.0/24 ● net_proj_one : “user_one” tenant - 50.50.1.0/24 ● net_proj_two : “user_one” tenant - 50.50.2.0/24 ● net_proj_new : “user_new” tenant - 60.60.1.0/24
* LibvirtHybridOVSBridgeDriver
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
Network node
net_proj_one net_proj_two net_proj_new
Big picture - Neutron OVS plugin GRE
OpenStack Havana OpenvSwitch plug-in GRE tunneling - LibvirtGenericVIFDriver
Compute node - 1
br-ex
qg~
VM VM
br-
tun
tap~ tag: 1
tap~ tag:2
br-int
Tunnel
qg~
qg~
eth0
qr~
tap~ tap~ tap~
br-int
qr~ qr~
patch
patc
h
br-
tun
patc
h
gre
~
patch
Data 192.168.10.0/24
OVS port
OVS Bridge
● qg~~~ : external gateway interface ● qr~~~ : virtual router interface
Packet conversion
Neutron OVS plugin GRE - Compute node
OpenStack Havana OpenvSwitch plug-in GRE tunneling - LibvirtGenericVIFDriver
Compute node - 1
VM VM
tap~ tag: 1
tap~ tag:2
Tunnel
br-
tun
patch
VM
tap~ tag:2
Security Group[1] set_tunnel id
mod_vlan_vid
VM
tap~ tag:3
br-int patch
Neutron OVS plugin GRE - Compute node
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-tun
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=87770.027s, table=0, n_packets=0, n_bytes=0,
priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:1,output:1
cookie=0x0, duration=87770.09s, table=0, n_packets=8786, n_bytes=1893724, priority=4,in_port=1,dl_vlan=1
actions=set_tunnel:0x1,NORMAL
cookie=0x0, duration=87769.693s, table=0, n_packets=3031, n_bytes=617650,
priority=3,tun_id=0x1,dl_dst=fa:16:3e:db:08:63 actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=87769.966s, table=0, n_packets=6320, n_bytes=4432680,
priority=3,tun_id=0x1,dl_dst=fa:16:3e:e0:73:95 actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=87771.753s, table=0, n_packets=2921, n_bytes=951454, priority=1 actions=drop
Packet conversion
Neutron OVS plugin GRE - Network node
janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-tun
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=474674.446s, table=0, n_packets=7899, n_bytes=2572502,
priority=3,tun_id=0x3,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:2,output:1
cookie=0x0, duration=473163.123s, table=0, n_packets=7876, n_bytes=2565284,
priority=3,tun_id=0x4,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:3,output:1
cookie=0x0, duration=633937.826s, table=0, n_packets=10543, n_bytes=3426814,
priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:1,output:1
cookie=0x0, duration=473163.329s, table=0, n_packets=16484, n_bytes=3348666, priority=4,in_port=1,dl_vlan=3
actions=set_tunnel:0x4,NORMAL
cookie=0x0, duration=474674.541s, table=0, n_packets=16864, n_bytes=3389132, priority=4,in_port=1,dl_vlan=2
actions=set_tunnel:0x3,NORMAL
cookie=0x0, duration=633937.905s, table=0, n_packets=62044, n_bytes=37320316, priority=4,in_port=1,dl_vlan=1
actions=set_tunnel:0x1,NORMAL
cookie=0x0, duration=472911.069s, table=0, n_packets=16335, n_bytes=3551350, priority=3,tun_id=0x4,dl_dst=fa:16:3e:89:fd:ce
actions=mod_vlan_vid:3,NORMAL
cookie=0x0, duration=474336.184s, table=0, n_packets=16360, n_bytes=3560332,
priority=3,tun_id=0x3,dl_dst=fa:16:3e:d8:d5:29 actions=mod_vlan_vid:2,NORMAL
cookie=0x0, duration=474674.351s, table=0, n_packets=525, n_bytes=52427, priority=3,tun_id=0x3,dl_dst=fa:16:3e:69:ca:97
actions=mod_vlan_vid:2,NORMAL
cookie=0x0, duration=473162.912s, table=0, n_packets=197, n_bytes=19365, priority=3,tun_id=0x4,dl_dst=fa:16:3e:d6:b8:07
actions=mod_vlan_vid:3,NORMAL
cookie=0x0, duration=633937.746s, table=0, n_packets=6207, n_bytes=630043, priority=3,tun_id=0x1,dl_dst=fa:16:3e:c7:ec:bd
actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=474794.912s, table=0, n_packets=36912, n_bytes=7440964,
priority=3,tun_id=0x1,dl_dst=fa:16:3e:8b:a6:d7 actions=mod_vlan_vid:1,NORMAL
cookie=0x0, duration=636252.069s, table=0, n_packets=163, n_bytes=36046, priority=1 actions=drop
Packet conversion
Namespcae Namespcae Namespcae
Neutron OVS plugin GRE - Network node
OpenStack Havana OpenvSwitch plug-in GRE tunneling - LibvirtGenericVIFDriver
br-
tun
Tunnel
eth0
patc
h
gre
~
qr~
tap~
qg~
qr~
qg~
qr~
qg~
br-int
br-ex
patch
Packet conversion
mod_vlan_id
set_tunnel id
tap~ tap~
net_proj_one
net_proj_two
net_proj_new
Network node
Floating-IP(NAT)
Neutron OVS plugin Security Group - GRE
FORWARD
neutron-filter-top
neutron-openvswi-FORWARD
neutron-openvswi-local
neutron-openvswi-sg-chain
neutron-openvswi-iTAP_NUMBER
neutron-openvswi-oTAP_NUMBER
neutron-openvswi-sg-fallback
neutron-openvswi-sg-fallback
Security group is applied here
Neutron OVS plugin Security Group - GRE Chain neutron-openvswi-sg-chain (4 references)
target prot opt source destination
neutron-openvswi-i21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap21767f1f-45 --physdev-is-
bridged
neutron-openvswi-o21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap21767f1f-45 --physdev-is-
bridged
neutron-openvswi-i7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap7903fd30-74 --physdev-is-
bridged
neutron-openvswi-o7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap7903fd30-74 --physdev-is-
bridged
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-i7903fd30-7 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
RETURN icmp -- 0.0.0.0/0 0.0.0.0/0
RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
RETURN udp -- 50.50.1.3 0.0.0.0/0 udp spt:67 dpt:68
neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0
Chain neutron-openvswi-o7903fd30-7 (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:DB:08:63
RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67
DROP all -- !50.50.1.2 0.0.0.0/0
DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
RETURN all -- 0.0.0.0/0 0.0.0.0/0
neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0
[1] Note, OpenStack uses iptables rules on the TAP devices such as “tap~~” to implement security groups. However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an Open vSwitch port.
Neutron OVS plugin NameSpace - GRE
janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
qg-fa243f49-d6 Link encap:Ethernet HWaddr fa:16:3e:9f:4b:63
inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe9f:4b63/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
qr-bc654dc2-f1 Link encap:Ethernet HWaddr fa:16:3e:c7:ec:bd
inet addr:50.50.1.1 Bcast:50.50.1.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fec7:ecbd/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.122.1 0.0.0.0 UG 0 0 0 qg-fa243f49-d6
50.50.1.0 * 255.255.255.0 U 0 0 0 qr-bc654dc2-f1
192.168.122.0 * 255.255.255.0 U 0 0 0 qg-fa243f49-d6
Neutron OVS plugin Floating-IP(NAT) - GRE
janghoon@Network-node:~$ sudo ip netns show
qdhcp-4c2f2346-ffaa-41a0-ab76-34cadf0163f5
qrouter-e1b88ce4-51e9-4744-be80-d70d04c6a59b
qdhcp-c19e22a0-1700-4b3b-91e5-2c961ef0a353
qrouter-244fff3f-f935-4bdd-949d-739f1ce81dd0
qdhcp-f37b681a-4be8-47b8-8063-3d17d24ee1ae
qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0
janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 iptables -L -n -t nat
Chain neutron-l3-agent-PREROUTING (1 references)
target prot opt source destination
REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697
DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.1.2
Chain neutron-l3-agent-float-snat (1 references)
target prot opt source destination
SNAT all -- 50.50.1.2 0.0.0.0/0 to:192.168.122.51
Chain neutron-l3-agent-snat (1 references)
target prot opt source destination
neutron-l3-agent-float-snat all -- 0.0.0.0/0 0.0.0.0/0
SNAT all -- 50.50.1.0/24 0.0.0.0/0 to:192.168.122.50
Floating-IP(NAT)
NameSpace
Neutron ML2
The Modular Layer 2 (ML2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety of layer 2 networking technologies found in complex real-world data centers. It currently works with the existing openvswitch, linuxbridge, and Hyper-V L2 agents, and is intended to replace and deprecate the monolithic plugins associated with those L2 agents.
Neutron
TypeDriver
VLAN
ML2 Plugin
GRE VxLAN Flat
MechanismDriver
OpenvS
witc
h
Hyper-
V
OpenD
aylig
ht
Aris
ta
Cis
co N
exu
s
pSwitch
TypeDriver : TypeDrivers maintain any needed type-specific network state, and perform provider network validation and tenant network allocation. MechanismDriver : The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied given the specific networking mechanisms that have been enabled. https://wiki.openstack.org/wiki/Neutron/ML2
Neutron ML2 Installation
Network node Compute node - 1 Compute node - 2
Neutron ML2-agent Neutron
ML2-agent
Nova compute
eth0
eth1 eth2 eth1 eth2
eth0
eth1 eth2
eth0
Neutron server
Neutron metadata-agent
Neutron L3/dhcp-agent
Neutron ML2-agent
Nova compute
Management 192.168.20.0/24
External network 192.168.122.0/24
Data 192.168.10.0/24
Compute node - 3
Compute node - 2
Network node-1
L3 Agent
Neutron Multi network node
Compute node - 1
Tenant A
Tenant B
Tenant A
Tenant C
Tenant D
Tenant C
Network node-2
L3 Agent
Network node-2
Compute node - 3
Compute node - 2
Network node-1
vRouter A - Master
Neutron High Availability(L3 agent)
Compute node - 1
Subnet 1
Subnet 3
Subnet 2
Subnet 5
vRouter B - Backup
vRouter C - Backup
vRouter D - Master
vRouter C - Master
vRouter D - Backup
vRouter A - Backup
Subnet 3
Subnet 4
vRouter B - Master
Tenant X
Tenant Y
Tenant Z
VRRP
Network node-1
Neutron server
eth1 eth2
eth0
Neutron ML2 plugin
Neutron metadata-agent
Neutron L3/dhcp-agent
External network
Management
Data
KeepAlived
Network node-2
Neutron server
eth1 eth2
eth0
Neutron ML2 plugin
Neutron metadata-agent
Neutron L3/dhcp-agent
KeepAlived Compute node - 1
Nova compute
eth1 eth2
eth0
Neutron ML2 plugin
Compute node - 2
eth1 eth2
eth0
Neutron ML2 plugin
Nova compute
Neutron High Availability(L3 agent)
Namespace OVS bridge
Network node-1
qdhcp-
br-
tun
br-int
qrouter-
ha~
ns~ qr~ qg~
br-ex
Network node-2
qdhcp-
br-
tun
br-int
qrouter-
qr~ qg~
br-ex
ns~
KeepAlived KeepAlived
ha~
ubuntu@ubuntu-5:~$ sudo ip netns exec qrouter-d8625260-88a1-4312-b788-c04fc9094356 tcpdump -n -i ha-27fe59da-
a8
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ha-27fe59da-a8, link-type EN10MB (Ethernet), capture size 65535 bytes
16:16:25.213440 IP 169.254.192.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 50, authtype none, intvl 2s, length 20
16:16:27.214607 IP 169.254.192.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 50, authtype none, intvl 2s, length 20
16:16:29.215796 IP 169.254.192.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 50, authtype none, intvl 2s, length 20
16:16:31.216986 IP 169.254.192.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 50, authtype none, intvl 2s, length 20
Neutron High Availability(L3 agent)
Network node-1
qdhcp-
br-
tun
eth0
br-int patch-tun
patc
h-in
t
qrouter-
tap tap tap
ha~ ns~ qr~
qg~
br-ex
tap
Network node-2
qdhcp-
br-
tun
eth0
br-int patch-tun
patc
h-in
t
gre
~
qrouter-
tap tap tap
ha~ ns~ qr~
qg~
br-ex
tap
Namespace OVS bridge
gre
~
Neutron High Availability(L3 agent)
Network node-1
qdhcp-
br-
tun
eth0
br-int patch-tun
patc
h-in
t
qrouter-
tap tap tap
ha~ ns~ qr~
qg~
br-ex
tap
Network node-2
qdhcp-
br-
tun
eth0
br-int patch-tun
patc
h-in
t
gre
~
qrouter-
tap tap tap
ha~ ns~ qr~
qg~
br-ex
tap
Namespace OVS bridge
gre
~
Neutron High Availability(L3 agent)
Namespace OVS bridge
Network node-1
qdhcp-
br-
tun
br-int
qrouter-
ha~
ns~ qr~ qg~
br-ex
KeepAlived
ubuntu@ubuntu-5:~$ cat /var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/keepalived.conf vrrp_sync_group VG_1 { group { VR_1 } notify_master "/var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/notify_master.sh" notify_backup "/var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/notify_backup.sh" notify_fault "/var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/notify_fault.sh" } vrrp_instance VR_1 { state BACKUP interface ha-27fe59da-a8 virtual_router_id 1 priority 50 nopreempt advert_int 2 track_interface { ha-27fe59da-a8 } virtual_ipaddress { 192.168.10.118/24 dev qg-8fffbd7e-8a } virtual_ipaddress_excluded { 50.50.1.1/24 dev qr-dee474e1-1e } virtual_routes { 0.0.0.0/0 via 192.168.10.51 dev qg-8fffbd7e-8a } }
Neutron High Availability(L3 agent)
Network node Tenant A
Namespace OVS bridge
br-
tun
br-int
qrouter-
ha~
br-ex
KeepAlived
qrouter-
ha~
KeepAlived
HA network : 169.254.192.1 ~ 254
Segmentation id : 0x6
Tenant B
qrouter-
ha~
KeepAlived
qrouter-
ha~
KeepAlived
HA network : 169.254.192.1 ~ 254
Segmentation id : 0x7
● One KeepAlived instance per vRouter
● One HA network per tenant
○ Each HA network has separate
segmentation id
○ allow_overlapping_ips = True
● Maximum 255 HA routers per tenant.
Neutron High Availability(L3 agent)
DVR (Distributed Virtual Router) - Installation
Network node
Neutron server
eth1 eth2
eth0
Neutron ML2 plugin
Neutron metadata-agent
Neutron L3/dhcp-agent
External network
Compute node - 1
Nova compute
eth1 eth2
eth0
Neutron ML2 plugin
Neutron metadata-agent
Neutron L3-agent
Management
Data
Compute node - 2
Nova compute
eth1 eth2
eth0
Neutron ML2 plugin
Neutron metadata-agent
Neutron L3-agent
DVR (Distributed Virtual Router) - Packet flow
Compute node - 1
br-
ex
VM VM
GRE Tunnel
VM
br-int
Network node
br-
ex
br-tun
br-int
br-
tun
Compute node - 2
VM VM VM
br-int
br-
tun
1.SNAT
External network
3. East-West traffic
2. Floating IP
OVS bridge
DVR (Distributed Virtual Router) - SNAT : Network node
Namespace
OVS bridge Network node
qdhcp- br-
tun
eth0
br-int patch-tun
patc
h-in
t
gre
~
snat- qrouter-
tap tap tap
sg~ 50.50.6.
2 ns~ qr~
qg~ 192.168.10.109
SNAT br-ex
tap
packet flow
DVR (Distributed Virtual Router) - SNAT : Compute node
Compute node
Namespace
OVS bridge
VM
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.
1
patch-tun
tap~
Linux bridge
sg~
on
netw
ork
node
packet flow
traffic flow
DVR (Distributed Virtual Router) - SNAT : Compute node
Namespace
OVS bridge
Linux bridge
Compute node
VM
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.
1
patch-tun
tap~ sg~
(50.5
0.6
.2)
on
netw
ork
node
ubuntu@ubuntu-8:~$ sudo ip netns exec qrouter-
20838b7d-a7ac-4da9-92aa-adec797d600e ip rule
ls
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
842139137: from 50.50.6.1/24 lookup
842139137
ubuntu@ubuntu-8:~$ sudo ip netns exec qrouter-
20838b7d-a7ac-4da9-92aa-adec797d600e ip route
show table 842139137
default via 50.50.6.2 dev qr-9722faba-b7
DVR (Distributed Virtual Router) - Floating IP/DNAT : Compute node
Compute node
Namespace
OVS bridge
VM
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.
1
patc
h-tu
n
tap~
Linux bridge
packet flow
br-ex
tap
eth0
fip-
fpr~ rfp~
fg~
Route Route
NAT
veth pair
DVR (Distributed Virtual Router) - Floating IP/DNAT : Compute node
Compute node
Namespace
OVS bridge
VM
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.
1
patc
h-tu
n
tap~
Linux bridge
packet flow
br-ex
tap
eth0
fip-
fpr~ rfp~
fg~
Route Route
NAT
veth pair
ubuntu@ubuntu-6:~$ sudo ip netns exec qrouter-20838b7d-a7ac-4da9-
92aa-adec797d600e ip rule ls
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
32770: from 50.50.5.5 lookup 16
842138881: from 50.50.5.1/24 lookup 842138881
842138881: from 50.50.5.1/24 lookup 842138881
842139137: from 50.50.6.1/24 lookup 842139137
ubuntu@ubuntu-6:~$ sudo ip netns exec qrouter-20838b7d-a7ac-4da9-
92aa-adec797d600e ip route show table 16
default via 169.254.31.29 dev rfp-20838b7d-a
DVR (Distributed Virtual Router) - Floating IP/DNAT : Compute node
Compute node
Namespace
OVS bridge
VM
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.
1
patc
h-tu
n
tap~
Linux bridge
packet flow
br-ex
tap
eth0
fip-
fpr~ rfp~
fg~
Route Route
NAT
veth pair
ubuntu@ubuntu-6:~$ sudo ip netns exec fip-02f9d340-
2caa-4c05-86fb-460c9580f9df ip route show
default via 192.168.10.1 dev fg-f3887d61-2d
192.168.10.114 via 169.254.31.28 dev fpr-20838b7d-a
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
VM 50.50.6.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
VM 50.50.5.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
ICMP Request
ICMP Reply
i.e., ping 50.50.5.3 -> 50.50.6.3
DVR (Distributed Virtual Router) - East-West traffic flow : network topology
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
VM 50.50.6.3
ubuntu@ubuntu-6:~$ sudo ip netns exec qrouter-20838b7d-
a7ac-4da9-92aa-adec797d600e ip link
2: qr-ecffa2a6-dd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen
1000
link/ether fa:16:3e:15:1e:e0 brd ff:ff:ff:ff:ff:ff
5: qr-9722faba-b7: <BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default
qlen 1000
link/ether fa:16:3e:71:3d:5a brd ff:ff:ff:ff:ff:ff
ubuntu@ubuntu-8:~$ sudo ip netns exec qrouter-20838b7d-
a7ac-4da9-92aa-adec797d600e ip link
2: qr-ecffa2a6-dd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu
1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen
1000
link/ether fa:16:3e:15:1e:e0 brd ff:ff:ff:ff:ff:ff
5: qr-9722faba-b7: <BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default
qlen 1000
link/ether fa:16:3e:71:3d:5a brd ff:ff:ff:ff:ff:ff
VM 50.50.5.3
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
VM 50.50.6.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
VM 50.50.5.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
ICMP Request 50.50.5.3 -> 50.50.6.3
Segmentation ID : 50.50.5.0/24 : 0x1 50.50.6.0/24 : 0x3
MAC 50.50.6.3 : fa:16:3e:ff:85:9b 50.50.6.1 : fa:16:3e:71:3d:5a 50.50.5.1 : fa:16:3e:15:1e:e0 50.50.5.3 : fa:16:3e:ce:8c:35
DVR Host MAC : Compute Node -1 : fa:16:3f:5e:a0:cf Compute Node -2 : fa:16:3f:72:60:33
SRC MAC :
fa:16:3e:71:3d:5a
SRC IP : 50.50.5.3
DST MAC : fa:16:3e:ff:85:9b
DST IP : 50.50.6.3
SRC MAC :
fa:16:3e:71:3d:5a
SRC IP : 50.50.5.3
DST MAC : fa:16:3e:ff:85:9b
DST IP : 50.50.6.3
SRC MAC :
fa:16:3e:ce:8c:35
SRC IP : 50.50.5.3
DST MAC :
fa:16:3e:15:1e:e0
DST IP : 50.50.6.3
GRE tunnel 0x3
SRC MAC : fa:16:3f:5e:a0:cf
SRC IP : 50.50.5.3
DST MAC : fa:16:3e:ff:85:9b
DST IP : 50.50.6.3
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
VM 50.50.6.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
VM 50.50.5.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
ICMP Reply 50.50.6.3 -> 50.50.5.3
SRC MAC :
fa:16:3e:15:1e:e0
SRC IP : 50.50.6.3
DST MAC : fa:16:3e:ff:85:9b
DST IP : 50.50.5.3
SRC MAC :
fa:16:3e:15:1e:e0
SRC IP : 50.50.6.3
DST MAC :
fa:16:3e:ce:8c:35
DST IP : 50.50.5.3
SRC MAC : fa:16:3e:ff:85:9b
SRC IP : 50.50.6.3
DST MAC :
fa:16:3e:71:3d:5a
DST IP : 50.50.5.3
Segmentation ID : 50.50.5.0/24 : 0x1 50.50.6.0/24 : 0x3
MAC 50.50.6.3 : fa:16:3e:ff:85:9b 50.50.6.1 : fa:16:3e:71:3d:5a 50.50.5.1 : fa:16:3e:15:1e:e0 50.50.5.3 : fa:16:3e:ce:8c:35
DVR Host MAC : Compute Node -1 : fa:16:3f:5e:a0:cf Compute Node -2 : fa:16:3f:72:60:33
GRE tunnel 0x1
SRC MAC : fa:16:3f:72:60:33
SRC IP : 50.50.6.3
DST MAC :
fa:16:3e:ce:8c:35
DST IP : 50.50.5.3
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
VM 50.50.6.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
VM 50.50.5.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
ICMP Request 50.50.5.3 -> 50.50.6.3
Segmentation ID : 50.50.5.0/24 : 0x1 50.50.6.0/24 : 0x3
MAC 50.50.6.3 : fa:16:3e:ff:85:9b 50.50.6.1 : fa:16:3e:71:3d:5a 50.50.5.1 : fa:16:3e:15:1e:e0 50.50.5.3 : fa:16:3e:ce:8c:35
DVR Host MAC : Compute Node -1 : fa:16:3f:5e:a0:cf Compute Node -2 : fa:16:3f:72:60:33
table=0, n_packets=9178, n_bytes=1009035, idle_age=17470, hard_age=65534, priority=1 actions=NORMAL
table=0, n_packets=2066, n_bytes=214544, idle_age=5, hard_age=65534, priority=1,in_port=1 actions=resubmit(,1)
table=1, n_packets=1765, n_bytes=172970, idle_age=5, hard_age=65534, priority=1,dl_vlan=2,dl_src=fa:16:3e:71:3d:5a
actions=mod_dl_src:fa:16:3f:5e:a0:cf,resubmit(,2)
table=2, n_packets=1849, n_bytes=183458, idle_age=5, hard_age=65534, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00
actions=resubmit(,20)
table=20, n_packets=1765, n_bytes=172970, idle_age=5, hard_age=65534, priority=2,dl_vlan=2,dl_dst=fa:16:3e:ff:85:9b
actions=strip_vlan,set_tunnel:0x3,output:3
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
VM 50.50.6.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
VM 50.50.5.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
ICMP Request 50.50.5.3 -> 50.50.6.3
Segmentation ID : 50.50.5.0/24 : 0x1 50.50.6.0/24 : 0x3
MAC 50.50.6.3 : fa:16:3e:ff:85:9b 50.50.6.1 : fa:16:3e:71:3d:5a 50.50.5.1 : fa:16:3e:15:1e:e0 50.50.5.3 : fa:16:3e:ce:8c:35
DVR Host MAC : Compute Node -1 : fa:16:3f:5e:a0:cf Compute Node -2 : fa:16:3f:72:60:33
table=0, n_packets=1789, n_bytes=175146, idle_age=17, hard_age=65534, priority=2,in_port=3,dl_src=fa:16:3f:5e:a0:cf actions=resubmit(,1) table=1, n_packets=1765, n_bytes=172970, idle_age=17, hard_age=65534, priority=4,dl_vlan=2,dl_dst=fa:16:3e:ff:85:9b actions=strip_vlan,mod_dl_src:fa:16:3e:71:3d:5a,output:8
table=0, n_packets=1857, n_bytes=184993, idle_age=18, hard_age=65534, priority=1,in_port=2 actions=resubmit(,3)
table=3, n_packets=1993, n_bytes=195880, idle_age=18, hard_age=65534, priority=1,tun_id=0x3 actions=mod_vlan_vid:2,resubmit(,9)
table=9, n_packets=1789, n_bytes=175146, idle_age=18, hard_age=65534, priority=1,dl_src=fa:16:3f:5e:a0:cf actions=output:1
DVR (Distributed Virtual Router) - East-West traffic flow : Compute node
Compute node-2
Namespace OVS bridge
VM 50.50.6.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~
Linux bridge packet flow
Compute node-1
tap~
qr~ 50.50.5.1
VM 50.50.5.3
br-int
br-
tun
qvo~
qbr~
tap~
qvb~
patch-int
qrouter-
qr~ 50.50.6.1
patch-tun
tap~ tap~
qr~ 50.50.5.1
ICMP Request 50.50.5.3 -> 50.50.6.3
Segmentation ID : 50.50.5.0/24 : 0x1 50.50.6.0/24 : 0x3
MAC 50.50.6.3 : fa:16:3e:ff:85:9b 50.50.6.1 : fa:16:3e:71:3d:5a 50.50.5.1 : fa:16:3e:15:1e:e0 50.50.5.3 : fa:16:3e:ce:8c:35
DVR Host MAC : Compute Node -1 : fa:16:3f:5e:a0:cf Compute Node -2 : fa:16:3f:72:60:33
table=0, n_packets=1789, n_bytes=175146, idle_age=17, hard_age=65534, priority=2,in_port=3,dl_src=fa:16:3f:5e:a0:cf actions=resubmit(,1) table=1, n_packets=1765, n_bytes=172970, idle_age=17, hard_age=65534, priority=4,dl_vlan=2,dl_dst=fa:16:3e:ff:85:9b actions=strip_vlan,mod_dl_src:fa:16:3e:71:3d:5a,output:8
table=0, n_packets=1857, n_bytes=184993, idle_age=18, hard_age=65534, priority=1,in_port=2 actions=resubmit(,3)
table=3, n_packets=1993, n_bytes=195880, idle_age=18, hard_age=65534, priority=1,tun_id=0x3 actions=mod_vlan_vid:2,resubmit(,9)
table=9, n_packets=1789, n_bytes=175146, idle_age=18, hard_age=65534, priority=1,dl_src=fa:16:3f:5e:a0:cf actions=output:1
Open Virtual Network project - OVN
● At present, ○ Packet switching -> Linux Bridge, OpenvSwitch ○ Routing -> Policy routing, routing table ○ Security -> iptables, ebtables
● OVN complements the existing capabilities of OpenvSwitch to add native support for virtual network abstractions, such as virtual L2 and L3 overlays and security groups.
● OVN will include logical switches and routers, security groups, and L2/L3/L4 ACLs, implemented on top of a tunnel-based (VXLAN, NVGRE, Geneve, STT, IPsec) overlay network.
Open Virtual Network project - OVN
Compute node
ovs-vswitchd ovsdb-server
ovn-controller
OVN-DB
OVN-Northbound DB
ovs-nbd
OVN plug-in OpenStack (Neutron)
Compute node
ovs-vswitchd ovsdb-server
ovn-controller
OpenFlow OVSDB protocol
OVSDB protocol
OpenFlow
OVSDB protocol
OVSDB protocol