Embed Size (px)
DESCRIPTION
Citation preview
● Network as a Service : Neutron
● Nova-network
● Neutron - OpenvSwitch plugin VLAN
● Neutron - OpenvSwitch plugin GRE
● Neutron - Software Defined Networking
● Neutron - Modular Layer 2
Index
Network as a Service - Neutron
Nova-network
eth0
Flat DHCP Network Manager VLAN Network Manager
VM VMVM
Bridge dnsmasq
VM VMVM
Bridge 1 Bridge 2
eth0
vlan 100 vlan 101
dnsmasq dnsmasqG/W
G/W G/W
Network Resources
Network Resources
* Network NameSpace
BMWNameSpace
eth0 eth1 eth2
Address
Routing table
Process Process
Process Process
Netfilter rules
eth0 eth1 eth2
BenzNameSpace
NetworkResources
NetworkResources
ProcessProcess
Process
Process
FordNameSpace
NetworkResources
Share
without Network NameSpace with Network NameSpace
Network NameSpace provides isolation of the system resources associated with networking. Thus, each network namespace has its own network devices, IP addresses, IP routing tables, /proc/net directory, port numbers, and so on. - http://lwn.net/Articles/531114/
Installation - OpenvSwitch plugin VLAN, GRE
Controller node
Keystone
Network node Compute node - 1 Compute node - 2
Nova
Glance Horizon
Neutron serverNeutron
openvswitch-plugin
Nova compute
eth1 eth2
eth0
eth1 eth2
eth0
eth1 eth2
eth0
eth1 eth2
eth0
Management 192.168.20.0/24
Data 192.168.10.0/24
External network 192.168.122.0/24
Neutron openvswitch-plugin
Neutron metadata-agent
Neutron L3/dhcp-agent
Neutron openvswitch-plugin
Nova compute
Network Topology
● ext_net : external network - 192.168.122.0/24● net_proj_one : “user_one” tenant - 50.50.1.0/24● net_proj_two : “user_one” tenant - 50.50.2.0/24● net_proj_new : “user_new” tenant - 60.60.1.0/24
Network node
net_proj_one net_proj_two net_proj_new
Big picture - Neutron OVS plugin VLAN
OpenStack Havana OpenvSwitch plug-in VLAN mode- LibvirtGenericVIFDriver
Compute node - 1
br-ex
qg~
VM VM
br-eth1
tap~tag: 1
tap~tag:2
qg~ qg~
eth0
qr~
tap~ tap~ tap~
br-int
qr~ qr~
phy-br-eth1 Data 192.168.10.0/24
OVS port
OVS Bridge
● qg~~~ : external gateway interface● qr~~~ : virtual router interface
int-br-eth1
eth1 eth1 br-eth1
phy-br-eth1
VM
tap~tag:2
br-intint-br-eth1
Neutron OVS plugin VLAN - Compute node
OpenStack Havana OpenvSwitch plug-in VLAN mode- LibvirtGenericVIFDriver
Compute node - 1
VM VM
tap~tag: 1
tap~tag:2
br-
eth1
VM
tap~tag:2
Security Group[1]
Packet conversion
mod_vlan_vid
VM
tap~tag:3
br-intphy-br-eth1 int-br-eth1
eth1
veth pair
mod_vlan_vid
Neutron OVS plugin VLAN - Compute node
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-eth1NXST_FLOW reply (xid=0x4): cookie=0x0, duration=90455.716s, table=0, n_packets=6, n_bytes=468, priority=2,in_port=2 actions=drop cookie=0x0, duration=89606.096s, table=0, n_packets=9484, n_bytes=2312018, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:1024,NORMAL cookie=0x0, duration=90456.248s, table=0, n_packets=6813, n_bytes=1325511, priority=1 actions=NORMAL
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-intNXST_FLOW reply (xid=0x4): cookie=0x0, duration=90458.482s, table=0, n_packets=64, n_bytes=4644, priority=2,in_port=1 actions=drop cookie=0x0, duration=89608.755s, table=0, n_packets=6499, n_bytes=1283680, priority=3,in_port=1,dl_vlan=1024 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=90459.075s, table=0, n_packets=9820, n_bytes=2323195, priority=1 actions=NORMAL
Packet conversion
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=3,in_port=1,dl_vlan=1024,actions=mod_vlan_vid:1,normal']Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-eth1', 'hard_timeout=0,idle_timeout=0,priority=4,in_port=2,dl_vlan=1,actions=mod_vlan_vid:1024,normal']
openvswitch-agent.log
NamespcaeNamespcaeNamespcae
Neutron OVS plugin VLAN - Network node
OpenStack Havana OpenvSwitch plug-in VLAN mode- LibvirtGenericVIFDriver
eth0
qr~
tap~
qg~
qr~
qg~
qr~
qg~
br-int
br-ex
Packet conversion
mod_vlan_id
tap~ tap~
net_proj_one
net_proj_two
net_proj_new
Network node
Floating-IP(NAT)
eth1
br-eth1
int-br-eth1 phy-br-eth1
veth pair
mod_vlan_id
Neutron OVS plugin VLAN - Network node
janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-intNXST_FLOW reply (xid=0x4): cookie=0x0, duration=7370.307s, table=0, n_packets=6, n_bytes=468, priority=2,in_port=6 actions=drop cookie=0x0, duration=7368.424s, table=0, n_packets=0, n_bytes=0, priority=3,in_port=6,dl_vlan=2048 actions=mod_vlan_vid:2,NORMAL cookie=0x0, duration=7367.991s, table=0, n_packets=764, n_bytes=191460, priority=3,in_port=6,dl_vlan=1024 actions=mod_vlan_vid:3,NORMAL cookie=0x0, duration=7369.073s, table=0, n_packets=0, n_bytes=0, priority=3,in_port=6,dl_vlan=500 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=7370.924s, table=0, n_packets=549, n_bytes=104066, priority=1 actions=NORMAL
janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-eth1NXST_FLOW reply (xid=0x4): cookie=0x0, duration=7373.826s, table=0, n_packets=14, n_bytes=1104, priority=2,in_port=2 actions=drop cookie=0x0, duration=7372.725s, table=0, n_packets=13, n_bytes=922, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:500,NORMAL cookie=0x0, duration=7371.663s, table=0, n_packets=519, n_bytes=103966, priority=4,in_port=2,dl_vlan=3 actions=mod_vlan_vid:1024,NORMAL cookie=0x0, duration=7372.09s, table=0, n_packets=9, n_bytes=634, priority=4,in_port=2,dl_vlan=2 actions=mod_vlan_vid:2048,NORMAL cookie=0x0, duration=7374.384s, table=0, n_packets=764, n_bytes=191460, priority=1 actions=NORMAL
Packet conversion
* LibvirtHybridOVSBridgeDriver
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
Network node
net_proj_one net_proj_two net_proj_new
Big picture - Neutron OVS plugin GRE
OpenStack Havana OpenvSwitch plug-in GRE tunneling- LibvirtGenericVIFDriver
Compute node - 1
br-ex
qg~
VM VM
br-tun
tap~tag: 1
tap~tag:2
br-int
Tunnel
qg~ qg~
eth0
qr~
tap~ tap~ tap~
br-int
qr~ qr~
patch
patch b
r-tu
np
atch
gre~ g
re~
patch
Data 192.168.10.0/24
OVS port
OVS Bridge
● qg~~~ : external gateway interface● qr~~~ : virtual router interface
Packet conversion
Neutron OVS plugin GRE - Compute node
OpenStack Havana OpenvSwitch plug-in GRE tunneling- LibvirtGenericVIFDriver
Compute node - 1
VM VM
tap~tag: 1
tap~tag:2
Tunnel
br-
tun
patch
gre
~
VM
tap~tag:2
Security Group[1]set_tunnel id
mod_vlan_vid
VM
tap~tag:3
br-intpatch
Neutron OVS plugin GRE - Compute node
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-tunNXST_FLOW reply (xid=0x4): cookie=0x0, duration=87770.027s, table=0, n_packets=0, n_bytes=0, priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:1,output:1 cookie=0x0, duration=87770.09s, table=0, n_packets=8786, n_bytes=1893724, priority=4,in_port=1,dl_vlan=1 actions=set_tunnel:0x1,NORMAL cookie=0x0, duration=87769.693s, table=0, n_packets=3031, n_bytes=617650, priority=3,tun_id=0x1,dl_dst=fa:16:3e:db:08:63 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=87769.966s, table=0, n_packets=6320, n_bytes=4432680, priority=3,tun_id=0x1,dl_dst=fa:16:3e:e0:73:95 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=87771.753s, table=0, n_packets=2921, n_bytes=951454, priority=1 actions=drop
Packet conversion
NamespcaeNamespcaeNamespcae
Neutron OVS plugin GRE - Network node
OpenStack Havana OpenvSwitch plug-in GRE tunneling- LibvirtGenericVIFDriver
br-tun
Tunnel
eth0
patch
gre~
qr~
tap~
qg~
qr~
qg~
qr~
qg~
br-int
br-ex
patch
Packet conversion
mod_vlan_id
set_tunnel id
tap~ tap~
net_proj_one
net_proj_two
net_proj_new
Network node
Floating-IP(NAT)
Neutron OVS plugin GRE - Network node
janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-tunNXST_FLOW reply (xid=0x4): cookie=0x0, duration=474674.446s, table=0, n_packets=7899, n_bytes=2572502, priority=3,tun_id=0x3,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:2,output:1 cookie=0x0, duration=473163.123s, table=0, n_packets=7876, n_bytes=2565284, priority=3,tun_id=0x4,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:3,output:1 cookie=0x0, duration=633937.826s, table=0, n_packets=10543, n_bytes=3426814, priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:1,output:1 cookie=0x0, duration=473163.329s, table=0, n_packets=16484, n_bytes=3348666, priority=4,in_port=1,dl_vlan=3 actions=set_tunnel:0x4,NORMAL cookie=0x0, duration=474674.541s, table=0, n_packets=16864, n_bytes=3389132, priority=4,in_port=1,dl_vlan=2 actions=set_tunnel:0x3,NORMAL cookie=0x0, duration=633937.905s, table=0, n_packets=62044, n_bytes=37320316, priority=4,in_port=1,dl_vlan=1 actions=set_tunnel:0x1,NORMAL cookie=0x0, duration=472911.069s, table=0, n_packets=16335, n_bytes=3551350, priority=3,tun_id=0x4,dl_dst=fa:16:3e:89:fd:ce actions=mod_vlan_vid:3,NORMAL cookie=0x0, duration=474336.184s, table=0, n_packets=16360, n_bytes=3560332, priority=3,tun_id=0x3,dl_dst=fa:16:3e:d8:d5:29 actions=mod_vlan_vid:2,NORMAL cookie=0x0, duration=474674.351s, table=0, n_packets=525, n_bytes=52427, priority=3,tun_id=0x3,dl_dst=fa:16:3e:69:ca:97 actions=mod_vlan_vid:2,NORMAL cookie=0x0, duration=473162.912s, table=0, n_packets=197, n_bytes=19365, priority=3,tun_id=0x4,dl_dst=fa:16:3e:d6:b8:07 actions=mod_vlan_vid:3,NORMAL cookie=0x0, duration=633937.746s, table=0, n_packets=6207, n_bytes=630043, priority=3,tun_id=0x1,dl_dst=fa:16:3e:c7:ec:bd actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=474794.912s, table=0, n_packets=36912, n_bytes=7440964, priority=3,tun_id=0x1,dl_dst=fa:16:3e:8b:a6:d7 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=636252.069s, table=0, n_packets=163, n_bytes=36046, priority=1 actions=drop
Packet conversion
Neutron OVS plugin Security Group - VLAN, GRE
FORWARD
neutron-filter-top
neutron-openvswi-FORWARD
neutron-openvswi-local
neutron-openvswi-sg-chain
neutron-openvswi-iTAP_NUMBER
neutron-openvswi-oTAP_NUMBER
neutron-openvswi-sg-fallback
neutron-openvswi-sg-fallback
Security group is applied here
Neutron OVS plugin Security Group - VLAN, GRE
Chain neutron-openvswi-sg-chain (4 references)target prot opt source destination neutron-openvswi-i21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap21767f1f-45 --physdev-is-bridgedneutron-openvswi-o21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap21767f1f-45 --physdev-is-bridgedneutron-openvswi-i7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap7903fd30-74 --physdev-is-bridgedneutron-openvswi-o7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap7903fd30-74 --physdev-is-bridgedACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain neutron-openvswi-i7903fd30-7 (1 references)target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALIDRETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHEDRETURN icmp -- 0.0.0.0/0 0.0.0.0/0 RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22RETURN udp -- 50.50.1.3 0.0.0.0/0 udp spt:67 dpt:68neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0
Chain neutron-openvswi-o7903fd30-7 (2 references)target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:DB:08:63RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67DROP all -- !50.50.1.2 0.0.0.0/0 DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALIDRETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHEDRETURN all -- 0.0.0.0/0 0.0.0.0/0 neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0
[1] Note, OpenStack uses iptables rules on the TAP devices such as “tap~~” to implement security groups.However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an Open vSwitch port.
Neutron OVS plugin NameSpace - VLAN, GRE
janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 ifconfiglo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 qg-fa243f49-d6 Link encap:Ethernet HWaddr fa:16:3e:9f:4b:63 inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe9f:4b63/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 qr-bc654dc2-f1 Link encap:Ethernet HWaddr fa:16:3e:c7:ec:bd inet addr:50.50.1.1 Bcast:50.50.1.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fec7:ecbd/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 route Kernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Ifacedefault 192.168.122.1 0.0.0.0 UG 0 0 0 qg-fa243f49-d650.50.1.0 * 255.255.255.0 U 0 0 0 qr-bc654dc2-f1192.168.122.0 * 255.255.255.0 U 0 0 0 qg-fa243f49-d6
Neutron OVS plugin Floating-IP(NAT) - VLAN, GRE
janghoon@Network-node:~$ sudo ip netns showqdhcp-4c2f2346-ffaa-41a0-ab76-34cadf0163f5qrouter-e1b88ce4-51e9-4744-be80-d70d04c6a59bqdhcp-c19e22a0-1700-4b3b-91e5-2c961ef0a353qrouter-244fff3f-f935-4bdd-949d-739f1ce81dd0qdhcp-f37b681a-4be8-47b8-8063-3d17d24ee1aeqrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0
janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 iptables -L -n -t natChain neutron-l3-agent-PREROUTING (1 references)target prot opt source destination REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.1.2
Chain neutron-l3-agent-float-snat (1 references)target prot opt source destination SNAT all -- 50.50.1.2 0.0.0.0/0 to:192.168.122.51
Chain neutron-l3-agent-snat (1 references)target prot opt source destination neutron-l3-agent-float-snat all -- 0.0.0.0/0 0.0.0.0/0 SNAT all -- 50.50.1.0/24 0.0.0.0/0 to:192.168.122.50
Floating-IP(NAT)
NameSpace
Installation - SDN
Controller node
Keystone
Network node Compute node - 1 Compute node - 2
Nova
Glance Horizon
Quantum plugin ryu-agent Quantum plugin
ryu-agent
Nova compute
eth1 eth2
eth0
eth1 eth2
eth0
eth1 eth2
eth0
eth1 eth2
eth0
Management 192.168.20.0/24
Data 192.168.10.0/24
External network 192.168.122.0/24
Ryu-manager
Quantum metadata-agent
Quantum L3/dhcp-agent
Quantum plugin ryu-agent
Nova compute
Quantum - Server
Overview
Controller node Network node
Compute node Compute node
Quantum - Server
AMQP
Ryu-manager
ovs-vswitchd
ryu-agent
ovs-vswitchd
ryu-agent
REST API
OpenFlow OVSDB protocol
Big picture - Neutron Ryu plugin
OpenStack Grizzly Ryu plugin GRE tunneling
OVS port
OVS Bridge
● qg~~~ : external gateway interface● qr~~~ : virtual router interface
Network node
net_proj_one net_proj_two net_proj_new
Compute node - 1
br-ex
qg~
VM VM
tap~tag: 1
tap~tag:2
br-int
Tunnel
qg~ qg~
eth0
ns~ ns~ ns~
br-int
qr~ qr~
gre~ g
re~
Data 192.168.10.0
/24qr~
Packet conversion
Neutron Ryu plugin - Compute node
Compute node - 1
VM VM
tap~ tap~
VM
tap~
Security Group[1]
VM
tap~
br-int
OpenStack Grizzly Ryu plugin GRE tunneling
gre
~Tunnel
set_tunnel id
Neutron Ryu plugin - Compute node
janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-intNXST_FLOW reply (xid=0x4): cookie=0x0, duration=90146.068s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=3 actions=drop cookie=0x0, duration=90146.989s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=4 actions=drop cookie=0x0, duration=90146.068s, table=0, n_packets=3273, n_bytes=643066, tun_id=0x2,in_port=4 actions=resubmit(,2) cookie=0x0, duration=90146.068s, table=0, n_packets=4720, n_bytes=1164172, in_port=3,dl_src=fa:16:3e:cf:dc:42 actions=set_tunnel:0x2,resubmit(,1) cookie=0x0, duration=90146.068s, table=1, n_packets=6, n_bytes=468, priority=8192,tun_id=0x2 actions=resubmit(,2) cookie=0x0, duration=90146.068s, table=1, n_packets=1504, n_bytes=483460, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:ff:ff:ff actions=output:4,resubmit(,2) cookie=0x0, duration=90146.068s, table=1, n_packets=3000, n_bytes=659756, tun_id=0x2,dl_dst=fa:16:3e:a2:0e:f1 actions=output:4,resubmit(,2) cookie=0x0, duration=90146.068s, table=1, n_packets=210, n_bytes=20488, tun_id=0x2,dl_dst=fa:16:3e:ee:aa:8c actions=output:4,resubmit(,2) cookie=0x0, duration=90146.068s, table=2, n_packets=3216, n_bytes=680712, priority=8192,tun_id=0x2 actions=drop cookie=0x0, duration=90146.068s, table=2, n_packets=1610, n_bytes=487912, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:ff:ff:ff actions=output:3 cookie=0x0, duration=90146.068s, table=2, n_packets=3167, n_bytes=638614, tun_id=0x2,dl_dst=fa:16:3e:cf:dc:42 actions=output:3
Flow table
NamespaceNamespaceNamespace
Neutron Ryu plugin - Network node
eth0
qr~
qg~
qr~
qg~
qr~
qg~
br-ex
Packet conversion
set_tunnel id
net_proj_one
net_proj_two
net_proj_new
Network node
Floating-IP(NAT)
OpenStack Grizzly Ryu plugin GRE tunnelingg
re~
tap~ tap~
tap~ tap~ tap~
tap~
Namespace
ns~
Namespace Namespace
ns~ ns~
br-int
tap~ tap~ tap~
veth pair
Neutron Ryu plugin - Network node
janghoon@network:~$ sudo ovs-ofctl dump-flows br-intNXST_FLOW reply (xid=0x4): cookie=0x0, duration=144003.213s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=3 actions=drop cookie=0x0, duration=142257.013s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=4 actions=drop cookie=0x0, duration=144003.261s, table=0, n_packets=0, n_bytes=0, priority=16384,in_port=2 actions=drop cookie=0x0, duration=142256.093s, table=0, n_packets=7335, n_bytes=1825414, tun_id=0x2,in_port=4 actions=resubmit(,2) cookie=0x0, duration=144003.261s, table=0, n_packets=4748, n_bytes=977976, in_port=2,dl_src=fa:16:3e:a2:0e:f1 actions=set_tunnel:0x2,resubmit(,1) cookie=0x0, duration=144003.213s, table=0, n_packets=544, n_bytes=58344, in_port=3,dl_src=fa:16:3e:ee:aa:8c actions=set_tunnel:0x2,resubmit(,1) cookie=0x0, duration=144003.261s, table=1, n_packets=27, n_bytes=5010, priority=8192,tun_id=0x2 actions=resubmit(,2) cookie=0x0, duration=142256.093s, table=1, n_packets=113, n_bytes=4746, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:ff:ff:ff actions=output:4,resubmit(,2) cookie=0x0, duration=142256.093s, table=1, n_packets=4914, n_bytes=998000, tun_id=0x2,dl_dst=fa:16:3e:cf:dc:42 actions=output:4,resubmit(,2) cookie=0x0, duration=144003.261s, table=2, n_packets=5177, n_bytes=1031490, priority=8192,tun_id=0x2 actions=drop cookie=0x0, duration=144003.253s, table=2, n_packets=504, n_bytes=49439, tun_id=0x2,dl_dst=fa:16:3e:ee:aa:8c actions=output:3 cookie=0x0, duration=144003.261s, table=2, n_packets=4733, n_bytes=1041550, tun_id=0x2,dl_dst=fa:16:3e:a2:0e:f1 actions=output:2 cookie=0x0, duration=144003.261s, table=2, n_packets=2495, n_bytes=769266, priority=16384,tun_id=0x2,dl_dst=ff:ff:ff:ff:ff:ff actions=output:2,output:3
Flow table
Neutron Ryu plugin Security Group
FORWARD
quantum-filter-top
quantum-ryu-agen-FORWARD
quantum-ryu-agen-local
quantum-ryu-agen-sg-chain
quantum-ryu-agen-iTAP_NUMBER
quantum-ryu-agen-oTAP_NUMBER
quantum-ryu-agen-sg-fallback
quantum-ryu-agen-sg-fallback
Security group is applied here
Neutron Ryu plugin Security Group
Chain quantum-ryu-agen-sg-chain (2 references)target prot opt source destination quantum-ryu-agen-ib7fa734b-e all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tapb7fa734b-e0 --physdev-is-bridgedquantum-ryu-agen-ob7fa734b-e all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapb7fa734b-e0 --physdev-is-bridgedACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain quantum-ryu-agen-ib7fa734b-e (1 references)target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALIDRETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHEDRETURN tcp -- 192.168.228.122 0.0.0.0/0 tcp dpt:80RETURN udp -- 50.50.2.2 0.0.0.0/0 udp spt:67 dpt:68quantum-ryu-agen-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0
Chain quantum-ryu-agen-ob7fa734b-e (2 references)target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:CF:DC:42RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67DROP all -- !50.50.2.4 0.0.0.0/0 DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALIDRETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHEDRETURN all -- 0.0.0.0/0 0.0.0.0/0 quantum-ryu-agen-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0
[1] Note, OpenStack uses iptables rules on the TAP devices such as “tap~~” to implement security groups,.However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an Open vSwitch port.
Neutron Ryu plugin NameSpace
janghoon@network:~$ sudo ip netns exec qrouter-f7f07d55-4fd6-4f95-a45f-d6b1f0cf8d18 ifconfiglo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1
qg-afcc5de0-46 Link encap:Ethernet HWaddr fa:16:3e:62:e4:4b inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe62:e44b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
qr-33616671-f3 Link encap:Ethernet HWaddr fa:16:3e:ee:aa:8c inet addr:50.50.2.1 Bcast:50.50.2.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:feee:aa8c/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 janghoon@network:~$ sudo ip netns exec qrouter-f7f07d55-4fd6-4f95-a45f-d6b1f0cf8d18 routeKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Ifacedefault 192.168.122.1 0.0.0.0 UG 0 0 0 qg-afcc5de0-4650.50.2.0 * 255.255.255.0 U 0 0 0 qr-33616671-f3192.168.122.0 * 255.255.255.0 U 0 0 0 qg-afcc5de0-46
Neutron Ryu plugin Floating-IP(NAT)
janghoon@network:~$ sudo ip netns exec qrouter-f7f07d55-4fd6-4f95-a45f-d6b1f0cf8d18 iptables -L -n -t natChain quantum-l3-agent-PREROUTING (1 references)target prot opt source destination REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.2.4
Chain quantum-l3-agent-float-snat (1 references)target prot opt source destination SNAT all -- 50.50.2.4 0.0.0.0/0 to:192.168.122.51
Chain quantum-l3-agent-snat (1 references)target prot opt source destination quantum-l3-agent-float-snat all -- 0.0.0.0/0 0.0.0.0/0 SNAT all -- 50.50.2.0/24 0.0.0.0/0 to:192.168.122.50
Floating-IP(NAT)
Ryu-Controller
[DEFAULT]app_lists = ryu.app.gre_tunnel,ryu.app.quantum_adapter,ryu.app.rest,ryu.app.rest_conf_switch,ryu.app.rest_quantum,ryu.app.rest_tunnel,ryu.app.tunnel_port_updater
wsapi_host = 0.0.0.0wsapi_port = 8080ofp_listen_host = 0.0.0.0ofp_tcp_listen_port = 6633
quantum_url=http://192.168.20.10:9696quantum_admin_username=quantumquantum_admin_password=*********quantum_admin_tenant_name=servicequantum_admin_auth_url=http://192.168.20.10:35357/v2.0quantum_auth_strategy=keystonequantum_controller_addr = tcp:192.168.20.11:6633
Configuration - ryu.conf
Neutron ML2
The Modular Layer 2 (ML2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety of layer 2 networking technologies found in complex real-world data centers. It currently works with the existing openvswitch, linuxbridge, and hyperv L2 agents, and is intended to replace and deprecate the monolithic plugins associated with those L2 agents.
Neutron
TypeDriver
VLAN
ML2 Plugin
GRE VxLAN Flat
MechanismDriver
Op
envSwitch
Hyp
er-V
Op
enDaylig
ht
Arista
Cisco
Nexus
pSwitch
TypeDriver : TypeDrivers maintain any needed type-specific network state, and perform provider network validation and tenant network allocation. MechanismDriver : The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied given the specific networking mechanisms that have been enabled.
https://wiki.openstack.org/wiki/Neutron/ML2
Neutron ML2
Network node Compute node - 1 Compute node - 2
Neutron ML2-agentNeutron
ML2-agent
Nova compute
eth0
eth1 eth2 eth1 eth2
eth0
eth1 eth2
eth0
Neutron server
Neutron metadata-agent
Neutron L3/dhcp-agent
Neutron ML2-agent
Nova compute
* Another option
Cisco and Canonical are collaborating to offer customers the Nexus 1000V virtual networking solution on Ubuntu Linux & Ubuntu OpenStack cloud orchestration for the first time. The solution will enable Nexus 1000V customers to embrace Ubuntu OpenStack, the largest commercial distribution of the open source cloud platform.
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-1000v-kvm/solution-overview-c22-730808.html
![[OpenStack Day in Korea 2015] Keynote 5 - The evolution of OpenStack Networking](https://img.pdfslide.us/doc/110x75/55a7530d1a28abf03c8b4675/openstack-day-in-korea-2015-keynote-5-the-evolution-of-openstack-networking.jpg)
