Upload
juergen-brendel
View
221
Download
2
Embed Size (px)
Citation preview
Simplifying the network stack with Romana
Pani NetworksOpenStack / Kubernetes Meetup, Wellington, May 2016
romana.io Simplifying the network stack with Romana @romanaproject
Agenda
● “Cloud native”, why does it matter?● A better network for cloud native architectures● Demos
romana.io Simplifying the network stack with Romana @romanaproject
About us
● Team background:– Data center networks
– Low-level traffic management
● Created L2 overlay network startup– Bought by Cisco
● OpenStack networking● There's got to be a better way
– Time is right
What is 'cloud native'?
romana.io Simplifying the network stack with Romana @romanaproject
The past: Enterprise networking
● Full control● Applications need L2 and L3
– May need hard-wired IP addresses
– Broadcasts
● Servers are pets, not cattle: “Careful!”– VM migration
● Complex!
romana.io Simplifying the network stack with Romana @romanaproject
Cloud native applications
● Automate all the things!– Infrastructure as code
– Cattle, not pets: “Meh... just kill it.”
– Workloads come and go quickly
– Build for resiliance
● IP is all you need– No hardcoded IP addresses, discovery
– No special network requirements
– Basic IP connectivity
The problem
romana.io Simplifying the network stack with Romana @romanaproject
We have a mismatch
● Building cloud native applications…● … on top of enterprise networking
– SDN controllers use overlay L2 domains
– VLAN, VXLAN, OVS, etc.
● Complexity and brittleness– Lose benefits of simplicity
– Lose performance (encap, blinded hardware)
– Difficult to maintain and trouble shoot
romana.io Simplifying the network stack with Romana @romanaproject
The price you pay: Complexity
VXLAN Decap
VXLAN Decap
VXLAN Encap
VXLAN Encap
2 Top of Rack Round Trips
East/West Traffic
Per Instance Security
romana.io Simplifying the network stack with Romana @romanaproject
The price you pay: Performance
Router
Endpoint A Endpoint B
Router
L2 overlay A
L2 overlay B
VRouter
romana.io Simplifying the network stack with Romana @romanaproject
Why do we do this to ourselves?
● We don't need any L2 features● Except maybe traffic segmentation
– Multi tenancy
– Tiers and policies
The solution
romana.io Simplifying the network stack with Romana @romanaproject
Networking the way it was intended
● Use native L3 capabilities● No overlays● De-emphasize IP address ranges● Still provide segmentation, multi tenancy● Simple, clear and scalable network setup
romana.io Simplifying the network stack with Romana @romanaproject
Truly cloud native networking
● Project Romana● Open source● Apache 2.0 license● Mostly written in Go● Kubernetes and OpenStack
romana.io Simplifying the network stack with Romana @romanaproject
Truly cloud native networking
● Use only IP routing– No overlays
– All workload addresses are 'real'
– Simplicity!
● Use smart addressing– Encode tenant or segment in IP address
– Assign “virtual” addresses with host prefixes
– Massive (!) collapse of route table
● Routes are static– No route updates, no broadcasts for new endpoint
romana.io Simplifying the network stack with Romana @romanaproject
Romana Architecture
● On each host: Agent– Configures routes– Connects endpoint interfaces– Sets policy implementations
● Controller: Cooperating microservices– Each service with RESTful interface– Specialized for different tasks
● Environment: Different integration points– APIs, drivers for various parts of OpenStack or
Kubernetes
romana.io Simplifying the network stack with Romana @romanaproject
Romana Architecture
Host A Host B Host C
Agent Agent Agent
Tenant
Topology
IPAM
Root
Environment (OpenStack or Kubernetes)
Policy
Beautifully simple networking
romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:192.168.8.11
Host B
eth0:192.168.8.22
Host C
eth0:192.168.8.33
romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:192.168.8.11
romana-gw:10.0.0.1/16
Host B
eth0:192.168.8.22
romana-gw:10.1.0.1/16
Host C
eth0:192.168.8.33
romana-gw:10.2.0.1/16
romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:192.168.8.11
romana-gw:10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Host B
eth0:192.168.8.22
romana-gw:10.1.0.1/16
10.1.3.52
10.1.9.2
Host C
eth0:192.168.8.33
romana-gw:10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6
romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:192.168.8.11
romana-gw:10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Host B
eth0:192.168.8.22
romana-gw:10.1.0.1/16
10.1.3.52
10.1.9.2
Host C
eth0:192.168.8.33
romana-gw:10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6
romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:192.168.8.11
romana-gw:10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Host B
eth0:192.168.8.22
romana-gw:10.1.0.1/16
10.1.3.52
10.1.9.2
Host C
eth0:192.168.8.33
romana-gw:10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6
romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:192.168.8.11
romana-gw:10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Routes:10.1/16 → 192.168.8.2210.2/16 → 192.168.8.33
Host B
eth0:192.168.8.22
romana-gw:10.1.0.1/16
10.1.3.52
10.1.9.2
Routes:10.0/16 → 192.168.8.1110.2/16 → 192.168.8.33
Host C
eth0:192.168.8.33
romana-gw:10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6
Routes:10.0/16 → 192.168.8.1110.1/16 → 192.168.8.22
romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
Rack A Rack B
romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
Host A2 Routes
0.0.0.0 192.168.1.200→10.68/14 192.168.1.1→10.76/14 192.168.1.3→10.80/14 192.168.1.4→
romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
ToR A Routes
10.128/10 192.168.2.200→10.68/14 192.168.1.1→10.72/14 192.168.1.2→10.76/14 192.168.1.3→10.80/14 192.168.1.4→
Host A2 Routes
0.0.0.0 192.168.1.200→10.68/14 192.168.1.1→10.76/14 192.168.1.3→10.80/14 192.168.1.4→
romana.io Simplifying the network stack with Romana @romanaproject
Larger network: Full L3
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
ToR A Routes
10.128/10 192.168.2.200→10.68/14 192.168.1.1→10.72/14 192.168.1.2→10.76/14 192.168.1.3→10.80/14 192.168.1.4→
Host Routes
0.0.0.0 192.168.1.200→
Scalable distributed firewalland
traffic policies
romana.io Simplifying the network stack with Romana @romanaproject
Romana: Traffic segmentation
● Tenant traffic separated:– Tenants don't get whole CIDR prefix or L2 domain
– But fully isolated from other tenants' traffic
● Tenants can define segments:– Like tiers, provide isolation and policies
● Use segment and tenant bits in IP addresses:– Apply policies (iptables) based on that
– Segments can stretch across hosts
romana.io Simplifying the network stack with Romana @romanaproject
Semantic and topological addressing
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9 8 7 6 5 4 3 2 1 0
0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1
10
Network prefix bitsThe network prefix. In this example, we are using the 10/8
address space.
6
Host ID Segment IDWe currently
store tenant ID in upper bits of segment ID.
4 67
Endpoint ID
Widths are configurable, don't have to use byte boundaries.
romana.io Simplifying the network stack with Romana @romanaproject
Semantic and topological addressing
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9 8 7 6 5 4 3 2 1 0
0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1
10
Network prefix bitsThe network prefix. In this example, we are using the 10/8
address space.
6
Host ID Segment IDWe currently
store tenant ID in upper bits of segment ID.
4 67
Endpoint ID
Widths are configurable, don't have to use byte boundaries.
Encode thetenant ID
romana.io Simplifying the network stack with Romana @romanaproject
Host BHost A
Allowing traffic within tenant
10.0.0.5 10.1.0.12
iptables:check src/dst addrs“tenant/segment bits
must match”
Src: 10.0.0.5Dst: 10.1.0.12
Same tenant/segment bits
romana.io Simplifying the network stack with Romana @romanaproject
Host BHost A
Isolating tenant traffic: Default
10.0.0.5 10.1.128.9
iptables:check src/dst addrs“tenant/segment bits
must match”
Src: 10.0.0.5Dst: 10.1.128.9
Different tenant/segment bits
Differenttenant
romana.io Simplifying the network stack with Romana @romanaproject
Host BHost A
Apply network policy between segments (full isolation as default)
10.0.0.5 10.1.1.9
iptables:Does policy chain
exist?Otherwise: DROP
Src: 10.0.0.5Dst: 10.1.1.9
Same tenant, different segment
policy-chain:From segment 0?Protocol TCP?To port 80?
Demo 1:
Kubernetes + Romana clusteron top of Catalyst OpenStack cloud
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Jump host withpublic IP address
romana.io Simplifying the network stack with Romana @romanaproject
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Install OpenStackcommand line tools
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
$ neutron port-update \ e925b70e-031e-4ef7-a27c-583b4b775290 \ --allowed-address-pairs type=dict list=true \ mac_address=fa:16:3e:e1:df:59,ip_address=10.0.0.0/8
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
$ git clone https://github.com/romana/romana$ cd romana/romana-install$ ./romana-setup -p static -i my-inventory -s kubernetes install
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Romanainstaller
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Kubernetes + Romana
Romana clusteraddress range:
10/8
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Kubernetes + Romana
Podswith containers.
Pods have RomanaIP addresses.
romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - What you will see
● Creation of pods● Network configuration● Application of network policies
Demo 2:
Mixing containers with legacy workloads
romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - Overview
bar-1 bar-2foo
Kubernetes + Romana
romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - Overview
bar-1 bar-2foo
Kubernetes + Romana
vm-workload
Legacy applicationin VM
romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - Overview
bar-1 bar-2foo
Kubernetes + Romana
vm-workload
Direct connection:- No gateway- No encap/decap- No NAT
romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - What you will see
● Creation of pods● Contact pod from VM● See the packet route
Demo 3:
Romana + Kubernetes clusteron top of Romana + OpenStack cluster
romana.io Simplifying the network stack with Romana @romanaproject
Baking layered cakes
● Kubernetes on OpenStack? Why?– On demand clusters
– Full tenant isolation
● Really nice with fully routed networking– No double encapsulation
– Logical, efficient packet forwarding
● Not all workloads fit into containers– Seamless connection between pods and VMs
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
HW1 HW2 HW3 HW4
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
HW1 HW2 HW3 HW4
$ ./romana-setup -p static -i hw-inventory -s devstack install
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
HW1 HW2 HW3 HW4
OpenStack + Romana
Romana cluster 1address range:
10/8
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
OpenStack VMs
VMs haveIP addresses
ofRomana cluster 1
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
$ ./romana-setup -p static -i vm-inventory -s kubernetes install
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3
Kubernetes + Romana
VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
Romana cluster 2address range:
172.16/12
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3
Kubernetes + Romana
VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
Podswith containers.
Pods haveIP addresses
ofRomana cluster 2
romana.io Simplifying the network stack with Romana @romanaproject
OpenStack + Romana
Kubernetes + Romana
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
romana.io Simplifying the network stack with Romana @romanaproject
OpenStack + Romana
Kubernetes + Romana
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
Remember this one?
2 Top of Rack Round Trips
East/West Traffic
Per Instance Security
Without pure L3 networklayered clusters
would be even morecomplex.
romana.io Simplifying the network stack with Romana @romanaproject
OpenStack + Romana
Kubernetes + Romana
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
But with Romana, networkingeven in layered clusters becomes
really easy...
romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - What you will see
● Creation of pods● Pods and VMs with fully routable addresses● Ease of use showcase: Trouble shooting
romana.io Simplifying the network stack with Romana @romanaproject
Conclusion
● Cloud native architectures simplify things● Need cloud native networking to enjoy benefits● Romana:
– Cloud native without compromises
– Native network performance
– Mostly static config: Solid network
– Very easy to work with and understand
● Easy to try:– Simple installers for Kubernetes and OpenStack
romana.io Simplifying the network stack with Romana @romanaproject
Thank you!
● Romana Links– http://romana.io - Project home
– http://romana.io/blog - Blog
– https://github.com/romana/romana - Sources
● Contact– @romanaproject - Twitter
– [email protected] - Email
– https://romana.slack.com/ - Slack channel