Embed Size (px)
Citation preview
ISP Design – Using a full table RR to improve eBGPperformance with MikroTikroutersP R ES E N T E D BY:
KEVIN MYERS , NETWORK ARCHITECT
www.iparchitechs.com
Profile: About Kevin Myers
Background:• 20+ years in Networking• Designed/Built Networks on 6 continents• MikroTik Certified Trainer • MikroTik, Cisco and Microsoft Certified
Community Involvement:
Packet Pushers (Podcast Guest / Blogger)
Group Contributor (RouterOS / WISP Talk and others)
Delegate/Roundtable contributor (NFD14)
MT Forum (Forum Veteran – Member since 2012)
Network Collective (Podcast Guest)
Profile: About IP ArchiTechs
Expert NetworkingWhitebox | ISP | Data Center | Enterprise
Locations in: US | Canada | South America
✓ Global Consulting✓ Managed Networks✓ Monitoring✓ Load Testing✓ Development
Call us at: +1 855-645-7684E-mail: [email protected]: www.iparchitechs.com
Profile: About IP ArchiTechs
Now in Europe!IPA Opened an office in Nis, Serbia in 2018
Design: Why use a route reflector for full tables
Goal of this presentation: When the presentation is finished, hopefully you will have walked away with a few key concepts:
• Performance limitations of using full mesh peering between BGP border routers
• How to leverage open source software to create a high performance BGP RR for MikroTik border routers
• Design benefits of using a BGP full table RR
Design: CHR vs. Hardware for full tables?
vs.
• Which platform is better?
• Throughput capabilities?
• x86 CPU vs. ARM/Tilera?
Design: CHR vs. Tilera/ARM for BGP Border?
Platform
CPUMPLS router CPU requirements depend on load and explicit/implicit null
x86Better for heavy computational work. Higher power draw.
TileraOptimized for packet transfer. Designed to be low power draw.
ARMIn between x86 and
Tilera for performance.
ThroughputAt 1530 bytes (L2), and 8970 bytes (L2)
x86More CPU and power is required to move data at the same speed as a CCR
TileraHandles throughput at different frame sizes slightly better than x86
ARMHandles throughput at different frame sizes similar to Tilera
Performance for full tables
x86x86 hardware with HyperV is the best RouterOS based solution for full BGP tables
TileraLowest performance for full tables
ARMPerformance is in between Tilera and x86 but the best hardware platform from MikroTikfor rapid convergence
Design: What is Free Range Routing?
• FRRouting (FRR) is an open source IP routing protocol suite for Linux and Unix platforms
• includes protocol daemons for BGP, IS-IS, LDP, OSPF, PIM, and RIP.Use MikroTik CCRs or CHRs for BGP border routers to handle throughput
• FRR has its roots in the Quagga project. In fact, it was started by many long-time Quagga developers who combined their efforts to improve on Quagga's well-established foundation.
Design: Why use Free Range Routing?
• FRR is multithreaded for BGP and can improve the performance of MikroTik BGP border routers by acting as a route reflector
• Free software since it is open source – can be compiled from the source or downloaded as VM using CumulusVX.
• Has been traditionally used as an IX route server and can handle a large volume of Full Tables
Design: Which design for full table route reflection?
• Free Range Routing for BGP RR – Open source routing package that supports BGP across multiple CPUs. Doesn’t need throughput – out of path
• Use MikroTik CCRs or CHRs for BGP border routers to handle throughput
• Leverage the strength of Open Source software to increase the performance and scalability of RouterOS as a BGP border router
Free Range
Routing
+
Design: BGP RR - logical lab setup
BGP AS 8675309
AS 5459
BGP AS 11200
bgp2.tutone.net
CCR
Full Table
BGP RR
100.76.1.0/29
VLAN 761
AS 6695
Test Network for full table RR
tutone.net bgp1.tutone.net
CCR
AS 1200
100.76.3.0/29
VLAN 763
bgp3.tutone.net
CCR
100.76.2.0/29
VLAN 762.11
.1
.2
CHR
CHR
CHR
BGP AS 16695
BGP AS 15459
.3
Free Range
Routing
100.126.0.0/24
Design: BGP RR – switch–centric lab setup
Physical
Test Network for
BGP RR
HP 1U x86 Server
14 UID 1iLO 4
500W
94%
CHR
Baltic Networks Vengeance HV
BGP Border routers
CHR Border routers
Core switch
CHR BGP Full Feeds
Design: Full Table BGP RR performance
• Concept of testing
• Performance with 3 full tables on CCRs & BGP full table RR
• Performance with 3 full tables on CCRs & full mesh peerings
• Performance with 4 full tables on CCRs + CHR & BGP full table RR
• Performance with 4 full tables on CCRs + CHR & full mesh peerings
• Performance with 3 full tables on CHR & BGP full table RR
• Performance with 3 full tables on CHR & full mesh peerings
BGP Border BGP RR Hypervisor
CCR1036 (2)CCR1072 (1)
Free Range Routing ESXi 6.0
Total routes: 1,757,631 IP Transit Convergence: 2:55 Full Convergence: 8:51
Design: Full Table BGP RR performance
BGP Border BGP RR Hypervisor
CCR1036 (2)CCR1072 (1)
None ESXi 6.0
Total routes: 1,757,631 IP Transit Convergence: 6:58 Full Convergence: 8:59
Design: Full Table BGP performance (Full Mesh Peerings)
Design: BGP RR - logical lab setup
BGP AS 8675309
AS 5459
BGP AS 11200
bgp2.tutone.net
CCR
Full Table
BGP RR
100.76.1.0/29
VLAN 761
AS 6695
Test Network for full table RR – 4 peers (CCR+CHR)
tutone.net bgp1.tutone.net
CCR
AS 1200
100.76.3.0/29
VLAN 763
bgp3.tutone.net
CCR
100.76.2.0/29
VLAN 762.11
.1
.2
CHR
CHR
CHR
BGP AS 16695
BGP AS 15459
.3
Free Range
Routing
100.126.0.0/24
100.76.4.0/29
VLAN 764
AS 13538
bgp4.tutone.net
CHR
BGP AS 23538
CHR
BGP Border BGP RR Hypervisor
CCR1036 (2)CCR1072 (1)CHR (1) HyperV
Free Range Routing ESXi 6.0
Total routes: 2,272,474 IP Transit Convergence: 5:33 Full Convergence: 16:51
Design: Full Table BGP RR performance
BGP Border BGP RR Hypervisor
CCR1036 (2)CCR1072 (1)CHR (1) HyperV
None ESXi 6.0
Total routes: 2,272,474 IP Transit Convergence: 7:27 Full Convergence: 17:01
Design: Full Table BGP performance (Full Mesh Peerings)
Design: BGP RR - logical lab setup
BGP AS 8675309
AS 5459
BGP AS 11200
bgp5.tutone.net
CHR
Full Table
BGP RR
100.76.4.0/29
VLAN 764
AS 6695
Test Network for full table RR – CHR Border
tutone.net bgp4.tutone.net
CHR
AS 1200
100.76.6.0/29
VLAN 766
bgp6.tutone.net
CHR
100.76.5.0/29
VLAN 765.11
.1
.2
CHR
CHR
CHR
BGP AS 16695
BGP AS 15459
.3
Free Range
Routing
100.126.0.0/24
BGP Border BGP RR Hypervisor
CHR (3) HyperV Free Range Routing ESXi 6.0
Total routes: 2,272,474 IP Transit Convergence: 00:17 Full Convergence: 00:41
Design: Full Table BGP RR performance
BGP Border BGP RR Hypervisor
CHR (3) HyperV Free Range Routing ESXi 6.0
Total routes: 2,272,474 IP Transit Convergence: 00:35 Full Convergence: 2:00
Design: Full Table BGP performance (Full Mesh Peerings)
Design: Full Table BGP RR performance
• Performance Conclusions
• BGP RR can help improve the performance of existing CCRs in the load time of the route table from the upstreams
• Does not significantly impact overall convergence time
• As the number of IX and IP Transit peers increases, RR becomes more useful for performance and scale
• CHR + BGP RR achieves the best performance and scale options
Design: BGP RR design
Design Advantages of BGP RR
Design: BGP RR design – Customer peering
BGP AS 8675309
AS 5459
BGP AS 11200
bgp5.tutone.net
CHR
Full Table
BGP RR
100.76.4.0/29
VLAN 764
AS 6695
Full table RR to provide IP Transit
tutone.net bgp4.tutone.net
CHR
AS 1200
100.76.6.0/29
VLAN 766
bgp6.tutone.net
CHR
100.76.5.0/29
VLAN 765.11
.1
.2
CHR
CHR
CHR
BGP AS 16695
BGP AS 15459
.3
Free Range
Routing
100.126.0.0/24
BGP AS 1 – Downstream Customer
RR Client Peering
Customer UpstreamCustomer Aggregation
Public /30
Design: BGP RR design – Customer peering
BGP AS 8675309
AS 5459
BGP AS 11200
bgp5.tutone.net
CHR
Full Table
BGP RR
100.76.4.0/29
VLAN 764
AS 6695
Use BGP Communities to select upstreams
tutone.net bgp4.tutone.net
CHR
AS 1200
100.76.6.0/29
VLAN 766
bgp6.tutone.net
CHR
100.76.5.0/29
VLAN 765.11
.1
.2
CHR
CHR
CHR
BGP AS 16695
BGP AS 15459
.3
Free Range
Routing
100.126.0.0/24
BGP AS 1 – Downstream Customer
RR Client Peering
Customer UpstreamCustomer Aggregation
Public /30
65000:5459
65000:6695
65000:1200
Filter 65000:5459 at the PE
Design: Questions?
Questions??
