Upload
mark-bond
View
248
Download
6
Tags:
Embed Size (px)
Citation preview
VXLAN Nexus 9000 Module 5 – MP-BGP EVPN
@onecloudinc.com
2© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Agenda
MP-BGP EVPN Overview VXLAN with EVPN Control Plane Route Optimization using EVPN VXLAN EVPN Features Add Control Plane EVPN For
Ingress Replication VXLAN EVPN Configuration VXLAN EVPN Design Options Scalability Limits
3© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
MP-BGP EVPN Overview
4© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Multiprotocol-BGP (MP-BGP): Carries multiple address-family reachability info (IPv4, IPv6, VPN, EVPN)
MP-iBGP: Internal BGP session between internal peers
MP-eBGP: External BGP session between external peers (in different BGP autonomous systems)
Route-reflector (RR): Propagate BGP prefixes to iBGP peers
Route Distinguisher (RD): 8-byte field, VRF parameters; unique value to make VPN IP routes unique
VPNv4 address: RD + VPN prefix
Route Target (RT): 8-byte field, unique value to define the import/export rules for VPNv4 routes
VRF Table: Hold customer routes at PE/VTEP
VNI: Virtual Network Interface
PE: Provider Edge Router
CE: Customer Router
MP-BGP and E-VPN Terminology
5© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Extension to Border Gateway Protocol (BGP)
Multi-Protocol BGP (MP-BGP) carries multiple address family information IPv4 (Unicast and Multicast) IPv6 (Unicast and Multicast) VPNv4, MVPN, EVPN (Ethernet VPN)
Address families controls type of routed protocol information exchanged
Used to distribute VPN routing information between PE’s
Allows to distinguish overlapping customer routes using Route Distinguisher (RD)
Allows reachability among different virtual networks using Route Target (RT)
Customer routes held in different VPN tables
Multi-Protocol BGP (MP-BGP)
6© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Exchange of VPN Policies Among PE Routers
MP-BGP with MPLS VPN Route Distribution
Full mesh of BGP sessions among all PE routers BGP Route Reflector
Multi-Protocol BGP extensions (MP-iBGP) to carry VPN policies
PE-CE routing options Static routes eBGP OSPF IS-IS
Label Switched Traffic
P
PPE
CE
PE-CELink
PE-CELink
P
P
CEPE
PE
CE
CE
Blue VPN Blue VPN
Red VPN Red VPN
BGP Route Reflector
PE
7© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN with EVPN Control Plane
8© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN Evolution
Uses Multi-Protocol BGP (IBGP or EBGP) w EVPN Address Family for: Dynamic Tunnel Discovery Dynamic Host reachability information
VTEP VTEP VTEP VTEP VTEP
RouteReflector
RouteReflector
IBGP Route Reflector*(on spine or different
box)
VXLAN Overlay
BGP Peers on VTEPs
Scalable BGP EVPN Control Plane
9© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Control Protocol for VXLAN
Protocol Learning
9
East
South
VTEP
West
BGP propagates routes for the host to all other VTEPs
2
VTEP
VTEP
Overlay Forwarding TableHost1 <MAC,IP> , VTEP IP AHost2 <MAC,IP> , VTEP IP B
3
1
VTEPs advertise host routes (IP+MAC) to RR
IP A IP B
IP C
BGP Route Reflect
or
Overlay Forwarding TableHost1 <MAC,IP> , VTEP IP A
3VTEPs obtain host routes for remote hosts and install in RIB/FIB
BGP MPLS Based Ethernet VPN (draft-ietf-l2vpn-evpn-02) Network Virtualization Overlay Solution using EVPN (draft-sd-l2vpn-evpn-overlay-02)
IETF
Protocol Learning
10© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
BGP : The Advantages
Scale Policy Security
11© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
With EVPN Address Family
Carries Host L2-address Carries Host IP/L3 address
Bridge Route
Overlay
Optimally
Active/Active Multipathing
Resilient & Efficient
No Tromboning
12© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Using MP-BGP to distribute overlay information Use MP-BGP with EVPN Address Family for Tunnel Endpoint Discovery and Host Reachability Distributed protocol – no single point of failure Proven BGP scalability Physical and virtual endpoints can become BGP peers to participate
Why not use a Central Controller? Single point of control could become a single point of failure Scalability Scope of the controller is limited to devices that understand the communication protocol
used by the controller
Why VXLAN EVPN Control Plane?
13© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
IGP for underlay network reachability
Node reachability for overlay
Quick reaction to fabric link/node failure
Enhanced for mesh topologies
Underlay Control Protocol doesn’t distribute
Host Routes
Host originated control traffic
Server subnet informationOSPF Adjacencies
VXLAN Underlay - Unicast
L3 Core
Step 1 – OSPF/IGP as Underlay Control Plane
OSPF Control Plane
14© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Multicast for underlay network
VXLAN VTEP peer discovery
Forward BUM traffic (broadcast, unknown and multicast)
Multicast Enabled Fabric
PIM Adjacencies
VXLAN Underlay - MulticastStep 2 – Enable Multicast on Underlay network
L3 Core
15© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Host Route Distribution decoupled from the Underlay IGP protocol
Use MP-BGP on the leaf nodes to distribute internal host/subnet routes and external reachability information
MP-BGP enhancements to carry up to 100s of thousands of routes and reduce convergence time
Step 3 – Host and Subnet Route Distribution
iBGP Adjacencies
VXLAN Overlay MP-BGP EVPN Control Plane
L3 Core
Host/Subnet Route Injection External
Subnet Route Injection
MP-BGP Control Plane
Route-Reflectors deployed for scaling purposes
RR
RR
16© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Control-Plane EVPNPeer and Host discovery
17© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN Peer and Host Learning Options
Host Learning
Data-Plane Control-Plane
Core
Multicast
UnicastVlan 2 vn-segment 4098Interface nve 1 host-reachability protocol bgp member vni 4098 ingress-replication protocol bgp
Vlan 2 vn-segment 4098Interface nve 1member vni 4098 ingress-replication protocol static
Vlan 2 vn-segment 10000Interface nve 1 host-reachability protocol bgp member vni 4098 mcast-group 225.1.1.1
Vlan 2 vn-segment 4098Interface nve 1member vni 10000 mcast-group 225.1.1.1
Flood and LearnPeer Learning: DP
EVPN-MulticastPeer Learning: BGP
Static Ingress-ReplicationPeer Learning: CLI
EVPN Ingress-ReplicationPeer Learning: BGP
18© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Dynamic VTEP Peer Discovery
VTEP-2
VTEP-1
VTEP-3
H-MAC-1H-IP-1VLAN-1 /VNI-1
BGP Update:H-MAC-1H-IP-1VTEP-1VNI-1
BGP Update
BGP Update
Route Reflector
Dynamic VTEP peer discovery:
VTEP1 peer via EVPN MP-BGP Update
Local learning of host info:
H-MAC-1 (MAC table)
H-IP-1 (VRF IP host table )
1
2
3 3
44
Dynamic VTEP peer discovery:
VTEP1 peer via EVPN MP-BGP Update
With EVPN Control Plane: Underlay multicast is NOT needed for VTEP peer discovery
19© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
MAC Host IP
VNI VTEP
H-MAC-1
H-IP-1 VNI-1
VTEP-1
Dynamic Host Reachability Advertisement
BGP Update:H-MAC-1H-IP-1VTEP-1VNI-1
BGP Update:H-MAC-1H-IP-1VTEP-1VNI-1
Install host info to RIB/FIB:
H-MAC-1 MAC table
H-IP-1 VRF IP host table
Install host info to RIB/FIB:
H-MAC-1 MAC table
H-IP-1 VRF IP host table
2
44
MAC Host IP
VNI VTEP
H-MAC-1
H-IP-1 VNI-1 VTEP-1
MAC Host IP
VNI VTEP
H-MAC-1
H-IP-1 VNI-1 VTEP-1
H-MAC-1
H-IP-1
VLAN-1 /VNI-1
Leaf must discover first locally connected devices
Local learning of host info:
H-MAC-1 (MAC table)
H-IP-1 (VRF IP host table )
1
Detection of local hosts using ARP/ND/DHCP (New mac learn)
Detection of remote hosts via Received MP-BGP updates
Route Reflector
VTEP-3
VTEP-1
VTEP-2
BGP Update:H-MAC-1H-IP-1VTEP-1VNI-1
3 3
20© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Use MP-BGP with EVPN Address Family on VTEPs to distribute:
Internal host MAC/IP addresses
Subnet routes
External reachability information
MP-BGP enhancements to carry up to 100s of thousands of routes with reduced convergence time
VTEP Functions are on leaf layer
Spine nodes don’t need to be VTEP
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update• Host-MAC• Host-IP• Internal IP Subnet• External Prefixes
MP-BGP for VXLAN EVPN Control PlaneEVPN Control Plane – Reachability Distribution
L2 L3 L4
S2 S3 S4 LeafVTEPVTEPVTEPVTEP
Spine
21© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Any subnet anywhere => Any leaf can instantiate any subnet
All leafs share gateway IP and MAC for a subnet (No HSRP)
ARPs are terminated on leafs, no flooding beyond leaf
Facilitates VM Mobility, workload distribution, arbitrary clustering
Seamless L2 or L3 communication between physical hosts and virtual machines
GW IP: 11.11.11.1GW MAC: 0011:2222:3333
GW IP: 10.10.10.1GW MAC: 0011:2222:3333
L3
L2
Anycast Gateway
Anycast Gateway at the Leaf
VTEP-1
VTEP-2
VTEP-3
VTEP-4
VTEP-5
VTEP-6
Optimized Network
22© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VTEP-1 detects Host1 and advertise an EVPN route for Host1 with seq# 0 Host1 Moves behind VTEP-3 VTEP-3 detects Host1 and advertises an EVPN route for Host1 with seq #1 VTEP-1 sees more recent route and withdraws its advertisement
MAC IP VNI Next-Hop Encap Seq
MAC-1 IP-1 5000 VTEP-1 VXLAN 0
MAC IP VNI Next-Hop Encap Seq
MAC-1 IP-1 5000 VTEP-3 VXLAN 1
VXLAN BGP Control Plane
Host 1MAC1IP 1
VNI 5000
EVPN Control Plane - Host Movement
VTEP-4VTEP-3VTEP-2VTEP-1
NLRI:• Host MAC1, IP1 • NVE IP 1• VNI 5000• Next-Hop: VTEP-3
Ext. Community:• Encapsulation:
VXLAN• Cost/Sequence: 1
NLRI:• Host MAC1, IP1 • NVE IP 1• VNI 5000• Next-Hop: VTEP-1
Ext. Community:• Encapsulation:
VXLAN• Cost/Sequence: 0
23© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VTEP-1 detects Host1 and advertise an EVPN route for Host1 with seq# 0 Host1 Moves behind VTEP-3 VTEP-3 detects Host1 and advertises an EVPN route for Host1 with seq #1 VTEP-1 sees more recent route and withdraws its advertisement
MAC IP VNI Next-Hop Encap Seq
MAC-1 IP-1 5000 VTEP-1 VXLAN 0
MAC IP VNI Next-Hop Encap Seq
MAC-1 IP-1 5000 VTEP-3 VXLAN 1
VXLAN BGP Control Plane
Host 1MAC1IP 1
VNI 5000
EVPN Control Plane - Host Movement
VTEP-4
VTEP-3
VTEP-2
VTEP-1
NLRI:• Host MAC1, IP1 • NVE IP 1• VNI 5000• Next-Hop: VTEP-3
Ext. Community:• Encapsulation: VXLAN• Cost/Sequence: 1
NLRI:• Host MAC1, IP1 • NVE IP 1• VNI 5000• Next-Hop: VTEP-1
Ext. Community:• Encapsulation: VXLAN• Cost/Sequence: 0
24© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Summary Classic Vs EVPN Control Plane
Classic VXLAN EVPN Control Plane
Peer Discovery Data-driven flood-&-learn MP-BGP
Host Route Learning Local hosts: Data-driven flood-&-learnRemote hosts: Data-driven flood-&-learn
Local Host: Data-drivenRemote host: MP-BGP
BUM Traffic forwarding Multicast replicationUnicast/Ingress replication
Multicast replicationUnicast/Ingress replicationMulticast Dependency Reduced
Route Optimization Hair-pinning, Need extra VTEPs and Routers
Routing on source VTEP, Anycast Gateway
25© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Control Plane EVPN Summary
• Head-end replication to allow unicast-mode only operation• Introduce a control plane to allow for dynamic VTEP
discoveryMulticast Dependency
• Workload MAC addresses are known once they are connected to the VXLAN capable devices
• Leverage the control plane also to exchange L2/L3 address-to-VTEP association information
Flood and Learn based Learning
• Introduce VXLAN GatewaysExternal Connectivity
26© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Route Optimization using EVPN
27© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN Routing
Different integrated Route/Bridge (IRB) Modes
Overlay Networks do follow two slightly different integrated Route/Bridge (IRB) semantics
Asymmetric
Ingress VTEP performs both Layer-2 bridging and Layer-3 routing lookup, whereas the egress VTEP performs only Layer-2 bridging lookup
Symmetric
Both the ingress and egress VTEPs perform Layer-2 and Layer-3 lookups
Cisco follows Symmetric IRB
Host 1H-MAC-1H-IP-1VNI-A
VTEP-4VTEP-3VTEP-2VTEP-1
Host 2H-MAC-2H-IP-2VNI-B
SVI A
SVI B
IP Transport Network
Routing ?
28© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Asymmetric IRB (Cont’d)
Host 1H-MAC-1H-IP-1VNI-A
VTEP-4VTEP-3VTEP-2VTEP-1
Host 2H-MAC-2H-IP-2VNI-B
VNI A VNI B VNI BVNI A
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1D-IP: H-IP-2
Ingress VTEP routes packets from source VNI to destination VNI. D-MAC in the inner header is the destination host MAC
1S-MAC: H-
MAC-1D-MAC: H-
MAC-2S-IP: H-IP-1D-IP: H-IP-2
S-IP: VTEP-1D-IP: VTEP-4
VNI: VNI-B
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1D-IP: H-IP-2 Egress VTEP
bridges packets in the destination VNI
2
Asymmetric Routing and Bridging on the ingress VTEP Bridging on the egress VTEP Both source and destination VNIs need to reside on the ingress VTEP
29© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Symmetric IRB
Routing on both ingress and egress VTEPs Layer-3 VNI
Tenant VPN indicator One per tenant VRF
VTEP Router MAC Ingress VTEP routes packets onto the Layer-3 VNI Egress VTEP routes packets to the destination Layer-2 VNI
30© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Symmetric IRB (Cont’d)
Host 1H-MAC-1H-IP-1VNI-A
VTEP-4Router MAC-4
VTEP-3VTEP-2VTEP-1Router MAC-1
Host 2H-MAC-2H-IP-2VNI-B
VNI A
L3 VNI
VNI B
L3 VNI
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1D-IP: H-IP-2
Ingress VTEP routes packets from source VNI to L3 VNI. D-MAC in the inner header is the egress VTEP router MAC
1S-MAC: Router-MAC-
1D-MAC: Router-MAC-
4S-IP: H-IP-1D-IP: H-IP-2
S-IP: VTEP-1D-IP: VTEP-4VNI: L3 VNI
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1D-IP: H-IP-2
Egress VTEP routes packets from L3 VNI to the destination VNI/VLAN2
31© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Symmetric IRB - Benefits
VTEPs don’t need to learn and maintain MAC address information for the remote hosts attached to egress VNIs for which it doesn’t have local hosts
Better utilization of the MAC address table and ARP adjacencies on a VTEP.
Routing and bridging is more scalable than with asymmetric IRB.
Cisco NX-OS implements symmetric IRB to achieve optimal learning and scaling.
32© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN Multi-Tenancy and VNI Types (Cont’d)
L3
Layer-3 VNI (VRF VNI)
Layer-2 VNI (Network
VNI)
Layer-2 VNI (Network VNI)
VTEP
VTEP
VTEP
VTEP
vlan 200 vn-segment 20000vlan 201 vn-segment 20100
vlan 3900 name l3-vni-vlan-for-tenant-1 vn-segment 39000
interface Vlan3900 description l3-vni-for-tenant-1-routing no shutdown vrf member evpn-tenant-1 ip address 39.0.0.1/16 fabric forwarding mode anycast-gateway
vrf context evpn-tenant-1 vni 39000 rd auto address-family ipv4 unicast route-target import 39000:39000 route-target export 39000:39000 route-target both auto evpn
interface Vlan200 no shutdown vrf member evpn-tenant-1 ip address 20.0.0.1/24 fabric forwarding mode anycast-gateway
interface Vlan201 no shutdown vrf member evpn-tenant-1 ip address 20.1.0.1/24 fabric forwarding mode anycast-gateway
33© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN Multi-Tenancy and VNI Types
Tenant A (VRF A)
VLAN B
Layer-2 VNI B’
VLAN C
Layer-2 VNI C’
SVIA
SVI B
VLAN A
Layer-3 VNI A’
SVIX
1 Layer-3 VNI per Tenant (VRF) for routing
VNI X’ is used for routed packets
1 Layer-2 VNI per Layer-2 segment
Multiple Layer-2 VNIs per tenant VNI A’ and B’ are used for bridged
packets
34© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Multi-tenant Packet Forwarding in Symmetric IRB
L3
VTEP-1 VTEP VTEP VTEP-2
Host 1H-MAC-1H-IP-1VNI-AL3-VNI-AVRF-A
Host 2H-MAC-2H-IP-2VNI-BL3-VNI-AVRF-A
IP Transport Network
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1D-IP: H-IP-2
S-MAC: Router-MAC-1
D-MAC: Router-MAC-2
S-IP: H-IP-1D-IP: H-IP-2
S-IP: VTEP-1D-IP: VTEP-2
VNI: L3-VNI-A
Use VTEP addresses in the outer header to route encapsulated packets to the egress VTEP
S-MAC: Router-MAC-1D-MAC: Router-MAC-4
S-IP: H-IP-1D-IP: H-IP-2
S-IP: VTEP-1D-IP: VTEP-2
VNI: L3 –VNI-A Use L3-VNI to identify the tenant VRF
Tenant AVRF-AL3-VNI-A
H-IP-2
Tenant BVRF-BL3-VNI-B
Tenant CVRF-CL3-VNI-C
Tenant AVRF-AL3-VNI-A
H-IP-2
S-MAC: H-MAC-1
D-MAC: H-MAC-2
S-IP: H-IP-1
D-IP: H-IP-2
35© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Symmetric IRB has optimal utilization of ARP and MAC tables on a VTEP. Symmetric IRB scales better for end hosts. Symmetric IRB scales better in terms of the total number of VNIs a VXLAN overlay
network can support.
Multi-vendor interoperability: Some vendors implemented Asymmetric IRB. It’s been agreed upon among multiple vendors that Symmetric IRB is the ultimate
solution. Cisco implemented Symmetric IRB. Cisco will introduce backward compatibility with asymmetric IRB by adding the
support for it.
Symmetric IRB vs Asymmetric IRB
36© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Remote host route learning through MP-BGP
Optimal VXLAN Routing with Symmetric IRB and Anycast Gateway
LeafVTEP-4
VTEP-3
VTEP-2
VTEP-1
VTEP-5
Spine
SVI 100
SVI 200
SVI 100
SVI 200
Host-based fabric routing and bridging with optimal and flexible VXLAN VNI placement Every VTEP is an anycast
gateway for its VXLAN subnets. Anycast gateway VTEPs share: The same virtual Gateway
IP The same virtual MAC
address
Distributed inter-vxlan host-based
routing on local VTEP
Host IP Port
IP-A Eth1/1
IP-B VTEP-4
Host IP VTEP
IP-A VTEP-2
IP-B Eth1/1
VLAN 100SVI VLAN 100VNI 5100Host IP-A
VLAN 200SVI VLAN 200VNI 5200Host IP-B
SVI 100
SVI 200
SVI300
37© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN Fabric External Routing
27
VTEP-1
VTEP-2
VTEP-3
VTEP-4
2
3
Host-1 (VLAN 100) External Network or Internet
VTEP-6
VTEP-5
Routing Protoco
l of Choice
IP Routing
Global Default VRF
Or User Space VRFs
VXLAN Overlay
EVPN VRF/VRFs Space
Border Leaf
RR RR
VXLAN Overlay
EVPN MP-BGP
1
Host 1MAC_1/ IP_1
38© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN Fabric External Routing (Cont’d)
Tenant VRF or Default VRF
VRF OSPF Process
Overlay
EVPN VRF A
Overlay
EVPN VRF B
Overlay
EVPN VRF C
VRFA
For Layer 3 interfaces, use one per overlay VRF instance. The routing protocol neighbor is in the EVPN VRF address family.Layer 3 interfaces on the external devices can be in either tenant VRF instances or the global default VRF instance.
External Router
VRFB
VRFC Interface-Type Options:
Physical Routed Ports Subinterfaces VLAN SVIs over Trunk Ports
Border Leaf
39© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN EVPN Features
40© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
ARP Suppression in MP-BGP EVPN
Host-1 in VLAN 10 sends an ARP request for Host-2’s IP-2 address.
1
Host 1MAC1IP 1
VLAN 10VXLAN 5000
Host 2MAC2IP 2
VLAN 10VXLAN 5000
S2 S3 S4
VTEP1
VTEP2
VTEP3
VTEP4
VTEP-1 intercepts the ARP request and checks in its ARP suppression cache. It finds a match for IP-2 in VLAN 10 in its ARP suppression cache.* 2
IP Address
MAC Address
VLAN Physical Interface Index (ifindex)
Flags
IP-1 MAC-1 10 E1/1 Local
IP-2 MAC-2 10 Null Remote
IP-3 MAC-3 10 Null Remote
VTEP-1 sends an ARP response back to Host-1 with MAC-2.*
3
Host-1 learns the IP-2 and MAC-2 mapping.
4
* If VTEP-1 doesn’t have a match for IP-2 in its ARP suppression cache table, it will flood the ARP request to all other VTEPs in this VNI
ARP suppression reduces network flooding due to host learning
41© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
interface nve1
no shutdown
source-interface loopback0
host-reachability protocol bgp
member vni 20000
suppress-arp
mcast-group 239.1.1.1
member vni 21000
suppress-arp
mcast-group 239.1.1.2
member vni 39000 associate-vrf
member vni 39010 associate-vrf
ARP Suppression in MP-BGP EVPN (Cont’d) ARP Suppression can be enabled on a per-VNI basis under the interface nve1
configuration.
S2 S3 S4
VTEP1
VTEP2
VTEP3
VTEP4
n9396-vtep-1.sakommu-lab.com# sh ip arp suppression topo-info
ARP L2RIB Topology informationTopo-id ARP-suppression mode100 L2 ARP Suppression200 L2/L3 ARP Suppression201 L2/L3 ARP Suppression
42© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Head-end Replication
Leaf
Spine
VTEP1
VTEP2
VTEP3
VTEP4
Host-1 sends BUM traffic into the VXLAN VNI
1
VTEP-1 receives the overlay BUM traffic, encapsulates the packets into unicast VXLAN packets, sends one copy to each remote VTEP peer in the same VXLAN VNI
2
Multicast-Free
Underlay
Head-end Replication (aka. Ingress replication):
Eliminate the need for underlay multicast to transport overlay BUM traffic
43© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Local Scoping of VLANs –ToR Local
---- N-Way ----
VTEP-1 VTEP-2 VTEP-3 VTEP-n
Host-1 (VLAN 10) Host-2 (VLAN 60)
5000
16 million possible VNIs global scope
Host -1MAC_1
/10.1.1.1
VLANS are Locally Scoped at Top of Rack/ Gateway
VNI 5000 maps to VLAN 10
Possible VLAN IDs 1-4K
Host -2MAC_2/10.1.
1.4
VLANS are Locally Scoped at Top of Rack/ Gateway
VNI 5000 maps to VLAN 60
Possible VLAN IDs 1-4K
44© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Local Scoping of VLANs – Port Local
---- N-Way ----
VTEP-1 VTEP-2 VTEP-3 VTEP-n
Host-1 (VLAN 10) Host-2 (VLAN 60)
16 million possible VNIs global scope
Host -1MAC_1
/10.1.1.1
VNI 10000 maps to (E1/1, VLAN 10)
VLANS are Locally Scoped VLAN to VNI mapping is per-
port significantPossible VLAN IDs 1-4K
(Eth1/1, Vlan10) => VNI 10000 (Eth1/2, Vlan10) => VNI 10001 (Eth1/2, Vlan11) => VNI 10000
Host -2MAC_2/10.1.
1.3
VNI 10000 maps to (E1/2, VLAN 10)
VLANS are Locally Scoped VLAN to VNI mapping is per-
port significantPossible VLAN IDs 1-4K
45© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN Control Plane Advantages
A multi-tenant fabric solution with host-based forwarding
Industry standard protocol for multi-vendor interoperability
Build-in multi-tenancy support Leverage MP-BGP to deliver VXLAN with L3VPN characteristics
Truly scalable with protocol-driven learning Host MAC/IP address advertisement through EVPN MP-BGP
Fast convergence upon host movements or network failures MP-BGP protocol driven re-learning and convergence
Upon host movement, the new VTEP will send out a BGP update to advertise the new
location of the host
46© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN Control Plane Advantages (Cont’d)
Optimal traffic forwarding supporting host mobility Anycast IP gateway for optimal forwarding for host generated traffic No need for hair-pinning to to reach the IP gateway
ARP suppression Minimize ARP flooding in overlay
Head-end Replication with dynamically learned remote-VTEP list Head-end replication enables multicast-free underlay network Dynamically learned remote-VTEP list minimizes the operational overhead of head-end
replication
VTEP peer authentication via MP-BGP authentication Added security to prevent rogue VTEPs or VTEP spoofing
A multi-tenant fabric solution with host-based forwarding
47© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Ingress Replication for Control Plane EVPN
48© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VxLAN Overview
VxLAN provides a way to extend Layer2 extension across Layer 3 infrastructure using MAC-in-UDP encapsulation and tunneling.
VxLAN uses a 24-bit Virtual Network ID (VNID), allowing a maximum of 16 million VxLAN segments to coexist in the same administrative domain.
VxLAN packets are IP routed through the underlying network based on its its Layer 3 header.
49© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN Peer and Host Learning Options
Host Learning
Data-Plane Control-Plane
Core
Multicast
UnicastVlan 2 vn-segment 4098Interface nve 1 host-reachability protocol bgp member vni 4098 ingress-replication protocol bgp
Vlan 2 vn-segment 4098Interface nve 1member vni 4098 ingress-replication protocol static
Vlan 2 vn-segment 10000Interface nve 1 host-reachability protocol bgp member vni 4098 mcast-group 225.1.1.1
Vlan 2 vn-segment 4098Interface nve 1member vni 10000 mcast-group 225.1.1.1
Flood and LearnPeer Learning: DP
EVPN-MulticastPeer Learning: BGP-RnH
Static Ingress-ReplicationPeer Learning: CLI
EVPN Ingress-ReplicationPeer Learning: BGP-IMET
50© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Flooding of Broadcast/Unknown-unicast/Multicast (BUM) packets across underlying core network: Option 1: Use multicast IP core
Each VNI is mapped to a multicast group. For packets coming on access side interfaces miss MAC lookup, they are encapsulated with VNI group address as DIP and sent out along the multicast tree to core
Option 2: Use Ingress Replication (IR) Some customers would want to avoid using multicast in their core Remote VTEPs are automatically learnt through Inclusive Multicast Ethernet Tag (IMET)
routes. When missing MAC lookup, a packet is replicated to all remote VTEPs in the VNI, each is
encapsulated with one VTEP IP as unicast Destination IP Support multiple VTEPs per VNI and a VTEP in multiple VNIs
VXLAN Flooding with BGP-EVPN
51© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VTEP-2
VTEP-1
VTEP-3
Host-1
Host-3
Ethernet 1/1: 102.1.1.1Loopback0 : 202.1.1.1VxLAN L2 Gateway
IP Network
Ethernet 1/1: 100.1.1.1Loopback0 : 201.1.1.1VxLAN L2 Gateway
E1/1
E1/1
Router-2
Router-3
Hoist-4
Router-1
feature nv overlay feature vn-segment-vlan-based
vlan 2 vn-segment 4098vlan 3 vn-segment 4099
interface nve1 no shutdown source-interface loopback0 member vni 4098 ingress-replication protocol bgp member vni 4099 ingress-replication protocol bgp
Ethernet 1/1: 103.1.1.1Loopback0 : 203.1.1.1VxLAN L2 Gateway
E1/1
Host-2
Hoist-5
VLAN 2 VLAN 3 VLAN 2
VLAN 2
VLAN 3
feature nv overlay feature vn-segment-vlan-based
vlan 2 vn-segment 4098
interface nve1 no shutdown source-interface loopback0 member vni 4098 ingress-replication protocol bgp
Topology and configuration
52© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN EVPN Configuration
53© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN with MP-iBGP EVPN Configuration
LeafVTEPVTEPVTEPVTEP VTEP VTEP
SpineRRRR
Spine nodes are iBGP route reflector
54© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
feature nv overlay
feature vn-segment-vlan-based
feature bgp
nv overlay evpn
Enable VXLAN and MP-BGP EVPN Control Plane
Enable VXLAN
Enable VLAN-based VXLAN (the currently only mode)
Enable OSPF if it’s chosen to be the underlay IGP routing protocol
Enable VLAN SVI interfaces if the VTEP needs to be IP gateway and route for the VXLAN VLAN IP subnet.
Enable EVPN control plane for VXLAN
feature ospf
feature pim
feature interface-vlan
Other features may need to be enabled
Enable BGP
Enable IP PIM multicast routing in the underlay network
Initial configuration – Per Switch
55© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
vrf context evpn-tenant-1 vni 39000 rd auto address-family ipv4 unicast route-target import 39000:39000 route-target export 39000:39000 route-target both auto evpn
Create VXLAN tenant VRFCreate a VXLAN Tenant VRF
Specify the Layer-3 VNI for VXLAN routing within the tenant VRF
Define VRF Route Target and import/export policies in address-family ipv4 unicast
Define VRF RD (route distinguisher)
vrf context evpn-tenant-2 vni 39010 rd auto address-family ipv4 unicast route-target import 39010:39010 route-target export 39010:39010 route-target both auto evpn
Example to create a 2nd tenant VRF following the above steps
EVPN Tenant VRF
56© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
vlan 3900 name l3-vni-vlan-for-tenant-1 vn-segment 39000
interface Vlan3900 description l3-vni-for-tenant-1-routing no shutdown vrf member evpn-tenant-1 vrf context evpn-tenant-1 vni 39000 rd auto address-family ipv4 unicast route-target import 39000:39000 route-target export 39000:39000 route-target both auto evpn
Configure Layer-3 VNI per EVPN Tenant VRF Routing Instant
Create the VLAN for the Layer-3 VNI. One Layer-3 VNI per tenant VRF routing instance
Create the SVI interface for the Layer-3 VNIPut this SVI interface into the tenant VRF context
Associate the Layer-3 VNI with the tenant VRF routing instance.
Layer-3 VNI Per Tenant for EVPN Routing
57© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
vlan 3901 name l3-vni-vlan-for-tenant-2 vn-segment 39010
interface Vlan3901 description l3-vni-for-tenant-2-routing no shutdown vrf member evpn-tenant-2
vrf context evpn-tenant-2 vni 39010 rd auto address-family ipv4 unicast route-target import 39010:39010 route-target export 39010:39010 route-target both auto evpn
Configure Layer-3 VNI per EVPN Tenant VRF Routing Instant
Define Layer-3 VNI for a 2nd tenant following the same steps in the previous slide
EVPN Layer-3 VNI Per Tenant for Routing Instance
58© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
vlan 200 vn-segment 20000vlan 210 vn-segment 21000
Map VLANs to VXLAN VNIs and Configure their MP-BGP EVPN Parameters
Map VLAN to VXLAN VNI
evpn vni 20000 l2 rd auto route-target import auto route-target export auto vni 21000 l2 rd auto route-target import auto route-target export auto
Under EVPN configuration, define RD and RT import/export policies for each Layer-2 VNIs
EVPN Layer-2 Network VXLAN VNI
59© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
interface Vlan200 no shutdown vrf member evpn-tenant-1 ip address 20.1.1.1/8 fabric forwarding mode anycast-gateway
interface Vlan210 no shutdown vrf member evpn-tenant-1 ip address 21.1.1.1/8 fabric forwarding mode anycast-gateway
Create SVI interface for Layer-2 VNIs for VXLAN routing
Create SVI interface for a Layer-2 VNI.Associate it with the tenant VRF.
Enable distributed anycast gateway for this VLAN/VNI
All VTEPs for this VLAN/VNI should have the same SVI interface IP address as the distributed IP gateway.
EVPN Layer-2 Network VXLAN VLAN SVI Interface
60© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
fabric forwarding anycast-gateway-mac 0002.0002.0002
interface Vlan210 no shutdown vrf member evpn-tenant-2 ip address 21.1.1.1/8 fabric forwarding mode anycast-gateway
Configure virtual IP addressAll VTEPs for this VLAN should have the same virtual IP address
Configure distributed gateway virtual MAC addressOne virtual MAC per VTEPAll VTEPs should have the same virtual MAC address
Enable distributed gateway for this VLAN
EVPN Distributed Gateway
61© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
interface nve1 no shutdown source-interface loopback0 host-reachability protocol bgp member vni 20000 suppress-arp mcast-group 239.1.1.1 member vni 21000 suppress-arp mcast-group 239.1.1.2 member vni 39000 associate-vrf member vni 39010 associate-vrf
interface loopback 0 ip address 10.1.1.11/32 ip ospf network point-to-point ip router ospf 1 area 0.0.0.0 ip pim sparse-mode
Configure VXLAN tunnel interface nve1
Specify loopback0 as the source interface
Define BGP as the mechanism for host reachability advertisement
Add Layer-3 VNIs, one per tenant VRF
Associate tenant VNIs to the tunnel interface nve1Define the mcast group on a per-VNI basisEnable arp suppression on a per-VNI basis
The loopback interface to source VXLAN tunnels
VXLAN Tunnel Interface Configuration
62© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
router bgp 100 router-id 10.1.1.11 log-neighbor-changes address-family ipv4 unicast address-family l2vpn evpn neighbor 10.1.1.1 remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended neighbor 10.1.1.2 remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended
vrf evpn-tenant-1 address-family ipv4 unicast advertise l2vpn evpn vrf evpn-tenant-2 address-family ipv4 unicast advertise l2vpn evpn
Address-family ipv4 unicast for prefix-based routing
Define MP-BGP neighbors. Under each neighbor define address-family ipv4 unicast and l2vpn evpn
Under address-family ipv4 unicast of each tenant VRF instance, enable advertising EVPN routes
Send extended community in l2vpn evpn address-family to distribute EVPN route attributes
Address-family l2vpn evpn for evpn host routes
MP-BGP Configuration on VTEP
63© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
router bgp 100 router-id 10.1.1.1 log-neighbor-changes address-family ipv4 unicast address-family l2vpn evpn retain route-target all template peer vtep-peer remote-as 100 update-source loopback0 address-family ipv4 unicast send-community both route-reflector-client address-family l2vpn evpn send-community both route-reflector-client neighbor 10.1.1.11 inherit peer vtep-peer neighbor 10.1.1.12 inherit peer vtep-peer neighbor 10.1.1.13 inherit peer vtep-peer neighbor 10.1.1.14 inherit peer vtep-peer
Address-family ipv4 unicast for prefix-based routing
iBGP RR client peer template
Send both standard and extended community in address-family l2vpn evpn
Send both standard and extended community in address-family ipv4 unicast
Address-family l2vpn evpn for EVPN vxlan host routesRetain route-targets attributes
MP-BGP Configuration on iBGP Route Reflector
64© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN Fabric with MP-eBGP EVPN
VTEPVTEPVTEPVTEP VTEP VTEP
Spine
AS 6500
1
AS 6500
2
AS 6500
3
AS 6500
4
AS 6500
5
AS 6500
6
AS 65000
[BGP configuration on a spine switch as in Figure
16 design]
route-map permit-all permit 10
set ip next-hop unchanged
router bgp 65000
router-id 10.1.1.1
address-family ipv4 unicast
redistribute direct route-map permitall
address-family l2vpn evpn
nexthop route-map permit-all
retain route-target all
neighbor 192.167.11.2 remote-as 65001
address-family ipv4 unicast
address-family l2vpn evpn
send-community extended
route-map permit-all out
neighbor 192.168.12.2 remote-as 65002
address-family ipv4 unicast
address-family l2vpn evpn
send-community extended
route-map permit-all out
Set next-hop policy to not change the next-hop attributes.
Retain routes with all route targets when advertising the EVPN BGP routes to eBGP peers.
Set outbound policy to advertise all routes to this eBGP neighbor.
65© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
VXLAN Fabric MP-eBGP EVPN (Cont’d)
[BGP configuration on a spine switch as in Figure 16 design]
route-map permit-all permit 10
set ip next-hop unchanged
router bgp 65000
router-id 10.1.1.1
address-family ipv4 unicast
redistribute direct route-map permitall
address-family l2vpn evpn
nexthop route-map permit-all
retain route-target all
neighbor 192.167.11.2 remote-as 65001
address-family ipv4 unicast
address-family l2vpn evpn
send-community extended
route-map permit-all out
neighbor 192.168.12.2 remote-as 65002
address-family ipv4 unicast
address-family l2vpn evpn
send-community extended
route-map permit-all out
Set next-hop policy to not change the next-hop attributes.
Retain routes with all route targets when advertising the EVPN BGP routes to eBGP peers.
Set outbound policy to advertise all routes to this eBGP neighbour.
66© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN Fabric External Routing
LeafVTEPVTEPVTEPVTEP VTEP VTEP
SpineRRRR
Global Default VRF
Or User Space VRFs
Border Leaf
VXLAN Overlay
EVPN MP-BGP
IP Routing
Routing Protocol
of Choice
VXLAN Overlay
EVPN VRF/VRFs Space
67© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN External Routing with BGP
IP Routing in the Default VRF
Instance
VTEP
VTEP
VTEP
VTEP
SpineRRRR
VXLAN Overlay
EVPN VRF Instance Space VTEP
Border Leaf interface Ethernet2/9.10 mtu 9216 encapsulation dot1q 10 vrf member evpn-tenant-1 ip address 30.10.1.1/30
interface Ethernet1/50.10 mtu 9216 encapsulation dot1q 10 ip address 30.10.1.2/30
Router bgp 100 vrf evpn-tenant-1 address-family ipv4 unicast network 20.0.0.0/24 neighbor 30.10.1.2 remote-as 200 address-family ipv4 unicast prefix-list outbound-no-hosts out
router bgp 200 address-family ipv4 unicast network 100.0.0.0/24 network 100.0.1.0/24 neighbor 30.10.1.1 remote-as 100 address-family ipv4 unicast
Sample Configuration
68© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
router bgp 100 router-id 10.1.1.16 log-neighbor-changes address-family ipv4 unicast address-family l2vpn evpn neighbor 10.1.1.1 remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended neighbor 10.1.1.2 remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended vrf evpn-tenant-1 address-family ipv4 unicast network 20.0.0.0/24 neighbor 30.10.1.2 remote-as 200 address-family ipv4 unicast prefix-list outbound-no-hosts out
ip prefix-list outbound-no-hosts seq 5 deny 0.0.0.0/0 eq 32 ip prefix-list outbound-no-hosts seq 10 permit 0.0.0.0/0 le 32
On the VXLAN Border Leaf
The eBGP neighbor is on the outside.It’s in address-family ipv4 unicast of the tenant VRF routing instance.
For better scalability, apply prefix-list to filter out /32 IP host routes. Advertise prefix routes only to the external eBGP neighbor.
EVPN VXLAN External Routing with BGPSample Configuration – On the Border Leaf
69© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN External Routing with BGPn9396-vtep-1# sh ip bgp vrf evpn-tenant-1 100.0.0.0 BGP routing table information for VRF evpn-tenant-1, address family IPv4 UnicastBGP routing table entry for 100.0.0.0/24, version 70Paths: (1 available, best #1)Flags: (0x08041a) on xmit-list, is in urib, is best urib route vpn: version 75, (0x100002) on xmit-list
Advertised path-id 1, VPN AF advertised path-id 1 Path type: internal, path is valid, is best path, no labeled nexthop Imported from unknown dest AS-Path: NONE, path sourced internal to AS 10.1.1.16 (metric 3) from 10.1.1.1 (10.1.1.1) Origin IGP, MED not set, localpref 100, weight 0 Received label 39000 Extcommunity: RT:100:39000 ENCAP:8 Router MAC:6412.2574.6ae7 Originator: 10.1.1.16 Cluster list: 10.1.1.1
VRF advertise information: Path-id 1 not advertised to any peer
VPN AF advertise information: Path-id 1 not advertised to any peern9396-vtep-1# n9396-vtep-1# sh ip route vrf evpn-tenant-1 100.0.0.0/24IP Route Table for VRF "evpn-tenant-1"'*' denotes best ucast next-hop'**' denotes best mcast next-hop'[x/y]' denotes [preference/metric]'%<string>' in via output denotes VRF <string>
100.0.0.0/24, ubest/mbest: 1/0 *via 10.1.1.16%default, [200/0], 01:01:14, bgp-100, internal, tag 100 (evpn)segid: 0x9858 tunnelid: 0xa010110 encap: 1 n9396-vtep-1#
This is the external route.
The tenant is VRF L3 VNI.
The next hop is the VTEP address of the border leaf.
10.1.1.16 is the BGP router ID of the border leaf. 10.1.1.1 is the spine route reflector.
This is the iBGP route. The next hop is the VTEP address of the border leaf.
70© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN External Routing with OSPFSample Configuration
IP Routing in the Default VRF
Instance
VTEP
VTEP
VTEP
VTEP
SpineRRRR
VXLAN Overlay
EVPN VRF and VRF Instance Space VTEP
Border Leaf
interface Ethernet2/9.10 mtu 9216 encapsulation dot1q 10 vrf member evpn-tenant-1 ip address 30.10.1.1/30 ip router ospf 1 area 0.0.0.0
interface Ethernet1/50.10 mtu 9216 encapsulation dot1q 10 ip address 30.10.1.2/30 ip router ospf 1 area 0.0.0.0
route-map permit-bgp-ospf permit 10route-map permit-ospf-bgp permit 10router ospf 1 router-id 10.1.1.16 vrf evpn-tenant-1 redistribute bgp 100 route-map permit-bgp-ospfrouter bgp 100 router-id 10.1.1.16 log-neighbor-changes address-family ipv4 unicast address-family l2vpn evpn retain route-target all vrf evpn-tenant-1 address-family ipv4 unicast advertise l2vpn evpn redistribute ospf 1 route-map permit-ospf-bgp
71© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN External Routing with OSPF
ip prefix-list bgp-ospf-no-hosts seq 5 permit 0.0.0.0/0 eq 32 route-map permit-bgp-ospf deny 5 match ip address prefix-list bgp-ospf-no-hosts route-map permit-bgp-ospf permit 10route-map permit-ospf-bgp permit 10
router ospf 1 router-id 10.1.1.16 vrf evpn-tenant-1 redistribute bgp 100 route-map permit-bgp-ospf
router bgp 100 router-id 10.1.1.16 log-neighbor-changes address-family ipv4 unicast address-family l2vpn evpn retain route-target all neighbor 10.1.1.1 remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended neighbor 10.1.1.2 remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended vrf evpn-tenant-1 address-family ipv4 unicast advertise l2vpn evpn redistribute ospf 1 route-map permit-ospf-bgp
Redistribute BGP routes to OSPF. Filter out /32 IP host routes.
A BGP router will modify route targets in l2vpn evpn routes when it is an autonomous system boundary router. The original route target must be retained.
Redistribute OSPF to BGP. Advertise the redistributed routes to L2VPN EVPN.
Sample Configuration - The relevant configuration on the border leaf
72© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
EVPN VXLAN External Routing with OSPF
n9396-vtep-1# sh vrf evpn-tenant-1 detail VRF-Name: evpn-tenant-1, VRF-ID: 3, State: Up VPNID: unknown RD: 10.1.1.11:3 VNI: 39000 Max Routes: 0 Mid-Threshold: 0 Table-ID: 0x80000003, AF: IPv6, Fwd-ID: 0x80000003, State: Up Table-ID: 0x00000003, AF: IPv4, Fwd-ID: 0x00000003, State: Up
n9396-vtep-1# sh bgp l2vpn evpn rd 10.1.1.11:3 100.0.0.0BGP routing table information for VRF default, address family L2VPN EVPNRoute Distinguisher: 10.1.1.11:3 (L3VNI 39000)BGP routing table entry for [5]:[0]:[0]:[24]:[100.0.0.0]:[0.0.0.0]/224, version 396Paths: (1 available, best #1)Flags: (0x00001a) on xmit-list, is in l2rib/evpn
Advertised path-id 1 Path type: internal, path is valid, is best path, no labeled nexthop Imported from 10.1.1.16:3:[5]:[0]:[0]:[24]:[100.0.0.0]:[0.0.0.0]/120 AS-Path: NONE, path sourced internal to AS 10.1.1.16 (metric 3) from 10.1.1.1 (10.1.1.1) Origin IGP, MED not set, localpref 100, weight 0 Received label 39000 Extcommunity: RT:100:39000 ENCAP:8 Router MAC:6412.2574.6ae7 Originator: 10.1.1.16 Cluster list: 10.1.1.1
Path-id 1 not advertised to any peer
n9396-vtep-1#
The external route learned through MP-BGP EVPN is imported into the tenant VRF.
This is the Layer 3 VNI of the tenant VRF routing instance.
The next hop is the VTEP address of the border leaf.
The internal VTEPs learn the external routes through MP-BGP EVPN
73© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Scalability Limits
74© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Unicast Routing Verified Scalability Limits
Feature 9500 Series Verified Limit 9300 Series Verified Limit
BFD sessions (echo mode) 512 (IPv4 only)512 (IPv6 only)256 (IPv4) + 256 (IPv6)
256 (IPv4 only)256 (IPv6 only)128 (IPv4) + 128 (IPv6)
BGP neighbors 2000 (IPv4 only)2000 (IPv6 only)1000 (IPv4) + 1000 (IPv6)
512 (IPv4 only)512 (IPv6 only)256 (IPv4) + 256 (IPv6)
EIGRP routes 20,000 20,000
EIGRP neighbors 360 (IPv4 only)360 (IPv6 only)180 (IPv4) + 180 (IPv6)
128 (IPv4 only)128 (IPv6 only)64 (IPv4) + 64 (IPv6)
HSRP groups per interface or I/O module 490 250
IPv4 ARP 45,000 (default system routing mode)60,000 (max-host routing mode)
45,000
Unicast Routing
75© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Unicast Routing Verified Scalability Limits (Cont’d)
Feature 9500 Series Verified Limit 9300 Series Verified Limit
IPv4 host routes 208,000 (default system routing mode)60,000 (max-host routing mode)
208,000 (default system routing mode)16,000 (ALPM routing mode)
IPv6 host routes 40,000 (default system routing mode)30,000 (max-host routing mode)
40,000 (default system routing mode)8,000 (ALPM routing mode)
IPv6 ND 40,000 40,000
IPv4 unicast routes (LPM) 128,000 (default system routing mode)16,000 (max-host routing mode)128,000 with no IPv6 routes (64-bit ALPM routing mode)
12,000 (default system routing mode)128,000 (ALPM routing mode)
IPv6 unicast routes (LPM) 20,000 (default system routing mode)4000 (max-host routing mode)80,000 with no IPv4 routes (64-bit ALPM routing mode)
7000 (6000 routes < /64, 1000 routes > /64) (default system routing mode)20,000 (ALPM routing mode)
IPv4 and IPv6 unicast routes (LPM) in 64-bit ALPM routing mode
x IPv6 routes and y IPv4 routes, where 2x +y <= 128,000
Not applicable
Unicast Routing
76© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Unicast Routing Verified Scalability Limits (Cont’d)
Feature 9500 Series Verified Limit 9300 Series Verified Limit
MAC addresses 90,000 90,000
OSPFv2 neighbors 1000 256
OSPFv3 neighbors 300 256
VRFs 1000 1000
VRRP groups per interface or I/O module 250 250
Unicast Routing
77© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Nexus 2000 Series Fabric Extenders (FEX) Verified Scalability Limits
Feature 9500 Series Verified Limit
9300 Series Verified Limit
Fabric Extenders and Fabric Extender server interfaces
Not applicable 16 and 768
VLANs per Fabric Extender Not applicable 2000 (across all Fabric Extenders)
VLANs per Fabric Extender server interface
Not applicable 75
Port channels Not applicable 500
78© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Interfaces Verified Scalability Limits
Feature 9500 Series Verified Limit
9300 Series Verified Limit
Generic routing encapsulation (GRE) tunnels
8 8
Port channel links 32 8
SVIs 490 250
vPCs 275 100 (280 with Fabric Extenders)
79© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Layer 2 Switching Verified Scalability Limits
Feature 9500 Series Verified Limit
9300 Series Verified Limit
MST instances 64 64
MST virtual ports 85,000 48,000
RPVST virtual ports 22,000 12,000
VLANs 4000 3900
VLANs in RPVST mode 500 500
80© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Multicast Routing Verified Scalability Limits
Feature 9500 Series Verified Limit
9300 Series Verified Limit
IPv4 multicast routes 32,000 8000
Outgoing interfaces (OIFs) 40 (see CSCum58876) 40 (see CSCum58876)
81© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Layer 2 Switching Verified Scalability Limits
Feature 9500 Series Verified Limit
9300 Series Verified Limit
MST instances 64 64
MST virtual ports 85,000 48,000
RPVST virtual ports 22,000 12,000
VLANs 4000 3900
VLANs in RPVST mode 500 500
82© 2015-2016 OneCloud and/or its affiliates. All rights reserved.
Thank You