E-VPN on UW System Network

Michael Hare

Purpose of presentation• A high level introduction to E-VPN

• A simple lab demonstration

• For our documentation, see https://kb.wisc.edu/uwsysnet/internal/page.php?id=56133

• For a deeper dive, see http://www.juniper.net/us/en/training/jnbooks/day-one/proof-concept-labs/using-ethernet-vpns/

E-VPN• MPLS service for loop free multipoing bridging using BGP as a control

plane• Similar deliverable as VPLS but with some additional features such as

support for active/active CE multihoming

• Supports features such as default gateway sync, which is not discussed in this presentation

E-VPN basic theory of operation• EVI: E-VPN instance, as defined on a PE.

• ES: Ethernet Segment. Ethernet link(s) between customer and provider network. When LAN is multihomed (multiple CE or multiple PE), the set of links is considered a single ES and must be assigned a network-wide unique ESI (Ethernet Segment Identifier). Many multihoming examples show a single CE directly connected to two PEs. However, an ES is also multihomed if a CE has a path to more than one PE through the CE's switched network. In latter case, single-active must be used (PEs for this ES cannot do active/active load balancing).

• ETI: Ethernet Tag Identifier, describes a unique broadcast domain in an EVI. For our purposes, ETI is a vlan ID. Note that an ETI inside an EVI may contain multiple unique ESI.

• DF: Designated Forwarder. Only for a multihomed ESI, DF replaces spanning tree and is responsible for forwarding multi-destination traffic (Broadcast, unknown unicast, and multicast traffic, or BUM). Non-DFs, or Backup Forwarders, drop BUM traffic on ingress.

Learning MAC addresses• When a PE router detects a new MAC address on its EVI access interface, it adds the address to its appropriate local Layer 2

forwarding table, or MAC-VRF. The PE then transmits a MAC Advertisement route using MP-BGP to all remote PEs.

• The inclusion of the ESI in the MAC Advertisement route is critical for implementing aliasing, or load balancing. Multi-homed PEs advertise their connectivity to a common ESI by transmitting Auto-Discovery routes to all remote PEs. When a given remote PE subsequently learns of a MAC address from that ESI, it knows that the destination is reachable via the set of multi-homed PEs. The PE can then load balance traffic to the multiple PEs connected to the common ES.

• 2:143.235.32.38:700::200::00:24:97:32:a7:42/304 (1 entry, 0 announced) *BGP Preference: 170/-101 Route Distinguisher: 143.235.32.38:700 Task: BGP_65010.143.235.32.112+179 Communities: target:65010:700 Import Accepted Route Label: 306720 ESI: 00:00:07:00:00:00:00:00:00:00 Secondary Tables: EVPN-2.evpn.0

Looking at the bridging/forwarding tables• From the bridging perspective

m7h@r-mx104-lab-ac-re0> show bridge mac-table instance EVPN-2 ……MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC)

Routing instance : EVPN-2 Bridging domain : V200, VLAN : 200 MAC MAC Logical NH RTR address flags interface Index ID 00:11:20:3e:3e:81 DC 1048594 1048594 00:11:20:3e:3e:c2 DC 1048594 1048594 00:24:97:32:a7:42 DC 1048577 1048577 ……

Looking at the bridging/forwarding tables (2)• From the E-VPN database perspective

m7h@r-mx104-lab-ac-re0> show evpn database instance EVPN-2 extensive Instance: EVPN-2

VLAN ID: 200, MAC address: 00:11:20:3e:3e:81 Source: 143.235.32.113, Rank: 1, Status: Active Timestamp: Sep 11 10:18:23 (0x55f2f0bf) State: <Local-Adv-Allowed Local-Adv-Done>

VLAN ID: 200, MAC address: 00:11:20:3e:3e:c2 Source: 143.235.32.113, Rank: 1, Status: Active Timestamp: Sep 11 15:29:13 (0x55f33999) State: <Local-Adv-Allowed Local-Adv-Done>

VLAN ID: 200, MAC address: 00:24:97:32:a7:42 Source: 00:00:07:00:00:00:00:00:00:00, Rank: 1, Status: Active Remote origin: 143.235.32.38 Timestamp: Sep 11 15:29:13 (0x55f33999) State: <Local-Adv-Allowed Local-Adv-Done>

Routing tables• bgp.evpn.0: contains all EVPN related routes carried in BGP. You’ll see

this on P and PE. To decode, see https://tools.ietf.org/html/rfc7432#section-7.

• __default_evpn__.evpn.0: carries multihomed ES routes. You’ll see this on P and PE.

• $EVI.evpn.0 is like bgp.evpn.0 but is routing-instance specific. You’ll only see this in PE’s that are part of the given EVI.

E-VPN NLRI

In E-VPN routes, the first integer is the E-VPN NLRI Route type.

• 1 - Ethernet Auto-Discovery (A-D) route [used for multihoming]• 2 - MAC/IP Advertisement route [endpoint reachability info]• 3 - Inclusive Multicast Ethernet Tag route [for BUM forwarding]• 4 - Ethernet Segment route [DF/multihoming election]

Looking at the bridging/forwarding tables (3)•

m7h@r-mx104-lab-ac-re0> show route table EVPN-2.evpn.0 evpn-mac-address 00:24:97:32:a7:42

EVPN-2.evpn.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both

… 2:143.235.32.38:700::200::00:24:97:32:a7:42/304 *[BGP/170] 00:03:47, localpref 100, from 143.235.32.112 AS path: I, validation-state: unverified > to 143.235.33.217 via ae0.3477 to 143.235.33.144 via xe-0/3/0.3475, Push 299888 …

Determining the DF for an ESI• DF election is performed ESI per EVI. This facilitates load balancing of BUM traffic amongst PEs, a feature known as Service

Carving. See https://tools.ietf.org/html/rfc7432#section-8.5

m7h@r-mx104-lab-ac-re0> show evpn instance EVPN-2 esi 00:00:07:00:00:00:00:00:00:00 extensive Instance: EVPN-2…… Local interface: ae1.200, Status: Up/Blocking Number of remote PEs connected: 1 Remote PE MAC label Aliasing label Mode 143.235.32.38 306720 306720 single-active Designated forwarder: 143.235.32.38 Backup forwarder: 143.235.32.106……

• Here is another way to check e-vpn forwarding state on a CE link

m7h@r-mx104-lab-ac-re0# run show interfaces ae1.200 detail | match EVPN Protocol bridge, MTU: 1522, Generation: 222, Route table: 4, Mesh Group: __all_ces__, EVPN multi-homed status: Blocking

Config snippets: client handoff interfacem7h@r-mx104-lab-ac-re0# show interfaces ae1

apply-groups-except ethernet-standards;description "s-lab-4 Port-channel1 uwplatteville switch 2 simulator";enable;per-unit-scheduler;flexible-vlan-tagging;encapsulation flexible-ethernet-services;esi { <---------- omit this if single homed 00:00:00:00:00:00:00:00:07:00; single-active;}aggregated-ether-options { link-speed 1g; lacp { active; }}unit 200 { description "s-lab-4 Port-channel1 uwplatteville switch 2 simulator"; family bridge { interface-mode trunk; vlan-id-list [ 200 201 202 ]; }}

Config snippets: E-VPN instance• m7h@r-mx104-lab-ac-re0# show routing-instances EVPN-2

instance-type virtual-switch;interface ae1.200;route-distinguisher 143.235.32.106:700;vrf-target target:65010:700;protocols { evpn { extended-vlan-list 200-202; default-gateway advertise; }}bridge-domains { V200 { vlan-id 200; } V201 { vlan-id 201; } V202 { vlan-id 202; }}

Tracking ESI/RD/vrf targets: https://kb.wisc.edu/uwsysnet/internal/page.php?id=55002

Config snippets: BGP• set protocols bgp group iBGP-reflector family evpn signaling

• m7h@r-mx2010-lab-re0# show policy-options policy-statement select-iBGP-reflector-routes-mx2010 ……term bgp-evpn {

from { protocol bgp; rib bgp.evpn.0; } then next policy; }

Keeping it running

• Monitoring• We track BGP NLRI counts and watch syslog.

That’s all, folks

FIN