Highly Available Wide Area Network Design · •Introduction • Cisco IOS and IP Routing • Convergence Techniques • Design and Deployment • Final Wrap Up Agenda

Highly AvailableWide Area Network Design

David Prall, CCIE #6508, Principal Systems Engineer

BRKRST-2042

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to communicate with David after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKRST-2042

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment

• Final Wrap Up

Agenda

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6BRKRST-2042

Your speaker

• David Prall

• Principal Systems Engineer

• World Wide Enterprise Networking

• [email protected]

• CCIE 6508 (R&S/SP/Security)

• Started at Cisco July 10, 2000

• Washington, DC

mailto:[email protected]


QuestionsQuestions?During the session grab a microphone, raise your hand, and ask your question.

Feel free to ask questions.

Use Cisco Spark to ask questions I can respond to afterwards, and communicate with me after the session





How



Goals

• Efficiently utilize available bandwidth

• Dynamically respond to all types of disruptions

• Leverage most effective design techniques that meet the design requirements

• Review today’s technology

8BRKRST-2042


Where Can Outages Occur?

• How does outage manifest?

• How quickly can network detect?

• How long is bidirectional reconvergence?

MPLS - SP A

MPLS - SP B

C-A-R2

C-A-R4

C-B-R1 C-B-R4

C-A-R3

C-A-R1

HQ-W1

HQ-W2

BR-W1

BR-W2

Link or Device Failure

Link or Device Degraded


Session Scope

• What methods are used for path selection and packet forwarding

• How does the network detect outages

• Focus on network survivability and effective utilization rather than sub-second convergence

• Modern Design using SD-WAN

• Does not address “zero loss” considerations

• Please review BRKRST-2365 Unified HA Network Design - The Evolution of the Next Generation Network

10BRKRST-2042

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11

Defining Availability

• System Availability: a ratio of the expected uptime to the experienced downtime over a period of time of the same duration

• Branch WAN High Availability: Between 99.99% and 99.999%

• Ultra High Availability: Between 99.9999% and 99.999999%

Availability Downtime / Year

98.000000% 7.3 Days

99.000000% 3.65 Days

99.500000% 1.825 Days

99.900000% 8.76 Hrs

99.990000% 52.56 Min

99.999000% 5.256 Min

99.999900% 31.536 Sec

99.999990% 3.1536 Sec

99.999999% .31536 Sec

Branch WAN

HA Targets

Ultra HA

Targets

BRKRST-2042


Building Highly Available WANsRedundancy and Path Diversity Matter

ISR

MPLS

ISR

MPLS MPLS Internet

ISR

MPLS

SINGLE

ROUTER,

SINGLE PATH

SINGLE

ROUTER,

DUAL PATHS

DUAL

ROUTERS,

DUAL PATHS

Internet Internet

ISR

ISR

Internet

ISR

MPLS Internet

ISR ISR

Internet Internet

ISR

99.95%* 99.90%*

99.995% 99.995% 99.995%

99.999% 99.999%

Downtime

per Year

4–9 Hours

Downtime

per Year

8 Hours

46 Minutes

5+ Minutes

26+ Minutes

Branch WAN

HA Solution

ISR

MPLS MPLS

ISR

99.999%

* Typical MPLS and Business Grade Broadband Availability SLAs and Downtime per Year, calculated with Cisco AS DAAP tool.

BRKRST-2042


Deployment Options

13BRKRST-2042

MPLS

Internet

LTE

MPLS

Internet

LTE

MPLS/MPLS

MPLS/Internet

MPLS/LTE

Internet/Internet

Internet/LTE

LTE/LTE

100’s of Combinations

• Introduction


• Multiple Links/Multiple Paths

• Load Sharing



• Final Wrap Up

Agenda


p 10.0.0.0/8 is variably subnetted, 14 subnets, 5 masks

B p 10.0.0.0/8 [20/0] via 172.16.0.6, 00:12:36

B p 10.3.0.0/16 [20/0] via 172.16.0.6, 00:12:36

B p 10.4.0.0/16 [200/0], 00:13:52, Null0

C p 10.4.0.41/32 is directly connected, Loopback0

D p 10.4.1.0/24 [90/307200] via 10.4.49.2, 00:14:32, Ethernet0/0

C p 10.4.49.0/30 is directly connected, Ethernet0/0

L p 10.4.49.1/32 is directly connected, Ethernet0/0

B p 10.9.0.0/16 [20/0] via 172.16.0.6, 00:12:36

100.0.0.0/8 is variably subnetted, 9 subnets, 2 masks

B 100.64.0.0/24 [20/0] via 100.64.3.1, 00:13:43

C 100.64.3.0/24 is directly connected, Ethernet0/2

L 100.64.3.2/32 is directly connected, Ethernet0/2


B 172.16.0.0/31 [20/0] via 172.16.0.6, 00:12:36

C 172.16.0.6/31 is directly connected, Ethernet0/1

L 172.16.0.7/32 is directly connected, Ethernet0/1

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP

a - application route

+ - replicated route, % - next hop override, p - overrides from PfR

Routing Table Basics


Administrative Distance

• The distance command is used to configure a rating of the trustworthiness of a routing information source, such as an individual router or a group of routers

• Numerically, an administrative distance is a positive integer from 1 to 255. In general, the higher the value, the lower the trust rating

• An administrative distance of 255 means the routing information source cannot be trusted at all and should be ignored

16BRKRST-2042

Route Source

Default

Distance

Connected Interface

0

Static Route 1

EIGRP Summary Route

5

BGP External (eBGP)

20

EIGRP Internal 90

OSPF 110

IS-IS 115

RIP 120

EIGRP External 170

BGP Internal (iBGP)

200

Unknown 255

INFORMATIONAL


router#show ip route 10.0.14.0 255.255.255.0

Routing entry for 10.0.14.0/24

Known via "eigrp 1", distance 90, metric 307200, type internal

Redistributing via eigrp 1

Last update from 10.0.121.2 on Ethernet0/1, 00:01:32 ago

Routing Descriptor Blocks:

* 10.0.121.2, from 10.0.121.2, 00:01:32 ago, via Ethernet0/1

Route metric is 307200, traffic share count is 1

Total delay is 2000 microseconds, minimum bandwidth is 10000 Kbit

Reliability 255/255, minimum MTU 1500 bytes

Loading 1/255, Hops 1

Route Selection• How is administrative distance

used to determine which route should be installed?

• Only identical routes are compared

• Identical prefixes with different prefix lengths are not the same route

• The route from the protocol with the lower administrative distance is installed

17BRKRST-2042

10.0.14.0/24 10.0.14.0/2510.0.14.0/24

EIGRP OSPFOSPF

These Two Routes Are Identical

EIGRP Internal Installed

EIGRP Internal = 90

OSPF = 110


Route Selection

• What about longest prefix comparison?

• Only identical routes are compared

• Identical prefixes with different prefix lengths are not the same route

• The route with the longest prefix is installed

18BRKRST-2042

10.0.14.0/24 10.0.14.0/2510.0.14.0/25

10.0.14.0/24

EIGRP OSPFOSPF

These Two Routes Are Identical

router#show ip route 10.0.14.0 255.255.255.0 longer-prefixes


D 10.0.14.0/24 [90/307200] via 10.0.121.2, 00:01:35, Ethernet0/1

O 10.0.14.0/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2

O 10.0.14.128/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2

OSPF InstalledLonger Prefixes

More Specific OSPF Override EIGRP

• Introduction


• Multiple Links/Multiple Paths

• Load Sharing



• Final Wrap Up

Agenda


Load Sharing

• Assume the same routing process attempts to install two routes for the same destination in the RIB

• The routing process may allow the second route to be installed based on its own rules

20BRKRST-2042

IGP OSPF IS-IS EIGRP

Route Cost Must be equal to installed route

Must be equal to installed route

Must be less than the variance times the lowest cost installed route

Maximum Paths Must be fewer than maximum-paths configured under the routing process (default = 4, maximum = 32)

Note: BGP default value for maximum-paths = 1


CEF Load Sharing

Per-Destination Per-Packet1

Default behaviour of IOS Universal

Algorithm “show cef state”

Requires “ip load-sharing per-packet”

interface configuration1

Per-flow using destination hash Per-packet using round-robin method

Packets for a given source/destination

session will take the same path

Packets for a given source/destination

session may take different paths

More effective as the number of

destinations increase

Ensures traffic is more evenly

distributed over multiple paths

Ensures that traffic for a given session

arrives in order

Potential for packets to arrive out of

sequence

1Not available in IOS-XE based images


Load Sharing

router#show ip route 192.168.239.0


Known via "eigrp 100", distance 170, metric 3072256, type external


Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago


* 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0


....

192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1


....

The Traffic Share Count Is Critical to Understanding the Actual Load Sharing of Packets Using These Two Routes

3072256/3072256 = 1


Load Sharing – with EIGRP Variance



Known via "eigrp 100", distance 170, metric 3072256, type external


Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago


* 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0


....

192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1


....If the Lower Metric Is Less than the Second Metric, the Traffic Share Count Will Be Something Other than 1 (EIGRP with Variance Configured)

3072256/3072256 = 1

2x Faster Link Gets 2 Flows vs. 1 Flow

3072256/1536128 = 2




Known via "bgp 1", distance 20, metric 0

Tag 2, type external

Last update from 10.0.122.2 00:00:16 ago


10.0.122.2, from 10.0.122.2, 00:00:16 ago


....

* 10.0.121.2, from 10.0.121.2, 00:00:16 ago


....

router#show ip bgp 192.168.239.0

BGP routing table entry for 192.168.239.0/24, version 9

Paths: (2 available, best #2, table default)

Multipath: eBGP

....

10.0.122.2 from 10.0.122.2 (10.0.0.2)

Origin IGP, metric 0, localpref 100, valid, external, multipath(oldest)

DMZ-Link Bw 312 kbytes

rx pathid: 0, tx pathid: 0

....

10.0.121.2 from 10.0.121.2 (10.0.0.2)

Origin IGP, metric 0, localpref 100, valid, external, multipath, best

DMZ-Link Bw 625 kbytes

rx pathid: 0, tx pathid: 0x0

24BRKRST-2042

Load Sharing – with eBGP dmzlink-bw

2x Faster Link Gets 2 Flows vs. 1 Flow

Only routes learnedvia eBGP Neighbors


CEF Hashing and Exact Route

• Now that we have load balancing

• Which exact path are the flows using

• “show ip cef exact-route <src-addr> [src-port] <dest-addr> [dest-port]”

#show ip cef exact-route 1.1.1.1 2.2.2.2

1.1.1.1 -> 2.2.2.2 =>IP adj out of GigabitEthernet1, addr 10.255.0.1

• Introduction



• Interface Detection

• Routing Protocols

• Static Routing and EOT

• First Hop Redundancy Protocols

• Performance Routing

• Cisco SD-WAN (Viptela)


Agenda


Interface Detection – Carrier-Delay

• Carrier-delay

• If a link goes down and comes back up before the carrier delay timer expires, the down state is effectively filtered, and the rest of the software on the switch is not aware that a link-down event occurred.

• Imposes a 2 second pause by default before processing interface events

• Disabling carrier-delay speeds convergence upon interface events

• Disabling carrier-delay can increase control-plane usage during repetitive interface events (flapping)

• IOS-XR releases support asymmetric configuration, allowing down events to be configured with one value and up events with another. Thus, providing immediate down detection and still enforcing a pause on up events in case of flapping.

BRKRST-2042


Interface Detection - Dampening• Dampening

• Imposes a logarithmic delay based on interface events

• Coupled with disabling carrier-delay, dampening protects the control-plane from repetitive events by increasing the delay before processing up events should the interface flap.

BRKRST-2042

#conf t

(config-if)#interface GigabitEthernet1

(config-if)#carrier-delay 0

(config-if)#dampening

(config-if)#end

#show dampening interface

1 interface is configured with dampening.

No interface is being suppressed.

Features that are using interface dampening:

IP Routing

• Introduction










Agenda


Routing Protocol Timers

Keepalive (B)

Hello (E,I,O)

Update (R)

Invalid (R)

Holdtime (B,E,I)

Dead (O)

Holddown (R)

Flush (R)

BGP 60 180

EIGRP

(< T1)5 (60) 15 (180)

IS-IS

(DIS)10 (3.333) 30 (10)

OSPF

(NBMA)10 (30) 40 (120)

RIP/RIPv2 30 180 180 240

Note: Cisco Default Values

INFORMATIONAL


Routing Protocol Neighbor Behavior

R2

R3

R1

Link Down

Line Protocol Down

Link Up

Loss 100%

Link Up

Neighbor Down

Link Up

Loss ~5%

BGP ~ 1 s 180 180 Never

EIGRP

(< T1)~ 1s 15 (180) 15 (180) Never

IS-IS

(DIS)~ 1s 30 (10) 30 (10) Never

OSPF

(NBMA)~ 1s 40 (120) 40 (120) Never

RIP/RIPv2 ~ 1s 240 240 Never

R4

Recovery Times by Protocol

Note: Using Cisco Default Values

INFORMATIONAL


Routing Protocol Neighbor BehaviorAdjust Hello Timers

R4#show ip bgp vpnv4 vrf cisco neighbor

BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link

BGP version 4, remote router ID 192.168.201.10

BGP state = Established, up for 1d10h

Last read 00:00:19, hold time is 180, keepalive interval is 60 seconds

R2

R3

R1 R4 BR-W1

When Configuring the Holdtime Argument for a Value of Less

than Twenty Seconds, the Following Warning Is Displayed:

% Warning: A Hold Time of Less than 20 Seconds Increases the

Chances of Peer Flapping

R4#show ip bgp vpnv4 vrf cisco neighbor

BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link

BGP version 4, remote router ID 192.168.201.10

BGP state = Established, up for 00:01:23

Last read 00:00:03, hold time is 21, keepalive interval is 7 seconds

BR-W1#

router bgp 65110

neighbor 192.168.101.9 timers 7 21


Bidirectional Forwarding Detection (BFD)

• Extremely lightweight hello protocol

• IPv4, IPv6, MPLS, P2MP

• 10s of milliseconds (technically, microsecond resolution) forwarding plane failure detection mechanism.

• Single mechanism, common and standardized

• Multiple modes: Async (echo/non-echo), Demand

• Independent of Routing Protocols

• Levels of security, to match conditions and needs

• Facilitates close alignment with hardware

33BRKRST-2042


Drivers for BFD

• Link-layer detection misses some types of outages

• e.g. Control Plane failure

• Control Plane failure detection is very conservative

• 15-40 seconds in default configurations

• Link-layer failure detection is not consistent across media types

• Less than 50ms on APS- protected SONET

• A few seconds on Ethernet

• Several seconds or more on WAN links

• Provides a measure of consistency across routing protocols

• Most current failure detection mechanisms are an order of magnitude too long for time-sensitive applications

34BRKRST-2042


Routing Protocol Neighbor BehaviorBidirectional Forwarding Detection

R1

R2

(Gi4)

R1#show bfd neighbors details

IPv4 Sessions

NeighAddr LD/RD RH/RS State Int

10.3.255.10 4104/1 Up Up Gi4

Session state is UP and using echo function with 50 ms interval.

Session Host: Software

OurAddr: 10.3.255.9

Handle: 2

Local Diag: 0, Demand mode: 0, Poll bit: 0

MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3

Received MinRxInt: 1000000, Received Multiplier: 3

Holddown (hits): 0(0), Hello (hits): 1000(1371)

Rx Count: 985, Rx Interval (ms) min/max/avg: 34/1978/1226 last: 290 ms ago

Tx Count: 1372, Tx Interval (ms) min/max/avg: 71/1137/879 last: 721 ms ago

Elapsed time watermarks: 0 0 (last: 0)

Registered protocols: EIGRP CEF

Uptime: 00:20:06

Last packet: Version: 1 - Diagnostic: 0

State bit: Up - Demand bit: 0

Poll bit: 0 - Final bit: 0

C bit: 0

Multiplier: 3 - Length: 24

My Discr.: 1 - Your Discr.: 4104

Min tx interval: 1000000 - Min rx interval: 1000000

Min Echo interval: 50000

interface GigabitEthernet4

ip address 10.3.255.9 255.255.255.252

bfd interval 50 min_rx 50 multiplier 3

router eigrp 1

network 10.3.0.0 0.0.255.255

bfd all-interfaces


ip address 172.17.2.9 255.255.255.254


router bgp 65000

neighbor 172.17.2.8 fall-over bfd

(Gi2)

Configured in milliseconds (ms)

Displayed in microseconds (µs)


Routing Protocol Neighbor BehaviorBidirectional Forwarding Detection

R1

R2

(Gi4)

<show bfd neighbors details cont’d>

IPv4 Sessions

NeighAddr LD/RD RH/RS State Int

172.17.2.8 4102/1 Up Up Gi2

Session state is UP and using echo function with 333 ms interval.

Session Host: Software

OurAddr: 172.17.2.9

Handle: 1

Local Diag: 0, Demand mode: 0, Poll bit: 0

MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3

Received MinRxInt: 1000000, Received Multiplier: 3

Holddown (hits): 0(0), Hello (hits): 1000(6076)

Rx Count: 4977, Rx Interval (ms) min/max/avg: 4/1970/1069 last: 491 ms ago

Tx Count: 6077, Tx Interval (ms) min/max/avg: 754/1180/879 last: 655 ms ago

Elapsed time watermarks: 0 0 (last: 0)

Registered protocols: BGP CEF

Uptime: 01:29:04

Last packet: Version: 1 - Diagnostic: 0

State bit: Up - Demand bit: 0

Poll bit: 0 - Final bit: 0

C bit: 0

Multiplier: 3 - Length: 24

My Discr.: 1 - Your Discr.: 4102

Min tx interval: 1000000 - Min rx interval: 1000000

Min Echo interval: 333000


ip address 10.3.255.9 255.255.255.252


router eigrp 1

network 10.3.0.0 0.0.255.255

bfd all-interfaces


ip address 172.17.2.9 255.255.255.254


router bgp 65000

neighbor 172.17.2.8 fall-over bfd

(Gi2)

Configured in milliseconds (ms)

Displayed in microseconds (µs)


Routing Protocol Neighbor BehaviorDetecting Unreachable Neighbor (Hello Timers vs. BFD)

BFD: Elapsed Time Between 100 - 150 ms with 50ms interval (example 50ms)

R1#show clock

*09:35:44.408 UTC Sat Jan 27 2018

R1#

*Jan 27 09:35:45.571: %BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD

session ld:4101 handle:2,is going Down Reason: ECHO FAILURE

*Jan 27 09:35:45.575: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG:

bfd_session_destroyed, ld:4101 neigh proc:EIGRP, handle:2 act

*Jan 27 09:35:45.580: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor

10.3.255.10 (GigabitEthernet4) is down: BFD peer down notified

R1#show clock

*09:58:27.716 UTC Sat Jan 27 2018

R1#

*Jan 27 09:58:40.612: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor

10.3.255.10 (GigabitEthernet4) is down: holding time expired

EIGRP Default: Elapsed Time Between 10 – 15 Sec

R1 R2

100% Packet Loss (Link Up)

12.896 seconds

1.172 seconds1

1injecting 100% loss after hitting show clock in the lab

• Introduction










Agenda


EOT, Static Routing, and DDR

• Enhanced Object Tracking (EOT)

• Static Routing Options

• Floating Static Routes

• Reliable Static Routing (RSR) using EOT

• Dial on Demand Routing (DDR)

• Backup Interface

• Dialer Watch

• EEM Script

• DMVPN State Tracking

• More information:

• https://www.cisco.com/c/en/us/support/docs/dial-access/dial-on-demand-routing-ddr/10213-backup-main.html

39BRKRST-2042


Enhanced Object Tracking (EOT)Local Significance

Track Options Syntax

Line-Protocol State of Interface

track object-number interface type number line-protocol

track 1 interface serial 2/0 line-protocol

IP-Routing State of Interface

track object-number interface type number ip routing

track 2 interface ethernet 1/0 ip routing

IP-Route Reachability

track object-number ip route IP-Addr/Prefix-len reachability

track 3 ip route 10.16.0.0/16 reachability

Threshold* of IP-Route Metrics

track object-number ip route IP-Addr/Prefix-len metric threshold

track 4 ip route 10.16.0.0/16 metric threshold

Router#show track 100

Track 100

Interface Serial2/0 line-protocol

Line protocol is Up

1 change, last change 00:00:05

Tracked by:

GLBP FastEthernet0/1 1

Router#show track 103

Track 103

IP route 10.16.0.0 255.255.0.0 reachability

Reachability is Up (EIGRP)

1 change, last change 00:02:04

First-hop interface is FastEthernet0/0

Tracked by:

GLBP FastEthernet0/1 1

IPv6 Support added in 15.3(3)S 15.4(1)T

* EIGRP, OSPF, BGP, Static Thresholds Are Scaled to Range of (0 – 255)


Enhanced Object TrackingExternal Significance


IP SLAs Operationtrack object-number ip sla type number state

track 5 ip sla 4 state

Reachability of an IP SLAs Host

track object-number ip sla type number reachability

track 6 ip sla 4 reachability

dhcp

dns

ethernet

frame-relay

ftp

http

icmp-echo1

icmp-jitter

mpls

path-echo

path-jitter

tcp-connect1

udp-echo1

udp-jitter1

voip

Types of IP SLA Probes:

1Available for IPv6


Enhanced Object TrackingCompound Operations


list boolean

track object-number list boolean {and|or}

and - both are up for object to be up

or - one is up for object to be up

track 5 list boolean or

object 51

object 52 not ! Negates state of object

list threshold

track object-number list threshold {weight|percentage}

track 6 list threshold weight

object 61 weight 20 ! Twice as important

object 62 ! Default weight 10

object 63

object 64

threshold weight up 30 down 25


Reliable Static RoutingTracking IP SLA

BR-W1

IP SLA

(.9)

IP SLA


object 400

object 401



ip sla 400

icmp-echo 10.100.100.100 source-ip 10.1.2.120

timeout 100

frequency 10

ip sla schedule 400 life forever start-time now

ip sla 401


timeout 100

frequency 10


!

ip route 10.100.100.100 255.255.255.255 Ethernet 0/1 192.168.101.9

ip route 10.100.200.100 255.255.255.255 Ethernet 0/1 192.168.101.9

ip route 10.100.100.100 255.255.255.255 Null0 2

ip route 10.100.200.100 255.255.255.255 Null0 2

ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4

ip route 10.100.0.0 255.255.0.0 192.168.201.9 200

BR-W1#show ip route 10.100.0.0 255.255.0.0

S 10.100.0.0/16 [1/0] via 192.168.101.9

192.168.101.8/29

192.168.201.8/29

(.9)

BR-W1#show ip route track-table

ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [up]

Static Host Route Guarantees probe destination only reachable via desired path

Floating Null Route Guarantees probe destination isn’t reachable when interface down


Reliable Static RoutingTracking IP SLA

BR-W1

IP SLA

(.9)

192.168.101.8/29 192.168.201.8/29

(.9)

Unable to Reach

IP SLA Responders

BR-W1#show ip route 10.100.0.0 255.255.0.0 longer-prefixes

S 10.100.0.0/16 [200/0] via 192.168.201.9

S 10.100.100.100/32 [1/0] via 192.168.101.9, Ethernet0/1

S 10.100.200.100/32 [1/0] via 192.168.101.9, Ethernet0/1

BR-W1#

*Mar 12 03:57:28.367: %TRACKING-5-STATE: 400 ip sla 400 reachability Up->Down

*Mar 12 03:57:37.374: %TRACKING-5-STATE: 401 ip sla 401 reachability Up->Down

*Mar 12 03:57:38.137: %TRACKING-5-STATE: 4 list boolean or Up->Down

BR-W1#show ip route track-table

ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [down]

Floating Static Installed

IP SLA

IPv6 Reliable Static Routing added in 15.4(1)T


EEM ScriptExample: IPv6 Static Route Event Tracking

BR RTR

WAN RTR

ipv6 route 2001:DB8::12/128 2001:DB8:B::5

ip sla 610

icmp-echo 2001:DB8::12 source-interface GigabitEthernet0/1.99

threshold 1000

frequency 10


track 600 list threshold percentage

<snip additional tracked objects>

object 610

threshold percentage down 40 up 60

track 610 ip sla 610

event manager applet DISABLE-STATIC-IPv6

event track 600 state down

action 1 cli command "enable"

action 2 cli command "configure terminal"

action 3 cli command "no ipv6 route ::/0 2001:DB8:B::5"

action 4 cli command "end"

action 99 syslog msg “DEFAULT IPv6 ROUTE DISABLED"

BR-RTR#

14:22:14: %TRACKING-5-STATE: 610 ip sla 610 state Up->Down

14:22:14: %TRACKING-5-STATE: 600 list threshold percentage Up->Down

14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:DISABLE-STATIC-IPv6)

14:22:14: %HA_EM-6-LOG: DISABLE-STATIC-IPv6: DEFAULT IPv6 ROUTE DISABLED

2001:DB8:B::5

WAN RTR

Unable to Reach

IP SLA Responder

IP SLA

15.4(1)T added Reliable Static Routing


event manager applet ENABLE-STATIC-GIG0-0

event track 62 state up



action 3 cli command "ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10"


action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 ENABLED"

ip sla 110

icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0

vrf INET-PUBLIC1 ! fVRF configuration

threshold 1000

frequency 15


ip sla 111


vrf INET-PUBLIC1

threshold 1000

frequency 15





object 60

object 61

IP SLA

Probes

Note: This method is compatible with dual Internet DHCP design.

Black Hole Route DetectionIPSLA with EEM

Lost connection to ISP but DHCP route stays in the routing table

event manager applet DISABLE-STATIC-GIG0-0




action 3 cli command "no ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10"


action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 DISABLED"

(config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ?

<cr>


Interface GigabitEthernet0/0

vrf forwarding INET-PUBLIC1

ip address dhcp

ip sla 110


vrf INET-PUBLIC1 ! fVRF configuration

threshold 1000

frequency 15


ip sla 111


vrf INET-PUBLIC1

threshold 1000

frequency 15





object 60

object 61

IP SLA

Probes

Note: This method is compatible with dual Internet DHCP design.

47BRKRST-2042

Black Hole Route DetectionIPSLA with Recursive Routing

Lost connection to ISP but DHCP route stays in the routing table

ip route 192.0.2.33 255.255.255.255 GigabitEthernet0/0 dhcp 10

ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 192.0.2.33 10 track 62

(config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ?

<cr>


EEM ScriptExample: 3G Backup with Event Tracking

3G-RTR

NAS

(Ce0/0/0)

WAN RTR

ip sla 100


threshold 1000

frequency 15



event manager applet ACTIVATE-3G




action 3 cli command "interface cellular0/0/0"

action 4 cli command "no shutdown"


action 99 syslog msg "Activating 3G interface"

192.168.4.22

VPN RTR

http://www.cisco.com/go/cvd/wan VPN Remote Site over 3G/4G Technology Design Guide

14:22:14: %TRACKING-5-STATE: 60 ip sla 100 reachability Up->Down

14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:ACTIVATE-3G)

14:22:14: %HA_EM-6-LOG: ACTIVATE-3G: Activating 3G interface

14:22:34: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to up

14:22:34: %DIALER-6-BIND: Interface Ce0/0/0 bound to profile Di1

14:22:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to up

14:22:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel11, changed state to up

14:22:40: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON

14:22:42: %DUAL-5-NBRCHANGE: EIGRP-IPv4 201: Neighbor 10.4.36.1 (Tunnel11) is up: new

adjacency

http://www.cisco.com/go/sba

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKRST-2042

DMVPN Interface State ControlExample: 3G Backup with DMVPN

3G-RTR

NAS

(Ce0/0/0)

WAN RTR


object 101 not

track 101 interface Tunnel100 line-protocol

interface Tunnel200

if-state track 2

tunnel source Cellular0/0/0

end

#show track 2

Track 2

List boolean or

Boolean OR is Down

7 changes, last change 00:07:55

object 101 not Up

Tracked by:

IF-State Control 2

192.168.4.22

VPN RTR

17:24:18.682: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel100, changed state to down

17:24:18.682: %TRACK-6-STATE: 101 interface Tu100 line-protocol Up -> Down

17:24:18.744: %TRACK-6-STATE: 2 list boolean or Down -> Up

17:24:28.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel200, changed state to up

17:24:29.276: %BGP-5-ADJCHANGE: neighbor 192.168.200.12 Up

17:24:37.505: %BGP-5-ADJCHANGE: neighbor 192.168.200.22 Up

#show track 2

Track 2

List boolean or

Boolean OR is Up

8 changes, last change 00:00:32

object 101 not Down

Tracked by:

IF-State Control 2

• Introduction










Agenda


First Hop Redundancy Protocols (FHRP)

• Hot Standby Router Protocol (HSRP)

• Virtual Router Redundancy Protocol (VRRP)

• Gateway Load Balancing Protocol (GLBP)

Failure Protection for the First Hop IP Router

BR-W2BR-W1


Drivers for FHRPs

• Provide routing redundancy for access layer

• How to handle failover when end-hosts have only a single IP default gateway and cached ARP entry

• Provide routing redundancy for devices that depend on static routing

• Some firewalls do not support dynamic routing

• Independent of routing protocols

• Works with any routing protocol and static routing

• Capable of providing sub-second failover

• Provides load sharing capabilities (GLBP) transparent to end host

52BRKRST-2042


(.2) (.3)HSRP

(.1)

Active Router

Standby Router

VIP

BR-W1 BR-W2

Default Gateway: (.1)DG MAC: MAC VIP

interface FastEthernet0/0

ip address 10.1.2.2 255.255.255.0

standby version 2

standby 4 ip 10.1.2.1

standby 4 priority 110

standby 4 preempt

standby 6 ipv6 autoconfig


standby 6 preempt

ipv6 address 2001:DB8:5:1::1/64

BR-W1#show standby brief

P indicates configured to preempt.

|

Interface Grp Pri P State Active Standby Virtual IP

Fa0/0 4 110 P Active local 10.1.2.3 10.1.2.1

Fa0/0 6 110 P Active local FE80::A8BB:CCFF:FE00:3400

FE80::5:73FF:FEA0

:6


ip address 10.1.2.3 255.255.255.0

standby version 2

standby 4 ip 10.1.2.1

standby 4 preempt

standby 6 ipv6 autoconfig

standby 6 preempt

ipv6 address 2001:DB8:5:1::2/64

Hot Standby Routing Protocol (HSRP)

HSRP—Global IPv6 Addresses Available

for Static Deployments



Fa0/0 4 100 P Standby 10.1.2.2 local 10.1.2.1

Fa0/0 6 100 P Standby FE80::A8BB:CCFF:FE00:3300

local FE80::5:73FF:FEA0

:6

53BRKRST-2042



(.2) (.3)HSRP

(.1)

Active Router

VIP

Local

FailuresBR-W1 BR-W2

Default Gateway: (.1)DG MAC: MAC VIP



|


Fa0/0 4 100 P Active local unknown 10.1.2.1

Fa0/0 6 100 P Active local unknown FE80::5:73FF:FEA0

:6



(.2) (.3)HSRP

(.1)

Active Router

VIP

BR-W1 BR-W2

Complex Failure

Requires “Enhanced

Object Tracking (EOT)”

(.2) (.3)HSRP

Standby Router

Upstream/Remote

Failures

BR-W1 BR-W2

Active Router

(.1) VIP

BR-W1#

track 100 interface serial2/0 line-protocol

!


standby version 2


standby 4 track 100 decrement 20


standby 6 track 100 decrement 20


BR-W1#show glbp brief

Interface Grp Fwd Pri State Address Active router Standby router

Fa0/0 4 - 100 Active 10.1.2.1 local 10.1.2.3

Fa0/0 4 1 - Active 0007.b400.0401 local -

Fa0/0 4 2 - Listen 0007.b400.0402 10.1.2.3 -

Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600

local FE80::A8BB:CCF

F:FE00:3400

Fa0/0 6 1 - Active 0007.b400.0601 local -

Fa0/0 6 2 - Listen 0007.b400.0602 FE80::A8BB:CCFF:FE00:3400

-

56BRKRST-2042

Gateway Load Balancing Protocol (GLBP)

(.2) (.3)GLBP

(.1)

Default Gateway: (.1)DG MAC: AVF A

AVF A

VIP VIP(.1)

BR-W1 BR-W2

AVG

Default Gateway: (.1)DG MAC: AVF B

AVF B

SVG

AVG = Active Virtual Gateway

SVG = Standby Virtual Gateway

AVF = Active Virtual Forwarder

BR-W1#show run int fa0/0


ip address 10.1.2.2 255.255.255.0

glbp 4 ip 10.1.2.1

glbp 4 preempt

glbp 4 weighting 110 lower 100

glbp 6 ipv6 autoconfig

glbp 6 preempt


ipv6 address 2001:DB8:5:1::1/64



Fa0/0 4 - 100 Standby 10.1.2.1 10.1.2.2 local

Fa0/0 4 1 - Listen 0007.b400.0401 10.1.2.2 -

Fa0/0 4 2 - Active 0007.b400.0402 local -

Fa0/0 6 - 100 Standby FE80::7:B4FF:FE00:600

FE80::A8BB:CCFF:FE00:3300

local

Fa0/0 6 1 - Listen 0007.b400.0601 FE80::A8BB:CCFF:FE00:3300

-

Fa0/0 6 2 - Active 0007.b400.0602 local -




Fa0/0 4 - 100 Active 10.1.2.1 local unknown

Fa0/0 4 1 - Active 0007.b400.0401 local -

Fa0/0 4 2 - Active 0007.b400.0402 local -

Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600

local unknown

Fa0/0 6 1 - Active 0007.b400.0601 local -

Fa0/0 6 2 - Active 0007.b400.0602 local -

BR-W2#

*May 26 19:09:14.260: %GLBP-6-STATECHANGE: FastEth0/0 Grp 4 state Standby -> Act

ive

*May 26 19:09:15.326: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Liste

n -> Active

*May 26 19:09:15.826: %GLBP-6-STATECHANGE: FastEth0/0 Grp 6 state Standby -> Act

ive

*May 26 19:09:16.856: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 6 Fwd 1 state Liste

n -> Active

57BRKRST-2042

Gateway Load Balancing Protocol (GLBP)

(.2) (.3)GLBP

(.1) VIPLocal

Failures

BR-W1 BR-W2AVF A

AVF B

AVG

Default Gateway: (.1)DG MAC: AVF A

Default Gateway: (.1)DG MAC: AVF B





GLBP with Enhanced Object Tracking




(.2) (.3)GLBP

(.1) VIP

BR-W1 BR-W2AVF A

AVF B

AVG

Complex Failure

Requires “Enhanced

Object Tracking (EOT)”

(.2) (.3)GLBP

VIP(.1)

BR-W1 BR-W2AVG

Upstream/Remote Failures

Requires “Enhanced Object

Tracking”AVF A

AVF B

Branch


Enhanced Object Tracking (EOT)Tracking IP SLA ip sla 100


timeout 100

frequency 10


ip sla 200


timeout 100

frequency 10


ip route 10.100.100.100 255.255.255.255 FastEthernet0/1 192.168.101.9

ip route 10.100.200.100 255.255.255.255 FastEthernet0/1 192.168.101.9

ip route 10.100.100.100 255.255.255.255 Null0 2

ip route 10.100.200.100 255.255.255.255 Null0 2

(.2) (.3)GLBP

AVF B

BR-W1 BR-W2

AVF A

IP SLA

(.1)VIP

IP SLA

(.1) VIP

Lo0 10.100.100.100 Lo0 10.100.200.100

BR-W1#show ip sla statistics

IPSLA operation id: 100

Latest RTT: 1 milliseconds

Latest operation start time: *04:42:11.444 UTC Tue Feb 17 2009

Latest operation return code: OK

Number of successes: 46

Number of failures: 0

Operation time to live: Forever

IPSLA operation id: 200

Latest RTT: 1 milliseconds

Latest operation start time: *04:42:11.356 UTC Tue Feb 17 2009

Latest operation return code: OK

Number of successes: 24

Number of failures: 0

Operation time to live: Forever


Enhanced Object TrackingTracking IP SLA

BR-W1#




object 100

object 200


ip address 10.1.2.2 255.255.255.0

glbp 4 ip 10.1.2.1

glbp 4 priority 110

glbp 4 preempt


glbp 4 load-balancing weighted

glbp 4 weighting track 1 decrement 20

BR-W1#show glbp

FastEthernet0/0 – Group 4

State is Active

1 state change, last state change 00:09:59

Virtual IP address is 10.1.2.1

Hello time 3 sec, hold time 10 sec

Next hello sent in 2.336 secs

Redirect time 600 sec, forwarder timeout 14400 sec

Preemption enabled, min delay 0 sec

Active is local

Standby is 10.1.2.3, priority 105 (expires in 7.808 sec)

Priority 110 (configured)

Weighting 110 (configured 110), thresholds: lower 100,

upper 110

Track object 1 state Up decrement 20

Load balancing: weighted

Group members:

aabb.cc00.0110 (10.1.2.2) local

aabb.cc00.0410 (10.1.2.3)

There are 2 forwarders (1 active)

Forwarder 1

State is Active

<SNIP>

Forwarder 2

State is Listen

<SNIP>

(.2) (.3)GLBP

AVF B

BR-W1 BR-W2

AVF A

IP SLA

(.1)VIP

IP SLA

(.1) VIP


BR-W2#show glbp

FastEthernet0/0 – Group 4

State is Standby

1 state change, last state change 00:28:16

Virtual IP address is 10.1.2.1

Hello time 3 sec, hold time 10 sec

Next hello sent in 1.856 secs

Redirect time 600 sec, forwarder timeout 14400 sec

Preemption enabled, min delay 0 sec

Active is 10.1.2.2, priority 110 (expires in 10.400 sec)

Standby is local

Priority 105 (configured)

Weighting 110 (configured 110), thresholds: lower 100, upper 110

Track object 1 state Up decrement 20

Load balancing: weighted

Group members:

aabb.cc00.0110 (10.1.2.2)

aabb.cc00.0410 (10.1.2.3) local

There are 2 forwarders (2 active)

Forwarder 1

State is Active

<SNIP>

Forwarder 2

State is Active

<SNIP>

BR-W1#

*Feb 17 05:17:25: %TRACKING-5-STATE: 100 ip sla 100 state Up->Down

*Feb 17 05:17:25: %TRACKING-5-STATE: 200 ip sla 200 state Up->Down

*Feb 17 05:17:26: %TRACKING-5-STATE: 1 list boolean or Up->Down

*Feb 17 05:17:38: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state

Active -> Listen

Enhanced Object TrackingComposite Failure

BR-W1 Remains

Active Virtual

Gateway (AVG)

BR-W2 Becomes

Active Virtual

Forwarder (AVF)

for both A and B

(.2) (.3)GLBP

(.1)

AVF A

AVF B

VIP

Unable to Reach Either

IP SLA Responder

IP SLA IP SLA

BR-W1 BR-W2AVG

• Introduction










Agenda


Small Office

But, Are the Applications Performing

Adequately?

E-Mail

63BRKRST-2042

Enterprise WAN Challenge

Headquarters

Branch Office

Internet VPN

MPLS

ATM

Frame Relay

WAN AvailabilityTwo Paths, Two Providers


Performance Routing (PfR)

• Uses Reachability, Delay, Loss, Jitter, and Load to determine the best path

• PfR Components

• BR—Border Router (Forwarding Path)

• MC—Master Controller (Decision Maker)

Remote Office

Campus

SOHO

Bottlenecks

Best Metric Path

Alternate Path

MPLS orPrimary ISP

SiSi

ISP B ISP CISP AMC

MC/BR

MC/BR

BR

BR

BR


Traditional Topology

• Routing protocol selects path

• Blackhole reconvergence can take minutes

• Will not recover from brownouts

MPLS - SP A

MPLS - SP B

Headquarters Branch

A-R2

A-R4

B-R1 B-R2

A-R3

A-R1

HQ-W1

HQ-W2

BR-W1

BR-W2


PfR Enabled TopologyL3—L7 Aware

MPLS - SP A

MPLS - SP B

Hub MC BR

BR

C-A-R4

C-B-R1 C-B-R2

HQ-W1

HQ-W2

HQ-MC1

BR-W2

C-A-R1

BR-W1

BR

Live Traffic Monitoring

Smart Probe

Domain Policy Configured and Deployed by PfR HubMaster Controller

MC = Master Controller

BR = Border Router

HS

RP

C-A-R2

C-A-R3

Headquarters Branch

MC/BR

HQ-IPSLA

PfR uses routing protocol and policy to select path

Live Traffic Monitoring significantly improves reconvergence due to blackholes and brownouts

• Introduction










Agenda


Overlay Management Protocol (OMP)• TCP based extensible control plane protocol

• Runs between vEdge routers and vSmart

controllers and between the vSmart controllers- Inside TLS/DTLS connections

• Leverages address families to advertise

reachability for TLOCs, unicast/multicast

destinations (statically/dynamically learnt service

side routes), service routes (L4-L7), BFD stats (TE

and H-SDWAN) and Cloud onRamp for SaaS

probe stats (gateway)- Uses attributes

• Distributes IPSec encryption keys, and data and

app-aware policies (embedded NETCONF)

vSmart vSmart

vSmart

vEdge vEdge

Note: vEdge routers need not connect to all vSmart Controllers

68BRKRST-2042


Bidirectional Forwarding Detection (BFD)

vEdge vEdge

vEdge

vEdge vEdge

• Path liveliness and quality measurement detection

protocol

- Up/Down, loss/latency/jitter, IPSec tunnel MTU

• Runs between all vEdge and vEdge Cloud routers in

the topology- Inside IPSec tunnels

- Operates in echo mode

- Automatically invoked at IPSec tunnel establishment

- Cannot be disabled

• Uses hello (up/down) interval, poll (app-aware)

interval and multiplier for detection- Fully customizable per-vEdge, per-color

69BRKRST-2042


Path Quality and Liveliness Detection

Hello Interval (ms)

Multiplier (n)

Hello Interval (ms)

Poll Interval (ms)Poll IntervalPoll Interval

App-Route Multiplier (n)

• Each vEdge router sends BFD hello packets

for path quality and liveliness detection- Packets echoed back by remote site

• Hello interval and multiplier determine how

many BFD packets need to be lost to declare

IPSec tunnel down

• Number of hello intervals that fit inside poll

interval determines the number of BFD

packets considered for establishing poll

interval average path quality

• App-route multiplier determines number of

poll intervals for establishing overall average

path quality

Liveliness

Quality

72BRKRST-2042


Critical Applications SLA

Path1: 10ms, 0% loss, 5ms jitter



vManage App Aware Routing PolicyApp A path must have:

Latency < 150ms

Loss < 2%

Jitter < 10ms

vEdge Routers continuously

perform path liveliness and

quality measurements

Internet

MPLS

4G LTE

IPSec Tunnel

Remote Site

Regional

Data CenterPath 2

73BRKRST-2042


Transport Redundancy - Meshed

MPLS Internet

vEdge routers are directly connected to all the

transports- No need for L2 switches front-ending the vEdge

routers

When transport goes down, vEdge routers detect

the condition and bring down the tunnels built

across the failed transport- BFD times out across tunnels

Both vEdge routers still draw the traffic for the

prefixes available through the SD-WAN fabric

If one of the vEdge routers fails (dual failure),

second vEdge router takes over forwarding the

traffic in and out of site- Both transport are still available

vEdgevEdge

76BRKRST-2042


Transport Redundancy – TLOC Extension

MPLS Internet

vEdgevEdge

vEdge routers are connected only to their

respective transports

vEdge routers build IPSec tunnels across directly

connected transports and across the transports

connected to the neighboring vEdge router- Neighboring vEdge router acts as an underlay

router for tunnels initiated from the other vEdge

If one of the vEdge routers fails (dual failure),

second vEdge router takes over forwarding the

traffic in and out of site- Only transport connected to the remaining vEdge

router can be used

77BRKRST-2042


Path and Remote-End Redundancy

vEdge routers leverage BFD for detecting

tunnel liveliness

• If intermediate network path through the

SD-WAN fabric fails or if the remote-end

vEdge router (e.g. data center) fails, BFD

hellos will time out and remote site vEdge

router will bring down its relevant IPSec

tunnels

• Traffic will be rerouted after the failed

condition had been detected- BFD hello timer and multiplier can be

tweaked for faster detection

InternetMPLS

Data

Center

Remote

Site

78BRKRST-2042


Summary of Convergence Techniques

R2

R3

R1

Link Down Link Up

Neighbor Down

Link Up

Loss ~5%

Upstream Blackhole

Upstream Brownout

Routing Protocols

BFD N/A1 N/A1

EOT

RSR using EOT (w/IP SLA)

PfR

SD-WAN

R4

Effectiveness of Various Techniques for Different Outage Types

Excellent Option

SubOptimal Option

Bad Option

1BFD Multihop support for Static and BGP routes

• Introduction




• MPLS Dual Carrier

• MPLS + Internet

Agenda


Dual WAN (MPLS—Dual Carrier)PE-CE Protocol: BGP

• Default behavior: 1-way load sharing

• Load is shared from HQ to Branch

BR-W1#show ip route

B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

HQ-CORE1#show ip route

D EX 10.1.2.0/24 [170/258816] via 10.1.1.110, 02:24:22, Vlan10

[170/258816] via 10.1.1.210, 02:24:22, Vlan10

A-R4

HQ-CORE1

HQ-W2

BR-W1

A-R1

10.100.0.0/16

MPLS - SP A

MPLS - SP BB-R1 B-R4

10.1.2.0/24

EIGRP eBGP eBGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.101.8/29

192.168.201.8/29

• Only one link used Branch to HQ


Dual WAN (MPLS—Dual Carrier)

• IGP (EIGRP examples)

• Routes redistributed from BGP into IGP (match & tag)

• BGP routes are treated as IGP external

• BGP

• No iBGP required between HQ-W1 & HQ-W2 (CE routers)

• Routes redistributed from IGP into BGP except those tagged as originally sourced from BGP

PE-CE Protocol: BGP Layer 3 Campus Locations

A-R4

HQ-CORE1

HQ-W2

BR-W1

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

EIGRP eBGP eBGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.101.8/29

192.168.201.8/29


HQ-W1#

router eigrp networkers

address-family ipv4 unicast autonomous-system 65110

topology base

redistribute bgp 65110 metric 45000 100 255 1 1500

address-family ipv6 unicast autonomous-system 65110

topology base

redistribute bgp 65110 metric 45000 100 255 1 1500

HQ-CORE1

10.100.0.0/16

HQ-CORE2

10.1.1.0/24

BR

BR

HQ-W1

HQ-W2

eBGP

eBGP

10.1.1.0/2410.1.2.0/24

EIG

RP

Routes into EIGRP

HQ-W1#

router bgp 65110

address-family ipv4

redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES

address-family ipv6

redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES

!

route-map BLOCK-TAGGED-ROUTES deny 10

match tag 65100 65200

route-map BLOCK-TAGGED-ROUTES permit 20

!

Routes into BGP

83BRKRST-2042

Dual WAN (MPLS—Dual Carrier)Mutual Route Redistribution Detail

AS 65100

AS 65200

iBG

P

BGP Redistribution to IGP automatically tags routes with neighbor AS Number


Dual WAN (MPLS—Dual Carrier)PE-CE Protocol: BGP Layer 2 Single Router Branch

• Is it possible to load share from Branch to HQ?

• BGP Multipath

• Allows installation of multiple BGP paths to same destination

• Requirements (all must be equal)• Neighbor AS or

AS-PATH

• Weight

• Local Preference

• AS-PATH length

• Origin

• Med

BR-W1#show ip bgp

Network Next Hop Metric LocPrf Weight Path

* 10.100.0.0/16 192.168.201.9 0 65200 65200 ?

*> 192.168.101.9 0 65100 65100 ?

A-R4

HQ-CORE1

HQ-W2

BR-W1

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

EIGRP eBGP eBGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.201.8/29

192.168.101.8/29

BR-W1#show ip route

B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00


Dual WAN (MPLS—Dual Carrier)PE-CE Protocol: BGP Layer 2 Single Router Branch

• Is it possible to load share from Branch to HQ?

• maximum-paths 2

• Requires hidden command:• bgp bestpath as-path multipath-relax

router bgp 65110

bgp bestpath as-path multipath-relax

address-family ipv4

maximum-paths 2

address-family ipv6

maximum-paths 2

BR-W1#show ip route

B 10.100.0.0/16 [20/0] via 192.168.201.9, 00:03:44

[20/0] via 192.168.101.9, 00:03:44

A-R4

HQ-CORE1

HQ-W2

BR-W1

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

EIGRP eBGP eBGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.201.8/29

192.168.101.8/29


Performance Routing (PfRv3)Basic Configuration—Dedicated MC, BRs

Hub MC

BR

BR

Headquarters

HQ-W1

HQ-W2

HQ-MC


BR = Border Router

PfR = Performance Routing

HQ-MC#

domain default

vrf default

master hub

source-interface Loopback0

password pfrv3

HQ-W2#

domain default

vrf default

border


master 10.9.0.93

password pfrv3


domain path mplsB

HQ-W1#

domain default

vrf default

border


master 10.9.0.93

password pfrv3


domain path mplsA


Performance Routing (PfRv3)Simplification of configuration via MC to MC peering

HQ-MC#show run | s domain default

domain default

vrf default

master hub


site-prefixes prefix-list LOCAL_PREFIX

password pfrv3

load-balance

class VOICE sequence 10

match dscp ef policy voice

path-preference mplsA fallback mplsB

class VIDEO sequence 20

match dscp af41 policy real-time-video


class critical-application sequence 30

match dscp af31 policy low-latency-data


!

ip prefix-list LOCAL_PREFIX seq 5 permit 10.9.0.0/16

Local Prefixes (pre-configured)

Each MC learns and

announces its inside prefixes.

MC source-interfaceaddress is local site-id

Head-end Configuration

Load Balance all othernon-performance traffic-classes

Performance Traffic-Classes

Built-In Policy Template

Custom Policy Available


domain default

vrf default

border


master local

password pfrv3

master branch


password pfrv3

hub 10.9.0.93

!

interface FastEthernet0/0.120

bandwidth 4000

interface FastEthernet0/0.220

bandwidth 4000

88BRKRST-2042

Performance Routing (PfRv3)Basic Configuration—Combined MC and BR


BR = Border Router

Branch

MC/BR

BR-W1

BR-W1#show ip bgp

Network Next Hop Metric LocPrf Weight Path

*> 10.100.100.0/24 192.168.101.9 10000 0 65100 65100 ?

* 192.168.201.9 2000 0 65200 65200 ?

*> 10.100.200.0/24 192.168.101.9 10000 0 65100 65100 ?

* 192.168.201.9 2000 0 65200 65200 ?

Bandwidth affects utilization calculations

BGP Configured to Choose Preferred Path

Choosing a single path can be desired, PfR will utilize the BGP Table or EIGRP Topology for Parent Route verification

BR-W1#show ip route

B 10.100.100.0 [20/0] via 192.168.101.9, 03:32:30

B 10.100.200.0 [20/0] via 192.168.101.9, 03:32:30


BR-W1#show domain default master traffic-classes summary

Dst-Site-Pfx Dst-Site-Id APP DSCP TC-ID APP-ID State

SP PC/BC BR/EXIT

10.9.0.0/16 10.9.0.93 N/A default 1 N/A CN

mplsB 2/1 10.3.0.31/FastEthernet0/0.220

10.9.0.0/16 10.9.0.93 N/A ef 4 N/A CN

mplsA 7/8 10.3.0.31/FastEthernet0/0.120

10.9.0.0/16 10.9.0.93 N/A af41 3 N/A CN

mplsA 5/6 10.3.0.31/FastEthernet0/0.120

10.9.0.0/16 10.9.0.93 N/A af31 2 N/A CN

mplsA 3/NA 10.3.0.31/FastEthernet0/0.120

Total Traffic Classes: 4 Site: 4 Internet: 0

89BRKRST-2042

Performance Routing (PfRv3)Load Sharing Example

• Both MPLS carriers are now being utilized

• More prefixes and flows result in better load sharing

A-R4

HQ-CORE1

HQ-W2

BR-W1

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.201.8/29

192.168.101.8/29


BR-W1#show domain default master exits

BR address: 10.3.0.31 | Name: Fast0/0.120| type: external | Path: mplsA |

Egress capacity: 1500 Kbps | Egress BW: 292 Kbps | Ideal:241 Kbps |

over: 60 Kbps | Egress Utilization: 19 %

DSCP: af31[26]-Number of Traffic Classes[1]

DSCP: af41[34]-Number of Traffic Classes[1]

DSCP: ef[46]-Number of Traffic Classes[1]

BR address: 10.3.0.31 | Name: Fast0/0.220| type: external | Path: mplsB |

Egress capacity: 2000 Kbps | Egress BW: 254 Kbps | Ideal:321 Kbps |

under: 59 Kbps | Egress Utilization: 12 %

DSCP: default[0]-Number of Traffic Classes[1]

90BRKRST-2042

Performance Routing (PfRv3)Load Sharing Example

• Both MPLS carriers are now being utilized

• More prefixes and flows result in better load sharing

A-R4

HQ-CORE1

HQ-W2

BR-W1

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.201.8/29

192.168.101.8/29


Performance Routing (PfRv3)Multiple Paths—Select Best Path by Application Requirements

• Monitor relevant path characteristics (one-way delay, loss, jitter, …)

• path A: <5 ms delay, 0% loss

• path B: < 50 ms delay, 0% loss

• Live Application Performance with Performance Monitor (TCP and RTP)

• Path without active traffic must be evaluated

• Smart Probe sent 1/3 Monitor Interval on Active path for validation, default 30 seconds. 10 packets every 500ms on fallback path.

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

BR-W1


HQ-MC#

domain default

vrf default

master hub

<SNIP>




<SNIP>

Performance Routing (PfRv3)Multiple Paths—Select Best Path by Application Requirements

BR-W1#show domain default master policy



class type: Dscp Based


priority 2 packet-loss-rate threshold 1.0 percent

priority 1 one-way-delay threshold 150 msec

priority 3 jitter threshold 30000 usec

priority 2 byte-loss-rate threshold 1.0 percent

Number of Traffic classes using this policy: 1

<SNIP>

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

BR-W1

93


Performance Routing (PfRv3)Active Prefix Monitoring

BR-W1# show domain default master traffic-classes dscp ef

Dst-Site-Prefix: 10.9.0.0/16 DSCP: ef [46] Traffic class id:4

TC Learned: 05:55:01 ago

Present State: CONTROLLED

Current Performance Status: in-policy

Current Service Provider: mplsA since 05:54:31

Previous Service Provider: mplsA for 1711 sec

BW Used: 8 Kbps

Present WAN interface: FastEthernet0/0.120 in Border 10.3.0.31

Present Channel (primary): 7

Backup Channel: 8

Destination Site ID: 10.9.0.93

Class-Sequence in use: 10

Class Name: VOICE using policy voice

BW Updated: 00:00:01 ago

Reason for Route Change: Uncontrolled to Controlled Transition

--------------------------------------------------------------------------------


A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

BR-W1


Performance Routing (PfRv3)Path Disruption: Loss

BR-W1#show domain default master traffic-classes dscp ef

Dst-Site-Prefix: 10.9.0.0/16 DSCP: ef [46] Traffic class id:4

TC Learned: 23:00:00 ago

Present State: CONTROLLED

Current Performance Status: in-policy

Current Service Provider: mplsB since 00:00:28 (hold until 61 sec)

Previous Service Provider: mplsA for 82741 sec

(A fallback provider. Primary provider will be re-evaluated 00:02:32 later)

BW Used: 4 Kbps

Present WAN interface: FastEthernet0/0.220 in Border 10.3.0.31

Present Channel (primary): 8

Backup Channel: 7

Destination Site ID: 10.9.0.93

Class-Sequence in use: 10

Class Name: VOICE using policy voice

BW Updated: 00:00:00 ago

Reason for Route Change: Loss

--------------------------------------------------------------------------------


A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A


10.1.2.0/24

HQ-W1

HQ-CORE2

BR-W2

BR-W1

2% Packet Loss

• Introduction




• MPLS Dual Carrier

• MPLS + Internet

Agenda


DUAL WAN (MPLS + Internet)PE-CE Protocol: BGP, Tunnel Protocol: EIGRP

• Headquarters WAN Edge

• W1 learns Branch route via eBGP

• W2 learns Branch route via EIGRP

• Headquarters Core

• W1 redistributes eBGP into EIGRP, results in EIGRP external

• W2 does not require redistribution, results in EIGRP internal

• Core1, Core2 install Branch route via W2

HQ-W2#show ip route

D 10.1.2.0/24 [90/26882560] via 10.0.1.2, 00:00:04, Tunnel1

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A

10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

192.168.101.8/29BR-W1

Internet

VPN Tunnel

EIGRP

EIG

RP

HS

RP

10.0.1.0/29

HQ-W1#show ip route

B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01


D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:02:32, Vlan10

HQ to Branch Traffic Flows

Across Tunnel



• Single Router Branch WAN Edge

• W1 learns HQ route via eBGP and EIGRP Internal

• eBGP Administrative Distance preferred A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A

10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

192.168.101.8/29

Internet

EIGRP

10.0.1.0/29

VPN Tunnel

BR-W1#show ip route

B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58

B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06Branch to HQ Traffic Flows

Across MPLS

BR-W1



• Dual Router Branch WAN Edge

• W1 learns HQ route via eBGP

• W2 learns HQ route via EIGRP

• No redistribution configured

• HSRP Primary is on W1

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A

10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

192.168.101.8/29

BR-W1

Internet

VPN Tunnel

EIGRP

EIG

RP

HS

RP

10.0.1.0/29

BR-W2#show ip route

D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 00:10:56, Tunnel1

D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 00:10:57, Tunnel1

BR-W1#show ip route

B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58

B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06



|


Fa0/1 1 110 P Active local 10.1.2.220 10.1.2.1

Branch to HQ Traffic Flows

Across MPLS

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 101


• How to force HQ to Branch traffic across MPLS (primary)?

• Adjust administrative distance• For EIGRP routes learned via tunnel• Ensure distance is higher than that of EIGRP external

(170)

HQ-W2#show ip route

D EX 10.1.2.0/24 [170/261120] via 10.1.1.110, 00:07:25, GigE0/0

HQ-W1#show ip route

B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01Now:

HQ to Branch Traffic Flows Across

MPLS

HQ-W2#

router eigrp 65110

network 10.0.1.0 0.0.0.7

distance 195 10.0.1.0 0.0.0.7


D EX 10.1.2.0/24 [170/258816] via 10.1.1.110, 00:08:44, Vlan10

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A

10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

192.168.101.8/29

BR-W1

Internet

VPN Tunnel

EIGRP

EIG

RP

HS

RP

10.0.1.0/29

HQ-W2#

Router eigrp 65100

network 10.0.1.0 0.0.0.7

router eigrp 65110

redistribute eigrp 65100

‒ Redistribute between two EIGRP Processes Forcing External as done between BGP and Campus EIGRP

Requires additional changes

Only change is on hub

or Proper Pre-Planning


DUAL WAN (MPLS + Internet)MPLS Failure

• Failure within MPLS cloud

• Dependent on provider

• Worst Case

• Link up neighbor down

• Primary dependency BGP timers

• End to end convergence time as long as BGP Holdtime

• Configuration options

• BFD would be much faster

• Application Restoration as fast as PfRdetects

HQ-W2#show ip route

D 10.1.2.0/24 [195/26882560] via 10.0.1.2, 00:06:46, Tunnel1

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A

10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

192.168.101.8/29

BR-W1

Internet

VPN Tunnel

EIGRP

EIG

RP

HS

RP

10.0.1.0/29


D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:09:18, Vlan10

After Failure:

HQ to Branch Traffic Flows Across

Tunnel

HQ Route Tables


DUAL WAN (MPLS + Internet)MPLS Failure

• Failure within MPLS cloud

• Suboptimal routing at Branch

• HSRP primary remains unchanged at BR-W1

• Use EOT and move HSRP primary to BR-W2

After Failure:

Branch to HQ Traffic Flows

Across Tunnel

BR-W1#show ip route

D 10.100.100.0/24

[90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1

D 10.100.200.0/24

[90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1

BR-W2#show ip route

D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 01:08:44, Tunnel1

D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 01:08:45, Tunnel1

Branch Route Tables

A-R4

HQ-CORE1

HQ-W2

A-R1

10.100.0.0/16

MPLS - SP A

10.1.2.0/24

EIGRP BGP BGP

HQ-W1

HQ-CORE2

10.1.1.0/24

BR-W2

192.168.101.8/29

BR-W1

Internet

VPN Tunnel

EIGRP

EIG

RP

HS

RP

10.0.1.0/29

• Introduction




• Final Wrap Up

• Key Takeaways

Agenda


Key Takeaways

• Outages can manifest in many different ways. Network design should be based on application requirements to survive various outages.

• Cisco IOS has inherent load sharing capabilities. Analyse your network topology and use these to your advantage.

• End-to-end convergence time is a critical metric. Understand how localized topology changes affect end-to-end resiliency.

• Multiple links/paths not only increase network reliability but can improve application performance.

105BRKRST-2042


Key Takeaways

• IP SLA based monitoring can detect outage types that are virtually undetectable by traditional “hello based” techniques.

• Performance Routing permits path selection based on current real time characteristics.

• Performance Routing permits full utilization of available bandwidth

• Most effective network designs incorporate a combination of convergence techniques

• Cisco SD-WAN is utilizing these features, while simplifying deployment and management, and increasing application availability.

106BRKRST-2042


Cisco Spark

Questions? Use Cisco Spark to communicate with David after the session





How



• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Complete Your Online Session Evaluation

http://ciscolive.com/Online

http://www.ciscolive.com/global/on-demand-library/


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Related sessions

109BRKRST-2042

Thank you

Documents

Highly Available Wide Area Network Design · •Introduction • Cisco IOS and IP Routing • Convergence Techniques • Design and Deployment • Final Wrap Up Agenda