108
1 © 2000, Cisco Systems, Inc. Networkers 2001, Australia March 28-30, Brisbane Networkers 2001, Australia March 28-30, Brisbane

Troubleshooting BGP Net Workers, 2001)

Embed Size (px)

Citation preview

Page 1: Troubleshooting BGP Net Workers, 2001)

1© 2000, Cisco Systems, Inc.

Networkers 2001, Australia

March 28-30, Brisbane

Networkers 2001, Australia

March 28-30, Brisbane

Page 2: Troubleshooting BGP Net Workers, 2001)

2© 2000, Cisco Systems, Inc.

Troubleshooting BGPTroubleshooting BGP

Phil SmithPhil Smith

2Presentation_ID © 2001, Cisco Systems, Inc.

Page 3: Troubleshooting BGP Net Workers, 2001)

3© 2000, Cisco Systems, Inc.

BGP in Large Scale NetworksBGP in Large Scale Networks

StableStable

SimpleSimple

ScalableScalable

Page 4: Troubleshooting BGP Net Workers, 2001)

4© 2000, Cisco Systems, Inc.

Avoid the Problem in the First Place

Avoid the Problem in the First Place

• Use simple configurationsmaintain a consistent policy throughout the AS

• Promote stable networksnail-down your routesuse loopback interfaces

• Grow into your networkuse peer-groups and RRs for scalability

Page 5: Troubleshooting BGP Net Workers, 2001)

5© 2000, Cisco Systems, Inc.

AgendaAgenda

• Basic Tools

• Peer Establishment

• UPDATE Exchange

• Selection Algorithm

• Route Reflectors

• Route Flap Damping

Page 6: Troubleshooting BGP Net Workers, 2001)

6© 2000, Cisco Systems, Inc. 6© 2000, Cisco Systems, Inc.

22101351_06_2000_c2

Tool TimeTool Time

Basic ToolsBasic Tools

Page 7: Troubleshooting BGP Net Workers, 2001)

7© 2000, Cisco Systems, Inc.

BGP Troubleshooting ToolsBGP Troubleshooting Tools

• show commands

• debug output

• Log messages

Page 8: Troubleshooting BGP Net Workers, 2001)

8© 2000, Cisco Systems, Inc.

show Commandsshow Commands

router#show ip bgp ?A.B.C.D IP prefix <network>/<length>, e.g., 35.0.0.0/8A.B.C.D Network in the BGP routing table to displaycidr-only Display only routes with non-natural netmaskscommunity Display routes matching the communitiescommunity-list Display routes matching the community-listdampened-paths Display paths suppressed due to dampeningfilter-list Display routes conforming to the filter-listflap-statistics Display flap statistics of routesinconsistent-as Display only routes with inconsistent origin ASsneighbors Detailed information on TCP and BGP neighbor connectionspaths Path informationpeer-group Display information on peer-groupsquote-regexp Display routes matching the AS path "regular expression"regexp Display routes matching the AS path regular expressionsummary Summary of BGP neighbor status| Output modifiers<cr>

Page 9: Troubleshooting BGP Net Workers, 2001)

9© 2000, Cisco Systems, Inc.

show Commands (Cont.)show Commands (Cont.)

router#show ip bgp neighbors x.x.x.x ?advertised-routes Display the routes advertised to a BGP neighbordampened-routes Display the dampened routes received from neighborflap-statistics Display flap statistics of the routes learned from

neighborpaths Display AS paths learned from neighborreceived Display information received from a BGP neighborreceived-routes Display the received routes from neighborroutes Display routes learned from neighbor| Output modifiers<cr>

Page 10: Troubleshooting BGP Net Workers, 2001)

10© 2000, Cisco Systems, Inc.

router#show ip bgp

BGP table version is 9, local router ID is 7.72.6.1Status codes: s suppressed, d damped, h history, * valid, > best, i - internalOrigin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path*> 3.0.0.0 0.0.0.0 0 32768 i*> 5.0.0.0 0.0.0.0 0 32768 i*> 6.0.0.0 6.72.6.2 4294967294 0 200 i* i 6.72.6.2 4294967294 100 0 200 i*> 7.0.0.0 0.0.0.0 0 32768 i*> 8.0.0.0/5 0.0.0.0 0 32768 i*> 17.0.0.0 6.72.6.2 4294967294 0 200 i* i 6.72.6.2 4294967294 100 0 200 i*> 23.0.0.0 6.72.6.2 4294967294 0 200 i* i 6.72.6.2 4294967294 100 0 200 i*> 35.0.0.0 6.72.6.2 4294967294 0 200 i* i 6.72.6.2 4294967294 100 0 200 i

The BGP TableThe BGP Table

Page 11: Troubleshooting BGP Net Workers, 2001)

11© 2000, Cisco Systems, Inc.

The BGP Table (Cont.)The BGP Table (Cont.)

router#show ip bgp 6.0.0.0 BGP routing table entry for 6.0.0.0/8, version 2Paths: (2 available, best #1)Advertised to non peer-group peers:

7.25.14.4 7.72.6.3 7.75.7.1 200

6.72.6.2 from 6.72.6.2 (7.72.6.2)Origin IGP, metric 4294967294, localpref 100, valid, external, best

2006.72.6.2 from 7.75.7.1 (7.75.7.1)Origin IGP, metric 4294967294, localpref 100, valid, internal

Page 12: Troubleshooting BGP Net Workers, 2001)

12© 2000, Cisco Systems, Inc.

show ip bgp Summaryshow ip bgp Summary

router#show ip bgp summary BGP router identifier 7.72.6.1, local AS number 100BGP table version is 9, main routing table version 98 network entries and 12 paths using 1176 bytes of memory3 BGP path attribute entries using 144 bytes of memory1 BGP AS-PATH entries using 24 bytes of memoryBGP activity 8/0 prefixes, 12/0 paths

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd6.72.6.2 4 200 6885 6882 9 0 0 4d18h 47.25.14.4 4 300 6882 6883 9 0 0 4d18h 07.72.6.3 4 100 6880 6886 9 0 0 4d18h 07.75.7.1 4 100 6884 6885 9 0 0 4d18h 4

Page 13: Troubleshooting BGP Net Workers, 2001)

13© 2000, Cisco Systems, Inc.

show ip bgp neighborsshow ip bgp neighborsrouter#show ip bgp neighbors 6.72.6.2BGP neighbor is 6.72.6.2, remote AS 200, external linkIndex 1, Offset 0, Mask 0x2BGP version 4, remote router ID 7.72.6.2BGP state = Established, table version = 9, up for 4d21hLast read 00:00:56, last send 00:00:48Hold time 180, keepalive interval 60 secondsNeighbor NLRI negotiation:

Configured for unicast routes onlyPeer negotiated unicast and multicast routesExchanging unicast routes only

Received route refresh capability(old) from peerMinimum time between advertisement runs is 30 secondsReceived 7044 messages, 0 notifications, 0 in queueSent 7041 messages, 0 notifications, 0 in queuePrefix advertised 4, suppressed 0, withdrawn 0Route refresh request: received 0, sent 0Inbound path policy configuredIncoming update prefix filter list is in-filterOutgoing update prefix filter list is out-filterRoute map for incoming advertisements is ebgp-router-inRoute map for outgoing advertisements is ebgp-router-out

Connections established 1; dropped 0Last reset neverNumber of unicast/multicast prefixes received 4/0External BGP neighbor may be up to 255 hops away.

Connection state is ESTAB, I/O status: 1, unread input bytes: 0Local host: 3.72.6.1, Local port: 179Foreign host: 6.72.6.2, Foreign port: 11014

Page 14: Troubleshooting BGP Net Workers, 2001)

14© 2000, Cisco Systems, Inc.

router#debug ip bgp ? A.B.C.D BGP neighbor addressdampening BGP dampeningevents BGP eventskeepalives BGP keepalivesupdates BGP updates<cr>

debug ip bgpdebug ip bgp

• Remember—can be dangerous!Use only in the lab or if advised by the TAC

• To make a little safer:logging buffered <size>

no logging console

Page 15: Troubleshooting BGP Net Workers, 2001)

15© 2000, Cisco Systems, Inc.

Session Establishment (debug ip bgp )

Session Establishment (debug ip bgp )

16:06:30: BGP: 7.72.6.1 sending OPEN, version 416:06:31: BGP: 7.72.6.1 OPEN rcvd, version 416:06:31: BGP: 7.72.6.1 rcv OPEN w/ OPTION parameter len: 1216:06:31: BGP: 7.72.6.1 rcv OPEN w/ option parameter type 2 (Capability) len 616:06:31: BGP: 7.72.6.1 OPEN has CAPABILITY code: 1, length 416:06:31: BGP: 7.72.6.1 OPEN has MP_EXT CAP for afi/safi: 1/116:06:31: BGP: 7.72.6.1 rcv OPEN w/ option parameter type 2 (Capability) len 216:06:31: BGP: 7.72.6.1 OPEN has CAPABILITY code: 128, length 0

16:06:31: BGP: 7.75.7.1 passive open16:06:31: BGP: 7.75.7.1 OPEN rcvd, version 416:06:31: BGP: 7.75.7.1 sending OPEN, version 416:06:31: BGP: 7.75.7.1 rcv OPEN w/ OPTION parameter len: 1216:06:31: BGP: 7.75.7.1 rcv OPEN w/ option parameter type 2 (Capability) len 616:06:31: BGP: 7.75.7.1 OPEN has CAPABILITY code: 1, length 416:06:31: BGP: 7.75.7.1 OPEN has MP_EXT CAP for afi/safi: 1/116:06:31: BGP: 7.75.7.1 rcv OPEN w/ option parameter type 2 (Capability) len 216:06:31: BGP: 7.75.7.1 OPEN has CAPABILITY code: 128, length 0

Page 16: Troubleshooting BGP Net Workers, 2001)

16© 2000, Cisco Systems, Inc.

Session Establishment(debug ip bgp events)

Session Establishment(debug ip bgp events)

17:31:39: BGP: 7.72.6.1 went from Idle to Active17:32:00: BGP: 7.72.6.1 went from Active to OpenSent17:32:00: BGP: 7.72.6.1 went from OpenSent to OpenConfirm17:32:00: BGP: 7.72.6.1 went from OpenConfirm to Established

17:31:59: BGP: 7.75.7.1 went from Idle to Active17:32:00: BGP: 7.75.7.1 went from Active to Idle17:32:00: BGP: 7.75.7.1 went from Idle to Connect17:32:00: BGP: 7.75.7.1 went from Connect to OpenSent17:32:00: BGP: 7.75.7.1 went from OpenSent to OpenConfirm17:32:00: BGP: 7.75.7.1 went from OpenConfirm to Established

Page 17: Troubleshooting BGP Net Workers, 2001)

17© 2000, Cisco Systems, Inc.

Looking at the UpdatesLooking at the Updates

router#debug ip bgp updates?<1-199> Access list<1300-2699> Access list (expanded range)<cr>

router#debug ip bgp x.x.x.x updates?<1-199> Access list<1300-2699> Access list (expanded range)<cr>

Use an access-list to limit the output!Use an access-list to limit the output!

Page 18: Troubleshooting BGP Net Workers, 2001)

18© 2000, Cisco Systems, Inc.

debug ip bgp Updatesdebug ip bgp Updates

BGP: 6.72.6.2 computing updates, neighbor version 0, table version 13, starting at 0.0.0.0

BGP: 6.72.6.2 send UPDATE 3.0.0.0/8, next 3.72.6.1BGP: , metric 0, path 100BGP: 6.72.6.2 send UPDATE 5.0.0.0/8 (chgflags: 0x0), next 3.72.6.1BGP: 6.72.6.2 send UPDATE 7.0.0.0/8 (chgflags: 0x0), next 3.72.6.1BGP: 6.72.6.2 1 updates enqueued (average=56, maximum=56)BGP: 6.72.6.2 update run completed, ran for 0ms, neighbor version 0,

start version 13, throttled to 13, check point net 0.0.0.0

Peer Address Prefix Being Advertised NEXT_HOP

Page 19: Troubleshooting BGP Net Workers, 2001)

19© 2000, Cisco Systems, Inc.

debug ip bgp Updates (Cont.)debug ip bgp Updates (Cont.)

BGP: 6.72.6.2 rcv UPDATE w/ attr: nexthop 6.72.6.2, origin i, metric294, path 200 100

BGP: 6.72.6.2 rcv UPDATE about 3.0.0.0/8 -- DENIED due to: as-pathcontains our own AS;

BGP: 6.72.6.2 rcv UPDATE about 7.0.0.0/8 -- DENIED due to: as-pathcontains our own AS;

BGP: 6.72.6.2 rcv UPDATE w/ attr: nexthop 6.72.6.2, origin i, metric494, path 200

BGP: 6.72.6.2 rcv UPDATE about 6.0.0.0/8BGP: 6.72.6.2 rcv UPDATE about 17.0.0.0/8BGP: 6.72.6.2 rcv UPDATE about 23.0.0.0/8BGP: 6.72.6.2 rcv UPDATE about 35.0.0.0/8

Prefixes in the Same UPDATE

Attributes Apply to

All Prefixes

Peer Address

Page 20: Troubleshooting BGP Net Workers, 2001)

20© 2000, Cisco Systems, Inc.

Logging Neighbor Changes

• Generate a log message whenever a BGP neighbor changes state, also indicate reason for reset

• Syntax (router subcommand):[no] bgp log-neighbor-changes

Typical log messages:%BGP-5-ADJCHANGE: neighbor x.x.x.x Up

%BGP-5-ADJCHANGE: neighbor x.x.x.x Down-Remote AS changed

Page 21: Troubleshooting BGP Net Workers, 2001)

21© 2000, Cisco Systems, Inc.

show ip bgp neighbors x.x.x.xshow ip bgp neighbors x.x.x.x

router#show ip bgp neighbors 7.75.7.1BGP neighbor is 7.75.7.1, remote AS 200, external link...Received 194 messages, 1 notifications, 0 in queue

Sent 194 messages, 0 notifications, 0 in queuePrefix advertised 0, suppressed 0, withdrawn 0Route refresh request: received 0, sent 0Connections established 7; dropped 7Last reset 00:04:11, due to BGP Notification received, hold time expiredNumber of unicast/multicast prefixes received 0/0External BGP neighbor may be up to 255 hops away.No active TCP connection

Page 22: Troubleshooting BGP Net Workers, 2001)

22© 2000, Cisco Systems, Inc.

Come Meet the Neighbors!Come Meet the Neighbors!

Peer EstablishmentPeer Establishment

22© 2000, Cisco Systems, Inc.

22101351_06_2000_c2

Page 23: Troubleshooting BGP Net Workers, 2001)

23© 2000, Cisco Systems, Inc.

Peer EstablishmentPeer Establishment

• Routers establish a TCP session

Port 179—permit in ACLs

IP connectivity (route from IGP)

• OPEN messages are exchanged

Peering addresses must match the TCP session

Local AS configuration parameters

Capabilities negotiation

Page 24: Troubleshooting BGP Net Workers, 2001)

24© 2000, Cisco Systems, Inc.

Common ProblemsCommon Problems

• Sessions are not established

No IP reachability

Incorrect configuration

Peering addresses

OPEN parameters

Page 25: Troubleshooting BGP Net Workers, 2001)

25© 2000, Cisco Systems, Inc.

Can’t Establish Session -Symptoms

Can’t Establish Session -Symptoms

• The peering session is not established!

State may change between active, idle and connect

routerA#show ip bgp summary BGP router identifier 7.72.6.1, local AS number 100BGP table version is 4, main routing table version 46 network entries and 6 paths using 774 bytes of memory2 BGP path attribute entries using 96 bytes of memory1 BGP AS-PATH entries using 24 bytes of memoryBGP activity 6/0 prefixes, 6/0 paths

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd6.72.6.2 4 200 0 0 0 0 0 never Idle7.25.14.4 4 300 4 5 4 0 0 00:01:43 07.72.6.3 4 100 0 0 0 0 0 never Active7.75.7.1 4 100 7 5 4 0 0 00:01:55 3

Page 26: Troubleshooting BGP Net Workers, 2001)

26© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting I

Can’t Establish Session—Troubleshooting I

• Is the remote-as assigned correctly?

router bgp 100neighbor 6.72.6.2 remote-as 200neighbor 7.72.6.3 remote-as 100

Local AS eBGP Peer

iBGP Peer

Page 27: Troubleshooting BGP Net Workers, 2001)

27© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting I (Cont.)Can’t Establish Session—Troubleshooting I (Cont.)

• Verify IP connectivitycheck the routing tableuse ping/trace to verify two way reachabilityinspect for ACLs in the path to the neighbor

routerA#show ip route 7.72.6.3Routing entry for 7.72.6.3/32

Known via "ospf 123”, distance 110, metric 87, type intra areaLast update from 27.27.27.254 on POS5/0, 00:09:33 agoRouting Descriptor Blocks:* 27.27.27.254, from 7.72.6.3, 00:09:33 ago, via POS5/0

Route metric is 87, traffic share count is 1

routerA#ping 7.72.6.3Sending 5, 100-byte ICMP Echos to 7.72.6.3, timeout is 2 seconds:!!!!!Success rate is 100 percent (5/5), round-trip min/avg/max = 28/30/32 ms

Page 28: Troubleshooting BGP Net Workers, 2001)

28© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting I (Cont.)Can’t Establish Session—Troubleshooting I (Cont.)

routerA#debug ip bgpBGP debugging is on10:51:02: BGP: 7.72.6.3 open active, delay 6864ms10:51:09: BGP: 7.72.6.3 open active, local address 27.27.27.25310:51:09: BGP: 7.72.6.3 open failed: Connection refused by remote host

• Is the remote router configured for BGP?What IP address is the remote router configured to receive?

router bgp 100no synchronizationbgp log-neighbor-changesneighbor 7.72.6.1 remote-as 100

Page 29: Troubleshooting BGP Net Workers, 2001)

29© 2000, Cisco Systems, Inc.

The TCP session is alwaysThe TCP session is alwayssourced from thesourced from the closestclosest IP IP address to the destination!address to the destination!

Can’t Establish Session—Troubleshooting I (Cont.)Can’t Establish Session—Troubleshooting I (Cont.)

• Configuration:Router A

router bgp 100neighbor 27.27.27.254 remote-as 100

Router C

router bgp 100neighbor 27.27.27.253 remote-as 100

A C

27.27.27.254

27.27.27.253

If redundant paths exist, If redundant paths exist, useuse loopback interfacesloopback interfaces to to

establish the session.establish the session.

Page 30: Troubleshooting BGP Net Workers, 2001)

30© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting I (Cont.)Can’t Establish Session—Troubleshooting I (Cont.)

• Solution:

make sure both routers source the information from the appropriate interface

routerA#debug ip tcp transactions11:19:48: BGP: 7.72.6.3 open active, delay 9916ms11:19:53: TCP: sending RST, seq 0, ack 309812912111:19:53: TCP: sent RST to 7.7.7.6:11719 from 7.72.6.1:179

router bgp 100neighbor 7.72.6.3 remote-as 100neighbor 7.72.6.3 update-source Loopback0

Information sourcedInformation sourcedfrom the IP address in from the IP address in interface Loopback0interface Loopback0

Page 31: Troubleshooting BGP Net Workers, 2001)

31© 2000, Cisco Systems, Inc.

Can’t Establish Session—Symptoms

Can’t Establish Session—Symptoms

• The eBGP session is still having trouble!

routerA#show ip bgp summary BGP router identifier 7.72.6.1, local AS number 100BGP table version is 4, main routing table version 46 network entries and 6 paths using 774 bytes of memory2 BGP path attribute entries using 96 bytes of memory1 BGP AS-PATH entries using 24 bytes of memoryBGP activity 6/0 prefixes, 6/0 paths

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd6.72.6.2 4 200 0 0 0 0 0 never Idle7.25.14.4 4 300 385 385 4 0 0 06:22:17 07.72.6.3 4 100 42 49 4 0 0 00:00:15 07.75.7.1 4 100 388 385 4 0 0 06:22:30 3

Page 32: Troubleshooting BGP Net Workers, 2001)

32© 2000, Cisco Systems, Inc.

routerA#configure terminal Enter configuration commands, one per line. End with CNTL/Z.routerA(config)#ip route 6.72.6.2 255.255.255.255 1.1.1.5

routerA#ping 6.72.6.2Type escape sequence to abort.Sending 5, 100-byte ICMP Echos to 6.72.6.2, timeout is 2 seconds:!!!!!Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

Can’t Establish Session -Troubleshooting II

Can’t Establish Session -Troubleshooting II

• Verify IP connectivitycheck the routing tableuse ping/trace to verify two way reachability

routerA#show ip route 6.72.6.2%Network not in table

Page 33: Troubleshooting BGP Net Workers, 2001)

33© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting II (Cont.)Can’t Establish Session—Troubleshooting II (Cont.)

• Peering with a loopback interface

Advantages

Interface is always up

Multiple physical paths may exist to reach it

Disadvantages

Physical link failure may take longer to detect

Page 34: Troubleshooting BGP Net Workers, 2001)

34© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting II (Cont.)Can’t Establish Session—Troubleshooting II (Cont.)

• The debug output indicates the neighbour’sconfigured peering address

routerA#debug ip bgprouterA#debug ip tcp transactions13:25:30: TCP: sending RST, seq 0, ack 203010066913:25:30: TCP: sent RST to 6.72.6.2:11041 from 3.72.6.1:179

Neighbour is tryingNeighbour is tryingto peer with this IPto peer with this IP

addressaddress

router bgp 100neighbor 6.72.6.2 remote-as 200neighbor 6.72.6.2 update-source Loopback1

Page 35: Troubleshooting BGP Net Workers, 2001)

35© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting II (Cont.)Can’t Establish Session—Troubleshooting II (Cont.)

• Hint: by default, eBGP peers should be directly connected

in this case, the peering address doesn’t match a connected interface in the local router

13:33:30: TCP: sending RST, seq 0, ack 251012964513:33:30: TCP: sent RST to 6.72.6.2:11045 from 3.72.6.1:179

Page 36: Troubleshooting BGP Net Workers, 2001)

36© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting II (Cont.)Can’t Establish Session—Troubleshooting II (Cont.)

routerA#show ip bgp neighbors 6.72.6.2BGP neighbor is 6.72.6.2, remote AS 200, external linkIndex 1, Offset 0, Mask 0x2BGP version 4, remote router ID 0.0.0.0BGP state = Idle, table version = 0Last read 00:00:06, last send neverHold time 180, keepalive interval 60 secondsNeighbor NLRI negotiation:Configured for unicast routes only

Minimum time between advertisement runs is 30 secondsReceived 0 messages, 0 notifications, 0 in queueSent 0 messages, 0 notifications, 0 in queuePrefix advertised 0, suppressed 0, withdrawn 0Route refresh request: received 0, sent 0Connections established 0; dropped 0Last reset neverNumber of unicast/multicast prefixes received 0/0External BGP neighbor not directly connected.No active TCP connection

Page 37: Troubleshooting BGP Net Workers, 2001)

37© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting II (Cont.)Can’t Establish Session—Troubleshooting II (Cont.)

• At this point, the session should come up

router bgp 100neighbor 6.72.6.2 remote-as 200neighbor 6.72.6.2 ebgp-multihop 255neighbor 6.72.6.2 update-source Loopback1

Page 38: Troubleshooting BGP Net Workers, 2001)

38© 2000, Cisco Systems, Inc.

Can’t Establish Session—Symptoms

Can’t Establish Session—Symptoms

• Still having trouble!

Connectivity issues have already been checked and corrected.

routerA#show ip bgp summary BGP router identifier 7.72.6.1, local AS number 100… Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd6.72.6.2 4 200 10 26 0 0 0 never Active

router bgp 100neighbor 6.72.6.2 remote-as 200neighbor 6.72.6.2 ebgp-multihop 255neighbor 6.72.6.2 update-source Loopback1

Page 39: Troubleshooting BGP Net Workers, 2001)

39© 2000, Cisco Systems, Inc.

Can’t Establish Session—Troubleshooting II (Cont.)Can’t Establish Session—Troubleshooting II (Cont.)

• If an error is detected, a notification is sent and the session is closed

In this case the remote router had a bad configuration

14:06:37: BGP: 6.72.6.2 open active, local address 3.72.6.114:06:37: BGP: 6.72.6.2 went from Active to OpenSent14:06:37: BGP: 6.72.6.2 sending OPEN, version 414:06:37: BGP: 6.72.6.2 received NOTIFICATION 2/2

(peer in wrong AS) 2 bytes 000114:06:37: BGP: 6.72.6.2 remote close, state CLOSEWAIT14:06:37: BGP: service reset requests14:06:37: BGP: 6.72.6.2 went from OpenSent to Idle14:06:37: BGP: 6.72.6.2 closing

Page 40: Troubleshooting BGP Net Workers, 2001)

40© 2000, Cisco Systems, Inc.

OPEN MessageOPEN Message

Optional ParametersOptional Parameters

BGP IdentifierBGP Identifier

Opt. Parm. Len.Opt. Parm. Len.

Hold TimeHold Time

My Autonomous SystemMy Autonomous System

VersionVersion

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Page 41: Troubleshooting BGP Net Workers, 2001)

41© 2000, Cisco Systems, Inc.

RFC2842 – May2000(Proposed Standard)

Capability Code (1 Octet)Capability Code (1 Octet)

Capability Length (1 Octet)Capability Length (1 Octet)

Capability Value (Variable)Capability Value (Variable)

Capabilities NegotiationCapabilities Negotiation

• Allows for the advertisement of capabilities (type 2)

• Backwards compatible

New error subcode introduced to indicate which capabilities arenot supported—thesession must be reset

Page 42: Troubleshooting BGP Net Workers, 2001)

42© 2000, Cisco Systems, Inc.

Where’s the Beef?Where’s the Beef?

UPDATE ExchangeUPDATE Exchange

42© 2000, Cisco Systems, Inc.

22101351_06_2000_c2

Page 43: Troubleshooting BGP Net Workers, 2001)

43© 2000, Cisco Systems, Inc.

UPDATE ExchangeUPDATE Exchange

• Once the session has been established, UPDATEs are exchanged

all the locally known routes

only the bestpath is advertised

• Incremental UPDATE messages are exchanged afterwards

Page 44: Troubleshooting BGP Net Workers, 2001)

44© 2000, Cisco Systems, Inc.

Propagation DecisionsPropagation Decisions

• bestpath received from eBGP peer

advertise to all peers

• bestpath received from iBGP peer

advertise only to eBGP peers

a full iBGP mesh must exist

Page 45: Troubleshooting BGP Net Workers, 2001)

45© 2000, Cisco Systems, Inc.

Common ProblemsCommon Problems

• Missing routes

No iBGP full mesh

Filters: routes are not received/sent

• Slow convergence

Page 46: Troubleshooting BGP Net Workers, 2001)

46© 2000, Cisco Systems, Inc.

UPDATE FiltersUPDATE Filters

• Type of filters

Prefix filters

AS_PATH filters

Community filters

Any attribute may be used in a route-map

• Applied incoming and/or outgoing

Page 47: Troubleshooting BGP Net Workers, 2001)

47© 2000, Cisco Systems, Inc.

Missing Routes—Troubleshooting Steps

Missing Routes—Troubleshooting Steps

• Determine which filters are applied to the BGP session

show ip bgp neighbors x.x.x.x

Look at the configuration

• Examine the route and pick out the relevant attributes

show ip bgp x.x.x.x

Page 48: Troubleshooting BGP Net Workers, 2001)

48© 2000, Cisco Systems, Inc.

Missing Routes—Troubleshooting Steps (Cont.)

Missing Routes—Troubleshooting Steps (Cont.)

• Compare the route against the filters

• If no match is found

Use route-refresh or soft-reconfiguration

Filter the updates through an ACL to determine where the problem is

Page 49: Troubleshooting BGP Net Workers, 2001)

49© 2000, Cisco Systems, Inc.

Missing Routes—SymptomsMissing Routes—Symptoms

• Missing 4.0.0.0/8 in 7.75.7.1 (routerA)

not received from 7.72.6.3 (routerB)

routerB#sh ip bgp nei 7.75.7.1 advertised-routes | include 4.0.0.0*> 4.0.0.0 0.0.0.0 0 32768 i

routerB shows that the route was advertised to routerA!

Page 50: Troubleshooting BGP Net Workers, 2001)

50© 2000, Cisco Systems, Inc.

Missing Routes—TroubleshootingMissing Routes—Troubleshooting

routerA#show access-lists 10Standard IP access list 10

permit 4.0.0.0

routerA#debug ip bgp 7.72.6.3 updates 10BGP updates debugging is on for access list 10 for neighbor 7.72.6.3

routerA#clear ip bgp 7.72.6.3 in01:22:41: BGP: 7.72.6.3 rcv UPDATE w/ attr: nexthop 7.72.6.3, origin i, metric 0, path 20001:22:41: BGP: 7.72.6.3 rcv UPDATE about 4.0.0.0/8 -- DENIED due

to: distribute/prefix-list;

Page 51: Troubleshooting BGP Net Workers, 2001)

51© 2000, Cisco Systems, Inc.

Missing Routes—Troubleshooting (Cont.)

Missing Routes—Troubleshooting (Cont.)

router bgp 100no synchronizationbgp log-neighbor-changesneighbor 7.72.6.3 remote-as 200neighbor 7.72.6.3 ebgp-multihop 255neighbor 7.72.6.3 update-source Loopback0neighbor 7.72.6.3 prefix-list filter in!ip prefix-list filter seq 5 deny 4.0.0.0/8ip prefix-list filter seq 10 permit 0.0.0.0/0 le 32

Page 52: Troubleshooting BGP Net Workers, 2001)

52© 2000, Cisco Systems, Inc.

Slow Convergence—SymptomsSlow Convergence—Symptoms

• The eBGP peering is established, but convergence is not complete even after several hours

• Possible causesRemote router is not healthy (OutQ)

Lower layer problems (IP)

routerA#show ip bgp summary...Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd150.10.10.1 4 100 3550 3570 847 0 206 05:53:51 100

Page 53: Troubleshooting BGP Net Workers, 2001)

53© 2000, Cisco Systems, Inc.

Slow Convergence—Troubleshooting

Slow Convergence—Troubleshooting

routerA#show ip route 150.10.10.1

Routing entry for 150.10.10.1/32

Routing Descriptor Blocks:

10.105.1.71, from 150.20.20.1, 00:06:14 ago, via POS2/1/0

* 156.1.1.1, from 150.20.20.1, 00:06:14 ago, via POS2/1/1

routerA#ping 150.10.10.1

Sending 5, 100-byte ICMP Echos to 150.10.10.1: !!!!!

Success is 100 percent, round-trip min/avg/max = 4/64/296 ms

Reply to request 0Record route:

(156.1.1.2)(195.5.5.1)(10.105.1.134)(150.10.10.1)(10.105.1.76)(195.5.5.2)(156.1.1.1)(211.211.211.1) <*>

Reply to request 1Record route:

(10.105.1.69)(140.10.50.5)(150.10.10.1)(140.10.50.6)(10.105.1.71)(211.211.211.1) <*>

router bgp 100neighbor 150.10.10.1 remote-as 200neighbor 150.10.10.1 ebgp-multihop 2neighbor 150.10.10.1 update-source Loopback0

Ping with route Ping with route record option.record option.

Page 54: Troubleshooting BGP Net Workers, 2001)

54© 2000, Cisco Systems, Inc.

Slow Convergence—Troubleshooting (Cont.)

Slow Convergence—Troubleshooting (Cont.)

eBGP Peering

OC-3 OC-3 OC-3

OC-3 OC-3

T3

A B

router bgp 100neighbor 150.10.10.1 remote-as 200neighbor 150.10.10.1 ebgp-multihop 2neighbor 150.10.10.1 update-source Loopback0

Longest path has more Longest path has more than 2 hops to the than 2 hops to the

destination. Use higher destination. Use higher TTL!TTL!

Page 55: Troubleshooting BGP Net Workers, 2001)

55© 2000, Cisco Systems, Inc. 55© 2000, Cisco Systems, Inc.

22101351_06_2000_c2

Pick One, Only One!Pick One, Only One!

Route Selection ProcessRoute Selection Process

Page 56: Troubleshooting BGP Net Workers, 2001)

56© 2000, Cisco Systems, Inc.

Route Selection ProcessRoute Selection Process

• A common policy should be maintained across the AS to guarantee loop-free operation

Not all routers may select the same path

• Filters may be used to modify or add attributes, affecting the selection algorithm

Page 57: Troubleshooting BGP Net Workers, 2001)

57© 2000, Cisco Systems, Inc.

Common ProblemsCommon Problems

• Inconsistent decision/policy

MED

External paths

Communities

By default, communities are not propagated

neighbor x.x.x.x send-community

Page 58: Troubleshooting BGP Net Workers, 2001)

58© 2000, Cisco Systems, Inc.

Inconsistent Decision—Symptom I

Inconsistent Decision—Symptom I

• The bestpath changes every time the peering is reset.

routerA#sh ip bgp 160.100.0.0BGP routing table entry for 160.100.0.0/16, version 40Paths: (3 available, best #3, advertised over IBGP, EBGP)100

204.146.33.10 from 204.146.33.10 (204.146.33.1)Origin IGP, metric 0, localpref 100, valid, internal

300204.146.33.66 from 204.146.33.66 (204.146.33.2)Origin IGP, metric 20, localpref 100, valid, internal

300204.146.33.6 from 204.146.33.6 (10.4.1.1)Origin IGP, metric 30, valid, external, best

Page 59: Troubleshooting BGP Net Workers, 2001)

59© 2000, Cisco Systems, Inc.

Inconsistent Decision—Symptom I (Cont.)

Inconsistent Decision—Symptom I (Cont.)

routerA#sh ip bgp 160.100.0.0BGP routing table entry for 160.100.0.0/16, version 2Paths: (3 available, best #3, advertised over EBGP)

100204.146.33.10 from 204.146.33.10 (204.146.33.1)

Origin IGP, metric 0, localpref 100, valid, internal300204.146.33.6 from 204.146.33.6 (10.4.1.1)

Origin IGP, metric 30, valid, external300204.146.33.66 from 204.146.33.66 (204.146.33.2)

Origin IGP, metric 20, localpref 100, valid, internal, best

• Same paths, but different result!

Page 60: Troubleshooting BGP Net Workers, 2001)

60© 2000, Cisco Systems, Inc.

Inconsistent Decision—Symptom I (Cont.)

Inconsistent Decision—Symptom I (Cont.)

• Different result…again!!

routerA#sh ip bgp 160.100.0.0BGP routing table entry for 160.100.0.0/16, version 12Paths: (3 available, best #3, advertised over EBGP)

300204.146.33.6 from 204.146.33.6 (10.4.1.1)

Origin IGP, metric 30, valid, external300204.146.33.66 from 204.146.33.66 (204.146.33.2)

Origin IGP, metric 20, localpref 100, valid, internal100204.146.33.10 from 204.146.33.10 (204.146.33.1)

Origin IGP, metric 0, localpref 100, valid, internal, best

Page 61: Troubleshooting BGP Net Workers, 2001)

61© 2000, Cisco Systems, Inc.

Deterministic MEDDeterministic MED

• By default, the prefixes are compared in order of arrival

it may result in inconsistent decisions

use bgp deterministic-med

the bestpath is recalculated as soon as the command is entered

enable in all the routers in the AS

Page 62: Troubleshooting BGP Net Workers, 2001)

62© 2000, Cisco Systems, Inc.

Deterministic MED—OperationDeterministic MED—Operation

• The paths are ordered by peer-AS

• The bestpath for each group is selected

• The overall bestpath results from comparing the winners in each group

Page 63: Troubleshooting BGP Net Workers, 2001)

63© 2000, Cisco Systems, Inc.

Deterministic MED—ResultDeterministic MED—Result

• The bestpath will always be the same!

routerA#sh ip bgp 160.100.0.0BGP routing table entry for 160.100.0.0/16, version 15Paths: (3 available, best #1, advertised over EBGP)

100204.146.33.10 from 204.146.33.10 (204.146.33.1)

Origin IGP, metric 0, localpref 100, valid, internal, best300204.146.33.66 from 204.146.33.66 (204.146.33.2)

Origin IGP, metric 20, localpref 100, valid, internal300204.146.33.6 from 204.146.33.6 (10.4.1.1)

Origin IGP, metric 30, valid, external

Page 64: Troubleshooting BGP Net Workers, 2001)

64© 2000, Cisco Systems, Inc.

Inconsistent Decision—Symptom II

Inconsistent Decision—Symptom II

• The bestpath changes every time the peering is reset

routerA#show ip bgp 7.0.0.0BGP routing table entry for 7.0.0.0/8, version 15Paths: (2 available, best #2)

Not advertised to any peer2001.1.1.5 from 1.1.1.5 (1.1.1.1)

Origin IGP, metric 0, localpref 100, valid, external20021.21.21.254 from 21.21.21.254 (7.75.7.1)

Origin IGP, metric 0, localpref 100, valid, external, best

Page 65: Troubleshooting BGP Net Workers, 2001)

65© 2000, Cisco Systems, Inc.

Inconsistent Decision—Symptom II (Cont.)

Inconsistent Decision—Symptom II (Cont.)

• The “oldest” external is the bestpath.All other attributes are the sameStability enhancement!

routerA#show ip bgp 7.0.0.0 BGP routing table entry for 7.0.0.0/8, version 17Paths: (2 available, best #2)

Not advertised to any peer20021.21.21.254 from 21.21.21.254 (7.75.7.1)

Origin IGP, metric 0, localpref 100, valid, external2001.1.1.5 from 1.1.1.5 (1.1.1.1)

Origin IGP, metric 0, localpref 100, valid, external, best

Page 66: Troubleshooting BGP Net Workers, 2001)

66© 2000, Cisco Systems, Inc. 66© 2000, Cisco Systems, Inc.

22101351_06_2000_c2

Route ReflectorsRoute Reflectors

Playing with MirrorsPlaying with Mirrors

Page 67: Troubleshooting BGP Net Workers, 2001)

67© 2000, Cisco Systems, Inc.

Route ReflectorsRoute Reflectors

• Provide additional control to allowrouter to advertise (reflect) iBGPlearned routes to other iBGP peers

Method to reduce the size of the iBGP mesh

• Normal BGP speakers can coexist

Only the RR has to support this feature

Page 68: Troubleshooting BGP Net Workers, 2001)

68© 2000, Cisco Systems, Inc.

Route Reflector

Clients ClientsClusters

Non-Client

Lines Represent Both Physical Links and BGP Logical ConnectionsLines Represent Both Physical Links and BGP Logical Connections

Route Reflectors—TerminologyRoute Reflectors—Terminology

Page 69: Troubleshooting BGP Net Workers, 2001)

69© 2000, Cisco Systems, Inc.

Reflection DecisionsReflection Decisions

• Once the best path is selected:

From non-client reflect to all clients

From client → reflect to all non-clients AND other clients

From eBGP peer → reflect to all clients and non-clients

Page 70: Troubleshooting BGP Net Workers, 2001)

70© 2000, Cisco Systems, Inc.

Common ProblemsCommon Problems

• Missing routes

• Routing loops and “close calls”

Page 71: Troubleshooting BGP Net Workers, 2001)

71© 2000, Cisco Systems, Inc.

Missing Routes—SymptomsMissing Routes—Symptoms

• At least one route is missing from at least one router in the network.

routerA#show ip bgp 4.0.0.0% Network not in tablerouterA#show ip bgp summary BGP router identifier 7.25.14.4, local AS number 100BGP table version is 1, main routing table version 1… Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd7.72.6.2 4 100 7 7 1 0 0 00:04:18 0

Page 72: Troubleshooting BGP Net Workers, 2001)

72© 2000, Cisco Systems, Inc.

Missing Routes—TroubleshootingMissing Routes—Troubleshooting

• Check routers for filtersrouterA#router bgp 100no synchronizationneighbor 7.72.6.2 remote-as 100routerB#

router bgp 100no synchronizationbgp cluster-id 0.0.0.5neighbor 7.25.14.4 remote-as 100neighbor 7.25.14.4 route-reflector-clientneighbor 7.72.6.1 remote-as 100

routerB#show ip bgp 4.0.0.0% Network not in table

routerC

Page 73: Troubleshooting BGP Net Workers, 2001)

73© 2000, Cisco Systems, Inc.

Missing Routes—Troubleshooting IMissing Routes—Troubleshooting I

• Follow the path where the routes should have been learned

routerC#router bgp 100no synchronizationbgp cluster-id 0.0.0.5neighbor 7.72.6.2 remote-as 100neighbor 7.75.7.1 remote-as 100neighbor 7.75.7.1 route-reflector-client

routerC#show ip bgp | include 4.0.0.0*>i4.0.0.0 7.72.6.3 0 100 0 200 i

Same Cluster-ID!

Page 74: Troubleshooting BGP Net Workers, 2001)

74© 2000, Cisco Systems, Inc.

Missing Routes—Troubleshooting IIMissing Routes—Troubleshooting II

• Alternative way to find duplicate cluster-id

use route-refresh + debug ip bgp updates ACL

routerB#clear ip bgp 7.72.6.1 in21:45:40: BGP: 7.72.6.1 rcv UPDATE w/ attr: nexthop

7.72.6.3, origin i, localpref 100, metric 0, path 221:45:40: BGP: 7.72.6.1 rcv UPDATE about 4.0.0.0/8 --

DENIED due to: reflected from the same cluster;

Page 75: Troubleshooting BGP Net Workers, 2001)

75© 2000, Cisco Systems, Inc.

• Clusters with multiple RRs

If the RRs have the same cluster-id, all the clients must peer with all the reflectors

Lines Represent Both PhysicalLinks and BGP Logical Connections

Lines Represent Both PhysicalLinks and BGP Logical Connections

RRRR

AA

BB CC

Missing Routes—Troubleshooting (Cont.)

Missing Routes—Troubleshooting (Cont.)

DD

EEIf not needed for administration, If not needed for administration,

don’t assign a clusterdon’t assign a cluster--id for id for added flexibility!added flexibility!

Page 76: Troubleshooting BGP Net Workers, 2001)

76© 2000, Cisco Systems, Inc.

Lines RepresentPhysical Connections

Lines RepresentPhysical Connections

Routing Loop—SymptomRouting Loop—Symptom

routerD#traceroute 7.1.1.1

1 1.1.1.2 24 msec 24 msec 40 msec

2 156.1.1.1 28 msec 48 msec 24 msec

3 156.1.1.2 24 msec 24 msec 24 msec

4 156.1.1.1 28 msec 28 msec 24 msec

5 156.1.1.2 28 msec 28 msec 28 msec

6 156.1.1.1 28 msec 28 msec 32 msec

A

B C

D

rtrBrtrB

rtrCrtrC

Loop!Loop!

Page 77: Troubleshooting BGP Net Workers, 2001)

77© 2000, Cisco Systems, Inc.

Routing Loop—TroubleshootingRouting Loop—Troubleshooting

routerC#show ip bgp 7.0.0.0

BGP routing table entry for 7.0.0.0/8

1

150.10.10.1 (metric 115) from 150.10.10.1 (150.20.20.1)

Origin IGP, valid, external, best

routerC#show ip route 150.10.10.1

Routing entry for 150.10.10.1/32

Routing Descriptor Blocks:

* 156.1.1.1, from 150.20.20.1, via Ethernet2/1/1

routerB#show ip bgp 7.0.0.0

BGP routing table entry for 7.0.0.0/8

1

156.1.1.2 from 156.1.1.2 (212.212.212.1)

Origin IGP, valid, internal, best

routerB#show ip route 156.1.1.2

Routing entry for 156.1.1.0/24

Routing Descriptor Blocks:

* directly connected, via Ethernet1

Verify Routing Information

Addresses on Same Subnet!

Page 78: Troubleshooting BGP Net Workers, 2001)

78© 2000, Cisco Systems, Inc.

Routing Loop—Troubleshooting (Cont.)

Routing Loop—Troubleshooting (Cont.)

• Check configurationrouterC#router bgp 134neighbor 150.10.10.1 remote-as 1neighbor 150.10.10.1 ebgp-multihop 255neighbor 150.10.10.1 update-source Loopback0neighbor 156.1.1.1 remote-as 134neighbor 156.1.1.1 route-reflector-clientneighbor 156.1.1.1 next-hop-self

!ip route 150.10.10.1 255.255.255.255 s0 250

A-RR

B C-RR

D

Lines RepresentPhysical Connections

Lines RepresentPhysical Connections

Page 79: Troubleshooting BGP Net Workers, 2001)

79© 2000, Cisco Systems, Inc.

Routing Loop—Troubleshooting (Cont.)

Routing Loop—Troubleshooting (Cont.)

• Solution

Establish the eBGP peering permanently through the“backup” link

Use LOCAL_PREF or MED to break any tie!

A-RR

B C-RR

D

Lines RepresentPhysical Connections

Lines RepresentPhysical Connections

Page 80: Troubleshooting BGP Net Workers, 2001)

80© 2000, Cisco Systems, Inc.

Close Call—SymptomClose Call—Symptom

• The bestpath is not being followed to an external destination

routerA#show ip route 4.4.4.4Routing entry for 4.0.0.0/8Known via "bgp 1", distance 200, metric 0Tag 2, type internalLast update from 6.72.6.3 00:25:45 agoRouting Descriptor Blocks:* 6.72.6.3, from 7.75.7.1, 00:25:45 ago

Route metric is 0, traffic share count is 1AS Hops 1, BGP network version 0

Expected to go out though the NEXT_HOP in the update.

Page 81: Troubleshooting BGP Net Workers, 2001)

81© 2000, Cisco Systems, Inc.

Close Call—Symptom (Cont.)Close Call—Symptom (Cont.)

• All eBGP peers are configured with their interface address

The NEXT_HOP is expected in the trace

routerA#traceroute 4.4.4.4

1 1.1.1.4 0 msec 0 msec 0 msec2 8.25.14.3 8 msec 4 msec 4 msec3 172.18.176.1 8 msec 4 msec 4 msec4 161.44.0.56 8 msec 8 msec 8 msec5 161.44.0.18 8 msec 8 msec 4 msec 6 4.4.4.4 4 msec 5 msec 4 msec

Page 82: Troubleshooting BGP Net Workers, 2001)

82© 2000, Cisco Systems, Inc.

Close Call—Troubleshooting (Cont.)

Close Call—Troubleshooting (Cont.)

• Verify configuration

Check for alternate routesrouter bgp 100no synchronizationneighbor 7.75.7.1 remote-as 100neighbor 7.75.7.1 update-source Loopback0

routerA#show ip bgp 4.0.0.0BGP routing table entry for 4.0.0.0/8, version 2Paths: (1 available, best #1)

Not advertised to any peer200

6.72.6.3 (metric 103) from 7.75.7.1 (7.75.7.1)Origin IGP, metric 0, localpref 100, valid, internal, best

routerC

Page 83: Troubleshooting BGP Net Workers, 2001)

83© 2000, Cisco Systems, Inc.

Close Call—Troubleshooting (Cont.)

Close Call—Troubleshooting (Cont.)

• routerC is a RR with two clients

router bgp 100no synchronizationbgp log-neighbor-changesneighbor 7.72.6.1 remote-as 100neighbor 7.72.6.1 update-source Loopback0neighbor 7.72.6.1 route-reflector-clientneighbor 7.72.6.2 remote-as 100neighbor 7.72.6.2 update-source Loopback0neighbor 7.72.6.2 route-reflector-clientneighbor 6.72.6.3 remote-as 200

routerA

routerB

Page 84: Troubleshooting BGP Net Workers, 2001)

84© 2000, Cisco Systems, Inc.

Close Call—Troubleshooting (Cont.)

Close Call—Troubleshooting (Cont.)

routerC#show ip bgp 4.0.0.0BGP routing table entry for 4.0.0.0/8, version 2

200, (Received from a RR-client)8.25.14.3 (metric 3) from 7.72.6.1 (7.72.6.1)

Origin IGP, metric 0, localpref 100, valid, internal200

6.72.6.3 (metric 2) from 6.72.6.3 (6.72.6.3)Origin IGP, metric 0, localpref 100, valid, external, best

routerC#traceroute 4.4.4.4

1 6.72.6.3 0 msec 4 msec 0 msec2 161.44.0.56 0 msec 4 msec 0 msec3 161.44.0.18 0 msec 4 msec 0 msec4 4.4.4.4 0 msec 4 msec 0 msec

Expected Path!

Page 85: Troubleshooting BGP Net Workers, 2001)

85© 2000, Cisco Systems, Inc.

Close Call—Troubleshooting (Cont.)

Close Call—Troubleshooting (Cont.)

• Output from the other client

follows the same exit as routerA

routerB#traceroute 4.4.4.4

1 8.25.14.3 8 msec 16 msec 8 msec2 172.18.176.1 16 msec 12 msec 16 msec3 161.44.0.48 12 msec 16 msec 12 msec4 161.44.0.15 16 msec 12 msec 16 msec5 4.4.4.4 8 msec 8 msec 8 msec

Page 86: Troubleshooting BGP Net Workers, 2001)

86© 2000, Cisco Systems, Inc.

Close Call—Troubleshooting (Cont.)

Close Call—Troubleshooting (Cont.)

• routerB is following the correct path!

routerB#show ip bgp 4.0.0.0BGP routing table entry for 4.0.0.0/8, version 132008.25.14.3 (metric 2) from 8.25.14.3 (8.25.14.3)

Origin IGP, metric 0, localpref 100, valid, external, best2006.72.6.3 (metric 50) from 7.75.7.1 (7.75.7.1)

Origin IGP, metric 0, localpref 100, valid, internal

Page 87: Troubleshooting BGP Net Workers, 2001)

87© 2000, Cisco Systems, Inc.

Close Call—Troubleshooting (Cont.)

Close Call—Troubleshooting (Cont.)

• The logical connection between routerC (RR) and routerA provides the route

The physical path is followed

EEDD

CC

BB

AA

CC

AA

Logical ConnectionLogical Connection4.0.0.0/8

Page 88: Troubleshooting BGP Net Workers, 2001)

88© 2000, Cisco Systems, Inc.

Route Flap DampingRoute Flap Damping

Stabilising the NetworkStabilising the Network

88© 2001, Cisco Systems, Inc. www.cisco.com22101351_06_2000_c2

Page 89: Troubleshooting BGP Net Workers, 2001)

89© 2000, Cisco Systems, Inc.

Route Flap Damping

• Route flapGoing up and down of path or change in attribute

BGP WITHDRAW followed by UPDATE = 1 flap

eBGP neighbour going down/up is NOT a flap

Ripples through the entire Internet

Wastes CPU

• Damping aims to reduce scope of route flap propagation

• Implementation documented in RFC2439

Page 90: Troubleshooting BGP Net Workers, 2001)

90© 2000, Cisco Systems, Inc.

Route Flap DampingOperation

Route Flap DampingOperation

• Add penalty (1000) for each flap

• Exponentially decay penaltyhalf life determines decay rate

• Penalty above suppress-limitdo not advertise route to BGP peers

• Penalty decayed below reuse-limitre-advertise route to BGP peers

Page 91: Troubleshooting BGP Net Workers, 2001)

91© 2000, Cisco Systems, Inc.

Route Flap DampingRoute Flap Damping

Reuse limit

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

0

1000

2000

3000

4000

Time

Penalty

Suppress limit

NetworkAnnounced

NetworkRe-announced

NetworkNot Announced

Page 92: Troubleshooting BGP Net Workers, 2001)

92© 2000, Cisco Systems, Inc.

Route Flap DampingOperation

• Only applied to inbound announcements from eBGP peers

• Alternate paths still usable

• Controlled by:Half-life (default 15 minutes)

reuse-limit (default 750)

suppress-limit (default 2000)

maximum suppress time (default 60 minutes)

Page 93: Troubleshooting BGP Net Workers, 2001)

93© 2000, Cisco Systems, Inc.

Route Flap DampingConfiguration

Fixed dampingrouter bgp 100bgp dampening [<half-life> <reuse-value> <suppress-penalty> <maximum suppress time>]

Selective and variable dampingbgp dampening [route-map <name>]route-map <name> permit 10match ip address prefix-list FLAP-LISTset dampening [<half-life> <reuse-value> <suppress-penalty> <maximum suppress time>]

ip prefix-list FLAP-LIST permit 192.0.2.0/24 le 32

Page 94: Troubleshooting BGP Net Workers, 2001)

94© 2000, Cisco Systems, Inc.

Route Flap DampingConfiguration

Route Flap DampingConfiguration

• Care required when setting parameters

• Penalty must be less than reuse-limit at the maximum suppress time

• Maximum suppress time and half life must allow penalty to be larger than suppress limit

Page 95: Troubleshooting BGP Net Workers, 2001)

95© 2000, Cisco Systems, Inc.

Route Flap DampingConfiguration

Route Flap DampingConfiguration

• Examples - ûbgp dampening 30 750 3000 60

reuse-limit of 750 means maximum possible penalty is 3000 - no prefixes suppressed as penalty cannot exceed suppress-limit

• Examples - übgp dampening 30 2000 3000 60

reuse-limit of 2000 means maximum possible penalty is 8000 - suppress limit is easily reached

Page 96: Troubleshooting BGP Net Workers, 2001)

96© 2000, Cisco Systems, Inc.

Route Flap DampingConfiguration

Route Flap DampingConfiguration

• Examples - ûbgp dampening 15 500 2500 30

reuse-limit of 500 means maximum possible penalty is 2000 - no prefixes suppressed as penalty cannot exceed suppress-limit

• Examples - übgp dampening 15 750 3000 45

reuse-limit of 750 means maximum possible penalty is 6000 - suppress limit is easily reached

Page 97: Troubleshooting BGP Net Workers, 2001)

97© 2000, Cisco Systems, Inc.

• Maximum possible value of penalty is

• Always make sure that suppress-limit chosen is LESS than max-penalty otherwise there will be no route damping

Route Flap DampingMaths!

Route Flap DampingMaths!

Page 98: Troubleshooting BGP Net Workers, 2001)

98© 2000, Cisco Systems, Inc.

Route Flap DampingStatus

Route Flap DampingStatus

gw>sh ip bgp flap-statistics

BGP table version is 8444024, local router ID is 202.12.29.64

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

Network From Flaps Duration Reuse Path

*> 63.74.112.0/22 203.62.252.21 3 00:18:08 1221 16779 1 701 13593

*> 63.74.120.0/22 203.62.252.21 3 00:18:08 1221 16779 1 701 13593

*d 63.109.97.0/24 203.62.252.21 12 00:42:07 00:28:20 1221 16779 1 701 17106

*> 63.140.98.0/24 203.62.252.21 4 00:29:48 1221 5696 15322

*> 64.254.102.0/24 203.62.252.21 2 00:03:11 1221 16779 1 701 17307

h 66.0.0.0 203.62.252.21 2 00:14:43 1221 16779 1833 701 1239 4231

*> 66.9.38.0/23 203.62.252.21 3 00:21:22 1221 16779 1 3356 16449

*d 66.40.0.0/18 203.62.252.21 11 00:35:00 00:26:30 1221 11388

*d 139.78.0.0 203.62.252.21 7 00:10:53 00:30:50 1221 5727 1 2828 7911 7294 5078

*d 147.39.200.0/24 203.62.252.21 7 00:41:02 00:17:10 1221 5727 7018 701 702 1913 1559

h 192.31.83.0 203.62.252.21 3 00:05:11 1221 16779 1 2828 7911 7294 5078

...

Flapping prefixes

Page 99: Troubleshooting BGP Net Workers, 2001)

99© 2000, Cisco Systems, Inc.

Route Flap DampingStatus

Route Flap DampingStatus

gw>sh ip bgp dampened-paths

BGP table version is 8443716, local router ID is 202.12.29.64

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

Network From Reuse Path

*d 202.8.238.0 203.62.252.21 00:04:10 1221 5727 9658 9386 i

*d 216.148.175.0 203.62.252.21 00:05:50 1221 5727 1740 11282 11282 i

*d 212.9.128.0/19 203.62.252.21 00:09:30 1221 16779 1833 1299 1755 8787 i

*d 200.23.202.0 203.62.252.21 00:12:20 1221 1 3561 6503 3454 3454 3454 i

*d 148.219.0.0 203.62.252.21 00:13:40 1221 1 1239 8151 278 i

*d 203.10.1.0 203.62.252.21 00:13:20 1221 2764 2764 2764 2764 9543 i

*d 206.137.4.0/22 203.62.252.21 00:14:10 1221 1 3561 6347 18435 i

*d 195.54.160.0/19 203.62.252.21 00:16:20 1221 5727 1239 1755 8525 ?

*d 192.146.206.0 203.62.252.21 00:21:50 1221 5727 1239 5078 5078 i

...

Damped prefixes

Page 100: Troubleshooting BGP Net Workers, 2001)

100© 2000, Cisco Systems, Inc.

Route Flap DampingMonitoring

Route Flap DampingMonitoring

gw#debug ip bgp damp

BGP dampening debugging is on

Mar 9 15:49:26.443 AEST: BGP: charge penalty for 204.134.66.0/24 path 1221 16779 1 1239 3912 with halflife-time 15 reuse/suppress 750/2000

Mar 9 15:49:26.443 AEST: flapped 1 times since 00:00:00. New penalty is 1000

...

Mar 9 15:49:26.667 AEST: BGP: charge penalty for 203.17.144.0/22 path 1221 5727 7018 3561 7474 7569 7570 4738 7572 7571 7476 with halflife-time 15 reuse/suppress 750/2000

Mar 9 15:49:26.667 AEST: flapped 5 times since 00:03:39. New penalty is 2766

Mar 9 15:49:26.667 AEST: BGP: suppress 203.17.144.0/22 path 1221 5727 7018 3561 7474 7569 7570 4738 7572 7571 7476 for 00:28:10 (penalty 2766)

Mar 9 15:49:26.671 AEST: halflife-time 15, reuse/suppress 750/2000

Monitoring route flaps

Page 101: Troubleshooting BGP Net Workers, 2001)

101© 2000, Cisco Systems, Inc.

Route Flap DampingMonitoring

Route Flap DampingMonitoring

Mar 9 16:00:32.021 AEST: BGP: unsuppressed 203.255.232.0/24, path 1221 6461 3786 1237

Mar 9 16:03:38.358 AEST: BGP: unsuppressed 204.228.71.0/24, path 1221 16779 1 3908 14041 12145

Mar 9 16:03:38.358 AEST: BGP: unsuppressed 204.228.64.0/24, path 1221 16779 1 3908 14041 12145

Mar 9 16:03:38.358 AEST: BGP: unsuppressed 198.59.93.0/24, path 1221 16779 1 3908 14041 12145

Mar 9 16:03:38.358 AEST: BGP: unsuppressed 198.59.87.0/24, path 1221 16779 1 3908 14041 12145

Mar 9 16:04:10.570 AEST: BGP: unsuppressed 207.40.160.0/20, path 1221 16779 1 1239 1791

Mar 9 16:04:10.570 AEST: BGP: unsuppressed 207.40.64.0/20, path 1221 16779 1 1239 1791

Mar 9 16:04:20.994 AEST: BGP: unsuppressed 203.91.145.0/24, path 1221 16779 1 701 703 9237

Mar 9 16:04:51.914 AEST: BGP: unsuppressed 203.168.64.0/20, path 1221 1 3561 4755 17632

Monitoring route flaps (more)

Page 102: Troubleshooting BGP Net Workers, 2001)

102© 2000, Cisco Systems, Inc.

Route Flap DampingOperations

Route Flap DampingOperations

• Be aware:

Many ISPs use flap damping

Unreachable remote networks may be due to damping, not disconnections

Damping applies to flapping prefixes, not unstable connections

Page 103: Troubleshooting BGP Net Workers, 2001)

103© 2000, Cisco Systems, Inc.

22101351_06_2000_c2

SummarySummary

103© 1999, Cisco Systems, Inc. www.cisco.com

Page 104: Troubleshooting BGP Net Workers, 2001)

104© 2000, Cisco Systems, Inc.

Summary/TipsSummary/Tips

• Isolate the problem!

• Use ACLs when enabling debug commands

• Enable bgp log-neighbor-changes

• IP reachability must exist for sessions to be established

Learned from IGP

Make sure the source and destination addresses match the configuration

Page 105: Troubleshooting BGP Net Workers, 2001)

105© 2000, Cisco Systems, Inc.

Summary/TipsSummary/Tips

• Use loopback interfaces for stability and where multiple paths exist

• Use common filters

Keep them simple!

• Maintain a consistent policy throughout the AS

• Use deterministic-med

Page 106: Troubleshooting BGP Net Workers, 2001)

106© 2000, Cisco Systems, Inc.

Summary/TipsSummary/Tips

• Select the appropriate knob/attribute for the job

Learn the decision algorithm

• Route reflectorsFollow the physical topology

Define a cluster-id only if administratively needed

• BGP DampingUseful for enhancing stability of Internet connected networks

Page 107: Troubleshooting BGP Net Workers, 2001)

107© 2000, Cisco Systems, Inc.

Please Complete Your Evaluation Form

Please Complete Your Evaluation Form

Troubleshooting BGPTroubleshooting BGP

107© 1999, Cisco Systems, Inc.

22101351_06_2000_c2

Page 108: Troubleshooting BGP Net Workers, 2001)

108© 2000, Cisco Systems, Inc.

22101351_06_2000_c2