Preface - Experience Issue212(Data Products).pdf · Preface Maintenance Experience Editorial Committee Maintenance Experience Newsroom Address: ZTE Plaza, No.55, Hi-tech Road South,

PrefaceMaintenance ExperienceEditorial Committee

Maintenance ExperienceNewsroom

Address: ZTE Plaza, No.55, Hi-tech Road

South, Shenzhen, P.R.China

Postal code: 518057

Contact: Ning Jiating

Email: [email protected]

Tel: +86-755-26771195

Fax: +86-755-26772236

Document support mail box: [email protected]

Technical support website: http://ensupport.zte.

com.cn

Maintenance Experience Editorial CommitteeZTE CorporationApril, 2010

In this issue of ZTE's Maintenance Experience, we continue to pass on various field reports and resolutions that are gathered by ZTE engineers and technicians around the world.

The contents presented in this issue are as below:● Four Special Documents● Eleven Maintenance Cases of ZTE's Data ProductsHave you examined your service polices and procedures

lately? Are you confident that your people are using all the tools at their disposal? Are they trained to analyze each issue in a logical manner that provides for less downtime and maximum customer service? A close look at the cases reveals how to isolate suspected faulty or mis-configured equipment, and how to solve a problem step by step, etc. As success in commissioning and service is usually a mix of both discovery and analysis, we consider using this type of approach as an example of successful troubleshooting investigations.

While corporate leaders maintain and grow plans for expansion, ZTE employees in all regions carry out with individual efforts towards internationalization of the company. Momentum continues to be built, in all levels, from office interns to veteran engineers, who work together to bring global focus into their daily work.

If you would like to subscribe to this magazine (electronic version) or review additional articles and relevant technical materials concerning ZTE products, please visit the technical support website of ZTE CORPORATION (http://ensupport.zte.com.cn).

If you have any ideas and suggestions or want to offer your contributions, you can contact us at any time via the following email: [email protected].

Thank you for making ZTE a part of your telecom experience!

Maintenance ExperienceBimonthly for Data ProductsNo.10 Issue 212，April 2010

Director: Qiu Weizhao

Deputy Director: Chen Jianzhou

Editors:Jiang Guobing, Zhang Shoukui, Wu Feng,

Yuan Yufeng, Tang Hongxuan, Li Gangyi,

Song Jianbo,Tian Jinhua, Wang Zhaozheng,

Liu Wenjun, Wang Yapping, Lei Kun, Ge Jun,

Wang Tiancheng, Yu Qing, Zhang Jiebin,

Fang Xi

Technical Senior Editors:Hu Jia, Tao Minjuan, Zhang Jianping

Executive Editor:Zhang Fan

Contents

Analysis on Common BGP Faults ...............................................................................................................2Handling of BGP Routing Protocol Fault .....................................................................................................3BGP Fault Caused by Poor-Quality Link .....................................................................................................6BGP Route Filter .........................................................................................................................................8Method to Advertise NAT Address Pool to EBGP Neighbor ........................................................................9Router Interruption Handling on ZXR10 T128 ...........................................................................................11IBGP Neighbor Relationship Establishment Failure ..................................................................................13Route Advertisement on BGP Route Reflector ..........................................................................................16BGP Route Learning Failure .....................................................................................................................20BGP Community Attribute Application .......................................................................................................24OSPF and BGP Route Oscillation .............................................................................................................27Advertisement Failure in BGP Routing Policy ...........................................................................................30Importing BGP Route Statically .................................................................................................................32Brief Discussion of BGP Load Sharing ......................................................................................................37BGP Performance Optimization on Large-Scale Network .........................................................................38

Maintenance Experience2

April 2010 Issue 212

Analysis on Common BGP Faults⊙ Li Wei / ZTE Corporation

1 Situations of Common BGP FaultThere are a lot of situations related

to BGP faults. The situations of three

common and typical faults are described

below.

Situation one: Neighbor relationship

cannot be established (that is, a device

cannot be in Established state).

Situation two: BGP routes do not

come to the routing table. That is, when

users input show ip route command,

there are no routes learnt through BGP,

or the routes are contained in the routing

table sometimes and not contained in the

routing table sometimes.

Situation three: When users input

show ip bgp command, the result shows

Key words: BGP, fault handling

that there is a route, but the route fails to be

advertised to the neighbor.

2 Fault Analysis2.1 Analysis for Situation One

(1) Check whether the configuration of AS

number is correct and valid.

(2) Check whether the IP address of the

neighbor is correct.

(3) If the loopback interface is used to establish

neighbor relationship, check whether update-source loopback command is conf igured.

By default, a router uses its optimal interface

instead of a loopback interface to establish TCP

connections.

(4) If the neighbor is an EBGP neighbor that

is not direct-connected physically, check whether

EBGP multi-hop is configured.

www.zte.com.cn

3Data Products

(5) Use ping command to check whether the

TCP connection is proper. As a router may have

several interfaces to reach the peer, it is required

to use extended ping to designate the source IP

address of ping packets.

(6) If the peer cannot be pinged through, use

show ip route command to check whether there

is a reachable route to the neighbor in the routing

table.

(7) If the peer can be pinged through, check

whether the ACL that is used to deny TCP Port 179

is configured. If this ACL is configured, modify the

configuration to admit TCP Port 179.

(8) Check the Router ID to see whether it

conflicts with that of the router with which the router

will establish neighbor relationship.

2.2 Analysis for Situation Two(1) Use show ip bgp command to check

whether the route is the optimal route.

(2) If the peer is an IBGP neighbor, check

whether the synchronization function is

disabled.

(3) Check whether the next hop of the

route is reachable.

( 4 )Check whe the r t he rou te i s

suppressed due to damping.

2.3 Analysis for Situation Three(1) Check whether the route is the

optimal route.

( 2 )Check whe the r t he rou te i s

aggregated.

(3) Check whether the route is covered

by network command.

(4) If the route is learnt from an IBGP

neighbor, and the route is intended to

advertise to another IBGP neighbor, it

is normal when a fault of Situation three

occurs.

(5) Check whether route advertisement

filter is configured. ■

Handling of BGP Routing Protocol Fault⊙ Li Minghui / ZTE Corporation

Key words: BGP, neighbor, Router ID, loopback address

Malfunction SituationA ZXUAS 10800E was used on a network.

When the office personnel checked the routes on

the uplink device CRS-001, they found that there

was a fault about the BGP routes and neighbor of

ZXUAS 10800E. The result was different from that

of corresponding commands on BAS devices of

other vendors.

It was necessary to check whether such a

situation indicated a fault and whether it would

affect services.

Malfunction AnalysisEng ineers checked the ZXUAS

10800E and there was nothing improper.

When engineers conducted service tests

on site, the services ran properly.

Engineers checked BGP neighbor

information and route information on

ZXUAS 10800E. They found that the

result was al l r ight. However, there

were problems about the BGP neighbor

information and route information on the



CRS-001#show bgp vpnv4 unicast nei 192.168.191.238

BGP neighbor is 192.168.191.238

Remote AS 65130, local AS 65130, internal link

Remote router ID 192.168.191.238

BGP state = Established, up for 1d06h

Last read 00:00:25, hold time is 90, keepalive interval is 30

seconds

Precedence: internet

Neighbor capabilities:

Route refresh: advertised and received

4-byte AS: advertised

Address family IPv4 Unicast: advertised and received

Address family VPNv4 Unicast: advertised and received

Received 297330 messages, 3 notifications, 0 in queue

Sent 1986915 messages, 3 notifications, 0 in queue

Minimum time between advertisement runs is 0 seconds

For Address Family: IPv4 Unicast

BGP neighbor version 33939827

Update group: 0.2

Route refresh request: received 0, sent 0

11 accepted prefixes, 11 are bestpaths

Prefix advertised 1934, suppressed 0, withdrawn 230, maximum limit

524288

Threshold for warning message 75%

An EoR was not received during read-only mode

……

The corresponding configuration on ZXUAS 10800E is shown below.

ZTE10800E#show bgp vpnv4 unicast neighbor 192.168.232.236

BGP neighbor is 192.168.232.236, remote AS 65130, internal link

Member of peer-group ha-jz-vpn for session parameters

BGP version 4, remote router ID 192.168.255.236

BGP state = Established, up for 1w5d

Last read update 00:01:39, hold time is 90 seconds, keepalive

interval is 30 seconds

Neighbor capabilities:

Route refresh: advertised and received

Address family VPNv4 Unicast: advertised and received

All received 82201 messages

34992 updates, 0 errs

uplink CRS-001 and NE5000E. Take CRS-001 as an example, the information is shown below.

www.zte.com.cn

5Data Products

2 opens, 0 errs

47207 keepalives

0 vpnv4 refreshs, 0 ipv4 refreshs, 0 ipv4 multicast refreshs, 0

ipv6 refreshs, 0 errs

0 notifications, 0 other errs

After last established received 72958 messages

30506 updates, 0 errs

0 opens, 0 errs

42452 keepalives

0 vpnv4 refreshs, 0 ipv4 refreshs, 0 ipv4 multicast refreshs, 0

ipv6 refreshs, 0 errs

0 notifications, 0 other errs

All sent 39031 messages

7 updates, 8 opens, 39015 keepalives

……

Engineers checked the upl ink device of

ZXUAS 10800E to view BGP neighbor and

route information. They found that no matter it

was an IPv4 neighbor or a VPNv4 neighbor, the

Router ID of ZXUAS 10800E was the address of

Loopback10, that is, the address used by VPNv4

services instead of the interconnected Loopback1

address.

SolutionAt last, engineers found that Router ID was

not configured on ZXUAS 10800E. When

the router ID is not configured, ZXUAS

10800E uses the smallest loopback

address on the device as its Router ID.

Therefore, when engineers checked

BGP neighbors, Router ID displayed on the

corresponding IPv4 neighbors and VPNv4

neighbors was the address of Loopback10,

which was different from other devices.

Engineers configured Router ID on the

device. The problem was solved. ■



BGP Fault Caused by Poor-Quality Link⊙ Li Changqing / ZTE Corporation

Key words: BGP, link quality, packet time-out, BGP neighbor

Malfunction SituationA ZXR10 T128 router connected to

another router. Both routers ran BGP. The

ZXR10 T128 worked properly for a time.

Recently, BGP alarms appeared on the device

continually. The alarm information is shown below.

An alarm 17946 level 6 occurred at 08:05:37 08/24/2008 GMT+8 sent by

UPC(RPU) 1 %BGP% Neighbor 10.70.128.78 recv bogus route : AS loop

An alarm 17961 level 6 occurred at 08:05:37 08/24/2008 GMT+8 sent

www.zte.com.cn

7Data Products

by UPC(RPU) 1 %BGP% 49w5d: BGP: CTVPN-1038 10.70.128.78 went from

OpenConfirm to Established



OpenSent to OpenConfirm


UPC(RPU) %BGP% 49w5d: BGP: CTVPN-1038 10.70.128.78 went from Connect

to OpenSent


UPC(RPU) 1 %BGP% 49w5d: BGP: CTVPN-1038 10.70.128.78 went from Idle

to Connect



Established to Idle


UPC(RPU) 1 %BGP% 49w5d: BGP: CTVPN-1038 10.70.128.78 reset due to

BGP Notification received


UPC(RPU) 1 %BGP% 49w5d: BGP: CTVPN-1038 10.70.128.78 NOTIFY rcvd

……

Malfunction Analysis(1) According to the alarm information, the

ZXR10 T128 received BGP NOTIFY packets.

Therefore, i t re-establ ished BGP neighbor

relationship.

(2) Engineers input show ip bgp neighbor command to check the detailed BGP neighbor

information. They found the information “last error

code is 6”. Engineers checked the code and found

that it was a notification Cease packet. According

to the requirement of BGP protocol, the router

would re-establish BGP neighbor relationship after

receiving this packet. Therefore, engineers needed

to check why the peer device sent the Cease

packet.

(3) Engineers checked alarm information

and BGP information on the peer device. They

found that there were alarms about BGP

keepalive packets time-out. The two

routers were connected directly. Engineers

conducted link tests. The results showed

that there were some packets lost.

This meant that the link quality was not

good enough, which caused loss of BGP

keepalive packets during receiving and

sending.

SolutionEngineers changed the link. After the

problem of link quality was solved, there

was no BGP alarm on the router. BGP

neighbor relationship could be established

properly and services ran properly. ■



BGP Route Filter⊙ Tang Tan / ZTE Corporation

Key words: BGP, ACL, route filter, wildcard-mask, prefix-list

Malfunction DescriptionAs shown in Figure 1, ZXR10 T1200

and Cisco 7609 connected to each other

through an EBGP connection. ZXR10

T1200 advertised segments 10.42.16.0/20

and 10.42.16.0/24 to Cisco 7609. A route

map was configured on ZXR10 T1200 to

control the segment 10.42.16.0/24 not to

be advertised to Cisco 7609.

*Nov 12 17:37:39.087: BGP: Applying map to find origin for

10.42.16.0/20

*Nov 12 17:37:39.087: BGP: Applying map to find origin for

10.42.16.0/24

Figure 1. Network Topology

After the configuration, users found that Cisco

7609 could not learn the route of 10.42.16.0/20.

Malfunction AnalysisTo solve the fault, engineers took the following

steps.

(1) After engineers deleted the configuration

related to the route map on ZXR10 T1200,

Cisco 7609 could learn the corresponding route.

Therefore, it was certain that there was a problem

about the configuration of the route map.

(2) Engineers checked debugging information

on ZXR10 T1200. They found that the route map

took effect on the segments 10.42.16.0/20 and

10.42.16.0/24, as shown below.

Engineers checked configuration related to the route map on ZXR10 T1200, as shown below.

acl standard number 10

rule 1 deny 10.42.16.0 0.0.0.255

rule 2 permit any

route-map deny permit 10

match ip address access-list 10

……

( 3 ) E n g i n e e r s f o u n d t h a t t h e

address segment matching acl 10 was

10.42.16.0/24 in the configuration. ACLs

use the wildcard-masks to match addresses. A

wildcard-mask is similar to a subnet mask, but

they are different in writing. For a wildcard-mask,

www.zte.com.cn

9Data Products

ip prefix-list deny seq 10 permit 10.42.16.0 24

route-map deny-JCX_TG deny 10

match ip address prefix-list deny

route-map deny-JCX_TG permit 20

Engineers applied the route map again. Cisco 7609 could receive routes properly.

SummaryWhen ACLs are used in configuration for accurate matching, it is required to configure the

wildcard-masks carefully. This can prevent a lot of addresses from matching the ACLs. ■

0 means the comparison is required and 1 means

the comparison can be ignored.

Segments 10.42.16.0/24 and 10.42.16.0/20 matched

the acl 10, therefore 10.42.16.0/20 was denied.

SolutionEngineers used a pref ic - l is t for

a c c u r a t e m a t c h i n g . T h e d e t a i l e d

commands are shown below.

Method to Advertise NAT Address Pool to EBGP Neighbor⊙ Zhang Fan / ZTE Corporation

Key words: BGP, network, EBGP, NAT, address pool

Malfunction AnalysisNAT was configured on a ZXR10 GER router

on a network. It was required to advertise the NAT

address pool to an EBGP neighbor locating in the

segment of the address pool.

In such situation, the segment should not be

advertised through network command simply,

as network command of BGP is only used to

advertise routes that have been in routing tables.

However, the segment which NAT address pool

belonged to was not in a routing table.

SolutionEngineers configured a black hole

route directing to the loopback address

for the segment that was used as the NAT

address pool on ZXR10 GER. And then

engineers used the network command of

BGP protocol to advertise the segment.

The problem was solved.

The configuration on ZXR10 GER is

shown below.



Interface loopback1

Ip address 10.0.0.11 255.255.255.255

!

Interface fei_2/6.8

Encapsulation dot1Q 8

Ip address 192.168.85.138 255.255.255.252

Ip mtu 1400

Ip nat outside

!

Interface gei_4/1

Ip address 10.0.1.82 255.255.255.252

Negotiation auto

Ip nat inside

!

Ip nat start

Ip nat pool natpool 192.168.72.1 192.168.72.6 prefix-length 29

Ip nat inside source list 10 pool natpool overload

!

Router bgp 65002

Timers bgp 60 180

No synchronization

Network 192.168.72.0 255.255.255.0

Neighbor 192.168.85.113 remote-as 8390

Neighbor 192.168.85.113 activate

Neighbor 192.168.85.117 remote-as 8390

Neighbor 192.168.85.117 activate

!

Ip route 192.168.72.0 255.255.255.0 loopback1

/*Configure a black hole route directing to the loopback address for

the segment that is used as the NAT address pool*/

!

Ip access-list standard 10

Permit 10.1.26.0 0.0.0.63

Permit 10.0.4.32 0.0.0.31

……

www.zte.com.cn

11Data Products

Key words: BGP, load sharing, static route

Router Interruption Handling on ZXR10 T128⊙ Luo Xiang / ZTE Corporation

Malfunction SituationAn ISP used two ZXR10 T128 routes on the

Metropolitan Area Network (MAN) as the egress

routers. One night, the maintenance personnel

found that the traffic on a 2.5 G POS interface of

ZXR10 T128-1 (2.5 G POS link egress) was 0, and

all traffic that came into and went out of the MAN

went through ZXR10 T128-2 (two 155 M POS link

egresses).

On-site engineers logged into the device

through the serial port. They found that the BGP

neighbor relationship between ZXR10 T128-1 and

the uplink router GSR was established properly.

The BGP routing table was normal and

there was no alarm information.

The customer said that the MAN

subscribers could not open web pages

because the on-line rate was too low,

and some special line subscribers could

not get online. The manifestation of this

malfunction was that the port traffic of

ZXR10 T128-1 was very low, only 1 MBps.

Engineers turned off the 2.5 G POS

interface on ZXR10 T128-1 to make all

MAN traffic go through ZXR10 T128-2.

Except that the on-line rate of subscribers



was still low due to the bandwidth limit,

services recovered to run properly.

Malfunction AnalysisEngineers checked the device in the

machine room in the wee hours. They

found that when they recovered the 2.5 G

POS interface on ZXR10 T128-1, the on-

line speed was rather low. Engineers could

only ping through some addresses. For some other

addresses, they could not ping through.

First, engineers thought it was the problem of the

NPCT board. They replaced the board and upgraded

the version. However, the problem was still on.

When engineers got the route information of

the GSR device, they found that a static route from

GSR to the loopback address of ZXR10 T128-1

was configured incorrectly, as shown below.

GSR>show ip route 192.168.180.0

Routing entry for 192.168.180.0/24, 2 known subnets

Attached (1 connections)

Variably subnetted with 2 masks

C 192.168.180.40/30 is directly connected, POS11/3

S 192.168.180.250/32 [1/0] via 192.168.180.42

The incorrect route and the correct

route to ZXR10 T128-1 formed load

sharing, which caused some packets to be

forwarded to a wrong next hop. Therefore,

the on-line speed was slow. If the load

sharing was on the base of destination

policy, i t was reasonable that some

address could be pinged through while some other

could not be.

SolutionEngineers deleted the wrong static route. When

the 2.5 G POS interface recovered, services ran

properly. The fault was solved. ■

www.zte.com.cn

13Data Products

⊙ Ren Lanbing / ZTE Corporation

IBGP Neighbor Relationship Establishment Failure

Key words: BGP, loopback address, neighbor, VPN

Network TopologyThe network topology of an ISP is shown

in Figure 1. Radio Network Controller (RNC)

works in load sharing mode. It connects to the

remote Serving GPRS Support Node (SGSN)

for communication. The services between RNC

and SGSN are classified into control plane and

subscriber plane. The services are isolated and

transmitted by the bearer network.

Two sub-interfaces on RNC are enabled

to connect to the CE devices (ZXR10 T600) to

bind different VPNs. Static routes are configured

on RNC to designate the IP addresses of the

corresponding sub-interfaces on CE devices as

the next hops of VPN routes to SGSN. The IP

addresses of sub-interfaces on corresponding

boards of RNC are designated as the next hops

of VPN routes to the control plane and subscriber

plane of RNC.

CEs and ARs are interconnected in EBGP

OPTION A mode. The two CEs interconnect to

each other through sub-interfaces. Each sub-

interface uses IS-IS as IGP bearing. Multiple

loopback addresses are configured on CEs to

establish IBGP neighbor relationship. The CEs

Figure1. Network Topology

redistribute static routes to their own VPNs.

Malfunction SituationAlthough IS-IS was configured to carry

over the routes of loopback addresses (bound

to different VRFd) on CEs, CEs failed to

use loopback addresses to establish IBGP

neighbor relationship of VPN routes.

Engineers checked the brief information of BGP routes on CE-1, as shown below.CE-1#show ip bgp summary

Neighbor Ver As MsgRcvd MsgSend Up/Down(s) State/PfxRcd

10.245.173.1 4 38351 3 9 14:54:28 24

10.245.175.138 4 64873 167 161 03:18:18 Connect

10.245.173.5 4 38351 0 0 00:00:00 24

10.245.175.139 4 64873 1421 1422 03:18:18 Connect



10.245.173.1 and 10.245.173.5 were IP addresses of the direct-connected sub-interfaces on

AR-1 and CE-1. 10.245.175.138 and 10.245.175.139 were loopback addresses of CE-2.

Engineers checked the brief information of BGP routes on CE-2, as shown below.

CE-2#show ip bgp summary


10.245.173.9 4 38351 35 57 00:28:33 24

10.245.175.136 4 64873 0 0 00:00:00 Connect

10.245.173.13 4 38351 35 56 00:28:33 24

10.245.175.137 4 64873 0 0 00:00:00 Connect

10.245.173.9 and 10.245.173.13 were IP addresses of the direct-connected sub-interfaces on

AR-2 and CE-2. 10.245.175.136 and 10.245.175.137 were loopback addresses of CE-1.

Solution(1) Engineers checked the configurations on the devices. The BGP configuration on CE-1 was

shown below.

router bgp 64873

address-family ipv4 vrf VPN_IuPS_SIG

redistribute connected

redistribute static

neighbor 10.245.173.1 remote-as 38351

neighbor 10.245.173.1 activate


/*Designate the peer loopback address of CE-2 as the IBGPneighbor*/


$

address-family ipv4 vrf VPN_IuPS_UP


redistribute static




/*Designate the peer loopback address of CE-2 as the IBGPneighbor*/


$

!

Engineers found that when CE-1 established IBGP neighbor relationship with CE-2, it was

not set to use the loopback address. Engineers modified the configuration to make CE-1 use a

loopback address to establish IBGP neighbor relationship, as shown below.

www.zte.com.cn

15Data Products

CE-1(config)#router bgp 64873

CE-1(config-router)#address-family ipv4 vrf VPN_IuPS_SIG

CE-1(config-router-af)#neighbor 10.245.175.138 update-source loopback1

%Code 4043: The command is used in common mode.

The bold information indicated that the device did not support to use a loopback address to

establish IBGP neighbor relationship in VPN.

(2) Engineers asked for technical support. They were told that loopback addresses could not

be used to establish IBGP neighbor relationship through sub-interfaces on CEs. Instead, only the

addresses on the interconnected sub-interfaces could be used. This was because the devices did

not support neighbor <ip-address> update-source loopback command in IPv4 VRF address

family configuration mode.

(3) Instead, engineers used the addresses on the direct-connected sub-interfaces on CEs

to establish IBGP neighbor relationship. The IBGP neighbor relationship was established

successfully.

The configuration result on CE-1 is shown below.



10.245.173.1 4 38351 3 9 19:15:26 24

/*the sub-interface connecting to AR-1 on CE-1*/

10.245.173.66 4 64873 49 43 00:22:08 30

/*the sub-interface connecting to CE-2 on CE-1*/

10.245.173.5 4 38351 0 0 00:00:00 24


10.245.173.70 4 64873 51 44 00:22:07 30


The configuration result on CE-2 is shown below.



10.245.173.9 4 38351 18 27 00:13:03 24


10.245.173.65 4 64873 37 43 00:19:20 7


10.245.173.13 4 38351 18 27 00:12:40 24


10.245.173.69 4 64873 39 45 00:19:20 9


SummaryWhen CEs interconnect to each other through sub-interfaces, only the interconnection

addresses can be used to establish IBGP neighbor relationship. Loopback addresses could not be

used to establish IBGP neighbor relationship in such situation. ■



Key words: BGP, Route Reflector (RR), IGP route

Route Advertisement on BGP Route Reflector⊙ Fu Tao / ZTE Corporation

Malfunction SituationAs shown in Figure 1, in AS 100,

R1 and R2 established IBGP neighbor


relationship with the Route Reflector (RR) R3. R1

and R2 were the clients of R3. R2 established

EBGP neighbor relationship with R4 in AS 200.

www.zte.com.cn

17Data Products

R1 needed to advertise segments 11.11.11.11 and 10.10.10.10 to R4. R4 needed to advertise

the segment 12.12.12.12 and the loopback addresses 4.4.4.4 to R1.

OSPF was configured on R1, R2 and R3 to provide TCP connections for IBGP neighbors.

Static routes were configured on R2 and R4 to direct to the loopback address of each other.

When engineers input show ip bgp command on R4, they found that R4 learnt the two

segments advertised by R1. The two routes were optimal, and the next hop was 2.2.2.2. However,

When engineers input show ip bgp command on R1, they found that R1 did not learn the

segment 12.12.12.12 advertised by R4.

Malfunction AnalysisTo find out the reason, engineers took the following steps.

(1) Engineers input show ip bgp command on R1. The result is shown below.

Network Next Hop Metric LocPrf Weight Path

*> 1.1.1.1/32 0.0.0.0 0 32768 ?

r>i3.3.3.3/32 3.3.3.3 0 100 0 ?

*> 10.0.0.0/30 0.0.0.0 0 32768 ?

* i10.0.0.4/30 3.3.3.3 0 100 0 ?

*> 0.0.0.0 0 32768 ?

r>i10.0.0.8/30 3.3.3.3 0 100 0 ?

*> 10.10.10.10/32 0.0.0.0 0 32768 i

*> 11.11.11.11/32 0.0.0.0 0 32768 i

Routes of segments 4.4.4.4 and 12.12.12.12 were not included in the routing table. Engineers

doubted that devices in AS 100 did not receive the segments advertised by R4. Therefore,

engineers checked R2 and R3.

(2) Engineers input show ip bgp command on R2. The result is shown below.


r>i1.1.1.1/32 1.1.1.1 0 100 0 ?

r>i3.3.3.3/32 3.3.3.3 0 100 0 ?

r> 4.4.4.4/32 4.4.4.4 0 0 200 ?

r>i10.0.0.0/30 1.1.1.1 0 100 0 ?

r>i10.0.0.4/30 3.3.3.3 0 100 0 ?

r>i10.0.0.8/30 3.3.3.3 0 100 0 ?

r> 10.0.1.0/30 4.4.4.4 0 0 200 ?

*>i10.10.10.10/32 1.1.1.1 0 100 0 i

*>i11.11.11.11/32 1.1.1.1 0 100 0 i

*> 12.12.12.12/32 4.4.4.4 0 0 200 ?

The result showed that R2 learnt the segment 4.4.4.4 from the EBGP neighbor R4. The route

was with the “>” mark and without the “*” mark. This meant that the route was optimal but not legal.

This was because loopback addresses were used during EBGP neighbor establishment between

R2 and R3. To ensure that TCP link kept through, static routes were configured on R2 and R4 to



direct to the loopback address of each other. As the priorities of the static routes were higher than

that of routes learnt from an EBGP neighbor, R2 would not use the BGP route to forward packets.

The route from R2 to 12.12.12.12 was learnt through BGP route advertisement principle, and it is

legal and optimal.

(3) Engineers checked whether R2 advertised the segment 12.12.12.12 to the IBGP neighbor

R3 (RR). They input show ip bgp command on R3. The result is shown below.


r>i1.1.1.1/32 1.1.1.1 0 100 0 ?

*> 3.3.3.3/32 0.0.0.0 0 32768 ?

* i4.4.4.4/32 4.4.4.4 0 100 0 200 ?

r>i10.0.0.0/30 1.1.1.1 0 100 0 ?

* i10.0.0.4/30 1.1.1.1 0 100 0 ?

*> 0.0.0.0 0 32768 ?

*> 10.0.0.8/30 0.0.0.0 0 32768 ?

* i10.0.1.0/30 4.4.4.4 0 100 0 200 ?

*>i10.10.10.10/32 1.1.1.1 0 100 0 i

*>i11.11.11.11/32 1.1.1.1 0 100 0 i

* i12.12.12.12/32 4.4.4.4 0 100 0 200 ?

The result showed that R3 (RR) learnt the two segments on R4 in AS 200 through the IBGP

neighbor. However, the routes were unavailable, as the routes were not with the “>” mark and the “*”

mark at the same time.

(4) Engineers found that when R3 learnt the routes advertised by the IBGP neighbor R2, the

next hops of the route were 4.4.4.4. This next hop was unreachable for R3, therefore the routes

could not be used or loaded to the global routing table.

(5) As segments 4.4.4.4 and 12.12.12.12 were not learnt by R1, engineers considered that R3

(RR) would only reflect the legal and optimal routes that were learnt from the IBGP neighbor R2

to other IBGP neighbors. This was the reason why R1 could not learn the segments 4.4.4.4 and

12.12.12.12 from R3 (RR).

Solutions(1)The first method was to make the R3 (RR) learn the legal next hop route.

When R2 advertised a route to R3, it advertised its loopback address 2.2.2.2 as the next hop.

In this way, the next hop of the route was 2.2.2.2 which was reachable for R3, so the route was

legal and optimal. Then R3 (RR) would reflect the two segments to R1.

The detailed configuration was to configure neighbor 3.3.3.3 next-hop-self command on R2.

After the configuration, engineers viewed the BGP routing table on R1, as shown below.

www.zte.com.cn

19Data Products


*> 1.1.1.1/32 0.0.0.0 0 32768 ?

r>i3.3.3.3/32 3.3.3.3 0 100 0 ?

*>i4.4.4.4/32 2.2.2.2 0 100 0 200 ?

*> 10.0.0.0/30 0.0.0.0 0 32768 ?

* i10.0.0.4/30 3.3.3.3 0 100 0 ?

*> 0.0.0.0 0 32768 ?

r>i10.0.0.8/30 3.3.3.3 0 100 0 ?

*>i10.0.1.0/30 2.2.2.2 0 100 0 200 ?

*> 10.10.10.10/32 0.0.0.0 0 32768 i

*> 11.11.11.11/32 0.0.0.0 0 32768 i

*>i12.12.12.12/32 2.2.2.2 0 100 0 200 ?

……

(2)The second method was to make R1 learn 4.4.4.4 and 12.12.12.12 through IGP, that is,

configure static routes to R4 on R2 and then redistribute the static routes to OSPF.

The configuration on R2 is shown below.

ip route 4.4.4.4 255.255.255.255 10.0.1.2

ip route 12.12.12.12 255.255.255.255 10.0.1.2

router ospf 1

redistribute static

After the configuration, the two segments were learnt by R1, as shown below.

O E2 4.4.4.4 [110/20] via 10.0.0.2, 00:00:40, FastEthernet0/0

O E2 12.12.12.12 [110/20] via 10.0.0.2, 00:00:18, FastEthernet0/0

Summary(1) For the EBGP neighbor R4, R2 could learn all routes advertised by R4.

(2) For R3 (RR), it only reflected the legal and optimal routes that were learnt from the IBGP

neighbor R2 to another IBGP neighbor R1.

(3) BGP protocol is used to transport routes, while IGP (such OSPF) is used to generate

routes.

(4) Generally the AS egress router advertises itself as the next hop to an IBGP neighbor.

(5) On a BGP router, for the routes in the same segment, if there are both IGP routes and BGP

routes, IGP routes are preferred. This is because the priority of an IGP route is higher than that of

a BGP route. ■



Malfunction SituationAs shown in Figure 1, a company used

a ZXR10 T128 router to establish IBGP

neighbor relationship with a user router

through a private network. The user router

established EBGP neighbor relationship

with CNC egress and China Telecom

egress. Ethernet services were connected

to the ZXR10 T128 router.

The ZXR10 T128 router could establish

IBGP neighbor relationship with the user

router properly, but it could not learn

BGP routes. If a default route directing to

the interface of the IBGP neighbor was

configured, ZXR10 T128 could learn BGP

routes.

BGP Route Learning Failure

Key words: BGP, IBGP, neighbor, default route

⊙ Jiang Tongjun / ZTE Corporation


www.zte.com.cn

21Data Products

Malfunction Analysis(1) The BGP configuration on the ZXR10 T128 router is shown below.

router bgp 23839

no synchronization

network 172.16.64.0 255.255.192.0

network 192.168.126.0 255.255.255.0

network 172.16.79.192 255.255.255.192



(2) The user router interconnected to the ZXR10 T128 router through a direct-connected route

to establish the neighbor relationship.

Neighbor Ver As MsgRcvd MsgSend Up/Down(s) State

10.10.10.9 4 23839 86234 7 00:03:11 Established

(3) The neighbor relationship had been established, but the ZXR10 T128 router could not learn

routes from the neighbor. There were only three routes advertised by the ZXR10 T128 router itself,

as shown below.

ZXR10#show ip protocol routing summary

Route Source count

connected: 30

static: 3

ospf: 0

rip: 0

bgp: 3

isis: 0

Total: 36

(4) After a default router was configured, the ZXR10 T128 router could learn the BGP routes

from the neighbor 10.10.10.9, and users could access to the network properly.

ZXR10(config)#ip route 0.0.0.0 0.0.0.0 10.10.10.9

ZXR10(config)#show ip route

IPv4 Routing Table:

Dest Mask Gw Interface Owner pri metric

0.0.0.0 0.0.0.0 10.10.10.9 gei_3/2 static 1 0

3.0.0.0 255.0.0.0 10.10.10.9 gei_3/2 bgp 200 0

4.0.0.0 255.0.0.0 10.10.10.9 gei_3/2 bgp 200 0

4.0.0.0 255.128.0.0 10.10.10.9 gei_3/2 bgp 200 0

4.21.41.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.23.112.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.23.113.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0



4.23.114.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.116.0 255.255.254.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.116.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.117.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.118.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.67.64.0 255.255.252.0 10.10.10.9 gei_3/2 bgp 200 0

4.79.181.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.79.248.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.128.0.0 255.128.0.0 10.10.10.9 gei_3/2 bgp 200 0

………

(5) If the default route was configured to direct to interface 192.168.126.1, the ZXR10 T128

router could also learn the BGP routes. But the source interfaces of the routers were not correct.

ZXR10(config)#ip route 0.0.0.0 0.0.0.0 192.168.126.1

ZXR10(config)#show ip route

IPv4 Routing Table:


0.0.0.0 0.0.0.0 192.168.126.1 gei_3/1 static 1 0

3.0.0.0 255.0.0.0 210.52.186.129 gei_3/1 bgp 200 0

4.0.0.0 255.0.0.0 210.52.186.129 gei_3/1 bgp 200 0

4.0.0.0 255.128.0.0 210.52.186.129 gei_3/1 bgp 200 0

4.21.41.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.23.112.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.23.113.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.23.114.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.36.116.0 255.255.254.0 210.52.186.129 gei_3/1 bgp 200 0

4.36.116.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.36.117.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.36.118.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.67.64.0 255.255.252.0 210.52.186.129 gei_3/1 bgp 200 0

4.79.181.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.79.248.0 255.255.255.0 210.52.186.129 gei_3/1 bgp 200 0

4.128.0.0 255.128.0.0 210.52.186.129 gei_3/1 bgp 200 0

…………

(6) The fault may have relation to the next hop attribute of BGP.

The next hop attribute is to use the IP address of the first router in the next AS along the BGP

path. The next hop will not be changed when a route is advertised through an IBGP link. If an

IBGP neighbor router does not have an IGP route reaching the next hop, the BGP route is only

displayed in the BGP routing table. It will not be loaded in the global routing table.

www.zte.com.cn

23Data Products

Solution(1) Engineers modified the next hop to the ZXR10 T128 router itself on the peer router, that is,

configure the following command on the BGP neighbor (the user router 10.10.10.9).

router bgp 23839

neighbor 10.10.10.10 next-hop-self

(2) Engineers deleted the default route configured on the ZXR10 T128 router.

no ip route 0.0.0.0 0.0.0.0 10.10.10.9

After the above configuration, the ZXR10 T128 router could learn BGP routes, as shown below.

ZXR10#show ip route

IPv4 Routing Table:


0.0.0.0 0.0.0.0 10.10.10.9 gei_3/2 bgp 200 0

3.0.0.0 255.0.0.0 10.10.10.9 gei_3/2 bgp 200 0

4.0.0.0 255.0.0.0 10.10.10.9 gei_3/2 bgp 200 0

4.0.0.0 255.128.0.0 10.10.10.9 gei_3/2 bgp 200 0

4.21.41.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.23.112.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.23.113.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.23.114.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.116.0 255.255.254.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.116.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.117.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.36.118.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.67.64.0 255.255.252.0 10.10.10.9 gei_3/2 bgp 200 0

4.79.181.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.79.248.0 255.255.255.0 10.10.10.9 gei_3/2 bgp 200 0

4.128.0.0 255.128.0.0 10.10.10.9 gei_3/2 bgp 200 0

…………



Network TopologyAs shown in Figure 1, R1, R2 and

R3 established full-mesh IBGP neighbor

Key words: BGP, community, metric, route map, back route

⊙ Fu Tao / ZTE Corporation

BGP Community Attribute Application


Loopback2 and Loopback3 on R1

were configured to simulate two direct-

connected segments connecting to R1.

The loopback addresses of R1, R2, R3

and R4 were 1.1.1.1, 2.2.2.2, 3.3.3.3 and

4.4.4.4 respectively.

Configuration Description(1) R1 established IBGP neighbor

relationship with R2 and R3. A route

map was conf igured fo r segments

172.16.0.1/32 and 172.16.1.1/32 on R1.

The community attributes of the two routes

were set to 100:100 and 200:200. R1

advertised the community attributes to

the IBGP neighbors R2 and R3. (Note: A

relationship with each other in AS 100. R2 and R3

established EBGP neighbor relationship with R4 in

AS 200.

community attribute can contain several segments.

It equals to make a mark for each segment to

make it easy for the same handling. Here, one

community only marks one segment.)

(2) After receiving the community attributes,

R2 and R3 se t the met r i c va lues fo r the

different community attributes of the segments

172.16.0.1/32 and 172.16.1.1/32.

(3) On R2, the metric value for the community

100:100 (that is, the segment 172.16.0.1/32) was

set to 200, and the metric value for the community


set to 100.

(4) On R3, the metric value for the community


set to 100, and the metric value for the community

www.zte.com.cn

25Data Products


set to 200.

(5) R2 and R3 advertised the communities (each

community contained multiple segments) that

were configured with metric values to the EBGP

neighbor R4.

(6) OSPF was configured on R1, R2 and R3.

Static routes were configured on R2 and R4 to

direct to the loopback address of each other. Static

routes were also configured on R3 and R4 to

direct to the loopback address of each other. (As

loopback addresses were used on the routers to

establish BGP neighbor relationship.)

Data Forwarding(1) After show ip bgp route command was

configured on R4, the result showed that

the next hop of the route to 172.16.0.1/32 was 3.3.3.3 (the loopback address of

R3), and the next hop of the route to

172.16.1.1/32 was 2.2.2.2 (the loopback

address of R2).

(2) According to the result of trace

command, the next hop of 172.16.0.1 was

3.3.3.3, and the next hop of 172.16.1.1

was 2.2.2.2.

Malfunction Analysis(1) The result of show ip bgp route

command on R4 showed that R4 learnt the

routes to 172.16.0.1/32 and 172.16.1.1/32,

and the next hops were correct.

R4#show ip bgp route

BGP table version is 4, local router ID is 4.4.4.4

Status codes: s suppressed, d damped, h history, * valid, > best, i -

internal,

r RIB-failure, S Stale

Origin codes: i - IGP, e - EGP, ? - incomplete


* 172.16.1.1/32 3.3.3.3 200 0 100 i

*> 2.2.2.2 100 0 100 i

*> 172.16.0.1/32 3.3.3.3 100 0 100 i

* 2.2.2.2 200 0 100 i

(2) On R4, the result of show ip route command

showed that the routes to the two segments were

also in the routing table, and the next hops were

correct. However, R4 could not ping through the two

segments.

(3) On R1, the result of show ip route command

showed that the route to the direct-

connected segment of R4 and the route to

the loopback address of R4 were not learnt

by R1. The result of show ip bgp route

command showed that R1 did not learn any

related routes from R4 through BGP.

R1#show ip bgp route

BGP table version is 4, local router ID is 4.4.4.4

Status codes: s suppressed, d damped, h history, * valid, > best, i -

internal,

r RIB-failure, S Stale

Origin codes: i - IGP, e - EGP, ? - incomplete




Solution(1) As R4 learnt the routes of the two

segments from R1, the packets (ICMP

packets) sent by R4 could reach R1.

However, R4 could not ping through

R1. According to the results of show ip route command and show ip bgp route command on R1, R1 did not learn related

routes to R4. Therefore, it was certain that

there was no back route.

(2) There might be some problem about

IBGP neighbor relationship establishment,

so there was no back route. As R4 could

learn the routes to the segments of R1,

the IBGP and EBGP neighbor relationship

should be normal. Engineers checked the

neighbor state of the router with show ip bgp neighbor command. They found that

the state was Established.

(3) Engineers analyzed the fault.

They found that when the ping packet

did not contain the source address, the

default source address was the egress

interface address of R4. But R1 did not

learn the segment which the interface

address belonged to, so there was no

response packet. Engineers redistributed

the direct-connected segment between R4 and

R2, and the direct-connected segment between R4

and R3 to BGP in BGP route configuration mode

(redistributing the segments connecting to R4 on

R2 and R3 respectively, or just redistributing them

on R4), the fault was solved.

(4) If the source address of the ping packet was

the loopback address, it was necessary to import

the loopback address to BGP so that neighbors

could learn it.

Summary(1) At present, the BGP community attribute can

be considered to make a mark for some segments

and make another mark for some other segments,

so that the segments with different marks can be

handled in the same way.

(2) The key points of a route are described below.

1) A router is only responsible for forwarding

packets to the next hop of a route entity according

to the routing table.

2) To forward a packet successfully, the router

must learn the route to the remote segment, and

the remote router also must learn the route to the

local router (that is, the local router must advertise

the segment to the remote router). A default route

is an exception. ■

www.zte.com.cn

27Data Products

Network TopologyAs shown in Figure 1, four ZXR10 T160G

switches were connected through GE. OSPF and

BGP learning were enabled on the devices. The

devices ran VRRP at the service side. The four

⊙ Zhou Dongwei / ZTE Corporation

OSPF and BGP Route Oscillation

Key words: BGP, T160G, OSPF, route oscillation


The server segment was in VLAN 10 and the

servers used multiple VRRP groups. The virtual

address of VRRP Group 1 was 192.168.9.1. This

address was used on T160G-2. The virtual address

of VRRP Group 2 was 192.168.9.2 which was the

same with the actual address of T160G-1.

All servers connected to the switches through

single links. The servers connecting to T160G-1

used the address of T160G-1 as the gateway,

and the gateway IP was 192.168.9.2. The servers

connecting to T160G-2 used the virtual address of

T160G-2 as the gateway, and the gateway IP was

192.168.9.1.

devices transmitted VLAN 10, VLAN 20

and VLAN 30 transparently. The devices

also ran STP, and a port on T160G-4 was

blocked.

Malfunction SituationAfter the upgrading of devices on the

DCN network, the CPU usage rates of

all devices kept high. Services were not

steady. There was packet loss and service

interruption.

Malfunction Analysis(1)Engineers checked the peer NE40

devices. They found that the devices

received a lot of BGP Update messages

and Withdraw messages sent by T160G

switches. The segments were the IP



segments of Province Company at DCN

side.

(2) Engineers d isconnected the

connections between NE40 devices

and T160G-1 and those between NE40

devices and T160G-2. CPU usage rates

became normal and services ran properly.

(3) Engineers checked the configurations. They

found that OSPF and BGP were redistributed to

each other on the four T160G switches. When BGP

routes were redistributed to OSPF, the tag values

were modified, as shown below.

router bgp 65508


redistribute static

redistribute ospf-int

redistribute ospf-ext

network 192.168.9.0 255.255.255.0

network 192.168.9.0 255.255.255.192

network 192.168.9.64 255.255.255.192

network 192.168.9.128 255.255.255.128



router ospf 100

router-id 192.168.46.1

network 192.168.9.0 0.0.0.255 area 0.0.0.0

redistribute bgp-int tag 10

redistribute bgp-ext tag 10

(4) According to the configuration,

OSPF and BGP were redistributed to each

other for the service running and route

advertisement on T160G switches. After

NE40 devices on the DCN advertised the

Province Company route segments to

T160G switches through EBGP, all T160G

switches learnt the routes from NE40

devices to communicate with the devices

in the Province Company segments.

Meanwhile, T160G switches redistributed

the routes to advertise them to other

T160G neighbor switches.

In normal situations, as the priority of

OSPF routes was low, T160G switches

did not use OSPF routes to select the egresses.

However, if there was an exception about the BGP

neighbor relationship between NE40 devices and

T160G switches, T160G switches would advertise

OSPF routes to NE40 devices through BGP, and

then NE40 devices would prefer the routes learnt

from T160G switches. This would cause oscillation

of route recalculation on DCN.

(5) As shown in Figure 2, if NE40-1 advertised

segments to T160G-1 first, T160G-3 would learn

the segments from T160G-1 through OSPF.

According to the device configurations, the BGP

routes redistributed to OSPF would lost AS

attributes, and T160G-3 would redistribute all

routes learnt through OSPF to BGP.

www.zte.com.cn

29Data Products

If the time when NE40-1 advertised segments

to T160G-3 was later than the time when T160G-3

learnt the segments from T160G-1 through OSPF,

T160G-3 would advertise the segments to NE40-1.

As the priority of EBGP routes was higher, NE40-1

preferred the routes learnt from T160G-3, and it

would withdraw the segments that it advertised

to T160G-1. As a result, T160G-1 and T160G-3

withdrew the segments. NE40-1 would receive the

Update messages that were used to withdraw the

segments. After that, NE40-1 recalculated routes.

After the recalculation, NE40-1 advertised the

routes to T160G-1 again. Then the devices took

the above procedure again. Such situation also

occurred on T160G-2. T160G-4 and NE40-2. Route

oscillation occurred on the whole network. Routes

were updated and withdrawn frequently.

SolutionEngineers deleted the conf igurat ion of

redistributing OSPF to BGP and used network command to advertise local segments in BGP

configuration. Then they recovered the connection

between T160G-1 and NE40-1 and that between

T160G-2 and NE40-1. Services and the devices

ran properly.

SummaryOn a rectangular network, if OSPF routes

and BGP routes are redistributed to each other,


some devices may receive routes that are

advertised by themselves. If the devices

fail to judge the sources of the routes,

faults of routing will occur, which will cause

service interruption.

When OSPF redistr ibutes a BGP

route, the AS attribute of the BGP route

is saved by the tag field in an LSA. If the

route is advertised to BGP from OSPF, the

devices can judge whether the route is in

the same AS according to the tag value.

This prevents route loops. In the case

described above, redistribute bgp-int tag 10 command was configured to modify the

tag value. Therefore, the devices failed to

judge the sources of the routes correctly,

which caused a loop. The fault could be

solved by deleting the configuration of tag

value.

The second method to solve the fault

is to cancel to redistribute BGP routes to

OSPF. NE40 devices will advertise default

routes to T160G switches through BGP,

and then T160G switches will advertise

default routes to other devices through

OSPF. In this way, dynamic redundancy

on service egresses is realized and loops

are prevented.

The third method to solve the fault is to

cancel to redistribute OSPF routes to BGP.

T160G switches advertise local routes

with network command and they will not

redistribute other routes that are learnt

through OSPF to BGP. In this way, loops

are prevented.

For a network with multiple egresses,

it is necessary to avoid redistributing

routes of different routing protocol to each

other. Prevent loops by methods such as

aggregation, redundancy, and so on. ■



Malfunction SituationAs shown in Figure 1, the ZXR10

8908 switch established EBGP neighbor

relationship with the router. The metric and

community values of the routes sent to the

Key words: BGP, metric, community, routing policy

Advertisement Failure in BGP Routing Policy⊙ Wu Qintao / ZTE Corporation

router by the ZXR10 8908 switch were modified

through a routing policy.

The community value of routes in the segment

10.51.161.0/19 was set to 779.

The metric values of routes in the segments

10.51.161.1/24 and 10.51.161.3/24 were set to

150.

After the configuration, the result on the router

showed that the routes in segments 10.51.161.0/19 and 10.51.161.1/24 only contained the community

value, and there was no metric value.

Malfunction AnalysisEngineers checked the BGP policy configuration

on the ZXR10 8908 switch, as shown below.Figure 1. Network Topology

route-map set-med-comm permit 10

match ip address 2

set community 779

!


match ip address 3

set metric 150

!


rule 1 permit 10.51.160.0 0.0.31.255

!


rule 1 permit 10.51.161.0 0.0.0.255

rule 2 permit 10.51.163.0 0.0.0.255

!

www.zte.com.cn

31Data Products

Engineers found that during the route map configuration, the community value for the larger

segment 10.51.161.0/19 was set first, and the metric value for the smaller segment 10.51.161.1/24 was set later.

The correct execution order is that execute rules for the small matching segment first and then

execute for the large matching segment. Therefore, it was necessary to modify the matching order

in the route map.

SolutionEngineers modified the matching order in the route map, as shown below.


match ip address 3

set metric 150

set community 779

!


match ip address 3

set community 779

!


rule 1 permit 10.51.160.0 0.0.31.255

!


rule 1 permit 10.51.161.0 0.0.0.255

rule 2 permit 10.51.163.0 0.0.0.255

router bgp 100



neighbor 59.43.170.65 prefix-list filter in

neighbor 59.43.170.65 route-map set-med-comm out

neighbor 59.43.170.65 send-community

neighbor 59.43.170.65 send-med

Engineers checked the routes on the router. The result showed that the community and metric

values of the segments met the expected result.

SummaryThe matching of a route map is related to execution order. When there is an entity matched,

the following rules will not be executed. Therefore, it is necessary to put the rules for a smaller

matching segment before the rules for a larger matching segment.

Pay attention to the rule order in other similar configurations where ACL is used. If the rules for

a smaller matching segment are put behind the rules for a larger matching segment, it equals to

that the rules for the smaller matching segment are not executed. ■



Network TopologyAs shown in Figure 1, two routers

connect to each other through EBGP to

interact with routing information.

⊙ Yang Zhiwei / ZTE Corporation

Importing BGP Route Statically

Key words: BGP, importing route static

neighbor relationship is established, routers need

to learn and advertise related routing information. A

device will learn BGP routing information advertised

by the peer devices. Meanwhile, it will advertise its

routing information according to its configuration.

Here we discuss the experiences about

advertising routing information to an EBGP

neighbor statically.

BGP is used to advertise routes. Each router

that runs BGP advertises local networks to the

Internet. In this way, hundreds of thousands of

routes are advertised by tens of thousands of

routers. This is why people can get services from

Internet freely.

Background DescriptionAt present, BGP is a common inter-

AS routing protocol. Generally, after BGP


www.zte.com.cn

33Data Products

The routes must exist in the IGP routing table

before they are advertised by BGP. Importing

routes to BGP by IGP is an important source of

BGP route update. It effects on the route stability

of the Internet. BGP routes can be imported either

dynamically or statically. Here we mainly discuss

importing routes statically.

Static import of routes can solve the problem of

route unsteadiness effectively. It is to import static

route entities to BGP. As a static route is configured

manually, it will not be affected by IGP route

fluctuation. Therefore, the static route is steady.

Its stability prevents continual updates caused by

route fluctuation. However, if the boundaries of

subnets of a network are not clear, import of static

routes may also cause problems, such as traffic

congestion, and so on.

Methods for Static ImportThere are two commands used to import

BGP routes statically, network command and

aggregate-address command.

● network command

A general method to advertise BGP routes is to

use network command. It is necessary to specify

the destination segment and the mask in

this command. A cluster of routes in the

IGP routing table matching the condition

will be added to the BGP routing table, and

then these routes will be advertised after

policy filter.

The cluster of routes contains all

subnets in the specified segment.

For example, after network 10.61.8.0

255.255.254.0 command is configured

in BGP, if segment 10.61.8.0/23 and all

subnets of this segment (such as segment

10.61.9.0/30, and so on) are included in

the routing table, all these segments will

be added to the BGP routing table. If there

are no routes and subnets in segment

10.61.8.0/23, no route will be added to the

BGP routing table.

For example, the device 59.43.185.17

and the device 59.43.185.18 establish

EBGP neighbor relationship. The network

command is configured on 59.43.185.18

to advertise routes to the EBGP neighbor.

The IGP routes on 59.43.185.18 are

shown below.

ZXR10# show ip route

IPv4 Routing Table:


10.61.4.0 255.255.255.192 10.61.4.226 fei_3/1 ospf 110 11

10.61.4.162 255.255.255.255 10.61.4.210 fei_3/8 ospf 110 2

10.61.4.165 255.255.255.255 10.61.4.226 fei_3/1 ospf 110 11

10.61.4.166 255.255.255.255 10.61.4.210 fei_3/8 ospf 110 11

10.61.4.196 255.255.255.252 10.61.4.210 fei_3/8 ospf 110 2

10.61.4.228 255.255.255.252 10.61.4.210 fei_3/8 ospf 110 11

10.61.8.0 255.255.255.0 10.61.4.210 fei_3/8 ospf 110 20

10.61.9.0 255.255.255.252 10.61.9.1 loopback10 direct 0 0

The BGP configuration to advertise routes is shown below.

router bgp 65510

no synchronization

aggregate-address 10.61.4.0 255.255.255.0 count 0 summary-only




network 10.61.8.0 255.255.254.0



neighbor 59.43.185.17 route-map filter in




neighbor 10.61.4.162 update-source loopback1

!

The routes that are advertised to the BGP neighbor are shown below.

ZXR10 #show ip bgp neighbor out 59.43.185.17

Routes Sent to This neighbor:

Dest NextHop Metric LocPrf Path

10.61.8.0/23 59.43.185.18 i

10.61.9.0/30 59.43.185.18 i

10.61.4.0/24 59.43.185.18 i

The network 10.61.8.0 255.255.254.0 command advertises the segment 10.61.8.0/23.

Meanwhile, it also advertises the subnets 10.61.8.0/23 and 10.61.9.0/30.

● aggregate-address command

If the aggregate-address command is configured in BGP on a device, the device will search

the local IGP routing table. If there are routes meeting the aggregation condition, these routes will

be aggregated and then advertised to the EBGP neighbor. The complete format of this command

is shown below.

aggregate-address <ip-address> <net-mask> [count <count>] [as-set] [summary-only] [strict]The parameters in this command are described below.

Parameter Description

<ip-address> The aggregation network to be generated, in the dotted decimal notation

<net-mask> The aggregation mask to be generated, in the dotted decimal notation

count <count>

It specifies that the routes to be aggregated should meet the count of the

aggregation subnets. The range of this parameter is 0~255, with the default

value 1.

as-set It generates AS-set path information.

summary-only It filters special routes in the update.

strict

According to RFC1771, only routes with the same MED attribute and the

same NEXT_HOP attribute can be aggregated. Otherwise, the aggregation

condition should be loosened, and the MED attribute and the NEXT_HOP

attribute should not be considered.

www.zte.com.cn

35Data Products

The following configuration helps to understand the aggregate-address command further.

As shown in Figure 9, when the device 59.43.185.17 and the 59.43.185.18 device establish

BGP neighbor relationship, the configuration on 59.43.185.18 is shown below.

router bgp 65510

no synchronization






neighbor 59.43.185.17 route-map filter in




neighbor 10.61.4.162 update-source loopback1

!

(1) The segment 10.61.8.0 255.255.255.0 is advertised through aggregation. When we view

BGP “neighbor out” information, the route advertised to the neighbor is not found, as shown below.

ZXR10-1#show ip bgp neighbor out 59.43.185.17



10.61.4.0/24 59.43.185.18 i

The IGP routing table of 59.43.185.18 is shown below.


10.61.8.0 255.255.255.0 10.61.4.210 fei_3/8 ospf 110 20

In this condition, aggregate-address command fails to advertise the aggregation route.

(2) If the command is modified as shown below (other commands remain), routes still cannot

be aggregated and advertised.

aggregate-address 10.61.8.0 255.255.255.0 count 0

(3) If the command is modified as shown below (other commands remain), routes are

aggregated, and only the aggregation route is advertised.


(4) If the command is modified as shown below (other commands remain):

aggregate-address 10.61.8.0 255.255.254.0 count 0

Routes are advertised after aggregation, as shown below.



ZXR10-1#show ip bgp neighbor out 59.43.185.17



10.61.8.0/23 59.43.185.18 i

10.61.9.0/30 59.43.185.18 i

10.61.4.0/24 59.43.185.18 i

According to the above steps, the parameter summary-only means to advertise the

aggregation route only. If this parameter is not configured, the device will advertise the aggregation

route, as well as the detailed subnet information.

The parameter count specifies that the routes to be aggregated should meet the count of the

aggregation subnets. The range of this parameter is 0~255, with the default value 1. It means the

count of subnets in the IGP when the device advertises routes. If the value of this parameter is 0,

as long as there is a subnet of 10.61.8.0 255.255.254.0 in the IGP routing table, the device will

advertise the aggregation route 10.61.8.0/23.

If the value of parameter count is 1 or other number, it is necessary to configure the following

command.

aggregate-address <ip-address> <net-mask> subnet <subnet-address> <subnet-mask>

The parameters in this command are described below.

Parameter Description

<ip-address> The aggregation network to be generated, in the dotted decimal notation

<net-mask> The aggregation mask to be generated, in the dotted decimal notation

<subnet-address> The IP address of the aggregated subnet, in the dotted decimal notation

<subnet-mask> The IP mask of the aggregated subnet, in the dotted decimal notation

In this example, the following configuration can be used.

aggregate-address 10.61.8.0 255.255.254.0 subnet 10.61.8.0 255.255.255.0

aggregate-address 10.61.8.0 255.255.254.0 subnet 10.61.9.0 255.255.255.252

After corresponding subnet segments are specified, only when the route information of the

specified subnet segments exists in the IGP routing table, and the count of the subnets reaches

the value of the count parameter at the same time, BGP will aggregate and advertise the

corresponding routes. ■

www.zte.com.cn

37Data Products

Network TopologyWhen an enterprise connected to the upstream

provider through BGP and obta ined extra

bandwidth by using multiple links between two

routers, the fact might be that only one link was in

use, as shown in Figure 1.

Brief Discussion of BGP Load Sharing⊙ Li Wei / ZTE Corporation

Key words: BGP, EBGP multi-hop, EBGP multi-path

is configured on each interface.

This method resolves the next hop

address, and makes the traffic be shared

through the recursive route to the next

hop. In such situation, it is necessary to

configure ebgp-multihop command to

modify the count of hops to 2. Otherwise

the BGP session cannot be established.

(2) EBGP multi-path

EBGP multi-path provides another

method for load sharing on multiple links.

An EBGP session is configured on each

link between the two routers. These

EBGP sessions are bound to the interface

addresses directly. The routers will receive

multiple routes to the same destination.

There is a route on ach link, and all these

routes are the same. By configuring

maximum-paths command, EBGP multi-

path makes all routes be loaded on the

routers until the count of the paths reaches

the maximum number that is configured. ■


SolutionTo make the traffic be shared on the links,

there are two methods, EBGP multi-hop and

EBGP multi-path.

(1) EBGP multi-hop

This method is to use single EBGP session

between the two routers. This EBGP session

originates from the loopback interface address

instead of a physical interface address. A static

routes directing to the remote loopback address



Start of BGP Performance Optimization

BGP has been w ide l y used on

different kinds of large-scale networks.

Compared with IGP, BGP has the following

advantages.

● Having the ability to handle masses

of route prefixes

● Supporting various routing policy

controls

BGP also has its disadvantages. For

example, the time for route update and

convergence increases, and the protocol

operation is more complex.

⊙ Li Wei / ZTE Corporation

Key words: BGP, performance optimization, peer, MSS, input buffer queue, GR, BFD

BGP Performance Optimization on Large-Scale Network

On large-scale networks, routers usually need

to transmit tens of thousands routes. At this time,

it is necessary to use BGP to take the work of

transmitting masses of routes. This will inevitably

increase the load of resources (such as CPU,

memory, and so on) of the devices, and the time

for route update and convergence will be longer.

For example, on a network of the operator

class on the Internet, to improve the network

expansibility, BGP Route Reflector (RR) is widely

used. An RR may receive 300,000 routes from

its non-client IBGP peers. About 200,000 routes

will be loaded to the routing table to reflect to its

www.zte.com.cn

39Data Products

client peers. If the RR has ten clients, 2,000,000

routes will be advertised. This is in a relatively

ideal situation. If BGP neighbor relationship is

established after some troubles during a period,

the optimal paths may be changed for some

times, which increases the number of route

advertisements.

A lot of routes are advertised after they

are handled, and another large part of routes

undergoes osci l lat ions. These are ser ious

challenges to routers. Therefore, it is extremely

important to optimize BGP performance on large-

scale networks.

Aspects of BGP Performance Optimization

(1) The basic transport layer is TCP. TCP

operations provide chances for BGP to improve

the convergence performance. BGP tells the data

to be transmitted to TCP. Then TCP divides the

data into segments according to the length of the

data. The size of the segments is decided by the

MSS value negotiated by TCP. Each TCP segment

corresponds to an IP packet.

A too large MSS value may cause fragmentation

at IP layer on intermediate routers. The fragments

of the data increase the recombination loads

on the receiving routers inevitably. A too small

MSS value may reduce the usage rate of the

network. The amount of packets to be handled

also increases, which then increases the loads of

routers. Therefore, the setting of MSS value is very

important. A reasonable MSS value can reduce the

CPU load of routers shorten the convergence time

of BGP routes.

(2) Generally, BGP packets need to be handled

by CPU. When there are a lot of BGP waiting to

be handled, BGP packets enter the BGP input

buffer queue. If the BGP input buffer queue is too

short, some BGP packets will be discarded and

then retransmitted, which lengthens the BGP route

convergence time. Therefore, adjusting the length

of BGP input buffer queue is helpful to shorten the

BGP route convergence time.

(3) BGP peer group technology is

widely used. A peer group provides the

mechanism for BGP peers that the BGP

peers with the same departure policy

associate with each other. In a peer group,

as all peers have the same departure

policy, the update messages of them

are the same. This means that it is only

necessary to generate one BGP update

message and this message can be reused

by all other peers in the group. In a non-

peer environment, the amount of update

messages increases greatly. This is a

challenge to the CPU and memory of a

router. Therefore, deploying peer group

can optimize BGP performance.

(4) On a BGP network where there are

a lot of routes, BGP neighbor invalidation

or neighbor relationship re-establishment

will cause a lot of routes to be updated

or withdrawn. The route convergence

time will also be longer, and a lot of traffic

will be lost. BGP Graceful Restart (GR)

can ensure traffic not to be interrupted

during neighbor invalidation or neighbor

relationship re-establishment by using the

independence of data plane and control

plane.

(5) Due to the indirect connections

between BGP neighbors, BGP routers

need to rely on IGP convergence or BGP

Keepalive packets to perceive neighbor

states. In this way, it takes a BGP router

about 180 seconds to complete the

convergence. BGP BFD can be used to

shorten the time that a BGP router uses to

perceive neighbor states and then shorten

the convergence time.

(6) Where there are a lot of routes on

the network, it is very unfavorable to bring

route updates due to frequent BGP route

oscillations. Route dampening can be



used to suppress the frequent oscillation

to maintain the network stability.

(7) Generally, when BGP inbound

policy is changed, the router will not save

the prefixes of routes that are received

previously. Therefore, it is necessary to

reset BGP. The influence of resetting BGP

on a large-scale network is obvious.

Route refreshing technology can solve

this problem. Route refreshing is a kind of

ability that needs to be negotiated when

BGP neighbor relationship is established.

When BGP inbound policy is changed, route

refreshing can make BGP routers request the

remote peers to send their routing tables again. In

this way, BGP routers can apply the new inbound

policy without re-establishing BGP neighbor

relationship.

(8) BGP has some timers that can be used to

periodically scan the prefixes of routes that need

to be updated or withdrawn. If the performance of

routers is good enough, the convergence time can

be shortened by shortening timer interval. ■