Basic Troubleshooting[1]

8/4/2019 Basic Troubleshooting[1]

1/18

Introduction to Network

Troubleshooting

Ann Harding

April 2005


2/18

Preparation Learn the network

Topology \Network_Diagrams\PhysicalNetwork\HEAnet-PhysicalNetworkMap.vsd

\Network_Diagrams\Multicast\multicast-map.vsd

Protocols IPv4, IPv6, Multicast

Routing Policy, protocols

https://www.hea.net/docs/bgp.html, https://www.hea.net/docs/v6bgp.html,https://www.hea.net/docs/igp-costs.html

Equipment http://www.cisco.com/en/US/customer/support/index.html

http://www.juniper.net/customers/support/

Understand normal network behaviour https://www.hea.net


3/18

DefineProblem

ConsiderPossibilities

Gather Facts

Create Action Plan

Implement Action Plan

Observe Results

IterateProcess

Problem Resolved

Document Facts

Do Problem

Symptoms Stop?

START

FINISH

Y

N

TheInternet is brokenXYZ is down on Netsaint

I cant get to a website

What are the symptoms?When, how often, where?

What are possible causes?

Has anything changed?

What is happening?Do some checking:

Routers, Syslog, RANCID,

Network Management

Re-evaluate possible causes

Narrow down possibilities

Identify what you need to check

Section the troubleshooting

Change one variable at a time

Start at a source device

Check as you go

Limit invasive impact

Be prepared to roll back

Check each time you change

Refer to problem definition

Refer to tests in fact-gathering

Troubleshooting Methodology


4/18

ATM links

Eircom Symphony ATM ServiceClient and HEAnet have two different

physical access circuitsEircom provision thePVC through their

network

Client side of thePVC may or may not

have an ATM switch present

HEAnet ATM over Esat linksVirtual path and physical path similar

Telco unaware ofATM aspect

Client side of thePVC may or may

not have an ATM switch present

Two main setups


5/18

ATM links Docs reference: https://www.hea.net/docs/clientlinks-atm-eircom.html

https://www.hea.net/docs/clientlinks-atm-pointtopoint.html

Determine which switch and router the circuit is on and telnetto them.

See ifyou can ping the otherend of the link from the router

Check the logs using show log on the relevant switch andaccess router and look for linkup/linkdown messages.

Ifthe interim links are down, there is nothing for theVirtual Circuit torun over.

Check the ATM VC on the router and switch show atm vc to check the status ofthe virtual circuits.

Check the interfaces on the router and switch using show

interface


6/18

ATM links Ls1010 interfaceexample

ATM0/0/0 is up, line protocol is up

Hardware is oc3suni

Description: STM-1 DIAS (Esat HEAN0013)

MTU 4470 bytes, sub MTU 4470, BW 155520 Kbit, DLY 0 usec, rely

255/255, load 1 /255

Encapsulation ATM, loopback not set, keepalive not supported Last

input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Queueing strategy: fifo

Output queue 0/40, 0 drops; input queue 0/75, 0 drops

5 minute input rate 19000 bits/sec, 65 packets/sec

5 minute output rate 168000 bits/sec, 400 packets/sec

456031409 packets input, 2694828197 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

89 input errors, 93 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

1839217443 packets output, 2989243967 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 output buffer failures, 0 output buffers swapped out

This means the interface hardware is active.

Line protocol is up means that keepalives are successful.

Ifthree successive keepalives fail, line protocol will go down.

Neither input nor output errors should be accumulating.If in doubt, clear the counters and check the interface again.


7/18

ATM Links Router interfaceexample

mantova>sh int atm3/1.8

ATM3/1.8 is up, line protocol is up

Hardware is ATM OC12 LC, address is 0011.21b3.bf80 (bia0011.21b3.bf80)

Description: DIT 90Mbps (CBR AVC502353)

Internet address is 193.1.194.41/30

MTU 4470 bytes, BW 90000 Kbit, DLY 80 usec,

reliability 255/255, txload 22/255, rxload 4/255

Encapsulation ATM

1096645466 packets input, 298037834034 bytes

1552827776 packets output, 1414170448777 bytes

239959 OAM cells input, 241916 OAM cells output

ATM and protocol may show up, even ifthere is a problem with the circuitas it refers only to the connection between the router and theLS1010.

Where OAM management is being configured, this will pull this link down in

theevent ofa break anywhere on theVC path.


8/18

Serial and Gigabit Ethernet links Docs reference

https://www.hea.net/docs/clientlinks.html#clientlinks-serial

https://www.hea.net/docs/clientlinks-gigabit.html

Determine what router the circuit is on and telnet to it.

Test to see ifyou can ping the otherend of the link

Check the logs and look for linkup or linkdown

messages, or OSPF transitions. Check the interfacefor the affected circuit using the

show interface command


9/18

Serial interface exampleSerial1/0 is up, line protocol is upHardware is M4T Description: NQAI (DS0858730)

Internet address is 193.1.194.25/30

MTU 1500 bytes, BW 64 Kbit, DLY 20000 usec, reliability 255/255,

txload 1/255, rxload 23/255

Encapsulation HDLC, crc 16, loopback not set

Keepalive set (10 sec)Last input 00:00:04, output 00:00:00, output hang never

Last clearing of "show interface" counters 7w3d

Queueing strategy: fifo

Output queue 0/40, 261 drops; input queue 0/75, 0 drops

5 minute input rate 6000 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

3779614 packets input, 363897481 bytes, 3 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

534 input errors, 300 CRC, 0 frame, 0 overrun, 0 ignored, 234 abort

3868414 packets output, 1209721840 bytes, 1 underruns

1 output errors, 0 collisions, 34 interface resets

0 output buffer failures, 0 output buffers swapped out

68 carrier transitions DCD=up DSR=up DTR=up RTS=up CTS=up

This means the interface hardware is active.

Line protocol is up means that keepalives are successful.

Ifthree successive keepalives fail, line protocol will go down.

Neither input nor output errors should be accumulating.

If in doubt, clear the counters and check the interface again.

The number ofcarrier transitions should not be incrementing.

Clear counters and check.


10/18

IP connectivity Ping cisco output

! successful reply

. timeout waiting for reply U destination unreachable

N network unreachable

P protocol unreachable

Q source quench

M could not fragment

? Unknown packet type


11/18

IP connectivity trace(route|rt) cisco(linux|windows)

Terminates when destination responds or max

TTLexceeded

Cisco version includes AS routing info

Look at msec values, not how slow the info

appears on your screen! Remember hostnames are determined by DNS


12/18

IP

connectivity TCP/IPshow commands

show ip route[address] | show route[address]

Shows entries in routing table

Shows protocols that derive the route

Shows administrative distance and metric of the route

Shows the next hop router address

Shows the interface where the route can be reached


13/18

IP connectivityhyperion>sh ip route 193.1.219.90

Routing entry for 193.1.219.0/24

Known via "ospf 1", distance 110, metric 20

Tag 1000, type extern 2, forward metric 1

Last update from 193.1.196.174 on GigabitEthernet3/0/0,

00:04:06 ago

Routing Descriptor Blocks:

* 193.1.196.122, from 193.1.195.33, 00:04:06 ago, via

GigabitEthernet3/3/0

Route metric is 20, traffic share count is 1

Route tag 1000

193.1.196.174, from 193.1.195.33, 00:04:06 ago, via

GigabitEthernet3/0/0

Route metric is 20, traffic share count is 1

Route tag 1000


14/18

IP Connectivity

TCP/IPshow commands

show ip protocols

What routing protocols are running

Frequency ofupdates

Redistribution information Routes for which the routing process is injecting routes


15/18

IP ConnectivityROUTE SOURCE DEFAULT VALUEConnected Interface 0

Static route 1

EIGRP summary route 5

External BGP 20

Internal EIGRP 90

IGRP 100

OSPF 110

IS-IS 115

RIP 120

EGP 140

ODR 160

External EIGRP 170

Internal BGP 200

Unknown not entered into Cisco routing table 255


16/18

IP connectivity TCP/IPshow Commands

show ip access-list | show firewall [filter ]

Gives ACL number/name and policies and counts show ip interface| show interface detail

Gives all addresses on interface

Multicast group membership information

MTU

ACLs applied

show ip traffic | show interface queue

Shows information on reasons for packet discards


17/18

Exercises Use show ip route to trace the path from Titan to

Cyclops

Compare what you find out to the output of trace Use a show command to find out if there are ACLs

on NUIMs link

How would you check how many matches?

Check forerrors on the UCDs primary connection Ar e there any dropped packets or likely circuit problems?

On Deimos, what type ofroutes are being

redistributed into which routing protocols?


18/18

Reading Cisco Internetwork Troubleshooting

HEAnet Documentation

https://www.hea.net/docs/

http://www.cisco.com/warp/public/535/4.html

Cisco TCP/IP tutorial

Documents

Basic Troubleshooting[1]