2.12.1

Understanding Network FailuresUnderstanding Network Failures

1.0 Understanding Network Failures (program overview)

2.0 Intro to ping 2.1 Usage Intro (Strybd prototype)

2.2 Lab

2.3 Assessment

Plus addt’l e-Learning Modules, labs and assessments:

3.0 Intro to traceroute, lab, assessment

4.0 Intro to netstat, lab, assessment

5.0 Intro to ipconfig (ifconfig), lab, assessment

6.0 Intro to nslookup, lab, assessment

7.0 Intro to whois, lab, assessment

8.0 Pre-Assessment Modules (pre-tests for each module)

9.0 Assessment Modules

10.0 Labs: User Tools & Network Utilities (telnet short-cuts, PCHAR/TTCP,…?)

2.1 Lesson Objectives2.1 Lesson Objectives

In this lesson, students will utilize “In this lesson, students will utilize “pingping” ” to validate network connections, and to validate network connections, and analyze responses reported from “analyze responses reported from “pingping””

Audience information:Audience information:–Call Center I & II/CCNA I & IICall Center I & II/CCNA I & II–20 Minutes (duration)20 Minutes (duration)

2.1.12.1.1

User:s0 s1

e0

Center

EvaBoaz

e0

s0s0

6543

1

e0

s2

Server 1

2

Customer Support:

CS:

User:

Network failures: Network failures: The sky is falling! The sky is falling!

“Becky”

The Internet

Becky

??

2.1.22.1.2

Policy change or local failure?Policy change or local failure?

–Do the interfaces show a link light? Do the interfaces show a link light?

Before escalating this call . . .Before escalating this call . . .

For most users: For most users: The browser The browser isis “The Internet” “The Internet”

. . . the sky . . . the sky isn’tisn’t falling! falling!

–LAN/WAN connectivity? (LAN/WAN connectivity? (# ping yahoo.com# ping yahoo.com))

Example: Text messages are being dropped by Example: Text messages are being dropped by “Boaz” router“Boaz” router

??

2.1.32.1.3

–Does the interface show a link light? Does the interface show a link light?

Review: Before escalating a Review: Before escalating a customer call . . .customer call . . .

Consider local failures first!Consider local failures first!

Identify recent (local) modificationsIdentify recent (local) modifications

The browser The browser isis “The Internet” ( “The Internet” (for most users))

–Are new patches applied? Applied correctly?Are new patches applied? Applied correctly?

Many local network “failures” are due to Many local network “failures” are due to operator erroroperator error

Experience suggests . . .Experience suggests . . .

–Un-skilled users, un-trained personnel, invalid Un-skilled users, un-trained personnel, invalid configurations . . .configurations . . .

Suspect recent changes or modificationsSuspect recent changes or modifications–Have all required patches been applied correctly?Have all required patches been applied correctly?

–Check the logs (Check the logs (recent activity? upgrades?recent activity? upgrades?))

Circuit “Circuit “outagesoutages” are a common cause ” are a common cause of real (of real (actualactual) network faults) network faults

–Example: Heavy equipment workers & sea dredging have Example: Heavy equipment workers & sea dredging have cut cabling, power lines, deep sea fibre cut cabling, power lines, deep sea fibre ((very rare!)very rare!)

1.01.0 (Review) (Review): Common Causes of : Common Causes of Network FailuresNetwork Failures

DoS Attacks = Sluggish network segmentsDoS Attacks = Sluggish network segments

For our example, the Internet is down!For our example, the Internet is down!Example: “Example: “pingping” may be used to verify all subnets ” may be used to verify all subnets “up” during DoS attack“up” during DoS attack

Alert:Alert: s2 s2is “down”!is “down”!

Status: (Status: (ping or traceroute scriptping or traceroute script))–All Routers and sub-nets “up” (reachable), except . . All Routers and sub-nets “up” (reachable), except . . –Center-s2 (Serial_2) “unreachable” during attackCenter-s2 (Serial_2) “unreachable” during attack

─Example: Denial of Service Example: Denial of Service ((DoS):DoS): More common. . .? More common. . .?

2.1.4 (1.0)2.1.4 (1.0)

Eva

653

Server 1

s2

# ping # ping 192.16.10.62192.16.10.62

Echo Request:

Echo Reply:

How many intervening How many intervening devices, as shown? devices, as shown?

WS4

192.168.10.62

Center

Boaz

2

e0

s0s0 s1

e0

s0

e0

What if this ping What if this ping fails? fails?

Reduce scope of test. . .Reduce scope of test. . .

Center-sw1

Boaz-sw1

Sw1-8

Sw1-2

2.1.52.1.5

Round-trip:Round-trip: A Request/Reply “pair” A Request/Reply “pair”

2

Serial_0Serial_0s1

EvaBoaz

e0

s0s0

6543

Server 1

e0

s2

Example: Using Example: Using pingping

Initial troubleshootingInitial troubleshooting # ping <# ping <IP-address>> ( (e.g. e.g. pingping <<local nodeslocal nodes>>))

Demonstration:Demonstration: ““ping Serial_0ping Serial_0””

# ping 192.168.10.65# ping 192.168.10.65

Type <ESC> to abort. Type <ESC> to abort. Sending 5Sending 5, ,

100-byte ICMP Echos100-byte ICMP Echos toto

192.168.10.65192.168.10.65, timeout is 2 seconds:, timeout is 2 seconds: !!

Success rate isSuccess rate is 100 percent100 percent (5/5) (5/5), ,

round-trip round-trip MinMin//AvgAvg//MaxMax = 4/6/9 ms = 4/6/9 ms

!!!!!!!!

Center

e0

Serial_0

2.1.62.1.6

pingping uses ICMP Echo Request/Reply uses ICMP Echo Request/Reply

ICMP Message types:ICMP Message types:

–EchoRequest/EchoReply: “ping” connectivity–Dest unreachable: Packet delivery problem–Time exceeded: Packet discarded (TTL)–Redirect: Better route via “router_ip_address”

Using “Using “pingping” continued. . . ” continued. . .

There are many ways to utilize “There are many ways to utilize “pingping” . . .” . . .

–Specify data length, source and dest. addresses

Extended “Extended “pingping” (options)” (options)

–Specify “next hop”

–Set timeout interval (default: 2 seconds)

–Specify ping count (repeated ping attempts)

–Specify data pattern (sliding “1s”, or 0xABCD)

–Validate response data (data validity)

–Set: Don’t Fragment, include Timestamp, etc

“ “My internet is downMy internet is down” could be a sluggish ” could be a sluggish network segment, slow server, or network segment, slow server, or equipment fault . . . ?equipment fault . . . ?

–How many intervening devices? (firewall, appliance, How many intervening devices? (firewall, appliance, proxy server, CSU/DSU, …)proxy server, CSU/DSU, …)–Is it a recurring fault or temporary slowness or Is it a recurring fault or temporary slowness or random outages?random outages?

Initial Network TestsInitial Network Tests

Collecting accurate failure data is Collecting accurate failure data is crucialcrucial!!

–Could be an Could be an intervening application server, intervening application server, device or appliancedevice or appliance

Review: Initial Network Tests: Review: Initial Network Tests: What to consider?What to consider?

User: “User: “My internet is down . . .My internet is down . . .””

““ping yahoo.comping yahoo.com” = “Are you there?”” = “Are you there?”

–Intermittent faults may appear as temporary Intermittent faults may appear as temporary service outages (service outages (e.g. threshold exceeded, server e.g. threshold exceeded, server rebooting, . . .rebooting, . . .))

Standard diagnostics using “Standard diagnostics using “pingping”:”:

# ping 127.0.0.1# ping 127.0.0.1

pingping: Validate Connectivity: Validate Connectivity

# ping # ping <IP address of local host>

# ping # ping <default-gateway IP address>

# ping # ping <remote destination IP address>

# ping # ping <remote destination hostname>

What is a 20% success rate?What is a 20% success rate?

# ping 192.168.10.62# ping 192.168.10.62

Type <ESC> to abort.

Sending 5, 100-byte ICMPSending 5, 100-byte ICMP

Echoes Echoes to 192.168.10.62to 192.168.10.62

Success rate is 20 percent (1/5)Success rate is 20 percent (1/5), ,

round-trip _min/avg/max = 76/76/76 msround-trip _min/avg/max = 76/76/76 ms

timeout is 2 secondstimeout is 2 seconds::

ECHO Request (from WS2):

ECHO Request (from WS2):

ECHO Reply (to WS2):

pping responsesing responses:: (.)(.) = timeout, = timeout, (!)(!) = success, = success, (N)(N) = Net-Unreachable, = Net-Unreachable, (U)(U) = Dest-Unreachable = Dest-Unreachable

.. .. .. .. !!

2.1.72.1.7

–““Are you there?” (Are you there?” (ECHO Request sent from sourceECHO Request sent from source))

““ping 192.168.10.65ping 192.168.10.65” will validate network ” will validate network connectivity (connectivity (between source and destinationbetween source and destination))

–““I am connected” (I am connected” (ECHO ReplyECHO Reply received from destination received from destination))

–5 of 5 packets = 100% success rate5 of 5 packets = 100% success rate

See, also, See, also, www.cwdotson.com/NetFailures,dd2

Review: Using Review: Using pingping

Recall the ping responses: An exclamation (!) indicates which test result?

A) Failure; B) Success; C) Time out

Questions: Using Questions: Using pingping

Recall the ping responses, and exclamation (.) indicates:

A) Failure; B) Success; C) Time out

(True/False) Ping is an excellent performance monitor

(True/False) 2 of 5 successful packets indicates a success rate of 20%

False (40% success)False (40% success)

2.1.82.1.8

(True/False) When ping is executed, the source

issues an Echo Request to the destination.

B) SuccessB) Success

C) TimeoutC) Timeout

FalseFalse

TrueTrue

Intermittent faults: Difficult to identify & fixIntermittent faults: Difficult to identify & fix

–Occasional errors (“Time exceeded”)Occasional errors (“Time exceeded”)

Intermittent Vs. Recurring Intermittent Vs. Recurring FailuresFailures

–Errors may occur only under certain conditions Errors may occur only under certain conditions (e.g. temporary outages, threshold exceeded)(e.g. temporary outages, threshold exceeded)

Recurring faults: Easier to identify (Server, Recurring faults: Easier to identify (Server, router, or interface is “down”)router, or interface is “down”)

–Chronic fault (“Network unreachable”)Chronic fault (“Network unreachable”)

Limitations of “Limitations of “pingping””

# ping yahoo.com# ping yahoo.com

Type <ESC> to abort.

Sending 5, 100-byte ICMP

Echos to 209.131.36.159

timeout is 2 seconds: ! ! ! ! !! ! ! ! !

Success rate is Success rate is 100 percent100 percent (5/5), (5/5),

round-trip _round-trip _min/avg/max = 23/26/29 msmin/avg/max = 23/26/29 ms

pingping can validate “ can validate “connectivityconnectivity”” onlyonly!!

–““100%” success 100%” success expected!expected!

–ICMP packets do NOT represent “real ICMP packets do NOT represent “real world” trafficworld” traffic

–pingping: Response is for few, : Response is for few, smallsmall pkts pkts

CautionCaution:: pingping is a is a poorpoor tool for performance monitoring! tool for performance monitoring!–Network performance varies for ”real world” trafficNetwork performance varies for ”real world” traffic–Text traffic is much different than streaming video or VoIPText traffic is much different than streaming video or VoIP

–For small, idle networks 100% success For small, idle networks 100% success rates are common (not “real world”)rates are common (not “real world”)

Review: Review: pingping limitations limitations

pingping: Validates network paths: Validates network paths

–Sends a few, small packets (e.g. 100-byte, Sends a few, small packets (e.g. 100-byte, ICMP packets are not “real world” traffic)ICMP packets are not “real world” traffic)

Only confirms basic connectivity between remote Only confirms basic connectivity between remote nodesnodes

In this lesson we:In this lesson we:

Lesson SummaryLesson Summary

–Examined LAN/WAN failures (Examined LAN/WAN failures (DoS, circuit breaksDoS, circuit breaks))

–Used “Used “pingping” to validate a network ” to validate a network connection with remote nodesconnection with remote nodes

–Examined responses reported by “Examined responses reported by “pingping” to ” to analyze network performanceanalyze network performance

2.1.92.1.9

Hosts/Routers return “Dest. Unreachable” when:Hosts/Routers return “Dest. Unreachable” when:

Data cannot be completely delivered to receiving Data cannot be completely delivered to receiving application at the destination hostapplication at the destination host

–Example: ICMP messages sent back to WS2 is reponse to “ping” (e.g. # ping serial_0)

Destination UnreachableDestination Unreachable

–Network unreachable: No matching routeNetwork unreachable: No matching route–Host unreachable: Host unreachable: packet is routable but host not respondingpacket is routable but host not responding

–Can’t fragment: Older router/Large pktsCan’t fragment: Older router/Large pkts ( (mustmust fragmnt but “do not frag” bit set) fragmnt but “do not frag” bit set) –Protocol unreachable: Transport layer protocol “down” at hostProtocol unreachable: Transport layer protocol “down” at host–Port unreachable: Host application fault (port un-opened by Port unreachable: Host application fault (port un-opened by app)app)

Use Use pingping to trace a path ( to trace a path (identify “last” routeridentify “last” router))

telnettelnet to last last “traced” router or node to last last “traced” router or node

# telnet # telnet <IP address-router_lastknown>

Isolating IP Routing Problems:Isolating IP Routing Problems: