Download ppt - Juniper Trouble Shooting

1.Control and Forwarding plane

Synchronization

100-Mbpsfxp1Ethernet link is used between RE and PFE

For M320 case, 100-Mbps Ethernet switch is being used to provide a dedicated link to each FPC. For RE, these links are presented atbcm0

Fxp0: management interface

Fxp2: communication between Primary RE and backup RE

3)Forwarding table (FT) can hold over 800,000 routes.

Redundant RE: M10i support, not M7i

Built-in Adaptive Service: M7i. M10i needs an external AS PIC.

RE: the same

3 types of storages:

Compact Flash(ad0) : built-in at the board.

Hard Drive(ad1)

External storage

-PCMCIA card(da0??)

-USB(da1??)

Space bar to complete a command

Command:Help topic for general concepts

Command : help reference for configuration syntax

Rebooting system:request system reboot

Shut down system:request system halt

Log and Trace files are located at /var/log

Command:Show log | messages |file-name

At more prompt, use forward slash(/) to search or use h to get a context help screen

Log commands examples:

- show log messages | match so-0/3/1 | match TRAP--- AND --

- show log messages | mach fpc | sfm | kernel--- OR ---

Monitor log/trace in real time:monitor startfile-name| match fail

Stop monitoring in real time:monitor stop

Enable/disable real-time output to screen: Esc-Q

Stop traceing operation:delete flag open

Truncate(clear) log/trace files:clear logfile-name

Delete log/trace files:file deletefile-name

Entering configuration:Typeconfigureoredit

Exclusive configuration (configure exclusive) and Private configuration (configure private??)

Moving within the configuration hierarchy: edit (equivalent to cd), up, top, exit (to previous location in the hierarchy)

Show command at configuration mode vs. show command at operational-mode

Relative configuration commands Starting with JUNOS5.3 : top

- top show system login(show system login no matter where you are.Examples:

- top edit protocols ospf( to enter protocols ospf no matter where you are)

Viewing configuration in operational mode:show configuration < configuration path>

View configuration with set: show xxx | display set

Viewing candidate configuration: show chassis alarm,show (at the current sub-hierarchy)

Change the candidate configuration. Examples:

- set alarm sonet lol red

- delete alarm sonet pll

Display difference between the candidate and active configurations:

At the current statement-path, show | compare

Viewing difference in files. Example:

- file showfilename1| compare filefilename2

- show configuarion | compare rollback number

Removing statements:delete

Delete the statements and all its subordinate statements and identifieres.

Wildcard delete. Example:wildcard delte interfaces fe-*

Ignore portion of the configuration hierarchy:deactivate/activate

Disable an interface: set disable interface

Delete and disabled interface: delete interface disable

Activate a configuration

commit ----- candidate file is checked, actived and marked as the current

operational sofware configuration file.

commit check ----- only validate a candidate configuration without

placing it into effect.

rollback n -------- recover the previous configuration. And then commit

rollback 0is current configuration

First 3 roll back (1-3) are stored in solid-state flash disk

/config/juniper.conf.n (n=1-3)

rest roll back (4-49) are stored in hard disk/var/db/config

commit confirmedtime-out---- temporarily activate a configuration (default is 10 minutes). If the final commit is not executed, the system will performs a rollback 1, commit commands.

commit synchronize ---- after committed on the master RE internally

copied and committed on the backup RE automatically.

commit attime----- commit at some time

clear system commit ---- cancel a pending commit

Save a configuration

savefilename

save terminal-- for copy and paste into other others

show | display set create configuration for simplifying configuration editing.

Loading configuration files ( load and then commit)

load override filenameoverride the currentconfig with the loaded one. Do it at the root of the configuration hierarchy.

load mergefilename - combine the new and old

load merge terminal(then copy/paste hierarchical configuration)

load replacefilename statements with replace tag will replace the statements with the same name

load relativeload at where it is current at the configuration hierarchy.

Only save the configuration under certain hierarchy. To save the whole configure, issue this command at the top of the hierarchy.

# Save

Display the contents of the file you saved

# Run file show

To load a configuration after clear the current configuration

# delete

#load override

To recover a mistake made previously after committing.

#rollback 1

show log messages | last

Show log interactive-commands | match restart

Use sysctl a to display kernel parameters.

sysctl a | grep icmp(under shell prompt)

show chassis 0 pic slot 1 information.

Show chassis pic fpc-slot 0 pic-slot 1

Master switchover

Request chassis cfeb master switch

Request chassis routing-engine master switch

Find out who logins the system and kick out some particular users.

show system users

reequest system logout

help syslog

Example:

lab@santro-re0> help syslog ACCT_ACCOUNTING_FERROR

Name:ACCT_ACCOUNTING_FERROR

Message:Unexpected error from file

Help:Error occurred during file processing

Description:An error prevented the accounting statistics process from processing the indicated file.

Type:Error: An error occurred

Severity:warning

show configuration with inheritance

show configuration interfaces ge-4/3/3 | displayinheritance

set system syslog file messages any notice

Show chassis alarms

Show chassis craft-interface

Show log messages

Show log chassid

Monitor start [message | chassid]

Show chassis hardware

Show chassis fpc

Show pfe stat error

Show interface terse

Show interface detail

Show log

Show chassis pic fpc-slot 0 pic-slot 1

Example:

lab@santro-re0> show chassis pic fpc-slot 0 pic-slot 1

FPC slot 0, PIC slot 1 information:

Type10x 1GE(LAN), 1000 BASE

ASIC typeH chip

StateOnline

PIC version1.13

Uptime1 day, 22 hours, 25 minutes, 17 seconds

PIC port information:

FiberXcvr vendor

PortCable typetypeXcvr vendorpart numberWavelength

0GIGE 1000SXSMFINISAR CORP.FTRJ8519P1BNL-J2850 nm

If you need to reboot from PCMCIA card, you need to copy a special image called jinstall-mediaxxxx .

Interrupt normal boot

Hit space when the system is rebooting until it goes to either boot: or OK prompt.

If you get boot: prompt, the loader is not run yet. You need to do this:

Boot: /boot/loader

Change a boot device at OK prompt

Ok nextboot compact-flash

Ok reboot

Disable(admin down) an interface

Admin Link

So-0/1/1 down up

So-0/1/1.0 up down

Deactivate an interface

Admin Link

So-0/1/1 up up

Primary coopy of JUNOS resides on the flash memory.Use this command to create a backup copy:

request system snapshot

Mgd manages CLI

RE has different versions: RE-333, RE-400, RE-600, RE-1600. Each RE is supported by certain platforms.

RE uses Intel processor from P III to P IV.

Use this command to find out what RE is being used:show chassis hardware.

Hard disk monitoring: Self-Monitoring Analysis and Reporting Technology System(SMART). From 5.5, SMART is enabled by default. To disable:

set system processes disk-monitoring disable

Configuration file compression: default starting Release 7.0 (maybe). To enable:

set system compress-configuration-file

RE versions

RE5(RE-400): only supported in M7i and M10i

RE4(RE-600): All M and T series. Except M7i/M10i/M320. The only RE to have flash memory upgrade

RE3 (RE-333): M5/10/20/40/40e, and M160

RE-1600: M320 and T320/T640. Using Broadcom chipset for Ethernet connectivity to PFE.

While used on M320, the GE link is supported as bcm0. While on T-series, 100- Mbps is supported(???)

Different names but referring to the route lookup module:

M40 System Control Board (SCB)

M20 System Switch Board (SSB)

M5/10 FPC and SCB are combined into a single boardcalled the Forwarding Engine Board (FEB)

M7i/10i Compact FEB (CFEB)

M40e and M160 Switching and Forwarding Module (SFM).4 SFM on M160, each one provides 25% of lookupcapability.2 SFM on M40e, only one can be active.

Special stuff on M40e and M160 platform:

MCS card(Miscellaneous Control Subsystem): provide control and monitoring functions for the various components in the chassis

PCG(PFE clock generation): 125-MHZ signal. Redundant PCGS

M320 is different than T and M-series. It is a combio of two using I and J chips.

T640 PFC2 has single PFE, PFC3 has two PFE

T-Series nonblocking cross-bar switch fabric Switch Interface Boards(SIBs).

T320: 3 SIBs with 2 are active. SIB 1 and 2 are active, SIB0 is standby. SIB0 has only one high-speed line (HSL) connected to FPC. SIB1 and SIB2 has 2 HSL. So when SIB0 becomes active, system performance is degraded.

T640: 5 switch fabric cards or SIBs, 4 are active, 1 standby. Something like Ciscos GSR.

M320: 4 SIBs.

M320 FPC1: use single I chip

M320 FPC2: dual I chip, thus two PFE

M320 FPC3: dual J chip, thus two PFE

IP service PIC is to hardware assist complex packet processing and has no physical ports.

IP service PIC include:

1)Tunnel service PIC for IP-IP, GRE tunnel and PIM-SM tunnel.

2)Multlink PIC:Multilink Point-to-Point (MLPPP) and Multilink Frame Relay (MLFR, FRF 1.5)

Hot-Pluggable except M20 and M40 which need to remove FPC.

Take PIC offline before physically removing it. Otherwise would cause system damage or PFE reset.

Packet loss is expected on M-serials except M320 because of FPC reset.

Support 1 to 4 PICs.M160 OC-192 has an FPC support only one PIC.

Each FPC on M-serial pooled to create shared memory switch fabric. So hot-swap FPC cause system to repartition the shared memory pool; 200 ms packet loss.

FPC is hot-swappable in all platforms except M5 and M10 which is using FEB. However M7i and M10i are OK even using CFEB.

Build-in FPC at some high-speed quad-wide PICs such as OC-48c/STM-16 for M20/40. OC-192c/STM-64 SONET/SDH on M160.

New FPC to support reuse of old PICs:

M160 FPC1:intend to reuse M20/40 PIC

M160 FPC2: design to support M160 only PIC, such as OC-48c

FPC3: support native T-series PICs.

T640 only support FPC2 and FPC3.

How to power off FPC?

set chassis fpc power off

General functions

Names very by platforms

M40 System Control Board (SCB)

M20 System Switch Board (SSB)

M5/10 FPC and SCB are combined into a single boardcalled the Forwarding Engine Board (FEB)

M7i/10i Compact FEB (CFEB)

M40e and M160 Switching and Forwarding Module (SFM). 4 SFM on M160, each one provides 25% of lookupcapability.2 SFM on M40e, only one can be active.

Enhanced System Boards:

- 2 ndgeneration Internet Processor II ASIC (not on M5/10 and M7i/10i)

- support 840K routing entries, double from old board 420K.

- Double on-chip memory to 16MB on IP II

- CPU memory 128 M for M40, 256M for M20, M40e and M160.

- Increased CPU speed to 256 MHZ.

- First shipped with JUNOS 5.5 Sep 2002.

Performance: 40 Mpps, 40 byte with 80K prefixes at routing table.

Packet processing features:

Filtering, sampling, logging, counting, load balancing

All M-series have enhanced S-board which as IP II ASIC. M5/10 doesnt have enhanced S-board.

T-series might contain as many as 16 IP II ASIC. Each FPC has one or two PFE which contains its own IP II ASIC.

What is it?

Collection of mechanisms on M-series and T-series

View System status messages

Trouble shooting

Where is it?

On the front of the chassis

What does it have?

System status LEDs

FPC/PIC online/offline buttons.

LCD screen provide status reporting for the entire system.

What alternatives on other platforms?

M7i: FIC (Fixed Interface Card)provide PIC offline/online buttons

M10i: HCM (High-Availability Chassis Manager) Card provide PIC offline/online bottons.

Connect to console

Power cycle the RE and watch it booting up

Enter a space character at the boot loader quick help manue to get a command prompt (dont enter space too quickly)

Enter boot s

When system boots up, answer recovery to recover password

Follow the on-screen steps to change password

Commit the change

Reboot the system again.

Step 1: Get the stack trace from syslog messages

lab@hissy> show log messages | find "machine check"

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 machine check caused by error on the PC

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 error detect register 1: 0x08, 2: 0x00

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 error ack count = 0

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 error address: 0x08004014

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 PCI bus error status register: 0x02

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 was the PCI master

Dec 5 01:51:17 hissy tnp_sfm_3 C/BE bits: I/O read [0b0010]

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 error detection reg1: PCI cycle

Dec 5 01:51:17 hissy tnp_sfm_3 mpc106 PCI status reg: parity error

Dec 5 01:51:17 hissy tnp_sfm_3 ^B

Dec 5 01:51:17 hissy tnp_sfm_3 last message repeated 7 times

Dec 5 01:51:17 hissy tnp_sfm_3 Registers:

Dec 5 01:51:17 hissy tnp_sfm_3 R00: 0x000e8c4c R01: 0x0775dad4 R02: 0x0000334

4 R03: 0x00000000

Dec 5 01:51:17 hissy tnp_sfm_3 R04: 0x0775dae0 R05: 0x00142e34 R06: 0x06006b3

6 R07: 0x00006b36

Dec 5 01:51:17 hissy tnp_sfm_3 R08: 0x00142e4c R09: 0x88000000 R10: 0x0000000

0 R11: 0x00000000

Dec 5 01:51:17 hissy tnp_sfm_3 R12: 0x00100004 R13: 0x000cc411 R14: 0x0000c43

0 R15: 0x00040000

Dec 5 01:51:17 hissy tnp_sfm_3 R16: 0x00000000 R17: 0x00041410 R18: 0x0004c42

0 R19: 0x8004c618

Dec 5 01:51:17 hissy tnp_sfm_3 R20: 0x0002c490 R21: 0x00110000 R22: 0x0000000

Juniper Confidential. For Internal use only.

0 R23: 0x001151cc

Dec 5 01:51:17 hissy tnp_sfm_3 R24: 0x00000001 R25: 0x00000000 R26: 0x0775db1

4 R27: 0x06006b36

Dec 5 01:51:17 hissy tnp_sfm_3 Stack Traceback:

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 01: sp = 0x0775dad4, pc = 0x000e8c4c

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 02: sp = 0x0775db0c, pc = 0x0005cd9c

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 03: sp = 0x0775db34, pc = 0x00108914

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 04: sp = 0x0775db4c, pc = 0x00108888

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 05: sp = 0x0775db54, pc = 0x000eec84

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 06: sp = 0x0775db5c, pc = 0x00037e78

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 07: sp = 0x0775dc1c, pc = 0x000380f8

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 08: sp = 0x0775dcfc, pc = 0x000eeadc

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 09: sp = 0x0775dd2c, pc = 0x000eefd0

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 10: sp = 0x0775dd3c, pc = 0x000f0184

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 11: sp = 0x0775dd74, pc = 0x000b28cc

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 12: sp = 0x0775dd84, pc = 0x000b29f4

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 13: sp = 0x0775ddac, pc = 0x000b2a8c

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 14: sp = 0x0775ddcc, pc = 0x000b2c80

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 15: sp = 0x0775ddec, pc = 0x000b2d5c

Dec 5 01:51:17 hissy tnp_sfm_3 Frame 16: sp = 0x0775de04, pc = 0x0002665c

What do I want? I will copy the following into a file called stack

single% cat stack