14
1 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an Isilon node For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub. We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. _________________ ___________________________ Abstract This troubleshooting guide helps you to troubleshoot issues with the front panel of an Isilon node. January 15, 2018 EMC ISILON CUSTOMER TROUBLESHOOTING GUIDE TROUBLESHOOT ISSUES WITH THE FRONT PANEL OF AN ISILON NODE OneFS 7.1 – 8.1

EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

Embed Size (px)

Citation preview

Page 1: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

1 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Abstract

This troubleshooting guide helps you to troubleshoot issues with the front panel of an Isilon node.

January 15, 2018

EMC ISILON CUSTOMER TROUBLESHOOTING GUIDE

TROUBLESHOOT ISSUES WITH THE FRONT PANEL OF AN ISILON NODE

OneFS 7.1 – 8.1

Page 2: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

2 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Contents and overview

Before you begin

Page 3

Appendix A

If you need further assistance

Start troubleshooting

Page 4

Service lights

Page 5

Swap the front panels

Page 6

Restart the isi_lcd_d process

Page 9

Note Follow all of these steps, in order, until you reach a resolution.

1. Follow these

steps.

2. Perform

troubleshooting

steps in order.

3. Appendixes

Appendix B

How to use this flowchart

Restart the isi_lcd_d daemon

Page 10

Restart the isi_lcd_d process

Page 11

Page 3: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

3 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Configure screen logging through SSH

We recommend that you configure screen logging to log all session input and output during your troubleshooting session.

This log file can be shared with EMC Isilon Technical Support, if you require assistance at any point during troubleshooting.

Note: The screen session capability does not work in OneFS 7.1.0.6 and 7.1.1.2. If you are running either of these versions,

configure logging by using the local SSH client logging feature.

1. Open an SSH connection to the cluster and log in by using the root account .

Note: If the cluster is in compliance mode, use the compadmin account to log in. All compadmin commands must be

preceded by the sudo prefix.

2. Change the directory to /ifs/data/Isilon_Support by running the following command:

cd /ifs/data/Isilon_Support

3. Run the following command to capture all input and output from the session:

screen -L

This will create a file named screenlog.0 that will be appended to during your session.

4. Perform troubleshooting.

Before you begin

CAUTION!If the node, subnet, or pool that you are working on goes down during the course of

troubleshooting and you do not have any other way to connect to the cluster , you could

experience data unavailability.

Therefore, make sure that you have more than one way to connect to the cluster before

you start this troubleshooting process. The best method is to have a serial cable

available. This way, if you are unable to connect through the network, you will still be

able to connect to the cluster physically.

For specific requirements and instructions for making a physical connection to the

cluster, see article 304071 on the EMC Online Support site.

Before you begin troubleshooting, confirm that you can connect through either another

subnet or pool, or that you have physical access to the cluster.

Page 4: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

4 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Start troubleshooting

Remove the front panel from the

node, wait 30 seconds, and then

reseat the panel.

Start

IntroductionStart troubleshooting here. If you need

help to understand the flowchart

conventions that are used in this guide,

see Appendix B: How to use this

flowchart.

If you have not done so already, log in to

the cluster and configure screen logging

through SSH, as described on page 3.

Note For instructions on how to

remove a front panel, refer to the

Front Panel Replacement Guide

for your node type.

Is the front panel fully lit

and responsive after you

reseated it?

End troubleshooting

Go to Page 5

No

Yes

Page 5: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

5 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Service lights

Is the Isilon logo on the

front of the unit flashing and is

the Unit ID light on the back of

the node illuminated?

Run the following command on the

node with flashing lights for your

version of OneFS:

OneFS 8.0 – 8.1.0

isi servicelight disabled

OneFS 7.1 – 7.2

isi servicelight off

Yes

Go to Page 6

No

Page

5

You could have arrived here from:

Page 4 - Start troubleshooting

Is the front panel

fully lit and

responsive now?

End troubleshooting

Note the page number that you

are currently on.

Upload log files and contact Isilon Technical

Support, as instructed in Appendix A.

No

Yes

Page 6: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

6 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Swap the front panels

Is the front panel

functioning, except for

the decorative

lighting?

Remove a fully functioning front panel

from another node and install the

unresponsive front panel in its place.

No Yes

Page

6

You could have arrived here from:

Page 5 - Service lights

Is the

unresponsive front

panel lit and

responsive on the

new node?

No

Yes

Contact Support for a replacement panel.

Note the page number that you

are currently on.

Upload log files and contact Isilon Technical

Support, as instructed in Appendix A.

Go to Page 7

Note Decorative lighting is the LED strip below the

Isilon logo.

This indicates that the panel is

bad. Contact Isilon Technical

Support, as instructed in

Appendix A, to order a

replacement panel and continue

troubleshooting.

Page 7: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

7 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Swap the front panels (2)

Install the known good front panel

from the responsive node on the

problematic node.

Page

7

You could have arrived here from:

Page 6 - Swap the panels

Is the known good front

panel lit and responsive on

the problematic node?

No

Yes

Go to Page 8

End troubleshooting

Page 8: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

8 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Swap the front panels (3)

Page

8

You could have arrived here from:

Page 7 - Swap the panels (2)

Is the problem related to

the LCD control panel or

the alert LED?

No

Yes

Is the problem related to

one of the other LEDs?

Note the page number that you

are currently on.

Upload log files and contact Isilon Technical

Support, as instructed in Appendix A.

No

Go to Page 9Yes

Go to Page 10

Page 9: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

9 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Restart the isi_lcd_d daemon

Page

9

You could have arrived here from:

Page 8 - Swap the panels (3)

Open an SSH connection on the node

with the unresponsive front panel and

log in by using the root account.

Run the following command:

ps aux | grep isi_lcd_d

Does the output

look similar to the example

output shown on this

page?

Example ps aux | grep isi_lcd_d output

ps aux | grep isi_lcd_d

10141 2.0 0.1 28428 22352 ?? S 6:25PM 0:00.23 isi_lcd_d: manager (isi_lcd_d)

10140 0.0 0.1 22908 16656 ?? S 6:25PM 0:00.02 isi_lcd_d: server (isi_lcd_d)

10146 0.0 0.1 22908 16536 ?? S 6:25PM 0:00.00 isi_lcd_d: buttons (isi_lcd_d)

Go to Page 11

Go to Page 10

Yes

No

Restart the isi_lcd_d daemon by

running the following command:

isi_lcd_d start

Make note of the column values in bold

text. These are the process IDs (PIDs)

and you will need them for the next steps.

Page 10: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

10 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Restart the isi_lcd_d process

Page

10 You could have arrived here from:

Page 9 - Restart the daemon

Did the PIDs change from

when you ran the command

on the previous page?

Yes

No

Reboot the node:

reboot

Restart the isi_lcd_d process by

running the following command:

killall isi_lcd_d

Run the following command:

ps aux | grep isi_lcd_d

Compare the current PIDs with the PIDs from the output gathered on page 10.______

Return to Page 4

This indicates that the isi_lcd_d

process was successfully restarted.

Are the lights on

the front panel

fully functional?

No

End troubleshooting Yes

This indicates that the

isi_lcd_d process

might be hung.

Page 11: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

11 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Restart the isi_lcd_d process (2)

Page

11

You could have arrived here from:

Page 10 - Restart the daemon

Run the following command:

ps aux | grep isi_lcd_d

Does the output

look similar to the

example output shown

on this page?

Example ps aux | grep isi_lcd_d output

ps aux | grep isi_lcd_d

10141 2.0 0.1 28428 22352 ?? S 6:25PM 0:00.23 isi_lcd_d: manager (isi_lcd_d)

10140 0.0 0.1 22908 16656 ?? S 6:25PM 0:00.02 isi_lcd_d: server (isi_lcd_d)

10146 0.0 0.1 22908 16536 ?? S 6:25PM 0:00.00 isi_lcd_d: buttons (isi_lcd_d)

Reboot the node:

reboot

No

Is the front panel

fully lit and

responsive now?Yes

End

troubleshootingYes

No

Return to Page 4

Page 12: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

12 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Contact EMC Isilon Technical Support

If you need to contact Isilon Technical Support during troubleshooting, reference the page or step that you need help with.

This information and the log file will help Isilon Technical Support staff resolve your case more quickly.

Appendix A: If you need further assistance

Upload node log files and the screen log file to EMC Isilon Technical Support

1. When troubleshooting is complete, type exit to end your screen session.

2. Gather and upload the node log set and include the SSH screen log file by using the command appropriate for your

method of uploading files. If you are not sure which method to use, use FTP.

ESRS:

isi_gather_info --esrs --local-only -f /ifs/data/Isilon_Support/screenlog.0

FTP:

isi_gather_info --ftp --local-only -f /ifs/data/Isilon_Support/screenlog.0

HTTP:

isi_gather_info --http --local-only -f /ifs/data/Isilon_Support/screenlog.0

SMTP:

isi_gather_info --email --local-only -f /ifs/data/Isilon_Support/screenlog.0

SupportIQ:

Copy and paste the following command.

Note: When you copy and paste the command into the command-line interface, it will appear on multiple lines (exactly

as it appears on the page), but when you press Enter, the command will run as it should.

isi_gather_info --local-only -f /ifs/data/Isilon_Support/screenlog.0 --noupload \

--symlink /var/crash/SupportIQ/upload/ftp

3. If you receive a message that the upload was unsuccessful , refer to article 16759 on the EMC Online Support site for

directions on how to upload files over FTP.___________

Page 13: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

13 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Decision diamondYes No

Process stepProcess step with command:

command xyz

Go to Page #

Page

# Note Provides context and additional

information. Sometimes a note is linked

to a process step with a colored dot.

CAUTION!Caution boxes warn that

a particular step needs

to be performed with

great care, to prevent

serious consequences.

End point Document ShapeCalls out supporting documentation

for a process step. When possible,

these shapes contain links to the

reference document.

Sometimes linked to a process step

with a colored dot.

Optional process step

Directional arrows indicate

the path through the

process flow.

IntroductionDescribes what the section helps you to

accomplish.

You could have arrived here from:

Page # - Page title

Appendix B: How to use this flowchart

Page 14: EMC Isilon Customer Troubleshooting Guide: Troubleshoot Issues

14 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an

Isilon node

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Copyright © 2018 Dell Inc. or its subsidiaries. All rights reserved.

Dell believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS-IS. DELL MAKES NO

REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE

INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED

WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED IN THIS

PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE.

Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other

trademarks may be the property of their respective owners.

EMC Corporation

Hopkinton, Massachusetts 01748-9103

1-508-435-1000 in North America 1-866-464-7381

www.EMC.com