Doberman Installation guide.book

www.mellanox.com

BridgeX® GT Series BX5020 Gateway Platform IB to Ethernet/Fibre Channel Installation Guide

PN: MBX5020Rev 1.0

Mellanox Technologies350 Oakmead Parkway Suite 100 Sunnyvale, CA 94085 U.S.A.www.mellanox.comTel: (408) 970-3400Fax: (408) 970-3403

Mellanox Technologies, Ltd.PO Box 586 Hermon BuildingYokneam 20692IsraelTel: +972-4-909-7200Fax: +972-4-959-3245

© Copyright 2010. Mellanox Technologies. All rights reserved.Mellanox, BridgeX, ConnectX, Virtual Protocol Interconnect, InfiniBlast, InfiniBridge, InfiniHost, InfiniRISC, InfiniScale, and InfiniPCI are registered trademarks of Mellanox Technologies, Ltd. CORE-Direct, FabricIT, and PhyX are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners.All other marks and names mentioned herein may be trademarks of their respective companies.

Gateway Installation Guide

Document Number: 3227

Rev 1.0

Mellanox Technologies2

NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCU-MENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRON-MENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGE-MENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PROD-UCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Gateway Installation Guide Rev 1.0

Mellanox Technologies 3

Table of Contents

Table of Contents 3List of Figures 4List of Tables 5Revision History 6About this Manual 7

Intended Audience 7Related Documentation 7

Installing the Gateway in the Rack 8Tools and Parts 8

Grounding the Gateway 13Configuring the Gateway for the First Time 14FabricIT BXM 20

Downloading FabricIT Software and Documents 20

Starting an SSH Connection to the Gateway 21Starting a WebUI Connection to the Gateway 22Resetting the Gateway 23

List of FiguresRev 1.0


List of Figures

Figure 1: Rack Installation Kit Parts 8Figure 2: Which Side of the Rack Do You Want the Connectors? 9Figure 3: Which Side of the Rack Do You Want the Connectors? 10Figure 4: Making Room for the Power Cord 11Figure 5: Screwing on the Bracket 11Figure 6: Screwing on the Rail 11Figure 7: Placement of the Caged Nuts 12Figure 8: Put the Power Cord Through the Bracket 12Figure 9: Connect to the CONSOLE Interface 14Figure 10: Web User Interface 22Figure 11: Reset the Gateway 23



List of Tables

Table 1: Revision History 6Table 2: Reference Documents and Web Sites 7Table 3: Serial Terminal Program Configuration 14Table 4: Configuration Wizard Session - IP Configuration by DHCP 14Table 5: Configuration Wizard Session - IP Zeroconf Configuration 16Table 6: Configuration Wizard Session - Static IP Configuration 17

Revision HistoryRev 1.0


Revision History

Table 1 - Revision History

Rev Date Description of Changes

1.0 September, 2010 Initial release



About this Manual

This manual provides an installation instructions and initial configuration instructions to enable you to get the gateway up and running. For more detailed information about this product see the User Manual.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting up the gateway platform.

The manual assumes familiarity with the InfiniBand® architecture specification.

Related Documentation

The documentation set accompanying the QSFP gateway platform includes the following:

Table 2 - Reference Documents and Web Sites

Gateway Firmware and Firmware Update Tools See http://www.mellanox.com => Downloads => Firmware Tools. Note that the Gateway System described in this manual is based on Mellanox Technologies’ PhyX bridging device.

FabricIT BridgeX Management Software CLI User’s Manual

Talk to your Mellanox representative for information regarding licensing and implementation of the FabricIT BXM Software System.

Mellanox OFED Stack for Linux User’s Manual See http://www.mellanox.com => Downloads => InfiniBand SW/Drivers, Click “Mellanox OpenFabrics Enterprise Distribution for Linux (MLNX_OFED)” Select the Linux User’s ManualThe embedded OS and tools on the CPU in the management module is a subset of the Mellanox OFED stack.

This symbol indicates a recommendation.

This symbol indicates a Warning that may cause damage to property, loss of information, or personal injury.

Installing the Gateway in the RackRev 1.0


Installing the Gateway in the Rack

Tools and Parts

Tools required and customer supplied parts

Parts Included in the Installation Kit:

Figure 1: Rack Installation Kit Parts

Before you install your new gateway:• unpack the system • make sure that all the parts have been sent• check the parts in the box against the parts list to be sure that everything is there• check the parts for visible damage that may have occurred during shipping

Note: If anything is damaged or missing, contact your customer representative immediately.

• Phillips Screwdrivers #1 and #2 • Grounding screw• ESD Strap • Grounding wire sufficient to reach a valid ground• ESD mat

• 2 rails • 18 recessed flat head screws • 2 rail slides • 8 caged nuts • 2 brackets • 8 pan head screws M6

X18

Rail

Rail slide

Bracket



Procedure

The following procedure can be done by one person, but inserting the gateway is much easier with two people.1. Place the ESD mat on the floor where you will be working and put on the ESD strap. Make

sure the ESD strap is touching your skin and that the other end is connected to a verified ground.

2. Choose which side of the gateway you want even with the vertical rack support. Either the side with the power supply units or the side with the IB connectors can be even with the ver-tical rack support. The other side of the gateway will be further inside of the rack.

Things to consider before choosing where to mount the rails and rail slides.

Figure 2: Which Side of the Rack Do You Want the Connectors?……….

……….

The figure above shows the power side next to the door and the connector side away from the door. This configuration has more room for the cables and a larger bending radius.



Figure 3: Which Side of the Rack Do You Want the Connectors?

• Do you want the connector side recessed in the rack to allow for larger cable bending radius?• Will the connector side be recessed past other equipment in the rack and will this be problem-

atic?

Note: The rails are to be installed on the side of the rack where the gateway will be recessed into the rack.

3. Screw the brackets onto the gateway. Use the flat head screws to connect the bracket. There are two options for mounting the bracket. One option will place the gateway even with the vertical support of the rack and the second option will recess the gateway further into the rack.

The distance between the rack and the door can be as little as 4 cm on one side of the rack and as much as 18 cm on the other side of the rack. Keep in mind that there can be as many as 36 cables connected to the gateway.

……….

……….

The figure below shows the connector side next to the door and the power side away from the door. This configuration may be necessary to conform to your rack config-uration.



Figure 4: Making Room for the Power Cord

Figure 5: Screwing on the Bracket

Note: The side of the gateway with these brackets will be the side that is even with the vertical rack support.

4. Screw the rails onto the gateway. Use the 5 flat head screws to connect each rail to the gate-way.

Figure 6: Screwing on the Rail

Power cord

In this position you will use 4 flat head screws

In this position you will use 3 flat head screws

Place to put the power cable.

In this position the front of the gate-way is even with the rack vertical sup-port

In this position the front of the gateway is ~5cm behind the rack vertical support

The 5 screws



5. Clip 4 caged nuts into the holes in the rack you will be using to connect the rail slides. Check that both sides of the gateway, left and right, are in the same position number on the rack.

6. Clip 4 more caged nuts into the holes in the rack you will be using to connect the brackets. Check that both sides of the gateway, front and back, are in the same position number on the rack.

Note: The rail slides are to be installed on the side of the rack where the gateway will be recessed into the rack.

7. Using two of the bolts for each rail slide, install the rail slides to the rack. Tighten the bolts to 9.2 Nm or 81.5 pound inches.

Figure 7: Placement of the Caged Nuts

Figure 8: Put the Power Cord Through the Bracket

8. Place the four bolts for the caged nuts within reach. 9. Slide the gateway into the rails. This is easier with two people.10. Put the gateway into place and screw the bolts into the nuts from step 8. If the power cable is

on this side of the gateway feed the power cable into the slot in the bracket before screwing it to the vertical support.

11. Tighten all of the screws to 9.2 Nm or 81.5 pound inches.12. Plug in the power cables.

Note: When the gateway is plugged in, the status LED may be RED for up to three minutes until the system completes booting up.

13. Check the Status LEDs and confirm that all of the LEDs show status lights consistent with normal operation.

14. You can start connecting all of the cables to the gateway.

20 The caged nuts are separated by a single space



Grounding the Gateway

You must install an external ground to all IT components. To this purpose there are two threaded screw holes marked as grounds. Connect a ground wire to one of these grounds and connect the other end to a valid ground. Do not rely on the power connection ground. If you choose to not use the ground screw, make sure that there is a valid ground connection between the gateway and the grounded rack. Test the ground using an Ohm meter.

Note: Grounding wires, screws, and connectors for the ground are not supplied by Mellanox.

Configuring the Gateway for the First TimeRev 1.0


Configuring the Gateway for the First Time

1. Connect the host PC RS232 connector to the CONSOLE port of the gateway using the sup-plied cable.

Figure 9: Connect to the CONSOLE Interface

Note: No remote IP connection is available at this stage.

2. Configure a serial terminal program (for example, HyperTerminal, Minicom, or Tera Term) on your host PC with the settings described in Table 3.

3. Login (from serial terminal program) as admin and use admin as password. This starts the Mellanox configuration wizard.

4. Go through the Mellanox configuration wizard. Table 4 shows an example of a wizard ses-sion.

Table 3 - Serial Terminal Program Configuration

Parameter Setting

Baud Rate 9600

Data bits 8

Stop bits 1

Parity None

Flow Control None

Table 4 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 1 of 2)

Wizard Session Display (Example) Comments

Mellanox configuration wizardDo you want to use the wizard for initial configuration? yes

You must perform this configura-tion the first time you operate the gateway or after resetting the gateway password. Type ‘y’ and then press <Enter>.

Step 1: Hostname? [bridge-1] If you wish to accept the default hostname, then press <Enter>. Otherwise, type a different host-name and press <Enter>.

MGT CONSOLEMBX5020

< D

ENFC

1

2 3 4 5

Connect the host PC to here.



Step 2: Use DHCP on eth0 interface? Yes Perform this step to obtain an IP address for the gateway. (eth0 is the management port of the gate-way.)If you wish the DHCP server to assign the IP address, type ‘yes’ and press <Enter>.

If you type ‘no’ (no DHCP), then you will be asked whether you wish to use the ‘zeroconf’ config-uration or not. If you enter ‘yes’ (yes Zeroconf), the session will continue as shown in Table 5.

If you enter ’no’ (no Zeroconf), then you need to enter a static IP, and the session will continue as shown in Table 6.

Step 3: Admin password (Enter to leave unchanged)? <new_password>Step 4: Confirm admin password? <new_password>

To avoid illegal access to the machine, please type a password and then press <Enter>. Then confirm the password by re-entering it.

Note that password characters are not printed.

You have entered the following information:1. Hostname: <bridge-1>2. Use DHCP on eth0 interface? : yes3. Admin password (Enter to leave unchanged): (CHANGED)

To change an answer, enter the step number to return to.Otherwise hit <enter> to save changes and exit.

Choice: <Enter>

Configuration changes saved.To return to the wizard from the CLI, enter the “configuration jump-start” command from configuration mode. Launching CLI...

bridge-1 >

The wizard displays a summary of your choices and then asks you to confirm the choices or to re-edit them.

Either press <Enter> to save changes and exit, or enter the configuration step number (1, 2 or 3) that you wish to return to.

Note:To run the command “configuration jump-start”, you must be in Config mode.

Table 4 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 2 of 2)

Wizard Session Display (Example) Comments



Table 5 - Configuration Wizard Session - IP Zeroconf Configuration

Wizard Session Display - IP Zeroconf Configuration (Example)

Mellanox configuration wizard Do you want to use the wizard for initial configuration? y Step 1: Hostname? [bridge-112428]Step 2: Use DHCP on eth0 interface? [no]Step 3: Use zeroconf on eth0 interface? [no] yesStep 4: Default gateway? [192.168.10.1]Step 5: Primary DNS server?Step 6: Domain name?Step 7: Admin password (Enter to leave unchanged)?Step 8: Set BXM base port ID? [0] You have entered the following information: 1. Hostname: bridge-112428 2. Use DHCP on eth0 interface: no 3. Use zeroconf on eth0 interface: yes 4. Default gateway: 192.168.10.1 5. Primary DNS server: 6. Domain name: 7. Admin password (Enter to leave unchanged): (unchanged) 8. Set BXM base port ID? [0] To change an answer, enter the step number to return to.Otherwise hit <enter> to save changes and exit. Choice: Configuration changes saved. To return to the wizard from the CLI, enter the “configuration jump-start”command from configure mode. Launching CLI...

bridge-1 >



5. Before attempting a remote (for example, SSH) connection to the gateway, check the eth0 interface configuration. Specifically, verify the existence of an IP address. To check the cur-rent eth0 configuration, enter the following commands:

bridge-112428> enable // Enter ‘enable’ mode of CLI

bridge-112428# show interface eth0 // # indicates ‘enable’ mode

Table 6 - Configuration Wizard Session - Static IP Configuration

Wizard Session Display - Static IP Configuration (Example)

Mellanox configuration wizard Do you want to use the wizard for initial configuration? y Step 1: Hostname? [bridge-112428]Step 2: Use DHCP on eth0 interface? [yes] nStep 3: Use zeroconf on eth0 interface? [no]Step 4: Primary IP address? 192.168.10.4Mask length may not be zero if address is not zero (interface eth0) Step 5: Netmask? [0.0.0.0] 255.255.255.0Step 6: Default gateway? 192.168.10.1Step 7: Primary DNS server?Step 8: Domain name?Step 9: Admin password (Enter to leave unchanged)?Step 10: Set BXM base port ID? [0] You have entered the following information: 1. Hostname: bridge-112428 2. Use DHCP on eth0 interface: no 3. Use zeroconf on eth0 interface: no 4. Primary IP address: 192.168.10.4 5. Netmask: 255.255.255.0 6. Default gateway: 192.168.10.1 7. Primary DNS server: 8. Domain name: 9. Admin password (Enter to leave unchanged): (unchanged) 10. Set BXM base port ID? [0] To change an answer, enter the step number to return to.Otherwise hit <enter> to save changes and exit. Choice: Configuration changes saved.

To return to the wizard from the CLI, enter the “configuration jump-start”command from configure mode. Launching CLI...

bridge-1 >



The following is an example of the output:

Interface eth0 state

Admin up: yes

Link up: yes

IP address: 192.168.10.6

Netmask: 255.255.255.0

Speed: 100Mb/s (auto)

Duplex: full (auto)

Interface type: ethernet

Interface source: physical

MTU: 1500

HW address: 00:00:50:3A:8E:80

Comment:

RX bytes: 123102

RX packets: 1271

RX mcast packets: 0

RX discards: 0

RX errors: 0

RX overruns: 0

RX frame: 0

TX bytes: 122850

TX packets: 1119

TX discards: 0

TX errors: 0

TX overruns: 0

TX carrier: 0



TX collisions: 0

bridge-112428#

FabricIT BXMRev 1.0


FabricIT BXM

FabricIT BXM is a software based management system that can be run with either a command line interface or with a WebUI interface. Gateway management comes with the gateway system. See the FabricIT BXM User Manual for instructions and commands available to manage the gateway.

Downloading FabricIT Software and Documents

To download FabricIT software and documentation, please visit the following Web page:

FabricIT http://www.mellanox.com/content/pages.php?pg=FabricIT_BXM_login.

For information on installation, please refer to the FabricIT BridgeX Management Software CLI User’s Manual.



Starting an SSH Connection to the Gateway

Note: SSH connection also supports TelNet.

1. Set up an Ethernet connection between the gateway and a local network machine (“the remote machine” henceforth) using a standard RJ-45 connector.

2. Connect to the remote machine.3. Start a remote shell to the gateway using the following command:

rem_mach1 > ssh –l <username> <IP_address>

Mellanox FabricIT Bridge Management

Password:<password>

Last login: Thu Apr 30 17:33:14 2009 from x.x.x.x

Mellanox BridgeFabricIT BXM Management Software CLI User’s Man-ual

bridge-112428>

When prompted for a password enter “admin”.

Note that the IP address used above is the same IP address that was assigned to the Mellanox configuration wizard in the “Configuring the Gateway for the First Time” section.

4. You can enter any supported command now.

Note: For a complete reference of commands, please see FabricIT BXM Management Software CLI User’s Manual.

Starting a WebUI Connection to the GatewayRev 1.0


Starting a WebUI Connection to the Gateway

Perform the following steps to start a WebUI connection to the gateway.1. Set up an Ethernet connection between the gateway and a local network machine (“the

remote machine” henceforth) using a standard RJ-45 connector.2. Open a Web browse – Internet Explorer 7.0 or Mozilla Firefox 3.0.

Note: Make sure the screen resolution is set to 1024*768 or higher.

3. Type in the IP address of the gateway or its DNS name in the following format: http://<bridge_IP_address> where <bridge_IP_address> is the IP address of the gateway or its DNS name.

4. Enter your username and password to log in. Note that the default username is admin.5. Click Login.

Figure 10: Web User Interface

6. Use admin for both the login (Account) and the password.



Resetting the Gateway

Should the gateway stop working, try resetting the gateway. On the connector side panel under the system LEDs is a reset button. This reset button requires a tool to be pressed, a paper clip will do.

Figure 11: Reset the Gateway

This button resets the gateway when the button is pushed. A quick push of this button resets the whole gateway.When the button is held down for 5 seconds the gateway is reset and the password is reset to the factory default password. The LEDs will flicker for a fraction of a second to indicate reset is occurring.

The MBX 5020 Series User Manual can be found at: http://www.mellanox.com

=>Products => Gateways => Gateway Systems => BX 5020

DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset button. Sharp objects can cause damage, use a flat object such as a paper clip.

STATUSPSU 1PSU 2FAN

RST

Press the rest button.