EMC Connectrix VDX -6740B Deployment Guide for EMC VSPEX ... · EMC CONNECTRIX VDX-6740B Deployment Guide for VSPEX BLUE with VMware®EVO:RAIL 53-1003742-05 3 Contents Connectrix

53-1003742-05

24 April 2015

EMC Connectrix VDX -6740B

Deployment Guide for

EMC VSPEX BLUE

(VMware® EVO:RAIL)

© 2015, Brocade Communications Systems, Inc. All Rights Reserved.

ADX, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, The Effortless

Network, VCS, VDX, Vplane, and Vyatta are registered trademarks, and Fabric Vision and vADX are trademarks of Brocade

Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be

trademarks of others.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any

equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document

at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be

currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in

this document may require an export license from the United States government.

The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to the

accuracy of this document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs that

accompany it.

The product described by this document may contain open source software covered by the GNU General Public License or other open

source license agreements. To find out which open source software is included in Brocade products, view the licensing terms applicable to

the open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.

Copyright © 2015 EMC Corporation. All rights reserved. Published in the USA.

EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. VMware

is a registered trademark of VMware, Inc., in the United States and other jurisdictions. © Copyright 2015 EMC Corporation. All rights reserved.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

http://www.brocade.com/support/oscd

http://www.emc.com/legal/emc-corporation-trademarks.htm

EMC CONNECTRIX VDX-6740B Deployment Guide for VSPEX BLUE with VMware® EVO:RAIL

53-1003742-05

3

Contents

Connectrix VDX-6740B-B Deployment Guide for VSPEX BLUE (VMware® EVO:RAIL)

Overview........................................................................................................... 4 Prepare and Plan Connectrix VDX-6740B Deployment…………………………6

Configuration Connectrix VDX-6740B............................................................... 9

Example Configuration............................................................................................................ 18

Basic Configuration Worksheet............................................................................................... 23

VLAN Worksheet......................................................................................................................24

Creating A Serial Connection...................................................................................................25

EMC CONNECTRIX VDX-6740B Deployment Guide EMC VSPEX BLUE with VMware® EVO:RAIL

53-1003742-05

4

Connectrix VDX- 6740B Deployment Guide for VSPEX BLUE

(VMware® EVO:RAIL)

Overview

The goal of this document is to guide an installer through the process of connecting EMC

VSPEX BLUE with EVO:RAIL appliances to EMC Connectrix VDX-6740B top-of-rack switches

by Brocade and configuring the switches to connect to an end customer's IP network.

VSPEX BLUE is a 2U 4 node hardware/software hyper-converged appliance running EMC VSPEX

BLUE Software and VMware® EVO:RAIL that combines VMware compute, networking, and

storage resources to create a simple, easy to deploy, all-in-one solution. VSPEX BLUE with

EVO:RAIL enables power-on to VM creation in minutes, radically easy VM deployment, easy non-

disruptive patch and upgrades and simplified scale-out.

In the current release, up to eight VSPEX BLUE appliances can be grouped together as a single

cluster serviced by a single Virtual SAN Datastore. Each VSPEX BLUE appliance contains four server

nodes and each server node has two 10GbE ports which connect to the VDX switches as

shown in Figure 1 below. This guide walks you through the steps to add a single cluster of VSPEX BLUE appliances to either one or two Connectrix VDX-6740B top-of-rack switches. Additional clusters may be connected to the Connectrix VDX-6740B switches. Both the planning the network for connecting the VSPEX Blue appliances to the switch and configuration steps are listed in the document in the following sections: ● Prepare and Plan Connectrix VDX-6740B Deployment……………………..................... 6 ● Configure Connectrix VDX-6740B...................................................................................9

If you are adding more than one cluster, print out multiple copies of Table 4 on page 24 and fill them

out separately for each cluster. For the second and additional clusters, refer to the VDX port

assignments shown in Table 1 on page 13 and Table 2 on page 14 and use the next available port

groupings for each additional cluster. For example, if you used VDX ports 1-4 and 33-36 for the first

cluster and the second cluster is the same number of EVO:RAIL appliances, you can use VDX ports

5-8 and 39-42.


53-1003742-05

5

FIGURE 1 EMC VSPEX BLUE with Brocade CONNECTRIX VDX-6740B Top-of-Rack switch solution

The VSPEX BLUE appliances must be connected to the IP network before they can be configured and

this requires both physical connectivity to the customer's network as well as some basic knowledge of

the customer's network.

Basic requirements include knowledge of

• The customer's default IP gateway address and subnet mask.

• Whether VDX switch management will be in-band or out-of-band, and the IP address for switch

management.

• The number and type of Ethernet ports required for VSPEX BLUE appliance connectivity.

• The number and type of Ethernet ports for the VDX inter-switch links (ISLs) which are used to

interconnect the switches to each other.

• The number and type of Ethernet ports for the VDX network uplinks used to connect the switches to

the customer’s existing network.


53-1003742-05

6

Prepare and Plan

The Connectrix VDX-6740B switch comes with 24 10GbE ports enabled. If more than 24 10GbE ports

are needed, or if 40GbE ports are needed, one or more license keys need to be installed on the VDX.

It is assumed the installer has the required license keys.

There are references to "Rbridge" and "Rbridge id" in the document. An Rbridge refers to a single Connectrix VDX-6740B switch, and rbridge-id refers to the unique switch identifier.

There are references to in-band and out-of-band management in the document. This refers to whether

switch management traffic is routed to an "out-of-band" management port on the switch (#5 in Figure 2

on page 7) or whether management traffic is routed "in-band" to one of the other Ethernet ports. In

this document, the serial console management port is used to configure the switch via a serial cable

attached to the laptop. After this initial configuration, the VDX switch will be configured for either in-

band or out-of-band management.

This guide requires no working knowledge of the Connectrix VDX-6740B, it walks the installer through

all of the required steps. Some of the configuration commands or terms may be unfamiliar to the

installer, but the guide is designed to walk the installer through the configuration step-by-step without

requiring any further knowledge of the commands or terms used. Simply following step-by-step

instructions and if you're not sure about a given value, use the recommended value.

Prepare and Plan Connectrix VDX-6740B Switch Deployment

This guide assumes that the installer has a basic familiarity with the VSPEX BLUE appliance

requirements, including how many appliances will be installed, and whether the EVO:RAIL

deployment will use VLANs or will be flat. The installer should also know how many ports are

available on the upstream switches that the Connectrix VDX-6740B will connect to and whether they

are 10GbE or 40GbE ports.

1. Prepare Installer Laptop

a. Serial console port: The laptop needs to connect to the console port of the Connectrix VDX-

6740B (port #4 in Figure 2). A serial cable with an RJ-45 connector is provided with the

Connectrix VDX-6740B for the purpose of connecting to the console port of the VDX. An

appropriate adaptor may be required for the laptop.

b. Serial console software: The installer will be required to perform the configuration via the serial

port. A terminal emulator program, for example "PuTTY" or "Screen," is required for console

access.

c. Must be able to set up and use the serial console port, using a terminal emulator program on

the laptop such as PuTTY or Screen.

d. For more detail on configuring the serial connection to the serial console port, refer to Creating

A Serial Connection on page 25 below.

FIGURE 2 CONNECTRIX VDX-6740B front side


53-1003742-05

7

2. Prepare Connectrix VDX 6740-B switch

a. NOS 5.0.1a or later must be installed

b. The VDX is expected to be in the "factory default" configuration. If it is not in the factory default

state, reset the VDX as follows. First, login to the switch, the defaults are:

Username = admin

Password = password

You can either do a complete reset to factory default, which erases all configuration and

installed license keys, or you can use an optional configuration parameter to reset all user

configurations except the management IP address and any installed licenses (such as

additional 10GbE or 40GbE port licenses).

Do one of the following two steps:

i. Perform a simple reset to factory default erasing all configuration and installed license

keys. Enter the global command: write erase

ii. Reset all user configurations except the management IP address and optional licenses.

Enter the global command: write erase rbridgeid <n>

Where <n> is the configured RbridgeID number. If you are not sure of the RbridgeID

number, you can display it using the show rbridge-id command.

c. Check if optional 10GbE or 40GbE licenses have been installed on the switch.

i. If a 10GbE or 40GbE license needs to be installed, apply the licenses to the CONNECTRIX

VDX-6740B using the instructions in next step. Otherwise skip to the next step.

ii. To install a license key, enter the license add licstr command followed by the license

key. The license key is printed between the XML start <licKey> and end </licKey> tags

in the license file

Be sure to copy the entire string, including spaces and non-alphanumeric characters. If the

license key includes spaces, you must enclose the entire string in double quotation marks.

Repeat this command for each software license key found in the license file. See the Figure

3 example below.

FIGURE 3 Adding a license key to a switch


53-1003742-05

8

Configure

3. Plan Connectrix VDX-6740B switch Networking settings for VSPEX Blue (EVO:RAIL)

To ensure correct functioning of VSPEX Blue appliance and end-to-end user experience the VLAN

and IP Address network settings on the Connectrix VDX-6740B switches need to support the

network requirement driven by VMware Loadmouth auto-discovery capability and requirements also

driven by Virtual SAN uses IPv4 multicast. The VLAN ID fields are pre-populated by the VSPEX

Blue (EVO:RAIL) appliance by default. These can be easily be changed on the VSPEX Blue

appliance during initial configuration and will need to be set on Connectrix VDX-6740B prior to

VSPEX Blue initial configuration using EVO:RAIL Manager.

NOTE

For steps 3a - 3e, use the worksheets in Table 3 on page 23 and Table 4 on page 24 to

record the answers.

5. How many VSPEX Blue (EVO:RAIL) appliances will be deployed? (default = 1, range = 1-8)

6. Will the network be flat or will it use VLANs? If flat, go to the next stop (c). If using VLANs,

you may need to ask the customer what their VLAN addressing scheme is. If they are not

sure, then use the defaults below including a single shared VM Network VLAN.

i. Management VLAN ID (default =1, the Native VLAN.)

Assign the name "Management" for this VLAN. The VDX Management VLAN must be

VLAN ID 1 because EVO:RAIL management traffic is untagged by default and needs

to be forwarded on the VDX's Native VLAN, which is VLAN ID 1. Traffic forwarded on

VDX VLAN ID 1 is also untagged by default.

ii. vSphere vMotion VLAN ID (default = 20)

Assign the name “vMotion” for this VLAN.

iii. Virtual SAN VLAN ID (default = 30)

Assign the name “VSAN” for this VLAN.

iv. VM Network VLAN

How many VM networks are required? (Optional, default = 1, range = 1 - 4K)

VM network VLAN naming and numbering can be defined by the customer, or if they

do not have a preference you can use the names suggested in this step.

As a best practice, VLAN names can be “VM Network A”, “VM Network B”, and

continue alphabetically. The VLAN ID's can start with 110, then increment in 10's. So

the second is 120, third is 130, etc. If a conflict exists with Management, vMotion or

Virtual SAN VLANs, then skip to next VLAN ID number.

7. How many Connectrix VDX-6740B switches are going to be provisioned? (range = 1-2).

Each switch needs uplink ports to connect to upstream switches in the network.

Note: For VSPEX Blue deployment with two redundant switches, you will also need to

interconnect them using “inter-switch link” ports (ISLs). If one switch, there is no need for

any ISL connections then to next step d.

i. If two switches will be provisioned, are the ISLs 10GbE or 40GbE? Both switches will

have the same ISL configuration.

(1) For 10GbE Inter-switch links, there are 2 to 8 ports in the trunk group (default =

2, range = 1-8).

(2) For 40GbE Inter-switch links, there are 1 to 2 ports are in the trunk group

(default = 2, range = 1-2)

8. How many 10GbE or 40GbE uplink ports are needed to connect to the upstream network?

Note: For VSPEX Blue deployments with two redundant switches, both switches will have

the same number or 10GbE or 40GbE uplinks ports from each switch.

i. Plan the number of 10GbE uplink ports (default = 2, range = 1 to 8)

or

ii. Plan the number of 40GbE uplink ports (default = 2, range - 1-4 minus any 40GbE

ports selected for ISL trunks)


53-1003742-05

9

Configure

e. Will the out-of-band VDX Ethernet Management port be configured, or will in-band management

be used instead? The out-of-band VDX Ethernet Management port is port #5 in Figure 2 above.

Note: You will need to ask the customer for the management IP address, subnet mask, and

gateway. The out-of- band management port is used when the customer deploys a separate out-

of-band management network.

i. You will need the VDX Management IP Address - this is required for either in-band or out-

of- band management

ii. You will also need the IP default gateway and subnet mask in CIDR notation for

management network. For example, 10.1.1.1/24.

Configure Connectrix VDX-6740B Switch

You will now configure the switch or switches using the parameters and values chosen and recorded in

Table 3 on page 23 and Table 4 on page 24 worksheets as explained in the steps below.

1. Power on the switch and log in

Connect the laptop's serial cable to the VDX console port, and log in. When you first log in to the

switch, the defaults are as follows:

Username = admin

Password = password

Enter the command configure terminal and then the global command:

username admin password <new-password>

2. Configure an Rbridge ID for each switch.

It is a mandatory switch identifier. Each switch needs a unique number. If you have two switches,

you can simply use 1 and 2. Enter the value(s) you used into the worksheet in Table 3 on page 23

below.

rbridge-id <number>

Where <number> is a unique number from 1-239.

3. Optionally, enter a name for the switch using the command

switch-attributes host-name <your chosen name>

4. In-band VDX management setup

Note: If out-of-band VDX switch management is to be used instead, skip to the next step.

Enter the IP default gateway address. From the rbridge-id 1 context, enter the management virtual

routing and forwarding context (VRF) by typing the command:

vrf mgmt-vrf

Next enter the IPv4 context by typing the command:

address-family ipv4 unicast

Enter the command:

ip route 0.0.0.0/0 <w.x.y.z>


53-1003742-05

10

Configure

Where <w.x.y.z> is the IP default gateway address you chose in Table 3 on page 23 below.

Enter the command: end

Next create the virtual interface (VE) that is to be used for in-band management and bind it to

management VLAN-ID 1:

interface ve 1

Add this interface to the management VRF created earlier:

vrf forwarding mgmt-vrf

Set the IP address and subnet mask for in-band management:

ip address <a.b.c.d>/<subnet mask>

Where <a.b.c.d>/<mask> is the IP address and subnet mask you chose in Table 3 on page

23. Enable the virtual interface:

no shutdown

The following is an example of the configuration above: rbridge-id 1 switch-attributes host-name foo vrf mgmt-vrf address-family ipv4 unicast ip route 0.0.0.0/0 10.5.1.1 !

! interface Ve 1 vrf forwarding mgmt-vrf ip proxy-arp ip address 10.5.1.2/24 no shutdown

!

5. Out-of-band VDX Management port setup

Note: If you have set up in-band VDX switch management, skip to the next step. For setting up the

out-of- band VDX Management port, follow the steps below.

Enter the command configure terminalthen enter the global command:

rbridge-id <enter value you used from the worksheet in Table 3>

Next enter the management virtual routing and forwarding context (VRF) by typing the command:

vrf mgmt-vrf

Enter the IPv4 context by typing the command:

address-family ipv4 unicast

Enter the command:

ip route 0.0.0.0/0 <w.x.y.z>

Where <w.x.y.z> is the IP default gateway address you chose in Table 3 on page 23 below.

Enter the command:

end

To return to the command prompt.

Enter the command configure terminal then enter the global command:

interface Management <rbridge-id number you configured in step 12>/0

Enter the command:


53-1003742-05

11

Configure

ip address <a.b.c.d>/<subnet mask>

Where <a.b.c.d>/<mask> is the IP address and subnet mask you chose in Table 3 on page 23

below.

Enter the command:

end

To return to the command prompt.

The following is an example of the out-of-band VDX management port setup.

rbridge-id 1 switch-attributes host-name foo vrf mgmt-vrf address-family ipv4 unicast ip route 0.0.0.0/0 10.5.1.1

interface Management 1/0 ip address 10.5.1.70/16

!

6. Configure the VSPEX BLUE appliance-facing Ethernet ports and VLAN interfaces

The number of Ethernet ports is based on the number of appliances and switches selected.

If the network uses VLANs, you must create each VLAN interface, then enable the Ethernet ports

and specify which VLANs are allowed to be included in each Ethernet port’s trunk, per Table 3 on

page 23 and Table 4 on page 24 below. The Management VLAN must be IPv4 IGMP snooping

and IPv6 MLD snooping enabled, and VSAN VLANs must be IPv4 IGMP snooping enabled. The

snooping querier function will also be enabled.

Note: If the network is flat, you do not need to configure additional VLANs.

Generally speaking, any Ethernet port can be used to connect to any host, appliance or upstream

switch. However for optimal performance, it is best to balance the use of the ports across the two

"Packet Buffer Banks", one from ports 1 - 32, the other from ports 33 - 52. Each packet buffer bank

has its own dedicated memory, so by balancing the use of ports across the two packet buffer banks

you take full advantage of the buffering capabilities of the switch. See Figure 4 below.

Use Table 1 and Table 2 to determine which Connectrix VDX-6740B switch ports to enable based on

the values chosen in Table 3 on page 23 worksheet. These port selections make optimal use of the

switches buffering capabilities.

FIGURE 4 CONNECTRIX VDX-6740B: port configuration and buffer banks


53-1003742-05

12

Configure

FIGURE 5 CONNECTRIX VDX-6740B Inter-Switch Link (ISL) trunk groups

You may deploy one or two switches for VSPEX BLUE. A single switch provides the lowest cost

option though it is a single point of failure. A single switch topology is not recommended if you

plan to deploy more than 4 VSPEX BLUE appliances. Two switches provide redundancy for high

availability and is the recommended approach.

Figure 6 is an example of a single switch topology and Figure 7 is an example of dual-switch

topology.

In the single switch topology, each node connects to a single VDX switch. The VDX may connect

to one or two upstream switches.

FIGURE 6 Example of a single switch topology

In the dual-switch topology, each node will connect to both switches. When two switches are

deployed, inter-switch links (ISLs) are needed. ISLs are used to interconnect the switches, you can

combine multiple Ethernet links to create trunks. These trunks are a bundle of Ethernet interfaces

and are automatically configured when they are connected. When creating a trunk, all of the

Ethernet ports used in the trunk must belong to ports in the same trunk group. See Figure 5 above.


53-1003742-05

13

Configure

The steps that follow guide you which ports to use based on the answers given in the worksheet

shown in Table 4 on page 24 below.

FIGURE 7 Example of a dual-switch topology

In the current version of VMware EVO:RAIL, you can have up to eight VSPEX BLUE appliances

represented as one cluster and serviced by one Virtual SAN datastore. Each VSPEX BLUE

appliance is comprised of four ESX hosts configured into a single cluster under the management of

vCenter Server, and serviced by a single Virtual SAN datastore. The following are key rules and

assumptions for deployment:

• Each VSPEX BLUE appliance has exactly eight 10GbE connections to the VDX top-of-rack switch.

• All VSPEX BLUE appliances are grouped into a single VSPEX BLUE cluster.

• Up to 8 VSPEX BLUE appliances can be deployed for a total of 32 server nodes.

Table 1 below defines the optimal VDX port selection for a single-switch topology. Table 2

below defines the optimal VDX port selection for a dual-switch topology.

TABLE 1

Single-switch topology: appliance-facing network port assignments

Number of VSPEX BLUE Appliances VDX Port Assignments

1 Ports 1-4,

33-36

2 Ports 1-8,

33-40

3 Ports 1-12,

33-44

4 Ports 1-16,

33-48


53-1003742-05

14

Configure

TABLE 2 Dual-switch topology: appliance-facing network port assignments

Number of VSPEX BLUE Appliances VDX Port Assignments

1 Ports 1-2,

33-34

2 Ports 1-4,

33-36

3 Ports 1-6,

33-38

4 Ports 1-8,

33-40

5 Ports 1-10,

33-42

6 Ports 1-12,

33-44

7 Ports 1-14,

33-36

8 Ports 1-16,

33-48

For the following steps, refer to the worksheets in Table 3 on page 23 and Table 4 on page 24

for the VLAN IDs and names you've chosen, and to Table 1 above for the port assignments.

7. Configure the Management VLAN interface

The Connectrix VDX-6740B switches use the default Native VLAN ID, using the command:

interface vlan 1

Then enter the VLAN name using the command:

name <VLAN name>

For the Management VLAN, you also need to enable IPv4 IGMP snooping, IPv6 MLD snooping,

and the snooping queriers. Enter the following commands:

ip igmp snooping enable

ip igmp snooping querier enable

ipv6 mld snooping enable

ipv6 mld snooping querier enable

Enter the command: exit

You are now returned to the top level configuration mode.

8. Disable tagging of untagged traffic on the Native VLAN (Management VLAN)

Enter the global command: no vlan dot1q tag native


53-1003742-05

15

Configure

9. Configure VLANs

interface vlan <x>

Where <x> is the name of the VLAN ID.

Then enter the VLAN name using the command:

name <VLAN name>

For the VSAN VLANs, you need to enable IPv4 IGMP snooping and the snooping querier. Enter

the following commands for those VLANs:

ip igmp snooping enable

ip igmp snooping querier enable

Enter the command: exit

To return to the top level configuration mode. Repeat the above steps for all VLAN

interfaces. The following is an example of this portion of the configuration.

! no vlan dot1q tag native ! interface Vlan 1 name Management ip igmp snooping enable ip igmp snooping querier enable ipv6 mld snooping enable ipv6 mld snooping querier enable ! interface Vlan 30 name VSAN ip igmp snooping enable ip igmp snooping querier enable

!

10. Configure the server-facing network interfaces

Recall that each switch has a unique Rbridge number, refer to the worksheet in Table 3 on page

23 for the values you chose. For CONNECTRIX VDX-6740B's, the slot number is always 0.

Configure the ports on the switch which connect to the VSPEX BLUE nodes based on the

values in Table 1 for a single switch topology and Table 2 for a dual-switch topology.

Enter the command:

interface tengig <rbridge-id/slot/port>

Note that to make port configuration quicker, you can configure a range of ports with one

command. For example to configure ports 1 – 4 enter the command:

interface tengig 1/0/1-4

Next put the port into trunk mode by entering the following two commands:

switchport

switchport trunk mode

11. Configure VLANs to Trunk

If the network is flat (no VLANs), go to next step

If using VLANs, refer to Table 4 on page 24 for the VLAN IDs and names you have chosen.

These are then added to the trunk with the following command. Management VLAN 1 is

forwarded by default and should not be included. Use a comma to separate multiple VLAN IDs.

switchport trunk allowed vlan add <vlan ID, vlan ID, vlan ID…>

12. Enabling Inter-Switch Links (ISL trunks).


53-1003742-05

16

Configure

You can use any of these ports of ISL connectivity, these are the ports that connect the switches

together.

Port Type Use Ports in Range

10GbE 17 - 24

40GbE 49 - 52

For ISLs, there is no configuration required. Just plug in the correct optics and cables and it will

automatically form the ISL.

13. Configure the network-facing uplink ports as LAGs

As shown in the example below. This example is shown for both 10GbE and 40GbE interfaces. You

will need to configure only the interfaces you are using for the uplink connections.

Port Type Use Ports in Range

10GbE 25 - 32

40GbE 49 - 52

Network-facing ports (uplinks) are configured as LAGs or as trunk ports. Typically, if you have an L2

network (no routing) with VLANs, you bundle the uplink ports into a LAG.

Link aggregation allows you to bundle multiple physical Ethernet links to form a single logical trunk

providing enhanced performance and redundancy. The aggregated trunk is referred to as a Link

Aggregation Group (LAG). The LAG is viewed as a single link by connected devices, the Spanning

Tree Protocol, IEEE 802.1Q VLANs, and so on. When one physical link in the LAG fails, the other

links stay up and there is no disruption to traffic. To configure links to form a LAG, the physical links

must be the same speed and all links must go to the same upstream switch unless the upstream

switches support a multi-chassis LAG capability, such as Cisco vPC, MLAG or Brocade VLAG.

Once you are in the config mode (config t), enter the following:

interface Port-channel <n>

Where <n> is the port-channel interface number. It can be any number from 1-6144. Next, put the

port-channel in trunk mode and bring it up by typing the following commands.

switchport

switchport mode trunk

no shutdown

Now, pick the interfaces from each switch (Rbridge1 and Rbridge2) which are connected to the

upstream switches (Switch A and Switch B) and configure them to belong to port-channel <n>.

Refer to Figure 7 above.

If you are using 10G interfaces for uplinks, you can configure them as shown below

interface tengig 1/0/31-32

channel-group <n> mode active

Where <n> is the port channel interface number.

If you are using 40G interfaces for uplinks, you can configure 40G ports 49-50 as shown below.

interface fortygig 1/0/49-50

channel-group <n> mode active

Where <n> is the port channel interface number.

Management VLAN 1 is forwarded by default. No other VLANs need to be forwarded on the

network-facing ports.


53-1003742-05

17

Configure

Here is an example of this section of the configuration where the port-channel number is 10.

! interface TenGigabitEthernet 1/0/31 channel-group 10 mode active type standard ! interface TenGigabitEthernet 1/0/32 channel-group 10 mode active type standard ! interface Port-channel 10 switchport switchport mode trunk no shutdown !

14. Set MTU

For best performance in an EVO:RAIL environment it is a good idea to set the interface MTU to the

largest value, also known as enabled "jumbo frames". However it is important that the upstream

switches and hosts connected to the VDX are also configured to the same interface MTU size. The

VDX supports a maximum MTU of 9216. Make sure to check the host and upstream network

interface settings. Not all hosts or switches can support full 9216 byte jumbo frames. If you are not

sure, skip this step and check later.

As an example, assume that your server's interface and upstream network switch ports are set to an

MTU of 9000. Then enter the command: mtu 9000

Enter the command end to return to the command prompt.

As a simple example, the following is an MTU configuration for an appliance facing trunk port and

network-facing uplink port. ! interface TenGigabitEthernet 1/0/1 mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 ! interface Port-channel 10 mtu 9000 switchport switchport mode trunk no shutdown !

15. Configure Buffer Performance Optimization

To optimize the switches forwarding performance for a VSPEX BLUE with EVO:RAIL environment,

you should allocate a little more buffering for bursty traffic.

Enter the command configure terminal. Then enter the global command:

rbridge-id <number>

Use the rbridge number you chose in Table 3 on page 23 below.

Enter the following two commands:

qos rcv-queue limit 2000

qos tx-queue limit 2000

Enter the end command to finish the configuration.

16. Save the Configuration.

This is the most important step!

copy running-config startup-config


53-1003742-05

18

Example Configuration


Consider the following appliance/switch combination. The VLAN configuration method was chosen

and each switch has two VM networks.

Number of VSPEX BLUE Appliances Number of Switches VDX Port Assignments

2 2 Ports 1-4, 33-36

In addition, there are two 10GbE uplink ports configured and connect two 10GbE ISLs in this

example configuration. Note that no configuration steps are needed for the ISLs, you just

plug them into the correct ports.

You can verify the VLAN configurations by issuing the show vlan brief command. The command

show fabric isl shows you ISL connectivity (for example below, Ten 1/0/23 and 1/0/24 are ISLs).

Two samples of the required VDX configuration commands on Rbridge1 are shown below, one for out-

of-band and one for in-band management. You can verify the configuration by issuing the show

running config command. Note this is the partial output of the show running config

command. It includes some default commands that were not entered in the steps above, for example

the fabric isl enable command is a default command shown on all interfaces regardless of

whether or not they are used as ISL ports.

Rbridge1 (with Out-of-band VDX management):

no vlan dot1q tag native ! interface Vlan 1 name Management ip igmp snooping enable ip igmp snooping querier enable ipv6 mld snooping enable ipv6 mld snooping querier enable

! interface Vlan 20 name VMOTION

! interface Vlan 30 name VSAN ip igmp snooping enable ip igmp snooping querier enable

! interface Vlan 110 name VM NETWORK_A

! interface Vlan 120 name VM NETWORK_B

! rbridge-id 1 switch-attributes chassis-name VDX6740 switch-attributes host-name VDX-TOR-1 vrf mgmt-vrf address-family ipv4 unicast ip route 0.0.0.0/0 10.5.1.1 ! address-family ipv6 unicast ! ! qos tx-queue limit 2000 qos rcv-queue limit 2000

! interface Management 1/0 vrf forwarding mgmt-vrf no tcp burstrate ip icmp unreachable ip icmp echo-reply no ip address dhcp ip address 10.5.1.70/24


53-1003742-05

19


ipv6 icmpv6 unreachable no shutdown

! interface TenGigabitEthernet 1/0/1 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/2 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/3 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/4 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/23 fabric isl enable fabric trunk enable no shutdown ! interface TenGigabitEthernet 1/0/24 fabric isl enable fabric trunk enable no shutdown ! interface TenGigabitEthernet 1/0/31 fabric isl enable fabric trunk enable channel-group 10 mode active type standard lacp timeout long no shutdown ! interface TenGigabitEthernet 1/0/32 fabric isl enable fabric trunk enable channel-group 10 mode active type standard lacp timeout long no shutdown ! interface TenGigabitEthernet 1/0/33 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120


53-1003742-05

20


switchport trunk tag native-vlan spanning-tree shutdown

no shutdown ! interface TenGigabitEthernet 1/0/34 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown

! interface TenGigabitEthernet 1/0/35 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown


! interface Port-channel 10 vlag ignore-split mtu 9000 switchport switchport mode trunk switchport trunk tag native-vlan spanning-tree shutdown no shutdown

!

Rbridge1 (with In-band VDX management):

no vlan dot1q tag native ! interface Vlan 1 name Management ip igmp snooping enable ip igmp snooping querier enable ipv6 mld snooping enable ipv6 mld snooping querier enable

! interface Vlan 20 name VMOTION

! interface Vlan 30 name VSAN ip igmp snooping enable ip igmp snooping querier enable

! interface Vlan 110 name VM NETWORK_A

! interface Vlan 120 name VM NETWORK_B

! rbridge-id 1 switch-attributes chassis-name VDX6740 switch-attributes host-name VDX-TOR-1 vrf mgmt-vrf address-family ipv4 unicast


53-1003742-05

21


ip route 0.0.0.0/0 10.5.1.1 ! address-family ipv6 unicast ! ! qos tx-queue limit 2000 qos rcv-queue limit 2000 ! interface ve 1 vrf forwarding mgmt-vrf ip proxy-arp ip address 10.5.1.2/24 no shutdown ! interface TenGigabitEthernet 1/0/1 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/2 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/3 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/4 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown ! interface TenGigabitEthernet 1/0/23 fabric isl enable fabric trunk enable no shutdown ! interface TenGigabitEthernet 1/0/24 fabric isl enable fabric trunk enable no shutdown ! interface TenGigabitEthernet 1/0/31 fabric isl enable fabric trunk enable channel-group 10 mode active type standard lacp timeout long no shutdown ! interface TenGigabitEthernet 1/0/32 fabric isl enable fabric trunk enable


53-1003742-05

22


channel-group 10 mode active type standard lacp timeout long

no shutdown ! interface TenGigabitEthernet 1/0/33 fabric isl enable fabric trunk enable mtu 9000 switchport switchport mode trunk switchport trunk allowed vlan add 20,30,110,120 switchport trunk tag native-vlan spanning-tree shutdown no shutdown




! interface Port-channel 10 vlag ignore-split mtu 9000 switchport switchport mode trunk switchport trunk tag native-vlan spanning-tree shutdown no shutdown

!


53-1003742-05

23

Basic Configuration Worksheet

TABLE 3 Basic Configuration Worksheet

Worksheet Setting Required Worksheet Value

Number of VSPEX BLUE Appliances yes

Number of VDX switches yes (at least one)

Number of 10GbE network-facing

ports (uplinks)

At least one uplink

Number of 40GbE network-facing

ports (uplinks)

At least one uplink

Number of 40GbE inter-switch links

(ISLs)

At least one ISL if using dual-switch

topology

Number of 10GbE inter-switch links

(ISLs)

At least one ISL if using dual-switch

topology

Management IP address for Rbridge

1

yes

Management IP address for Rbridge

2

yes

Management default gateway and

mask

yes

IP default gateway for in-band data yes

Switch 1 Rbridge ID yes

Switch 2 Rbridge ID yes


53-1003742-05

24

VLAN Worksheet

VLAN Worksheet

TABLE 4 VLAN Worksheet

Worksheet Setting Required VLAN ID VLAN Name

Management VLAN yes 1

vSphere vMotion VLAN yes

Virtual SAN VLAN yes

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

VM Network VLAN no

The table above shows entries for up to ten VM Network VLANs. If needed, more VM Network VLANs

can be added.


53-1003742-05

25

Creating A Serial Connection

You perform all configuration tasks using a serial connection from a workstation or terminal to the

switch.

Complete the following steps to create a serial connection to the switch:

1. Connect the serial cable to the serial port on the switch and to an RS-232 serial port on the

workstation or terminal device.

If the serial port on the workstation or terminal device is RJ45 instead of RS-232, remove the adapter

on the end of the serial cable and insert the exposed RJ45 connector into the RJ45 serial port on the

workstation.

2. Open a terminal emulator application (such as HyperTerminal on a PC, or TERM, TIP, or Kermit in a

UNIX environment), and configure the application as follows:

In a Windows environment, enter the following values: 9600 bits per second, 8 databits, no parity, 1

stop bit, and no flow control.

In a UNIX environment using TIP, enter the following string at the prompt:

tip /dev/ttyb -9600

If ttyb is already in use, use ttya instead.

The serial port is located on the port side of the CONNECTRIX VDX-6740B switches. The

CONNECTRIX VDX-6740B uses an RJ-45 connector for the serial port. An RJ-45 to DB9 adapter is

also provided with the switch. The cable supplied is a rollover cable.

NOTE

To protect the serial port from damage, keep the cover on the port when not in use. The serial port can

be used to connect to a workstation to configure the IP address for the CONNECTRIX VDX-6740B

before connecting the switch to a fabric or IP network. The serial port's parameters are fixed at 9600

baud, 8 data bits, and no parity, with flow control set to None.

TABLE 5 Serial cable pinouts

PIN Signal Description

1 Not supported NA

2 Not supported NA

3 TXD Transmit data

4 GND Logic ground

5 Not supported NA

6 RXD Receive data

7 Not supported NA

8 Not supported NA

Documents

EMC Connectrix VDX -6740B Deployment Guide for EMC VSPEX ... · EMC CONNECTRIX VDX-6740B Deployment Guide for VSPEX BLUE with VMware®EVO:RAIL 53-1003742-05 3 Contents Connectrix