BEST PRACTICES GUIDE:

Nimble Storage Best Practices for Networking

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 2

Contents

Network Connectivity ............................................................................................................................................... 3

Management Network ................................................................................................. 4

Data Network ................................................................................................................ 5

Choosing iSCSI Switches ........................................................................................................................................ 7

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 3

Network Connectivity

This section will help you to properly connect your Nimble arrays to a redundant Ethernet network

to ensure optimal performance and availability. Nimble Storage arrays are designed with

redundant controllers that provide high availability access to your storage in the event that the

active controller fails. In each of the associated diagrams you will see both solid lines and dashed

lines. The solid lines represent an active connection while the passive lines represent passive

connections that will become active in the event of a Nimble controller fail-over. It is also

important to wire each sibling interface on each controller to the same switch. For example,

Controller A, Eth1 connects to Switch 1 and Controller B, Eth1 also connects to Switch 1.

Best Practice: To make wiring easier, match Odd Numbered Ports with Switch 1 and Even

Numbered Ports with Switch 2. E.g. Odd-to-Odd and Even-to-Even.

Nimble Storage arrays are typically configured to connect to a management network and to data

networks.

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 4

Management Network

Management Network Diagram

Resiliency of the management network is important to permit access by administrators to the

Nimble Storage arrays for management purposes. The management network is typically wired to

the eth1 and eth2 ports that are located as labeled in the Management Network Diagram. The

Management IP Address can float between network ports that are designated as management

ports.

Management IP Screen

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 5

Data Network

In-general, you should configure the two stacked ports for management only which leaves the

remaining (4) 1 Gigabit ports or (2) 10 Gigabit ports available for the data network. While the

Nimble Storage management features permit mixing management and data networks, this

configuration is rarely needed and requires special care to configure properly. If you are unsure of

your networking needs, then you can contact Nimble Storage technical support for further

assistance.

Best Practice: If operating system attaching to Nimble Storage arrays permits the choosing

of load balancing algorithm for multi-path I/O, you should choose Least Queue Depth (LQD). The

Least Queue Depth algorithm is superior to Round Robin algorithms since it takes into

consideration pending I/O operations to avoid overloading a particular connection.

1 Gigabit Network Wiring

Best Practice: Enable Jumbo frames on data ports to maximize throughput. You must

enable Jumbo frames on each network connection including the Nimble array, the switch and

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 6

servers. Failure to enable Jumbo frames on one or more connections will not achieve the benefit

of the larger Ethernet frame size.

10 Gigabit Networking Wiring

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 7

Choosing iSCSI Switches

Network switches provide a critical part of an iSCSI storage area network. There are many

different classifications of switches and it is important to understand the characteristics that

make a switch good for supporting iSCSI storage traffic. Use the following table when evaluating

network switches:

Non-blocking Backplane A switch used for iSCSI data communication

should have a backplane that provides

enough bandwidth to support full duplex

connectivity for all ports at the same time. For

example, a 24 port Gigabit switch backplane

should provide at least 48 Gigabits per second

of bandwidth or (1 Gbps * 2 for Full Duplex *

24 Ports).

Flow Control (802.3x) Flow control provides a mechanism for

temporarily pausing the transmission of data

on Ethernet networks when a sending node

transmits data faster than the receiving node

can accept it. You should enable flow control

on all hosts, switch, and array ports to ensure

graceful communication between network

nodes. Nimble Storage array NICs have flow

control enabled by default.

Buffer Space per Switch Port Switches are used to provide communication

between hosts and arrays and for Nimble

Storage scaling communication between

arrays. Each switch port should have at least

512 Kilobytes of buffer memory per port to

ensure full performance between connected

nodes.

Support for Jumbo Frames Ethernet frames that transport data are

typically 1,500 Bytes in size. While this does a

good job of balancing application network

traffic between network clients and servers,

while host to storage communication tends to

be measured in Kilobytes. Jumbo frames were

created to better handle the flow of iSCSI SAN

traffic and consist of 9,000 Byte frames.

Enable Jumbo frames to improve storage

throughput and reduce latency.

B E S T P R A C T I C E S G U I D E : N I M B L E S T O R A G E B E S T P R A C T I C E S F O R N E T W O R K I N G 8

Can Disable Unicast Storm Control Storage traffic can appear bursty to switches

which can be mistaken by some switches as a

packet storm and blocked. Disabling Unicast

Storm Control ensures that the storage traffic

is transmitted unfettered.

Nimble Storage, Inc.

2740 Zanker Road, San Jose, CA 95134

Tel: 408-432-9600; 877-364-6253) | www.nimblestorage.com | info@nimblestorage.com

© 2012 Nimble Storage, Inc. All rights reserved. CASL is a trademark of Nimble Storage Inc. BPG-NET-0313