43935463 N5k N2k Ethernet FCoE Lab Guide Student v2 1

Nexus 5000/Nexus 2000 Lab Guide

Overview This lab is designed to help attendees understand how to deploy Nexus 5000 and Nexus 2000 in the datacenter. It covers the configuration and management perspective for Nexus 5000 and Nexus 2000. Attendees will learn how to design and deploy

Lab List This guide includes these activities:

Lab 1: Basic configuration of N5k

Lab 2: Deploy N5k/N2k using EtherChannel

Lab 3: Deploy N5k/N2k using static pinning

Lab 4: Configuration of FCoE

Nexus 5000 and Nexus 2000 Overview

The Cisco Nexus™ 5000 Series Switches comprise a family of line-rate, low-latency, lossless 10 Gigabit Ethernet, Cisco® Data Center Ethernet, and Fibre Channel over Ethernet (FCoE) switches for data center applications. The switch family is highly serviceable, with redundant, hot-pluggable power supplies and fan modules. Its software is based on data center-class Cisco NX-OS Software for high reliability and ease of management.

The Cisco Nexus 2148T provides 48 Gigabit Ethernet server ports and 4 10 Gigabit Ethernet uplink ports in a compact one rack-unit (1RU) form factor. The Cisco Nexus 2000 Series Fabric Extender integrates with the parent switch, allowing "zero-touch" provisioning as well automatic configuration and software upgrade when a rack is connected to the parent switch. This integration allows large numbers of servers to be supported with the same feature set of the parent Cisco Nexus 5000 Series switch, along with specific configuration parameters including security and quality of service (QoS). The FEX does not run Spanning Tree Protocol even when multiple links are connected to the parent switch, thus giving customers loop-free, active-active connectivity between the two.

Lab Topology and Access

EveryoneortwostudentswillshareonePOD.EachonePODincludesoneNexus5000andoneNexus

2000switch.TheN2kisconnectedtointerfacee1/19ande1/20ofNexus5000.

Lab Topology

IP address and console assignment

Network: 128.107.65.192/27 255.255.255.224 Gateway 128.107.65.193 Terminal Server: 128.107.65.194 Servers IP Address Subnet Console APC APC Port Server-1 128.107.65.211 128.107.65.192 128.107.65.221 22,23 Server-2 128.107.65.212 128.107.65.192 128.107.65.221 20,21 Server-3 128.107.65.213 128.107.65.192 128.107.65.221 18,19 Server-4 128.107.65.214 128.107.65.192 128.107.65.221 15,17 Server-5 128.107.65.215 128.107.65.192 128.107.65.221 7,9 Server-6 128.107.65.216 128.107.65.192 128.107.65.221 5,6 Server-7 128.107.65.217 128.107.65.192 128.107.65.221 3,4 Server-8 128.107.65.218 128.107.65.192 128.107.65.221 1,2 N5k IP Address Subnet Console APC APC Port TM1 128.107.65.201 128.107.65.192 2033 128.107.65.220 23 TM2 128.107.65.202 128.107.65.192 2034 128.107.65.220 22 TM3 128.107.65.203 128.107.65.192 2035 128.107.65.220 21 TM4 128.107.65.204 128.107.65.192 2036 128.107.65.220 20 TM5 128.107.65.205 128.107.65.192 2037 128.107.65.220 19 TM6 128.107.65.206 128.107.65.192 2038 128.107.65.220 18 TM7 128.107.65.207 128.107.65.192 2039 128.107.65.220 17 TM8 128.107.65.208 128.107.65.192 2040 128.107.65.220 15 Core IP Address Subnet Console APC APC Port 6506-1 N/A N/A 2042 128.107.65.221 8 Terminal Server 128.107.65.194 login --> labops 128.107.65.220 6

Password: Terminal server Login password: labops. Enable password: nuovanuova Nexus 5000: admin/nbv12345 Windows Server: administrator/nuovanuova APC: apc/apc

Lab 1 Basic configuration of Nexus 5000

Lab Objective InthisexerciseyouwillconfigurethebasicconnectivitytoNexus5000andgetfamiliarwiththemostcommonlyusedNX‐OSCLI

Lab Procedure Step 1. Access the switch

Telnet to 128.107.65.194, sign on with a password of labops

Issue the show host command:

Nuova_TermServer>show host

[snip]

Host Port Flags Age Type Address(es)

tm1 2033 (perm, OK) 21 IP 1.1.1.1

tm2 2034 (perm, OK) 21 IP 1.1.1.1

tm3 2035 (perm, OK) 21 IP 1.1.1.1

tm4 2036 (perm, OK) 21 IP 1.1.1.1

tm5 2037 (perm, OK) 21 IP 1.1.1.1

tm6 2038 (perm, OK) 21 IP 1.1.1.1

tm7 2039 (perm, OK) 21 IP 1.1.1.1

tm9 2044 (perm, OK) 21 IP 1.1.1.1

tm8 2040 (perm, OK) 21 IP 1.1.1.1

tm6500 2042 (perm, OK) 21 IP 1.1.1.1

mds 2047 (perm, OK) 21 IP 1.1.1.1

Console in to the 5020 for your pod, for example, POD 1 would be TM1

Nuova_TermServer>tm1

Translating "tm1"

Trying tm1 (128.107.65.194, 2033)...

% Connection refused by remote host

Deleting login session

If you see the above error message you will need to get enable access to the terminal server and clear the line as shown below. The enable password is nuovanuova. If you do not see the error message above you can proceed to step 2

Nuova_TermServer>ena

Password:

Nuova_TermServer#clear line 33

[confirm] <- Hit enter

[OK]

Nuova_TermServer#tm1

Translating "tm1"

Trying tm1 (128.107.65.194, 2033)... Open Hit enter here

switch# <- Should see this prompt

Step 2. Configuretheswitchnameforthe5020inyourpod,managementVRF,enableSSHserver,andverifyconnectivitythroughthemanagementport.

switch(config)# hostname TM1 POD specific

TM1(config)# int mgmt 0

TM1(config-if)# ip address 128.107.65.201/24 Replace with the address for the 5020 in

TM1(config-if)# vrf context management

TM1(config-vrf)# ip route 0.0.0.0/0 128.107.65.193

TM1(config-vrf)# sh run int mgmt0

version 4.0(1a)N2(1)

interface mgmt0

ip address 128.107.65.201/24

TM1(config-vrf)# exit

TM1(config)# ssh key rsa 1024

rsa keys already present, use force option to overwrite them

TM1(config)# no ssh server enable

XML interface to system may become unavailable since ssh is disabled

TM1(config)# no ssh key

TM1(config)# ssh key rsa 1024

generating rsa key(1024 bits).....

.

generated rsa key

TM1(config)# ssh server enable

TM1(config)# end

Executeshowcommanddirectlyfromconfiguration

mode

DisableSSHanddeleteSSHkeyifthiserrormessageisprinted

Ping the default gateway for the management network.

TM1# ping 128.107.65.193 vrf management

PING 128.107.65.193 (128.107.65.193): 56 data bytes

64 bytes from 128.107.65.193: icmp_seq=0 ttl=254 time=2.167 ms





--- 128.107.65.193 ping statistics ---

5 packets transmitted, 5 packets received, 0.00% packet loss

round-trip min/avg/max = 1.748/1.984/2.167 ms

TM1#

Step3.TelnetorSSHtotheNexus5000inyourPODandgetfamiliarwithNX‐OSCLICheckthesoftwareversionandimagefilelocation

TM1# sh version

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Copyright (c) 2002-2009, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained herein are owned by

other third parties and are used and distributed under license.

Some parts of this software are covered under the GNU Public

License. A copy of the license is available at

http://www.gnu.org/licenses/gpl.html.

Software

BIOS: version 1.2.0

loader: version N/A

kickstart: version 4.0(1a)N2(1)

system: version 4.0(1a)N2(1)

BIOS compile time: 06/19/08

kickstart image file is: bootflash:/n5000-uk9-kickstart.4.0.1a.N2.1.bin

kickstart compile time: 2/25/2009 0:00:00 [02/25/2009 08:29:12]

NX‐OSsoftwareversion

Imagefilelocation

system image file is: bootflash:/n5000-uk9.4.0.1a.N2.1.bin

system compile time: 2/25/2009 0:00:00 [02/25/2009 08:56:57]

Hardware

cisco Nexus5020 Chassis ("40x10GE/Supervisor")

Intel(R) Celeron(R) M CPU with 2074308 kB of memory.

Processor Board ID JAB123900G8

Device name: TM1

bootflash: 974848 kB

Kernel uptime is 2 day(s), 22 hour(s), 51 minute(s), 8 second(s)

Last reset at 340459 usecs after Tue Mar 10 23:55:39 2009

Reason: Reset by installer

System version: 4.0(0)N1(2)

Service:

plugin

Core Plugin, Ethernet Plugin

ChecktheexpansionmoduleofNexus5000

TM1# show module

Mod Ports Module-Type Model Status

--- ----- -------------------------------- ---------------------- ------------

1 40 40x10GE/Supervisor N5K-C5020P-BFS active *

2 8 4x10GE + 4x1/2/4G FC Module N5K-M1404 ok

3 8 4x10GE + 4x1/2/4G FC Module N5K-M1404 ok

Mod Sw Hw World-Wide-Name(s) (WWN)

--- -------------- ------ --------------------------------------------------

1 4.0(1a)N2(1) 1.0 --

2 4.0(1a)N2(1) 1.0 20:41:00:0d:ec:a3:db:80 to 20:44:00:0d:ec:a3:db:80

Expansionmodules

3 4.0(1a)N2(1) 1.0 20:81:00:0d:ec:a3:db:80 to 20:84:00:0d:ec:a3:db:80

Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------

1 000d.eca3.db88 to 000d.eca3.dbaf JAB123900G8

2 000d.eca3.dbb0 to 000d.eca3.dbb7 JAB1250004E

3 000d.eca3.dbb8 to 000d.eca3.dbbf JAB12500048

TM1#

Lab 2 Deploy N5k/N2k with EtherChannel

Lab Objective InthisexerciseyouwilllearnhowtoaddNexus2000toNexus5000usingEtherChannelasconnectionbetweenNexus2000andNexus5000

Lab Procedure N2kismanagedbytheparentswitchN5k.AftertheN5kdiscoverstheconnectedN2kitfirstchecksthesoftwarecompatibilitybetweenN2kandN5k.TheconfigurationdataisthenpushedtoN2k.TheconfigurationismaintainedbyN5k.ThereisnolocalconfigurationdatastoredonN2k.N5kcollectsandmaintainthetrafficstatisticsaswell.

TherearetwowaystoconnectN2ktoN5k:

• Etherchannel:FirstistobundleallthelinksbetweenN2kandN5k(Upto4)asanEtherChannelandtheEtherChannelwillcarrythetrafficforallthe48N2kports.

• “Staticpinning”:EachofthelinksbetweenN5kandN2kcarriestrafficforagroupofN2kports.

InthislabexercisewewilllearnhowtoconnectN2ktoN5kusingEtherChannel

Step1AddN2krelatedconfigurationonN5kTM1(config)# fex 100

TM1(config-fex)# pinning max-links 1

TM1(config-fex)# description "fex 100 pod1"

TM1(config-fex)# int e1/19-20

TM1(config-if-range)# channel-group 100

TM1(config-if-range)# int port-channel 100

TM1(config-if)# switchport mode fex-fabric

TM1(config-if)# fex associate 100

TM1(config-if)# end

Step2CheckN2kstatusandverifythatN2kisupandonline.

DisplayallN2ksconnectedtothisN5k

TM1# sh fex

FEX FEX FEX FEX

Number Description State Model Serial

------------------------------------------------------------------------

EtherChannelmodeisON.NoLACPbetweenN5kandN2k

EtherChannelisconsidersas

onelogicallink

100 fex 100 TM1 Online N2K-C2148T-1GE JAF1244ATEQ

DisplaythedetailinformationaboutaN2k

TM1# sh fex 100 detail

FEX: 100 Description: fex 100 TM1 state: Online

FEX version: 4.0(1a)N2(1) [Switch version: 4.0(1a)N2(1)]

FEX Interim version: 4.0(1a)N2(1)

Switch Interim version: 4.0(1a)N2(1)

Extender Model: N2K-C2148T-1GE, Extender Serial: JAF1244ATEQ

Part No: 73-12009-02

Card Id: 70, Mac Addr: 00:0d:ec:b1:3e:82, Num Macs: 64

Module Sw Gen: 19 [Switch Sw Gen: 19]

pinning-mode: static Max-links: 1

Fabric port for control traffic: Eth1/1

Fabric interface state:

Po100 - Interface Up. State: Active

Eth1/1 - Interface Up. State: Active


Fex Port State Fabric Port Primary Fabric

Eth100/1/1 Down Po100 Po100




Eth100/1/5 Up Po100 Po100





Eth100/1/10 Down Po100 Po100

Eth100/1/11 Down Po100 Po100

Eth100/1/12 Down Po100 Po100

Eth100/1/13 Down Po100 Po100

Eth100/1/14 Down Po100 Po100

Eth100/1/15 Down Po100 Po100

Eth100/1/16 Down Po100 Po100

Eth100/1/17 Down Po100 Po100

Eth100/1/18 Down Po100 Po100

Eth100/1/19 Down Po100 Po100

Eth100/1/20 Down Po100 Po100

Eth100/1/21 Down Po100 Po100

Eth100/1/22 Down Po100 Po100

Eth100/1/23 Down Po100 Po100

Eth100/1/24 Down Po100 Po100

Eth100/1/25 Down Po100 Po100

Eth100/1/26 Down Po100 Po100

Eth100/1/27 Down Po100 Po100

Eth100/1/28 Down Po100 Po100

Eth100/1/29 Down Po100 Po100

Eth100/1/30 Down Po100 Po100

Eth100/1/31 Down Po100 Po100

Eth100/1/32 Down Po100 Po100

Eth100/1/33 Down Po100 Po100

Eth100/1/34 Down Po100 Po100

Eth100/1/35 Down Po100 Po100

Eth100/1/36 Down Po100 Po100

Eth100/1/37 Down Po100 Po100

Eth100/1/38 Down Po100 Po100

Eth100/1/39 Down Po100 Po100

Eth100/1/40 Down Po100 Po100

Eth100/1/41 Down Po100 Po100

Eth100/1/42 Down Po100 Po100

Eth100/1/43 Down Po100 Po100

Eth100/1/44 Down Po100 Po100

Eth100/1/45 Down Po100 Po100

Eth100/1/46 Down Po100 Po100

Eth100/1/47 Down Po100 Po100

Eth100/1/48 Down Po100 Po100

Logs:

[03/18/2009 00:01:47.921366] Module register received

[03/18/2009 00:01:47.922299] Registration response sent

[03/18/2009 00:01:47.950304] Module Online Sequence

[03/18/2009 00:01:49.252440] Module Online

TM1#

Step3ConfigureN2kinterfacesTM1(config)# vlan 10X Replace X with your POD number

TM1(config-vlan)# int ethernet 100/1/1

TM1(config-if)# switchport access vlan 10X Replace X with your POD number

Pleasenotethatthefexinterfacessupport802.1qtrunkingandcanbeconfiguredastrunkport.

Step4ConfigureinterfaceconnectedtoCatalyst6500TM1(config)# feature lacp

TM1(config)# interface port-channel 1

TM1(config-if-range)# exit

TM1(config)# interface ethernet 1/1-2

TM1(config-if-range)# channel-group 1 mode active

Ethernet1/1 Ethernet1/2 added to port-channel1

TM1(config-if-range)# interface port-channel 1

TM1(config-if)# switchport mode trunk

TM1(config-if)# switchport trunk allowed vlan 10X Again POD specific

TM1(config-if)# no shut

Verify the Port channel is up and both interfaces are up and active in the port channel

TM1# show port-channel summary

Flags: D - Down P - Up in port-channel (members)

I - Individual H - Hot-standby (LACP only)

s - Suspended r - Module-removed

S - Switched R - Routed

U - Up (port-channel)

-------------------------------------------------------------------------------

Group Port- Type Protocol Member Ports

Channel

-------------------------------------------------------------------------------

1 Po100(SU) Eth LACP Eth1/1(P) Eth1/2(P)

Step5Verifyserverconnectivity

FindouttheIPaddressoftheserverinyourPOD.RemoteDesktoptotheserverandassignIPaddressfortheNICportconnectedtoFEX.TheIPaddressfortheserveris100.100.X.10with“X”beingyourPODID.

FirststarttheNetworkconnectionwindowbyclickingthenetworkiconontherightbottomofthedesktop.

Thendoubleclickthenetworkconnectionnamed“Nexus2000”.

AssignIPaddresstothe1GENICportnamed“Nexus2000”

TheIPaddressshouldbe“100.100.X.10”“X”beingthePODID.

POD ServerNICaddress Catalyst6500address

1 100.100.1.10 100.100.1.1

2 100.100.2.10 100.100.2.1

3 100.100.3.10 100.100.3.1

4 100.100.4.10 100.100.4.1

5 100.100.5.10 100.100.5.1

6 100.100.6.10 100.100.6.1

7 100.100.7.10 100.100.7.1

8 100.100.8.10 100.100.8.1

PingtheCatalyst6500SVIipaddress:100.100.X.1

Step6SimulatethereplacementofN2kbyshutdown/noshutdownthefabriclinkbetweenN5kandN2k.InthisstepwewillsimulatethescenariowhereweneedtoreplaceanexistingN2kbydisconnectandreconnecttheN2ktoN5k.NotethattheconfigurationofN2karerestoredautomaticallyafterthenewN2kcomesupandonline.

ChecktheN2kinterfacesconfiguration

TM1(config-if-range)# sh run int ethernet 100/1/1


interface Ethernet100/1/1

spanning-tree port type edge

switchport access vlan 101

Shutdownallthefabricinterfaces

TM1(config)# int ethernet 1/19-20

TM1(config-if-range)# shut

2009 Mar 24 22:33:28 TM1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet1/2 is down

2009 Mar 24 22:33:28 TM1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet1/2 to Ethernet1/1

2009 Mar 24 22:33:28 TM1 %NOHMS-2-NOHMS_ENV_FEX_OFFLINE: FEX-100 Off-line (Serial Number JAF1241BLGA)

2009 Mar 24 22:33:28 TM1 %ETHPORT-5-IF_DOWN_MODULE_REMOVED: Interface Ethernet100/1/1 is down (module removed)





2009 Mar 24 22:33:31 TM1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet1/1 is down

2009 Mar 24 22:33:31 TM1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: port-channel1 is down

2009 Mar 24 22:33:31 TM1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet1/1 to none

CheckalltheN2kinterfacesarenolongexistonN5k.NX‐OSmaintainstheconfigurationsothatwhentheFEX100comesuponlinetheconfigurationwillberestored.


^

% invalid command detected at '^' marker.

Invalid range

Bringupthefabricinterfaces

TM1(config-if-range)# no shut

TM1(config-if-range)# 2009 Mar 24 22:33:59 TM1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet1/2 is up

2009 Mar 24 22:33:59 TM1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from none to Ethernet1/2

2009 Mar 24 22:34:00 TM1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet1/1 is up

2009 Mar 24 22:34:09 TM1 %SATCTRL-FEX100-2-SOHMS_ENV_ERROR: FEX-100 Module 1: Check environment alarms.

2009 Mar 24 22:34:10 TM1 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-100 On-line (Serial Number JAF1241BLGA)

ChecktheN2kcomesonlineandalltheN2kinterfacesrelatedconfigurationarerestoredautomatically



interface Ethernet100/1/1

spanning-tree port type edge

switchport access vlan 101

TM1(config-if-range)# sh int ethernet 100/1/1

Ethernet100/1/1 is up

Hardware is 1000 Ethernet, address is 000d.ecb1.2cc2 (bia 000d.ecb1.2cc2)

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA

Step7Checkthetrafficmappingbetweenfabricinterfaceandserverfacinginterfaces.TM1# sh interface Po100 fex-intf Fabric FEX Interface Interfaces --------------------------------------------------- Po100 Eth100/1/1 Eth100/1/2 Eth100/1/3 Eth100/1/4 Eth100/1/5 Eth100/1/6 Eth100/1/7 Eth100/1/8 Eth100/1/9 Eth100/1/10 Eth100/1/11 Eth100/1/12 Eth100/1/13 Eth100/1/14 Eth100/1/15 Eth100/1/16 Eth100/1/17 Eth100/1/18 Eth100/1/19 Eth100/1/20 Eth100/1/21 Eth100/1/22 Eth100/1/23 Eth100/1/24 Eth100/1/25 Eth100/1/26 Eth100/1/27 Eth100/1/28 Eth100/1/29 Eth100/1/30 Eth100/1/31 Eth100/1/32 Eth100/1/33 Eth100/1/34 Eth100/1/35 Eth100/1/36 Eth100/1/37 Eth100/1/38 Eth100/1/39 Eth100/1/40 Eth100/1/41 Eth100/1/42 Eth100/1/43 Eth100/1/44 Eth100/1/45 Eth100/1/46 Eth100/1/47 Eth100/1/48

Lab 3 Deploy N2k with Static Pinning WhenN2kisconnectedtoN5kwithstaticpinning,thetrafficof48serverfacingportaredistributedamongtheuplinksinafixedfashion.The48portsaredividedintothesamenumberofgroupsasthenumberofuplinksbetweenN2kandN5k.Trafficforeachsetoftheseportsiscarriedbyoneuplink,unliketheN2k/N5kdeploymentscenariowithPortChannelwheretrafficdistributionisdeterminedbythehashalgorithmresult.

Step1Firstremovetheconfigurationofpreviouslab.

TM1(config)# no int Po100

TM1(config-fex)# int e1/19-20

TM1(config-if-range)# no switchport mode

TM1(config-if-range)# no fex associate 100

TM1(config-if-range)# no fex 100

TM1(config)#

AddN2krelatedconfigurationonN5k

TM1(config)# fex 100

TM1(config-fex)# pinning max-links 2

Change in Max-links will cause traffic disruption.

TM1(config-fex)# description "POD1 FEX 100"

TM1(config-fex)# int ethernet 1/19-20

TM1(config-if-range)# switchport mode fex-fabric

TM1(config-if-range)# fex associate 100

Step2CheckN2kstatusandverifythatN2kisupandonline.TM1# sh fex 100

FEX: 100 Description: POD1 FEX 100 state: Online

FEX version: 4.1(3)N1(1) [Switch version: 4.1(3)N1(1)]

Extender Model: N2K-C2148T-1GE, Extender Serial: JAF1307BHCE

Part No: 73-12009-05

pinning-mode: static Max-links: 2

Fabric port for control traffic: Eth1/19

Fabric interface state:



TM1#

Step3ConfigureN2kinterfaces

ThisnumberhastomatchwiththeactuallinksbetweenN2kandN5k.

TM1(config)# vlan 10X Replace X with your POD number

TM1(config-vlan)# int ethernet 100/1/1

TM1(config-if)# switchport access vlan 10X Replace X with your POD number

Step4Checkthetrafficmappingbetweenfabricinterfaceandserverfacinginterfaces.TM1# sh interface ethernet 1/19 fex-intf Fabric FEX Interface Interfaces --------------------------------------------------- Eth1/19 Eth100/1/24 Eth100/1/23 Eth100/1/22 Eth100/1/21 Eth100/1/20 Eth100/1/19 Eth100/1/18 Eth100/1/17 Eth100/1/16 Eth100/1/15 Eth100/1/14 Eth100/1/13 Eth100/1/12 Eth100/1/11 Eth100/1/10 Eth100/1/9 Eth100/1/8 Eth100/1/7 Eth100/1/6 Eth100/1/5 Eth100/1/4 Eth100/1/3 Eth100/1/2 Eth100/1/1 TM1# sh interface ethernet 1/20 fex-intf Fabric FEX Interface Interfaces --------------------------------------------------- Eth1/20 Eth100/1/48 Eth100/1/47 Eth100/1/46 Eth100/1/45 Eth100/1/44 Eth100/1/43 Eth100/1/42 Eth100/1/41 Eth100/1/40 Eth100/1/39 Eth100/1/38 Eth100/1/37 Eth100/1/36 Eth100/1/35 Eth100/1/34 Eth100/1/33 Eth100/1/32 Eth100/1/31 Eth100/1/30 Eth100/1/29 Eth100/1/28 Eth100/1/27 Eth100/1/26 Eth100/1/25

Step5 VerifyserverconnectivitybypingtheSVIinterfaceofCatalyst6500“100.100.X.1”.Step6 Bringdownonefabricinterfaceandverifytheserverinterfacespinnedtothisfabricinterfacesarebroughtdown

TM1(config)# int e1/19

TM1(config-if)# shut

TM1(config-if)# sh int ethernet 100/1/1

Ethernet100/1/1 is down (fabricIfDown)

Hardware: 1000 Ethernet, address: 000d.ecc9.ec42 (bia 000d.ecc9.ec42)

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA

Port mode is access

auto-duplex, 1000 Mb/s

<snip> ….

TM1(config-if)#

Lab 4 Configuration of FCoE

Lab Objective

In this exercise you will configure basic connectivity to the switch, 802.1q trunking, LACP Port Channels, Fibre Channel Port Channels, and FCoE connectivity.

Note: The exercises in the lab were performed on POD 1. Please use the correct switches, VSANs, VLANs, IP addresses, and host associated with your assigned POD.

Lab Procedure

Continuing from the N5K/N2K lab, access the Nexus 5000 switch for your pod via the terminal server.

Step 1. In this step you will enable FCoE feature for the Nexus 5000 switch.

First turn on the FCoE feature. This will require a reboot of the switch to go into effect.

TM1# configure terminal

TM1(config)# feature fcoe

2008 Aug 13 22:56:01 switch %$ VDC-1 %$ %PFMA-2-FC_LICENSE_DESIRED: FCoE/FC feature will be enabled after the configuration is saved followed by a reboot

TM1(config)# copy running-config startup-config

[########################################] 100%

Packaging and storing to flash: \

TM1(config)# reload

Step 2. Creating a VSAN for the Fibre Channel Database

Once the switch comes back up, login into the switch and create the vsan needed for your POD. Please note that the convention used would be VSAN 20X, where X is your Pod #. Example would be for Pod1, would need to create VSAN 201, Pod 2 would be VSAN 202, etc.

TM1#configure terminal

TM1(config)# vsan database

TM1(config-vsan-db)# vsan 20X Again POD specific

TM1(config-vsan-db)# end

Step 3. Creating a SAN Port-Channel

TM1(config)# interface san-port-channel 10X Again POD specific

TM1(config-if)# interface fc2/3-4

TM1(config-if)# channel-group 10X force Again POD specific

fc2/3 fc2/4 added to san-port-channel 10X and disabled

please do the same operation on the switch at the other end of the channel, then do "no shutdown" at both ends to bring them up

Please note that the MDS9134 has already been initially configured to be in switching mode and the Port-Channels has been created on it as well. So the students nor instructor "DO NOT" need to do anything on the MDS switch.


TM1(config)# interface san-port-channel 10X Again POD specific


TM1(config-if)# end

Verify the SAN Port Channel is up and Trunking.

TM1# show interface san-port-channel 10X

san-port-channel 10X is trunking

Hardware is Fibre Channel

Port WWN is 24:0b:00:0d:ec:6d:90:40

Admin port mode is auto, trunk mode is on

snmp link state traps are enabled

Port mode is TE

Port vsan is 1

Speed is 8 Gbps

Trunk vsans (admin allowed and active) (1,20X)

Trunk vsans (up) (1,20X)

Trunk vsans (isolated) ()

Trunk vsans (initializing) ()

[snip]

TM1# show san-port-channel summary

--------------------------------------------------------------

Group Port Type Protocol Status Member Ports

Channel

-----------------------------------------------------------------

101 san-port-channel 101 FC NONE U fc2/3 (P)

fc2/4 (P)

-----------------------------------------------------------------

Step 4. Configuring a VLAN to be FCoE capable. This also does the VLAN to VSAN mapping.

Note: The VLAN number will be following the convention of VLAN 20X, where X is your Pod #. Example would be VLAN 201 for Pod 1, VLAN 202 for Pod 2, etc.

TM1# configure terminal

TM1(config)# vlan 20X Again POD specific

TM1(config-vlan)# fcoe vsan 20X Again POD specific

TM1(config-vlan)# show vlan fcoe

VLAN VSAN Status

-------- -------- --------

201 201 Operational

Step 5. Create the virtual Fibre Channel interface, and move it to the VSAN assigned to your pod.

TM1(config-vlan)# interface vfc5 The "vfc" number is not relevant and can be any number the student chooses. But for consistency, please use #5 for all pods. All student pods should have interface "vfc5" when this is completed.

TM1(config-if)# bind interface Ethernet1/5


Add Virtual Fibre Channel (vfc5) Interface to the VSAN database for your Pod.

TM1 (config-if)#vsan database

TM1 (config-vsan-db)#vsan 20X interface vfc5 Again POD specific

Step 6. Allow the FCoE VLAN to traverse interface ethernet 1/5 TM1(config-vsan-db)# interface ethernet 1/5

TM1(config-if)# switchport mode trunk

TM1(config-if)# switchport trunk allowed vlan 10X, 20X Again POD specific

TM1(config-if)# switchport trunk native vlan 10X Again POD specific

Step 7. Verify vfc interface is up Note: If the student does a "show interface vfc5", the vFC interface will be down awaiting the FCoE VLAN assignment, please be patient and the CNA port will come up.

vfc5 is down (STP not forwarding in FCoE Mapped VLAN)

Verify your CNA host is logged into the fabric and registered with the name server

TM1# show interface vfc5

vfc5 is up

Bound interface is Ethernet1/5

Hardware is GigabitEthernet

Port WWN is 20:04:00:0d:ec:b2:51:7f

Admin port mode is F

snmp link state traps are enabled

Port mode is F, FCID is 0x3c0001

Port vsan is 201

Beacon is turned unknown

5 minutes input rate 344 bits/sec, 43 bytes/sec, 0 frames/sec

5 minutes output rate 472 bits/sec, 59 bytes/sec, 0 frames/sec

983 frames input, 99240 bytes

0 discards, 0 errors

1089 frames output, 126152 bytes

0 discards, 0 errors

Step 8. Verifying zoning of the pod is active

TM1# show zoneset active vsan 20X

zoneset name pod1 vsan 201

zone name pod1-server vsan 201

* fcid 0x73001e [pwwn 21:00:00:04:cf:26:96:89] Disk

* fcid 0x3c0001 [pwwn 10:00:00:00:c9:85:05:48] CNA

Note: Below is the list of all the active zonesets for all the Pods.



* fcid 0x73001e [pwwn 21:00:00:04:cf:26:96:89]

* fcid 0x3c0001 [pwwn 10:00:00:00:c9:85:05:48]



* fcid 0xdb006e [pwwn 21:00:00:04:cf:67:74:ff]

* fcid 0x470001 [pwwn 10:00:00:00:c9:85:03:c8]



* fcid 0xa10002 [pwwn 22:00:00:20:37:f7:da:31]

* fcid 0x2b0001 [pwwn 10:00:00:00:c9:85:08:b0]



* fcid 0x4d0000 [pwwn 10:00:00:00:c9:85:00:d8]

* fcid 0xe10002 [pwwn 21:00:00:20:37:f7:da:31]



* fcid 0x3300cd [pwwn 50:00:00:e0:10:5c:24:e1]

* fcid 0x910001 [pwwn 10:00:00:00:c9:85:08:98]



* fcid 0x7000e2 [pwwn 22:00:00:04:cf:fe:4e:a0]

* fcid 0x280000 [pwwn 10:00:00:00:c9:85:01:f8]



* fcid 0xa80073 [pwwn 22:00:00:04:cf:67:47:fe]

* fcid 0x820001 [pwwn 10:00:00:00:c9:85:02:6c]



* fcid 0xd400e8 [pwwn 21:00:00:04:cf:fb:20:a9]

* fcid 0xdf0001 [pwwn 10:00:00:00:c9:85:02:71]

Step 9. Checking on the Server Pod sees the new SAN Disk. Use “Remote Desktop” to the server for your POD and verify the host can see storage

Step 10. Open Disk Management.

Step 11. Find the LUN allocated for your server based on the size.

Documents

43935463 N5k N2k Ethernet FCoE Lab Guide Student v2 1