ISCSI Optimization

Force 10 MXL IO Module

Module 6 : iSCSI Optimization

Dell Force10 TAC Training2 Confidential 04/22/2023

iSCSI Optimization

Dell Force10 TAC Training

• iSCSI– TCP/IP-based protocol

(Layer3)– establishing and managing

connections between storage devices and initiators in a storage area network

• iSCSI Optimization– Monitor iSCSI sessions– Apply QoS policies on iSCSI

traffic– Snoop the creation and

termination of iSCSI connections

iSCSI Optimization Overview


iSCSI Optimization Functionality

• Auto-detection– active EqualLogic storage arrays

• Automatic configuration– switch ports after detection of storage arrays

• Manual configuration– detect Compellent storage arrays where auto-detection is not supported

• iSCSI monitoring sessions– Monitors and tracks active iSCSI sessions

› connections on the switch› port information› iSCSI session information

• iSCSI QoS– user-configured iSCSI class of service (CoS) profile

› applied to all iSCSI traffic

• iSCSI DCBX TLVs are supported


iSCSI Monitoring/Snooping

• iSCSI session monitoring (up to 256)– monitors iSCSI connections made between hosts and servers

› snooping iSCSI commands (defined in RFC 3720)– login– logout– login response– logout response

– iSCSI snooping ports› default: TCP ports 3260 and 860› user can configure it to snoop on non standard ports

– new session› system will raise a syslog with session details

– each TCP connection is identified by:› ISID (Initiator defined session identifier)› iSCSI qualified name

– Initiator’s IQN– Target’s IQN

› IP Address– Initiator– Target

› TCP Port– Initiator– Target

– view current iSCSI sessions through CLI command


iSCSI Class of Service (CoS)

• iSCSI CoS– Classifier rules are used to direct the iSCSI data traffic to queues

› given preferential QoS treatment over other data passing through the switch

› helps to avoid session interruptions during times of congestion that would otherwise cause iSCSI packets to be dropped.

– COS treatment should be applied to – iSCSI flows as described by iscsi target configuration

› support dot1p priority based profile with and without remark at egress› support DSCP based profile with and without remark at egress

– EqualLogic Inter-array traffic› (TCP ports 9876, 20002, 20003, 25555)› Fixed and not configurable


Topology Diagram

Navasota IOM’s StackedUp to 6 IOMs

S4810s/Nexus5K StackedUp to 3 S4810sUses 10G InterfacesUp to 8, 10G Stacking Cables

Stacked blades

Stacked Switches

DCB Ethernet

m1000e

EQL Arrays

Single LAG supportsfull bandwidth


Scalability and Known Limitations

• All Copy to CPU iSCSI packets – rate limited at 500 packets per second (pps)– unresolved ARP messages use queue 5

› may not learn all connections› may also miss FIN/RST packet for learnt session

– the session/connection information stays till we age out


Enabling iSCSI Optimization

Dell Force10 TAC TrainingConfidential10 04/22/2023

iSCSI Default Configuration

• iSCSI Enabled– Flow Control

› enabled on all interfaces– EQL auto-detection

› Jumbo Frames (MTU 12000)› No Storm Control› Spanning Tree

– port-fast

%STKUNIT0-M:CP %IFMGR-5-IFM_ISCSI_ENABLE: iSCSI has been enabled causing flow control to be enabled on all interfaces. EQL detection and enabling iscsi profile-compellent on an interface may cause some automatic configurations to occur like jumbo frames on all ports and no storm control and spanning tree port-fast on the port of detection

Dell Force10 TAC TrainingConfidential11 04/22/2023

iSCSI Default Configuration

• iSCSI Configuration– iSCSI Enabled (enabled by default)– iSCSI COS (enabled by default)

› dot1p or dscp– iSCSI TCP Ports

› 3260› 860

FTOS# show iscsi Display iSCSI Configuration

iSCSI is enablediSCSI COS : dot1p is 4 no-remarkSession aging time: 10Maximum number of connections is 256------------------------------------------------iSCSI Targets and TCP Ports:------------------------------------------------TCP Port Target IP Address3260860


Configuring iSCSI CoS

• iSCSI Class of Service (CoS)– dot1p

› remark– dscp

› remark

FTOS(conf)# iscsi dot1p <0-7> remark Configure iSCSI CoS dot1p

FTOS(conf)# iscsi dscp <0-63> remark Configure iSCSI CoS dscp

FTOS# show iscsi Display iSCSI Configuration



Configuring iSCSI Aging Time

• iSCSI Aging Time– Configure 5 – 43,200 minutes– Default: 10 minutes

FTOS(conf)# iscsi aging time <5-43200> Configure Aging Time (minutes)(default: 10)

FTOS(conf)# do show iscsi Display iSCSI Configuration



Add iSCSI TCP target port

• iSCSI TCP Ports (default)– 3260– 860

FTOS(conf)# iscsi target port TCP-RANGE Add iSCSI TCP target ports

FTOS(conf)# do show iscsi Display iSCSI Target TCP Ports

iSCSI Targets and TCP Ports:------------------------------------------------TCP Port Target IP Address32608604000

FTOS(conf)# iscsi target port TCP-RANGE ip-address

Add iSCSI TCP target ports / IP

FTOS(conf)# do show iscsi Display iSCSI Target TCP Ports

iSCSI Targets and TCP Ports:------------------------------------------------TCP Port Target IP Address32608604000 10.10.10.10


Manual iSCSI Profile (Compellent)

• iSCSI Profile-Compellent– enable auto-configuration when Compellent is on interface– MTU 12000 (Jumbo Frames)

FTOS(conf)# interface te 0/33 Enter Interface Config mode

FTOS(conf-if-te-0/33)# iscsi profile-compellent Configure iSCSI Compellent profile on interface

%STKUNIT0-M:CP %IFMGR-5-IFM_ISCSI_AUTO_CONFIG: This switch is being configured for optimal conditions to support iSCSI traffic which will cause some automatic configuration to occur including jumbo frames and flow-control on all ports; no storm control and spanning-tree port fast to be enabled on the port of detection

FTOS(conf-if-te-0/33)# show config Display Interface Configuration

!interface TenGigabitEthernet 0/33 no ip address mtu 12000 iscsi profile-compellent shutdown


Viewing iSCSI Optimization Information


View iSCSI Sessions

FTOS# show iscsi session Display iSCSI Sessions

Session 0: ----------------------------------------------------------------------Target: iqn.2001-05.com.equallogic:0-8a0906-0e70c2002-10a0018426a48c94-iom010 Initiator: iqn.1991-05.com.microsoft:win-x9l8v27yajg ISID: 400001370000 Session 1: ----------------------------------------------------------------------Target: iqn.2001-05.com.equallogic:0-8a0906-0f60c2002-0360018428d48c94-iom011 Initiator: iqn.1991-05.com.microsoft:win-x9l8v27yajgISID: 400001370000


View iSCSI Sessions detailed

FTOS# show iscsi session Display iSCSI Sessions

Session 0: ---------------------------------------------------------------------------------------- Target: iqn.2001-05.com.equallogic:0-8a0906-0e70c2002-10a0018426a48c94-iom010 Initiator: iqn.1991-05.com.microsoft:win-x9l8v27yajg Up Time: 00:00:00:16 (DD:HH:MM:SS) Time for aging out: 44 secs ISID: 400001370000 Initiator Initiator Target Target Connection IP Address TCP Port IP Address TCP Port ID192.168.2.125 49272 192.168.2.2 0 3260 3232 Session 1: ---------------------------------------------------------------------------------------- Target: iqn.2001-05.com.equallogic:0-8a0906-0f60c2002-0360018428d48c94-iom011 Initiator: iqn.1991-05.com.microsoft:win-x9l8v27yajg Up Time: 00:00:00:13 (DD:HH:MM:SS) Time for aging out: 47 secs ISID: 400001370000 Initiator Initiator Target Target Connection IP Address TCP Port IP Address TCP Port ID192.168.2.125 49274 192.168.2.20 3260 45


Troubleshooting and Debugging iSCSI Optimization


Issue: iSCSI auto-detection not happening• 1. Check is iSCSI task enabled ( this is applicable to all of iSCSI

feature)

iscsi cos enableForce10(conf)# iscsi cos enableForce10(conf)# do show iscsiiSCSI is enablediSCSI COS : dot1p is 4 no-remarkSession aging time: 10Maximum number of connections is 256------------------------------------------------iSCSI Targets and TCP Ports:------------------------------------------------TCP Port Target IP Address3260860FTOS#


Issue: iSCSI auto-detection not happening• 2. Check the “system description” TLV for hex string that states

it is Equallogic storage.

debug lldp interface tengigabitethernet 0/51 packet detail bothFTOS#debug lldp interface tengigabitethernet 0/51 packet detail both 2d22h35m : Recieved LLDP pkt on Te 0/51 of length 81 :2d22h35m : Packet dump:2d22h35m : 01 80 c2 00 00 0e 78 2b cb 69 96 4b 81 00 03 e72d22h35m : 88 cc 02 07 04 78 2b cb 69 96 4b 04 05 05 65 742d22h35m : 68 32 06 02 00 78 0e 04 00 80 00 80 0c 1d 44 652d22h35m : 6c 6c 20 45 71 75 61 6c 4c 6f 67 69 63 20 53 742d22h35m : 6f 72 61 67 65 20 41 72 72 61 79 00 00 62 1c 3a2d22h35m : 7e2d22h35m : TLV: Chassis ID, Len: 7, Subtype: Mac address (4) Value: 78:2b:cb:69:96:4b2d22h35m : TLV: Port ID, Len: 5, Subtype: Interface name (5) Value: eth22d22h35m : TLV: TTL, Len: 2, Value: 1202d22h35m : TLV: SYSTEM CAPAB, Len: 4, Existing: Station only, Enabled: Station only2d22h35m : TLV: SYSTEM DESC, Len: 29, Value: Dell EqualLogic Storage Array2d22h35m : TLV: ENDOFPDU, Len: 0


Issue: iSCSI auto-detection not happening• 3. If spanning-tree port state is not enabled, check “switchport”

is enabled on the interface

debug lldp interface tengigabitethernet 0/51 packet detail bothFTOS(conf)#interface tengigabitethernet 0/51FTOS(conf-if-te-0/51)#show config!interface TenGigabitEthernet 0/51no ip addressswitchportno shutdown


Issue: iSCSI auto-detection not happening• 4. Check, first detection of EQL/Compellent is already done

show loggingFTOS#show logging3d1h46m: %STKUNIT0-M:CP %IFMGR-5-IFM_ISCSI_AUTO_CONFIG: This switch is being configured for optimal conditions to support iSCSI traffic which will cause some automatic configuration to occur including jumbo frames and flow-control on all ports; no storm control and spanning-tree port fast to be enabled on the port of detection3d1h46m: %STKUNIT0-M:CP %LLDP-5-LLDP_EQL_DETECTED: EqualLogic Storage Array detected on interface Te 0/49


Issue: iSCSI session monitoring is not happening• Check L2/L3 setting are proper VLAN, subnets etc.• Check number of sessions, max session Navasota can monitor is 256 sessions.

show iscsi session | grep SessionFTOS#show iscsi session | grep SessionSession 0 :

• Check iSCSI sessions are getting created• Open shell prompt as mentioned in appendix• Execute following command in shell prompt “w iscsiOptDebug 0x8”• Exit shell prompt “ctrl+c” and execute command “exit”• Initiate iSCSI session from initiator and check for following debugs• 4. Check does iSCSI sessions are created on non-default tcp port ( 3260). If

it is on non-default add that port number in the iSCSI CoS table.• (debug output for cases 3 and 4 in next slide )


Issue: iSCSI session monitoring is not happeningFTOS#iscsiOptMain(): recv info on sockFdiscsiOptRecvTcpPkt(): __FUNCTION__IP Version: 4 HL: 5 TOS: 0IP Len: 56 ID: 43261 Off: 0IP TTL: 64 Proto: 6 Csum: bd49IP Src: 0xa0a0001 Dst: 0xa0a0065TCP Src: 35096 Dest: 3260 ----- iSCSI target TCP portTCP Seq: 584691560 Ack: 0TCP Rsvd: 0 Off: 9 Flags: STCP Sum: b4c2 URP: 0iscsiOptRecvTcpPkt(): f10RecvFrom len=74(< 122)iscsiOptMain(): recv info on sockFdiscsiOptRecvTcpPkt(): __FUNCTION__IP Version: 4 HL: 5 TOS: 0IP Len: 56 ID: 52382 Off: 0IP TTL: 64 Proto: 6 Csum: 99a8IP Src: 0xa0a0065 Dst: 0xa0a0001TCP Src: 3260 Dest: 35096TCP Seq: 409487739 Ack: 584691561TCP Rsvd: 0 Off: 9 Flags: S ATCP Sum: 3660 URP: 0

iscsiOptRecvTcpPkt(): f10RecvFrom len=74(< 122)iscsiOptMain(): recv info on sockFdiscsiOptRecvTcpPkt(): __FUNCTION__IP Version: 4 HL: 5 TOS: 0IP Len: 52 ID: 43262 Off: 0IP TTL: 64 Proto: 6 Csum: bd4cIP Src: 0xa0a0001 Dst: 0xa0a0065TCP Src: 35096 Dest: 3260TCP Seq: 584691561 Ack: 409487740TCP Rsvd: 0 Off: 8 Flags: A


Issue: iSCSI session monitoring is not happening• Check does any of the initiator initiated the session but it did not get login success response

from target• Follow the steps mentioned for debugging issue 3• Execute following command “iscsiDumpTcb”

iscsiDumpTcbf10ddb-> iscsiDumpTcbKey 10088IPC Key 65Task ID 0Service 279Instance 0EV Log Key 20052...Pending 0..


Issue: iSCSI CoS not happening

• 1. Check iSCSI CoS enabled

show iscsiFTOS#show iscsiiSCSI is enablediSCSI COS : dot1p is 4 no-remarkSession aging time: 10Maximum number of connections is 256------------------------------------------------iSCSI Targets and TCP Ports:-------------------------------------------TCP Port Target IP Address3260860FTOS#

• 2. Check tcp port and destination IP added in the iscsi COS table• Appendix:

• Start shell by executing command “start shell”(root/abracadabra31)• Attach iscsi process by executing command “f10appioclnt iscsiOpt”

Dell Force10 TAC Training28 For Internal Use Only04/22/2023

Lab 06.0iSCSI Optimization

Documents

ISCSI Optimization