-
Solution ID: emc223491 Date Created: 09/18/2009 Last Modified
Date: 02/07/2011
Here is the solution: 8 Gb link does not initialize to FA
adapter when using ARB(FF) as fill word on Brocade switches. The
Brocade 8 Gb-capable switches have the default fill words set to
IDLE. The fill word can be changed to ARB(FF) using a CLI command.
Once changed to ARB(FF), the link does not initialize when
connected to an EMC FA adapter. The link does come up when the fill
word is switched back to IDLE.
The EMC perferred idle mode is 3, ARB(ff)/ARB(ff) then
IDLE/ARB(ff).
The CLI command to check the fill word setting is:
portcfgshow
The CLI command to change the fill word is: portcfgfillword
This problem occurred on 8 Gb Brocade switches connected to 8 Gb
Open System and IBM z10 8 Gb FCP links. There are no issues using
either fill word on IBM 8 Gb FICON channels to a Brocade 8
Gb-capable switch. Here is the problem or goal: 8 Gb link does not
initialize to FA adapter when using ARB(FF) as fill word on Brocade
switches. The 8 Gb link will not come up to an EMC Fiber Adapter
(FA) Problem Environment: Product: Connectrix DS-300B Product:
Connectrix DS-5100B Product: Connectrix DS-5300B Product:
Connectrix ED-DCX-B Product: Connectrix ED-DCX-4S-B Director: Fiber
Adapter (FA) Product: Symmetrix VMAX Series Causes of this problem:
The T11 fiber committee changed the FC-PI-3 standard for 8 Gb
attachment on February 4, 2008. This change uses the ARB(FF)
primitive signal in place of IDLE as a fill word to maintain link
synchronization. During link initialization, 6 IDLE frames are sent
before sending ARB(FF) at port state LR3. Slower link speeds only
use the IDLE fill word and do not have this issue. Changes
affecting this problem: An 8 Gb capable Brocade switch was
installed. The default fill word was changed from IDLE to ARB(FF)
to comply with T11 standards. Additional Information: At FOS 6.3.1
there are now the three following options when using the
portcfgfillword CLI command:
0 | -idle-idle Sets IDLE mode in the Link Init and IDLE as the
fill word (default). 1 | -arbff-arbff Sets ARB(ff) in the Link Init
and ARB(ff) as the fill word. 2 | -idlef-arbff Sets IDLE mode in
the Link Init and ARB(ff) as the fill word. 3 | -aa-then-ia
Attempts hardware arbff-arbff (mode 1) first. If the attempt fails
to go into active state, this command executes software idle-arb
(mode 2). Mode 3 is preferable to modes 1 and 2 as it captures more
cases.
For the Symmetrix VMAX Series to eliminate problems with the
counter for er_bad_os increasing on the switch port when running at
8GB, set fill word to 3.
FICON_Pluto_upper_174-20:dennis> portcfgshow 2/3 Area Number:
67 Speed Level: AUTO(HW) Fill Word: 1(Arbff-Arbff) AL_PA Offset 13:
OFF
-
Trunk Port ON Long Distance LE VC Link Init ON Locked L_Port OFF
Locked G_Port OFF Disabled E_Port OFF ISL R_RDY Mode OFF RSCN
Suppressed OFF Persistent Disable OFF NPIV capability ON QOS E_Port
ON Port Auto Disable: OFF Rate Limit OFF EX Port OFF Mirror Port
OFF Credit Recovery ON F_Port Buffers OFF
FICON_Pluto_upper_174-20:dennis>
Script to disruptively change fillword to mode 3 for all ports
on card. Repeat command for each port card slot in switch.
for ((i=0;i
-
3. Clear the counters on the switch and check to see if the
"er_bad_os" field is incrementing. Example: #> statsclear #>
slotstatsclear #> portstatsshow 12/12 stat_wtx 0 4-byte words
transmitted stat_wrx 0 4-byte words received stat_ftx 0 Frames
transmitted stat_frx 0 Frames received stat_c2_frx 0 Class 2 frames
received stat_c3_frx 0 Class 3 frames received stat_lc_rx 0 Link
control frames received stat_mc_rx 0 Multicast frames received
stat_mc_to 0 Multicast timeouts stat_mc_tx 0 Multicast frames
transmitted tim_rdy_pri 0 Time R_RDY high priority tim_txcrd_z 0
Time TX Credit Zero (2.5Us ticks) tim_txcrd_z_vc 0- 3: 0 0 0 0
tim_txcrd_z_vc 4- 7: 0 0 0 0 tim_txcrd_z_vc 8-11: 0 0 0 0
tim_txcrd_z_vc 12-15: 0 0 0 0 er_enc_in 0 Encoding errors inside of
frames er_crc 0 Frames with CRC errors er_trunc 0 Frames shorter
than minimum er_toolong 0 Frames longer than maximum er_bad_eof 0
Frames with bad end-of-frame er_enc_out 0 Encoding error outside of
frames
er_bad_os 3456363056 Invalid ordered set=============>this is
incrementing extremely fast [output truncated..]
4. Change the Fill Word setting to mode 3. Reference EMC
Knowledgebase solution emc223491 "8 Gb link does not initialize to
FA adapter when using ARB(FF) as fill word on Brocade
switches".
5. Clear the counters and check the port counter to make sure
its not incrementing. You may need to block and unblock the port to
force a full re-login to get your N-port login. Example: #>
portdisable 12/12 #> portenable 12/12 #> statsclear #>
slotstatsclear #> portstatsshow 12/12 stat_wtx 0 4-byte words
transmitted stat_wrx 0 4-byte words received stat_ftx 0 Frames
transmitted stat_frx 0 Frames received stat_c2_frx 0 Class 2 frames
received stat_c3_frx 0 Class 3 frames received stat_lc_rx 0 Link
control frames received stat_mc_rx 0 Multicast frames received
stat_mc_to 0 Multicast timeouts stat_mc_tx 0 Multicast frames
transmitted tim_rdy_pri 0 Time R_RDY high priority tim_txcrd_z 0
Time TX Credit Zero (2.5Us ticks) tim_txcrd_z_vc 0- 3: 0 0 0 0
tim_txcrd_z_vc 4- 7: 0 0 0 0 tim_txcrd_z_vc 8-11: 0 0 0 0
tim_txcrd_z_vc 12-15: 0 0 0 0
-
er_enc_in 0 Encoding errors inside of frames er_crc 0 Frames
with CRC errors er_trunc 0 Frames shorter than minimum er_toolong 0
Frames longer than maximum er_bad_eof 0 Frames with bad
end-of-frame er_enc_out 0 Encoding error outside of frames
er_bad_os 0 Invalid ordered set=============>No longer
incrementing
Here is the problem or goal: VMware host not logging in as
N-port. VMware host intermittently loosing access to LUN's. Fill
Word Problem Environment: Product: Connectrix Brocade Switches All
System: VMware vSphere OS: VMware ESX
Solution ID: emc228910 Date Created: 12/04/2009 Last Modified
Date: 03/06/2012
Here is the solution: Brocade B-series: er_bad_os counter
incrementing when 8 Gb device is attached Issue portcfgfillword
command against port where 8 Gb device is to be attached.
In FOS 6.3.1 and later there are now the following options when
using the CLI portcfgfillword command:
0 | -idle-idle Sets IDLE mode in the Link Init and IDLE as the
fill word (default).
1 | -arbff-arbff Sets ARB(ff) in the Link Init and ARB(ff) as
the fill word.
2 | -idlef-arbff Sets IDLE mode in the Link Init and ARB(ff) as
the fill word.
3 | -aa-then-ia Attempts hardware arbff-arbff (mode 1) first. If
the attempt fails to go into active state, this command executes
software idle-arb (mode 2). Mode 3 is the preferable to modes 1 and
2 as it captures more cases. Here is the problem or goal: er_bad_os
counter incrementing Problem Environment: Product: Connectrix
ED-DCX-B Product: Connectrix DCX-4S-B Product: Connectrix DS-300B
Product: Connectrix DS-5100B Product: Connectrix DS-5300B Host
adapter: Emulex LPe12002-E Host adapter: Emulex LPe12000-E Host
Adapter 8 Gb HBA logged in at 8 Gb
-
Causes of this problem: The default implementation is to use
Idles for link initialization and fill words. Some newer 8 Gb
devices may experience problems with the use of Idle/Idle sequence
at 8 Gb. These devices may require the ARB (FF) / ARB (FF) sequence
to have a successful link initialization. Changes affecting this
problem: Newly installed 8Gb HBA / device Additional Information:
No errors seen in porterrshow Index Slot Port Address Media Speed
State Proto =================================================== 257
1 33 0fbf80 id 8G Online F-Port 10:00:00:00:c9:xx:xx:xx
portstatsshow 257 er_bad_os 3277982983 Invalid ordered set
/fabos/cliexec/porterrshow : frames enc crc crc too too bad enc
disc link loss loss frjt fbsy tx rx in err g_eof shrt long eof out
c3 fail sync sig
============================================================================
257: 2.8g 2.2g 0 0 0 0 0 0 0 0 0 0 0 0 0
This command is not applicable to a non 8G FC port. This command
disables and re-enables the port and the port comes online with the
new fill word setting. The configuration is stored in nonvolatile
memory and is persistent across switch reboots or power cycles.
