- 1. Fibre Channel over Ethernet (FCoE) John L Hufferd,
Brocade
2. SNIA Legal NoticeThe material contained in this tutorial is
copyrighted by the SNIA. Member companies and individuals may use
this material in presentations and literature under the following
conditions: Any slide or slides used must be reproduced without
modification The SNIA must be acknowledged as source of any
material used in the body of any document containing material from
these presentations. This presentation is a project of the SNIA
Education Committee. Fibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved.2 3.
Abstract A new concept is currently moving through theFibre Channel
(T11) standards committee calledFire Channel over Ethernet
(FCoE).The FCoE standard will specify theencapsulation of Fibre
Channel frames intoEthernet Frames and the amalgamation of
thesetechnologies into a network fabric that cansupport Fibre
Channel protocols and otherprotocols such as TCP/IP, UDP/IP etc.The
tutorial will show the Fundamentals of theFCoE concept and describe
how it might beexploited in a Data Center environment.Fibre Channel
over Ethernet (FCoE) 2008 Storage Networking Industry Association.
All Rights Reserved. 3 4. AgendaIntroductionGoals &
RequirementsConsolidationArchitectureDiscovery & Link
InstantiationTopologiesScenariosSummary Fibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved.4 5. Introduction This presentation provides an overview
of Fibre Channel over Ethernet (FCoE)One should think about FCoE as
placing the FC protocol on a new physical link Ethernet links
instead of physical FC links But it is still Fibre ChannelThe
protocol is being defined in the INCITS Fibre Channel (T11)
technical committeeMany details of the protocol still need to be
defined but the significant major issues have been resolvedTarget
T11 Standards completion is the 2H08 Fibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved.5 6. The Origins of Data Center Ethernet2008-2009
CEE(Converged Enhanced Ethernet) 10Gbps Increasing
PerformanceIntegration of Carrier-grade features 1Gbps Incremental
Protocol Enhancements100Mbps Logical Partitioning10MbpsEvolution
from shared media to dedicated mediaIntroduction of
Ethernet1973Increasing Scalability, Feature, Function The
technology has evolved continuously, showing a great ability to
adapt to new technologies and increasing business requirements
Fibre Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.6 7. Goals/RequirementsFibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.7 8. FCoE Goals/Requirements(part
1)FCoE Fabrics must be built with FCoE SwitchesSwitches with
Ethernet ports that provide FCoE capabilities andservices Called,
in the standard, an FC Forwarder FCFSwitches that include the
functions of traditional FC switchesStandard Ethernet switches may
also exist in the fabric but switcheswith FCoE capabilities are
required FCoE fabrics must operate seamlessly with real FC
FabricsFC services must operate identically on FCoE fabrics
andFibre Channel fabricsFCoE must support all Fibre Channel
advanced features(e.g. virtual fabrics, IFR, security, etc.)
transparentlyFCoE is NOT a replacement for FCIP or iFCPFCIP &
iFCP use TCP/IPFCIP/iFCP is for inter-switch links beyond the Data
CenterFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved. 8 9. Integrated Ethernet
& FCoE switch (FCF) with FC connections Some implementations
may combine the features and capabilities of an Ethernet Switch
with the features and capabilities of a FC switch Support Ethernet
and IP standards for switching, pathing and routing Support FC
standards for switching, pathing and routing CEE Ethernet Ports
Support current and enhanced Ethernet Standards (with IP & FCoE
VF_Port & FCFCFCVE_Port capabilities) Adapt between FCoE and FC
An FCoE Port (N_Port, F_Port or E_Port) Has the same function as in
FC But is layered on top of Ethernet Called VN_Port, VF_Port and
VE_PortBecause many logical (virtual)Ports can share one physical
port Note: FCF Pathing and Forwarding utilizes the FSPF (Fabric
Shortest Path First) protocol Non-FCoE Ethernet traffic is relayed
using conventional 802.1 defined mechanisms such as STP (Spanning
Tree Protocol) and MSTP (Multiple STP)See additional FSPF and STP
explanation in Appendix Fibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved.9 10.
Connections to a Combo FCoE Switch Fibre Channel is carried over
lossless Ethernet as a L3 protocol SCSI SCSIApplications iSCSIFibre
ChannelUDPTCPIPFCoELossless Ethernet MAC (CEE)IP address
123.45.67.89(FCoE VN_Port)Ethernet port with IP & FCoE VF_Port
capabilities Combo Lossless Ethernet (CEE) Switchwith FCoE Switch
(FCF) capabilities Fibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved.10 11. FCoE
Goals/Requirements (part 2)FCoE requires specific Ethernet
extensions to be implementedLossless switches and fabrics (e.g.,
supporting IEEE 802.3 PAUSE)configurations are requiredJumboframe
support is required (not a standard, but widely
available)Deployments of FCoE should utilize the advances in
Ethernet currently being discussed in IEEE 802.1,
specifically:Priority-based Flow Control (PFC)Enhanced Transmission
Selection (ETS)DCB Capability eXchange Protocol (DCB CXP)These
802.1 advances are important for Consolidated Flows (Messaging,
Clustering and Storage)This set of functions has been called Data
Center Ethernet, or CEE Converged Enhanced Ethernet (intended for a
Data CenterEnvironment)FCoE should require no changes to FC
software Fibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved.11 12. Consolidation
Fibre Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved. 12 13. High End 10GE Server &
NIC/HBA Consolidation OS1 Web ServerOS2 App ServerOS3DB ServerWith
CEEHyper Visor (vmWare, Xen, etc) OS2 FC OS2OS3App ServerHBAApp
ServerDB ServerOS1OS1 OS3Web Server Web ServerDB Server Messaging
EthernetFC SANNICE-HBA E-HBA TCP acceleration. TCP/IP (CEE)
E-HBA(CEE) MPI, RDMA(CEE) MPIRDMA over EthernetFCoE Today
IB/Ethernet ClusterData CenterCEE NetworkData CenterCEE
NetworkDramatic reduction in adapter, switch ports and cabling 4-6
cables to 2 cables per server Seamless connection to the installed
based of existing SANs and LANs Requires high performance
lightweight frame mappers vs. heavy weight gateways FCoE has no
need to terminate and re-initiate a SCSI connection (e.g. iSCSI to
FC) Effective sharing of high bandwidth linksFibre Channel over
Ethernet (FCoE) 2008 Storage Networking Industry Association. All
Rights Reserved. 13 14. Architecture Fibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved. 14 15. FC Encapsulation Into Network Packets (2 FCoE
Related Packet types)FC-4FC-4 FC LevelsFC-3FC-3
(Unchanged)FC-2FC-2FCoE MappingFC-1MAC IEEE 802.3FC-0PHY
LayersFibre Channel over Ethernet (FCoE) Packets EthernetFCoE FC
Header SCSI Commands/Data FCSHeaderHeaderFC Imbedded Frames: Same
as in Physical FC Ethertype Protocol control information: Version,
SOF, EOF, etc.FCoE Frame Check Sequence Ethernet Header provides
things needed for the(CRC) physical network, including
EthertypeFCoE Initialization Protocol (FIP) PacketsEthernetFIP
HeaderDescriptors FCS Header Discovery and Login/Logout
ParametersEthertypeProtocol control information: Version, Op-codes,
etc.FIPFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved. 15 16. FIP Protocol and
FCoE Protocol Discovery Phase FCFs Discovery each other and form a
Fabric ENodes and FCFs DiscoverPotential VN_Port VF_Port
pairingCapabilities of Potential pairingLogin PhaseFCoE ENodes
chose among discovered FCFs Ports Initialization Creates
association between ENode Ports and FCF Ports Protocol
(FIP)VN_PortVF_Port Logical FC Link Two allowed alternatives for
the ENode MAC AddressingFabric Provided MAC Addresses (FPMA)Server
Provided MAC Addresses (SPMA)Chosen by FCF (FPMA & SPMA
Described latter) Uses: FLOGI, FLOGI ACC, LOGO, Data Transfer
PhaseFCoE Protocol PLOGI/PRLI All other FC protocol frames (ELS,
FC4 ULPs. etc.) Fibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved.16 17. FIP
Operation FormatWord 31-2423-1615-8 7-0 0 Destination MAC Address
(6 Bytes) 1 Optional IEEE 802.1q2 Source MAC Address (6 Bytes) 4
Byte Tag goes here 3 ET=FIP (16 bits) Ver (4b)Reserved (12
bits)4FIP Operation CodeReservedFIP Operation Code FIP SubCode See
Appendix forDescriptor list items Reserved List Length Descriptor
FIP subcodeFlagsDescriptor List Length 5 FP SP S F Flags F S S F 6P
P Descriptor list varies Descriptor List In size nPAD to minimum
length or mini-Jumbo lengthEthernet frame size n+1 Ethernet FCS Is
64Bytes to 2220BytesCapability Bits (SPMA or FPMA)Solicited bitFCF
bit Fibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved.17 18. FCs Encapsulation
in Ethernet (FCoE)Word 31-24 23-16 15-87-0 0 Destination MAC
Address (6 Bytes) 1 Optional IEEE 802.1q2Source MAC Address (6
Bytes) 4 Byte Tag goes here 3 ET=FCoE (16 bits)Ver (4b)Reserved (12
bits)4Reserved 5Reserved 6 ReservedSOF (8 bits) 7 Encapsulated FC
Frame This field varies FC Frame = Minimum 28 Bytes (7 Words)In
sizeMaximum 2180 Bytes (545 Words) n(including FC-CRC)n+1EOF (8
bits)ReservedEthernet frame n+2Ethernet FCSsizeIs 64Bytes to
2220BytesFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved. 18 19. FCF ModelLinkEnd
Point (LEP) Fibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved.19 20. ENode
(HBA) Model Link EndPoint(LEP)Each ENode (HBA) may have multiple
Physical Ethernet PortsEach Physical Port may have multiple Logical
VN_PortsFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved. 20 21. Model of the
ENODE with MultipleLogical FC interfaces In this Multiple FC NPIV
model this instances on a singleFC-3 /FC-4sis where logical FC Host
interface FC-3 /FC-4sFC-2functions For each logical N_Port
live(VN_Port) there is one FLOGIIn this model this and perhaps 100s
of FDISCVN_PortFC...VN_Port FCis where the Entity
EntityEncapsulation /De-Encapsulation Each VN_Port is seen by
FCoE_LEP FCoEFCoE_LEPFCoEfunctions live Entity the Host as a
separate Entity FCoE (logical) FC connection Controller MAC Address
ofFCoE_LEP (VN_Port) May or may not be The number of (logical)
FCLossless Ethernet MACEthernet_Portthe same as theFCoE controller
connections is MAC Address of Burnt-in implementation dependentMAC
Only one MAC Address is required for the FCoE Controller and the
VN_Ports on a single physical MAC (aka Server Provided MAC Address
SPMA) FCF may chose to specify new MAC addresses for each VN_Port
(aka Fabric Provided MAC Address FPMA)Fibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved.21 22. Multiple Logical FC connections via a single
Ethernet MAC Examples of single MACs with connections to two
different FCFs SwitchThe Logical FC Link is defined by a MAC
Address pair A VN_Port MAC Address A VF_Port MAC AddressFor a
logical FC link the FCoE Frames are always sent to and received
from a specific FCFs MAC Address Therefore, pathing to and from the
FC driver is always defined by the MAC Address of the partner FCFs
VF_PortFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved.22 23. Functions of an
FCoE Initiator ASICAB CHost PCIe interface Has a Normal NIC
interface FCoE(A) to the HostASIC Has one or more Normal FC
interfaces (B,C) to thefunction Host FCoE functions not seen byFCoE
NIC Function FCthe Host Function FCoE functions perform the
Encapsulation and De-Lossless encapsulationEthernetMAC The FCoE
function External Instantiates a Logical FC Port N_Port, called a
VN_Port Fibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved. 23 24. HBA with Multiple
Logical FC Interfaces The FCoE controller will perform the FIP
functions and will instantiate new VN_Ports as FCoE Link End Point
(LEP) With the same MAC address as the FCoE Controller (SPMA) Or
with a new MAC address specified by the FCF (FPMA)A B CFCoE
ChipMACAddress of Burnt-in MAY have one burnt-in MACMAC address for
both IPFCoE_LEP VN_Portand FCoE/FIP packetsFC-3 /FC-4sEntity Entity
FCoE FC OrNIC Controller FCoEFCoE Function MAY have different
Burnt- Etype= in MACs for IP andFC-3 /FC-4s FCoE_LEP VN_PortFCoE or
FCoE/FIP packetsFIP? Used to separate HWEntityEntityFCoE based FCoE
from otherFC Lossless Ethernet MACEthernet_PortEthernet Traffic
Most NICs come with MAC several Burnt-in MACAddress of Burnt-in MAC
Addresses Fibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved.24 25. FIP
(FCoE Initiation Protocol) Discovery and Link InstantiationFibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved. 25 26. ENode to FCF Logical Links
An ENode must Locate FCFs with a Discovery protocol and then
establish a Logical FCLink with an FCF (VN_PortVF_Port) before an
Normal FC frame flow VF_PortVN_PortFCoE_LEPsFCoE_LEPFCF-MAC(A)
MAC(H1) FCF A VF_Port H1FCoE_LEPsFCfabric VN_PortFCF-MAC(B)
FCoE_LEP Lossless VF_Port Note: The FCFs must
VN_PortEthernetFCoE_LEPsdiscover each other and MAC(H2)
Switchcreate a Fabric before the FCoE_LEP FCF B ENode Discover FCFs
VF_PortH2FCoE_LEPs ClassicalEthernet
VN_PortFCoE_LEPVF_Port,LANVF_Port_Name VN_Ports, FCF-MAC(x): A MAC
address of a Lossless Ethernet port of FCF xVN_Port_Names MAC(y): A
MAC address of a Lossless Ethernet port of ENode yFibre Channel
over Ethernet (FCoE) 2008 Storage Networking Industry Association.
All Rights Reserved.26 27. Initial Login Flow ladder (2 Phases)
FCoE InitializationEnd-NodeFCFProtocolDiscovery Multicasto ALL
tFCFs DiscoveryUnicast FCF MAC Addresset.al. in Jumbo FramePhase
FLOGIFLOGI ACC with the FCFs Login Phasechosen VN_Port MAC
addressas a descriptor value FC Command (Using the FCF Normal FC
selected MAC Address as the SA)ProcessingFC Command responses(using
the commands SA as a DA)See Appendix for more details in Discovery
Consideration and ActionsFCOE ProtocolFibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved. 27 28. Subsequent Login Flow ladder (1 Phase) FCoE
InitializationEnd-NodeFCF ProtocolFLOGIFLOGI ACC with the FCFs
Login Phasechosen VN_Port MAC addressas a descriptor value FC
Command (Using the FCF selected MACNormal FC Address as the SA)
ProcessingFC Command responses(using the commands SA as a DA) See
Appendix for more details in Discovery Consideration and
ActionsFibre Channel over Ethernet (FCoE)FCOE Protocol 28 2008
Storage Networking Industry Association. All Rights Reserved. 29.
TopologiesFibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved.29 30. FCoE
Switch & FC FabricNote: FCoE serversand storage willprobably
use an FCoEHBA (or chip set)FCoE Switch FCFC An FCoE Switch may
connect to a normal FC switch Via the FC E-Port Ethernet FCFibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.30 31. Multiple FCoE Switch (FCF)
TopologiesFCoE Switch LosslessFC FC Ethernet FC Lossless Ethernet
FCFabricFabricLossless Ethernet switches configured into a Lossless
Ethernet A Lossless Ethernet Fabric can be made up of Combo FCoE
Switches Fabric can Front the FCoE Switch LosslessEthernet 3 FCoE
FCoE Switch SwitchFCoESwitchLossless FC FCoE FCoEEthernetFC 2
Switch SwitchLosslessFCoEEthernet Lossless EthernetFC 1Switch
FCLosslessEthernet 4FCoE Switches deployed at the edges of the
Lossless Ethernet Fabric A VE_Port in an FCF may connect to other
VE_Port in another FCF FCoE Switches connected via VE_Ports and
Lossless EthernetAnd an FCF FC E_Port may connect to an FC switch
E_Port Fibre Channel over Ethernet (FCoE) Ethernet 2008 Storage
Networking Industry Association. All Rights Reserved. FC31 32.
Status of Current Data CenterNetworkingData Center Outfacing (IP)
Network Server & Storage NetworkManagement GroupManagement
Group There can be 3 different System Area Network (SyAN) networks
Clustering Fabric System Area Network (SyAN) InfiniBand MyrinetUsed
for Clustering/Low Latency Ethernet CTC Storage area Network (SAN)
Used to Access to Storage Remote OfficesLAN/WAN External Networks
Used for General messaging Used for Client-Server
MessagingLAN/WANUsed for NASMessagingData Center NAS Often divided
into at least 3 management domains Storage Area Network - Fibre
ChannelData Center Server (clustering)NetworkData Center Storage
NetworkOutfacing (IP) Network LAN/WANs Note: with multiple Data
Centers there may alsobe interconnects with DWDM, FCIP/iFCP, etc.
Messaging NASFibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved. 32 33.
Current Fabrics RemoteClustering Network Offices
iSCSIStorageMainframeFocus: Low Ficon Latency & High Bandwidth
FCOutfacing FCFC IP Network FC
Network(LAN/WAN)MessagingNASFiconLocal & RemoteBusiness Campus
Ficon Storage Controller FC Link Focus: Protection,Clustering
Network Ficon Bandwidth/Congestion LinkFile Storage
ManagementArrays (NAS)EthernetLink Data Center Server & Storage
Network Management Group IP Network Management Group There is a FC
Storage Network, a Clustering Network & an Outfacing Network
Note: with multiple Data Centers there may also be interconnects
with DWDM, FCIP/iFCP, etc.Fibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved.33 34.
View of the Future Data Center Networks Data Center Server &
Outfacing (IP) Network Storage Network Management Group Management
Group The Server/Storage Networkswill become a
ConsolidatedFabricRemote Managed by the Data CenterOfficesSystem
and Storage Management Group Includes Storage andStorage &
System clustering provisioningOther IP NetsArea NetworkLogically
Single Fabric(LAN/WAN) SAN and SyAN managedMessaging(with FCoE) NAS
as a single fabric Focus: High Bandwidth and Low LatencyThe
Outfacing (IP) ManagementGroup remains the same Note: with multiple
Data Centers there may also Focus: Protection, Bandwidthbe
interconnects with DWDM, FCIP/iFCP, etc. and Congestion Management
Fibre Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.34 35. Phase in: FC & High
Performance LosslessEthernet Unify into a Data Center Fabric
MainframeRemoteOffices Ficon iSCSI Storage EthernetSW
FCDataCenterOutfacing FC & EthernetFabric IP Network FCSW
Blades(LAN/WAN)Including iSCSIMessaging GatewaysNASFCoE FiconFC
Ethernet BusinessSWCampus with iSCSI Ficon Storageconnections File
Storage ControllerArrays (NAS) Note: with multiple Data Centers
there may also be interconnects with DWDM, FCIP/iFCP, etc. FCoE
permits intermixing of multiple Connection types/protocolsFC Link
Clustering messaging, General Messaging, and Storage Ficon Link The
DataCenter Fabric will Trunk to the Outfacing Network (including
iSCSI sys) EthernetLink Some Customers may want keep a mixed
environment on-goingFibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved. 35 36.
ScenariosFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved.36 37. Scenario 1: FCoE
& IP Flows ClassicalInternet EthernetLossless EthernetLossless
FCoEEthernetSwitchesFC FCFCoE SwitchFCoE FlowsIP FlowsFibre Channel
over Ethernet (FCoE) 2008 Storage Networking Industry Association.
All Rights Reserved. 37 38. Scenario 2: FCoE Right &
WrongClassical InternetEthernetLossless EthernetLossless
FCoEEthernetSwitches FCFC FCoE SwitchFCoE FlowsInvalid FCoE
FlowsFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved.38 39. Summary Fibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.39 40. SummaryFCoE is a simple,
efficient mechanism for encapsulating Fibre Channel in Ethernet
framesFCoE is being standardized in INCITS Fibre Channel (T11)
technical committee Target completion is 2H08 Maximum benefit of
Fibre Channel is achieved: Evolutionary model of FC Switches and FC
SANs Emphasis placed on capitalizing on the benefits of Converged
Enhanced Ethernet (CEE) Being discussed in the IEEE 802.1 standards
working groupFibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved. 40 41. Thank
You! 42. Q&A / Feedback Please send any questions or comments
on thispresentation to SNIA: [email protected] thanks to
the following individuals for their contributions to this
tutorial.SNIA Education CommitteeClaudio DeSantiHoward
GoldsteinWalter Dey Robert Snively Suresh Vobbilisetty Silvano Gai
Joe PelissierJohn HufferdFor additional information refer to
http://www.t11.org/fcoeFibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved. 42
43. AppendixAdditional Requirements FCoE Relation to ISO Layers
Flows Additional Topologies FSPF and STP FIP Considerations and
Actions FIP Descriptors Pause vs. BB_CreditFibre Channel over
Ethernet (FCoE) 2008 Storage Networking Industry Association. All
Rights Reserved. 43 44. FCoE Goals/Requirements (part 3) FCoE must
be a direct mapping of Fibre Channel over an Ethernet network FCoE
must be layered on top of Ethernet FSPF used to route FCoE packets
Ethernet Spanning Tree (STP), MSTP, etc, is at a layer below FCoE
to allow an evolutionary approach towards consolidation of fabrics
The Fibre Channel N_Port, F_Port and E-Port constructs must be
retainedWith FCoE, ports may be connected with Logical Ethernet
Links May pass through Ethernet switches Identified by pairs of end
point MAC addresses Physical Ethernet Links can replace physical FC
Links Physical Ethernet Links can carry all Ethernet traffic,
including FCoE, but combined traffic needs the CEE capabilities
Fibre Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved. 44 45. FCoE Goals/Requirements
(part 4)Combo FCoE Switches may be built that support normal
Ethernet traffic, FCoE traffic , & FC traffic The FCoE
solutions should appear as a Fibre Channel to a Fibre Channel
experienced customer FCoE should keep the Fibre Channel operations
independent from Ethernet forwarding Keeps management
/Troubleshooting simple Common physical structures, different
logical structuresBased on Ethertype (Ethertype = FCoE) Storage
Management should be unchanged Fibre Channel over Ethernet (FCoE)
2008 Storage Networking Industry Association. All Rights
Reserved.45 46. FCoE Relation to ISO LayersFibre Channel over
Ethernet (FCoE) 2008 Storage Networking Industry Association. All
Rights Reserved. 46 47. FlowsFibre Channel over Ethernet (FCoE)
2008 Storage Networking Industry Association. All Rights
Reserved.47 48. Logical Fabric Topology An FCoE Switch receives
FCoE frames addressed to its FC-MAC address and forwards them based
on the D_ID of the encapsulated FC frameLogical LosslessTransaction
Path FCoE EthernetEthernet SwitchFC A H2#2 FCoESwitch #1 CLossless
#3FCFCLosslessEthernet #4 BEthernet FCoE SwitchH4Lossless S2
Ethernet H1H3 S1 An FCoE Switch rewrites the SA and DA of an FCoE
frame Path #1Path #2Path #3 Path #4EthernetDestinationFCoE-A
MACFCoE-B MACFCoE-C MAC FCoE-S1 MAC& Source FCoE-H2 MAC FCoE-A
MACFCoE-B MAC FCoE-C MAC EncapsulatedFC FrameD_ID FC_ID for S1FC_ID
for S1 FC_ID for S1FC_ID for S1 S_IDFC_ID for H2FC_ID for H2 FC_ID
for H2FC_ID for H2Fibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved. 48 49. Single
Ethernet Fabric with FCoE Switches H3Ethernet H2
EthernetSwitchSwitchFCoE#1SwitchFC FCEthernetSwitchCFCoE ASwitch
#2#3S2 EthernetEthernet Switch Lossless EthernetSwitchH1 S1Path #1
Path #2Path #3EthernetDestinationFCoE-A MAC FCoE-C MACFCoE-S1
MAC& Source FCoE-H2 MACFCoE-A MACFCoE-C MAC EncapsulatedFC
FrameD_IDFC_ID for S1 FC_ID for S1FC_ID for S1LogicalTransaction
Path S_IDFC_ID for H2 FC_ID for H2FC_ID for H2EthernetFC Fibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.49 50. FC Host to FCoE Storage H3
S2H5Ethernet#1 H2 Ethernet#3Switch FCoESwitch Switch FCEthernetC FC
FCoE Switch ASwitch#2 EthernetEthernetS3 Switch Lossless
EthernetSwitchH1 S1Path #3 Path #2EthernetDestinationFCoE-S2
MACFCoE-A MAC& Source FCoE-A MAC FCoE-C MACPath #1
EncapsulatedFC Frame D_ID FC_ID for S2 FC_ID for S2FC_ID for
S2LogicalTransaction PathS_ID FC_ID for H5 FC_ID for H5FC_ID for
H5EthernetFC Fibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved.50 51.
Additional Topologies Fibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved.51 52.
Additional Topologies (1) Lossless Ethernet FCoE FCoESwitchSwitch
(E) Switch (A)FC FC FCoE Lossless FCoE Switch (B) Ethernet Switch
(F) SwitchExample of Topologies with Rack Mount serversFibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.52 53. Additional Topologies (2)
Lossless Ethernet FCoEFCoESwitchSwitch (A)Switch (E)FC FC Lossless
FCoE FCoE Ethernet Switch (B) Switch (F)Switch Example of
Topologies with Blade ServersEquivalent to Blade servers with
N_Port_ID Virtualization (NPIV) Support Fibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved.53 54. FSPF and STP Fibre Channel over Ethernet (FCoE)
2008 Storage Networking Industry Association. All Rights
Reserved.54 55. FSPF & STP Concepts with FCoE (basic)Fibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved. 55 56. FSPF & STP Concepts
with FCoE (interconnected)Fibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved. 56
57. Equivalent FC topologyFibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved. 57
58. FIP Considerations and ActionsFibre Channel over Ethernet
(FCoE) 2008 Storage Networking Industry Association. All Rights
Reserved. 58 59. FCoE Discovery Considerations The Discovery phase
of the FCoE Initialization Protocol (FIP) uses two types of
messages, Solicitations and Advertisements The FIP Discovery Phase
helps define the FCF Ports that are available for the Link
instantiation Phase The ENodes discover the FCF ports that can
become VF_Ports and FCFs discover other FCF ports that can become
VE_PortsENodes Solicit (via Multicast) Advertisements from FCFs
whilespecifying their capabilitiesIn response FCFs Advertise their
availability and capabilities backto the ENodesFCFs Multicast their
existent to other FCF The FIP Discovery phase exchanges
solicitation and/or Advertisements between (HBA and/or FCF) FCoE
ControllersFibre Channel over Ethernet (FCoE) 2008 Storage
Networking Industry Association. All Rights Reserved. 59 60. FCF
Discovery Actions FCF supporting VE_Ports: Discovers other VE_Port
capable FCF-MACs, connected to the same Lossless Ethernet segment,
by: Transmitting a multicast Solicitation to All-FCFs (with the FCF
bit set to one) Receives back Jumbo Unicast Advertisements from VE
capable MACs To verify the support of Ethernet Jumbo frames in the
path In response to receipt of a Multicast to All-FCFs from another
FCF Has its own VE capable MACs sends their own Jumbo Unicast
Advertisements Instantiates VE_Port to VE_Port connections and
Exchanges FC ELP (Extended Link Protocol) and Fabric configuration
(using Ethertype=FCoE) with the other FCFs VE_Port capable
MACsFibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved. 60 61. ENode Actions
When an ENode becomes operational: The ENode discovers the VF_Port
capable FCF-MACs connected to the same Lossless Ethernet
segmentTransmits a multicast Solicitation to All-FCFs (with the FCF
bit setto zero)Receives Jumbo unicast Advertisements from select
compatibleVF_Port capable FCF-MACsStores the discovered FCF-MACs in
an FCF list When an ENode receives an Advertisement that a new FCF
is available, it may send a unicast Solicitation to it and receive
a Jumbo unicast Advertisement in reply To verify the support of
Ethernet Jumbo frames in the path May then perform FLOGIs (with
Ethertype-FIP) to a vendor specific subset of the FCF-MACs in the
FCF list Fibre Channel over Ethernet (FCoE) 2008 Storage Networking
Industry Association. All Rights Reserved.61 62. FIP Descriptors
Fibre Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.62 63. FIP Descriptors (1) Fibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved.63 64. FIP Descriptors (2)Fibre
Channel over Ethernet (FCoE) 2008 Storage Networking Industry
Association. All Rights Reserved. 64 65. Pause vs. BB_CreditBoth
mechanisms are used to avoid dropping framesWith different
trade-offsThe Pause mechanism requires at least the (2 x RTT
xbandwidth) product on a link as buffer spaceBut allows Buffer
handling in an arbitrary wayWell suited for networks with limited
(bandwidth x delay) product(e.g. within the data center)The Pause
frame is handled by the MAC layerSimilar to the R_RDY handling by
the FC-1 levelThe BB_Credit mechanism prevents loosing framesover
any linkBut links go under-utilized if link credits (& buffers)
are < thatneeded for (RTT x BW)Requires buffer handling in
maximum frame size unitsFibre Channel over Ethernet (FCoE) 2008
Storage Networking Industry Association. All Rights Reserved.
65
