FCoE Slides Only

2008 IBM Corporation

Fibre Channel over Ethernet: An Introduction

Peter MescherSTG Solution Central Team, RTP, NC, USA

Session ID: SSA01

New Enterprise Forum: Information Infrastructure - IBM System Storage and Storage Networking Technical Symposium

2008 IBM Corporation2 17-Aug-082

DisclaimersNOTES:All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices are suggested US list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography.Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use.

The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.IBM makes no representation or warranty regarding third-party products or services including those designated as ServerProven, ClusterProven or BladeCenter Interoperability Program products. Support for these third-party (non-IBM) products is provided by non-IBM Manufacturers.IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA.



Agenda

Overview of Fibre Channel over Ethernet (FCoE) as a product

Protocol Summary



What Is Fibre Channel over Ethernet?

A direct mapping of Fibre Channel onto Ethernet Aims to reduce cost through sharing of Ethernet

infrastructure New standard specified by ANSI T11 Committee

within FC-BB-5 Backed by virtually entire storage networking industry,

including IBM, Brocade, Cisco, QLogic, HP, and Intel



General Characteristics

Simple No complicated flow control or frame loss detection

Entire standard 24 pages

Viable alternative to native Fibre Channel Generally intended for, but not limited to, 10Gb

Ethernet



Basic FCoE Installation



Why use FCoE?

Large hosts generally use two networks right now: Ethernet and Fibre Channel

FCoE eliminates need for additional Host Adapters Reduces Cost Reduces slot and bus space usage

Very helpful for Blades



FCoE vs. FCIP/iFCP

Not really equivalent FCoE Direct replacement for FC links FCIP/iFCP Generally used for Distance Extension

FCoE unsuitable for long distances or complex networks

No FCIP or iFCP storage devices or HBAs



FCoE vs. iSCSIFCoE

Local-area, lossless links no routing allowed

Simple encapsulation of Fibre Channel

Low overhead 1% over regular FC

Fibre Channel well-understood by storage administrators

Substantial frame loss catastrophic

iSCSI Allows many hops, lossy

connections, high latency Substantial complexity on top

of TCP very different from FCP

Overhead varies typically higher.

TCP/IP well-understood by just about everybody else

Frame loss merely painful



FCoE vs. iSCSI long-term No overwhelming advantages for one vs. other FCoE appropriate for existing SAN installs

More SCSI-like No complex, gateway for non-iSCSI-native systems

iSCSI serious choice for new solutions Takes advantage of substantial existing IP infrastructure

and knowledge

Lossless Network needed for FCoE helpful to both iSCSI likely will win, in the end



Software vs. Hardware Implementations

FCoE is available as a software driver for most operating systems

Primary use is proof-of-concept and interoperability test

Virtually all production deployments expected to be hardware based

New Converged Network Adapters (CNA) In reality, FC HBAs with an Ethernet MAC layer



FCoE Disadvantages

No flow control (not even buffer credits) Relies on one or more optional Ethernet extensions

802.3-2005 PAUSE mechanism 802.1Qau Congestion Management

Required to deal with potential FC congestion, even if Ethernet uncongested

Assumes high-quality network Lossless Ethernet network required



Possible Industry Deployment

FCoE intended for 10Gb Ethernet links 1Gb functional, not recommended

1. Blades with Ethernet/FCoE splitter on in-Chassis switch

2. Edge switches / Director blades3. Entire Directors / Core Ethernet switches (same

thing?)4. Storage



Summary of FCoE as a Product

FCoE, in summary: A new standard designed to converge Fibre Channel

SANs and Ethernet networks Simple in concept and design Will be initially used with smaller servers Fundamentally different from iFCP and FC/IP In competition with iSCSI

Questions? (Information on protocol details to follow)



Terms

FCoE Mapper (FCM) Provides translation between Ethernet and Fibre Channel

Present in NICs and switches FCoE Forwarder (FCF) Ethernet / Fibre Channel

switch Also provides FCoE discovery and login services Does not need actual Fibre Channel ports

FCoE ENode (ENode) Ethernet port w/ FCoE abilities (software or hardware)



Terms (cont.)

FC-MAP Fabric ID Usually 0E:FC:0z (z = a Fabric ID number, which

defaults to 0.) Used for first three bytes of Fabric-Assigned MAC

Addresses



FCoE Standard

Listed in ANSI T11, FC-BB-5, Chapter 7 Two protocols in one standard:

FCoE The encapsulation standard FCoE Initialization Protocol (FIP) Protocol to provide

device discovery and login services Use different EtherTypes

Facilitates FIP snooping by non FCF switches



FCoE and the Protocol Stack



FCoE Frame Format

Other than normal Ethernet headers, no per-frame overhead

Mini-Jumbo Ethernet frames strongly suggested Not a difficult requirement; virtually all do



FCoE Frame Forwarding

Relies on Ethernet Spanning Tree or other forwarding scheme

No built-in routing such as FSPF Load balancing generally poor



More-Complex FCoE Topology



FCoE Network Without Much FC



Addressing

Two options Server Provided MAC Address (SPMA)

Typical Ethernet standard globally unique address Fabric Provided MAC Address (FPMA)

Format XX:XX:XX:YY:YY:YY XX:XX:XX = FC-MAP YY:YY:YY = FCID

No relation to WWPN of adapter



Addressing (cont.)

Multiple MACs per adapter expected One for regular Ethernet traffic (i.e. TCP/IP)

ENode MAC Address One for each FCoE address

VN_Port MAC Address

NPIV supported



FCoE Discovery

Utilizes special Fabric Initialization Protocol (FIP) frames Separate from regular FCoE frames Used for FLOGI/FDISC, LOGO, ELP and FCoE

discovery Source MAC probably burned-in MAC Address



FCoE Discovery and FC Login Process

1. Send multi-cast Ethernet frame to All-FCF-MACs standard address (or specific FCF) with Discovery Solicitation payload. Contains:

MAC Address used to receive response The hosts name Maximum Receive Size If an FCF itself, include FC-MAP (if using FPMA)



FCoE Discovery and FC Login Process (cont.)

2. FCF responds with Discovery Advertisement Advertisement contains:

Priority 1-byte value used to determine which FCF-MAC to use, if multiple respond

MAC Address of the FCF-MAC If using FPMA, the FC-MAP of the FCF Name of the Switch Name of the Fabric



FCoE Discovery and FC Login Process (cont.)

3. ENode sends FIP FLOGI Just like FC FLOGI, only in a FIP frame Returned FCID also used to build FPMA MAC

Address For FCFs, FIP ELP sent instead

For NPIV, FIP FDISC is used Works just like FC counterpart

4. The FCF responds with a FIP LS_ACC (or ELP_SW_ACC)



Summary

FCoE has two parts FCoE frames simple Ethernet frame plus Fibre

Channel frame. FIP protocol handles device discovery, fabric login

functions, and address assignment

Documents

FCoE Slides Only