JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 36, NO. 1, JANUARY 1, 2018 19

OTN Interface Standards for Rates Beyond 100 Gb/sSteven S. Gorshe , Fellow, IEEE

(Tutorial)

Abstract—The optical transport network (OTN) protocoldefined by the ITU-T (Recommendation G.709) has become thebackbone of service provider long haul and metro networks. Itsflexibility has allowed it to support the converged transparenttransport for the full variety of both constant bit rate andpacket-oriented digital WAN client signals, and its overhead isoptimized to reduce the equipment and operational costs forservice provider networks. The emerging technologies that enableinterface rates beyond 100 Gb/s have created multiple challengesand opportunities for the next generation of OTN. This paperfirst discusses the challenges and considerations behind the newextension of OTN for rates beyond 100 Gb/s (OTN B100G). Thispaper then provides a tutorial for the new OTN B100G standardin light of the flexible and modular manner in which it addressedthese considerations. This paper also gives a tutorial overview ofthe new flexible OTN “FlexO” protocol. FlexO was developed asa modular PHY technology for OTN B100G, allowing it to reuseIP and modules from the Ethernet ecosystem. In addition, thispaper provides a tutorial overview of the new flexible Ethernet“FlexE” technology from the optical internetworking forum.FlexE provides both an important new client for transport overOTN, and technology that has been incorporated into the OTNB100G standard. The complementary synergies between OTNand 200G/400G Ethernet and FlexE will become apparent.

Index Terms—DWDM, optical network, OTN, transport net-work, WDM.

I. INTRODUCTION

THE ITU-T G.709 Optical Transport Network (OTN) stan-dard was originally developed for simple transport of sig-

nals over a DWDM (dense wavelength division multiplexed)network. The OTN signal frame format provided a flexible dig-ital “wrapper” that used the same frame format and percentageof signal overhead regardless of its rate. Since its initial release,shortly after 2000, G.709 has evolved to become an increasinglyflexible transport protocol. OTN supports converged transpar-ent transport for the full variety of both constant bit rate (CBR)and packet-oriented digital WAN client signals with rates �1Gbit/s. This flexibility allows maximum utilization of optical re-sources with low protocol complexity. When signal rates beyond100 Gbit/s were on the horizon, it was recognized that a new ap-proach should be taken to evolving OTN for higher rates. Afterproviding some background on OTN, this paper discusses some

Manuscript received July 5, 2017; revised August 29, 2017; accepted Septem-ber 4, 2017. Date of publication September 7, 2017; date of current versionFebruary 24, 2018. This paper is an invited paper based on the “Beyond 100GOTN Interface Standardization” tutorial Th1I.1 at OFC 2017.

The author is with the Carrier Products Business Unit, Microsemi Corp.,Portland, OR 97223 USA (e-mail: steve.gorshe@microsemi.com).

Digital Object Identifier 10.1109/JLT.2017.2750141

Fig. 1. OTN base frame format.

of the challenges and motivations for a different approach, andthen describes the resulting recently released version of G.709to support rates up through multiple Tbit/s.

As explained in this paper, the new extension to G.709 forrates beyond 100 Gbit/s (B100G) differs from the legacy OTNin four significant ways:

1) OTN B100G adopts a frame structure to support a varietyof rates in increments of 100 Gbit/s rather than specifyingcertain discrete rates.

2) The method for mapping packet clients into OTN B100Gis based on Ethernet techniques rather than the ITU-TG.7041 Generic Framing Procedure (GFP).

3) Forward error correction (FEC) is now required, howeverit is specified separately for each interface type rather thanmade a part of the base frame format as it was for legacyOTN.

4) A modular physical layer interface, called “FlexO” hasbeen added in a companion ITU-T Recommendation.

Notes on terminology: Except where specified, all rates men-tioned in this paper are nominal rates rounded to the nearestconvenient value. In reality, most of the actual rates are slightlyhigher in order to accommodate the required overhead, etc.Ethernet signals are named based on their MAC rate. For exam-ple, 10 GbE is 10 Gbit/s Ethernet and 100 GbE is 100 Gbit/sEthernet.

II. BACKGROUND

The OTN signal frame is illustrated in Fig. 1. An ODU (Op-tical Data Unit) is the basic wrapper signal that includes thepayload area for carrying client signals and the overhead fornetwork management. The OTU (Optical Transport Unit) addsinformation for frame and multiframe recovery that are requiredwhen the signal is transmitted over the optical media, and ad-

0733-8724 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

20 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 36, NO. 1, JANUARY 1, 2018

ditional performance monitoring overhead for the optical span.The OPU (Optical Payload Unit) is the portion of the frame inwhich client signals are carried. The OPU can contain eithera single client signal or multiple clients multiplexed into theOPU. The OPU payload area is structured as a set of tributaryslots (TS), with each client signal occupying an integer numberof TS. The OPU overhead identifies the client signal occupyingeach TS and provides the means for adapting between the clientsignal rate and the aggregate rate of the TS set used by thatclient. This adaptation is called justification, and provides themethod for adding padding or “stuff” words to the client signalto match the TS set rate.

The initial version of G.709 was optimized for carrying thewidely deployed SONET/SDH (Synchronous Optical Network/ Synchronous Digital Hierarchy) optical transport signals. Con-sequently, it was defined with a set of discrete rates at roughly2.5, 10 and 40 Gbit/s, corresponding to the standard ratesof those client signals. As data client signals (e.g., Ethernetand Fiber Channel) became increasingly important, G.709 wasamended and revised to include new ODU and OTU rates to sup-port their transport. Specifically, by 2010, G.709 had defined adiscrete set of OTUk transport signals, with k = 1, 2, 3 and 4corresponding to 2.5, 10, 40 and 100 Gbit/s OTN signals. TheOTU4 was optimized for carrying 100 GbE. The ODUj sig-nals carried by these OTUk included the same four nominalrates, but also included the 1.25 Gbit/s ODU0 and an m ×1.25 Gbit/s ODUflex that could be multiplexed into the OPUkof the OTUk. The ODU0 was added primarily for carrying Gbit/sEthernet (GbE). The ODUflex was introduced for encapsulatingboth arbitrary-rate CBR signals that are carried transparently,and packet data client signals. ODUflex signals carrying CBRclients are referred to as ODUflex(CBR), and those carryingpacket data streams are referred to as ODUflex(GFP) due tothe client data packets being first encapsulated by G.7041 GFPframes.

The need and challenges for OTN rates beyond 100 Gbit/sprovided an opportunity to take a fresh look at OTN. The chal-lenges included:

1) Higher bit rate signals, beginning with 40 GbE and100 GbE Ethernet, had already focused on interfaces withmultiple physical lanes rather than a single serial interface.For example, 40 GbE and 100 GbE initially specified 4× 10 Gbit/s and 10 × 10 Gbit/s interfaces, respectively,where the different physical lanes could be transmittedover separate electrical wires, separate fibers, or separatewavelengths within the same fiber. OTN had specified sin-gle lane serial interfaces, but adopted this multi-lane op-tion, primarily for the electrical intra-system interfaces,so that the PHY modules developed for 40 GbE and100 GbE could be reused for transport interfaces.

2) The Shannon channel capacity limits are catching up tooptical transport network capabilities. Specifically, withcurrently available technology, the standard 50 GHz chan-nel spacing used for dense wavelength division multiplex-ing (DWDM) imposes limits on transporting signals overreasonable distances when they have rates much over 100Gbit/s.

3) The old paradigm of adding new discrete rates for OTNhad largely reached its practical limits, making a modularrate and frame structure approach more attractive.

4) Different B100G interface types have different FEC per-formance capability requirements. Consequently, whilethe legacy OTUk frame format had fixed dedicated over-head for an RS(255, 239, 8) Reed-Solomon, it was moreappropriate to specify the FEC on a per-interface basisand not make the FEC overhead in integral part of theframe structure.

5) The 1.25 Gbit/s Tributary Slot (TS) rate of legacy OTN istoo fine-grained to be practical with B100G signals.

6) The higher OTN data rates and the emerging new dataclient signals, such as 200 GbE, 400 GbE and the OIF’sFlexible Ethernet (FlexE), have made the legacy byte-oriented mapping approach for data clients impractical.This motivated the desire for a wide-word type of map-ping.

7) In order to optimize the use of each wavelength, includingfor transmission reach, there was a desire to be able totransmit the OTN signals at the rate required for the clientpayload being carried rather than at the full discrete rateof the OTUCn signal.

8) B100G interfaces should re-use as much IP from the100 Gbit/s OTN interfaces as practical.

9) At that time, the IEEE 802.3bs Task Force working on400 Gbit/s Ethernet was examining several new ap-proaches that were different from its previous interfaces.The new OTN format needed to not only carry 400 GbE,but also re-use its technology and PHY components when-ever possible in order to benefit from the Ethernet com-ponent cost curves.

The first three challenges all motivated defining OTN B100Gas a modular structure that included support for multi-lane inter-face structures. This was the first of the foundational agreements.Other foundational agreements will be discussed below as theyrelate to the topic.

III. OTN B100G FRAME STRUCTURE

As mentioned above, legacy OTN signals used an identicalframe structure regardless of the ODU rate. In contrast, OTNB100G borrowed the modular approach of SONET/SDH andinterleaves base frames to create higher rate signals. Specifically,the B100G frame format is an interleaving of n of the basicframes. The basic frame is an ODUC (ODU, where the “C”reflects its 100 Gbit/s nominal rate), and the transmitted signalis an OTUCn. Unlike with SONET/SDH, the entire OPUCnpayload area of the resulting signal is a single entity with ntimes the size of the OPUC.

A key early decision was that the new B100G signals wouldnot be switched within the network, since the ability to switchwould make them a new layer network for the carrier tomanage. Instead, B100G was defined for only point-to-pointconnectivity, with the client signals carried within the B100Gsignal being switched.

GORSHE: OTN INTERFACE STANDARDS FOR RATES BEYOND 100 GB/S 21

Fig. 2. OTN B100G frame format. (a) OTUC/ODUC/OPUC slice frameformat. (b) OTUCn/ODUCn/OPUCn frame.

Another closely related foundational agreement is related tothe need to be able to use multiple wavelengths when transmit-ting an OTUCn signal. It was agreed that the entire group ofwavelengths associated with an OTUCn interface signal wouldgo through the same fiber and optical switches (i.e., the sameOptical Multiplex Section trails) such that the OTUCn signalcan be managed as a single entity. Consequently, very limiteddeskew is required if multiple wavelengths are used. Note thatwith the recently revised ITU-T nomenclature, a WDM wave-length used to carry an OTN signal is referred to as an OpticalTributary Signal (OTSi). An Optical Tributary Signal Group(OTSiG) is the set of OTSi signals that support carrying a singleclient such as an OTUCn.

The resulting signal is shown in Fig. 2. As is apparent, thebasic ODUC frame format is identical to the ODUk except forsome differences in the overhead field definitions that will benoted below. This allows IP reuse from legacy OTN. Note thatthe format for interleaving the ODUC overhead into the ODUCnis not specified, since it depends on the interface.

The OTUCn, ODUCn, and OPUCn overhead fields are es-sentially the same as for the previous generation of OTN.Overhead that pertains to the entire interface (e.g., Trail TraceIdentifier for proper connectivity checking, remote defect indi-cations, and delay measurement overhead) only appear on thefirst OTUC/ODUC of the OTUCn/ODUCn rather than on all nslices. See [1], [3], or [4] for more information on the specificoverhead fields.

The rate of the base OTUC signal was chosen in order tomeet the criteria that the OPUC1 must be capable of carryingan ODU4 client, the OPUC4 must be capable of carrying a 400GbE client, and the resulting signal rate should be reasonablyefficient within the constraints of the first two criteria. The signalrate information is summarized in Table I.

As noted above, there is a desire to be able to transmit theB100G signal at only the rate required for the payload that itcarries, hence optimizing the wavelength utilization and signal

TABLE IOTN B100G RATE INFORMATION

Note: All rates are in Gbit/s ±20 ppm.

reach. As described below in Section V, the way this is accom-plished with OTN B100G is by keeping all the overhead of theOTUCn signal, but removing every TS that will not be used onthis interface.

The TS size was chosen to be nominally 5 Gbit/s, which gives(100n/5) = 20n TS in an OPUCn. This TS rate provided thedesired coarser granularity, and was the lowest rate that wouldefficiently accommodate the new data client signals, including25 GbE and 16 Gbit/s Fiber Channel. Newer data clients typi-cally have rates that are divisible by 5 Gbit/s. Legacy data clientsignals with rates below 5 Gbit/s will be first mapped into alegacy OTN signal. The exact TS and OPUCn payload rates areshown in Table I. In order to reduce device and managementcomplexity, it was agreed that there would be a maximum of10n tributaries in the OPUCn.

The OPUCn Tributary Slot and overhead structure is illus-trated in Fig. 3. In order to simplify data path designs inhigh-speed devices, it was decided that each TS occupies 16consecutive bytes per appearance in the OPUC. In other words,a TS uses 16-byte (128-bit) words.

Fig. 4 shows an example in which a client using three TS(TS2.3, 1.4 and 1.15) is mapped into an OPUC2. Differentcolors are used here for each of the 3 × 16-byte words shownfor that client. The sequence number of the 16-byte segment ofthe 3 × 16-byte word is shown.

IV. CLIENT SIGNAL MAPPING/MULTIPLEXING

With legacy OTN, client signals could either be mapped di-rectly into an OPUk, filling that entire OPUk, or mapped intothe OPU of a lower rate ODU, which was then multiplexed intoa higher rate OPUk. One consequence of the decision not toallow switching ODUC signals is that no clients are mappeddirectly into an OPUCn. All client signals must first be mappedinto a legacy ODUj (j = 2, 3, 4, flex) signal, which is thenmultiplexed into the OPUCn. The resulting considerations andnew mapping approaches are described in the remainder of thissection.

All ODUk signals carrying client information are multiplexedinto the OPUCn using the Generic Mapping Procedure (GMP)for rate compensation between the ODUk rate and the aggregaterate of the OPUCn TS set that it occupies. GMP was introducedwith the previous generation of OTN. Here, GMP works byhaving the source node, once per OPUCn multiframe, trans-mit a count of the number of ODUk data words that it willmap into that TS (or TS set) during the next multiframe. Theword locations that are not filled with the ODUk data are filled

22 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 36, NO. 1, JANUARY 1, 2018

Fig. 3. OPUCn TS and overhead structure. (a) Within a single OPUC slice.(b) Across all n OPUC slices.

with pad words, and the location of the pad words is determinedbased on modulo arithmetic involving the count value. The GMPoverhead is illustrated in Fig. 5. For robustness, the transmittersignals count increments and decrements (±1 and ±2) by in-verting a specific pattern of the count field bits (C1-C10). Thebit inversions, combined with the CRC, allow the receiver tocorrectly interpret the count in the presence of bit or burst er-rors. See [2] or [3] for a detailed description and illustrationof GMP operation. Since 952 is the maximum number of datawords that a client ODUk can occupy in an OPUCn multiframe,the OPUCn overhead uses a 10-bit count value rather than the14-bit value used with other OTN signals. Moving to a 9-bitcount field also freed more bits for use in providing fine-grainphase information about the client signal mapping (i.e., concep-tually, the fraction of a data word that the source was not able totransmit in that multiframe). As shown in Fig. 5, this fine-grainphase information (D1-D18) now expands in to the first two bitsof OTUCn frame column 16.

Fig. 4. Mapping example for a client using 3 TS into an OPUC2.

Fig. 5. GMP overhead.

Some additional details about the ODUflex, mentioned abovein Section II, would be helpful in understanding the remainderof this section. As noted, there were originally two types ofODUflex defined. The ODUflex(CBR) for carrying CBR clientsis simply a (239/238) × (client rate) client wrapper mapping.The rate of an ODUflex(GFP), which uses GFP encapsulationfor carrying the packets of a packet data client, is generatedwith local source clock. The OPUflex(GFP) is filled with GFPData and Idle frames, with the GFP Idle frames used for rateadaptation between the client packet rate and the OPUflex rate.Mapping a specific data flow or VLAN into each ODUflex(GFP)allows pseudo-wire or VLAN switching within the OTN domainvia ODU switching. As explained below, a new type of ODUflexwas defined for use with OTN B100G.

There were two new considerations for data client signalswith rates >100 Gbit/s. One was a desire for larger mappinggranularity wider word sizes and the other was the emergenceof the FlexE client signal type in the Optical InternetworkingForum (OIF).

A. Constant Bit Rate (CBR) Client Signal Mapping

OTN B100G continues to use the ODUflex(CBR) forCBR client signals. However, client signals using the popu-lar 64B/66B block code are mapped into the OPUflex on the2-bit boundaries of the 66-bit blocks in order to simplify blockdelineation at the receiver.

GORSHE: OTN INTERFACE STANDARDS FOR RATES BEYOND 100 GB/S 23

B. Packet Data Client Signal Mapping

For B100G, the granularity of the ODUflex signals carryingpacket data is m × 5 Gbit/s (i.e., the ODUflex occupies m ofthe OPUCn TS). Higher client data rates and an important newFlexE client signal led to a new type of data client mapping withB100G.

As described above, legacy OTN used GFP to encapsulatethe packets of data client signals, with the GFP frames mappedbyte-wise into the OPUk. While GFP continues to be used forpacket data clients with rates � 100 Gbit/s, a different approachwas adopted for clients with rates >100 Gbit/s.

The rationale for the new mapping is that most high ratedata clients have adopted the wide-word 64B/66B block codethat contains 64 data bits per code word rather than 8 bits likethe 8B/10B code commonly used for lower rate data signals.The larger code blocks are more bandwidth efficient, and moresuitable to the wide data buses commonly used within ICs thatprocess the high rate signals. Since Ethernet is by far the mostcommon packet data client, there are data path architectural ad-vantages in OTN mapper ICs to using Ethernet for all packetdata, including encapsulating non-Ethernet clients (e.g., MPLS)into Ethernet frames. With this approach, the Ethernet 64B/66Bcodes are mapped directly into the OPUflex rather than extract-ing the Ethernet frames and then using an intermediate GFPstage. Since GFP Idle frames are no longer available for rateadaptation, the mapping adopts the IEEE 802.3 rate adaptionmethod of inserting or removing Ethernet Idle codes charactersbetween Ethernet frames (i.e., adjusting the size of the inter-packet gap). Consequently, this type of ODUflex is referred toas ODUflex(IMP) where IMP = Idle Mapping Procedure.

Since the ODUflex(IMP) carries a character stream ratherthan packets, it is conceptually somewhere between ODU-flex(CBR) and ODUflex(GFP). While the ODUflex(IMP) map-per and demapper must be aware of the packet boundaries (i.e.,the inter-packet gap) to perform the Idle insertion/removal, theydon’t need to process the packets.

The specific steps for mapping the 64B/66B stream into anOPUflex(IMP) are as follows. The packet client signal is mappedas an m× 25 Gbit/ s (m = 1, 2, . . .) Ethernet stream. Note thatEthernet clients with rates of 10 and 40 Gbit/s are also supported.

1) The data stream presented to the mapper consists of64B/66B characters containing Ethernet MAC frames andinter-frame gap Idle (or Ordered set) characters.

2) The character stream is first 64B/66-encoded into a FlexEsignal. (See the FlexE description below.)

3) The resulting scrambled FlexE signal is mapped into theODUflex(IMP). Idle characters (or Ordered sets) are in-serted between Ethernet frames at this point in order tomatch the rate of the FlexE stream with the OPUflex rate.

4) Next the FlexE character stream is scrambled, using aself-synchronous scrambler with the same 1 + x39 + x58

method and polynomial specified in IEEE 802.3 Clause49.2.6.

5) Finally, the resulting ODUflex(IMP) is multi-plexed/mapped into the OPUCn, where GMP isused for rate justification, as with all ODUk clients beingmultiplexed into an OPUCn.

The FlexE client signal rates are defined by the OIF tobe k× 5.156,25 Gbit/ s ± 100 ppm, with k = 2, 8, 5n (n ≥1). Consequently, the nominal ODUflex(IMP) bit rate is(239/238)(s × 5.15625 Gbit/s). Note that the 64B/66B encodedclients are mapped into the OPUflex(IMP) using the same 2-bitalignment discussed above for general 64B/66B encoded CBRclient streams.

C. Flexible Ethernet (FlexE) Client Signals

Another critical factor that influenced the OTN B100G packetclient data mappings was the parallel work done in the OIF todefine the above-mentioned “Flexible Ethernet” (FlexE) inter-face. FlexE defines a mechanism for supporting a variety ofEthernet sub-rates, in addition to an efficient method to bondmultiple parallel links. Specifically, it defines a TDM structurethat allows multiple Ethernet MAC flows that are each less thanthe PMD (Physical Medium Dependent) rate to share a set ofPMDs. It also allows bonding a set of PMDs to carry MAC flowsthat are higher rate than a single PMD. The primary motivationfor developing FlexE was to provide flexible interface rates fordata center interconnections.

A FlexE client signal is an Ethernet MAC flow that is aCBR-type signal. A FlexE Group consists of between 1 and254 100GBASE-R Ethernet PHYs, all sharing a common PHYclock source. The TDM structure that allows multiplexing theFlexE clients is based on periodic “calendar slots” that can befilled with a 64B/66B character from the MAC flow assigned tothat calendar slot. The frame format for each PHY consists of aslot carrying the FlexE signal overhead followed by 1023 setsof 20 calendar slots for carrying FlexE client data. The calendarassociated with each individual PHY is called a sub-calendarof the overall calendar for the FlexE Group. A FlexE clientMAC flow can own any number of calendar slots, and thesecalendar slots can either go over the same PMD or be spreadacross multiple PMDs. As noted above, the nominal rate of eachcalendar slot is 5 Gbit/s (corresponding to 20 calendar slots per100 Gbit/s PMD).

The FlexE transmit data flow is illustrated in Fig. 6. As can beseen here, FlexE sits in the middle of the Ethernet data flow, i.e.,between the Ethernet MAC that creates the FlexE client dataflows and the Ethernet scrambler, PMA, PMD and lower layers.

In addition to using FlexE for mapping packet data clientsinto the OPUCn, as described above, OTN B100G has flexibleoptions for carrying FlexE signals. Specifically, when the clientsignal originates as a FlexE signal, there are three options formapping it into an OPUCn. One option is to terminate the FlexEsignal and map the FlexE client flows into the OPUCn as in-dividual clients. A new FlexE signal can then be created at theOTN demapper. The second option is called “FlexE unaware”because it treats each 100 Gbit/s FlexE PMD stream as if it isan individual 100 GbE signal. Hence, the mapper and demapperhave no need to know that a FlexE signal is present.

The third option for mapping a FlexE signal into an OPUCnis called “FlexE aware.” One of the features of FlexE resultsfrom each FlexE client flow being assigned only the calendarsslots required for that client flow’s bandwidth. Consequently, it

24 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 36, NO. 1, JANUARY 1, 2018

Fig. 6. FlexE data flow for the transmit direction.

is possible that the bandwidth of a PMD is not fully utilized. Ifit is known that a PMD will remain only partially utilized, theunneeded calendar slots are grouped at the end of the calendarand flagged as being “Unavailable.” The PMD can then re-move these calendar slots and transmit at a corresponding lower(“crunched”) CBR PHY rate in order to reduce system power,achieve longer signal reach and/or improve error performance.When a FlexE-aware OTN mapper receives either a crunchedFlexE signal or a full-rate FlexE signal with Unavailable calen-dar slots, it will map the lower rate CBR stream correspondingto the non-Unavailable calendar slots. Consequently, the ODU-flex carrying the FlexE client will require one fewer OPUCnTS for each Unavailable FlexE calendar slot that is removed,thus making more efficient use of the OPUCn capacity. Theresulting signal is mapped into an OPUflex using the specialbit-synchronous mode of GMP (BGMP). BGMP generates theGMP count values deterministically rather than deriving themfrom the input client rate and buffer fill.

V. SUB-RATE OTUCN (OTUCN-M)

Some carrier applications need optimization for powerand/or transmission distance. Example applications includeinterconnections between two routers where the peak rateof the packet flow is less than the n × OTUC rate, and theinterconnections between two OTN crossconnects where therequired capacity is less than the n × OTUC rate. B100G OTNincludes the option of transmitting a signal that has the full setof OTUCn/ODUCn overhead, but has an OPUCn consistingof only the active Tributary Slots. Specifically, an OTUCn-Msignal consists of n copies of the OTUC, ODUC and OPUCoverhead, and M of the 5 Gbit/s TS. Since this application is

vendor specific, G.709 does not define the frame format details.Conceptually, this is somewhat similar to the FlexE crunchingof unavailable calendar slots described above.

VI. FLEXIBLE OTN (FLEXO) INTERFACE

As discussed above, one of the important decisions in creatingthe OTN B100G standard was separating the OTN signal for-mat from the transmitted line format and associated FEC. Thisseparation allows creating PHY formats optimized for differentapplications that are independent of the B100G signal formatexcept for its rate. The first PHY interface defined for OTNB100G signals was conceptually inspired in part by FlexE. It iscalled Flexible OTN (FlexO) and is defined in ITU-T Recom-mendation G.709.1. [5] The basics of FlexO are described inthis section.

Like FlexE, FlexO is a modular interface consisting of a setof 100 Gbit/s optical PHY streams that are bonded togetherto carry an OTUCn. This allows using any value of “n” foran OTUCn interface rather than defining only certain discretevalues of n (e.g., just n = 4 and 10). Since the ODUCn signalis constructed from n of the 100 Gbit/s ODUC slices, it is wellsuited to using an n × 100 Gbit/s modular PHY, with each100 Gbit/s PHY carrying an OTUC slice. Note that if a set ofm 100 Gbit/s PHYs are available, a subset of n PHYs can bechosen to carry an OTUCn (n < m). For example, this wouldallow choosing the subset of PHYs with the best optical channelcharacteristics, or carrying multiple OTUCn signals over a setof PHYs.

Since the initial version of FlexO is intended for client sideinterfaces, it was ideal from a component cost and availabilitystandpoint to reuse the existing pluggable optical PHY modulesdeveloped for 100 GbE and OTU4. FlexO makes partial reuseof the lane architecture and FEC structure from 100 GbE and400 GbE Ethernet in order to leverage Ethernet IP.

IEEE 802.3 defined a relatively strong Reed-Solomon FECfor use with Ethernet signals having rates ≥100 Gbit/s. TheRS(544,514) “KP4” code covers blocks of 514 10-bit sym-bols in each FEC codeword, and can correct up to 15 erroredsymbols per codeword. The overall performance is effectivelycomparable to the RS (255, 239) “GFEC” code defined forlegacy OTN, which was based on 8-bit symbols. However, theRS (544, 514) achieves its performance with lower codingoverhead (544/514 = 1.0584 and 255/239 = 1.0669). Thisreduced FEC overhead ratio is very similar to the rate ratiobetween an ODUC1 and a legacy OTU4 using GFEC. Conse-quently, applying the KP4 FEC to an ODUC1 made it possibleto achieve a bit rate within <5 ppm of the OTU4 rate, whichallows using existing 100 GbE/OTU4 PHY modules. Hence, aflexible interface based on the RS (544, 514) FEC was ideal interms of both the ability to reuse existing optical modules andreusing Ethernet KP4 FEC IP.

The FlexO signal is a constant-rate stream of RS(544, 514)codewords containing the OTUCn data as its payload. This al-lows direct reuse of an IEEE 802.3 KP4 IP/engine. The signaland frame format is illustrated in Fig. 7. There are 128 FECcodewords per FlexO frame, and eight frames per multiframe.

GORSHE: OTN INTERFACE STANDARDS FOR RATES BEYOND 100 GB/S 25

Fig. 7. FlexO frame format.

Fig. 8. FlexO overhead illustration.

Each frame begins with Lane Alignment Markers (AM) andFlexO overhead. The AMs were chosen to directly reuse thosefrom 400 GbE, with one 120-bit AM per 25 Gbit/s logical lanefor that interface. The Fixed Stuff is inserted to achieve thenear-OTU4 rate. The OTUCn data begins on a 128-bit blockboundary of the first frame. While the boundary slides betweensuccessive codewords, there is an integer number of 128-bitOTUCn data blocks per frame.

There are 320 bits reserved for FlexO overhead in the firstcodeword of each frame. The overhead, shown in Fig. 8, isassociated with managing the FlexO interface. Specifically, itincludes information required by the receiver to properly re-assemble the OTUCn (Group and PHY IDs, and a map of whichPHYs are used by that group), FlexO Communications Channel(FCC) for PHY configuration negotiation, and a status field toindicate remote PHY failures. The FlexO overhead of the firstmember of the FlexO Group also includes the OSMC (OTNSynchronization Message Channel), which is used to carry IEEE1588 PTP messages for frequency and time-of-day informationto synchronize OTN nodes. The OSMC is located in FlexOoverhead rather than OTUCn since the FlexO is the PHY layer.

For the FlexO interface (FOIC), the FlexO data stream isround robin striped onto 25 Gbit/s logical lanes on a 10-bit

Fig. 9. FlexO transmit data flow.

Fig. 10. Data flow from the input client signal to the optical line signal.

(i.e., RS10 symbol) basis. This is identical to the format usedby IEEE 802.3 with the KP4 FEC. The FlexO frame beginson the first lane. The AMs are inserted into the beginning ofthe FlexO frame such that the lane distribution will place thecorrect AM sequence onto each lane. Hence the name “lane”alignment marker. The FlexO transmit data flow is illustratedin Fig. 9. Currently only 25 Gbit/s electrical lanes are used,creating a 1-to-1 logical-to-electrical PHY mapping. Future50 Gbit/s electrical lanes would carry the two AM sets asso-ciated with the two logical lanes carried over the electrical PHYlane.

26 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 36, NO. 1, JANUARY 1, 2018

As 200 and 400 Gbit/s PHY modules become available for200 GbE and 400 GbE, the FlexO specification will be extendedfor using them. Another extension to FlexO is using a strongerFEC for longer transmission reach. Such a long reach extensionwas being developed at the time this paper was written.

Fig. 10 illustrates the resulting data flow from the client signal,through mapping into an OPUCn to the FlexO encoding and lanestriping, and onto one or more wavelengths on the fiber.

VII. CONCLUSION

Originally introduced in 2000, OTN has proven capable ofcontinuing to evolve and adapt to cover new applications andtechnology in the transport network. The new OTUCn B100Gstandard is the next stage of that evolution. The modular frameand rate structure readily allows:

1) A convenient way for the OTUCn signal to be divided fortransmission over multiple wavelengths.

2) Supporting multi-vendor interfaces of different rates.3) A straightforward adaptation to carry future high-speed

client signals, such as Terabit/s Ethernet.4) The introduction of FlexO for a modular interface that can

exploit 100 GbE/OTU4 PHY modules.While the initial application for OTUCn signals will be long

haul connections, it will see metro network applications as 400GbE and FlexE become available and data-center connectivityincreases in importance. OTUCn should also be well suited forfuture 5G wireless transport applications as they increase thebandwidth demands in the backhaul and mid-haul networks.As a TDM protocol, OTN inherently carries client frequencyinformation. The FlexO OSMC allows delivering precision tim-ing synchronization information with OTUCn. These featuresmake OTUCn a very practical technology for supporting timesensitive networking applications like 5G transport.

With its mapping and rate flexibility, its associated FlexOPHY that takes advantage of Ethernet modules, and its overheadoptimized for service provider transport networks, the B100GOTUCn preserves OTN’s role as the ideal transport technologyfor service provider transport networks.

REFERENCES

[1] Interfaces for the Optical Transport Network (OTN), ITU-T Recommenda-tion G.709, 2016.

[2] Optical Transport Networks From TDM to Packet, Publication of ITU-T,2010, pp. 2-41 to 2-62.

[3] S. Gorshe, A tutorial on ITU-T G.709 optical transport networks (OTN).White Paper ESC-2081250. (2011) [Online]. Available: https://www.microsemi.com/products/optical-networking/otn/otn#white-papers

[4] S. Gorshe, A tutorial on the new ITU-T G.709 OTN evolution forrates beyond 100 Gbit/s. White Paper ESC-2170438. (2017) [Online].Available: https://www.microsemi.com/products/optical-networking/otn/otn#white-papers

[5] Flexible OTN Short-Reach Interfaces, ITU-T Recommendation G.709.1,2016.

[6] IEEE Standard for Ethernet, IEEE Standard 802.3:2015, 2015.[7] Flexible Ethernet (FlexE) Implementation Agreement, OIF.

Steven S. Gorshe (M’83–SM’91–F’07) received the B.S.E.E. degree from theUniversity of Idaho, Moscow, ID, in 1979, and the M.S.E.E. degree and thePh.D. degree in electronics/computer engineering from Oregon State Univer-sity, Corvallis, OR, USA, in 1982 and 2002, respectively.

Since 1982, he has been in research and development of telecommunica-tions systems and ICs, beginning his career with GTE Lenkurt, moving to NECAmerica in 1988, where he was a Principal Engineer and Chief Architect withNEC eLuminant Technologies. In 2000, he moved to PMC-Sierra, since acquiredby Microsemi Corp., Portland, OR, where he is currently a Distinguished En-gineer working on technology for optical transmission and access systems. Heis currently a Rapporteur for ITU-T Q11/15, which is responsible for opticaltransport network standards. His standards activity there and in other bodiesincludes more than 400 contributions, and multiple technical editorships. Hehas 39 patents granted/pending, is a coauthor of two textbooks, four chapters,and more than 29 papers.

Dr. Gorshe was Editor-in-Chief for IEEE COMMUNICATIONS MAGAZINE.He was also the Director of Magazines and is currently a Board of GovernorsMember-at-Large for the IEEE Communications Society.