Developing OTN Interface Standards for Beyond 100G

Maximizing network Capacity, Reach and Value Over land, under sea, worldwide www.xtera.com 1

Developing OTN Interface Standards for Beyond 100G

Abstract

Work is underway to define Ethernet and OTN interfaces for client signals greater than 100 Gbit/s. For the OTN interface, three possible directions are being investigated thus far. Many questions must be answered and contributions are expected. Experts from all over the world got together in late February 2013 in Dallas, Texas, USA at an ITU-T Q11/15 meeting hosted by Xtera Communications to continue their work in this area. This paper includes some of the latest developments from this meeting.


Introduction

Industry drivers for increased transport and switching capacity continue to include growth of the Internet, greater prevalence of video content, and more dependence on network based storage. The new applications continue to be IP/Ethernet-based packet applications that rely on routers and packet switches. In many cases, these devices operate more efficiently with fewer high-capacity ports than they do with many low-capacity ports, even if the total bandwidth supported in the two cases is the same. Partly for that reason, IEEE 802.3 specified 100 Gigabit Ethernet (GbE) in the past few years. The Telecommunication Standardization Sector of International Telecommunication Union (ITU-T) extended the OTN (Optical Transport Network) standards to support 100GbE and other combinations of lower rate clients totaling around 100 Gbit/s.

Figure 1: OTN Frame structure

As networks continue to grow in capacity requirements, the efforts are already underway to define the next higher rate client interface. The beyond 100G standards activities started with work done by the IEEE as part of the Bandwidth Assessment project. The completion of this project at the end of 2011 kicked off a series of activities relating to high bit rate systems beyond 100G.

The first step is to decide on the bit rate for the next high speed interface. In general, large bandwidth users and carriers favored the terabit per second system. Most component and system vendors supported 400-Gbit/s approaches. The IEEE held several meetings trying to weigh the pluses and minuses of each option in order to choose between 400 Gbit/s and 1 Tbit/s. During the March 2013 meeting in Orlando, Florida, USA, the IEEE approved the formation of a 400GbE working group. The first working group meeting will be held at the next IEEE meeting in May 2013. The decision is based on the ability to re-use much of the technologies developed for 100GbE. This approach could reduce the time to market and


development costs, therefore encourages earlier adaptation of the next generation higher bit rate systems.

While the IEEE 802.3 400GbE working group is starting its work on 400GbE, corresponding activities at the ITU-T have also started. Because most optical network operators prefer to manage their optical links using OTN performance measurements rather than Ethernet ones, the ITU-T is beginning work on defining the next OTN frame structure, beyond 100G. It is noted that in this paper, the bit rates referenced are nominal and not actual bit rates. Interested readers should see the relevant standards for the actual bit rates of any signal. For example, the so-called 100-Gbit/s OTU4 (Optical Transport Unit 4) actually has a bit rate of 111,809,973.568 kbit/s plus or minus 20 parts per million [1].

OTN Line Signal (ITU-T G.709)

OTUk Line Rate (kbit/s)

0PUk Payload Rate (kbit/s)

OTU Frequency Accuracy (Parts per Million)

OTU1 2,666,057 2,488,320 +/- 20

OTU 2 10,709,225 10,037,629 +/- 20

OTU 2e 11,095,727 10.356,012 +/- 100

OTU3 43,018,413 40,150,519 +/- 20

OTU3e2 44,583,355 41,611,131 +/- 100

OTU4 111,809,973 100,376298 +/- 20

Background on ITU-T Structure

The ITU-T is a technical subcommittee of the United Nations, tasked with developing global standards related to the telecommunications industry. It is subdivided into Study Groups, where Study Group 15 is the lead study group for optical networking. Study groups are subdivided further into Working Parties and Questions, where the actual technical work is conducted. Question 11 of Study Group 15 is the group responsible for developing the next OTN frame structure to be included in ITU-T Recommendation G.709 [1].

All the work is contribution-driven, meaning it is conducted based on discussion around proposals submitted by member companies. The best solutions result from collaboration by industry experts from all over the world, focused on a solution that will be the best for the global industry and not favoring any particular company or region of the world. A good global standard largely eliminates the need for additional regional standards and makes it possible for companies to build and sell products globally, by complying with the relevant ITU-T standards (called Recommendations).

Having dependence on a single set of standards also reduces development costs and offers the opportunity for volume based pricing. Maintaining the lowest cost brings the potential for higher supplier, manufacturer, and service provider profits. Most importantly, adherence to global standards should also result in lower cost of service for consumers.

For these reasons, companies collaborate to develop OTN standards.


Components of an OTN Interface Definition Beyond 100G

The work to define and standardize an OTN interface beyond 100G is not limited to the definition of a new frame structure (to be added to ITU-T G.709 Recommendation). Standardization in complementary areas is also necessary and will require enhancements to other ITU-T Recommendations (or the development of new ones). The other areas include a physical interface component ([2], developed in Question 6 of Study Group 15 – Q6/15), an equipment management component ([3, 4], developed in Q9/15 in the past, in Q11/15 going forward), and potentially even components impacting overall network architecture ([5], developed in Q12/15) and network management ([6, 7], developed in Q14/15).

Q11/15 is leading the way by defining the framing structure, collaborating with the other Questions along the way. Once a framing structure is defined, then the other aspects can be completed as well.

Framing Structure for the Next OTN Interface

In the early stages, all proposals for how to frame a client signal at a bit rate greater than 100 Gbit/s were being accepted and added to a “G.709 Beyond 100G Living List.” This list was used to track all the questions that need to be answered, and the contributions and proposals related to each one. It should eventually facilitate an objective comparison of the options and selection of a final version for inclusion in a future version of ITU-T G.709 Recommendation.

Initially, all of the proposals for the OTN frame beyond 100G fit into one of three categories.

1. The first was to define a fixed OTN frame (ODU5 – Optical Data Unit 5 – and OTU5 – Optical Transport Unit 5) at a single bit rate, much like most lower rate OTN signals (ODUk/OTUk, k= 1, 2, 2e, 3, 4).

2. The second was to define a “flexible” single-OTU with a higher bit rate that would always be managed as a single signal like the existing OTUk. It was “flexible” in that the frame structure might be applied to any number of higher bit rates (e.g. 400 Gbit/s, 500 Gbit/s, and 1 Tbit/s).

3. The final option was to define a “flexible” multi-OTU structure where the new beyond 100G signals may be inverse multiplexed into multiple separate ODU/OTU, possibly even reusing the existing ODUk/OTUk. The multiple signals might be managed separately or together, given they would each carry full ODUk/OTUk overhead. At the receiving end, the constituent ODUk/OTUk components would be deconstructed and the original beyond 100G signals reassembled.


Figure 2: Single ODUx/OTUx/OChx

Figure 3: Multiple ODUxi/OTUxi/OChxi

slice

Fiber

Central Frequency

single signal frequency slot

...

Multiple Optical Carriers

Flexi Grid

Central Frequency



Frequency Slot

ODUx(H)

OTUxi OTUxi

n*ODU4

Central Frequency



OTUxi

OChxi OChxi OChxi

ODUxi ODUxi ODUxi

Frequency SlotFrequency Slot ...

ODUx(L)

Client

OTLk.n electrical lanes not

shown

slice

Fiber

...


Single Optical Carrier



50GHz (Fixed Grid)

50GHz (Fixed Grid)

50GHz (Fixed Grid)



Frequency Slot

Central Frequency

Central Frequency

Central Frequency

...

OChx

ODUx(H)

Packet

n*ODU4

Frequency Slot Frequency Slot

ODUx(L)

OTL1

00

G.4

#1

OTLk.n OTLk.n OTLk.n

1T = 10* 100Gbit/s

25G electrical lane signal

OTUx(H)O

TL10

0G

.4 #

2

OTL1

00

G.4

#3

OTL1

00

G.4

#

OTL1

00

G.4

#1

OTL1

00

G.4

#2

OTL1

00

G.4

#3

OTL1

00

G.4

#

OTL1

00

G.4

#1

OTL1

00

G.4

#2

OTL1

00

G.4

#3

OTL1

00

G.4

#

1Tbit/s = 10*100GOS OS OS

OCh: Optical ChannelOS: Optical Signal


Both proposed “flexible” options were based on smaller payload increments, called tributary slots. The frame structure would be defined based on the number of tributary slots it contained and might only be resized (if resizing is standardized) in increments of those tributary slots.

Today, ODUk are made up of 1.25-Gbit/s tributary slots (each one capable of carrying an ODU0-framed Gigabit Ethernet signal). However, there is concern that having such fine granularity at rates beyond 100G would be unnecessary and costly. So, proposals for the tributary slot size for a new interface above 100G currently range from using the same 1.25-Gbit/s tributary slot all the way up to using 400-Gbit/s tributary slots. Most early proposals were focused on using 100-Gbit/s tributary slots, but in the February meeting the consensus of Q11/15 was to use 10-Gbit/s tributary slots as the working assumption going forward. Signals at lower rates than the tributary slot will require an additional stage of multiplexing to be carried in the new beyond 100G interface. For example, a 1 GbE (or any other client signal below 10 Gbit/s) could not be efficiently mapped directly into an interface with 10 Gbit/s tributary slots, and would need to first be mapped into an ODU at 10G or higher, and then that higher rate ODU mapped into the new beyond 100G frame.

The tributary slot size to be used is only one of many questions that need to be answered, regardless of which framing choice is selected. These must be explored to understand the implications of the different options. The additional questions include those in the following list:

What is the actual bit rate of the new OTN frame(s)?

What bit rate(s) will be used for the multi-lanes on the line side?

How will client signals be mapped into the payload area?

Can multiple frequency slots or wavelengths be used to support the new interface?

Must those frequency slots be of equal size?

What FEC (forward error correction) will be used for the interface?

How much skew compensation must be supported?

The different proposals may have very different answers to the questions listed above and as a result, identifying the pros and cons of the different proposals will be a complex problem of many variables. It will be important for the interested parties to fully understand each proposal to be able to identify the important differences and implications of those differences.

For example, consider the fact that all digital signals beyond 100 Gbit/s are expected to be transmitted in multiple digital lanes, like 100 GbE and the OTN digital frame defined to carry it, OTU4. These parallel digital signals are then to be carried on multiple optical wavelengths or optical sub-carriers within a single optical wavelength. If multiple optical wavelengths are used, they may be closely packed in a single optical block of spectrum, or they might be scattered across the channels within a single fiber, across multiple fibers in the same cable, or even across multiple routes through a network.

If one were to select the “flexible” multi-OTU option, then there is overhead to monitor and manage the transport of signals across multiple wavelengths or even multiple fibers. However, managing the constituent OTU signals as a single entity may be far more complicated with that option, depending on how widely the constituent signals are spread.


Recombining the different digital lanes becomes more complex the further the signals are scattered in the optical transport, given the potentially higher and higher skew (time offset resulting from different propagation times) between the lanes. However, one may minimize waste of optical spectrum if the entire beyond 100G signal does not have to fit within a single wavelength or contiguous group of wavelengths.

Three Applications/Interfaces and FEC

Another thing to keep in mind in following or contributing to the work on the next OTN frame is the fact that there are multiple interfaces that will utilize it.

Existing OTUk (k=1, 2, 2e, 3, 4) frame structures are mapped into optical channels (OCh) and used in three different applications. The interfaces in the three applications have different levels of standardization, which will be true of the next rate as well.

The most highly standardized is the inter-domain interface (IrDI), used for interconnection between different types of equipment regardless of the manufacturer. It is fully standardized to ensure interoperability on the client side of transport equipment.

The second interface type has a frame definition and some functional specifications to support them, but is not fully standardized for interoperability between equipment of different vendors. This is the multi-vendor intra-domain interface (IaDI). It is used when one is traversing a standardized transport link, meeting the specifications of a “black link” [8, 9], for example. Interoperability between different vendor equipment may be possible, though it may require negotiations and adjustments between the two equipment manufacturers attempted to interoperate.

The third and final interface type is the single vendor IaDI. In this final case, no interoperability is expected with other vendors, so even the frame structure may be modified to accommodate a vendor’s unique development choices.

The two IaDI interface applications are typically used on the line side of optical transport equipment, though they may also be used on the client side where interoperability with other vendor equipment is not required.

With the different interface applications in mind, the standardization efforts will focus on the IrDI and multi-vendor IaDI specifications.

One should keep in mind that the different applications are likely to require different choices for FEC. FEC is used to correct errors digitally, improving performance of an optical connection or extending the reach over which it can be operated cleanly.

In the past, Q11/15 considered standardizing the use of a stronger FEC code for the OTU4 IrDI than the Reed-Solomon (255,239) (RS(255,239)), specified in G.709 for other OTUk. However, it was noted at the time that even the OTU4 IrDI is a multi-lane interface and not a serial stream of bits at over 100 Gbit/s. Analysis by Q6/15 showed that given the IrDI is a client side interface, and not a long haul interface, the performance of RS(255,239) was sufficient.


Most optical transport manufacturers offer stronger FEC on their long haul IaDI interfaces. Given those are not expected to be interoperable between different vendors’ equipment, the FEC does not require ITU-T standardization. As part of the new frame definition, Q11/15 will need to determine (with input from Q6/15) whether RS(255,239) is sufficient for the new IrDI or if a stronger FEC will need to be selected for standardization in G.709.

Latest Development Based on the February 2013 Interim Meeting in Dallas, Texas, USA

An ITU-T SG15/Q11 Experts Meeting was held in Dallas, Texas, USA attended by experts all over the world. The approach and objectives for this meeting were:

Eventually develop a concise (simple) specification of the digital signals (electrical lanes) that are passed between the electrical (Q11/15) domain and the optical (Q6/15) domain;

Minimize the overall complexity of converting a high rate (Beyond 100G or B100G) bit stream (OTUCn) from electrical into one or more optical signals that can be transmitted over network media channels and converting the optical back into the (original) electrical bit stream;

Explore the method to compensate for the skew between the electrical bit streams carried by the OCh signals.

Several assumptions were agreed to at this meeting. These assumptions are important to assure progress will be made on a uniform and consistent manner. The assumptions are:

Processing of the interface signal will be based on electrical lanes:

o Candidate rates include 28G, 40G, 56G

o Selection will be influenced by IEEE decisions on 400GbE

Objective to use a common electrical lane format and framer chip for:

o 400GbE;

o Multi-vendor Inter-Domain Interface (IrDI); and

o Single-vendor Intra-Domain Interface (IaDI).

Frame format based on n x 100G (frame increments) or m x lane rate increments (e.g. m x 28G, 40G, 56G), standardizing only certain values of n or m

o Initial standardized IrDI rate would be ~400G and capable of carrying 400GbE

The group agreed at the meeting that it was premature to make any major decisions on defining B100G OTN interfaces at that time. A more detailed list of questions that still need to be answered was created to help guide future contributions on the subject. Though many of the drivers for specifying new higher rate interfaces are known, there were differing opinions on the urgency for the new standards. Future meetings will continue to discuss proposals for how to specify the new OTN interfaces. As the options are better understood, the best solutions will be selected.


As that is done, the advantages, consequences, and risks related to specifying the interface too early or too late will be weighed. Once the interfaces are sufficiently specified and the advantages are greater than the negative consequences and risks, it is expected that the ITU-T will finalize the next OTN interface standards. Exactly when that will be is still unknown, though the earliest possible conclusion is in 2014.

Summary

There are a lot of activities in progress for the development of standards for OTN interfaces beyond 100G. The IEEE 802.3 group is also working hard to define the 400G Ethernet interfaces. With the ITU-T and the IEEE working closely together and in parallel, it is expected that the standards required for future beyond 100G systems will be available in the next one to three years (2014-2016).

References

[1] ITU-T Recommendation G.709/Y.1331 (02/2012) “Interfaces for the optical transport network,” pre-published.

[2] ITU-T Recommendation G.959.1 (02/2012) “Optical transport network physical layer interfaces.”

[3] ITU-T Recommendation G.798 (04/2012) “Characteristics of optical transport network hierarchy equipment functional blocks.”

[4] ITU-T Recommendation G.798.1 (04/2011) “Types and characteristics of optical transport network equipment.”

[5] ITU-T Recommendation G.872 (07/2010) “Architecture of optical transport networks.”

[6] ITU-T Recommendation G.874 (04/2012) “Management aspects of optical transport network elements.”

[7] ITU-T Recommendation G.874.1 (01/2002) “Optical transport network (OTN): Protocol-neutral management information model for the network element view.”

[8] ITU-T Recommendation G.698.1 (11/2009) “Multichannel DWDM applications with single-channel optical interfaces.”

[9] ITU-T Recommendation G.698.2 (11/2009) “Amplified multichannel dense wavelength division multiplexing applications with single channel optical interfaces.”


Maximizing Network Capacity, Reach and Value Over land, under sea, worldwide

Edition Date: May 2013

Version: 1.0

Documents

Developing OTN Interface Standards for Beyond 100G