LH DWDM HeavyReading

HEAVY READING | VOL. 5, NO. 2, FEBRUARY 2007 | LONG-HAUL DWDM: MARKET & TECHNOLOGY OUTLOOK

Use of this PDF file is governed by the terms and conditions stated in the Subscriber License Agreement included in this file. Any violation of the terms of this Agreement, including unauthor-ized distribution of this file to third parties, is considered a breach of copyright. Light Reading Inc. will pursue such breaches to the full extent of the law. Such acts are punishable in court by fines of up to $100,000 for each infringement.

Heavy Reading Independent quantitative research and competitive analysis of next-generation hardware and software solutions for service providers and vendors

KEY FINDINGS

The long-haul DWDM market grew by more than 30 percent in 2006, to $1.8 billion worldwide

The bandwidth glut is over, and operators are scrambling to add network capacity

Nearly all network operators either are now upgrading or will soon upgrade their DWDM backbones

Opex reduction tops the list of operator pri-orities for core DWDM

Residential Internet service is a far bigger driver of core band-width demand growth than video services

Vendors must focus on rolling out 40-Gbit/s Ethernet and keep developing 100-Gbit/s Ethernet for the future

Core switching must evolve to become better optimized for packet transport

VOL. 5, NO. 2, FEBRUARY 2007

Long-Haul DWDM: Market & Technology Outlook Long-haul DWDM transport was one of the most fiercely pun-ished markets after the telecom collapse of 2001, and it has been one of the slowest segments to return to health. In 2006, however, real signs of recovery finally became evident through-out this market, in every region of the globe: Operators had al-ready consumed whatever bandwidth glut may have existed in their backbone networks, and a raft of new broadband applica-tions and users were demanding more capacity. Operators are reacting beyond simply adding capacity to existing systems; many are opting to deploy wholly new systems, taking advantage of tremendous improvements in technology and eco-nomics over the past few years. This strategy shift has made the long-haul DWDM sector competitive and fresh for the first time in nearly seven years. This report examines the core DWDM market using a combina-tion of direct one-on-one interviews with key carriers and an ex-clusive worldwide survey of service provider professionals. The report evaluates the long-term health of the core DWDM trans-port sector; the current and future drivers of growth; and the ob-stacles and challenges that remain. It also provides a compara-tive analysis of suppliers in this demanding sector. AUTHOR: SCOTT CLAVENNA, CHIEF ANALYST, HEAVY READING

HEAVY READING | VOL. 5, NO. 2, FEBRUARY 2007 | LONG-HAUL DWDM: MARKET & TECHNOLOGY OUTLOOK 2

TABLE OF CONTENTS

LIST OF FIGURES................................................................................................ 3

I. INTRODUCTION & KEY FINDINGS.....................................................................4

1.1 Key Findings..........................................................................................................5 1.2 Report Methodology ..............................................................................................7 1.3 Report Structure ....................................................................................................8

II. LONG-HAUL DWDM MARKET ASSESSMENT ..................................................9

2.1 Demand Drivers ..................................................................................................10 2.2 Other Considerations ..........................................................................................11

III. OPERATOR DEPLOYMENT PLANS FOR CORE DWDM ................................13

3.1 Operator Backbone Expansion Plans .................................................................13 3.2 Operator Requirements & Impediments in the Core DWDM Market...................14 3.3 40-Gbit/s Trends in Operator Backbone Networks..............................................15 3.4 100-Gbit/s Ethernet & Core DWDM Networks ....................................................17

IV. CORE DWDM ARCHITECTURES & OPERATIONS .........................................20

4.1 "Alien Wavelengths" & the Integration of WDM Optics on Client Equipment ......20 4.2 Switching in the Core DWDM Network................................................................21

V. NETWORK OPERATOR INTERVIEWS AND ANALYSIS .................................25

5.1 AT&T Inc. ............................................................................................................25 5.2 Comcast ..............................................................................................................26 5.3 Interoute Communications Ltd. ...........................................................................27 5.4 Level 3 Communications Inc. ..............................................................................28 5.5 Verizon Communications Inc...............................................................................29

VI. VENDOR PROFILES AND STRATEGIC ANALYSIS ........................................32

6.1 Alcatel-Lucent......................................................................................................32 6.2 Ciena Corp. .........................................................................................................32 6.3 Ericsson AB.........................................................................................................33 6.4 Huawei Technologies Co. Ltd. ............................................................................34 6.5 Infinera Corp........................................................................................................37 6.6 Nortel Networks Ltd.............................................................................................38 6.7 Siemens AG ........................................................................................................40 6.8 Xtera Communications Inc. .................................................................................41 6.9 ZTE Corp.............................................................................................................41

APPENDIX A: ABOUT THE AUTHOR...........................................................................43

APPENDIX B: LEGAL DISCLAIMER ............................................................................44


LIST OF FIGURES*

SECTION I

Figure 1.1 Operator Plans for Backbone DWDM Upgrades....................................................5 Figure 1.2 Survey Respondents by Type ................................................................................7 Figure 1.3 Survey Respondents by Geographic Region .........................................................8 Figure 1.4 Survey Respondents by Job Function....................................................................8

SECTION II

Figure 2.1 Evolving Provider Transport Network Strategies....................................................9 Figure 2.2 Annual Growth Estimates for Backbone Transport Capacity ...............................10 Figure 2.3 Backbone Transport Capacity Growth Drivers .....................................................11

SECTION III

Figure 3.1 Worldwide Network Operator Plans for Backbone DWDM Upgrades .................13 Figure 3.2 Network Operator Backbone DWDM Capacity Expansion Plans ........................14 Figure 3.3 Top Attributes of Backbone DWDM Systems for Operators Worldwide ..............15 Figure 3.4 Current Limitations of Operator Backbone DWDM Networks, Ranked................15 Figure 3.5 Most Common New Wavelength Rates in Long-Haul DWDM Networks .............16 Figure 3.6 Primary Drivers of 40-Gbit/s Wavelengths Today ................................................16 Figure 3.7 40-Gbit/s Interfaces Requirements in Operator Networks Today ........................17 Figure 3.8 Virtual Concatenation for 100-Gbit/s DWDM Transport .......................................18 Figure 3.9 Inverse Multiplexing for 100-Gbit/s Ethernet: Ethernet LAG vs. VC ....................19 Figure 3.10 Network Operator Approaches to 100-Gbit/s Wavelengths .................................19

SECTION IV

Figure 4.1 Value of Integrating DWDM Optics on Client Equipment.....................................20 Figure 4.2 Concerns About Integrating DWDM Optics on Routers & Switches ....................21 Figure 4.3 ITU-T G.709 OTN in the Network.........................................................................22 Figure 4.4 Switching Requirements at Core Optical Network Nodes....................................22 Figure 4.5 Wavelength Add-Drop/Switching Requirements on DWDM Backbones .............23 Figure 4.6 Network-Level Automation Values in the Optical Core ........................................23 Figure 4.7 Operator Plans for GMPLS/ASON .......................................................................24 Figure 4.8 Value of GMPLS/ASON in Operator Networks ....................................................24

SECTION V

Figure 5.1 AT&T U.S. Domestic 40-Gbit/s OTN/DWDM Express Backbone ........................25 Figure 5.2 Comcast Regional Area Network Diagram...........................................................26 Figure 5.3 Verizon ROADM-Enabled Network Mode of Operation .......................................30

SECTION VI

Figure 6.1 Ericsson Optical Product Line ..............................................................................33 Figure 6.2 Huawei Optical Product Line ................................................................................34 Figure 6.3 Huawei Optical Contract Sales by Region, 2005 .................................................35 Figure 6.4 Huawei Optical Contract Sales by Product, 2005 ................................................35 Figure 6.5 Huawei Optical Backbone Deployments ..............................................................36 Figure 6.6 Huawei Optical Roadmap.....................................................................................36 Figure 6.7 Nortel's Metro Ethernet Network & Optical Strategy ............................................39

* All charts and figures in this report are original to Heavy Reading, unless otherwise noted.


I. Introduction & Key Findings Over the first half of this decade, the long-haul and ultra-long-haul dense wavelength division multiplexing (DWDM) markets lacked the energy, steady growth, and rapid innovation of metro optical networks, but in 2006 they began to show significant signs of improvement. Growth out-paced even the most sanguine forecasts, and the improving health of the wavelength services market, combined with ever-growing bandwidth demand, has set the stage for an ongoing surge in core DWDM investment by network operators. Long-Haul DWDM: Market & Technology Outlook provides a detailed look at this expanding market through an extensive and exclusive survey of network operators worldwide; a forecast of the equipment market; and information gathered from direct interviews with carriers and suppli-ers. The report analyzes an optical market segment that has been in the shadows of the higher-profile metro and regional markets of late. The findings of this latest research reveal a market in the midst of an impressive recovery, as operators not only shore up capacity on their installed base of long-haul DWDM gear to keep up with demand, but also overlay routes or entire back-bones with new gear to take advantage of the latest generation of equipment. The key questions now facing network operators include:

At what pace will backbone bandwidth demand continue to grow, and is there any way to "future proof" a core DWDM network?

How can network operators take advantage of new technologies to improve provisioning times and lifecycle costs, given their capital constraints?

How will the trend in the metro-aggregation network toward Ethernet and packet network-ing affect network operator decisions in the core?

Will growth in the core network be cyclical or linear? Will wavelength services migrate to optical transport network (OTN) services? And will

OTN migrate to a true networking layer, beyond Sonet/SDH?

Will 40-Gbit/s transport be squeezed out of the market by 100-Gbit/s Ethernet? This report explores each of these questions in detail, with some answers provided by an exclu-sive worldwide survey of 88 service provider employees, designed to elicit their core DWDM de-ployment plans and feature requirements. The survey was supplemented by direct interviews with DWDM operators and vendors. Network operators interviewed for and analyzed in this report are:

AT&T Inc. (NYSE: T) Interoute Communications Ltd. Level 3 Communications Inc. (Nasdaq: LVLT) Verizon Communications Inc. (NYSE: VZ)

Equipment vendors interviewed for and analyzed in this report are:

Alcatel-Lucent (NYSE: ALU) Ciena Corp. (Nasdaq: CIEN) Ericsson AB (Nasdaq: ERIC) Huawei Technologies Co. Ltd. Infinera Corp. Nortel Networks Ltd. (NYSE/Toronto: NT) Siemens AG (NYSE: SI; Frankfurt: SIE) Xtera Communications Inc. ZTE Corp. (Shenzhen: 000063; Hong Kong: 0763)


1.1 Key Findings The long-haul DWDM market is on fire. In 2006, the long-haul DWDM market including those systems designed for backbone networks, with spans in excess of 1,000 km grew more than 30 percent, to approximately $1.8 billion. This growth benefited nearly every long-haul DWDM ven-dor worldwide. There are many drivers, but the market is clearly ramping quickly to a cyclical peak, driven by converging factors of intense bandwidth demand from broadband Internet and mobile applications; a healthy, stabilized wholesale bandwidth and wavelength services market; new core overlays to address the requirements of newly merged operators; and generational up-grades due to aging DWDM infrastructure that is no longer economical to expand incrementally. The "bandwidth glut" is over. Operators worldwide are reporting limited bandwidth supply and more stable pricing. Many new backbone optical builds are underway, and there are few if any "distressed assets" in the carrier market that can disrupt pricing and supply in wholesale markets. The vast majority of network operators either are upgrading, or soon will upgrade, their DWDM backbones. According to our service provider survey, 52 percent of respondents say their company is currently expanding its DWDM backbone; another 42 percent will expand in 2007 or 2008. As shown in Figure 1.1, only a miniscule percentage of our respondent base either planned to expand their backbone in 2009, planned no expansion, or didn't know. Figure 1.1: Operator Plans for Backbone DWDM Upgrades

There is a strong trend toward the 10-Gbit/s Ethernet LAN PHY, as operators look to re-duce backbone costs by transitioning core router interfaces to Ethernet. In the core, the migration to an IP-over-optical architecture in which routers and multiservice switches directly link to WDM-based transport devices without the need for intermediary Sonet/SDH multiplexing systems has been underway for at least five years. This has typically been accomplished via 2.5-Gbit/s or 10-Gbit/s packet-over-Sonet interfaces on routers, but there has recently been a move to 10-Gbit/s Ethernet LAN PHYs in many operator cores. This transition is about econom-ics, as 10-Gbit/s Ethernet interfaces on routers typically cost less than packet-over-Sonet inter-faces. The Ethernet-over-wavelength approach represents the final link in the end-to-end OTN, and has allowed a number of operators to plan for a complete network transition to Ethernet. In the core, video services are trumped by residential Internet demand. In previous Heavy Reading research, video and IPTV were identified as the main bandwidth drivers in the aggrega-tion network; yet in this current survey of core network operators, video ranked near the bottom of the list of drivers. The top drivers were (in order): residential Internet access, wholesale private-line services, dedicated network services, and retail private-line services; video, surprisingly, gar-nered merely 3 percent of the vote. The explanation lies in current telecom video architectures, in


which carriers distribute video to their metro areas via satellite, caching, or some other efficient means. Once in the metro, things get complicated and video chews up capacity; but in the core, bandwidth is being driven more by the need to shore up Internet backbones and wholesale wave-length services than by any particular service. More operators are choosing overlays in the core. In our survey of network operators world-wide, 50 percent said they are upgrading core DWDM routes by simply adding capacity to de-ployed systems, while 27 percent are overlaying new DWDM systems on a route-by-route basis and another 23 percent are performing a complete backbone overlay of new DWDM systems. Operators may be getting the opex religion, finally. Operators put lifecycle costs at the top of their requirements for core DWDM systems, above initial installation cost and a range of next-gen features. Operators also point to the high ongoing operational costs (opex) and capital costs (capex) of adding additional capacity as key concerns. Vendors must focus on rolling out 40-Gbit/s Ethernet and keep developing 100-Gbit/s Ethernet for their next releases. Regardless of recent demonstrations and activity around 100-Gbit/s Ethernet, carriers all want to see 40 Gbit/s, so vendors have to respond. Cisco and its CRS-1 remains the main driver for 40 Gbit/s in long-haul DWDM, but this remains relegated to the core of the core IP networks, so volume will remain low, and it will be more of a nag than a big opportunity for DWDM vendors. In the survey, a fraction of operators responded that 40-Gbit/s economics can be superior to 10-Gbit/s economics, but anecdotal evidence and interviews does not support that claim today: 10-Gbit/s equipment is getting cheaper all the time, comes in nice small form factors, has a wealth of test equipment to support it, and has built up a solid opera-tional history. 40 Gbit/s is still challenged in every way, and is not on the same cost curve. 100-Gbit/s Ethernet is coming fast, and will negatively affect the 40-Gbit/s market. Most car-riers have a keen interest in seeing 100-Gbit/s Ethernet come to fruition soon. There are many solutions in the labs today from striped, multi-lane 5 x 20 Gbit/s, to a more sophisticated, serial-like 2 x 50 Gbit/s, leveraging 40-Gbit/s optoelectronics that can get carriers to some kind of 100-Gbit/s transport within three years. This puts real pressure on the 40-Gbit/s market, and may just spell its contraction, after a brief period of growth. OTN is moving to the client side slowly. By virtue of its improved performance management and longer reach, the optical transport network (OTN) has become well established on the net-work side of long-haul DWDM. But operators, mainly in Europe, are now looking at client-side OTN, mainly to offer inter-carrier OTN services and support interconnect from OTN-enabled metro aggregation gear. Future applications of OTN include a more complete migration into the metro network, through introduction of an optical channel data unit (ODU) that would contain a Gigabit Ethernet signal and provide networking of individual Gigabit Ethernet lines through a switched OTN. In Heavy Reading's survey of network operators, 62 percent of respondents saw a clear and compelling need to transition from Sonet/SDH to OTN. OTN faces some limitations to adoption, particularly outside of Europe, with the installed base of Sonet/SDH and DWDM equipment topping the list. The resistance is strongest in the U.S. and Asia, but smaller operators are starting to take advantage of OTN and will begin moving the market in that direction. Of the largest operators, Verizon (particularly Verizon Business) is an advocate of OTN migration, while AT&T remains on the sidelines for now. "Alien wavelengths" and direct-connect DWDM architectures are overhyped. Though Cisco has put a full solution onto the market, and most long-haul DWDM suppliers support some kind of alien wavelength architecture, a core DWDM solution that is largely based on passive multiplex-ing of International Telecommunication Union (ITU)-grid wavelengths coming off switches, routers, and metro gear just isn't taking hold with operators: Trials and requirements are com-monplace, but there has been very little purchasing. Carriers still prefer a hard demarcation point at the DWDM terminal, and consider the premium over alien wavelength architectures justified.


Core switching must evolve. All the operators we surveyed saw a real need to move beyond simple STS or VC4 switching systems to something more optimized for packet transport. Solu-tions such as Provider Backbone Bridges with Traffic Engineering and Transport MPLS are gain-ing interest in the metro-aggregation network of a number of large carriers, and setting the stage for a core switch that is packet-optimized in a similar fashion. This could become a new product line from transport switch vendors, rather than router or Ethernet switch vendors.

1.2 Report Methodology The methodology of this report was designed to gather information critical to equipment suppliers and service providers in the core DWDM transport market. Our primary research was conducted in two stages: First, Heavy Reading conducted a series of one-on-one interviews with employees of network operators typically senior network planners, directors of R&D, and network architects. Subjects covered included a review of current core DWDM deployments, transport requirements for future services, feature requirements for next-gen core DWDM, and timing of deployments. Second, Heavy Reading conducted a global online survey of telecom operators to gather informa-tion on their current core DWDM strategies and future technology plans. Heavy Reading's Fall 2006 DWDM Deployment Survey elicited 88 quality responses from employees of 88 unique ser-vice providers, including incumbent and competitive operators of every stripe. The majority of respondents were from incumbent PTTs and ex-PTTs, interexchange carriers (IXCs), Internet service providers (ISPs), and competitive local exchange carriers (CLECs). Fig-ure 1.2 breaks out the responses by service provider type. Figure 1.2: Survey Respondents by Type

The Heavy Reading survey reached a broad global audience, although it was concentrated in North America and Western Europe, as shown in Figure 1.3. The survey drew responses from a wide variety of service provider personnel. In this light, the survey results should be considered as a reflection of perceptions within operators, not as providing strict guidance on operator capex plans. Figure 1.4 illustrates the diversity of responses.


Figure 1.3: Survey Respondents by Geographic Region

Figure 1.4: Survey Respondents by Job Function

1.3 Report Structure Long-Haul DWDM: Market & Technology Outlook is structured as follows: Section II assesses the long-haul DWDM market, analyzing its current strength and growth pros-pects, and offers a five-year growth forecast, drawing on supplier data and carrier interviews. Section III details the results of Heavy Reading's service provider survey on technology require-ments for long-haul DWDM deployments. Section IV considers evolving service provider architectures for long-haul DWDM deployments. Section V profiles select service providers with long-haul DWDM plans of particular interest. Section VI examines the equipment vendors in the long-haul DWDM space.


II. Long-Haul DWDM Market Assessment The long-haul DWDM market was the first to crash in 2001, yet recently has been on a steady upswing as operator consolidation, ongoing bandwidth demand increases, and aging systems have combined to create opportunities for expansion and upgrades in nearly every operator net-work. In addition, carrier backbones themselves are evolving in a number of important ways. The table below, from AT&T, illustrates the new realities of a provider transport network, making clear just how comprehensive this evolution has been over the past five years. Figure 2.1: Evolving Provider Transport Network Strategies

OLD MODEL TODAY'S REALITY

Traffic growth steady and predictable onto multiple parallel backbones

Volatile traffic growth onto a single converged network

Client access rate is always less than the backbone rate

Client access channel rate approaches or equals backbone channel rate

Aggregation at ingress to backbone Greater amount of aggregation close to or done by the client

Backbone relatively static Agile backbone (ROADM, GMPLS)

Separate domestic and global pieces Seamless domestic/global network Source: AT&T One factor to note in this chart is the impact of operating a single converged network. While there is clearly a value in converging stovepipe networks onto a common IP/MPLS/DWDM backbone, this approach does make predicting bandwidth demand and growth rates on the backbone much more complex, if not impossible. Also, each of the different types of traffic traversing the common core will have varying characteristics and network requirements. Other key contributors to the recent growth of the long-haul DWDM market include:

Consolidated operators find the management of many separate vendors in their new backbones to be bothersome and an inhibitor to rapid deployment of capacity and ser-vices, and are often inspired to finance the overlay of a more unified core transport layer.

Next-generation network transition efforts are often about cutting opex in the face of erod-ing voice and wireline revenues. These efforts often involve creation of a more scaleable backbone, fueling deployments of long-haul and ultra-long-haul DWDM express layers.

Many core DWDM backbone routes in the U.S. and Europe are reported to be more than 75 percent utilized, creating demand for new overlays, or in some cases whole new backbones, if the operator chooses to move to next-gen equipment or a new vendor.

In China and emerging markets, there continues to be strong demand from both wireline and mobile operators for new backbone networks, creating a fierce battleground for sup-pliers and providing strong growth opportunities for vendors such as Huawei and ZTE.

Many systems installed in 1999-2000, even if not reaching capacity exhaust, are being retired in favor of newer systems that offer much improved economics and features.

Wavelength services are selling more briskly, as many ISPs, alternate operators, and mobile operators shore up backbones with 2.5-Gbit/s and 10-Gbit/s waves. The 10-Gbit/s Ethernet LAN PHY wavelength is particularly attractive to ISPs and content providers.

Private networks, research and education networks, and government networks are more often adopting long-haul DWDM to create end-to-end optical networks.


In the past two years, the results for the long-haul DWDM market, comprising those DWDM sys-tems designed for routes over 1,000 km, have been impressive. This market segment was not long ago the most impressive in the whole telecom industry, taking in more than $6 billion in 2001. Yet within a year the segment plummeted, and continued to fall over the next several years, hovering around $1 billion in annual revenues from 2002-2004. But there were real signs of growth in 2005, particularly overseas, and the market grew to $1.35 billion an increase of 22.7 percent over 2004. This was no fluke: Heavy Reading estimates the 2006 long-haul DWDM market at $1.77 billion, a increase of 31 percent over 2005. We also esti-mate that this growth rate will continue into 2007, as many operators worldwide continue to de-ploy full backbone overlays or add significant capacity to high-demand routes. How long will this growth be sustainable? Heavy Reading believes the optical market, particularly at the backbone, to be modestly cyclical. National backbone builds obviously don't happen every year, in every country. Most operators will undergo a transition from broad deployment to more focused capacity expansion by simply adding transponders to installed systems. China is cur-rently deploying a number of high-profile backbone networks, but will not continue this level of construction indefinitely. And in North America, the consolidation of network operators will ulti-mately have a negative effect on the total addressable market. That said, Heavy Reading expects the long-haul DWDM market to begin to plateau in 2009, and potentially contract by 2011. Predicting the timing of a contraction is difficult in the global telecom market today, because so many factors remain uncertain, but we do feel strongly that optical networks markets cannot grow indefinitely: They are infrastructure markets that mature, stabilize, contract, then rise again with a new wave of investment.

2.1 Demand Drivers In the network core, demand drivers always originate in the access and edge network, and spe-cific applications or services driving demand this deep in the network aren't always known. In our survey of network operators about their metro optical networks, there was a clear understanding of service and application drivers of bandwidth demand; while in the core, where the unit of cur-rency is often the 2.5-Gbit/s or 10-Gbit/s wavelength, operators are often not exposed to the un-derlying drivers of demand. In the core, operators are more attuned to overall bandwidth demand, and note shifts in customers, connectivity requests, and pricing fluctuations. Figure 2.2: Annual Growth Estimates for Backbone Transport Capacity

AT WHAT ANNUAL PACE IS TRANSPORT CAPACITY CURRENTLY GROWING ON YOUR DWDM BACKBONE NETWORK?

NUMBER OF RESPONSES

PERCENT OF TOTAL

Less than 10% 9 10%

10% to 30% 35 39%

30% to 50% 19 21%

50% to 100% 13 15%

More than 100% 3 3%

Don't know/Not sure 9 10%

Total 88 The responses to our survey reveal a widespread perception that core backbone traffic is growing at a significant pace, with nearly 40 percent of respondents seeing annual growth of 10 to 30 per-cent, and more than 35 percent of respondents seeing growth in excess of 30 percent per year. This pace of growth in the network core is significant, reflecting dramatic growth in the access


network, which is then reduced through the processes of aggregation and statistical multiplexing as it travels towards the network backbone. At these rates, core capacity will continue to be under constant pressure to scale cost-effectively, encouraging new lower-cost transport platforms. As noted above, the largest carriers are seeing a significant change in the operation of their core networks, addressing an increase in traffic volatility and uncertainty with a more agile, scaleable network. This occasions more than simply deploying DWDM systems that offer a lower cost per transported bit; there is a need to deploy systems that offer lower capex, opex, and agility through technologies such as reconfigurable optical add/drop multiplexing (ROADM), wavelength cross-connect, generalized multiprotocol label switching (GMPLS)/automatic switched optical network (ASON), and enhanced optical performance monitoring. This core growth will also continue to push the 40-Gbit/s and 100-Gbit/s markets forward. We are already seeing some major bandwidth users, such as large enterprise customers and research organizations, asking service providers about 100-Gbit/s wavelength services. The reason is fairly straightforward: 10-Gbit/s Ethernet switches are rapidly becoming the norm in large data centers, as nearly every server installed today attaches at Gigabit Ethernet rates. As connectivity require-ments continue to scale up, data centers will need to migrate beyond 10-Gbit/s Ethernet switching to 100-Gbit/s Ethernet. Once this has occurred, they will prefer to interconnect across the metro and wide area at these rates to support their business continuity and data protection plans. Figure 2.3: Backbone Transport Capacity Growth Drivers

WHAT IS THE BIGGEST DRIVER TRANSPORT CAPACITY GROWTH? NUMBER OF RESPONSES

PERCENT OF TOTAL

Retail private-line services (e.g., T1, T3, OCn) 11 13%

Wholesale private-line services (e.g., wavelength services) 15 17%

Dedicated network services (e.g., dedicated ring services) 14 16%

Frame Relay 1 1%

ATM 2 2%

E-Line retail (over fiber, over Sonet, over switched network) 4 5%

E-LAN retail 1 1%

E-Line wholesale 3 3%

E-LAN wholesale 2 2%

Dedicated Internet access 3 3%

Layer 3 VPN 9 10%

Residential video 3 3%

Residential Internet access 17 20%

Storage-area network (SAN) transport/data center extension 2 2%

Total 87

2.2 Other Considerations When considering the top drivers of backbone transport demand, it is interesting to contrast this with recent surveys of operators about traffic growth in their metro networks. Here, in the back-bone, operators point to residential Internet access first, followed by their high-capacity private-line and dedicated network services. Ethernet and video are near the bottom. In the metro, the


reverse is true: Video is already starting to have a major impact in the metros of many operators, and Ethernet is driving bandwidth well above traditional leased-line services and Internet access. An important observation to make here is that in most operator networks, IPTV and Ethernet are really metro and access services, and do not always traverse the core network. Ethernet services often provide internetworking between customer locations in a single metro, or from a customer location to the nearest service provider point of presence (POP). For IPTV, operators often dis-tribute broadcast video signals efficiently via satellite or a handful of dedicated circuits across the nationwide network, while more interactive, unicast services such as video on demand (VOD) originate in local or metro networks, isolating their bandwidth impact. For backbone operators, the impact of peer-to-peer networking, arising from broadband Internet access, is much more profound than the impact of these newer services, and will continue to be the primary driver of capacity. It is also important to consider Internet video, which is distinct from an IPTV or "telco TV" service, in that video is delivered via a user's broadband connection to the PC, rather than via a dedicated video services network. Internet video is not easy to quantify and track, unlike cable operator or telco VOD services, which deliver packaged video content from local or regional servers directly to a customer's set-top box. That said, Internet video is likely just lumped into "residential Internet access" in our survey, and must be considered a major driver. Lastly, backbone operators are often in the wholesale business, and in these cases simply lease wavelengths to customers, and do not therefore track the purpose for which they are used. So in this survey, where wholesale private-line services came in second, it is safe to assume that those wavelengths could be used for a variety of purposes, including building out an operator's video delivery backbone. The important takeaway here, regardless of how the market drivers are interpreted, is the rate at which bandwidth demand is growing and the diversity of the drivers. The public Internet remains the top driver, and while broadband subscriber growth has begun to mature in many markets, the usage of broadband connections continues to expand, as does the access connection rate. Exist-ing users consume more bandwidth, even as new subscriber numbers slow. Beyond the con-sumer, the transition to Ethernet-based services in the corporate market enables this customer base to make a significant leap in bandwidth usage at little extra expense. Finally, in the research and education markets, customers are often building their own high-speed optical infrastructures, using carriers for wholesale connectivity at 2.5-Gbit/s and 10-Gbit/s speeds.


III. Operator Deployment Plans for Core DWDM Heavy Reading's forecasts for the DWDM market are created through a blend of analyses: opera-tor surveys, interviews, and supplier interviews. In this section, we provide detailed results from Heavy Reading's Fall 2006 DWDM Deployment Survey, which shed a great deal of light on the DWDM market, and provide ample evidence to suggest that the market is in a period of rapid growth and transition, as more operators are deploying overlays of next-generation technology, while making a concerted transition to OTN, 40 Gbit/s, and a more dynamic optical layer.

3.1 Operator Backbone Expansion Plans Looking at plans for backbone upgrades with DWDM, the survey shows a clear level of backbone expansion activity currently underway (52 percent of respondents), while showing another 42 percent will be expanding over the next two years. In our conversations with core DWDM suppli-ers, all are seeing growth beyond their expectations currently and is the past three or four quar-ters. Sales are up around 25 percent in 2006, which is well above initial expectations of a 15 per-cent growth for the year. Similar numbers are expected for 2007. Figure 3.1: Worldwide Network Operator Plans for Backbone DWDM Upgrades

This activity is most pronounced in 2006 (when the survey was conducted), followed by a signifi-cant number in 2007. The numbers dwindle from there, which raises the question of whether the core DWDM market can continue to expand at rates in excess of 20 percent per year, or if this market is cyclical in nature. Arguably, this market could follow six- to seven-year cycles indefi-nitely, driven more by the introduction of new generations of technology into the marketplace than by construction and deployment cycles. Much of the anecdotal data gathered by Heavy Reading for this report indicates that in many cases, operators are deploying new backbone DWDM systems not because their currently de-ployed systems have approached maximum capacity, or "fill," but because continuing to expand capacity with current systems is too expensive. In many cases, an operator with sufficient budget sees an advantage in deploying an overlay backbone DWDM layer with the latest generation of technology, rather than continuing to pay a premium to add linecards to the installed base. In years such as 2006 and 2007, when indications are that a real spike in demand for backbone DWDM occurred worldwide (driven by the combined factors of emerging markets expansion, in-stalled base obsolescence, and rapid bandwidth demand from a more video-centric Web), it may be safe to assume that this is the peak of a cycle of infrastructure deployment, and from here the market will level and ultimately contract slightly.


The DWDM market, though now nearly two decades old, is still quite young in terms of tracking cycles so there is little historical information to draw upon to gauge the shape of the next cycle. The situation is further confounded by the fact that the drivers underlying this market are rapidly changing, quite unlike the more consistent cyclical drivers that define other commodity industries. That said, we believe the DWDM market has enough fuel to grow over the next three years, and then may begin to ease off by 2009. In the Heavy Reading survey, we asked respondents to identify how their companies are planning to expand the capacity of their DWDM backbones. The responses to this question will indicate the way in which the core DWDM market is growing, which will have an impact on how individual suppliers fare. The central question may be considered: Will spending predominantly flow to sup-pliers who already have an installed base of equipment in the network, or will the trend toward deployments of overlays benefit new entrants and support shifts in market share? Figure 3.2: Network Operator Backbone DWDM Capacity Expansion Plans

HOW IS YOUR COMPANY UNDERTAKING ITS BACK-BONE DWDM CAPACITY EXPANSION?

NUMBER OF RESPONSES

PERCENT OF TOTAL

We will light more wavelengths on existing DWDM systems on a route-by-route basis. 42 50%

We will overlay new DWDM systems (lighting new fibers) on a route-by-route basis. 23 27%

We will do a complete backbone overlay of new DWDM systems. 19 23%

Total 84 The results are encouraging for new entrants. Fully half of the respondents to our survey say their company is deploying core DWDM systems as overlays, opening up significant opportunities for new entrants and for market-share shifts among the larger incumbent suppliers. The most signifi-cant result here is the 27 percent of respondents who claim their company will perform a com-plete backbone overlay of new DWDM systems indicating a decisive move away from the route-by-route approach to backbone scaling that has been predominant over the past five years, and toward a much more ambitious approach. Indeed, one of our most important observations about the long-haul DWDM market is the speed at which market share has changed in recent years. The market has radically shifted first as Huawei stormed on the scene, buoyed by significant Chinese contracts, then by explosive busi-ness in emerging markets; and then as Infinera charged in, rapidly snapping up business with major wholesale backbone operators in the U.S. and Europe, while setting the stage for more Tier 1 carrier wins in the future.

3.2 Operator Requirements & Impediments in the Core DWDM Market Our online survey also asked operators to provide input on which features they considered most important in their equipment purchases for backbone DWDM infrastructure. They were asked to rank each feature on a scale of 1 to 10, with 10 being the most important. On average, respon-dents ranked lifecycle costs highest indicating an increased focus on the operations costs of DWDM systems, rather than capital costs, which had been the paramount consideration over the past five years. Ranking second was "transparent support of any protocol" a key endorsement of OTN or any other solution that can enable this functionality. Notice that support for "alien" wavelengths and ultra-long reach fell near the bottom of the list both features many DWDM vendors tout in their marketing, but which are clearly not of great importance to operators, as gauged by our survey.


Figure 3.3: Top Attributes of Backbone DWDM Systems for Operators Worldwide

ATTRIBUTE SCORE

Lifecycle cost 7.80

Transparent support of any protocol 7.68

Ease of design, installation, and service provisioning 7.44

First-installed cost 7.34

ROADM, wavelength crossconnect support 6.75

Integrated packet aggregation or switching 6.70

Network reach "multi-haul" systems that span metro-regional to long-haul 6.66

Automated optical control plane (GMPLS/ASON) 6.60

Sub-wavelength crossconnect support 6.58

Support for 40-Gbit/s wavelengths 6.20

Roadmap to 100-Gbit/s support 6.04

Support for "alien" wavelengths 6.02

Ultra-long reach > 2,000 km unregenerated span support 5.36 We next asked respondents to rank the limitations of their current backbone DWDM systems, to explore how these aligned with current technological developments. The leading response the capital cost of adding capacity supports our thesis that part of the core DWDM market's recent growth is spurred by aging infrastructure that is too expensive to scale. Second is operations cost, which is clearly rising in importance as operators look to scale their networks to support a flood of bandwidth from new broadband services. And in third place is lack of flexibility a con-cern that supports the current trend toward the adoption of ROADM, wavelength-selective switch-ing (WSS), and tenability in DWDM systems from metro to core, which in turn supports operator requirements for more rapid provisioning of wavelengths and services from their transport cores. Figure 3.4: Current Limitations of Operator Backbone DWDM Networks, Ranked

ATTRIBUTE SCORE

Capital cost of adding additional capacity 7.21

Ongoing operations costs 6.72

Lack of flexibility (ROADM, WSS, tunables) 6.62

Poor planning tools, complex network design 6.29

Reach limitations 5.83

No support for 10-Gbit Ethernet 5.67

No support for OTN 5.64

No support for 40-Gbit/s 5.21

3.3 40-Gbit/s Trends in Operator Backbone Networks DWDM has completely overtaken Sonet/SDH systems in carrier backbones, reducing them to linecards on DWDM systems, instead of standalone ADMs. The most common line rate deployed today is 10 Gbit/s, and there is a strong trend toward the 10-Gbit/s Ethernet LAN PHY, as opera-


tors look to reduce their backbone costs by transitioning core router interfaces to Ethernet. In our survey, 61 percent of respondents identified 10 Gbit/s as the most common new wavelength rate deployed in the long-haul DWDM network, while 28 percent said it was 2.5 Gbit/s. Figure 3.5: Most Common New Wavelength Rates in Long-Haul DWDM Networks

Surprisingly, nine operators (representing 10 percent of the survey) identified 40 Gbit/s as the most common new wavelength deployed. This was not corroborated in our interviews with major service providers, so it is likely that these respondents are from specialized operators, and not reflective of the overall market. The 40-Gbit/s market is certainly growing, but it remains a niche market, representing less than 10 percent of new wavelengths deployed, and is driven almost exclusively by 40-Gbit/s router port installations. This assessment, however, is somewhat contradicted by the results of our next survey question, regarding the drivers for deployment of 40-Gbit/s wavelengths. While we see 40-Gbit/s interfaces on routers as the primary driver, the survey responses are split: Of the 32 percent of respondents whose companies are deploying such wavelengths today, nearly half identify the need to inter-connect with 40-Gbit/s interfaces on new core routers as the strongest driver, while the other half point to the superior economics of 40-Gbit/s over 10-Gbit/s wavelengths. (The 7 percent of re-spondents who identify wholesale requests as the top driver could be counted in either camp.) Figure 3.6: Primary Drivers of 40-Gbit/s Wavelengths Today

IF YOUR COMPANY IS DEPLOYING 40-GBIT/S WAVELENGTHS TODAY, WHAT IS THE PRIMARY DRIVER?

NUMBER OF RESPONSES

PERCENT OF TOTAL

Need to interconnect with 40-Gbit/s interfaces on new core routers 10 12%

Superior economics to 10-Gbit/s wavelengths 10 12%

Wholesale service request from customers 6 7%

Other (please specify) 1 1%

We are not deploying 40-Gbit/s wavelengths. 54 67%

Total 81


Based on both our carrier interviews and data from technology suppliers, we continue to believe that the 40-Gbit/s market remains tied to router deployments with 40-Gbit/s interfaces. While 10-Gbit/s transponders on long-haul DWDM systems typically cost about $12,000 to $15,000 each, 40-Gbit/s transponders today cost three or even four times as much, offering no economic advan-tage. In addition to that price tag, 40-Gbit/s wavelengths also add cost throughout the network, in the form of more sophisticated and costly dispersion management solutions, newer amplifiers, and in some cases new fiber: Nearly all fibers deployed before 1994 are unsuitable for 40-Gbit/s wavelengths, and many operators have such poor inventories of their fibers that they can't be certain which are 40-Gbit/s-ready without field testing adding further cost, design complexity, and delays. In contrast, the costs of supporting 40-Gbit/s router interfaces are more easily justi-fied because of the savings on the router, while the limited volume of these connections allows an operator to "cherry-pick" specific fibers to use for an IP backbone. The prevailing wisdom in the market today is that 40 Gbit/s will start in the very core of the net-work and slowly make its way towards the edge, following the pattern 10 Gbit/s established when it overtook 2.5 Gbit/s. Our survey, however, does not support that thinking, at least at first glance. When asked to identify where in their company's network 40-Gbit/s interfaces are required today, the most common response was actually metro DWDM networks, followed by the more predict-able "ultra-long-haul/express DWDM core." In addition, the survey showed equal responses for long-haul DWDM and regional DWDM networks; thus, the results are spread almost evenly throughout the regions of operator networks in our survey. This data may or may not be in conflict with our observations that 40-Gbit/s deployments are currently tied to core router deployments, as DWDM systems of varying reaches are presently deployed to create backbone IP networks, de-pending the type of carrier and the geography in which they are deployed. Figure 3.7: 40-Gbit/s Interfaces Requirements in Operator Networks Today

WHERE IN YOUR COMPANY'S NETWORK ARE 40-GBIT/S INTERFACES REQUIRED TODAY?

NUMBER OF RESPONSES

PERCENT OF TOTAL

Ultra-long-haul/express DWDM core 8 24%

Long-haul DWDM network 5 15%

Regional DWDM network 5 15%

Metro DWDM network 9 27%

No current need for 40-Gbit/s interfaces 6 18%

Total 33

3.4 100-Gbit/s Ethernet & Core DWDM Networks The drive to achieve 100-Gbit/s Ethernet transport has recently grown quite intense, driven by pressures from the data center to support aggregation of 10-Gbit/s Ethernet servers and clients; and from the transport network to achieve speeds well above 40 Gbit/s in the core. Based on data from our service provider survey and our interviews with major network operators, the average growth of bandwidth demand across the core today is between 50 and 100 percent per year a rate nearly impossible to maintain economically with current 10-Gbit/s transport solutions. Most operators today deploy transport technology with an eye toward replacement in five years the lifespan of particular technology generation. Given that bandwidth demand will be increasing between 16 and 32 times over that five-year lifespan, operators are becoming increasingly con-cerned that without a new high-speed line rate, optical systems won't be able to accommodate the growth in demand by simply adding new 10-Gbit/s channels. Adding whole new line systems to accommodate growth can be quite costly and disruptive to network operations.


Yahoo!, for one, finds that at least 60 percent of its data center, 33 percent of its metro, and 8 percent of its wide-area connections are already 10-Gbit/s Ethernet, setting the stage for 100 Gbit/s as a next-gen solution. According the company's director of network architecture, there is a pressing need for 100 Gbit/s from major content providers, driven by the following realities:

Their metro networks are 100 percent Ethernet Internet Exchanges are 100 percent Ethernet SANs are migrating to Ethernet, away from Fibre Channel, InfiniBand and other protocols Long-haul transport moving to 10-Gbit/s Ethernet LAN PHY

For long-haul DWDM suppliers, the challenge of achieving 100 Gbit/s on a single wavelength is the same as with 40 Gbit/s, yet exponentially worse (not just 2.5 times worse). That said, there is little hope of rapidly migrating to 100-Gbit/s wavelengths for DWDM transport. Instead, the Insti-tute of Electrical and Electronics Engineers (IEEE) and transmission engineers are focusing on possible solutions that create a 100-Gbit/s virtual port on a transmission system. Examples of solutions or specifications that enable very high-speed virtual ports include virtual concatenation (VCAT), link aggregation, multi-link bonding, parallel, and coarse wavelength division multiplexing (CWDM) optical interfaces such as the Optical Internetworking Forum's VSR-4 and VSR-5 and the multi-wavelength 10GBASE-LX4 for 10-Gbit/s Ethernet over multimode fiber. As work proceeds on 100-Gbit/s transport, parallel efforts are underway to achieve 100 Gbit/s in the LAN and data center. In the electronics world, the challenge is overcoming the limitations of copper traces on circuit boards and system backplanes. There, parallel approaches that combine multiples of 6-, 10-, and 25-Gbit/s signals are likely to be more successful than serial approaches. Whereas radio frequency, power consumption, and heat dissipation are typically the limiting fac-tors in electronics, in optical transmission it is non-linear effects, which contribute to chromatic and polarization mode dispersion (PMD), greatly reducing transmission distances of high-speed optical signals. Though 40-Gbit/s wavelengths have finally arrived that are capable of traversing typical DWDM routes, it is unlikely that pure 100 Gbit/s will achieve this cost-effectively in the next decade. Instead, most observers believe a multi-lane/multi-wavelength approach will be neces-sary one in which ten wavelengths of 10 Gbit/s are virtually bonded (using VCAT, for example) to travel the network as a 100-Gbit/s signal. Figure 3.8 illustrates this solution. Figure 3.8: Virtual Concatenation for 100-Gbit/s DWDM Transport

Source: Alcatel-Lucent


Other proposals include 5 x 20 Gbit/s or 4 x 25 Gbit/s. The attractiveness of these solutions lies in their use of lower-cost, lower-power optoelectronics and their ease of deployment across existing fiber networks. The 10 x 10 Gbit/s solution using VCAT is perhaps the most attractive, as it uses existing 10-Gbit/s technology as its unit of currency, leveraging the installed base of DWDM sys-tems for transmission, limiting the need for new 100-Gbit/s-capable equipment at endpoints only. Figure 3.9 compares Ethernet Link Aggregation (LAG) to the transport network's VCAT tech-nique as a solution for transporting 100-Gbit/s Ethernet. The VCAT solution is attractive because of is its wide use in Sonet/SDH networks today for lower-speed Ethernet transport, as well as its ease of integration with the OTN standard, which is becoming increasingly common in provider transport backbones. Figure 3.9: Inverse Multiplexing for 100-Gbit/s Ethernet: Ethernet LAG vs. VC

ETHERNET LINK AGGREGATION: DISTRIBUTE TRAFFIC TO LANES AT PACKET LEVEL

VIRTUAL CONCATENATION: DISTRIBUTE TRAFFIC TO LANES BY BYTE/COLUMN SLICING

Each lane is an independent MAC Must choose same lane for every

packet in a flow to maintain ordering, but then latency differences between lanes are irrelevant (need not be com-pensated by the receiver)

Large aggregate flows create load bal-ancing problems must try to parse in-dividual flows from inside various tun-neling mechanisms

Individual flows may be hidden, e.g., with IPSec

Inverse multiplexing done below the packet layer in the protocol stack

No packet awareness within an individual lane

Receiver buffers traffic to align with highest latency lane

Never a load balancing problem N bit/second data stream is distributed evenly into X bitstreams of N/X bits/second.

Remultiplexing reconstitutes the original bit-stream from which packets can be extracted

Source: Alcatel-Lucent Most participants in the standardization process today believe a fully approved 100-Gbit/s stan-dard will be adopted by the IEEE in 2010, leaving a rather short window of opportunity for 40-Gbit/s DWDM networks and perhaps squeezing that solution out of the market. Figure 3.10: Network Operator Approaches to 100-Gbit/s Wavelengths

WHAT IS YOUR COMPANY'S APPROACH TO 100-GBIT/S WAVELENGTHS?

NUMBER OF RESPONSES

PERCENT OF TOTAL

Under investigation 39 45%

Trialing 3 3%

Planning for deployment when available 11 13%

No interest 11 13%


Total 86 Surveyed operators were predictably quite interested in this technology: Most claim to be investi-gating 100-Gbit/s DWDM and even planning for its future deployment. In response to carrier in-terest, most DWDM vendors are developing 100-Gbit/s technology today. In nearly every case, vendors are looking first to multi-wavelength solutions, as attempts at single-wavelength 100-Gbit/s transmission (often using differential quadrature phase shift keying [DQPSK] encoding) has not proven technologically or economically feasible for long-haul DWDM in the near term.


IV. Core DWDM Architectures & Operations 4.1 "Alien Wavelengths" & the Integration of WDM Optics on Client Equipment One of the most interesting changes in the DWDM market over the past three years has been the development and positioning of architectures and solutions that support "alien wavelengths" or the implementation of DWDM optics on clients of the DWDM network. The goal of this architec-ture is network simplicity and capex reduction, as it reduces a set of transponders in each net-work connection by placing DWDM optics on a router, switch, or multiservice provisioning plat-form (MSPP), which then interfaces passively to the DWDM network. The DWDM network, in the logical extreme of this architecture, is made up of passive mux/demux units, managed optical amplifiers, wavelength switches, and signal-conditioning equipment. Vendors have approached this vision in two ways. Neither solution has fared terribly well in the market today, although some operators find deploying this architecture in a limited fashion to be workable and economic:

Router vendors such as Cisco position this architecture as a complete fulfillment of the IP-over-DWDM vision, tying IP equipment more tightly to the transport network and re-ducing overall network cost.

Transport vendors such as Nortel position this architecture as a lower-cost, more flexible way to deploy end-to-end optical networks, with metro MSPP and DWDM equipment in-terfacing to an agile, multi-haul photonic layer.

In our interviews with operators, most expressed an interest in this type of solution as a means of reducing network cost, but found that ultimately their vendors could not deliver solutions that radi-cally lowered cost without imposing additional management and operational burdens on the net-work. Our online survey found more than 60 percent of operators positive about the value of inte-grating DWDM optics on routers and switches to reduce transponders in a DWDM network, with most of the balance undecided. Figure 4.1: Value of Integrating DWDM Optics on Client Equipment

DO YOU SEE VALUE IN INTEGRATING DWDM OPTICS ON ROUTERS AND SWITCHES TO REDUCE TRANSPONDERS IN A DWDM NETWORK?

NUMBER OF RESPONSES

PERCENT OF TOTAL

Yes 54 62%

No 7 8%

Not sure 26 30%

Total 87 However, our one-on-one interviews with operators and suppliers revealed a deeper skepticism about this solution. Most suppliers told us they sold very little gear in this configuration, though they are continually asked to prove that it is an available option. Carriers and suppliers have often discussed and even trialed such solutions, but by and large operators have ultimately chosen to continue as they have, with a classic DWDM terminal and transponder as the hard demarcation point between the transmission network and the data network. Moving some of the transport functionality onto data gear raises many issues about network con-trol and management, and often does not reduce capital costs as much as advertised: Because a transponder moved from a DWDM terminal to a router or switch typically only replaces two plug-gable short-reach XFPs (if it's a 10-Gbit/s connection, for example), the resulting savings per link averages only $1,000. Given the additional management burdens, most operators choose to forego the cost savings in favor of operational certainty.


As shown in Figure 4.2, operators gave fairly equal weight to a variety of concerns around inte-grating DWDM optics on client gear, with "adds complexity to wavelength planning and manage-ment" as the only standout in the group. This echoes what we observed from our industry con-versations, which we believe will continue to put negative pressure on the success of this type of deployment for years to come. Figure 4.2: Concerns About Integrating DWDM Optics on Routers & Switches

WHAT ARE YOUR CHIEF CONCERNS WHEN CONSIDERING INTEGRATING DWDM OPTICS ON ROUTERS AND SWITCHES?

NUMBER OF RESPONSES

PERCENT OF TOTAL

Poor/limited interoperability with DWDM system management 11 13%

Adds complexity to wavelength planning and management 28 33%

Loss of end-end control/management plane information and control 13 15%

Inferior support from router/switch vendors for DWDM transport 15 17%

No opinion/Don't know 19 22%

Total 86

4.2 Switching in the Core DWDM Network Core DWDM networks have incorporated some level of switching for the last six years, serving the functions of protection, restoration, service initiation, and wavelength routing. Switching at core nodes is typically accomplished via wavelength switches and Sonet/SDH switches. Wave-length switches tend to be all-optical, in that optical signals are not electronically regenerated; rather, they are switched via an optical mechanism from one port to another, most often for sim-ple protection purposes. Sonet/SDH core switches are more complex systems that sit at major junctures in a core optical network and perform line-rate and subrate switching to support all switching functions in the core including bandwidth grooming, aggregation, protection, and res-toration. The core switch has emerged as an important product category for most transport net-work vendors, and is presently undergoing a transformation of its own. The amount of wavelength-level traffic traversing any core node continues to rise, and the need for switching, bandwidth management, and automation continues to increase as operators look to protect valuable core facilities and provision customer wavelengths rapidly and cost-effectively. In addition, many operators are beginning to consider the value of integrating some level of packet switching at these core nodes, offloading much of the basic switching function from expensive core routers onto a transport-optimized switching system. There is also an ongoing transition to OTN. This standard (ITU-T G.709) is similar to Sonet/SDH in that it defines a range of interfaces to the optical network, is built on a layered structure, and supports in-service performance monitoring, protection, and management functions, yet also adds management of optical channels in the optical domain and forward error correction (FEC) to enable longer optical spans. The use of OTN on the line side of long-haul DWDM networks is common worldwide, while the adoption of more sophisticated OTN networking features has thus far been limited to operators in Western Europe and select others. That said, our interviews with operators provided evidence that the use of OTN is expanding, driven in many cases by a parallel transition to Ethernet-based transport. Ethernet's lack of a true multiplexing hierarchy, end-to-end performance management, and Sonet/SDH-like operations make it an ideal match for OTN, and many operators already foresee a future in which Ethernet, WDM, and OTN are married to create a fully functional packet transport network, without requir-ing any Sonet/SDH.


Figure 4.3: ITU-T G.709 OTN in the Network

Source: Verizon In our survey, we asked respondents to consider what type of switching their companies require at their core optical network nodes. The results were surprising, but there is ample evidence to suggest they reflect an emerging sentiment that Ethernet and some form of packet switching (rather than full Layer 3 routing) has its place in core transport switches. Today, most core switches perform STS1/VC4 switching and can also act as wavelength switches either by switching a full wavelength of traffic through an electronic fabric, or through an optical switching fabric such as a ROADM, WSS, or a simple optical protection switch. Some core switches include Ethernet interfaces, and can at least interface directly to the packet network to perform aggrega-tion in Layer 2 and switching through the transport network; but many operators see a need for true packet switching in these systems, requiring the addition of a separate packet fabric or a transition to a full packet fabric at the switch's core. Figure 4.4: Switching Requirements at Core Optical Network Nodes

WHAT TYPE OF SWITCHING DOES YOUR COMPANY REQUIRE AT CORE OPTICAL NETWORK NODES?

NUMBER OFRESPONSES

PERCENT OF TOTAL

Wavelength 54 62%

OTN/ODUk 18 21%

STS1/VC4 30 34%

VT1.5/VC12 9 10%

Ethernet/Packet 61 70%

Other 3 3%


Total 87 Taking this analysis a step further, it is now becoming more interesting to explore the use of ROADM/WSS in the network core, as this market has heretofore been largely associated with the metro, where wavelength provisioning and reconfiguration are more necessary and strategic. In


this case, we refer to pure wavelength add-drop and switching, which can often be integrated di-rectly into the DWDM transport system, and doesn't require a separate switching system. The results of the survey show no clear preference for a particular number of degrees (referring to the number of directions in which a ROADM or WSS can route wavelengths) though the more common two-degree (east-west) ROADM node with dynamic add-drop received the most votes. In the metro-regional network, operators are beginning to determine which nodes would be ideal for four-degree operation or more, to support larger junctions and mesh-based restoration. In our conversations with operators, many tell us that around 60 percent of their ROADM nodes will remain two-degree, 20 percent three-degree, 10 percent four-degree, and 10 percent more than four-degree. The core seems likely to follow these ratios. The final determination will come as traffic patterns are better understood and the economics of multi-degree ROADM and WSS become more favorable as volumes increase. Figure 4.5: Wavelength Add-Drop/Switching Requirements on DWDM Backbones

FOR WAVELENGTH ADD-DROP AND SWITCHING, WHAT WILL YOUR COMPANY REQUIRE FOR CORE NETWORK NODES OVER THE NEXT THREE YEARS?

NUMBER OF RESPONSES

PERCENTOF TOTAL

Two-degree nodes with fixed wavelength add-drop (OADM) 14 16%

Two-degree nodes with dynamic add-drop (ROADM) 24 28%

Four-degree nodes with dynamic wavelength-selective switching (WSS) 17 20%

Full optical crossconnect (8 degrees or greater) 14 16%


Total 87 The final element in any switched optical network is management and control. Optical networks have long promised to evolve from rather static arrangements of high-capacity connections to more dynamic architectures where bandwidth and connectivity are more fluid, and under software control. In our survey, we first asked operators where network-level automation would be most beneficial to their company. The leading answer by a significant margin was "provisioning wave-lengths" not a real surprise, given the current acceleration in the market and increased competi-tiveness. Core network operators have long found service delivery times to be a key differentiator, while other, more traditional, network operations features can be improved with automated con-trol, but don't necessarily add immediate competitive value. Figure 4.6: Network-Level Automation Values in the Optical Core

WHERE IS NETWORK-LEVEL AUTOMATION MOST BENEFICIAL TO YOUR COMPANY?

NUMBER OFRESPONSES

PERCENT OF TOTAL

Provisioning wavelengths 37 43%

Channel/wavelength upgrades 14 16%

Software upgrades and maintenance 12 14%

Troubleshooting 13 15%


Total 86 The most important part of this slow and steady evolution to a more dynamic optical network is clearly the ITU-T's standardization effort called ASON, or the automatic switched optical net-work. GMPLS is the protocol suite most commonly referred to as an enabler of the ASON archi-


tecture, and is currently in use in many optical systems as a kind of "ASON-lite" implementation, enabling enhanced control and management of that particular vendor's gear. Based on our survey results, as well as our interviews with carriers, it appears that a slim majority of network operators are planning to make use of GMPLS/ASON in their core DWDM networks. Many certainly do so today on their core optical switches, and as DWDM systems become more automated in their wavelength-provisioning features, many are relying on at least some of the ASON/GMPLS tools to enable these advanced features. That said, a number of operators did tell us that in most cases GMPLS/ASON would be used to support particular services or features, but would not soon be implemented network-wide, or in any multi-vendor fashion, as their vendors have yet to prove that is realistic in the near term. There is also an ongoing debate within many operators around the idea of network control that is, whether the packet network can be charged with controlling facilities in the transport layer, or if there should remain a hard demarca-tion between the two layers, with GMPLS/ASON focused primarily on network protection and bandwidth provisioning. Figure 4.7: Operator Plans for GMPLS/ASON

IS YOUR COMPANY PLANNING TO MAKE USE OF GMPLS/ASON IN ITS CORE DWDM NETWORK?

NUMBER OFRESPONSES

PERCENT OF TOTAL

Yes 44 51%

No 14 16%


Total 87 The next survey question sought more specific information on operator interest in GMPLS/ASON for their core DWDM networks. Asked to identify the primary value of the technology, the leading choice was overwhelmingly "transport service provisioning" a longstanding challenge in the op-tical network, where wavelength setup can be a labor-intensive, error-prone process requiring weeks, if not months, to accomplish. A more dynamic optical network enabled by GMPLS/ASON at least promises to reduce setup time and reduce errors, and operators clearly see value in that. Mesh networking is another feature often associated with GMPLS/ASON, and another that has taken much longer for operators to implement than was first advertised. The reasons for the delay are myriad, but all relate to the difficulty of transitioning from a transmission layer that is so well established along one paradigm ring-based protection and network design to another, particu-larly one that is reliant on a dynamic, distributed control plane. Verizon has talked openly about transitioning to mesh networks in its major regional networks supporting FiOS, but that stance remains fairly unique in the industry today. Figure 4.8: Value of GMPLS/ASON in Operator Networks

WHAT IS THE PRIMARY VALUE OF GMPLS/ASON FOR YOUR COMPANY'S CORE DWDM NETWORK?

NUMBER OFRESPONSES

PERCENT OF TOTAL

Optical layer network inventory/management 11 16%

Transport service provisioning 28 40%

Protection 16 23%

Support for migration to mesh networking 15 21%

Total 70


V. Network Operator Interviews and Analysis 5.1 AT&T Inc. AT&T (NYSE: T), having merged with both SBC and BellSouth, is now the largest operator in North America, and has laid out ambitious goals for its core network. It has begun consolidating several distinct networks onto a common IP/MPLS/Ethernet/OTN/DWDM backbone. Its global network comprises 535,000 fiber route miles, with 30 major Internet data centers on four conti-nents. AT&T offers dedicated MPLS access from more than 1,600 nodes in 137 countries, and wired Ethernet from more than 1,600 access points in 17 countries. Its network carries an aver-age of 7.0 petabytes of traffic per day, and is reportedly growing by at least 75 percent per year. AT&T's express backbone in the U.S. is currently based on Siemens ultra-long-haul DWDM sys-tems, supporting 40-Gbit/s wavelengths over 20,000 route miles and 27 major backbone router POPs, shown in Figure 5.1 below. Below this express layer, AT&T operates more than 1,300 long-haul DWDM systems to serve the second tier of core POPs. Figure 5.1: AT&T U.S. Domestic 40-Gbit/s OTN/DWDM Express Backbone

Source: AT&T According to AT&T, the OTN/DWDM express backbone serves both its converged IP backbone and its enterprise customer base, connecting major U.S. cities with express wavelengths mainly 10 Gbit/s today, but with potential for growth to 40 Gbit/s. The network complements AT&T's ex-tensive long-haul and metro networks for optical connectivity. Key features of the network include:

40-Gbit/s G.709 OTN framing over 80-channel DWDM for most efficient multiplexing utili-zation and to support ultra-high-rate client services such as 10-Gbit/s Ethernet.

Ultra-long reach for rapid provisioning ROADM-based reconfiguration and wavelength setup from core to metro. Both two-

degree and multi-degree ROADM

Tunable lasers for inventory and spares management Automated provisioning of bandwidth Growth path to tunable drops and ASON/GMPLS for agility in wavelength provisioning


AT&T's domestic express and metro-regional backbones employ common technologies, such as:

C-band G.692 DWDM line-haul with 80 channels (50GHz grid) or 40 channels (100GHz grid); both 10-Gbit/s and 40-Gbit/s client payload rates per channel.

G.709 OTN OTU2/ODU2/OPU2 (10 Gbit/s) OTU3/ODU3/OPU3 (40 Gbit/s) client pay-load multiplexing.

Longitudinal, single-span, multi-supplier interoperability of G.709 adapted client signals across intra-domain interfaces.

Optical modulation format is NRZ (10-Gbit/s channel signaling) or advanced (duobinary, DQPSK, etc.) modulation (40-Gbit/s channel signaling).

OC768 packet over Sonet is the framing protocol used for aggregated IP/MPLS/Ethernet client service interface at 40 Gbit/s (STS768c payload capacity is 38.338560 Gbit/s).

At Light Reading's Optical Expo 2006, AT&T Labs' Simon Zelingher said AT&T "will need 100 Gbit/s by the end of the decade," citing the need to adapt to the characteristics of and demand for VOIP, video, and online gaming, which require low latency and packet loss. Zelingher noted that a 40-Gbit/s DWDM system in one POP, capable of carrying 80 wavelengths at full capacity, is already 25 percent full. He described AT&T's single, integrated backbone a model he said will be deployed globally as a network that operates in four layers: hosting centers; IP routers and the "global packet layer," based on MPLS; intelligent optical switching; and fiber and DWDM.

5.2 Comcast Comcast, the largest U.S. MSOs with more than 22 million basic cable subscribers, has recently become a major player in the core DWDM market. Comcast had previously been utilizing DWDM in metro networks for capacity relief, but in the past two years it has been deploying DWDM more strategically, as part of an overall network architecture focused on scaling to meet multimedia services demands. Its IP network infrastructure is designed around three elements backbone, regional, and access networks. The backbone provides connectivity among major content nodes, core Internet POPs, a network operations center, cable maintenance centers, and networked data centers. Comcast's converged regional area networks (CRANs) are large, ring-architected networks responsible for most services delivery, distributing broadcast video from head-ends to local distribution hubs, and voice and data services from various data centers and services POPs. Figure 5.2: Comcast Regional Area Network Diagram

Source: Comcast


Comcast's new backbone network, supplied mainly by Nortel with its Common Photonic Layer equipment, interconnects these CRANs into a national backbone. Features include:

19,000 route miles of fiber (acquired from Level 3), over 100 nodes OC768 40-Gbit/s router ports deployed in 2006 (Cisco CRS-1 routers) Total transport capability of more than 2,500 Gbit/s in 2006 Consolidates peering and interconnection with other operators Managed QOS service delivery end-to-end with no third parties involved Centralized management functions ROADM and WSS support in Nortel's CPL Nortel OME 6500 for edge service aggregation

Comcast's VP of backbone architecture and engineering, Kevin McElearney, has noted that the company's backbone buildout continued throughout 2006, with the East and West Coast rings lit first, followed by remaining core market connections toward the end of 2006. Comcast has an ongoing project called the Open Transport Initiative that is focused on fostering interoperability among router and transport vendors, and today is concentrated on 10-Gbit/s Ethernet LAN PHY for transponder interoperability, and optical control channel interoperability. As part of this initia-tive, Comcast and Nortel also pledged to explore next-generation photonic line interfaces that define power levels, wavelengths, modulation schemes, optical signal-to-noise ratios, and wave-length identification. The driver for all of this expansion is a shift from narrowcast and unicast services. Today, the ra-tio of broadcast to unicast is 94/6; Comcast sees this evolving to 80/20 by 2010, which will have a significant impact on both regional and core network capacity.

5.3 Interoute Communications Ltd. Interoute has become a major network and data center operator in Europe through consolidation and aggressive network expansion. The company has invested 2.7 billion in its network, ac-quired VIA Net.works and PSINet assets to expand its direct customer base to 14,000 enter-prises, and recently received a 125 million investment from Dubai Holding to further its devel-opment toward the East. The range of services Interoute presently offers is quite broad, including hosting; media, voice, and VPN services to enterprise customers; and collocation, bandwidth, and infrastructure ser-vices on the wholesale market. Interoute's network consists for a SIP/SS7 peering platform; a multiservice network for corporate services; a network of data centers for hosting and security services; and a newly developed Ethernet services network based on 10-Gbit/s Ethernet. Interoute has observed that the market in which it operates (competitive pan-European) has been evolving rapidly moving from a simple carrier's carrier model to one of advanced services deliv-ery over a scaleable underlying network. The company sees a major shift going on from 2.5-Gbit/s to 10-Gbit/s services, and a welcome bottoming out of pricing for bandwidth in these mar-kets as operators consolidate. This is partly due to the end of bandwidth supplied by distressed assets in the market, which had a significant downward pressure on pricing. Interoute has seen demand on its network in 2005-2006 exceed its forecast by a factor of two. This surge has caught its vendors off-guard, and some of them are struggling to respond, with lead times extending to around 18 to 22 weeks in many instances. Interoute is seeing 400 percent year-on-year growth for Ethernet, and is rapidly expanding its 10-Gbit/s Ethernet core facilities to keep up. In the face of such rapid growth, the company decided in 2006 to launch a review of transport technology and systems for optical transport.


According to Interoute, the targets of this study included:

New network had to be built and operational before the end of 2007 (Interoute started the process in May 2006; the target footprint is 13,500 km)

Reduce the cost of wavelength delivery by more than 50 percent Provisioning times must be no more than ten days anywhere on the network Implementation had to be robust to deal with multiple changes on a daily/weekly basis

Interoute ultimately chose Infinera as its supplier, claiming its solution offered easy span engi-neering, rapid initial installation, and easier network monitoring than traditional DWDM. Interoute also cited Infinera's use of GMPLS/ASON and a simplified architecture for wavelength routing versus a traditional DWDM line system paired with an optical wavelength crossconnect. The new build with Infinera equipment covered most major traffic routes 13,500 km of new net-work with 39 nodes supporting service add-drop. The Infinera system design is based on 100 Gbit/s of traffic per line card; thus the system was initially deployed with a minimum of 100 Gbit/s for each route, with upgradeability in 100-Gbit/s increments.

5.4 Level 3 Communications Inc. Level 3 (Nasdaq: LVLT) operates both metro access networks and an intercity backbone, which provides high-capacity links between Tier 1 markets and medium-capacity links to Tier 2 and 3 markets, with extensions directly into key traffic aggregation centers. This backbone includes roughly 47,000 route miles, connecting 16 countries in the U.S. and Europe, and supports both wavelength services and private-line services (TDM and Ethernet). In its metro access networks, Level 3 provides a mix of traditional private-line services and Ethernet, and has lately been ex-panding its metro footprint through acquisition. The metro access networks today serve more than 120 markets and 6,200 traffic aggregation points, spanning roughly 24,000 route miles. Level 3 estimates growth in wavelength services demand ranges from 10 to 40 percent per year, depending on the market, while prices are falling by between 5 and 20 percent. VOIP services are both Level 3's strongest growth segment and its highest overall revenue contributor. In the future, Level 3 expects video over IP to be the next-largest contributor to growth. Level 3's has been rapidly expanding its footprint via acquisitions, the most important of which are Broadwing, WilTel, Progress Telecom, ICG, TelCove, and Looking Glass Networks. Level 3's 2005 communications capex stood at $298 million, including both transport and IP upgrades; 2006 capex was roughly $400 million, including several significant acquisitions. Level 3 expects future capex to come in at 12 to 14 percent of total communications revenue. For core DWDM transport, Level 3 is deploying Infinera gear today, largely because of its ability to reduce opex and rapidly provision capacity. Level 3 moved forward with Infinera's solution after concluding that it was cheaper than ROADM technology on both the opex and capex fronts. Level 3 has been rather negative on ROADMs and wavelength crossconnects in the core network, claiming that wavelength churn in the core is not sufficient to justify the added expense. The two major platform-level migrations Level 3 has made for its core network over the past two years are the move to Infinera's "Digital Optical Networking" system; and to Force10's Ethernet routing solution. Level 3 feels Infinera has dramatically lowered its capex and opex in the optical core, and that Force10 has lowered capex in the core IP network by at least a factor of ten. Level 3 has mentioned 100-Gbit/s Ethernet as its next interface focus, rather than looking for 40-Gbit/s optical solutions. The company is seeing 20 to 25 percent annual growth in wave services, much of it driven by corporate customers, rather than wholesale services to operators.


5.5 Verizon Communications Inc. Verizon (NYSE: VZ) has been showing consistent growth over the past year, with 6.6 million broadband data customers, 118,000 FiOS video and 522,000 data customers, and 18.3 percent overall revenue growth reported for the third quarter of 2006. Verizon is targeting 6 to 7 million FiOS data customers and 3 to 4 million FiOS video customers for 2010, indicating that FiOS will continue to be the main driver of Verizon's infrastructure upgrades for the foreseeable future. Verizon's approach to core optical networking is tied to larger efforts to evolve its entire transport infrastructure around the requirements of FiOS (its fiber-to-the-premises initiative) and its need to reduce network capex and opex as it transitions to packet-based services delivery. Verizon's network includes its original backbone (upgraded most recently with Lucent LambdaXtreme prod-uct) and MCI's former backbone (now implementing Ciena and Siemens gear for a ultra-long-haul DWDM express layer), which have been integrated to create a more unified core optical network. Verizon has a number of high-profile efforts underway to shore up its optical networks, including an RFP for a next-generation optical transport platform to serve the needs of the FiOS network, and work on 40-Gbit/s transport, GMPLS/ASON, and wavelength routing in the network core. For the FiOS network, Verizon was primarily focused on regional DWDM solutions that included the following features:

Integration of Sonet and DWDM functionality into a single element ADM-on-a-wavelength eliminates the need for an external Sonet ADM Larger optical reach allows transparent interconnection of a large number of offices with

minimal regeneration

Optical mesh allows rings to be optically interconnected WSS for optical switching between rings eliminates back-to-back nodes for ring to ring in-

tercon

Documents

LH DWDM HeavyReading